The fact that the two IIS based sites (Microsoft, Stack overflow) both fail with an outright error 400 complaining about a bad host name makes me wonder whether this might actually be a bug in IIS to overzealously validate the hostname.
The trailing dot is totally valid and there's no need to complain.
The 400 error is just the web server complaining that there is no web site to deliver to you for that hostname. It isn't complaining about the hostname being invalid, per se.
The IP is the one from stackoverflow.com. When querying for a host they are not configured (try any), you will get a 404 plus an explanation page that the selected Q&A community doesn't exist.
When you try the same host with a trailing dot, you get a generic IIS error 400.
This is too much of a coincidence and I value it as a strong sign that my hypothesis might be correct. I have no IIS at my disposal to replicate paulbeattie's experiment, but the outcome he described further helps confirming my suspicion (no matter the downvotes he got)
That's odd, I would have thought that '404 Not Found' would be the wrong response to send, although you could argue either way.
I used to develop a web server ages ago (Zeus Web Server), it would try and match the Host: header against all the sites it was configured to server, and if none matched, it would send a 400 error back.
(Incidentally, it would happily strip off the trailing dot from the hostname)
I think you might be right. I just tried it on an IIS server set to accept any hostname at the IP the site is bound to and it returned a 400 error as well. Removing the trailing dot made it work.
Yes and it really showed in the beginning: they had lots of issues and you could constantly see the cat saying "sorry, i am wurkin' on fixin' yer problem" (don't remember the exact msg but it was a lolzcat message).
SO is also hardly something impressive from a UI point of view: it is a very simple UI really not doing much. Usenet readers from 20 years ago were way more capable than SO...
I'm not seeing a direct connection between UI simplicity and server capability. I can serve a hideously complex UI with a very minimal demand on the server. Demand on the server is much more about actual traffic, not the UI being rendered in the browser.
Where have we got to the point where a relative hostname is accepted by 100% of websites yet an absolute hostname isn't? Surely it should be the other way round, this is a ridiculous state of affairs!
Up voted, because "forward example.com. to example.com" is exactly backwards. Think about what happens to the user who doesn't have 'example.com' (the relative hostname) resolve to the same address as 'example.com.' (the absolute hostname). Now they want to get to 'example.com.', they type 'example.com.' and they get forwarded to 'example.com', which resolves to example.com.localdomain.com according to their local DNS server. Oops.
It's like having everything rely on directories called "com" and "net", but when someone uses the correct, absolute path of "/com" or "/net" it somehow stops working. Nobody would stand for the latter happening with relative paths, yet somehow relative domains are the default! Madness!
My shock is that some webservers don't remove an ending dot before trying to match a virtual host (any besides IIS?), and that web browsers don't assume equivalence, for website SSL cert CN purposes, between "fqdn." and "fqdn".
I'm also shocked that chrome and firefox send a relative domain in Host: headers if that's what's typed into the url bar. Why wouldn't they send the fqdn? Is it specified this way in some RFC? What does this browser "feature" do other than require webservers in orgs with corpdomain.com search paths to add additional vhost aliases to ensure all the requests get to the right place? Switching vhosts to an intranet website based on the request's Host header does not provide any security in an intranet/dns-search-path setup.
It's because it makes the attacker's job easier. "fqdn." and "fqdn" are not the same thing. They may not resolve to the same machine. If one of them resolves to the attacker's machine, you certainly don't want them to be equivalent.
They're not equivalent, but most or all major CAs issue certs without terminating dots, apparently. So I fail to see how distinguishing them during browser checks for cert validity accomplishes anything.
If an attacker controls "fqdn.localsite.com." when the client's (Alice's) intended destination is "fqdn.", isn't the client fooled when Alice has a localsite.com dns search path, if the attacker (Mallory) manages to get a cert for "fqdn"? (Let's suppose Mallory has access to email for "hostmaster@fqdn." but can't get access to the actual "fqdn." webserver to do the attack that way; Mallory sets up "fqdn.localsite.com." to fool Alice's browser which is subject to a "localsite.com." search path.)
If it's the reverse, where Mallory is external and controls "fqdn." and Alice is expecting to get to "fqdn.localsite.com." because of dns search rules, but the dns search rules are broken and Alice gets sent to "fqdn." instead, Mallory wins because the CA-issued cert for the "fqdn." site won't have the terminating dot, right?
>Let's suppose Mallory has access to email for "hostmaster@fqdn." but can't get access to the actual "fqdn." webserver to do the attack that way; Mallory sets up "fqdn.localsite.com." to fool Alice's browser which is subject to a "localsite.com." search path.
Then you're probably still in trouble either way. But suppose the key for the actual cert for "fqdn." was generated with a bad random number generator like what happened with Debian a while back, or the attacker at one point broke into the webserver for "fqdn." and copied the private key and has since been locked but without revoking the compromised certificate, or it was revoked but the client isn't checking CRLs. Now the fact that you used "fqdn." and not "fqdn" in your cert saves you from Mallory's attack using "fqdn.localsite.com".
>They're not equivalent, but most or all major CAs issue certs without terminating dots, apparently.
The CAs will issue a cert for whatever you ask them to issue it for. The issue is that people ask them for certs without the trailing dot.
The real trouble is that we've basically trained the whole world to type "example.com" and not "example.com." and then when it comes to HTTPS, if you type " https://example.com " then the cert has to have "example.com" at least in addition to if not instead of "example.com." or it isn't going to work. But that's no excuse for not allowing the distinction in cases where it can be made, e.g. for subdomains that serve cookies but are never typed by the user, so that the cert for those can contain only the dot-terminated name and the attacker then has a harder time impersonating the subdomain to pilfer the cookie.
SSL libraries seem to treat CNs as fully-qualified, even without the dot. Do we really want a global ssl PKI where non-fqdn CNs are allowed? If not, why should anyone care about trailing dots for ssl cert validation purposes?
>Do we really want a global ssl PKI where non-fqdn CNs are allowed?
It might not have been a bad idea to prohibit it twenty years ago, but at this point it seems unlikely because everybody expects the contrary.
> If not, why should anyone care about trailing dots for ssl cert validation purposes?
Because if the cert has the dot and the address the client is connecting to doesn't, you may not be connected to the right machine.
If I was writing a piece of client software, what I would do sooner than just allowing them to be considered identical is to do a name lookup for "example.com." if that's what's in the cert presented by "example.com" and make sure the IP address is identical to the one for the "example.com" I'm connected to, and then be OK if they are and fail validation if they aren't.
They shouldn't assume equivalence. What should happen is SSl certificates should have both fully-qualified and non-fully-qualified hostnames in them, not just the latter.
Not the same thing, really, since domains are reversed compared to file paths. In domains, the root token is optional and assumed; like if 'home/user' meant the same thing as '/home/user'.
Well, reverse the path then. "news.ycombinator.com." is equivalent to "/com/ycombinator/news", but more commonly we use "news.ycombinator.com", equivalent to "com/ycombinator/news".
The root token is not optional and assumed at all. Instead, there are multiple "search paths", like with binaries on *nix and Windows/DOS. Now, usually, it will check "." ("/") somewhere along the way, but people often have intranet domains which are also checked, so what you expected to resolve to "news.ycombinator.com." might resolve to "news.ycombinator.com.mycompany.com."
That's what the spec says. In practice, no one uses fully-qualified domains, and users who generally do not even know they exist rightly assume any domain name ending in a TLD should take them to the site indicated by that root, and it's surprising and confusing if it does not.
Which is why it's pretty different from relative vs. absolute paths, which is a much-used and generally well-understood distinction.
This is really going to blow up with the new gTLDs though, there is so much more room for conflict between root DNS and local DNS with so many more TLDs.
That doesn't make sense. In filesystem-land /com/ycombinator/news is not equivalent to com/ycombinator/news. The latter depends on your current location in the filesystem, while the former does not.
The search domains vs. search paths comparison is interesting, but I'd argue not equivalent either. Filesystem search paths only apply to binary executables, for one thing. On unix-like systems, at least, it is recommended that you do not have '.' in your search path. Regardless, there isn't really a "default" search path. If it's empty, you get nothing, and you'll be unable to execute anything without specifying a path. If your search domain list is empty, you still get the default, which is the root.
I guess you can draw some parallels here and there, but it feels a bit forced to me.
Well, for the purposes of SSL verification, or any of these other tests, "localhost" and "localhost." are two very different machines--one is "your" localhost, while the other is "the" localhost [theoretically, the A record for a TLD called "localhost"]. Same with "foo.local" and "foo.local."
Any non-rooted domain actually has the potential to have what is called a "search domain" applied to it: if your own host's FQDN is bob.example.com, going to "foo.com" can actually take you to "foo.com.example.com.", just like going to "foo" can take you to "foo.example.com."
It's just that, conventionally, although it was always an option to use weird TLD-lookin' subdomain hierarchies within domains like that, nobody does it, because it's so confusing. I don't think much at all would break if we enforced a "if your non-rooted domain ends in a known TLD, canonicalize it to a rooted domain before comparison" rule in browsers, the OpenSSL and OpenSSH libraries, etc., and then require that going forward, all X.509 certs, CORS policies, and so forth which are meant to refer to an FQDN are issued only for the rooted version.
[If you're curious, DNS zone records themselves are always already canonicalized into rooted form whenever you edit them through any sensible interface.]
> "localhost" and "localhost." are two very different machines--one is "your" localhost, while the other is "the" localhost
I don't think this is correct. Check out RFC 2606--it implies that .localhost (the TLD) is localhost, which means "localhost" and "localhost." are the same (one will definitely not have a search domain added to it though).
I suspect the same applies to .local—I just checked and the multicast DNS draft RFC [1] refers to ".local." everywhere.
Because technically speaking they are different things. "example.com." refers to a domain name in a system with a nameless root (like the Internet DNS namespace managed by ICANN). "example.com" would refer to some different (non existent) domain name system with a root called "com".
The problem is that in practice many implementations that use DNS names display "example.com" to the user when they really mean "example.com." and many certificates are signed for "example.com" when they really mean to be signing for "example.com." A proper implementation would use "example.com." everywhere, but might choose to display "exmample.com" in the GUI if they consistently translate between the two.
I particularly wonder if one can set "www.example.com" to point to a different address from "www.example.com." (globally, not via changing /etc/hosts or your local dns server...)? If they are practically equivalent, then I can't think of much security benefit from treating them differently.
Even if changing the behaviour won't cause any security risk, I can't imagine this being a simple change to make. The SSL certificate contains a digital signature that is performed on a hash of many details, including the CN which is typically the FQDN (without the trailing dot). The `CN==requested-server-name?' matching can be performed at different layers and is probably deeply embedded within the specification and in different implementations. Add this to the practicality of needing to use both www.example.com and www.example.com. - it seems that the balance is in favour of keeping things simple and as the post suggested, use a redirect (or buying an extra certificate).
From the looks of it, Safari ignores the trailing dot when validating (as you can see when going to https://www.facebook.com./ ). Curl on the other hand doesn't
The . doesn't indicate the root nameservers at all - it indicates that the DNS name to be resolved is an absolute name, not a relative name. All nameservers - root or otherwise - specify the names they know relative to the absolute global domain namespace root, ".".
When resolving a hostname, your computer's DNS resolver will look for it in various 'search path' contexts, much like a shell looks for executables relative to all the directories in your PATH.
Asking it to resolve news.ycombinator.com will lead to it looking for news.ycombinator.com., but also maybe trying various connection-specific DNS suffixes your network interfaces have configured, or global search suffixes. Commonly macs, for example, try resolving names within the domain 'directory' of "home." - http://mymacbookpro/ will look for a machine called "mymacbookpro.home." as well as "mymacbookpro.home.". But http://mymacbookpro./ is specifying the absolute path - the resolver won't look for mymacbookpro.home., only mymacbookpro. Conversely, if you type in http://news.ycombinator.com/, it will look for a "news.ycombinator.com.home." as well as a "news.ycombinator.com.". Yes, you're right - that is an opening for a spoofing attack if you use a compromised or untrustworthy DNS service on your local network.
The root nameservers are just one place your DNS resolver looks to to ask questions like "who is the authoritative DNS server for domain names in .com.?" once it's decided to look up a name like news.ycombinator.com..
Every "web developer" should read the first three chapters of the Cricket book. The profound ignorance (that many web developers have) of DNS is inexcusable:
D'oh. It would seem to me that for the purposes of X.509/CORS/server config/etc. equality, domains which seem to be rooted in any of the TLD registries should be canonicalized first to add the dot on both sides (along with doing the IRI -> URI conversion and so forth that should happen either way.) Is there any reason that would be a bad idea?
I wouldn't call it a 'danger' more like an 'annoyance', unless someone is MITM you with a certificate for the name ending in dot or something like that
Maybe redirect, but I'd go for making it not work, period. (I don't mean 'broken', I mean blank page or forbidden)
I use RequestPolicy to whitelist access to other domains' resources. Usually most sites that have CSS on external resources show up as black-and-white, which is perfectly readable. But this one is displayed as blue text on red background... WTF?
It's something I see a lot over slow connections--more commonly, black text on a black background, where eventually a white "content-layer background" image will load behind the text, making it readable.
no, I read it as a solution to a problem that doesn't occur. It's way worse if somebody would type a comma or any other letter at the end of the domain name, as I just learned, the dot is the best character to mistake at the end of a domain
> [The trailing dot in the domain name is] there for a reason. It made the domain name a fully qualified one, and thus unambiguous and not prone to search path spoofing.
[snip]
> For example: Posit that the web browser uses the BIND DNS Client library and the search example.net directive is present in that library's resolv.conf configuration file.
> In the case that the URL provided by the user is http://example.com/, a web browser would pass example.com to the BIND DNS Client library for looking up its IP address(es). Because the RES_DNSRCH option is set and the domain name does not end in a trailing dot, the BIND DNS Client library would take the domain name to not be in fully-qualified form and apply its "search path" mechanism. It would look up example.com.example.net. and, if that domain name happened to exist, would return the IP address(es) corresponding to that domain name instead of the IP address(es) corresponding to example.com.. The web browser would contact the content HTTP server at the former IP address(es) without the user being aware of it.
> In the case that the URL provided by the user is http://example.com./, a web browser would pass example.com. to the BIND DNS Client library for looking up its IP address(es). Because the domain name ends in a trailing dot, the BIND DNS Client library would take the domain name to be in fully-qualified form and not apply its "search path" mechanism. It would look up example.com. and, would return the IP address(es) corresponding to that domain name.
> The fully-qualified domain name would thus prevent search path spoofing. (See RFC 1535 and RFC 1536 § 6 for more discussion of this.)
The search path spoofing argument is weak. If you depend on a DNS resolver that spoofs domain searches, what guaranteed that it wouldn't simply spoof the full resolution as well ?
In practice nobody uses the dot at the end except when writing BIND files.
The DNS resolver doesn't need to spoof anything. If you're on network example.net and there's a server on that network called "google" then typing "google" into your address bar will resolve to google.example.net in a completely legitimate way, with no DNS spoofing.
>If you depend on a DNS resolver that spoofs domain searches, what guaranteed that it wouldn't simply spoof the full resolution as well ?
A lot of DHCP servers do DDNS. That allows you to change your computername and get your computer registered in the local DNS as whatever-you-want followed by the local domain which is usually in the search domains list. You can't get registered as "google.com" but you can get registered as "google.com.example.com" when "example.com" is in the search domain, and now you're "google.com" (but not "google.com.") to everybody on the local network.
Yeah agreed but what's the most likely outcome. You rewriting your urls or you fixing your network / kicking the offending user ? You're not only impacting browsing experience but also all the scripts that are trying to connect to the domain like your google drive sync agent.
>Yeah agreed but what's the most likely outcome. You rewriting your urls or you fixing your network / kicking the offending user ?
Who is "you"? The person who can fix the URLs is rarely the same person who can kick the offending user off the local network. If the latter person is not doing their job (or is about to but hasn't yet), you're still better off fixing your URLs.
>You're not only impacting browsing experience but also all the scripts that are trying to connect to the domain like your google drive sync agent.
You don't have to break the old stuff, just start using the final dot for new stuff.
The trailing dot is totally valid and there's no need to complain.