This is something I came across today when we launched a new section for a website. It’s something I’ve never encountered before, and I ended up spending most of my afternoon debugging what was causing our issue. Let me begin by saying the correct spelling of Referrer does have two r’s, however it is misspelled in the HTTP Specification. To the end user, this should mean nothing as most software that uses the HTTP Spec uses the incorrect spelling, as it’s what was defined in the spec.
When we launched the site, we checked it out in all our normal browsers and everything was good. Then we fired up IE 6 (unfortunately the client in question needs IE 6 support), navigated to the new secti-OH CRAP! The page itself would load, but none of the external assets (images, stylesheets and scripts) were loading. Refresh the page and bingo! it works fine! What the heck is going on?!
It turns out IE 6 was sending not one, but two referers to the web server. While it’s odd to have 2 referers, it shouldn’t cause 4xx Errors when sending a request. From this, we were able to find out a few things:
- IIS accepts both Referer and Referrer as valid HTTP headers.
- IIS will not accept more than one Referer.
- IIS will accept more than one Referrer (note this is the dictionary spelling of referrer, not the spec spelling).
- IE 6 will add 2 referrers if the link you clicked is located inside of a Flash element for all external assets: the URI to the Flash file and the URI of the page you are on.
I’m not quite sure why IE 6 behaves the way it does, but the fact is that it does and it was causing IIS to give us 400 Bad Request responses. Think I’m crazy (and maybe I am, please prove me wrong)? Try sending this request to www.microsoft.com:
HEAD / HTTP/1.1 Host: www.microsoft.com Referer: http://google.com Referer: http://yahoo.com Connection: Close
In response you should get something similar to:
HTTP/1.1 400 Bad Request Content-Length: 339 Content-Type: text/html; charset=us-ascii Server: Microsoft-HTTPAPI/2.0 Date: Mon, 24 Aug 2009 22:53:04 GMT Connection: close
Now send the exact same request to, say, www.apple.com:
HEAD / HTTP/1.1 Host: www.apple.com Referer: http://google.com Referer: http://yahoo.com Connection: Close
and you get:
HTTP/1.1 200 OK Age: 255 X-Cache-TTL: 345 Accept-Ranges: bytes Date: Mon, 24 Aug 2009 22:48:41 GMT Content-Length: 9542 Content-Type: text/html; charset=utf-8 Expires: Mon, 24 Aug 2009 22:58:41 GMT Cache-Control: max-age=600 Server: Apache/2.2.8 (Unix) X-Cached-Time: Mon, 24 Aug 2009 22:50:56 GMT
And now, to prove point #3, send the following request to www.microsoft.com (note the change in spellings):
HEAD / HTTP/1.1 Host: www.microsoft.com Referrer: http://google.com Referrer: http://yahoo.com Connection: Close
and you get
HTTP/1.1 200 OK Cache-Control: no-cache Content-Length: 1020 Content-Type: text/html Last-Modified: Mon, 16 Mar 2009 20:35:26 GMT Accept-Ranges: bytes ETag: "67991fbd76a6c91:0" Server: Microsoft-IIS/7.0 P3P: CP="ALL IND DSP COR ADM CONo CUR CUSo IVAo IVDo PSA PSD TAI TELo OUR SAMo CNT COM INT NAV ONL PHY PRE PUR UNI" X-Powered-By: ASP.NET Date: Mon, 24 Aug 2009 23:09:03 GMT