Fireside Media Development Blog

This is something I came across today when we launched a new section for a website. It’s something I’ve never encountered before, and I ended up spending most of my afternoon debugging what was causing our issue. Let me begin by saying the correct spelling of Referrer does have two r’s, however it is misspelled in the HTTP Specification. To the end user, this should mean nothing as most software that uses the HTTP Spec uses the incorrect spelling, as it’s what was defined in the spec.

When we launched the site, we checked it out in all our normal browsers and everything was good. Then we fired up IE 6 (unfortunately the client in question needs IE 6 support), navigated to the new secti-OH CRAP! The page itself would load, but none of the external assets (images, stylesheets and scripts) were loading. Refresh the page and bingo! it works fine! What the heck is going on?!

It turns out IE 6 was sending not one, but two referers to the web server. While it’s odd to have 2 referers, it shouldn’t cause 4xx Errors when sending a request. From this, we were able to find out a few things:

1. IIS accepts both Referer and Referrer as valid HTTP headers.
2. IIS will not accept more than one Referer.
3. IIS will accept more than one Referrer (note this is the dictionary spelling of referrer, not the spec spelling).
4. IE 6 will add 2 referrers if the link you clicked is located inside of a Flash element for all external assets: the URI to the Flash file and the URI of the page you are on.

I’m not quite sure why IE 6 behaves the way it does, but the fact is that it does and it was causing IIS to give us 400 Bad Request responses. Think I’m crazy (and maybe I am, please prove me wrong)? Try sending this request to www.microsoft.com:

HEAD / HTTP/1.1
Host: www.microsoft.com
Referer: http://google.com
Referer: http://yahoo.com
Connection: Close

In response you should get something similar to:

HTTP/1.1 400 Bad Request
Content-Length: 339
Content-Type: text/html; charset=us-ascii
Server: Microsoft-HTTPAPI/2.0
Date: Mon, 24 Aug 2009 22:53:04 GMT
Connection: close

Now send the exact same request to, say, www.apple.com:

HEAD / HTTP/1.1
Host: www.apple.com
Referer: http://google.com
Referer: http://yahoo.com
Connection: Close

and you get:

HTTP/1.1 200 OK
Age: 255
X-Cache-TTL: 345
Accept-Ranges: bytes
Date: Mon, 24 Aug 2009 22:48:41 GMT
Content-Length: 9542
Content-Type: text/html; charset=utf-8
Expires: Mon, 24 Aug 2009 22:58:41 GMT
Cache-Control: max-age=600
Server: Apache/2.2.8 (Unix)
X-Cached-Time: Mon, 24 Aug 2009 22:50:56 GMT

And now, to prove point #3, send the following request to www.microsoft.com (note the change in spellings):

HEAD / HTTP/1.1
Host: www.microsoft.com
Referrer: http://google.com
Referrer: http://yahoo.com
Connection: Close

and you get

HTTP/1.1 200 OK
Cache-Control: no-cache
Content-Length: 1020
Content-Type: text/html
Last-Modified: Mon, 16 Mar 2009 20:35:26 GMT
Accept-Ranges: bytes
ETag: "67991fbd76a6c91:0"
Server: Microsoft-IIS/7.0
P3P: CP="ALL IND DSP COR ADM CONo CUR CUSo IVAo IVDo PSA PSD TAI TELo OUR SAMo CNT COM INT NAV ONL PHY PRE PUR UNI"
X-Powered-By: ASP.NET
Date: Mon, 24 Aug 2009 23:09:03 GMT

This entry was posted on Monday, August 24th, 2009 at 6:09 pm and is filed under Code. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.