Sunday, March 04, 2007

I've been using ADFS day in and day out for almost a year now and have run into a fairly stunning number of issues.  Most of these are simple problems with simple solutions, but they can cause hours or days of frustration to resolve.  One of the main themes that seems to pervade these little doses of IT misery is based on a misunderstanding of how cookies work and how ADFS uses cookies.  This post is my attempt to gather some of the lessons I've learned and hopefully make your time with ADFS a little easier. 

How Cookies Work in Web Browsers

Although many of us have a basic, foggy notion about what cookies do in web applications, many of us don't know all of the little rules that govern their behavior.  I'll try to run that down now based on my own understanding, which I believe is at least technically accurate enough for the purposes of the rest of this article.  Feel free to correct me if you know better or refer to the RFCs if you want more precision.

Basically, a cookie is a small piece of textual data that is sent by a web server to a web browser.  The job of the browser is to return this piece of data back to the server when it makes another request to the server, based on the rules that I'll detail shortly.  Cookies are generally used in web applications as a means of maintaining state in the web server across multiple requests made by the browser.

Cookies are implemented as well-known HTTP headers in the request data sent by the browser and the response data sent.  The request header name is called Cookie and the response header name is Set-Cookie.  The server always initiates the usage of cookies by returning a response with a Set-Cookie header in it that contains the data and some other instructions.  The browser's job is to then add a Cookie header on all subsequent requests to the server that contains exactly the data the server sent in the Set-Cookie header.  However, exactly when the browser will add the Cookie header to its request is based on the other instructions in the Set-Cookie header and the rules for cookie behavior. 

First, let's examine the "other instructions".  The Set-Cookie header may contain these bits of info:

  • Name
  • Value
  • Domain (optional)
  • Path (optional)
  • Expires (optional)
  • Secure flag (optional)
  • HttpOnly flag (optional)

Cookie Name and Value

The Cookie Name is just what it sounds like.  Since a web app may use many different cookies, this is how they can be distinguished.  The Cookie Value is just some textual data.  It could be anything.  Oftentimes, it may contain some data that has been encrypted that the browser can't even interpret it--only the server can.  This is not a problem.  The data is for the server anyway.  The browser's job is to dutifully sending it back to the server, not understand it.

Cookie Expires, Secure and HttpOnly

I'll skip ahead to the last three, as they easy to explain and not terribly problematic (for our purposes).  The optional Expires field may contain a date/time value that tells the browser when to stop sending the cookie back to the server.  This one is interesting because it alone determines whether a cookie is a session cookie or a file-based cookie

If there is no expiration specified, the cookie defaults to a session cookie and will be returned as long as the browser session continues.  In IE and FireFox (and probably most other browsers on Windows), the session is basically the duration that the browser process is open.  Note that a browser process may span many windows and may originate in a non-obvious place like an email or an instant messenger client, so you may sometimes be surprised to see a cookie being sent back to a server even after a window was closed (or seemingly all windows for that matter!). 

If the expiration date is set, then the cookie is "file-based" and will be written to disk.  It will survive across multiple browser processes being launched and stopped.  The browser will continue to send it until the expiration time is exceeded.

The Secure flag instructs the browser to only return the cookie back to the server if the request uses the HTTPS (SSL) protocol.  This flag is often used when the cookie contains sensitive data that could be useful by a bad guy if it was intercepted on the wire.  I consider it to be bad practice to ever use any kind of forms-based authentication cookie without the Secure flag set, although this then requires SSL.  If you are serious about security, you use SSL for secure websites.  End of story.

The HttpOnly flag is the new guy and not all browsers support it (although many do).  It basically says that this cookie will only be sent on the wire.  The browser will not allow the cookie to show up in the browser's Document Object Model (DOM) cookie collection.  This prevents scripts from being able to view and manipulate the cookie and can help prevent some kinds of cross-site scripting (XSS) attacks.

Cookie Domain

For our purposes, The Cookie Domain and Path are the most interesting values, as this is the heart of all of the trouble in ADFS, so we cover them last.  The domain specifies which hosts the browser will replay the cookie to.  If the domain is not specified, the browser will only replay the cookie to host that sent the Set-Cookie header in the first place.  Here, the host is determined by the host portion of the URL.  For example, in http://www.joekaplan.net/default.aspx, the host is www.joekaplan.net

Domains can also be specified hierarchically.  I could set the cookie domain to .joekaplan.net and then my browser would return the cookie to any host within the joekaplan.net DNS namespace such as www.joekaplan.net, blogs.joekaplan.net and junk.joekaplan.net (neither of which exist in the latter two examples, but that isn't important).

One might wonder if one can set a cookie for a domain other than the domain of the host.  The answer is "it depends", but usually no.  As one can imagine, this could (and has!) lead to all kinds of crazy hacking where one site writes over another site's cookies.  In IE, the ability to do this is locked down by the various security levels and assigned by zone.  I haven't seen many legitimate uses for such a design personally, so I'm happy this is restricted now.

Another important picky detail about the domain value is that it doesn't consider the port component of the hostname in the URL.  Basically, this means that it treats the host www.joekaplan.net:80 and www.joekaplan.net:8080 as the same host, even if those map to completely different websites on your web server.  This can lead to surprises.

Cookie Path

The final modifier is the Path.  The path value is like the domain, except that is specifies the scope in the path hierarchy that the cookie will be replayed to.  Here, the path is the part of the URL that comes after the host name.  If the path is set to /, then the cookie will be replayed to any part of the hierarchy of the website.  If the path is set to /site, then the cookie will only be replayed to requests for URLs at or below /site.

One tricky detail with the path value is that it is case-sensitive.  If you set the path to /site and your URL is http://www.joekaplan.net/Site/default.aspx (note the capital "S"), guess what?  The cookie will not be returned with the request!  This little gotcha can easily confound IIS developers, as IIS is generally quite happy to treat folders and virtual directories as case-insensitive.  Developers on "other" web server platforms where URLs (and file names) are case-sensitive tend to be a little more hip to this. :)

A Brief Tour of the ADFS Cookies

ADFS, like most other HTTP authentication protocols that don't use the built in HTTP authentication specifications (Basic, Digest, Negotiate, Kerberos, NTLM, etc.), use cookies for a lot of stuff.  There are three primary cookies to know about:

  • Authentication/Token cookie (_WebSsoAuth)
  • Home Realm Cookie (_LSRealm)
  • Logout Cookie (_LSCleanup)

Let's take each in turn.

Authentication/Token Cookie

The _WebSsoAuth cookie is certainly the most important one, as that is the cookie that represents the user's login to a federation resource.  This cookie is always a session cookie, in that the Expires field is never set, so it goes away when the browser process does.  Expiration of this cookie is handled by timestamps set in the data in the cookie itself and is a programmatic feature of the protocol and is not related to how the browser treats cookie expiration. 

The cookie basically contains the SAML 1.1 token issued by a federation server that allows access to a specific resource.  The format of the cookie for ADFS is in some crazy compressed format that MS came up with that isn't documented, but that is supposedly cool because the WS-Federation PRP spec doesn't actually contain any details about what this should look like and leaves it implementation specific.  These cookies are only consumed by the server/app that issued them (hopefully!), so this isn't a big deal.  Sometimes you may see a _WebSsoAuth0 cookie or additional numbering.  This can happen if the SAML token is very large and the compressed size of the XML in the SAML token may exceed the allowed size of a cookie.  ADFS breaks these up for you to overcome this.  This is a picky and unimportant detail for our discussion, but I figured I'd mention it in case you saw it and were curious.

The Set-Cookie header for this cookie usually looks something like this (data clipped for brevity):

Set-Cookie: _WebSsoAuth=eNrNWWuTqkgS9acYzpfd6LV5qAhGd8cWDxUVFMVnbMREASWiQ.....CdagA; path=/adfs/ls/; secure; HttpOnly

In this case, this is the authentication cookie issued by the federation server itself.  In ADFS, each federation server the user logs in to (at least one, maybe two if there is an account partner/resource partner set up) will issue a _WebSsoAuth cookie.  Additionally, each application that the user visits will issue a _WebSsoAuth cookie that is specific to the application. 

Each _WebSsoAuth cookie is different and is intended only for the app/server that issued it.  However, you probably noticed how they all have the same name.  Because they all have the same name, they must different by domain and path so that the browser will know which cookie to send to which app/server.  If you've ever configured an ADFS token-based or claims-based application, you will have noticed that you are required to specify the domain and path for the cookie when configuring the application.  All it takes is a naive mistake with this configuration to cause the browser to start sending the wrong cookie to the wrong place and ADFS chaos ensues.

Home Realm Cookie

The _LSRealm cookie is a file-based cookie issued by the resource federation server after you have successfully logged in that specifies the federation server URI (usually urn:federation:foo or something) of the server that authenticated you.  This is done so that the resource federation server doesn't have to prompt you for your home organization again if it has multiple potential partners.  We already discussed a little bit about these values and home realm discovery query strings in a post last summer, so I won't get into much more detail.  These things rarely cause trouble except that you have to clear your cookies if you want to get a resource federation server to prompt you for your home realm again after you log in the first time.  They are issued for one month as I recall; I'm too lazy to check for certain right now.  :)

Logout Cookie

The _LSCleanup cookie assists in the logout process by keeping track of which federation servers and applications you've visited.  When ADFS does a log out, it attempts to "drain the swamp" by logging you out of everything.  To do this, the built-in logout page attempts to visit the logout URL for every federation server you've visited and every app you've visited.  These logout URLs set the _WebSsoAuth cookie value for the cookies they issued to "null".  This cookie is also a session cookie.  These cookies can get screwed up just like the authentication cookies, but usually this fact never comes up because once the authentication cookies are hosed, things tend to break before you ever try to log out.

Common ADFS Problems Caused by Incorrect Cookie Settings

There are two ADFS problems I've seen frequently that cause much pain that are related to cookie problems and probably some variants that I'm forgetting:

  • Invalid SAML Audience
  • Infinite Loop Detected

Invalid SAML Audience

This occurs when one ADFS application get's another ADFS application's cookie.  Like I said before, each application has an application-specific cookie and there is a piece of data in the SAML token called the audience that says exactly where the cookie was supposed to go.  This is identified by the application's configured URL in the trust policy and the web site. 

There are a variety of ways you can create this error.  One easy way is by setting the cookie domain to something like .domain.com and having your apps set up with host names like app1.domain.com and app2.domain.com.  Since these are both basically children of .domain.com, the browser will happily send a _WebSsoAuth cookie to app2 that app1 issued if the paths are the same.  Since the path is often set to /, this is not uncommon.  You can also have pseudo-chaos that is even more difficult to debug if the path's aren't the same but contain some overlap in hierarchy.

I'm still trying to figure out a reason why you would ever set the cookie domain in an ADFS system.  Generally, you only ever want an ADFS cookie going back to the host that issued it, so why not leave it at the default setting of "null"?  In my experience, nothing but trouble ever comes from setting the domain.  If you think of a use case for setting this, please let me know.

I also mentioned overlapping paths.  These are a significant cause of trouble.  Let's say you have an app at the root of the site / and an app below that /Site.  Basically, if they have the same host name, you can't do that in ADFS.  They need to be strict siblings like /Site1 and /Site2

Overlapping paths can show up in subtle ways as well.  Sometimes, you may not think that overlapping paths are a problem because you aren't aware that you have overlapping domains.  If you use the ADFS Step-by-Step Guide to put together your ADFS demo lab, you may find yourself using host names like this to try out different application styles and using different TCP/IP ports for SSL to split them up in IIS:

  • https://apps.domain.com:8081/
  • https://apps.domain.com:8082/

While this makes things easy from the IIS perspective in terms of setting up multiple sites and it makes it easier to deal with few host names (and less DNS and certificate mucking around), remember that the browser treats those two hosts as being the same thing for cookie purposes.  You now have overlapping domains.  Therefore, it simply falls to the cookie path to make a distinction between them.  If you use a path like:

  • /claimsapp
  • /tokenapp

...for each one, you are fine, but if you naively put one or both of those apps in the root, you will have problems if you try to do SSO between them with the previously mentioned error.

Infinite Loop Errors and the Like

The other problem we can easily create is the "infinite loop" problem or "same token request within the last 20 seconds" problem on the federation server.  I've seen this happen when we have a capitalization different between the browser URL and the cookie path.  As we explained above, the cookie path is treated as case sensitive by the browser, but IIS is generally very happy to treat a URL folder or virtual directory path as case-insensitive.  The problem then manifests itself in ADFS with an interaction like this:

  1. User logs into federation server after being redirected to do so by the application they were trying to access
  2. Federation server generates SAML token for resource application and redirects browser to POST the token to that URL (app.domain.com/site/)
  3. Resource application receives the token via POST, verifies it and issues the browser a cookie with the token contents for use with subsequent visits.  The path in the Set-Cookie header is /Site.  App redirects the user back to itself with the exact URL originally requested.
  4. Browser receives the cookie, but it is using a URL like https://app.domain.com/site/default.aspx, not https://app.domain.com/Site/default.aspx.  Since those are clearly different (note the capital S!), the browser does not include the authentication cookie in the request.
  5. App receives the request from the browser without the cookie and decides "this guy should log in if I'm going to allow access, so go back to my federation server, get a token and I'll see you later" and redirects the user back to the federation server with instructions to sign in.
  6. Federation server is clever and notices that it just issued a token for this user for this app a second ago or so and throws an error because it knows that something just isn't right.

This one is a little subtle, but easily fixable.

Conclusion

Cookies are at the heart and soul of how ADFS works (with the Passive Requester Profile anyway), so understanding the rules that govern their behavior in the browser and how ADFS expects you to use them will save you potentially hours or days of troubleshooting misery. 

Tools for Viewing Headers and Cookies

When working with ADFS (or really just about any web app I've found), having a way to view the raw request and response headers associated with all of the traffic generated by the browser and server is invaluable.  There are great plugins for IE (ieHttpHeaders) and FireFox (Live Headers) that do a fine job of this and are free. 

(edited March 4, 2007, to add some additional formatting, fix some wording and add the section on viewing headers)

Sunday, March 04, 2007 5:09:13 PM (Central Standard Time, UTC-06:00)  #    Comments [2]  | 
Sunday, January 06, 2008 10:19:52 AM (Central Standard Time, UTC-06:00)
Hello Joe,

this article was very interesting and very helpful too.

but I have a question, I need to access the token cookie from multiple browser sessions, is there a way to do this? and if there is a way, can you please demonstrate it?

and another question, when I try to open an ADFS protected site I get a windows login screen, i want to see a web-based login form instead. is there a way to do this? and how?

Thanks in advance.
Monday, January 07, 2008 4:22:33 PM (Central Standard Time, UTC-06:00)
Hi Ayman,

Would you mind redirecting your questions to the ADFS discussion forum at www.directoryprogramming.net? I think it would be better to conduct the discussion there. Thanks!
Name
E-mail
Home page

Comment (HTML not allowed)  

Enter the code shown (prevents robots):

Theme design by Jelle Druyts