Tokens ain't Tokens

After writing my recent rant about token authentication, I realised there was more to say on the subject.

Not all tokens are created equal, so I feel it's important for people to understand the differences.

Here's a quick summary of some of the common token types, and their relative strengths and weaknesses.

Some terms.

User: typically the Human wanting to allow Agents limited access to their resources.

Issuer: the service generating tokens

Consumer: a service that trusts tokens from the issuer.

Agent: a service which is issued a token to grant it rights with a Consumer.

Grant: authorisation granted to an Agent by a User.

Token Types

Opaque vs. Parsable

Opaque tokens are meaningless to those who hold them and those who consume them. They provide no information beyond being an identifier.

Parsable tokens (like JWT) contain data as well as acting as an identifier. They can include a user ID, as well as authorisation claims, and so on.

Plain vs. Signed

A plain token can not be verified as having been issued by the trusted Issuer except by asking them directly. And the Issuer must keep a record of all issued and current tokens so as to be able to verify them.

Signed tokens contain extra information to "prove" they were produced by a "trusted" Issuer. This is typically done by appending a "signature" to the token (opaque or parsable).

Signatures may be done using either Shared Secret or Public Key approaches. See below for more discussion on the relative merits of each.

Unsigned tokens are only really practical if the Issuer and Consumer are the same system. Otherwise the Consumer must make a background connection to the Issuer to verify each token. However, in doing so they can also verify each time the token has not been revoked or amended.

Signed tokens do not require the Consumer to contact the Issuer in order to validate it as a legitimate token. However, they do have the issue of becoming outdated; since the Consumer does not contact the Issuer, these tokens can not be revoked. The common solution for this is to make these tokens short lived, and expire quickly.

To avoid having to have a User intervene frequently to authorise a new token, Issuers will often also issue a Refresh Token. This token is only ever sent between the Agent and the Issuer. When the Agent notices its token is expired, it requests a fresh token "automatically" from the Issuer. If the Grant has been revoked, the Issuer can refuse to issue a new token.

Some systems put the Refresh Token in a secure, http-only cookie. In this way JavaScript Agents in browsers can't even access it, and the browser's native cookie handling ensures it's only ever sent in requests to the Issuer.

Open vs. Encrypted

For Parsable tokens, they may be raw serialised data, or encrypted.

Raw data can be read by anyone who can see the token, including the Agent. Most JWT systems use this, which is why it's recommended to not use them for session data.

Encrypted tokens are encrypted to keep their contents secret from anyone not issued with a key.

Encryption may be done using either Shared Secret or Public Key approaches.

Shared Secret vs Public Key

Shared Secret systems involve the Issuer and Consumers having a copy of the same key. Typically this uses a system like HMAC.

Without knowing the secret you can not generate or verify a token.

Shared Secret systems tend to be simpler to implement, but require every system having the same key, making them all hacking targets.

This means the Issuer must trust every Consumer not to abuse the Shared Secret to generate their own tokens.

Public Key cryptography allows you to safely publicise the Public Key (used for decrypting, and verifying signatures) whilst carefully guarding the Private Key (used for encrypting, and signing).

This also means none of the Consumers can generate a valid token.

Which to use?

If your system is monolithic, simple opaque tokens can suffice. An example of this is Django's session auth, where we can think of the Session Cookie value as the Token.

When it comes to distributed systems, you have more things to consider.

Signed tokens allow Consumers to verify a token is valid without having to contact the Issuer, but suffer the revocation limits mentioned earlier.

Parsable tokens can carry details of granted authorisation rights, allowing for more nuanced Grants without the need to contact the Issuer to check for specific permissions.

They can also continue to work as Opaque tokens when simply possessing the token is sufficient authorisation to access the resource.

If your system is distributed, but private (all Consumers are under your control) you might consider using a Shared Secret signed token, as you can more readily trust your Consumers to not maliciously issue tokens.

If, however, you must support untrusted Consumers, you will want Private Key signed tokens so only the Issuer can generate tokens.

Examples

Most combinations of the above have been tried over the years, and (fortunately) few remain.

1. HTTP Basic Auth

   This can be considered an unsigned, parsable token, as it contains the username and password base64 encoded.

2. OAuth2

   Originally this used opaque tokens, since the Issuer is also the Consumer.

3. JWT

   Commonly, JWT are Parsable and Signed.

   They support both Shared Secret and Public Key signing.

   Additionally, though uncommon, they support encryption.

4. OIDC

   This system builds on top of Oauth2, replacing their Opaque tokens with Parsable JWTs.

   Since Parsable tokens are backward compatible with Opaque tokens, this allows existing OAuth2 systems to be retrofitted as ID services.