Understanding JSON Web Tokens: structure

Applications talk to each other constantly, over private networks and over the open internet. How do you secure those exchanges? How do you know who sent a message? How do you stop it being read or altered on the way, without eating your bandwidth? JSON Web Tokens answer all of that, and more.

First, five principles

Security in computing rests on a handful of guarantees. Five of them matter here.

  • Integrity: the data has not been altered. Alterations may be accidental, a faulty protocol or a corrupted signal, or deliberate, an impersonation or an injection of forged data.
  • Confidentiality: the data is not readable in transit. Think of the login form you submit. Would you want somebody intercepting your credentials?
  • Availability: the system works in every circumstance, without failure or interruption.
  • Non-repudiation: the origin of the data is provable. The sender can be identified and cannot deny having written it.
  • Authentication: only the intended recipient reaches the data. No other party can read it without explicit permission.

Others exist, traceability for instance, but they are out of scope here.

What is a JSON Web Token?

A JSON Web Token, or JWT, is an open JSON-based format for transferring data securely between two applications. It is designed for the web.

It addresses every principle above. Integrity comes from digitally signing the data. Confidentiality comes from encrypting the token. Non-repudiation and authentication come from the keys used for those operations. Availability is served by the format being simple enough to process quickly.

There are several representations, which we will meet along the way, but most often it looks like this:

eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiYWRtaW4iOnRydWV9.TJVA95OrM7E2cBab30RMHrHDcEfxjoYZgeFONFh7HgQ

Were there no other ways?

Of course there were, and the web did not wait for these tokens to secure its exchanges. A secure connection over HTTPS already covers part of it.

There is also SAML, the Security Assertion Markup Language, an XML format dedicated to secure transfers that offers the same guarantees. It is widely used, in single sign-on in particular.

So why a new format?

  • Simplicity. The structure and the representation of a token are plain: no ornament, no unused field.
  • Compactness. A JWT runs to roughly 500 bytes where a SAML assertion would take more than 2 500. Two kilobytes sounds like nothing, but on a server handling hundreds of thousands of exchanges per second it is several gigabytes of bandwidth saved every day.
  • Ease of use. A few lines of code create and read a token in full, and parsing takes microseconds with the faster algorithms.

One more benefit

A drastic reduction in database calls. This kind of token carries everything your application needs inside itself, so there is no entity to fetch and no query to run. Over time that lightens your servers appreciably, which serves availability in its turn.

Where are they used?

Mostly in authorisation protocols such as OAuth2, to protect a REST API for instance, and in identity protocols such as OpenID Connect. Nothing limits them to that: a secure chat system or a document exchange are just as legitimate.

Throughout this article the examples are in PHP, using web-token/jwt-framework. Others exist for PHP and for other languages, and jwt.io keeps a fairly complete list. I use this one for two reasons: I wrote it, and it is one of the few PHP libraries, possibly the only one, to implement every specification covered here. If it does not suit you, or if PHP is not your language, use whatever you prefer for what follows.

An open standard

The Internet Engineering Task Force, the IETF, is the group that produces most internet standards. A standard goes through writing, review and correction as a draft before being submitted for approval. Once approved it becomes a Request For Comments, an RFC, and receives a number in chronological order. RFCs and drafts alike are freely available.

If errors are found after approval, an erratum is issued. A standard may also be updated by another standard. The IETF is open to anyone, including you, should you wish to standardise a protocol, or simply report a mistake in an RFC.

JSON Web Tokens became standards in May 2015. Every aspect around them is described in its own document:

Further RFCs have since completed that list, and we will meet some of them. These five are the main ones. Let us start with RFC 7519, which describes the basic structure of a JWT.

The payload

In theory

The payload of a JSON Web Token is, drum roll, a JSON object. Otherwise these tokens would be called something else.

It is generally a set of members forming claims, each with a well-defined meaning. RFC 7519 standardises a number of them so that applications understand one another. It also allows application-specific members, as long as they do not reuse a standard name and both ends agree on what they mean.

None of the following is mandatory. Whether you use them depends entirely on the context.

  • iss (Issuer): who issued the token
  • sub (Subject): what the token is about, a user, a resource
  • aud (Audience): who the token is for
  • exp (Expiration Time): when it stops being valid
  • nbf (Not Before): when it starts being valid
  • iat (Issued At): when it was issued
  • jti (JWT ID): a unique identifier for the token

An example payload:

{
  "iss": "joe",
  "exp": 1300819380,
  "http://example.com/is_root": true
}

Here the issuer is joe, the token expired on 22 March 2011 at 19:43 UTC, and an application-specific member, http://example.com/is_root, is true.

Mind the character encoding: only UTF-8 is allowed. If your system uses another, convert.

In practice

<?php
$data = [
    'iss' => 'joe',
    'exp' => 1300819380,
    'http://example.com/is_root' => true,
];

echo json_encode($data);

The result carries no line break and no space:

{"iss":"joe","exp":1300819380,"http://example.com/is_root":true}

The header

In theory

The header is JSON too, unsurprisingly. Each of its members has a precise meaning, and RFC 7519 defines two:

  • typ, the media type. Optional, and often left out because the value will usually be JWT. Using it is nonetheless strongly advised, so your application can tell apart the tokens it receives.
  • cty, the content type. Also optional, and not recommended except in two cases: the token is nested, meaning an encrypted token whose content is a signed one, or the payload has a particular type, a JWK key for instance.

If the media type or content type you use starts with application/ and contains no other slash, you may drop the prefix: application/secevent+jwt becomes secevent+jwt.

The same RFC notes that some payload members may be duplicated in the header. We will see later why that is useful.

{ "typ": "JWT" }

Or:

{
  "iss": "My Great Service",
  "exp": 1502749759
}

Not much to show for it. Two optional headers, one of them discouraged, and a payload whose members are optional too. What is the point?

The point of JSON Web Tokens lies in what you can do to them: signing the data, or encrypting it. Other headers exist for those operations, and some of them are mandatory. That is where this gets interesting, and where the next part picks up.

In practice

Nothing exotic: the header is built exactly like the payload.

<?php
$data = [
    'iss' => 'My Great Service',
    'exp' => 1502749759,
];

echo json_encode($data);

Two JSON objects, then. What turns them into a token is the subject of the next article.

Tagged: , , , ,