Vulcain.rocks: Use preloading to create fast and idiomatic client-driven REST APIs

Vulcain is a brand new protocol using Preload hints and the 103 Early Hints status code to create fast and idiomatic client-driven REST APIs.

An open source gateway server (a module for the Caddy web server), which you can put on top of any existing web API to instantly turn it into a Vulcain-compatible API is also provided!

It supports hypermedia APIs (e.g. any API created with API Platform) but also any "legacy" API by documenting its relations using OpenAPI.

Grab What You Need... Burn The REST!

The protocol has been published as an Internet Draft that is maintained in this repository.

A reference, production-grade, implementation gateway server is also available in this repository. It's free software (AGPL) written in Go. A Docker image is provided.

Introduction

Over the years, several formats have been created to fix performance bottlenecks impacting web APIs: over fetching, under fetching, the n+1 problem...

Current solutions for these problems (GraphQL, JSON:API's embedded resources and sparse fieldsets, ...) are smart network hacks for HTTP/1. But these hacks come with (too) many drawbacks when it comes to HTTP cache, logs and even security.

Fortunately, thanks to the new features introduced in HTTP/2, it's now possible to create true REST APIs fixing these problems with ease and class! Here comes Vulcain!

Pushing Relations

Considering the following resources:

/books

{
    "member": [
        "/books/1",
        "/books/2"
    ]
}

/books/1

{
    "title": "1984",
    "author": "/authors/1"
}

/books/2

{
    "title": "Homage to Catalonia",
    "author": "/authors/1"
}

/authors/1

{
    "givenName": "George",
    "familyName": "Orwell"
}

The Preload HTTP header introduced by Vulcain can be used to ask the server to immediately push resources related to the requested one using 103 Early Hints or HTTP/2 Server Push:

GET /books/ HTTP/2
Preload: "/member/*/author"

In addition to /books, a Vulcain server will push the /books/1, /books/2 and /authors/1 resources!

Example in JavaScript:

const bookResp = await fetch("/books/1", { headers: { Preload: `"/author"` } });
const bookJSON = await bookResp.json();

// Returns immediately, the resource has been pushed and is already in the push cache
const authorResp = await fetch(bookJSON.author);
// ...

Full example, including collections, see also use GraphQL as query language for Vulcain.

Thanks to HTTP/2+ multiplexing, pushed responses will be sent in parallel.

When the client will follow the links and issue a new HTTP request (for instance using fetch()), the corresponding response will already be in cache, and will be used instantly!

For non-hypermedia APIs (when the identifier of the related resource is a simple string or int), use an OpenAPI specification to configure links between resources. Tip: the easiest way to create a hypermedia API is to use the API Platform framework (by the same author as Vulcain).

When possible, we recommend using Early Hints (the 103 HTTP status code) to push the relations. Vulcain allows to gracefully fallback to preload links in the headers of the final response or to HTTP/2 Server Push when the 103 status code isn't supported.

Query Parameter

Alternatively to HTTP headers, the preload query parameter can be used:

Filtering Resources

The Fields HTTP header allows the client to ask the server to return only the specified fields of the requested resource, and of the preloaded related resources.

Multiple Fields HTTP headers can be passed. All fields matching at least one of these headers will be returned. Other fields of the resource will be omitted.

Considering the following resources:

/books/1

{
    "title": "1984",
    "genre": "novel",
    "author": "/authors/1"
}

/authors/1

{
    "givenName": "George",
    "familyName": "Orwell"
}

And the following HTTP request:

GET /books/1 HTTP/2
Preload: "/author"
Fields: "/author/familyName", "/genre"

A Vulcain server will return a response containing the following JSON document:

{
    "genre": "novel",
    "author": "/authors/1"
}

It will also push the following filtered /authors/1 resource:

{
    "familyName": "Orwell"
}

Query Parameter

Alternatively to HTTP headers, the fields query parameter can be used to filter resources:

License and Copyright

tl;dr:

proprietary software can implement the Vulcain specification
proprietary software can be used behind the Vulcain Gateway Server without having to share their sources
modifications made to the Vulcain Gateway Server must be shared
alternatively, a commercial license is available for the Vulcain Gateway Server

The specification is available under the IETF copyright policy. The Vulcain specification can be implemented by any software, including proprietary software.

The Vulcain Gateway Server is licensed under AGPL-3.0. This license implies that if you modify the Vulcain Gateway Server, you must share those modifications. However, the AGPL-3.0 license applies only to the gateway server itself, not to software used behind the gateway.

For companies not wanting, or not able to use AGPL-3.0 licensed software, commercial licenses are also available. Contact us for more information.

Treeware

This package is Treeware. If you use it in production, then we ask that you buy the world a tree to thank us for our work. By contributing to the Treeware forest you’ll be creating employment for local families and restoring wildlife habitats.

Credits

Created by Kévin Dunglas. Sponsored by Les-Tilleuls.coop.

Some ideas and code used in Vulcain's reference implementation have been taken from Hades by Gabe Sullice, an HTTP/2 reverse proxy for JSON:API backend.

Introduction

Pushing Relations

Query Parameter

Filtering Resources

Query Parameter

See Also

License and Copyright

Treeware

Credits