ETag, If-Match, NGINX, and you!

I’m building a microservice for a client. That microservice allows users to update their cart from different versions of client’s site: their website, their mobile app, and any future implementations of their shop. All of these would be using one data store to keep track of what’s in the customers’ carts.

Which presents a problem. If Alex Smith opens the website and the app at the same time, they will see the same cart contents. Then Alex prepares to change quantity of item 1 on the website, then removes item 1 from the cart from the app, and then submits the quantity change on the website, we get to a point where the system is trying to update a cart item that no longer exists. This is the gist of the lost update problem.

Enter If-Match

If-Match is a request header you can send down with a value of ETag hash in it. According to the RFC that defines If-Match, the server should only ever act upon the resource if the ETags match, or there’s a current representation of the resource.

Server needs to implement hashing of the data and generating the ETag, and server also needs to actually do the comparison and rejection in case of a non-match. For example a hash of the json representation of the data being held will work.

For testing, I’m just setting an arbitrary ETag, so I can at least start doing comparisons. Middlewares are perfect in Laravel for this, and setting an ETag on a response object is even easier:

<?php
namespace App\Http\Middleware;

use Closure;
use Illuminate\Http\Request;

class AppendETag
{
    public function handle(Request $request, Closure $next)
    {
        $response = $next($request);

        $response->headers->add([
            'Foo' => 'somevalue',
        ]);

        $response->setEtag(sha1('someetaghere'));

        return $response;
    }
}

I set a Foo tag to really see whether my middleware works. This will make sense in a second.

That code also sets the ETag header on the response, and bam, I have access to it! Or at least I should.

Dude, where’s my header?

Except I don’t. I confirmed that the middleware is working, as other headers are being set, but not the ETag.

My middleware works, the Foo tag shows up, but the ETag header is... just missing.

NGINX ate my header

After a long (long, long, long, long, long) session of debugging and sticking dd() calls everywhere in Laravel core, I still could not find where that was stripping the header. I had assumed that there was a middleware being called after mine, but I got as late as $response->sendHeaders(), and the ETag is STILL THERE, the php function header() is still being called with the ETag in it.

Which can only mean one thing: NGINX messes it up.

Stack Overflow and some forum posts suggested it’s something to do with Gzip, which would make sense, given compression can be variable, so byte-by-byte comparison is moot then, but in this case I’m using ETag for my own reasons, and I still want it.

So I opened up the nginx.conf file, and started commenting things out until things started working.

Enter SSI

SSI, or server side includes, is a module that strips a bunch of tags from the response when it’s configured.

Unfortunately it also removes the ETag headers, as well as the Last-Modified one too. If I tell nginx (ssi_last_modified on;) to keep the last modified header on, it swaps the ETag header to a weak hash, which is still not great.

For the curious, the source where ETag headers are stripped in the NGINX source code is this:

    if (r == r->main) {
        ngx_http_clear_content_length(r);
        ngx_http_clear_accept_ranges(r);

        r->preserve_body = 1;

        if (!slcf->last_modified) {
            ngx_http_clear_last_modified(r);
            ngx_http_clear_etag(r);

        } else {
            ngx_http_weak_etag(r);
        }
    }

For now, SSI needs to be turned off on routes in NGINX where you want to use ETags. The default is off though, but Laravel Valet sets it to on.

Exit SSI

With ssi off;, or just not set, the same request I made above becomes this:

Photo by Kelly Sikkema on Unsplash