Aftermarket Lets AfterDiff Test a New Version on Live Traffic

Aftermarket 0.10.0 gives AfterDiff a live shadow mode. aftermarket shadow sits in front of the version you run today and serves every request from it, unchanged. A copy of each request goes to the new version in the background, and the two answers are compared with the same rules replay uses. The result is a running report of where the new version differs, written while real traffic flows.

The client only ever gets the current version’s answer. Copies wait in a small bounded queue, and when it’s full the copy is dropped and counted, so a slow or dead new version can’t hold anything up. Only reads are mirrored unless you ask for writes too, and record’s redaction keeps credentials out of the report.

In the tests, 300 random requests came back byte for byte the same with shadowing on, against a new version that answered differently, failed, hung or cut the connection. With six kinds of bad new version, every answer was right and none took over 400ms. The added time was about 0.2ms per request even with the new version down. On the customer-api example, the old client passed 12 of 12 checks both times: the report read 12 of 12 the same with the new version behind AfterAPI, and 12 of 12 different without it. Binaries are out for Linux, macOS and Windows.

An independent review then found five bugs, fixed in 0.10.1. None changed what clients got back, but one passed a proxy credential to the new version and four made the report count wrong. Three case studies followed on 0.10.1. A Datasette upgrade from 0.65.5 to the 1.0 alpha showed the new JSON shape, a redirect for SQL queries and a reworded 404, once four ignore rules took out the noise that differed even between two copies of the same version. The json-server rewrite, shadowed with writes, showed string ids, changed query parameters and a different DELETE answer. Under load, every request was served while the full queue dropped copies, and sampling at a tenth sent exactly 600 of 6000. The recipes cover ignore rules, writes and redaction. The previous note covers AfterBridge.