Notes/TaskFused

TaskFused Launches, Running Duplicate Requests Once Even When Workers Crash

TaskFused, the studio’s thirteenth project, does expensive work once when it’s asked for several times. Each piece of work gets a name. The first request creates an operation in a local SQLite file, and equal requests join it, whether they come from the same program, another process or a retry after a timeout. They all wait for the same result, and later requests get the stored one. Version 0.1 came out on 18 October as a Go library, a CLI and a loopback HTTP API, with binaries for Linux, macOS and Windows. The code is open source under Apache 2.0.

The idea is Go’s singleflight, made durable. Workers hold each attempt on a lease, so when one dies mid-job another takes the operation over under the same ID, and a late result from the dead worker is turned away. The same name with different input is refused, so two different jobs can’t end up sharing an answer.

The test write-up kills real worker processes and forces 26 storage failures across 13 kinds of write, each rolling back with nothing half-saved. In the recorded demo, 100 identical requests made one operation and one run, and a killed worker’s job finished on a new one about 1.1 seconds after the kill.

The limits are stated plainly. It runs on one host. It runs work at least once, so outside effects should use the operation ID as their own idempotency key. And you choose what counts as the same request by choosing the name. Where Ukue queues every job it’s given, TaskFused looks at a job’s name first and joins work that already exists.