Benchmark

Lanes benchmark: 46% to 0% broken builds

In shortIn the scenario where one agent renames a function while another agent writes a new caller, file locks alone broke the build in 6 of 13 runs (46%). With Lanes' shared context, 0 of 10. When edits were refused the old way, 4 of 13 agents (31%) stopped and asked their human what to do; with Lanes' queue, 1 of 216 sessions (0.5%) did. With 64 agents writing at once, no edit was lost. The check before an edit adds 39 ms on Linux and 68 ms on Windows at p95.

Updated 2026-11-12 · Lanes 0.3.0 · Source on GitHub

Summary

Behind these numbers: 389 real Claude Code sessions (173 in the early spikes, 216 in 108 benchmark runs) and 419 automated tests that run on Linux, Windows and macOS for every change.

How the benchmark works

Each run creates a fresh git repository from a template, starts a Lanes server and launches two real Claude Code agents with different tasks on separate clones, as if two teammates worked on two laptops. The second agent starts 8 seconds after the first. When both are done, the runner checks the final code on lanes/main by running it: a route must answer, a total must be right, the app must load.

Three setups ("arms") run the same scenarios:

• C: Lanes leases and checkpoint commits, no shared context. This is close to what file locking alone gives you.
• D: C plus team context as first worded, on 3 November 2026.
• D2: C plus team context as shipped in Lanes 0.3.0: a heads-up when an agent reads a teammate's file, a check at the end of a turn, and wording that asks the agent to match the teammate's change.

The method, the success bar for D2 and the metrics were written down before any run. All 90 runs of the main round are reported, none excluded.

The scenarios

The scenarios are chosen so that file locks cannot help: the second agent writes its new code in a new file that the first agent never touches.

Results, run by run

In an earlier round on 3 November, arm C failed I1 in 3 of 3 runs. The leases-only code did not change between the rounds, which is how the summary above counts 6 broken builds in 13 runs.

The read heads-up is what moved I1 to 10 of 10: it fired twice in every D2 run of I1, and in no run of D. The one D2 miss on I2 was the renaming agent's own mistake, not a collision.

No agent in D or D2 stopped to ask its user anything. No edit was left unpublished and no line was lost in any of the 90 runs.

Scale: 64 agents at once

A separate test drives the hooks the way real agents do, with no model involved, to find where the server breaks. Agents are spread over teammates, two repositories and four agent kinds (Claude Code, Codex, Gemini CLI and Cursor), with 10 or 60 open views.

• With 8, 16, 32 and 64 agents and 60 viewers, no edit was lost. In the 64-agent run, 7 edits waited committed in their clone for the next publish, as designed.
• The server stayed at about 70 MB of memory.
• Every server route answered in under 2 ms at p99, except the lease request, which includes the time spent waiting in the queue.
• The script edits far more often than a real agent: a write every 1.5 to 4.5 seconds per agent, 70% of them to one hot file. Waits on that hot file grow with the number of agents.

Hook latency

Every edit goes through the hook first, so it has to be fast. The bundled hook, for an edit of a free file, process start included, measured in CI at the 95th percentile: 39 ms on Linux, 68 ms on Windows, 79 ms on macOS. In real Claude Code sessions on Windows, hook start-up before the request was 31 ms at p50 and 84 ms at p95.

Getting there took one bundled file instead of TypeScript source (166 ms to 130 ms), caching the repository identity on disk (to 84 ms) and using node:http instead of fetch, which costs about 45 ms to load in every short-lived process.

Sources

Every number on this page links back to a file in the repository:

• Method, written before the runs
• Intent conflicts, round 1 and round 2, all 90 runs
• Scale test
• Hook latency

Questions