My agents livelocked on a paragraph of prose

The first time I ran several coding agents in parallel on the same repository, they stopped finishing work. Not crashed, stopped finishing. Each one was doing its job correctly, and collectively they made almost no progress for the better part of a day.

The cause was not a race condition, a lock, or a model doing something stupid. It was a checkbox in a Markdown file.

The mechanism

Every agent that shipped a piece of work updated three shared tracking documents: a checkbox line in the backlog, a status row in an index, and a change-log line. Sensible bookkeeping. With one agent, invisible.

With three, this happens:

1. Agent A and Agent B both finish. Both edit the same backlog file, on different lines.
2. A merges first. B’s branch now conflicts: in prose, in a document, with no bearing on B’s code.
3. B rebases. Its branch carries code, so the entire test matrix re-runs, many minutes of CI.
4. During that run, Agent C merges. B conflicts again.

Nobody is stuck. Everybody is busy. Throughput collapses. It’s a livelock, and it’s particularly nasty because every individual component is behaving exactly as designed. There’s no error to find, no test to fix, nothing that looks like a bug in any log you’d think to open.

The pathology is a specific one: a low-value shared write became coupled to an expensive verification. The bookkeeping wasn’t costly. Making the bookkeeping conflict with code was.

The fix, which is a file layout

The tempting move is coordination: a lock, a queue, a merge protocol, an agent whose job is arbitrating. I’d encourage you to resist that, because coordination is the thing you’re trying to avoid needing. I settled on three rules instead.

1. Single writer. Every file has exactly one writer at a time. A work-item’s brief is written only by the agent running that item. The queue is written only by the dispatcher. If two agents can write one file, that’s a design bug: not an accident to be managed, a defect to be removed.

2. New facts are new files. Anything with multiple producers (bug reports, run records, changelog entries, verification verdicts) becomes one file per fact in a directory. Bug reports go to bugs/bug-<date>-<slug>.md. Each release writes its own changelog/vX.Y.Z.md.

This is the whole trick, and it’s almost embarrassingly simple: distinct files never conflict in git. Two agents creating two files in one directory is a non-event. Readers glob. There is no merge, so there is nothing to resolve.

3. Indexes are compiled, not edited. The aggregate views (the backlog checkboxes, the status counts) stopped being written by hand. Each item’s status lives in its own brief’s frontmatter, and a small dependency-free script derives the aggregate view from all of them. It runs in exactly one place, after merge, on the main branch.

The compiler has to earn trust, so it’s built to be boring: idempotent (running twice produces no diff), minimal-diff (it patches lines, never rewrites prose), and monotonic (frontmatter that would un-ship a shipped line is a warned no-op, not an edit).

What it bought

Parallel agents now touch disjoint files by construction. Not by convention, not by politeness, not by a protocol they might reason their way around. By the shape of the directory tree.

There was a second win I didn’t design for. Status updates became documentation-only commits, and the CI configuration already skipped those paths. The expensive half of the original problem (a prose conflict triggering a full test matrix) disappeared twice over.

The generalisable bit

I think there’s a real principle here, and it applies well beyond coding agents:

Most multi-agent state problems are better solved by choosing a layout that cannot conflict than by adding a protocol that resolves conflicts.

Coordination is expensive, and it’s expensive in the specific way that gets worse with scale. Every lock is a queue. Every merge protocol is a thing to debug at 2am. Whereas “these two writers touch different files” costs nothing at any scale, and can’t be violated by an agent having a bad day or a prompt drifting.

The corollary I’d offer to anyone building on a repository: git is already a blackboard with a free audit log. You have durable shared state, atomic writes, full history, and conflict detection, for nothing. You almost never need to add a database to your agent system. You need to stop making your agents fight over the same lines.

The one new failure mode

Honesty requires the cost. Making the backlog partly generated introduced exactly one new mistake, and it’s mine to make: hand-editing a compiled line. It looks like it worked, right up until the compiler runs and overwrites it from the source of truth.

The banner at the top of the file explains this. I have still done it twice.