← Writing Build log

Nothing marks its own homework

How I verify work an agent produced, when the obvious reviewer is the agent that just wrote it, and why the trick is what you refuse to tell the reviewer.

Who should review code an agent just wrote?

the mechanism
implementer → its own reasoning → looks good
work-item path diff range · nothing else → verifier ∥ breaker
verifier re-derives done · breaker → runnable evidence
  1. 01 of 03

    The builder is convinced

    It has spent its whole context justifying exactly these decisions. Asking it to check them is asking it to re-read its argument and agree.

  2. 02 of 03

    A thin brief

    No summary, no reasoning, none of the implementer's context. Context from the builder carries the frame in which the decisions looked correct.

  3. 03 of 03

    Divergence is the signal

    Starved of context, the reviewers reconstruct the problem from the source of truth, sometimes differently. That divergence is the only signal not downstream of the builder.


An agent that has just written some code is the worst possible reviewer of it. Not because it’s careless. Because it’s convinced. It has spent its whole context constructing a justification for exactly these decisions, and asking it to check them is asking it to re-read its own argument and agree.

Human engineers have this problem too, which is why we invented code review. But the usual fix (get a second pair of eyes) doesn’t transfer cleanly, because the obvious way to brief the second agent is to hand it everything the first one knew. And the moment you do that, you’ve rebuilt the blind spot with extra steps.

Here’s what I landed on.

Two reviewers: one re-derives done, one tries to break it

Every change in Alter fans out to two independent agents before it can proceed.

The verifier re-derives what “done” means. It reads the work-item (the acceptance criteria written before implementation started) and checks the diff against them, independently. Not “is this code good,” but “does this actually satisfy what was asked, and how would I know?”

The breaker is adversarial. Its job is to make the change fail, and its output isn’t opinions. It’s runnable evidence. A failing test, a command with its output, a sequence that produces the wrong state. An unreproducible objection isn’t a finding.

Splitting them matters more than I expected. Conflating “does it meet the spec” with “can I break it” produces an agent that does neither well: it drifts toward whichever is easier on the day, and it lets a genuine gap in the acceptance criteria hide behind an interesting bug.

What you refuse to tell the reviewer is the design

This is the bit I’d actually defend as a design principle.

Both are spawned with a deliberately thin brief: the path to the work-item, and a diff range. That’s it. No summary of what was built. No note about which approach was chosen and why. No “the tricky bit was X, here’s how I handled it.” None of the implementer’s context at all.

The temptation to hand over more is enormous, because it looks like efficiency. It is the opposite. Context from the implementer is contaminated. It carries the frame in which the decisions looked correct. Give a verifier the builder’s reasoning and it will grade the reasoning rather than the artefact, and it will usually agree, because the reasoning is coherent. Coherent and wrong is the exact failure you’re trying to catch.

Starving the reviewer of context forces it to reconstruct the problem from the source of truth. Sometimes it reconstructs it differently. That divergence is the entire product. It’s the only signal in the system that isn’t downstream of the implementer’s assumptions.

The acceptance tester cannot read the code, so it cannot be reassured

The same principle appears again, one layer out. Acceptance testing runs against a deployed staging environment, driven from written use cases, by an agent that is structurally blind to the repository. It has no access to the code at all.

It cannot be reassured by an implementation. It can’t read a function and conclude the behaviour must be fine. It has to actually use the thing, like a person would, and report what happened. When it says a flow is broken, that finding has no possible contamination from how the flow was built.

I’ve come to think of independence as an information property, not a personnel one. Two different agents with the same context are one reviewer. One reviewer with genuinely different information is two.

Two fix cycles, then stop and surface

Findings feed back to the implementer for at most two fix cycles. After that the iteration stops and surfaces rather than continuing to grind.

This is a capacity decision more than a quality one. An unbounded verify-fix loop is how you turn a small piece of work into an afternoon of tokens, and the third cycle is almost always evidence that something is wrong above the code: an ambiguous acceptance criterion, a missing decision, a work-item that was never really ready. Those are worth a human’s attention, and they’re cheap to escalate and expensive to grind at.

Three rules, in the order I would defend them

Three rules, in order of how much I’d defend them:

  1. The reviewer’s input must be independent of the implementer’s reasoning. This is the whole thing. Everything else is implementation.
  2. Adversarial review must produce runnable evidence. Otherwise you’ve added a second opinion, not a second check.
  3. Bound the loop, and treat hitting the bound as a signal, not a failure to try harder.

None of this is exotic. It’s the same instinct as double-entry bookkeeping, or a blind trial, or a test written before the code: arrange things so that the check doesn’t depend on the thing being checked. Agents just make it much easier to accidentally skip, because the cheapest reviewer is always right there, already loaded with context, ready to tell you it looks good.