LLMs extract, code decides

At work I build agentic systems for a collections business under US regulation. The two I own read scanned correspondence and legal paperwork, check them against a system of record, and hand a colleague an answer: is this person who they say they are, does this account have the documents it needs, is this balance supported. Every one of those answers has legal weight. Some of them, wrong in the wrong direction, are a reportable event.

The instinct when you first hold a model that can read a scanned statement and tell you what it means is to ask it the real question. Not “what is the balance on this page” but “is this account ready for legal.” It will answer. It will answer fluently, with a rationale, and it will be right often enough that the demo lands.

I’ve come to think that’s the single most dangerous thing you can do with these systems in a regulated setting, and that the fix is a rule short enough to fit on a sticky note: the model extracts; the code decides.

A decision is anything an auditor could ask you to defend

A decision, in the sense I mean, is any conclusion an auditor might ask you to defend. Verified. Complete. Supported. Escalate. Reject. These have three properties in common that a model’s output doesn’t have and can’t be made to have:

1. They must be explainable in terms of a rule. “The balance on the charge-off statement matched the claimed balance under the approved tolerance for this seller” is a sentence a compliance officer can check. “The model judged the balance to be adequately supported” is not, however good the model’s rationale reads.
2. They must be stable across model versions. I’ve written about a prompt that started refusing after a routine model upgrade. That was a personal product; the failure was annoying. If the same thing happened to a check that clears accounts for legal placement, the failure would be a month of wrong verdicts nobody noticed, because the outputs would still be well-formed and confident.
3. They must be changeable by the people who own them, without a release. The seller list, the tolerance, the required-document set, the threshold at which a signal becomes a flag: these belong to the business and to compliance, and they change. A rule in a prompt is a rule only an engineer can change, in a way nobody can diff.

Extraction has none of these constraints. “What is the balance on this page” is a question whose answer can be checked against the page. It can be wrong, and the wrongness is visible, local, and measurable against a labelled set. That’s the property you want your model doing.

The model is only ever on the left of the line

So the architecture draws a line between what the document says and whether that’s acceptable, and the model is only ever on the left of it.

On the left: read this page into typed fields with a confidence and a bounding box. Predict this document’s type from a fixed list. Read this statement into a list of transactions (date, merchant, amount, channel, country, posted or returned), one citation per line.

On the right: is this predicted type different from the indexed type by more than the threshold. Does this required set, resolved from product and seller and jurisdiction, have at least one document present in the inventory. Does this transaction list trip any of the rules in the flag registry. Does the identity match score clear the verified threshold. What action does the decision table select for this combination of outcomes.

Every item on the right is a function over config. Every item on the left produces a value with a citation. The interface between them is a small set of typed contracts: a finding, made of comparisons, made of extractions, each carrying where it came from. The platform is basically this line, made into code.

Extraction is judgment too, but it fails locally

It is. A model reading a handwritten checkbox is making a judgment, and it will sometimes be wrong. The difference is what kind of wrong.

An extraction error is a wrong value with a page reference. It’s caught by a confidence gate, a schema validator, a fallback OCR tier, a golden set, or a colleague looking at the highlighted region and saying “no”. It fails locally.

A decision error is a wrong verdict with a plausible rationale. It’s caught by nothing structural, because the structure trusted the model with the verdict. It fails globally, and it fails in the direction the model finds most fluent, which is rarely the conservative one.

Keeping the model on the extraction side doesn’t make the system infallible. It makes its failures the checkable kind.

Three times the model started in the wrong seat

Three examples from the systems, each of which started with the model in the wrong seat.

Form detection. The first version of the correspondence pipeline let the classifier say what kind of form it was reading. It was usually right, and when it was wrong the wrong analyser ran on the wrong form and produced confident nonsense. Now a handful of pure functions detect forms from phrases in the OCR text, a deterministic resolver picks the type, and that overrides anything the model believes. There’s a test in the safety suite that asserts the override holds under adversarial letter content.

Fraud signals. The expert matrix lists signals like “purchases only in the first sixty days” and “multiple addresses in a short window”. The tempting build is a prompt that reads the statement and answers “are any of these present”. The build that shipped reads the statement once into a normalised transaction list, and eleven small deterministic rules consume the list. The model never decides fraud. It produces records; code applies thresholds; every flag quotes the statement line. And where the matrix said “multiple” without a number, the rule emits review until the business supplies one. It does not let the model pick.

The recommended action. The case pack ends with a next step for the colleague. It comes from a decision table keyed on intent, verification outcome, documentation outcome and the highest flag severity. The model’s job is to select and justify from that table (write the two lines explaining why this row) never to propose an action that isn’t in it. The table is versioned, compliance-approved config, and its key stays stable across taxonomy changes so evaluations remain comparable.

The same rule builds my own product

This is the same principle as the one I use for the agents that build my own product: the agent that produced something is not the one that gets to say it’s correct. There it’s a verifier and an adversarial breaker with runnable evidence. Here it’s a comparator function and a citation. In both cases the move is identical: separate the claim from the check, give the model the claim, and make the check something that can’t be talked out of its answer.

The models are getting better. That doesn’t move the line. A better extractor is a better extractor. A better decider is still a decider whose reasoning you can’t diff, whose behaviour changes when someone else ships an improvement, and whose verdict a regulator will ask you to explain in terms of a rule. The rule has to exist. It might as well be the thing that decides.