The agent platform

At work I own two agentic systems for a collections business operating under US regulation. One reads incoming disputes and complaints and investigates them before a colleague sees them. The other audits whether an account qualifies before it is placed with a law firm. They serve different teams, different documents, and different rules.

They are the same program.

Strip the domain off either and what’s left is: find the documents, read them, pull out structured values, compare those values against the system of record, and hand a human a finding with a citation for every conclusion. The second system was specced explicitly as “the same shape as the first: swap the domain logic and the orchestration, keep everything else.” So everything else became a platform.

Four steps, one finding, and the model in exactly one of them

Four steps, each a shared component, each with a fixed contract. The model appears in exactly one of them (extraction), and its only job there is to turn pixels into typed values with a confidence and a page reference. Presence is a set comparison. Fetch is a parameterised SELECT. Reconcile is a comparator function the calling project injects. The decision at the end is code.

The output of every check, in every project, is the same object: a finding, carrying a status, the field-by-field comparisons that produced it, the extractions those comparisons were built on, the evidence that was missing, and the citations. Each project maps that finding into its own native result. Neither consumes the raw contract, and neither can produce a conclusion without one. See the verification pattern and the data contracts.

The platform is small; its value is the invariants it makes hard to break

The platform is small. Its value is less the code than the invariants the code makes hard to break:

• Models extract; code decides. No conclusion with legal or compliance weight is a model’s judgment. The model proposes evidence; deterministic rules apply thresholds. The essay on why.
• Every conclusion carries a citation: a document, a page, a bounding region, or a database reference. A conclusion without one is rendered as an unverified suggestion, never as a result.
• Domain logic lives in versioned config, not code. Required-document sets, exception rows, comparison policies, thresholds: YAML validated by Pydantic, versioned under the pipeline version, editable by the people who own the rules without a release.
• The fetch layer is an allow-list. The only way to read the warehouse is through a validated registry of fields: declared tables, declared columns, one bound key, SELECT only. No string from a model or a user ever reaches SQL. The fetch registry.
• Fail closed. A missing input, a low-confidence extraction, or a citation that doesn’t resolve produces review, never pass. The essay.
• Deterministic orchestration on the batch path. No model routes between agents. An interactive agent exists for a colleague to ask questions; the pipeline itself is a fixed graph.
• Structured output everywhere. JSON mode, Pydantic validation, one retry on schema failure, then dead-letter. Determinism pinned: zero temperature, fixed seed, the parameters logged.
• Managed Identity to every dependency. The models, the document reader, the warehouse, the queue, the vault. Secrets exist only as residuals, resolved once at startup, failing fast if absent.

The full list, with the reasoning behind each, is on design decisions.

Shared imports nothing, and projects never import each other

The repo is a monorepo: one shared/ package and one folder per project. shared/ imports nothing from any project. Projects import from shared/ and from themselves. Projects never import each other.

That’s the whole governance model, and it’s what lets the two products ship on independent schedules while running identical, tested mechanics. When a comparator bug surfaces, it’s fixed once. When the second project needed a jurisdiction axis on its required-document model, it went in its own config and touched nothing shared.

Built, and in test under the first product

Built, and in test underneath the first product. The contracts, the fetch engine, the document reader, the evaluation harness and the observability layer are all exercised end to end by the disputes pipeline, which is complete and being tested against a golden set before pilot. The second product is most of the way through its use cases on the same platform.

Vision at batch scale, blocking I/O, and knowing what not to make an agent

Vision through an abstraction layer at scale. Routing GPT vision through LiteLLM was convenient in the prototype and unproven under batch load; it took real work to make image inputs, structured output and retries behave together across a queue-fed consumer.

Blocking I/O in an async runtime. The document reader, the warehouse driver and the PDF library all block. Every one is wrapped so the event loop never stalls, with concurrency bounded by a semaphore, and the first version got this wrong in ways that only showed up under load.

Deciding what not to make an agent. Presence checking, form detection and form-type resolution all started as sub-agents. They ended as pure functions. The routing they do is deterministic and authoritative (it overrides any model’s opinion of what form it is looking at) and a model in that seat was a liability, not a capability.

Pages

• The verification pattern: gather, extract, reconcile, cite.
• Data contracts: the finding, the citation, the comparison.
• The fetch registry: a read-only data layer as a SQL allow-list.
• Document reading: tiered vision-first extraction, with OCR as the fallback.
• Design decisions: the full list, and why.
• Evaluation and AI safety: golden sets, hard gates, and the override flywheel.