Stage one, as built

The full disputes system was designed around a pipeline that already existed. Stage one (read the letter, work out what it is, persist the result) was running as a batch processor before the investigation engines, the case pack or the platform contracts were written. It shipped early because it was the highest-volume, best-defined slice: classification alone removes the first several minutes of every case.

The pipeline

One async function, deterministic end to end:

OCR (per page)
  → form detection        pure functions over the text; no model
  → form resolution        deterministic and authoritative
  → form analysis          only for the resolved type; concurrent if several
  → knowledge-base load    cached; one workbook per classification axis
  → classify               text model, JSON mode, Pydantic-validated
  → persist                warehouse insert + a marker file downstream depends on

Form handling. Four outcomes: a single recognised form, several (each analyser runs concurrently and the results are merged), or none, in which case classification proceeds on the OCR text alone. One form type is resolved entirely from the OCR layer’s selection marks and geometry. Two are read by GPT vision under an observe-only prompt. The form type the resolver picks overrides any form type the classifier might emit. The model is not consulted on routing.

Classification. Grounded on a canonical folder of business-owned workbooks, one per axis: the dispute reason (single, cited), the customer’s requests (multi, each cited), and the complaint issue type (single, cited, optional, tiered by how well-defined the type is). Plus the identity fields and whether an attorney or a debt-management firm is writing. Output validates into a typed result; every value must be on its governed list; every conclusion carries a citation into the letter text. The prompt is versioned by date and pinned with the workbook version it was written against.

Persistence. Two warehouse tables. A status table keyed on the document (in progress, completed, failed, with a processed timestamp) is the idempotency ledger: the runner claims a document by inserting or flipping its row, and a redelivered or re-run document is skipped if already completed or in progress. An output table holds the classification result, the citations as JSON, and the form type. Adding the complaint axis meant two new columns and a one-time DDL change ahead of the batch, which is the kind of downstream contract the marker file also represents: preserve the shape exactly, because something you don’t own reads it.

The batch runner

Iterates a watch folder, claims each document through the status table, runs the pipeline, writes the row and the marker, and sets a terminal status. Concurrency is bounded by a semaphore. Every blocking call (the OCR client, the database driver, the PDF library) is wrapped so the event loop never stalls. The bound was tuned by watching the OCR service’s rate limits rather than the CPU.

What the design kept, and what it replaced

Almost everything above survives into the full system unchanged. The classifier becomes an agent in the orchestrator graph and gains an intent axis that routes deadline clocks. The form detection stays exactly as it is. The status table becomes the idempotency ledger for the queue consumer. The marker file stays until the case-pack store replaces its reader.

What went: an LLM root agent that called the classifier as a tool (replaced by a deterministic orchestrator; the LLM agent stays as a colleague Q&A front door), service-principal secrets in environment variables (Managed Identity), and a shared database cursor (a pool).

What the code review found

The stage-one review produced the first iteration’s backlog, and most items were the gap between “works on a laptop” and “runs in a container off a queue”: vision through the abstraction layer unproven at volume; a browser-based login flow and network share paths that would not exist inside Container Apps; and a cursor shared across concurrent tasks. None were bugs in the classification. All were the reasons the platform has a document reader and a fetch layer instead of each project having its own.