Balance support and misclassification
The two reasons placements get rejected, an unsupported balance and a document that isn't what its index says, and how each is decided by code over model-extracted evidence.
The team’s own rejection history pointed at two failure modes above all others. A claimed amount that doesn’t match the affidavit. And no acceptable statement supporting the charge-off balance. Underneath both sits a third, quieter one: a document that exists in the index under the right label and is something else when opened. The records system has one engine for each.
Misclassification detection
The vision classifier reads each retrieved file (title, layout, phrases, seller identity, table structure, signature blocks) and predicts a type from the catalogue’s allowed list, with a confidence and a page citation. That is where the model stops.
The mismatch decision is deterministic and lives in the project: compare predicted to indexed; a disagreement above the confidence threshold is a fail with the page cited; a low-confidence prediction is a review; agreement is a pass. The canonical case is a transaction history indexed as a final statement. The classifier catches it on layout alone, and the reviewer sees the page.
Two things keep this honest. The allowed list is scoped by product, so the classifier cannot “find” a document type that this product never has. And the misclassification-recall gate sits at 97%: a missed misindex is a document that doesn’t say what the placement claims it says.
Balance support
The question: does an acceptable document actually support the balance being asserted?
- Identify seller and product; from config, which balance must be supported and which document types are acceptable support for this seller. Some originators require a specific statement; for some the requirement depends on a charge-off date threshold; some are exception rows with no statement at all.
- Enumerate candidate documents from the inventory that match those types and pass their date windows.
- Extract the candidate balance from each, through the shared extractor, with page and region.
- Decide, in code: exact support, support under an approved tolerance or adjustment row, no supporting document, conflicting candidates, unreadable, seller exception, or review.
The tolerance point deserves emphasis. The non-goals say no tolerance is invented. Any “close enough” is a versioned config row that the business and compliance approved, with the seller it applies to. If no row exists, a difference of one cent is no support, and a reviewer decides. This was the single most tempting place to let the model be helpful, and the single most important place not to.
Complications the real accounts introduced
Labels that arrive late. Statements aren’t renamed to their roles (charge-off, last-pay, last-purchase) until a downstream process runs. Validate ahead of it and the engine has to infer the relevant statement from the closest date to the account’s own dates, and say that it did.
More than one account number. Numbers can change at or around charge-off. The reconciliation normalises and compares against the set of known identifiers, not one.
The field that isn’t in the system of record. One of the dozen data points isn’t held by the primary system at all; most sellers never provide it. Where it exists it comes from the warehouse. The fetch registry records that per field, so the comparison knows which source system it is citing.
The regression suite
Each historical rejection (a set of real accounts across several seller families, each with its known reason) is a golden case with an expected outcome and a business explanation. The exit gate for this component is that every one reproduces. Not “recall above a number”: every one. They are the reason the project exists, and a version that can’t reproduce them has no claim to replace the audit that found them.