Alter

The problem

Personal knowledge tools tend to be one of two things: a dumb store that never understands what you put into it, or a smart assistant that ships your private data to someone else’s cloud. Alter is built to be neither — a system that genuinely understands what you give it and compiles it into something useful, while running entirely on infrastructure you control.

Architecture

Alter is a TypeScript / NestJS microservice system. At its centre is a recognition-and-orchestration layer: an LLM classifies unstructured input, routes it across five interaction modes, and an agentic harness runs the multi-step, multi-mode workflows behind recognition, briefing, research, and decision-cascade flows — with automatic re-grounding on long-running reasoning, branching, and end-of-thread synthesis. Everything compiles into a per-user knowledge wiki with pgvector-backed hybrid (semantic + keyword) retrieval, fed by an event-driven reindex pipeline.

A multi-provider abstraction (Anthropic / OpenAI / Google / Perplexity) handles complexity-based cost routing and prompt caching, and enforces strict structured output through provider tool-use + Zod schemas with a retry-repair flow and safe fallback — never prompt-only JSON.

Key decisions & tradeoffs

• Prompts are tested, versioned components, not ad-hoc strings: every prompt and mode is backed by golden eval suites run as CLI eval runners with per-run cost budgets.
• Privacy is an invariant, not a feature: encryption-by-default on all content, CI-enforced privacy checks, and self-hosting so regulated data never leaves the environment.
• Cost is bounded by design: per-user LLM cost caps with graceful degradation; the provider router trades a thin abstraction layer for resilience and predictable spend.

What broke / what I’d do differently

The hardest lessons were operational rather than architectural: keeping evals honest as prompts evolved, and making failure modes — provider outages, malformed output, cost spikes — degrade gracefully instead of cascading. Detailed write-ups of specific incidents are coming.

Outcome

Alter runs in production as a daily-use system. The patterns it forced — evals-as-tests, structured-output-by-contract, multi-provider routing, privacy-as-invariant — generalise well beyond it, and several are being factored out as open components.