Observability

Error tracking is Sentry, but not the Sentry SDK. Each service carries a hard budget of fifteen dependencies, and adding @sentry/node across nine services would have blown it. So the client is ~250 lines of hand-written code that speaks Sentry’s envelope ingest protocol directly over fetch, plus a self-contained ~175-line mirror for the Next.js portal (so its image installs zero workspace packages), with an anti-drift spec keeping the two honest.

What makes it a real client rather than a shortcut:

• Nothing leaves unscrubbed. Before transport, every event is deep-copied and any key matching the telemetry system’s forbidden-content set is redacted, along with auth headers and JWTs. Same list, same definition of “content,” everywhere.
• Inert by default. With no DSN configured it’s a no-op, so local development stays clean and the code could merge dormant before the infrastructure existed.
• Fail-safe. A scrubbing or ingest error can never itself produce an error: no recursive error storms.
• The privacy bar is operational. Any DSN rotation re-runs a synthetic check: a content field and a JWT must come back [scrubbed] before the key goes to production.

Uptime is watched from outside the code entirely: four Better Stack monitors hit the portal and a /health/deep endpoint across staging and production, with email and mobile-push alerting. The app’s contribution is that /health/deep actually exercises Postgres and Redis and returns 503 when a dependency is down. A health check that can’t lie.

Watching the system, not just the servers

Everything above tells you whether the machine is running. None of it tells you whether the machine is working, and I learned the difference the hard way, from a real session with real people.

The telemetry substrate was fine. The operating layer on top of it did not exist. Reconstructing what had happened in a single afternoon took about fifteen hand-written database joins across four different tables and four containers’ logs. Errors that users had actually hit existed only in container logs: no events, no alerts, nothing that would ever reach me unless I went looking. There was no concept of a session anywhere in the system, so “did they come back?” was unanswerable. And failed generations burned real money that was recorded nowhere, because the cost table counted successes.

The fix was a journey read model derived from the existing event spine: no new service, no external analytics product, no parallel store. On top of it: proper request attribution (the gateway’s logger ran before authentication, so every request had been recorded as anonymous), error events promoted to first-class citizens with alerting, and a daily report that arrives by email whether or not I remember to look.

One distinction worth designing in from the start: an event log records what happened, not what a user did. Background jobs, retries and scheduled work all emit events, and they are not behaviour. Any metric meant to describe people needs an explicit, separate notion of a user-initiated event; otherwise the number you’re reading is partly a measurement of your own machinery.