Observability
Observability is the practice of recording what a system actually did in production (every input, output, step, timing and cost), so you can see its real behaviour, diagnose failures and prove what happened, instead of inferring from averages.
No jargon, no math
In plain words, what is it?
Monitoring tells you a number is wrong; observability lets you see why. The rule is simple: every request emits a trace, a full record of what happened. For an AI system the trace holds the input, the retrieved documents, the prompt, the model's answer, the tools it called, plus latency and cost. Traces stream into a store where dashboards aggregate them into trends and alerts fire when a metric leaves its band. When something looks off, you do not guess: you open the trace of the actual failing request and read the steps. For agents this matters doubly, because one user question can trigger a dozen model calls and tool calls, and the mistake hides in one of them. Where it breaks: if you log only errors, or nothing at all, the first sign of trouble is a customer complaint and debugging turns into archaeology.
- 01
Trace. Every request records its full journey: input, context, answer, tools, latency, cost.
- 02
Store. Traces land in one place, searchable and kept for months.
- 03
Watch. Dashboards turn traces into trends; alerts fire when a metric leaves its band.
- 04
Open. A failing trace shows the exact step that went wrong.
- 05
Fix. The alert that caught it becomes part of the system.
Terms that live nearby.
MLOps
MLOps is the set of practices, tools and pipelines that take a machine learning model from a data scientist's notebook into production and keep it running reliably: versioning data and code, training, evaluating, deploying and monitoring the model like any other business-critical software.
ReadLLMOps
LLMOps is the discipline of running large language models in production: managing prompts, context, costs, latency and outputs with the same rigour MLOps applies to predictive models, adapted to models that answer in words and can say almost anything.
ReadModel Drift
Model drift is the silent loss of accuracy that happens when live data stops resembling the data a model was trained on: the world moves, the model stays still.
Read§ 04
Want this running in your stack?
We design and ship AI systems with permissions, logs and human approval. It starts with a 30-minute scoping call.