The boring stack that keeps AI agents shipping

Quality is system design, not prompting

When an agent misbehaves, the first instinct is to edit the prompt. Sometimes that is right. Far more often the prompt was fine and the system around it had no way to notice the failure, no way to stop it, and no way to undo it. "Agent quality" turns out to be mostly a property of the plumbing: what the agent is allowed to touch, what it records, what happens when a call fails, and who gets to say no before anything irreversible happens.

That reframing changes where the effort goes. Instead of a week on prompt engineering, you spend a week on the parts below. They are not novel. That is the point. Novelty is where the bugs live.

The minimum viable agent platform

Four pieces, and we do not run an agent in production without all four.

A tool registry. Every action the agent can take is a named tool with a typed input and a typed output. No free-form shell unless the shell is itself a registered tool with a scope. If you cannot list the tools, you cannot reason about what the agent can do, and neither can the agent.

Permissions, scoped per agent. Which tools, on which resources, read or write. We treat this as a matrix and keep it in version control. A policy you cannot diff is a policy nobody reviews. The default is read-only, and write access is granted per workflow, not per agent.

State the agent does not own. Memory, task queues, and run history live outside the agent process in a store you can inspect and back up. An agent that owns its own state cannot be restarted safely, and cannot be replaced by a different model without starting over.

Logs for every call. Every tool invocation, every model call, every decision, with a run id that ties them together. This is the piece people skip because it is tedious, and it is the one that pays off first.

Logging and traceability

The question you will actually ask in production is not "what did the model say" but "why did the agent do that, on Tuesday, to that record." Answering it requires three things: a trace per run, structured fields rather than prose, and the inputs preserved alongside the outputs. We log the prompt version, the tool calls with arguments, the raw responses, and the final decision, and we keep them long enough to answer a customer question a month later.

One rule that has saved us repeatedly: a log line that says "N started" must have a sibling that says "N finished" or "N failed." A terminal state that is never written is indistinguishable from a hang, and it hides whole classes of bugs. We found a purchase funnel once that had been recording starts for months and had never once recorded a completion. Nobody noticed, because nothing was failing loudly. The terminal state was swallowed on both platforms, for two different reasons.

Retries, fallbacks, and the honest error

Models time out. APIs return 502s. Tokens are issued a few seconds in the future and rejected by a server whose clock disagrees. Each of these needs a different response, and the mistake is treating them all the same.

Retry only the errors that are actually transient, with a bounded count and backoff. Fall back to a smaller model or a cached answer only when the fallback is labeled as such in the output, so the downstream consumer knows it got the degraded path. And when neither applies, fail honestly: surface the error to a human with the trace id, rather than returning a plausible-looking result. A confident wrong answer is worse than a visible failure, and an agent that never errors is an agent that is hiding its errors.

Versioned prompts

Prompts are code. They go in the repo, they get a version, and the version is written into every log line the prompt produces. When behavior changes, you can bisect. When a customer asks why the agent did something last quarter, you can read the exact instructions it had at the time. Prompts edited live in a dashboard with no history are the AI equivalent of editing production config over SSH.

The cheap addition that makes this work: a small fixture set of inputs with expected outputs, run on every prompt change. It will not catch everything. It catches the regressions you would otherwise find out about from a user.

Human checkpoints, where they actually matter

Not every step needs a human. Putting an approval on everything trains people to click approve without reading, which is worse than no checkpoint at all. The checkpoints that matter sit in front of actions that are hard to reverse or that face outward: sending a message to a customer, moving money, deleting data, deploying, publishing. Everything upstream of those can run unattended, because everything upstream is a draft.

The pattern we use is triage, then draft, then approve, then execute. The agent does the first two on its own and stops. A person approves. The agent executes. The approval step is the product, not an inconvenience bolted onto it.

Observability: traces, evals, regression tests

Logs tell you what happened. Observability tells you whether it is getting better or worse. We keep three layers: traces for individual runs, a small evaluation suite that scores output quality on a fixed set of cases, and regression tests that pin down the behaviors we have already fixed once. The eval suite does not need to be large. Twenty representative cases, run on every change, will catch most of what matters, and the discipline of writing them forces you to decide what "good" means.

Rollback and idempotency

Two properties, and you want both. Idempotency: running the same step twice produces the same result, so a retry after a timeout does not create a duplicate record, send a second email, or charge a card again. Rollback: every change the agent makes can be undone, which in practice means snapshots before writes, soft deletes instead of hard ones, and a deploy that keeps the previous tree on disk.

Every production deploy we run leaves the previous version in place with a timestamp and a database backup beside it. The rollback is a directory rename. It has never been exciting and it has never once been regretted.

Three failures these pieces prevent

The duplicate post. A content pipeline published to four platforms, then crashed before writing the status back. Next run, it read the same "ready" entries and published them again. The fix was a writeback step and an idempotency key per post. The lesson was that the writeback is not bookkeeping; it is the thing that makes the pipeline safe to re-run.

The clobbered memory. An hourly export replaced an entire manifest instead of merging into it, silently wiping rows that had been written by a different process. Nothing failed. The data was just gone each hour. Row-level provenance and a merge instead of a replace fixed it; a log line that recorded "rows before" and "rows after" would have caught it on day one.

The agent that switched branches. Two agents sharing one repository, and one of them checked out a different branch mid-edit, deleting the other's working files. Scoped permissions and a pre-edit verification step prevent it. We now check the branch before every non-trivial edit cycle, which is boring, and which works.

A reference shape

trigger (schedule / webhook / human)
   |
   v
agent run  -- reads: memory store, task queue
   |          writes: trace (run id, prompt version, tool calls)
   v
tool registry  -- typed tools, per-agent permission matrix
   |
   v
draft  -->  human checkpoint (only before irreversible / outward actions)
   |
   v
execute  -- idempotent writes, snapshot before, rollback path kept
   |
   v
writeback  -- terminal state recorded: finished | failed, with reason

None of this is clever. All of it is the difference between an agent you demo and an agent you run. If you are building one and want a second set of eyes on the plumbing, tell us what it does and we will tell you which of these you are missing.