The anatomy of a production-grade coding harness

01 · The harness

A coding agent is a control loop around a fallible planner.

The model proposes the next useful action. The harness decides whether that action is available, allowed, executed, observed, and accepted. It also decides what context the next model call receives. This loop—not the chat panel—is the core application.

A demo loop can be small: send an issue, let the model edit files, run tests, return a patch. A production loop must survive partial state, flaky infrastructure, conflicting instructions, stale dependencies, secret boundaries, long approvals, and repeated failure. It must leave evidence for reviewers and operators.

Six boundaries make these responsibilities concrete. They are not six services that must ship on day one. They are six contracts that should be visible in the design, even when one process implements several of them.

02 · Boundary one

The task contract defines what done means.

An issue description is rarely a sufficient execution contract. It may explain the symptom but omit the affected version, supported behavior, non-goals, rollout limits, or acceptance tests. Humans fill these gaps from team context. An agent will fill them too, but its assumptions may remain invisible until review.

Normalize the task before the run. Record the objective, repository and base revision, allowed scope, required artifacts, acceptance criteria, forbidden actions, time and cost budgets, and escalation conditions. If a key field is unknown, route the task for clarification instead of converting uncertainty into code.

The contract should be immutable for one attempt. New user input creates a new version. This prevents a common audit problem: the final patch is compared with a task description that changed halfway through the run.

03 · Boundary two

The context builder supplies evidence, not a repository dump.

More context can reduce omissions and increase confusion at the same time. Generated files, vendored packages, old design documents, and unrelated incidents compete with the code that controls the task. Large prompts also repeat cost and may push the run toward compaction at the worst point.

Build context in stages. Start with the task contract, repository map, local instructions, and a small set of likely files. Let the agent retrieve more through search and read tools. Add current issue discussion, test failures, ownership, and recent relevant changes when they affect the next decision.

Every retrieved item should carry its source, revision or timestamp, and reason for inclusion. Mark untrusted text, especially issue bodies, documentation pulled from outside the repository, and tool output that could contain prompt injection. Context is data crossing a trust boundary.

Keep durable decisions outside the prompt. A compact state record should track the chosen approach, files changed, checks run, unresolved questions, and rejected paths. Reconstruct the next context from that state and current evidence instead of replaying the entire conversation.

The context window is working memory. Your state store is memory. Your repository and systems of record are truth.

04 · Boundary three

Tools turn intent into bounded operations.

A shell is powerful because it compresses a huge action space into one interface. It is dangerous for the same reason. Production harnesses should expose the shell inside an isolated environment and use narrower tools for external systems such as pull requests, tickets, secrets, deployments, and cloud resources.

Each tool needs a stable name, a precise description, a typed input schema, a structured result, and actionable error categories. The current MCP specification formalizes these elements and recommends input validation, access control, rate limits, output sanitization, user confirmation for sensitive operations, timeouts, and audit logs.

Design return values for the next decision. A create_pull_request tool should return the pull-request identifier, URL, base and head revisions, checks state, and whether review is required. A vague success string forces the model to query again and makes the trace harder to verify.

Separate read, propose, and write operations. An agent can search deployment history freely, generate a rollout plan under a budget, and still require a distinct capability to change production. This gives policy engines clear attachment points and lets teams grant autonomy gradually.

05 · Boundary four

Execution must be isolated, reproducible, and disposable.

Run untrusted code in a fresh workspace. Pin the base revision and dependency sources. Mount only required credentials, and mount them as late as possible. Disable outbound network access by default; allow known destinations per task. Enforce CPU, memory, disk, process, time, and output limits outside the model loop.

The sandbox protects the host and other work. It does not make the result correct. A harmful command may still damage the assigned workspace, leak a mounted credential to an allowed endpoint, or alter test fixtures to create a false pass. Protect critical paths as read-only, keep verifiers outside the writable area when possible, and compare final changes with the task scope.

Reproducibility requires an environment manifest. Record the image digest, repository revision, dependency lockfiles, tool versions, model identifier, harness version, and configuration. If an engineer cannot recreate the failure, the team cannot distinguish a model regression from environmental drift.

Dispose of the execution environment after artifacts and logs are captured. Long-lived workspaces accumulate hidden state and turn passing retries into mysteries.

06 · Boundary five

Verification maps every requirement to evidence.

Tests are necessary and can still be incomplete. A patch may pass unit tests while breaking a migration path, accessibility rule, performance budget, or security boundary. The verifier should begin from the task's acceptance criteria and select the smallest decisive check for each one.

Use layers. Run fast local checks while the agent works. Before completion, run the authoritative test suite in a clean environment. Add static analysis, dependency and secret scans, artifact inspection, and targeted behavioral checks where risk requires them. Verify the diff as well as the runtime result: unexpected files and disabled tests are evidence too.

Keep the authoritative verifier independent from the agent's writable state. Terminal-Bench scores the final container through task tests, and its failure analysis distinguishes missing, incorrect, and weak verification. The same distinction belongs in product telemetry. A run that never checked a criterion is different from one that checked and failed it.

Return a structured verification report. For each criterion, record pass, fail, blocked, or not run; the command or evaluator; the artifact tested; and a compact evidence link. Completion requires the configured set of required criteria to pass. The model does not get to waive them in prose.

07 · Boundary six

Durable state makes interruption and recovery ordinary.

Long runs will stop. Providers time out, containers fail, approvals wait overnight, and branches move. A production harness treats interruption as a state transition, not an exceptional collapse.

Persist the task contract, current phase, chosen plan, files and external resources created, tool-call results, remaining budgets, verification report, and pending approvals. Append events with stable identifiers. A resumed worker should be able to rebuild the next model input without the original process or an in-memory transcript.

Make external writes idempotent. Attach idempotency keys or check current state before retrying. Save a checkpoint before any high-impact operation. Define compensating actions for changes that cannot be rolled back automatically. After a partial failure, reconcile observed reality before issuing the next command.

Tracing ties these boundaries together. Use one run identifier across model calls, tools, sandbox jobs, verifiers, and approvals. OpenTelemetry's trace and span model is a useful common structure, but the important part is semantic consistency: operators should be able to follow one task from intake to accepted artifact.

A resumable run is easier to approve, cheaper to operate, and safer to debug than a heroic uninterrupted session.

08 · Rollout

Build the harness in the order that reduces unknowns.

The first version should not chase full autonomy. It should make one narrow workflow observable and repeatable. Use a capable model to find the architecture's limits, then optimize models and prompts after the controls work.

1. Choose one task family with frequent examples and objective acceptance checks.
2. Create versioned task contracts and a clean, reproducible execution image.
3. Expose read-only tools first; collect traces and compare proposed work with human work.
4. Add file writes inside the sandbox and require independent verification.
5. Create draft pull requests; measure acceptance, correction time, severe errors, cost, and latency.
6. Permit low-risk external actions only after policy and idempotency tests pass.
7. Add human approvals at high-impact boundaries, with durable pause and resume.
8. Review failures weekly and convert repeated classes into code, policy, or evaluation cases.