Where does an Agent actually run?
Harnesses, sandboxes, and the infrastructure around the loop
Last week we reduced an agent to a loop.
The model reasons, takes an action, observes the result, and decides what to do next.That drawing was useful. It was also missing almost everything needed to run an agent in production.
When the model says “run the tests,” where does that command execute? Which files can it read? Can it access the network? Whose credentials does it use? What happens if the command runs forever, modifies the wrong file, or downloads something malicious?
The model chose the action. Something else has to make the action possible without giving the model control of the entire system. That surrounding layer is the agent harness.
The loop needs a harness
A harness runs the loop around the model.
It prepares the context, calls the model, reads the requested action, checks whether that action is allowed, invokes the right tool, captures the result, and sends it back for the next iteration.
It also carries the parts the clean ReAct diagram leaves out: state, identity, limits, approvals, error handling, and traces.
The term agent harness has become more common this year because people are discovering that the model is only one part of agent performance. The same model can behave very differently depending on how the harness selects context, describes tools, handles failures, and decides when the task is complete.
Consider the coding example from last week. The model decides to run a test. The harness checks whether shell access is allowed, starts an isolated environment, runs the command, captures the output, records what happened, and returns the result to the model.
Model requests an action → harness checks it → sandbox runs it → result returns to the loop.
If the model is the part doing the reasoning, the harness is the part turning that reasoning into a running system.
A sandbox is not just another tool
From the model’s perspective, a sandbox may look like a tool called “run code” or “use computer.”
Architecturally, it is a different thing.
A tool defines an action the agent can request. The sandbox defines where that action happens and what it is allowed to touch.
Suppose the agent wants to run a Python script.
The tool describes the input, such as the code and arguments.
The sandbox decides which runtime to start, how much CPU and memory it gets, which files are mounted, whether it can reach the internet, which credentials are available, and when the environment should be destroyed.
That distinction matters because agents increasingly work with files, browsers, shells, package managers, and generated code. These are useful precisely because they are powerful. They are dangerous for the same reason.
Running model-generated code directly on the host is the equivalent of handing a stranger your laptop because they promised to use only one folder.
The sandbox creates a boundary around the work.
Agents do not look like normal API request
Most web applications are designed around short, stateless requests. Any replica can handle the next request, and local state disappears when the request ends.
An agent task may run for twenty minutes. It may install a package, create files, open a browser, pause while waiting for approval, and resume later with the same workspace. That workload wants something different: one isolated environment for one task, with a stable identity and state that survives long enough to finish the job.
It looks less like a normal API request and more like a temporary computer created for the agent.
This runtime shape is becoming common enough that the Kubernetes community now has a dedicated Agent Sandbox project. It is designed for isolated, stateful agent workloads that need a stable identity, persistent files, and stronger isolation for untrusted code.
Isolation has a cost
Starting a fresh sandbox for every task gives you a clean boundary, but it adds startup time. Keeping sandboxes warm improves latency, but idle environments cost money. Preserving the filesystem helps long-running work, but now you need rules for retention, cleanup, and storage. The isolation boundary also has to match the risk.
An agent calling a read-only weather API does not need the same environment as an agent executing arbitrary code or controlling a browser. A container may be enough for one workload. A separate kernel or lightweight virtual machine may be appropriate for another.
There is no single “agent sandbox” shape.
The useful questions are more practical.
What could this agent damage?
What data could it reach?
How long does the environment need to live?
What must survive if the task pauses?
How quickly does the next environment need to start?
The right boundary follows those answers.
A sandbox does not make an agent safe
Isolation limits where an action can go. It does not decide whether the action was sensible. An agent can stay perfectly inside its sandbox and still leak data through an allowed network connection. It can use valid credentials to call the wrong API. It can follow instructions hidden inside a document and take an action the user never intended.
The sandbox limits the blast radius. The harness still has to manage identity, network access, approvals, budgets, and the complete trace of what happened.
Credentials should be short-lived and scoped to the task.
Network access should be limited to the destinations the work needs.
Consequential actions should pause for approval.
Every model call, tool invocation, shell command, and retry should appear in the same trace.
Otherwise, the sandbox gives you an isolated environment without giving you control of the agent running inside it.
The solutions architect lens
I would start with the execution path rather than the model.
Where will the agent’s code run?
What can that environment reach?
Whose identity does it use?
What state must survive between steps?
What happens when the task crashes halfway through?
Can it resume, or does it start again and risk repeating an action?
Then I would look at the operating model.
How quickly must a sandbox start?
How many concurrent tasks can run?
Do environments stay warm or get created on demand?
Who patches the base images?
When are files deleted?
How do we investigate a task after the environment is gone?
These questions are less exciting than choosing a model. They are also the questions that determine whether the agent can move beyond a demo.
The series comes full circle
This series began by asking why AI infrastructure could no longer be treated as a pool of GPUs.
The full picture is clearer now.
Accelerators run the model calls.
General-purpose compute runs the harness and tool orchestration.
Sandboxes contain code, browsers, and files.
Storage carries state.
Networks connect tools and data.
Identity and policy decide what the agent may do.
Observability tells us what it actually did.
The model is still central, but it is not the system. An agent becomes useful when the model can act. It becomes deployable when the infrastructure around it keeps those actions bounded, recoverable, and visible.
What I’d take away
The harness runs the agent, not just the model. It manages the loop, context, tools, state, limits, identity, and traces.
A tool describes an action. A sandbox controls where that action happens. They work together, but they solve different problems.
Agent runtimes are stateful. Long-running tasks need private workspaces, stable identity, persistence, and a way to pause or resume.
Isolation limits the blast radius. Permissions, network controls, approvals, and observability still decide whether the system is safe.
Start with the execution boundary. Choose the sandbox and harness around what the agent can damage, not around what is fashionable.
That closes this series.
It started with the chip and ends with the system around the model. That system is where most of the hard production work lives.
If you want future essays in your inbox, subscribe.