The Model Is Not the Agent: Where Does AI Intelligence Actually Live?

Key Takeaways

• The same model, unchanged, can score dramatically differently on identical tasks depending only on the harness: the tools, memory, context management, and verification wrapped around it.
• A well-designed harness can let a smaller model outperform a larger one running with a worse harness, which means “how intelligent is this model” is not a question you can answer by looking at the model alone.
• This is the same move philosophers made about human cognition decades ago: a notebook, reliably used, can be functionally part of a mind rather than a tool a mind merely holds.
• The practical upshot is architectural: judging or building an AI system means judging the whole loop of model, context, tools, permissions, verification and feedback, not just the model at its center.
• QuECTO, a minimal agent harness I’ve been building, exists partly because this question felt worth being able to poke at directly rather than just write about.

Take one model. Freeze its weights. Put it inside two different agents, with different tools, different memory, different context management and different verification steps, and give both the same task.

One finishes it cleanly. The other flails, forgets what it already tried, calls the wrong tool, and quietly convinces itself of a wrong answer. Same model. Same weights. Same everything that supposedly makes it “intelligent.”

So here is the question this essay actually wants to sit with: if the exact same model behaves brilliantly in one agent and terribly in another, was the intelligence ever inside the model alone?

What Happens When the Same Model Behaves Differently in Two Agents?

Yes, measurably so: research that holds a model’s weights fixed and varies only the harness around it, meaning tools, memory and verification, has recorded double-digit swings in task performance. This isn’t a hypothetical. It’s a measured effect, and the numbers are larger than intuition suggests they should be.

A 2026 study on automatically evolving coding-agent harnesses took a fixed model and varied only the scaffolding around it: how tools were exposed, how middleware handled intermediate results, how long-term memory was structured. On Terminal-Bench 2, the baseline harness scored 69.7%. The evolved harness, same underlying model, reached 77.0%, beating even Codex-CLI’s hand-crafted harness at 71.9%. Nobody touched the weights. The entire gain came from the system wrapped around them, and the paper’s ablations found the gains concentrated in tools, middleware, and memory, not in prose-level system-prompt tweaks, which barely transferred at all.

The inverse result is even more pointed. In Google DeepMind’s AutoHarness work, researchers noted that in a Kaggle chess competition, 78% of Gemini-2.5-Flash’s losses came not from bad chess but from attempting illegal moves: a harness failure, not a reasoning failure. When they let the model synthesize its own code harness to eliminate that failure mode, the smaller Gemini-2.5-Flash, harnessed well, outperformed the larger Gemini-2.5-Pro, harnessed poorly.

Read that again: a smaller model beat a larger model, with nothing else changed but the system around it. If intelligence were simply a quantity stored in a model’s parameters, that result shouldn’t be possible. It happened anyway.

What Is an Agent, If Not Just a Model?

We talk about Claude, GPT, Gemini, and the rest as if the model is the intelligent system, as if “how smart is Claude” were a question you could answer by pointing at one artifact. But nothing that actually acts in the world is just a model. An agent, in practice, is closer to:

model + context + memory + tools + environment + permissions + verification + feedback

Anthropic’s own engineering writeup on this makes the same point from the builder’s side: context is “a critical but finite resource,” and getting agents to perform well is substantially a matter of curating what tokens land in front of the model at each step: compaction, structured memory, just-in-time retrieval, delegating to subagents to keep the working set small and high-signal. None of that is the model. All of it changes what the model effectively is, in the moment it has to act.

Push on this and the framing “the model is the intelligence, the rest is plumbing” starts to look backwards. The plumbing is not incidental to the intelligence. In a real sense, for the system that actually does the task, the plumbing is part of the intelligence.

Is Intelligence a Property of the Thinker, or of the System Around It?

Increasingly, both: philosophy already made this case about human cognition decades before language models existed. This is not a new question. It’s an old one, dressed in new hardware.

In 1998, Andy Clark and David Chalmers published a short, strange, and enormously influential paper called “The Extended Mind.” Their thought experiment: Otto has Alzheimer’s and carries a notebook everywhere, writing down everything he needs to remember and consulting it constantly, reliably, automatically. Inga has an intact biological memory and just remembers things. When Otto wants to visit the museum, he checks his notebook for the address; when Inga wants to, she recalls it. Clark and Chalmers’ claim is that Otto’s notebook is not a tool his mind uses from the outside. It is, functionally, part of his mind. Reliably available, consistently consulted, tightly coupled to behavior: on those criteria, the boundary of “where the mind is” runs past the skull.

The paper is philosophy, not engineering, and it drew plenty of pushback in the decades since. But notice how precisely the criteria transfer. A model’s tool calls, its retrieved memory, its verification loop: reliably available, consistently used, tightly coupled to what the agent actually does next. If Otto’s notebook counts as part of his cognition, it becomes hard to insist an agent’s memory store or tool layer is merely “external” to its intelligence. The Terminal-Bench and AutoHarness results above are, in effect, an empirical echo of a philosophical argument made a generation before either existed.

What Does the Human Analogy Actually Teach Us?

Push the analogy one step further and it stops being abstract.

Take a brilliant person and strip away every book they’ve read, every instrument they can use, every collaborator they can ask, every institution that trained them, every note they’ve ever written down, every piece of feedback anyone has ever given them. What’s left is still a capable brain, but it is not equivalent to that same person embedded in a lab, a library, a research group, a tradition. The genius is real in both cases. What changes is what the genius can do, and arguably what it even is, once you’re asking about behavior rather than raw potential.

Civilization is a harness. Language, peer review, citation, mentorship, tools, institutional memory: these aren’t decorations around individual human intelligence, they’re load-bearing parts of it. This is the same fault line I keep returning to in these essays: the difference between reacting and reflecting isn’t purely a fact about one brain’s wiring, it’s shaped by whether that brain has ever been handed the architecture of doubt, a discipline, a method, a set of external checks, or left to improvise one from scratch. It also raises a question the harness framing cannot settle on its own: if the whole loop is doing the work, is what the loop does thinking, or only inference?

Maybe intelligence was never purely intrinsic to the thinker. Maybe it’s always been partly a property of the system surrounding the thinker, and AI is just forcing us to notice that, because we can finally hold the “thinker” part constant and vary everything else.

What Does Systemic Intelligence Mean for Building and Judging Agents?

Once you take this seriously, it stops being a philosophical curiosity and becomes a design constraint.

It means the question “which model should we use?” is frequently the wrong first question, or at least an incomplete one, asked before “what harness will it run in?” A frontier model wired into a thin, badly-instrumented agent can lose to a mid-tier model wired into a well-built one, exactly as the AutoHarness result showed. Model selection and harness design are not sequential decisions where you pick the model first and bolt scaffolding on after; they trade off against each other.

It also means evaluation has to change shape, which is the same conclusion I reached from a different direction when writing about why evals break down for agentic AI: you cannot score “the model’s intelligence” by testing the model in isolation, because in deployment it never runs in isolation. You have to evaluate the trajectory of plan, tool calls, recovery and verification, because that trajectory is where the intelligence you actually care about is expressed. A benchmark score attached to a bare model is a claim about a system that will never ship.

And it means harness engineering, the unglamorous work of context management, tool design, memory architecture, permissioning and verification gates, is not infrastructure sitting below the interesting part. It’s increasingly one of the more interesting parts.

Where Does This Leave Us: Model, Harness, and the Space Between?

I don’t think the answer is “the model doesn’t matter.” A better model helps in every harness, and always will. The answer is stranger and more useful than that: model quality is necessary but not sufficient, and past some threshold, the system around the model can matter as much as the model itself. “How intelligent is this model” is, on its own, an increasingly underspecified question, like asking how fast a person is without saying whether they’re walking, driving, or flying. That is the premise the rest of these essays on AI agents build on: that the harness, the tools, and the evaluation decide what a system can actually do.

This question is partly what led me to build QuECTO, a minimal, statically-linked Rust harness: a ~1.2 MB core and a 3.3 MB coding agent, with no async runtime and almost no dependency tree between the model call and the tool call. Not because a smaller harness is automatically a better one, but because if model intelligence and agent behavior really are different things, you need a harness simple enough to see through, to change one variable at a time and actually watch what moves.

The same pattern shows up outside benchmarks, too, at the scale of a single household object. Asked to design a 3D-printable wedge ramp with no tools and no context beyond the vacuum’s own measurements, a fresh model produced a working part and caught a clearance problem nobody had mentioned, once its harness put the vacuum’s actual numbers in front of it and let it check its own geometry. The model’s raw grasp of the physical world was incomplete; the harness that fed it real constraints and verified the output closed most of the gap anyway.

There is also a level above this one, and it repeats the same lesson. If model quality doesn’t determine agent behaviour, agent quality doesn’t determine what a population of agents does either: Anthropic’s Frontier Red Team found groups of agents scoring 17–36% on a task where the same models working alone approached 100%. The property keeps failing to survive the step up.

If a model’s intelligence isn’t fully separable from the system it runs in, then neither, probably, is ours. The uncomfortable and clarifying part of this question is that it doesn’t stay contained to AI for very long. It just makes you ask, again, where you’d draw the line around your own mind, and whether the line was ever where you thought it was.

Further Reading

• Clark, A., & Chalmers, D. (1998). “The Extended Mind.” Analysis, 58(1), 7–19.
• Anthropic (2025). “Effective Context Engineering for AI Agents.” Anthropic Engineering Blog.
• Lou, X. et al. (2026). “AutoHarness: Improving LLM Agents by Automatically Synthesizing a Code Harness.” arXiv:2603.03329. ICLR 2026 Workshop on Recursive Self-Improvement.
• “Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses.” (2026). arXiv:2604.25850.

More essays at Call to Think · About this project