Agent Is Not the Model

TL;DR Highlight

Splitting an agent into model, inference service, and harness shows which layer to fix.

Who Should Read

An agent developer who can't tell whether odd LLM behavior comes from the model or from their own code.

Core Mechanics

An agent system has three layers
- A model sits under an inference service, which sits under a harness (the wrapper that shapes inputs and wires up tools).
- All three working together make the agent system.
MCP and Skills belong to the harness
- The model doesn't inherently know about an MCP server or a Skill.
- The harness decides what context and tools to expose.
Same model, different harness, different behavior
- A model like Sonnet behaves differently across agent systems.
- The harness shapes the inputs and interprets the outputs.
Name the layer, then fix it
- Slow or expensive points to the inference service or model choice.
- Bad tool use likely means the harness formats tools wrong or mis-parses responses.
The post includes a symptom-to-layer table

Community Reactions

Definitions of agent vary
- One commenter calls an agent an LLM chat loop with tool calling.
- By that view, no harness is needed beyond maybe a shell script running the loop.
Product names in the table are ambiguous
- Commenters note Claude Desktop hosts three harnesses, and Cursor and ChatGPT also contain several.
- One suggests 'default inference provider' is more accurate than 'inference service'.
Some use even more layers
- One commenter splits it into application, orchestrator, agent instances, service API, and model.
- Another says Claude Code sub-agents are often forks that differ only in context contents.
Pushback: nobody confuses them
- Several commenters say they've never heard anyone conflate agent and model.
- One says the post feels AI-generated.

How to Apply

Label three recent bugs by layer
- Pick three recent issues and classify each as model, harness, or inference service using the post's symptom table.
- Add a layer label in your tracker.
Check harness logs before swapping models
- When tool calls break, inspect tool definitions and response parsing first.
Compare the same model across harness variants
- Change only the prompt template or tool list and run the same input.
- If results differ, record it as a harness issue.
Add the four terms to your team glossary
- Write one line each for model, inference service, harness, and agent system.

Try It

copy & run

# Paste into Claude or ChatGPT (the post's symptom-to-layer table turned into a prompt)

You are an LLM agent debugging assistant. Classify symptoms using these layers.
- Model: function turning input tokens into output tokens
- Inference service: hosts the model and tracks usage
- Harness: shapes inputs, interprets outputs, connects tools

Symptom: {{symptom description}}
Tools in use: {{e.g. Claude CLI, custom LangChain agent}}

1) Most suspect layer and why 2) Three things to check in that layer 3) Other layers it could be.

Next: run this on three recent issues and compare the classifications.

Terminology

HarnessThe wrapper code in front of a model API that assembles inputs, connects tools, and interprets outputs. Like the building crew that actually touches the ground.

Inference serviceThe hosted layer that runs the model and bills usage, such as AWS Bedrock or Anthropic's API.

MCPA standard for connecting external tools to a model. The post treats it as a harness-layer feature.

SkillsBundles of capabilities the harness exposes to the model. The model doesn't inherently know them.