How the agent loop works

"Agent" is the most worn-out word of the year. But underneath the noise there is a concrete, fairly simple mechanism, and understanding it is the difference between using an agent with judgment and praying it works. The definition that has held up best is Anthropic's, which Simon Willison boils down to a sentence: an agent is a model using tools in a loop (simonwillison.net). No loop, no agent — just a one-off call to a model.

An agent is not a workflow

Before the loop, a distinction that saves a lot of grief. Anthropic separates two things the jargon blurs: a workflow is a system where the model and the tools are orchestrated by code paths you write in advance; an agent is a system where the model itself decides its trajectory and which tool to use, based on feedback from the environment, in a loop (Building Effective AI Agents). The workflow is predictable and cheap; the agent is flexible and expensive. Most problems are solved by the former — the agent is reserved for when you can't wire the path up front but you can verify the result.

The building block: the "augmented LLM"

The basic piece isn't the bare model but what Anthropic calls the augmented LLM: a model with access to retrieval (fetch data), tools (take actions) and memory. Lilian Weng describes it with the model as the "brain" and three pillars around it — planning (break into subgoals, self-correct), memory (short and long term) and tool use (LLM Powered Autonomous Agents). The loop is what sets all of that in motion.

Think, act, observe

The pattern that organizes the loop comes from a 2022 paper: ReAct (Reason + Act). The idea is to interleave reasoning and action instead of separating them: the model reasons a step ("I need the current price, I don't know it from memory"), takes an action (calls a search tool), observes the result, and that result feeds the next reasoning step (Yao et al., 2022). The reasoning plans and handles exceptions; the action brings in information from the outside world the model didn't have.

The round trip, concretely

In practice, that "act" has a very precise shape in today's APIs. When the model wants to use a tool, it doesn't run it: it stops its turn with a signal — in Claude's API, stop_reason: "tool_use" — and returns a block describing which tool it wants and with what arguments. Your code runs that operation and returns the result in a tool_result block, which re-enters the context on the next iteration (Tool use with Claude). OpenAI documents the same dance in five steps: request with tools → the model asks for a call → your app runs it → second request with the output → final answer or more calls (Function calling).

Strip away each provider's detail and the whole loop fits in a few lines:

# The agent loop, in essence
messages = [user_prompt]

while True:
    response = model.generate(messages, tools)
    messages.append(response)

    if response.stop_reason == "end":
        break                        # the model decided it's done

    # the model asked for one or more tools: YOUR code runs them
    results = [execute(call) for call in response.tool_calls]
    messages.append(results)         # observations go back into the context

That's it. There is no more magic at the core: a while that alternates model generation and tool execution, accumulating the entire conversation in messages until the model signals it's done. What changes between a toy and a serious agent isn't the loop — it's what surrounds those two lines.

Why the context grows on every turn

Notice that messages only does one thing: grow. Each turn adds the model's response and each tool's result. On long tasks this piles up fast, and that's where the first real problem appears. Anthropic puts it bluntly: "an agent running in a loop generates more and more data… that must be cyclically refined" (Effective context engineering). The context window is a finite attention budget, and past a certain point you hit context rot: the more tokens you stuff in, the worse the model retrieves what mattered. That's why serious agents compact, summarize or take notes — not for elegance, but because the loop, left unmanaged, drowns in its own history.

Where it breaks

The loop is simple, and for that very reason it has predictable failures. Worth knowing before you turn an agent loose in production:

• Cost grows on every iteration. Each turn resends the whole accumulated history and adds more. An agent that runs twenty turns doesn't cost twenty small calls: it costs twenty ever-larger ones.
• Errors propagate. A wrong bit of reasoning on turn 3 enters the context and poisons turns 4, 5, 6… The model drags its own mistake along as if it were fact. This is why Anthropic recommends the simplest pattern that passes the eval, not the most agentic one (Building Effective AI Agents).
• Stopping isn't free. The while True in the example depends on the model emitting "end". An agent can keep spinning, repeat actions, or fail to recognize it's done. In earnest you need an iteration limit, loop detection and an explicit stopping condition — the goal is to reach a target, not to iterate forever.

The takeaway

An agent is a while around a model that calls tools and keeps accumulating what it observes. Understanding that mechanism doesn't make it less useful — it makes it usable with judgment: you know cost scales with the turns, that the context degrades if you don't manage it, that an early error poisons the rest, and that you control the exact point where a tool runs or doesn't. None of that is visible from the outside, where there's only "an agent doing things". It's visible from inside the loop. And looking inside, before trusting, is exactly the middle ground this site is after: neither rejecting agents by reflex, nor running them blind.