What Is an Agent?

AI agents go beyond answering questions - they take actions, use tools, and run in loops until a goal is met. Learn how tools, memory, and the agent loop work, and where agents break down in practice.

TL;DR: An AI agent is an LLM wired into a loop - it reasons, picks a tool, runs it, reads the result, and repeats until the job is done. The loop is the key difference between a model that answers questions and one that actually gets things done. Agents break down in predictable ways, and understanding those failure modes is just as important as understanding how they work when everything goes right.

From Answering to Acting

A plain language model does one thing: it takes text in and produces text out. You ask it something, it responds. That's it. The interaction ends when the response ends.

An agent is different. Instead of stopping at a response, it can use that response to trigger an action - running a web search, writing a file, calling an API, executing code. Then it reads what came back from that action and decides what to do next. It keeps going, step by step, until it reaches its goal or runs out of room to work.

Think of the difference between asking a colleague "what's the weather in Tokyo?" and telling them "book me a flight to Tokyo for next Tuesday, cheapest option under $900." The first is a lookup. The second is a task. Agents are built for the second kind.

The Agent Loop

Almost every agent architecture bottoms out on the same basic cycle. You'll see it called the ReAct loop, the observe-think-act cycle, or just "the agent loop." The names vary; the shape is the same:

Observe - The agent reads its current context: the original goal, any prior tool results, memory it has access to.
Think - It reasons about what to do next. Which tool? What parameters? Or is the task done?
Act - It calls a tool or produces a final answer.
Update - The result of the action is added to context, and the loop starts again.

The loop runs until the agent decides the task is complete - or until it hits a hard stop like a token limit, a timeout, or an error it can't recover from.

This is where agents differ fundamentally from chatbots. A chatbot's "loop" is just you hitting enter again. An agent's loop runs autonomously, potentially across dozens or hundreds of steps, without a human in the middle.

Tools: How Agents Touch the World

By itself, an LLM is stateless and sandboxed. It can reason about the world but it can't do anything in it. Tools are what change that.

A tool is a function the model can call by name, with parameters, and get a structured result back. Common examples:

Web search - pass a query string, get back a list of results
Code execution - pass Python or JavaScript, get back the output
File read/write - read or write content at a path
API calls - POST to a database, email service, payment processor
Browser control - navigate a real browser, click, type, take screenshots

The model doesn't run these tools itself. It emits a structured request - something like {"tool": "web_search", "query": "SpaceX launch schedule June 2026"} - and the surrounding system (called a harness or runtime) executes it and hands the result back.

Anthropic describes this as "a contract between deterministic systems and non-deterministic agents." The tool always behaves the same way; the agent may call it differently depending on context, skip it, chain it with other tools, or occasionally get its parameters wrong. Well-designed tools return only high-signal information and give actionable error messages that tell the agent what to try next - not cryptic codes that waste context. Anthropic's own guidance on tool design is blunt on scope: build "a few thoughtful tools targeting specific high-impact workflows" rather than wrapping every endpoint.

MCP: The Standardized Tool Layer

Until recently, every team wiring tools to an agent had to build their own integration. The Model Context Protocol (MCP) changed that. Described by its creators as "a USB-C port for AI applications," MCP is an open standard that defines how AI agents connect to tools, data sources, and external services in a consistent way.

An MCP server exposes tools and data. An MCP client (your agent runtime) connects to those servers. Because the protocol is standard, a tool built for Claude works with any other MCP-compatible agent - you build once and integrate everywhere. In 2026, MCP support spans Claude, ChatGPT, Visual Studio Code, Cursor, and dozens of other platforms.

Memory: The Agent's Running State

LLMs have no built-in memory between sessions. Each time the context window fills or a session ends, the slate wipes clean. Agents solve this by externalizing memory - writing key information somewhere it can be read back in later.

There are four practical memory types, borrowed from cognitive science:

In-context memory - The live context window. Fast and immediately available, but limited in size and gone when the session ends. This is the agent's working memory.
Episodic memory - A record of what the agent did in previous runs: which steps it took, which tools it called, what failed and why. Prevents re-attempting dead ends. Anthropic's research on long-running agents emphasizes exactly this: "Failed approaches are important - without them, successive sessions will re-attempt the same dead ends."
Semantic memory - Factual knowledge stored externally, often in a vector database, retrieved by similarity search when needed. Lets an agent answer questions about your private documents without keeping all of them in context.
Procedural memory - Instructions and rules baked into prompts or versioned instruction files (like a CLAUDE.md). Tells the agent how to behave, what to avoid, and what conventions to follow.

In practice, memory is the hardest part to get right. An agent that loses track of what it already did will repeat work, contradict itself, or declare a task done when it's only half-finished.

Multi-Agent Systems: When One Agent Isn't Enough

Some tasks are too big or too parallel for a single agent. A research task that needs to explore ten independent threads simultaneously would take forever if done sequentially. The answer is a multi-agent system: one orchestrator agent breaks the task into sub-tasks and spawns worker agents to handle each one in parallel.

Anthropic's multi-agent research system uses exactly this pattern. A lead agent analyzes the query, plans an approach, and spawns specialized subagents that search different angles simultaneously. Anthropic reported that introducing parallelization cut research time by up to 90% for complex queries. The cost is tokens: multi-agent systems use about 15x more tokens than standard chat interactions, so they're only worth deploying when the task's value justifies the compute.

Multi-agent architectures also inherit and amplify single-agent failure modes. An error in one agent can propagate through the whole pipeline. Coordination overhead grows. Debugging becomes harder when six agents were all running at once.

Where Agents Break Down

This is the part that most introductions skip, and it's the most important part if you want to build something that actually works.

Compounding errors

The most common production failure. An agent calls a tool with slightly wrong parameters, gets back a plausible-looking result, and continues as if everything worked. Every subsequent step now builds on a flawed foundation. By step 10, the output is confidently wrong in ways that are hard to trace back to the original mistake.

Context exhaustion

Context windows are finite. A long-running agent can exhaust its context mid-task, losing track of what it was doing. Without good episodic memory, the next session starts blind and may re-do completed work or declare the task done prematurely. Anthropic's engineering team compared this to "engineers working in shifts, where each new engineer arrives with no memory of what happened on the previous shift."

Premature completion

Agents sometimes decide a task is done before it is. They'll attempt to implement too much at once, exhaust context before finishing, and call it complete anyway. Anthropic's "Ralph loop" pattern - where a harness kicks the agent back into context when it claims completion and asks whether the work is actually done - exists specifically to counter this.

Tool misuse

Calling the wrong tool, passing wrong parameter types, or misreading a tool's output. These failures are especially damaging because they can trigger retry storms - an agent that doesn't understand a tool error may call the same tool repeatedly with the same bad parameters. Clear, actionable error messages in tool responses are one of the most effective mitigations.

Hallucination under uncertainty

An agent that can't find information through its tools may fabricate it rather than admit failure. This is a model-level limitation that better tools and better prompts reduce but don't eliminate.

What Makes an Agent Actually Useful

An agent worth running in production needs more than a loop and some tools. The practical checklist:

Clear success criteria - Vague goals produce vague results. "Research climate policy" is a conversation starter. "Find 5 peer-reviewed papers published after 2023 comparing carbon taxes vs. cap-and-trade, summarize the key findings, and flag any papers with conflicting results" is an agent task.
Scoped tools - Don't give the agent access to every API you have. Fewer, well-designed tools with clear descriptions outperform a sprawling toolkit where many tools overlap.
Durable memory - Write progress to disk. Use git commits. Keep a progress log. Whatever form it takes, the agent needs a way to pick up where it left off without re-discovering everything.
Human checkpoints - The best-designed agent systems include human review at key decision points, especially before irreversible actions. An agent booking flights, sending emails, or modifying a database should ask before it acts, not after.
Verification before "done" - An agent should run its own test, check its own output, or at minimum re-read what it produced before declaring a task complete. "Should work" is not done.

# A minimal agent tool definition (Python, Claude API)
tools = [
    {
        "name": "web_search",
        "description": "Search the web for current information. Use for facts, news, or anything that may have changed recently.",
        "input_schema": {
            "type": "object",
            "properties": {
                "query": {
                    "type": "string",
                    "description": "The search query to use."
                }
            },
            "required": ["query"]
        }
    }
]

Key Takeaways

An agent is an LLM in a loop: observe, think, act, update - repeat until done or stopped.
Tools are the bridge between the model's reasoning and the real world. Design them to return high-signal results and actionable errors.
MCP standardizes how agents connect to tools so you build once and integrate everywhere.
Memory comes in four kinds: in-context (fast, ephemeral), episodic (run history), semantic (external knowledge), procedural (rules and instructions).
Multi-agent systems parallelize work but multiply token costs - use them when the task's complexity justifies it.
Compounding errors, context exhaustion, and premature completion are the three failure modes that kill agents in production. All three have known mitigations.

Try this next: Now that you understand how agents work, see What Is Tool Use? for a deeper look at how to design tools that agents can use reliably - including how to write descriptions that reduce misuse and structure errors so the agent knows what to try next.

Guide fundamentalscore8 min read