BonrockWhat Actually Makes Something an AI Agent
Everyone is building AI agents. Most of what they're calling agents is just a chatbot with extra steps. Here's the real distinction, and why it matters for what you build.
Every product with a chat interface is now calling itself an AI agent. Most of them aren't.
This matters for a reason beyond semantics: if you build a chatbot and call it an agent, you'll design it wrong, scope it wrong, and be surprised by the failures that follow. The word "agent" implies a specific set of properties, and building systems that have those properties requires different decisions than building systems that don't.
Chatbots Respond; Agents Pursue
A chatbot responds to input. An agent pursues a goal across multiple steps without someone confirming each one.
Ask Claude how to sort a list in Python. It gives you an answer and stops. Give a system access to your codebase, tell it to fix all the failing tests, and it spends twenty minutes reading files, writing code, running tests, debugging failures, and committing the result without asking you anything. The second system has a goal and works toward it across multiple steps.
The gap between them is autonomy and persistence. The agent is still running after you stop watching.
What Agents Are Made Of
Four things are required for an agent to actually work:
Tools. Without the ability to read files, call APIs, run code, search the web, or send messages, an LLM can only produce text. Producing text doesn't change anything in the world. Tools are how a system goes from knowing things to doing things.
Memory across steps. Agents don't complete tasks in a single turn. They need to know what they've already done, what they found, and what comes next. This can be conversation history, a scratchpad, or a database. The architecture of memory matters a lot; a naive approach of keeping the full history in context collapses at step 40.
A loop. The agent runs until done or stuck. Each cycle: look at the current state, decide what to do, act, observe the result, repeat. Without a loop, you have a system that takes one action per turn, which is assisted input rather than autonomous execution.
A goal, not a prompt. Someone has to tell the agent what success looks like. "Fix the failing tests" is a goal. "Here's a broken test" is input. The difference is that a goal persists across steps; input just informs one response.
Why Agents Are Harder Than They Look
The first thing that surprises people: agents make real mistakes with real consequences.
A chatbot's worst outcome is a bad answer. An agent's worst outcome is an action you can't take back: an email sent to the wrong person, code deployed to production, a file deleted, an API call charged. The blast radius of an agent failure is larger than a chatbot's. You need to design privilege boundaries deliberately, because assuming the agent will stay in its lane is the wrong assumption.
The second thing that surprises people: errors compound. An agent with a wrong assumption in step 2 will spend steps 3 through 15 confidently building on that assumption. Humans course-correct; agents need explicit mechanisms to do the same. The most useful design pattern here is having the agent summarize its understanding of the goal before acting on it. It's cheap to implement and catches a surprising percentage of problems early.
The third problem is context. Agents accumulate state as they work: history, tool results, notes to themselves. In long tasks, this outgrows what models handle well. Good agent systems actively manage this by compressing history, caching results separately, and summarizing what's happened. Systems that don't think about this hit a wall at exactly the wrong moment.
It's a Spectrum
What actually exists is a range from "LLM that answers questions" to "system that operates independently for days without human input," with most practical systems sitting somewhere in the middle.
The pattern that works well in production: the agent handles a bounded task autonomously, surfaces decisions it can't make confidently, and hands off to a human when it hits the edges of its authority. This is sometimes called "human-in-the-loop," though that framing undersells it; the real design work is identifying which escalation points matter and making sure the agent surfaces them reliably.
A customer support agent that resolves 80% of tickets automatically and routes the remaining 20% to humans is well-designed, because the goal is appropriate autonomy for the specific task, which is something different from maximum autonomy.
What Agents Are Actually Good For
The tasks where agents deliver disproportionate value have two properties: they require multiple steps, and each step's input depends on the previous step's output.
Processing an invoice sounds simple. It involves reading the document, extracting fields, checking them against a purchase order, applying rules about discounts and terms, routing to the right approver based on the amount, sending a notification, and updating the ERP. An agent handles all seven steps in one invocation. A chatbot needs a human at each one.
The tasks that don't suit agents: single-turn lookups, anything where the output needs human judgment at every step, anything where the stakes of an autonomous wrong decision are too high given the system's reliability. Trying to use agents for these introduces unnecessary risk.
Knowing which is which before you build is the useful judgment. The word "agent" is too cheap to be a reliable guide. The question of whether the task requires autonomous multi-step action: that's the one worth asking.
Found this helpful? Pass it on.