LLM
A neural network trained on text that predicts the next token. Given a prompt, it returns a completion.
- Text in, text out
- Knows what it learned in training
- Stateless: each call starts fresh
- Can only propose a tool call
Guide · Agent harness
AI agent vs LLM: the model that reasons, and the system that lets it act.
An LLM turns text into text. An AI agent puts that model in a loop with tools, memory and rules, so its next words can become a lookup, a calculation or a change in a system.
People ask whether ChatGPT or Claude is an LLM or an agent. The model is an LLM; what you use may wrap it as an agent. This guide separates the two precisely: what a model can do on its own, the parts added around it, how one agent run looks as a trace, why errors compound over steps, and how to decide whether a task needs an agent at all.
Updated · 8 sources
What wraps an LLM to make an agent
01
Goal and instructionswhat done looks like, and the rules to work within
instructions02
Loopcall the model, run what it chose, feed back the result, repeat
run03
Toolssearch, databases, APIs, code — run by the application, never by the model
function calling04
Memory and contextwhat this step needs: history, documents, earlier results
context05
Checks and approvalstests, policy and people between the model and the world
guardrails06
LLMreads everything above and proposes the next words
modelWhat is the difference between an AI agent and an LLM?
An LLM (large language model) is a model that takes text in and predicts text out; it has no tools, no memory between calls and no way to act. An AI agent is a system built around an LLM: it runs the model in a loop, gives it tools to call, keeps context across steps and checks each result until a goal is met. In AI agent vs LLM, the LLM is the reasoning engine and the agent is everything that lets that reasoning do work — often summed up as agent = model + harness.
Side by side
AI agent vs LLM, compared on what each can actually do.
OpenAI describes an agent as three parts — a model, tools, and instructions — run in a loop until an exit condition. Remove the loop and the tools, and what remains is the LLM.
AI agent
Software that uses an LLM to decide what to do next, does it with tools, and repeats until the goal is met.
- A goal in, a completed task out
- Reads live data through tools
- Keeps state across steps
- Runs tools within permissions
One callUnitOne run of many calls
Training data, fixed at a cutoffKnowledgeTraining data plus whatever its tools return
Only what is in this promptMemoryShort-term context plus stored memory
None; it writes a request for oneActionsThe application runs the requested tool
A wrong answerErrorsA wrong step that later steps build on
One passCost and timeMany passes; grows with the task
A model providerWho builds itYou, or the platform you run agents on
Where acting begins
What an LLM does on its own, and where tool calling begins.
Modern LLMs are trained to answer with a structured request for a tool instead of prose. That is the hinge between model and agent: the model writes the request, and the application decides whether to run it.
- Offer tools
The application sends the prompt and a list of tools with their input schemas.
- Model asks
The LLM returns a tool name and arguments — text, not an action.
- App runs it
The application executes the function, if policy allows.
- Return result
The output goes back to the model in a second request.
- Answer or repeat
The model answers, or asks for another tool.
An LLM cannot act by itself. OpenAI’s documentation states the step plainly: execute the code “on the application side” with input from the tool call. Everything an agent can do is decided by what the application around the model will run.
Inside an agent
Anatomy of an AI agent: one LLM and the parts around it.
Anthropic calls the starting point an augmented LLM — a model with retrieval, tools and memory. Lilian Weng’s widely cited 2023 overview names the same core: planning, memory and tool use. Production agents add instructions, checks and a record.
AI agentan LLM in a loop, with everything it needs to act
LLMreads the context, proposes the next step
Planningbreaks a goal into steps; revises after each result
Memorythe current context, plus a store it can recall from
Toolsfunctions and APIs the application will run for it
Instructionsthe job, the rules and when to stop or ask
Checkstests, policies and approvals on each step
Environmentthe systems, files and data it works in
Run recordwhat it saw, chose and did, in order
The loop Anthropic describes for its own agents
- Gather contextSearch, read files, query the systems the task touches.
- Take actionCall the tool or write the change the plan calls for.
- Verify workRun tests, check rules, compare with the goal.
Repeat until the work passes its checks or a person must decide.
One run, traced
One question, answered by an LLM and by an AI agent.
Question: “Which supplier did we spend the most with last quarter, and is any of it overdue?” The answer lives in your systems, not in any model’s training data.
user > Which supplier did we spend the most
with last quarter? Anything overdue?
model > call sql_query(
"spend by supplier, Q3, top 1")
app < Northwind Metals · 1,284,600.00
model > call list_invoices(
supplier="Northwind Metals",
status="unpaid", due_before=today)
app < 2 invoices · 41,200.00 · 12 and 19
days overdue
model > answer: Northwind Metals, 1.28m in Q3.
Two invoices (41,200.00) are overdue;
links to both below.The LLM on its own
- HasNo access to your spend data or invoices.
- Best caseSays it cannot know, and suggests where to look.
- Worst caseProduces a plausible supplier name and figure.
The agent
- HasTwo granted tools: a query and an invoice list.
- DoesTwo tool calls, each result fed back to the model.
- ReturnsAn answer built from live records, with links.
Evidence
Tools and a loop raise an LLM’s success rate, and let one wrong step carry forward.
Giving a model tools and a loop measurably improves results on interactive tasks. It also means one wrong step can carry through every step after it, which is why checks sit inside the loop.
How errors compound. Our arithmetic, not a measurement: if each step of a task is right 95% of the time, independently, and nothing checks the steps, a 20-step run is right end to end only about 36% of the time (0.95 to the power of 20 ≈ 0.36). Checks after each step, and a person on the risky ones, are what keep long runs reliable.
Decision guide
AI agent vs LLM call: choosing the lowest level that works.
Microsoft’s architecture guidance puts it as a rule: use the lowest level of complexity that reliably meets your requirements. A direct model call is cheaper, faster and easier to test than an agent.
What does the task need beyond the prompt?
Nothing — the input has it allA direct LLM call: summarise, classify, translate, extract fields. Add a schema for typed output.
Data or actions, in a known orderA workflow that calls the LLM at fixed steps; your code fetches the data and makes the calls.Read more →
Data or actions, chosen as it goesAn AI agent: one LLM with tools in a loop, with checks and approvals on the risky steps.Read more →
Signs a single call has outgrown itself
- The prompt keeps growingYou paste in data that a lookup could fetch.
- People copy answers into systemsThe next step is an action a tool could take.
- The path varies by caseWhich lookup comes next depends on what the last one found.
On Botminds
How Botminds turns the LLM you choose into a governed agent.
On Botminds the model is a choice and the harness is constant. Pick one model or a pool; the same loop, grants, approvals and records apply to every agent.
What the harness adds to any model
- The loopFast, thinking, auto and autonomous ways to run, on one harness.
- Tools by grantGranted by name and checked before every call.
- ContextAssembled within a budget; preview the exact prompt before a run.
- ApprovalsTools you choose wait for a person to approve, reject or edit.
- Typed outputExtraction and classification agents return fields against a schema.
- A recordEvery step, tool call, approval, answer and cost.
Models an agent can use
- Azure OpenAI and Azure AI Foundrydeployments in your subscription
- Google Geminialongside other providers
- Through the gatewayOpenAI, AWS Bedrock or models you host
- A poolon document and background runs, a failing call moves to the next model
See the agent harness and, in the Academy, LLMs and services.
Questions people ask
AI agent vs LLM: common questions.
The technology behind itAgent harnessThe runtime around every agent→What is the difference between an AI agent and an LLM?
In AI agent vs LLM, the LLM is a model that turns text into text and cannot act. An AI agent is a system that runs an LLM in a loop with tools, memory and rules, so the model’s choices become lookups and actions, checked step by step until a goal is met.
Is an LLM an AI agent?
No. An LLM on its own only produces text. It becomes part of an agent when software runs it in a loop, gives it tools to call, keeps state between steps and decides which of its requested actions to carry out.
Is ChatGPT an LLM or an AI agent?
ChatGPT is an application built on LLMs. When it only replies, it is a chat interface to a model; when it browses, runs code or uses connected tools to complete a task, it is acting as an agent.
Can an LLM use tools by itself?
No. Through function calling, the model returns the name and arguments of a tool it wants. The application runs the tool and sends the result back. What the application allows decides what the agent can do.
What makes an LLM agentic?
Control over the next step. When the model’s output decides which tool runs next and the loop continues until a goal is met, the system is agentic. When code fixes every step and the model only fills in text, it is a workflow.
Do AI agents need a large language model?
Today’s general-purpose AI agents use an LLM to read requests and choose actions. Older software agents used rules or planners. The LLM is what lets an agent handle open-ended language and messy documents.
What is the difference between an AI agent, an LLM and MCP?
The LLM reasons, the agent runs that reasoning in a loop with tools, and MCP — the Model Context Protocol — is a standard way for the agent to discover and call those tools. See MCP vs API.
Why are AI agents more expensive than LLM calls?
An agent makes many model calls per task, each carrying more context. Anthropic reports agents use about four times the tokens of chat interactions, so use an agent only where its choices add value.
Sources
- OpenAI: Function calling guidedevelopers.openai.com
- OpenAI: A practical guide to building agentscdn.openai.com
- Anthropic: Building effective agents (December 2024)anthropic.com
- Anthropic: Building agents with the Claude Agent SDK (September 2025)claude.com
- Anthropic: How we built our multi-agent research system (June 2025)anthropic.com
- Lilian Weng: LLM Powered Autonomous Agents (June 2023)lilianweng.github.io
- Yao et al.: ReAct, reasoning and acting in language models (2022)arxiv.org
- Microsoft Azure Architecture Center: AI agent orchestration patternslearn.microsoft.com
Read next
Agentic AI vs generative AIHow AI that writes becomes AI that acts, and what that adds→Agent harness vs agent frameworkWhat a framework gives you, and what a harness runs for you→AI agent vs chatbotWhat changes when the thing you chat with can act in your systems→All guidesEvery explainer, by topic→
See it on
your own work.
Bring one process and its documents. We’ll show you the agents, the controls and the record, running on your data.