Skip to content

04 · The Agent Loop: Where Agency Actually Comes From

In Lesson 3, we built this:

USER
↓
MODEL
↓
tool request
↓
PYTHON
↓
tool executes
↓
result goes back to MODEL
↓
final answer

That introductory example showed one tool-request/result cycle.

It was not yet a general loop that could keep handling additional requests.

Real tasks aren’t that predictable.

A model might need:

0 tools

or:

1 tool

or:

tool
↓
result
↓
another tool
↓
result
↓
another tool
↓
answer

We cannot write:

call_model()
run_tool()
call_model()

and assume that’s enough.

We need Python to keep going until the model stops asking for tools.

That is the agent loop.

And this is the lesson where most of the mystery disappears.

An agent is largely a loop around an LLM with access to tools.

No framework required.


Before Python, write the architecture.

RECEIVE user task
REPEAT:
ASK model what to do next
IF model requests tools:
execute requested tools
give results back to model
continue
OTHERWISE:
return model answer

That’s already most of an agent.

Add one safety constraint:

RECEIVE user task
FOR maximum 10 steps:
ASK model what to do next
IF model requests tools:
execute tools
give results back
continue
RETURN final answer
RETURN error because step limit was reached

That is the version we’re going to build.


Imagine the request:

What is the status of the latest invoice for max@example.com?

Suppose our model has two tools:

find_customer(email)
get_latest_invoice(customer_id)

The model doesn’t initially know the customer ID.

So it cannot immediately call:

get_latest_invoice(customer_id=...)

It must first do:

find_customer("max@example.com")

Tool result:

{
"customer_id": 42
}

Now the model knows enough to request:

get_latest_invoice(42)

Tool result:

{
"invoice_id": 4812,
"status": "unpaid",
"amount": 250
}

Now it can answer:

The latest invoice is unpaid, with $250 outstanding.

Notice the sequence:

MODEL
↓
find_customer
PYTHON
↓
customer_id = 42
MODEL
↓
get_latest_invoice
PYTHON
↓
invoice is unpaid
MODEL
↓
final answer

How many model calls did we need?

Three.

But another question might need only one.

So our Python cannot hardcode the number of turns.


First, no AI.

def find_customer(email: str) -> dict:
customers = {
"max@example.com": {
"customer_id": 42,
"name": "Max",
}
}
return customers.get(
email,
{"error": "customer_not_found"},
)
def get_latest_invoice(customer_id: int) -> dict:
invoices = {
42: {
"invoice_id": 4812,
"status": "unpaid",
"amount": 250,
}
}
return invoices.get(
customer_id,
{"error": "invoice_not_found"},
)

These are boring Python functions.

Good.

That’s exactly what tools should usually be.


We learned this in Lesson 3.

Never let the model dynamically execute arbitrary Python names.

AVAILABLE_TOOLS = {
"find_customer": find_customer,
"get_latest_invoice": get_latest_invoice,
}

This is our allowlist.

The model may request:

find_customer
get_latest_invoice

It may not request:

delete_database
open_shell
read_env_file

unless we explicitly expose those capabilities.


tools = [
{
"type": "function",
"name": "find_customer",
"description": """
Find a customer by email address.
Use this when you need the customer's internal ID.
""",
"parameters": {
"type": "object",
"properties": {
"email": {
"type": "string",
}
},
"required": ["email"],
"additionalProperties": False,
},
"strict": True,
},
{
"type": "function",
"name": "get_latest_invoice",
"description": """
Get the latest invoice for a customer using
the customer's internal ID.
""",
"parameters": {
"type": "object",
"properties": {
"customer_id": {
"type": "integer",
}
},
"required": ["customer_id"],
"additionalProperties": False,
},
"strict": True,
},
]

Again, notice the separation:

PYTHON FUNCTION

is the real capability.

TOOL DEFINITION

is the model-facing description of that capability.


Let’s write the skeleton before filling in details.

This excerpt belongs inside run_agent(): its return lines exit that function. The complete definition appears below.

MAX_STEPS = 10
for step in range(MAX_STEPS):
response = client.responses.create(...)
# Check response status and refusals before dispatch.
# We'll fill in those checks below.
tool_calls = [
item
for item in response.output
if item.type == "function_call"
]
if not tool_calls:
answer = response.output_text
if not answer.strip():
raise RuntimeError("Model returned no answer.")
return answer
# execute tools
# add results to context
# loop again

Look closely at:

if not tool_calls:
answer = response.output_text
if not answer.strip():
raise RuntimeError("Model returned no answer.")
return answer

After checking that the response completed without a refusal, that is how our agent decides it has a final text answer.

Not because Python magically understands the task.

Not because we created a special Agent object.

Simply because:

The model returned an answer instead of asking for another tool.


Suppose the model does:

find_customer(max@example.com)

and Python gets:

customer_id = 42

On the next model call, the model must know:

The user asked about max@example.com.
I requested find_customer.
The tool returned customer_id 42.

Otherwise, every iteration would effectively start from scratch.

For today’s lesson, we’ll manually maintain a list:

input_items = [
{
"role": "user",
"content": "What is the status of the latest invoice for max@example.com?",
}
]

As the interaction proceeds, we’ll add model outputs and tool results.

Conceptually:

input_items:
USER:
What's the latest invoice status?
MODEL:
function_call find_customer(...)
TOOL:
customer_id = 42
MODEL:
function_call get_latest_invoice(...)
TOOL:
status = unpaid
MODEL:
final answer

This is our agent’s working conversation.

We’ll study context and state properly in Lesson 5.

For now, just understand that the loop must preserve enough history for the next step to make sense.


The examples in this lesson use the Responses API with the documented gpt-5.5-2026-04-23 snapshot. The Python interfaces were checked with Python 3.13.18 and openai==3.55.1 on November 19, 2026. The checks use synthetic responses, not paid model calls; the sample answers below are illustrations. Model reference

No reasoning-effort or output-token override is set here. The model’s defaults apply. An iteration limit is not a token, cost, or elapsed-time guarantee.

Start with:

response = client.responses.create(
model="gpt-5.5-2026-04-23",
instructions="""
You are a customer support assistant.
Use the available tools when current account
information is required.
Do not invent customer or invoice information.
""",
input=input_items,
tools=tools,
)

The response could contain:

function_call

or:

message

or other response items.

We specifically look for function calls:

tool_calls = [
item
for item in response.output
if item.type == "function_call"
]

9. Check the response before treating it as finished

Section titled “9. Check the response before treating it as finished”

A response without function calls is not necessarily a successful answer.

It could be incomplete. It could contain a refusal. It could contain no text at all.

First, check the response status and any refusal items:

if response.status != "completed":
raise RuntimeError("Model response did not complete.")
if any(
part.type == "refusal"
for item in response.output
if item.type == "message"
for part in item.content
):
raise RuntimeError("Model refused the request.")

Messages contain a list of content parts. The outer loop selects message items; the inner loop examines their parts. any(...) is true if one part is a refusal. Reasoning and function-call items are skipped by that particular check, not discarded from the response history.

Then, after collecting tool_calls, check the final text:

if not tool_calls:
answer = response.output_text
if not answer.strip():
raise RuntimeError("Model returned no answer.")
return answer

These are small stop-on-error checks. Lesson 6 develops more useful application error handling. We are not treating an unsuccessful response as evidence that the task is finished.

Imagine the question were:

What does “invoice due date” mean?

The model might not need any tools.

So:

tool_calls = []

and the loop ends immediately.

This is an important property.

Our agent isn’t required to use tools.

Tools are capabilities, not mandatory steps.


10. If tools were requested, preserve the model output

Section titled “10. If tools were requested, preserve the model output”

Before executing them:

input_items += response.output

Why?

Because on the next iteration, we want the model to know what it previously requested.

When manually maintaining Responses API context, carry forward the relevant response output items in their original order, including reasoning items returned alongside function calls. Do not reconstruct the history from output_text alone. This loop retains the original input and every output item before appending tool results. OpenAI function-calling guidance

So our context becomes approximately:

USER:
What is the latest invoice status for max@example.com?
MODEL:
find_customer(email="max@example.com")

Now we’re ready to execute.


Do not assume the model will always request exactly one tool.

for tool_call in tool_calls:

First, find the allowed Python function:

function = AVAILABLE_TOOLS.get(tool_call.name)

Reject unknown tools:

if function is None:
raise ValueError(
f"Unknown tool: {tool_call.name}"
)

Then parse the arguments:

arguments = json.loads(tool_call.arguments)

Parsing JSON does not validate the argument contract. These two tools have small enough contracts to check directly:

if not isinstance(arguments, dict):
raise ValueError("Tool arguments must be an object.")
if tool_call.name == "find_customer":
if set(arguments) != {"email"} or type(arguments["email"]) is not str:
raise ValueError("Invalid customer lookup arguments.")
elif tool_call.name == "get_latest_invoice":
if set(arguments) != {"customer_id"} or type(arguments["customer_id"]) is not int:
raise ValueError("Invalid invoice lookup arguments.")

The field-set check catches missing or extra arguments. The type checks reject a string or boolean pretending to be an integer ID.

Execute:

result = function(**arguments)

Here, an unknown tool, malformed JSON, or invalid arguments stops the run with an exception. A normal lookup miss returns an error dictionary to the model. Lesson 6 adds a wrapper for controlled tool failures; these read-only fixture lookups do not implement real account permissions.

And then create the tool result:

tool_result = {
"type": "function_call_output",
"call_id": tool_call.call_id,
"output": json.dumps(result),
}

The call_id associates this result with that particular function request. Each requested call needs its own corresponding output, even if two calls use the same function. OpenAI function-calling guidance

Then append it:

input_items.append(tool_result)

Now the model can see what happened.


Put everything together.

import json
from openai import OpenAI
client = OpenAI()
def find_customer(email: str) -> dict:
customers = {
"max@example.com": {
"customer_id": 42,
"name": "Max",
}
}
return customers.get(
email,
{"error": "customer_not_found"},
)
def get_latest_invoice(customer_id: int) -> dict:
invoices = {
42: {
"invoice_id": 4812,
"status": "unpaid",
"amount": 250,
}
}
return invoices.get(
customer_id,
{"error": "invoice_not_found"},
)
AVAILABLE_TOOLS = {
"find_customer": find_customer,
"get_latest_invoice": get_latest_invoice,
}
tools = [
{
"type": "function",
"name": "find_customer",
"description": """
Find a customer by email address.
Use this to obtain the customer's internal ID.
""",
"parameters": {
"type": "object",
"properties": {
"email": {
"type": "string",
}
},
"required": ["email"],
"additionalProperties": False,
},
"strict": True,
},
{
"type": "function",
"name": "get_latest_invoice",
"description": """
Get the latest invoice for a customer
using the customer's internal ID.
""",
"parameters": {
"type": "object",
"properties": {
"customer_id": {
"type": "integer",
}
},
"required": ["customer_id"],
"additionalProperties": False,
},
"strict": True,
},
]
def run_agent(user_request: str) -> str:
input_items = [
{
"role": "user",
"content": user_request,
}
]
MAX_STEPS = 10
for step in range(MAX_STEPS):
response = client.responses.create(
model="gpt-5.5-2026-04-23",
instructions="""
You are a customer support assistant.
Use tools when current customer or
invoice information is required.
Never invent account information.
""",
input=input_items,
tools=tools,
)
if response.status != "completed":
raise RuntimeError("Model response did not complete.")
if any(
part.type == "refusal"
for item in response.output
if item.type == "message"
for part in item.content
):
raise RuntimeError("Model refused the request.")
tool_calls = [
item
for item in response.output
if item.type == "function_call"
]
if not tool_calls:
answer = response.output_text
if not answer.strip():
raise RuntimeError("Model returned no answer.")
return answer
input_items += response.output
for tool_call in tool_calls:
function = AVAILABLE_TOOLS.get(
tool_call.name
)
if function is None:
raise ValueError(
f"Unknown tool: {tool_call.name}"
)
arguments = json.loads(
tool_call.arguments
)
if not isinstance(arguments, dict):
raise ValueError("Tool arguments must be an object.")
if tool_call.name == "find_customer":
if set(arguments) != {"email"} or type(arguments["email"]) is not str:
raise ValueError("Invalid customer lookup arguments.")
elif tool_call.name == "get_latest_invoice":
if set(arguments) != {"customer_id"} or type(arguments["customer_id"]) is not int:
raise ValueError("Invalid invoice lookup arguments.")
result = function(**arguments)
input_items.append(
{
"type": "function_call_output",
"call_id": tool_call.call_id,
"output": json.dumps(result),
}
)
raise RuntimeError(
"Agent exceeded maximum steps."
)
answer = run_agent(
"What is the status of the latest invoice "
"for max@example.com?"
)
print(answer)

This is the complete teaching example. It assumes configured SDK credentials only if you choose to run it against the service. Reading it requires no account or API key.

The instructions are supplied on every model call. The loop preserves all response output items when tools are requested, then appends one result per call. If a status, refusal, argument, tool, or step-limit check fails, the exception stops the example; it does not manufacture a success message.

That’s a basic agent.

No LangChain.

No CrewAI.

No graph framework.

No special agent SDK.

Just:

Python
LLM
functions
loop

This matters more than memorizing the code.

We call:

run_agent(
"What is the status of the latest invoice "
"for max@example.com?"
)

Python creates:

input_items = [
{
"role": "user",
"content": "...",
}
]

Then enters:

for step in range(MAX_STEPS):

Python calls the model.

The model sees:

User wants latest invoice for max@example.com.
Available tools:
find_customer
get_latest_invoice

The model realizes:

I don't have customer_id.

It returns:

function_call:
find_customer(
email="max@example.com"
)

Python sees a tool call.

So it does not return to the user.

It executes:

find_customer(
email="max@example.com"
)

Result:

{
"customer_id": 42,
"name": "Max",
}

Python appends that result to the context.

Then:

loop again

The model now sees:

USER:
latest invoice for max@example.com
MODEL REQUEST:
find_customer(...)
TOOL RESULT:
customer_id = 42

Now the model can ask:

get_latest_invoice(
customer_id=42
)

Python executes:

get_latest_invoice(42)

Result:

{
"invoice_id": 4812,
"status": "unpaid",
"amount": 250,
}

Add result.

Loop again.


Now the model sees enough information:

customer_id = 42
latest invoice:
4812
unpaid
$250

It doesn’t request a tool.

Instead it returns something like:

The latest invoice is #4812.
It is currently unpaid, with $250 outstanding.

Our Python sees:

tool_calls = []

Then:

answer = response.output_text
if not answer.strip():
raise RuntimeError("Model returned no answer.")
return answer

The loop ends.


This distinction is subtle and important.

You might hear:

The agent keeps thinking until it finishes.

That’s a useful shorthand.

But technically, Python keeps calling the model.

The model does not sit there running forever.

It produces one response.

Then Python decides what happens.

PYTHON
call model
MODEL
return tool request
PYTHON
execute tool
call model again
MODEL
return another request
PYTHON
execute
call again
MODEL
return answer
PYTHON
stop

Agency is an emergent behavior created by the orchestration loop.

That word—orchestration—is sometimes made to sound complicated.

Here it means:

The Python code coordinating model calls and tool execution.


Traditional software might hardcode:

customer = find_customer(email)
invoice = get_latest_invoice(
customer["customer_id"]
)
return invoice

The path is known beforehand:

A
↓
B
↓
C

Our agent allows the model to determine which capability it needs next:

USER TASK
↓
MODEL chooses tool
↓
RESULT
↓
MODEL chooses next step
↓
RESULT
↓
...

That’s where flexibility comes from.

But flexibility has a price.

Traditional deterministic logic is:

predictable
testable
cheap
fast

Agent loops are:

flexible
less predictable
more expensive
slower
harder to test

So you should not replace ordinary code with an agent loop merely because you can.

This will become especially important in Lesson 10.


Consider this requirement:

Retrieve the user and then retrieve their latest invoice.

Should that be an agent?

Probably not.

You already know the workflow:

customer = find_customer(email)
invoice = get_latest_invoice(
customer["customer_id"]
)

Why pay for a model to decide something Python already knows?

Now consider:

Help the customer resolve whatever account problem they’re describing.

Possible needs:

invoice lookup
order lookup
refund policy
shipping status
account information
escalation

You don’t know the required path beforehand.

That’s where model-driven tool selection becomes useful.

A good heuristic:

Use deterministic workflows when the sequence is known. Use an agent loop when judgment is genuinely required to choose the next step.


Look at:

MAX_STEPS = 10

Without it:

while True:

could theoretically continue indefinitely.

Imagine:

MODEL:
calls tool A
TOOL A:
returns ambiguous result
MODEL:
calls tool B
TOOL B:
returns error
MODEL:
calls tool A
MODEL:
calls tool B
MODEL:
calls tool A
...

Every iteration costs:

time
tokens
money
external API calls

So we create a budget.

for step in range(MAX_STEPS):

This is one of the simplest forms of agent control.

You are saying:

You may reason and act autonomously, but only within a boundary I define.

That principle will keep reappearing.


Our simple rule is:

if not tool_calls:
answer = response.output_text
if not answer.strip():
raise RuntimeError("Model returned no answer.")
return answer

Meaning:

After checking completion and refusals, a nonempty text answer with no further tool requests ends this example’s loop.

That tells us the model has stopped requesting tools. It does not prove that its answer is correct or that the user’s task was actually accomplished. Lesson 8 introduces checks for those questions.

More complex systems might define completion differently.

For example:

structured final result
explicit finish tool
workflow status
human approval
successful database state

But we don’t need any of that yet.

Keep the simple version until the problem proves it inadequate.


21. What if the model requests multiple tools at once?

Section titled “21. What if the model requests multiple tools at once?”

This is why we used:

for tool_call in tool_calls:

instead of:

tool_call = tool_calls[0]

Imagine:

Check invoices 4812 and 4813.

The model may generate two independent tool calls.

Conceptually:

get_invoice_status(4812)
get_invoice_status(4813)

This illustration uses the invoice-by-ID tool from Lesson 3. Our complete example above exposes customer lookup and latest-invoice lookup instead. The same iteration pattern applies to either registry; we have not silently added a third tool to it.

Our current implementation executes them sequentially.

That’s fine.

You could eventually execute independent calls concurrently, but do not complicate the learning example yet.

Correct first.

Optimize later.


22. Tools can return errors without crashing the agent

Section titled “22. Tools can return errors without crashing the agent”

Suppose:

find_customer("unknown@example.com")

returns:

{
"error": "customer_not_found"
}

That’s actually useful.

Send it back to the model.

MODEL REQUEST:
find_customer("unknown@example.com")
TOOL RESULT:
error = customer_not_found

The model can then answer:

I couldn’t find an account associated with that email address.

This is often better than throwing an exception for every normal business failure.

Distinguish:

EXPECTED DOMAIN FAILURE
customer not found
invoice not found
order already shipped

from:

SYSTEM FAILURE
database unavailable
network timeout
malformed response

We’ll go much deeper into that distinction in Lesson 6.


Look at everything Python currently controls:

available tools
tool implementations
tool validation
maximum steps
context
execution
error behavior
completion

The model controls:

which available capability appears useful next

That balance is extremely important.

When people say:

Give the agent autonomy.

That should not mean:

give model unlimited control

It should mean:

allow model to make selected decisions
inside software-defined boundaries

Now that we’ve seen the Python, strip it back down:

FUNCTION run_agent(task)
history = task
REPEAT up to MAX_STEPS
response = ASK model using history
VERIFY response completed without a refusal
IF response requests no tools
VERIFY final text is not empty
RETURN final text
ADD model response to history
FOR EACH requested tool
VERIFY tool is allowed
PARSE arguments
VALIDATE argument fields and types
RUN tool
ADD tool result to history
RETURN step-limit error

That’s the heart of it.

This is the kind of reasoning the course is trying to teach: make the control flow visible before adding abstractions.

Before implementation:

What owns the loop?
What causes another iteration?
What counts as completion?
What are the allowed actions?
What happens on failure?
What prevents endless execution?

Those are software-engineering questions.

Not syntax questions.


Imagine an internal research assistant with three tools:

search_documents(query)
read_document(document_id)
get_customer(customer_id)

The user asks:

Find our policy on late-payment fees and tell me whether it applies to customer 73.

Mentally trace one possible execution.

A reasonable sequence might be:

MODEL:
need late-payment policy
MODEL REQUEST:
search_documents(
query="late payment fee policy"
)
PYTHON:
executes search
RESULT:
document_id = 991
MODEL:
need actual policy text
MODEL REQUEST:
read_document(
document_id=991
)
PYTHON:
executes read
RESULT:
policy text
MODEL:
need customer facts
MODEL REQUEST:
get_customer(
customer_id=73
)
PYTHON:
executes lookup
RESULT:
customer data
MODEL:
compare policy with customer facts
MODEL:
produce final answer

Now ask yourself:

What makes this an agent rather than a fixed workflow?

Answer:

Python did not predefine that exact sequence. The model selected the next allowed capability based on the information available at each step.

That’s the key.


The loop itself is simple. The problems around the loop are where production engineering begins.

The model repeatedly requests tools without converging.

Protection:

MAX_STEPS = 10

You may later add cost or time budgets too.

A loop that calls:

send_email
issue_refund
create_order

is much more dangerous than one calling:

search
lookup
read

Retries or bad reasoning could repeat real-world actions.

Side-effect tools need additional controls, which we’ll address in Lessons 6 and 7.

Each loop adds:

model output
tool calls
tool results

Eventually context becomes large, expensive, and noisy.

That’s exactly where we’re going in Lesson 5.

Imagine:

search_documents()

returns 200 complete documents.

Now every following model call carries huge amounts of irrelevant information.

Tool design must constrain what enters context.

Your model might be fine.

Your Python loop might be fine.

But:

external_api()

waits 90 seconds.

Production tools need timeouts.

A model requests a tool with valid but dangerous arguments

Section titled “A model requests a tool with valid but dangerous arguments”

Schema validation might tell you:

customer_id is an integer

It does not tell you:

current user is authorized to access that customer

Business rules still belong in Python.

The model never produces a useful final answer

Section titled “The model never produces a useful final answer”

It may reach the step limit.

Don’t hide that failure.

Return or record something explicit such as:

agent_step_limit_exceeded

rather than fabricating success.


After four lessons, we can finally write the architecture like this:

The status and refusal checks run before the decision shown here; an empty final answer also stops the run with an error.

USER TASK
↓
┌───────────────┐
│ MODEL │
│ decide next │
│ step │
└───────┬───────┘
│
tool request?
/ \
yes no
│ │
↓ ↓
┌──────────────┐ FINAL
│ PYTHON │ ANSWER
│ validate/run │
└──────┬───────┘
│
tool result
│
└──────────────→ MODEL

That arrow back to the model is the important addition.

It creates the loop.

And the loop creates the behavior we call agency.

There isn’t a mysterious agent object hidden underneath.

There is a normal Python program doing approximately:

while not finished:
ask_model()
execute_allowed_tools()

with increasingly sophisticated rules around it.


The compact model is now:

MODEL = reasoning
TOOLS = capabilities
PYTHON = authority
LOOP = agency

But one major question is still unresolved.

Our loop currently keeps piling information into:

input_items

What belongs there?

What should survive between steps?

Between separate user requests?

What should live in the database instead?

What should not be sent to the model?

And what does people saying an agent has “memory” actually mean?

That’s Lesson 5: State and Context.

We’ll separate three things that are constantly confused in agent discussions:

application state
conversation context
long-term stored data

Once those are clear, the architecture becomes much easier to reason about.