Skip to content

06 · Retries and Failure Handling: Making the Agent Dependable

In Lesson 4, we built the agent loop. In Lesson 5, we separated application state from model context.

Our architecture now looks like this:

USER REQUEST
↓
PYTHON
↓
MODEL
↓
TOOL REQUEST
↓
PYTHON EXECUTES TOOL
↓
TOOL RESULT
↓
MODEL
↓
FINAL ANSWER

So far, we’ve assumed everything works.

The model responds. The arguments are valid. The database is available. The tool succeeds.

In production, any of those operations can fail.

Today we’re going to handle those failures without introducing a retry framework, workflow engine, or complicated recovery architecture.

The principle is simple:

When an agent encounters a failure, Python must decide whether to retry, report the failure, or stop.

Do not delegate that responsibility entirely to the model.


1. There are three different places where things break

Section titled “1. There are three different places where things break”

Start by separating failures into three categories.

Your Python application calls the model:

response = client.responses.create(...)

But something goes wrong:

network timeout
rate limit
server unavailable
invalid API key
invalid request

The problem occurred while communicating with the model provider.

The model successfully requests:

get_invoice_status(4812)

But the Python function encounters:

database unavailable
invoice not found
external API timeout
invalid arguments
permission denied

The model API worked.

The tool did not.

Every individual operation might work, but the overall agent still fails.

For example:

MODEL:
search for customer
TOOL:
customer not found
MODEL:
search for customer again
TOOL:
customer not found
MODEL:
search for customer again
...

Or the agent performs ten successful tool calls without answering the user’s question.

These are three different engineering problems.

Don’t try to solve all of them with one generic retry function.


Let’s revisit our ordinary model call.

from openai import OpenAI
client = OpenAI()
response = client.responses.create(
model="gpt-5.5-2026-04-23",
input="Explain database transactions.",
)
print(response.output_text)

What happens if the API doesn’t respond?

Without exception handling, your Python program raises an exception.

For a learning script, that’s acceptable.

For an application serving real users, we need something better.

Fortunately, the official Python SDK already does much of the work.

The current OpenAI Python SDK automatically retries certain transient failures twice by default.

These include connection errors and HTTP statuses 408, 409, 429, and 500 or higher. It also provides configurable timeouts and retry limits. GitHub

So you do not need to immediately write your own exponential-backoff system.

Start here:

from openai import OpenAI
client = OpenAI(
timeout=30.0,
max_retries=2,
)

Let’s understand those settings.

timeout=30.0

Configures the SDK’s network timeouts with a scalar value of 30 seconds. The underlying transport has connect, read, write, and connection-pool timeouts. This is not a guaranteed 30-second wall-clock deadline for a whole request attempt.

max_retries=2

Allows up to two automatic retries after the initial attempt.

Conceptually:

REQUEST 1
↓
temporary failure
↓
REQUEST 2
↓
temporary failure
↓
REQUEST 3
↓
still failing
↓
raise exception

The SDK handles the backoff between attempts.

One important detail: this timeout setting is not a deadline for the entire multi-attempt operation or agent run. An attempt may involve multiple network waits; retries and backoff add more time. A real overall time limit needs its own application policy. Lesson 4’s step limit bounds model turns, not elapsed time.


3. Catch the failure at the application boundary

Section titled “3. Catch the failure at the application boundary”

Let’s wrap the model call in a small function.

from openai import (
OpenAI,
APIConnectionError,
APIStatusError,
RateLimitError,
)
client = OpenAI(
timeout=30.0,
max_retries=2,
)
def ask_model(question: str) -> str:
try:
response = client.responses.create(
model="gpt-5.5-2026-04-23",
input=question,
)
if response.status != "completed":
return "The model could not complete the request."
if any(
part.type == "refusal"
for item in response.output
if item.type == "message"
for part in item.content
):
return "The model could not answer this request."
if not response.output_text.strip():
return "The model returned no answer."
return response.output_text
except RateLimitError:
return "The AI service is busy. Please try again later."
except APIConnectionError:
return "Unable to connect to the AI service."
except APIStatusError as error:
print(f"API error: {error.status_code}")
return "The AI service encountered an error."

Notice the exception order.

RateLimitError is a more specific kind of API status error.

APIConnectionError covers connection-related failures, including API timeouts.

APIStatusError handles other HTTP errors returned by the service.

This example returns simple messages to keep the flow visible. In a real application, you would usually log the technical details and return a structured failure to the calling code.

The important thing is what we’re not doing:

while True:
try:
call_model()
except Exception:
continue

That would be terrible.

It could keep calling the API indefinitely.

We’re using bounded retries and explicit failure handling.


This is one of today’s most important ideas.

Imagine these two failures.

Failure A:

HTTP 503
Service temporarily unavailable

Retrying might work.

Failure B:

HTTP 401
Invalid API key

Retrying the same request with the same credentials won’t fix anything.

The problem is configuration, not availability.

Here’s a useful mental model:

Failure Usually retry? Why
Network timeout Yes, bounded Could be temporary
HTTP 503 Yes, bounded Service may recover
HTTP 429 rate limit Sometimes, with backoff Capacity may become available
HTTP 400 No Request needs correction
HTTP 401 No Credentials need correction
Permission denied No Authorization must change
Record not found No Repeating the lookup probably changes nothing

The distinction is:

Transient failure: something temporarily prevented an otherwise valid operation.

Permanent failure: the operation cannot succeed without changing something.

There is one additional complication: a timeout doesn’t necessarily mean the server did nothing. The request may have completed while its response was lost.

We’ll return to that shortly.


Now let’s return to our invoice agent.

We have a tool:

def get_invoice_status(invoice_id: int) -> dict:
invoices = {
4812: {
"status": "unpaid",
"amount": 250,
},
4813: {
"status": "paid",
"amount": 175,
},
}
return invoices.get(
invoice_id,
{"error": "invoice_not_found"},
)

The model requests:

get_invoice_status(9999)

Python executes the function.

The result is:

{
"error": "invoice_not_found"
}

Is that a system failure?

No.

The application worked perfectly.

The invoice simply doesn’t exist in our data.

This is what we call an expected domain failure.

Compare it with:

def get_invoice_status(invoice_id: int) -> dict:
database = connect_to_database()
return database.get_invoice(invoice_id)

What if:

connect_to_database()

raises a connection error?

Now we have an infrastructure failure.

Different problem.

Different response.


Here’s a small improvement I recommend.

Instead of returning arbitrary structures, give your tools a simple convention.

Successful result:

{
"ok": True,
"data": {
"status": "unpaid",
"amount": 250,
}
}

Unsuccessful result:

{
"ok": False,
"error": "invoice_not_found",
}

This is not a framework.

It’s simply an interface convention.

Our function becomes the following. The invoice fixtures have not changed: 4812 is unpaid, 4813 is paid, and 9999 is absent. Only the result envelope changes to ok plus data or error.

def get_invoice_status(invoice_id: int) -> dict:
invoices = {
4812: {
"status": "unpaid",
"amount": 250,
},
4813: {
"status": "paid",
"amount": 175,
},
}
invoice = invoices.get(invoice_id)
if invoice is None:
return {
"ok": False,
"error": "invoice_not_found",
}
return {
"ok": True,
"data": invoice,
}

Now every caller can check:

result = get_invoice_status(4812)
if result["ok"]:
print(result["data"])
else:
print(result["error"])

And our model can interpret the same structure when Python sends it as a tool result.

The function-calling API allows applications to return tool results as strings, including JSON representations of successful operations or errors. OpenAI Developers


Now imagine the tool occasionally depends on an external API.

We don’t want an unexpected exception to crash the entire agent.

We can build a small execution wrapper.

import json
def execute_tool(tool_call) -> dict:
if tool_call.name != "get_invoice_status":
return {
"ok": False,
"error": "unknown_tool",
}
try:
arguments = json.loads(tool_call.arguments)
if not isinstance(arguments, dict) or set(arguments) != {"invoice_id"}:
return {
"ok": False,
"error": "invalid_arguments",
}
invoice_id = arguments["invoice_id"]
if type(invoice_id) is not int:
return {
"ok": False,
"error": "invalid_arguments",
}
return get_invoice_status(invoice_id)
except (json.JSONDecodeError, KeyError, TypeError):
return {
"ok": False,
"error": "invalid_arguments",
}
except Exception as error:
# Log technical details internally.
print(f"Tool failed: {type(error).__name__}")
return {
"ok": False,
"error": "tool_unavailable",
}

Walk through the code.

First:

if tool_call.name != "get_invoice_status":

We’re enforcing the allowed capability.

Then:

arguments = json.loads(tool_call.arguments)

We’re parsing the model’s requested arguments.

Then:

if type(invoice_id) is not int:

We’re validating what the function expects.

Finally:

return get_invoice_status(invoice_id)

We execute the real tool.

And if something unexpected happens:

except Exception:

we return a controlled error.

That broad exception handler is appropriate here only because this is an outer execution boundary for a read-only tool. We should record unexpected failures rather than silently treating programming bugs as ordinary business errors.

Notice another important distinction:

Returning tool_unavailable does not mean the operation should automatically be retried.

It tells the agent something failed.

Your application still owns retry policy.


Remember Lesson 3?

The model requested a function.

Python executed it.

Then we returned the result using:

{
"type": "function_call_output",
"call_id": tool_call.call_id,
"output": json.dumps(result),
}

That still works when result represents failure.

For example:

result = {
"ok": False,
"error": "invoice_not_found",
}

We send it back.

The model now sees:

USER:
Has invoice 9999 been paid?
MODEL:
get_invoice_status(9999)
TOOL:
ok = false
error = invoice_not_found

The model can produce a useful answer:

I couldn’t find invoice 9999.

The agent didn’t crash.

It handled the business failure.

That’s a perfectly valid outcome.


9. A tool failure doesn’t always mean retry

Section titled “9. A tool failure doesn’t always mean retry”

This is where agent engineering starts requiring judgment.

Suppose the model asks:

search_documents("refund policy")

The tool returns:

{
"ok": false,
"error": "temporary_search_failure"
}

What should happen?

There are several reasonable possibilities.

OPTION A:
Retry search once.
OPTION B:
Try a different search capability.
OPTION C:
Tell user the search is unavailable.
OPTION D:
Stop and escalate.

But consider:

{
"ok": false,
"error": "permission_denied"
}

The answer should not be:

Try another tool until we find a way around permission.

Authorization failures are boundaries, not puzzles for an agent to solve.

Your Python application should enforce those boundaries regardless of the model’s next request.

We’ll make that explicit in Lesson 7.


10. Add failure awareness to the agent loop

Section titled “10. Add failure awareness to the agent loop”

Recall our loop from Lesson 4.

We can now modify its core. This is an excerpt inside an agent function. It assumes input_items, the tool definitions, and a call_model(input_items) helper that returns a Responses API response or raises an API exception. That helper is not ask_model() above: ask_model() returns user-facing strings, which this loop cannot inspect as response objects.

The application must handle API exceptions at its boundary, while the loop checks the returned response before dispatch:

MAX_STEPS = 10
for step in range(MAX_STEPS):
response = call_model(input_items)
if response.status != "completed":
raise RuntimeError("Model response did not complete.")
if any(
part.type == "refusal"
for item in response.output
if item.type == "message"
for part in item.content
):
raise RuntimeError("Model refused the request.")
tool_calls = [
item
for item in response.output
if item.type == "function_call"
]
if not tool_calls:
answer = response.output_text
if not answer.strip():
raise RuntimeError("Model returned no answer.")
return answer
input_items += response.output
for tool_call in tool_calls:
result = execute_tool(tool_call)
input_items.append(
{
"type": "function_call_output",
"call_id": tool_call.call_id,
"output": json.dumps(result),
}
)
raise RuntimeError("Agent exceeded maximum steps.")

What’s changed?

Previously, we assumed:

result = function(**arguments)

would always succeed.

Now:

result = execute_tool(tool_call)

always gives our loop a controlled result for the failures handled by that wrapper.

It might be:

{"ok": True, "data": ...}

or:

{"ok": False, "error": ...}

Either way, we can feed that result back to the model.

The response checks preserve Lesson 4’s completion boundary. The separate model-call helper still needs the API-exception policy discussed earlier, without converting exceptions into strings where this loop expects a response.

But the overall design remains simple.


11. The dangerous case: duplicate side effects

Section titled “11. The dangerous case: duplicate side effects”

Now we get to what I consider the single most important production issue in this lesson.

Imagine our agent has this tool:

def send_invoice_reminder(invoice_id: int):
send_email(...)

The model requests:

send_invoice_reminder(4812)

Python executes it.

The email gets sent.

But immediately afterward, the connection to the email provider drops before the provider’s success response reaches us.

Our application sees:

TIMEOUT

What actually happened?

We don’t know.

Possibility A:

Email was never sent.

Possibility B:

Email was sent successfully.
Response was lost.

Now imagine we automatically retry:

send_invoice_reminder(4812)

We might send the same email twice.

For emails, that’s annoying.

For this:

issue_refund(order_id=123)

it could be expensive.

For this:

charge_credit_card(amount=500)

it could be disastrous.

A timeout means you don’t know whether the operation completed.

It does not necessarily mean the operation failed.


12. Idempotency solves a different problem than retries

Section titled “12. Idempotency solves a different problem than retries”

Here’s an important engineering term worth knowing.

Idempotency means that repeating an operation does not produce additional unintended effects.

For example:

SET invoice status = paid

is naturally easier to make idempotent than:

ADD $250 to account balance

Run the first operation twice:

paid
paid

No additional state change is necessary.

Run the second twice:

balance += 250
balance += 250

You’ve added $500.

That’s a very different outcome.

We can design operations with an idempotency key.

Conceptually:

send_invoice_reminder(
invoice_id=4812,
operation_id="reminder-4812-2026-10-08",
)

The application or external provider records that operation ID.

If the same operation is requested again, it can recognize the duplicate.

Conceptually:

REQUEST:
operation_id = X
DATABASE:
Have we processed X?
IF completed:
RETURN previous result
IF currently processing:
RETURN pending / reconcile
IF new:
CLAIM operation X
PERFORM action
RECORD result

In real systems, this typically requires durable storage, uniqueness constraints, careful handling of concurrent requests, and ideally idempotency support from the downstream service.

A Python set() is not enough for production because it disappears on restart and cannot coordinate multiple workers.

Also, do not let the model invent a fresh operation ID every time it retries.

The application should generate and preserve the ID for the original logical action.

The key principle:

Retries handle temporary failures. Idempotency handles duplicate execution.

These solve two different problems.


My recommendation for a simple agent:

Keep retry policy primarily in Python.

For example:

READ-ONLY TOOL
temporary failure:
Python may retry once or twice
permanent failure:
return error

For a side-effecting tool:

SEND EMAIL / UPDATE RECORD / ISSUE REFUND
timeout:
do not blindly repeat
check operation status
use idempotency key
or require reconciliation

You can certainly tell the model:

If a tool is temporarily unavailable, try another approach.

But that is not a replacement for application-level safeguards.

There is another important detail here.

Suppose your agent loop gets:

{
"ok": false,
"error": "temporary_failure"
}

The model might immediately call the same tool again.

Then again.

Then again.

Your MAX_STEPS would eventually stop it, but you may have wasted several calls.

For a production application, Python should also maintain a small retry budget for individual operations.

Conceptually:

state = {
"steps_used": 0,
"max_steps": 10,
"tool_attempts": {},
}

You don’t need to implement a complicated mechanism today.

Just recognize that the agent’s step limit and a tool’s retry limit are different controls.


14. A useful distinction: failure versus uncertainty

Section titled “14. A useful distinction: failure versus uncertainty”

These are not always the same.

Consider three results.

Known success:

{
"ok": true,
"status": "email_sent"
}

Known failure:

{
"ok": false,
"error": "recipient_not_found"
}

Unknown outcome:

{
"ok": false,
"error": "delivery_status_unknown"
}

The third case is special.

It means:

We cannot establish whether the action completed.

That’s not something you should hide behind a generic error.

The correct next step may be to check the external service, look up the operation ID, or ask a human to investigate.

We’ll explore the human-in-the-loop side in Lesson 7.


Before putting even a simple agent into production, I would answer these questions:

Question Simple solution
What if the model API is unavailable? Timeout + bounded SDK retries
What if API credentials are invalid? Stop and report configuration error
What if the model response is incomplete? Treat as failure, not final answer
What if tool arguments are invalid? Validate and return a controlled error
What if a record doesn’t exist? Return explicit domain error
What if the database is unavailable? Log and report tool failure
What if the agent loops? Maximum steps
What if the agent retries a payment? Idempotency + operation tracking
What if an action’s outcome is unknown? Reconcile before repeating

That’s most of the reliability architecture you need to think about initially.

No workflow framework required.


For a small personal agent with read-only tools, I’d start with just this:

client = OpenAI(
timeout=30.0,
max_retries=2,
)

Then:

MAX_STEPS = 10

Then a wrapper:

def execute_tool(tool_call):
# Validate requested tool.
# Parse arguments.
# Execute allowed function.
# Catch expected failures.
# Return structured result.
...

And finally:

try:
answer = run_agent(task)
except Exception as error:
# Log enough to diagnose the problem.
# Do not claim success.
...

That outer try is a last-resort boundary, not a substitute for handling known errors inside the agent.

For read-only operations, this is a reasonable starting point.

Once we introduce actions that change the outside world, we need stronger guarantees.


Imagine you’re building an agent with these three tools:

def search_documents(query: str):
...
def get_customer(customer_id: int):
...
def send_email(customer_id: int, message: str):
...

The model is helping a small business answer customer questions.

Three failures occur.

search_documents()
↓
HTTP 503

What should happen?

Reasonable answer:

Python may retry with bounded backoff because this is a read-only operation experiencing a likely transient failure.

If retries fail, return an explicit error.

get_customer(9999)
↓
customer_not_found

What should happen?

Reasonable answer:

Don’t retry blindly.

The lookup worked. The record doesn’t exist.

Return the result to the model so it can ask the user to verify the customer ID.

send_email(42, "Your invoice is overdue.")
↓
TIMEOUT

What should happen?

Reasonable answer:

Do not immediately send another email.

First determine whether the original operation completed.

Use an operation ID or provider-side message lookup where available.

If the outcome cannot be established safely, stop and surface the uncertainty.

If you understand why Scenario C requires different handling from Scenario A, you’ve understood the most important concept in today’s lesson.


Let’s connect everything to actual operating conditions.

Imagine 500 agent sessions encounter the same external API failure.

Every agent immediately retries three times.

Suddenly:

500 failures
↓
1,500 additional requests
↓
service gets even more overloaded

Retries should have limits, backoff, and eventually system-level rate control.

Sometimes the correct response is to stop sending requests temporarily.

The OpenAI SDK already retries certain failures.

If you wrap it in your own three-attempt retry function, you may unintentionally generate up to nine underlying requests.

Don’t stack retry mechanisms without understanding what each layer does.

A function expects:

invoice_id: int

but receives:

{
"invoice_id": "banana"
}

The model may usually respect your schema, but your application should still validate at the execution boundary.

4. Sensitive exception details leak to the model

Section titled “4. Sensitive exception details leak to the model”

A database exception might contain internal hostnames, SQL statements, connection parameters, or personal information.

Don’t automatically send:

str(exception)

back to the model.

Log appropriate diagnostic details internally. Return a safe error code externally.

Imagine:

create customer
✓
create invoice
✓
send welcome email
✗

The task partially succeeded.

You cannot simply rerun the entire workflow without checking what already exists.

This is ordinary transaction and workflow engineering, made more important by agent autonomy.

The email tool returns:

{
"ok": false,
"error": "delivery_status_unknown"
}

But the model replies:

Your email was sent successfully.

That’s unacceptable.

For consequential operations, Python should verify recorded outcomes before reporting success.

Model-generated language is not evidence of successful execution.

Your logs show:

Agent failed.

That’s not enough.

A useful minimal log should record facts such as:

task_id
step_number
tool_name
duration
success/failure
safe error code

For model API failures, a request ID can also help with debugging. The OpenAI SDK exposes request IDs on responses and relevant exceptions. GitHub

We’ll make this more systematic in Lesson 8.


Let’s update the agent architecture we’ve been developing.

Before today:

MODEL
↓
TOOL REQUEST
↓
PYTHON
↓
TOOL
↓
RESULT
↓
MODEL

Now:

USER TASK
↓
┌────────┐
│ PYTHON │
└───┬────┘
↓
MODEL CALL
│
┌──────┴──────┐
│ │
SUCCESS FAILURE
│ │
↓ ↓
TOOL REQUEST RETRY OR STOP
│
↓
VALIDATE REQUEST
│
↓
EXECUTE TOOL
│
┌──────┴─────────┐
│ │
SUCCESS FAILURE
│ │
↓ ↓
RETURN RESULT CLASSIFY FAILURE
│
┌─────────┴────────┐
│ │
RETRY SAFE? NOT SAFE?
│ │
↓ ↓
BOUNDED RETRY REPORT / STOP

Notice who’s controlling everything.

Python.

The model still reasons about which capabilities would help complete the task.

But Python controls:

  • execution;
  • retries;
  • error classification;
  • step limits;
  • permissions;
  • operation identity;
  • authoritative success or failure.

That separation is what turns a clever AI demonstration into software you can trust.


We’ve now learned six major ingredients:

MODEL = reasoning
STRUCTURED OUTPUT = contracts
TOOLS = capabilities
PYTHON = authority
LOOP = agency
STATE = application truth
CONTEXT = what the model sees
RETRY POLICY = controlled recovery

Today, the two principles I especially want you to remember are:

A retry is only safe when you understand what repeating the operation might do.

And:

A failed request is not always a failed operation.

That second principle is particularly important for systems that send messages, modify records, charge money, or trigger external workflows.

The good news is that none of this requires sophisticated agent tooling.

You’re applying established software-engineering principles to a program that happens to use an LLM.

That’s exactly the right foundation.


Next: Lesson 7 — Human Approval and Permissions

Section titled “Next: Lesson 7 — Human Approval and Permissions”

Now that our agent can execute tools and recover from failures, we need to answer a more consequential question:

What should the agent be allowed to do without asking?

We’ll build a simple approval boundary using ordinary Python.

The model might request:

delete_document(document_id=73)

But Python will decide whether to:

EXECUTE

or:

REQUIRE HUMAN APPROVAL

We’ll also separate authentication, authorization, and approval, because those are three different things.

And we’ll do it without introducing an approval framework, state-machine library, or orchestration system.

Just explicit Python rules around the capabilities we’ve already built.