Skip to content

08 · Evals and Observability: Knowing Whether Your Agent Actually Works

We’ve now built an agent that can:

  • Reason about a task.
  • Choose and execute tools.
  • Maintain context.
  • Handle failures and retries.
  • Request human approval for consequential actions.

But we have a problem.

How do we know whether the agent is actually reliable?

Imagine you build an agent that answers customer invoice questions.

You test it:

USER:
Has invoice 4812 been paid?
AGENT:
Invoice 4812 is unpaid. The outstanding amount is $250.

Perfect.

You run it again. Perfect.

You show it to someone. Perfect.

So you deploy it.

Then real users start asking different questions.

The agent occasionally looks up the wrong invoice, invents information, calls unnecessary tools, or gives incorrect answers.

Your successful demos told you very little about its reliability.

Today we’re going to fix that.

Evals tell you whether the agent behaves correctly. Observability tells you what happened when it ran.

We’ll build both using ordinary Python.

No evaluation framework. No tracing platform. No monitoring infrastructure.

Just Python, JSON, and a few well-chosen measurements.


These terms are closely related but serve different purposes.

Imagine an agent executing this sequence:

USER:
What is the latest invoice for customer 42?
MODEL:
get_latest_invoice(42)
PYTHON:
executes tool
TOOL:
invoice 4812, unpaid
MODEL:
final answer

Observability means recording what happened.

For example:

Run: abc123
Step 1:
Model called
Duration: 820ms
Tool requested: get_latest_invoice
Step 2:
Tool executed
Duration: 12ms
Result: success
Step 3:
Model called
Duration: 650ms
Final answer produced
Total duration: 1482ms
Status: completed

Those numbers are illustrative.

The important thing is that we can reconstruct the execution.

Now imagine asking:

Did the agent retrieve the correct invoice, use an appropriate tool, and answer accurately?

That’s evaluation.

Conceptually:

EXPECTED:
Look up customer 42.
Retrieve the latest invoice.
Report unpaid status.
ACTUAL:
Looked up customer 42.
Retrieved invoice 4812.
Reported unpaid status.
RESULT:
PASS

The distinction is:

Observability Evals
What happened? Was it correct?
Which tools ran? Were the right tools used?
How long did it take? Was the result acceptable?
Which errors occurred? Did the agent handle errors correctly?
How many tokens were used? Was the result worth the cost?

You need both.


Let’s begin with observability.

Remember our agent loop from Lesson 4?

def run_agent(task):
for step in range(MAX_STEPS):
response = client.responses.create(...)
# Handle tool calls.
# Return final answer.

Suppose this function is called 100 times.

How do we distinguish one execution from another?

Give each execution a unique ID.

import uuid
run_id = uuid.uuid4().hex

For example:

run_id = 8f7c21b4...

Now every event associated with that execution can carry the same identifier.

RUN 8f7c21b4
model_call
tool_call
model_call
completed

This becomes particularly useful when multiple agent executions are happening simultaneously.

The run ID is simply a correlation identifier.

Nothing agent-specific about it.


Traditional application logging often emphasizes failures.

logger.error("Database connection failed")

That’s useful.

But for agents, we also want to understand successful execution paths.

For example:

agent_started
model_called
tool_requested
tool_completed
model_called
agent_completed

Why?

Because an agent can execute every operation successfully and still choose the wrong sequence.

Consider:

USER:
What is invoice 4812's status?
AGENT:
find_customer()
search_documents()
get_invoice_status()
search_documents()
get_invoice_status()
final answer

Everything succeeded.

But why did it need five tool calls?

A reasonable solution might require only one.

Without recording the execution, you would never notice the unnecessary work.


Let’s create a simple function.

import json
from datetime import datetime, timezone
def log_event(run_id: str, event: str, **details):
record = {
"timestamp": datetime.now(timezone.utc).isoformat(),
"run_id": run_id,
"event": event,
**details,
}
with open("agent_runs.jsonl", "a", encoding="utf-8") as file:
file.write(json.dumps(record) + "\n")

That’s it.

We’re writing events to a file:

agent_runs.jsonl

JSONL means JSON Lines.

Each line contains one JSON object.

For example:

{"run_id":"abc123","event":"agent_started"}
{"run_id":"abc123","event":"tool_called","tool":"get_invoice_status"}
{"run_id":"abc123","event":"agent_completed","steps":2}

Unlike a large JSON array, JSONL is convenient for append-only logging.

You can inspect it with ordinary command-line tools or Python.

For now, this is enough.


Let’s revisit a simplified version of the agent.

We’ll add logging to the same loop structure. This example uses the invoice-by-ID capability from Lesson 3 and the validated execute_tool() plus ok/data/error fixtures from Lesson 6. It does not reuse Lesson 4’s customer/latest-invoice registry unchanged.

Here is the one model-facing tool definition used by this example:

tools = [
{
"type": "function",
"name": "get_invoice_status",
"description": "Look up the current status of an invoice by its ID.",
"parameters": {
"type": "object",
"properties": {"invoice_id": {"type": "integer"}},
"required": ["invoice_id"],
"additionalProperties": False,
},
"strict": True,
}
]

The following excerpt assumes client, json, log_event(), and the Lesson 6 execution wrapper are defined. It adds logging to the loop; it is not a complete second application.

import time
import uuid
def run_agent(task: str):
run_id = uuid.uuid4().hex
log_event(
run_id,
"agent_started",
)
input_items = [
{
"role": "user",
"content": task,
}
]
MAX_STEPS = 10
for step in range(MAX_STEPS):
started = time.perf_counter()
response = client.responses.create(
model="gpt-5.5-2026-04-23",
instructions="Use available tools when needed.",
input=input_items,
tools=tools,
)
duration_ms = round(
(time.perf_counter() - started) * 1000
)
log_event(
run_id,
"model_response",
step=step + 1,
duration_ms=duration_ms,
response_id=response.id,
)
if response.status != "completed":
log_event(
run_id,
"agent_failed",
reason="model_incomplete",
)
raise RuntimeError("Model response incomplete")
if any(
part.type == "refusal"
for item in response.output
if item.type == "message"
for part in item.content
):
log_event(run_id, "agent_failed", reason="model_refused")
raise RuntimeError("Model refused the request")
tool_calls = [
item
for item in response.output
if item.type == "function_call"
]
if not tool_calls:
answer = response.output_text
if not answer.strip():
log_event(run_id, "agent_failed", reason="empty_answer")
raise RuntimeError("Model returned no answer")
log_event(
run_id,
"agent_completed",
steps=step + 1,
)
return answer
input_items.extend(response.output)
for call in tool_calls:
log_event(
run_id,
"tool_requested",
tool=call.name,
call_id=call.call_id,
step=step + 1,
)
# Reuse the validated execution wrapper
# we built in Lesson 6.
result = execute_tool(call)
log_event(
run_id,
"tool_completed",
tool=call.name,
call_id=call.call_id,
step=step + 1,
success=result.get("ok", False),
error=result.get("error"),
)
input_items.append(
{
"type": "function_call_output",
"call_id": call.call_id,
"output": json.dumps(result),
}
)
log_event(
run_id,
"agent_failed",
reason="step_limit_exceeded",
)
raise RuntimeError("Agent exceeded maximum steps")

This is a teaching version. In production, put top-level exception handling around the run so unexpected crashes also produce failure events, and measure tool execution durations separately.

But look at what we’ve gained.

We can now inspect:

Which model responses occurred?
Which tools were requested?
Which tools succeeded?
How long did the model calls take?
How many steps were required?
Did the agent complete?

And we did it with a tiny logging function.


We’re missing one useful measurement.

How expensive is the agent to run?

Agent loops can become costly because every additional model call consumes tokens.

The Responses API exposes usage information, including input and output token counts, through response.usage. GitHub

We can expand our logging:

usage = response.usage
log_event(
run_id,
"model_response",
step=step + 1,
duration_ms=duration_ms,
input_tokens=usage.input_tokens if usage else None,
output_tokens=usage.output_tokens if usage else None,
)

Now we can measure:

Run A:
2 model calls
3,200 input tokens
410 output tokens
Run B:
7 model calls
18,700 input tokens
2,100 output tokens

These are illustrative figures, not measurements from an actual run.

Now imagine both runs answered the same customer question correctly.

Which one would you prefer?

Usually Run A.

But we must also check reliability.

A cheaper agent that answers incorrectly isn’t an improvement.

Eventually you want to understand:

success rate
latency
token usage
cost per successful task

Not merely how frequently the model produces a response.

Actual dollar cost requires the model’s applicable token prices and other billable usage, not just total tokens.


Let’s examine one hypothetical execution.

RUN: 73ab21
USER TASK:
Has invoice 4812 been paid?
STEP 1
Model requested get_invoice_status
Input tokens: 610
Output tokens: 85
Duration: 740ms
TOOL
get_invoice_status
invoice_id: 4812
Success: true
Duration: 8ms
STEP 2
Model returned final answer
Input tokens: 780
Output tokens: 42
Duration: 520ms
RUN COMPLETED

This tells us much more than:

Agent completed successfully.

But there’s an important distinction.

A trace records observable model outputs and application actions. It does not expose the model’s entire internal reasoning process.

What matters operationally is what the system requested, executed, returned, and reported.

Also, don’t blindly log entire prompts, tool arguments, or customer records. Logs can become a source of sensitive-data exposure.

For many applications, tool names, safe identifiers, durations, error codes, token counts, and final statuses are sufficient.


Observability tells us what happened.

An eval checks what should have happened.

The simplest useful eval is a collection of test cases.

We don’t need a framework.

We need a small dataset.

Imagine this Python list:

TEST_CASES = [
{
"name": "unpaid_invoice",
"input": "Has invoice 4812 been paid?",
"expected_tool": "get_invoice_status",
"expected_invoice_id": 4812,
"expected_fact": "unpaid",
},
{
"name": "paid_invoice",
"input": "What is the status of invoice 4813?",
"expected_tool": "get_invoice_status",
"expected_invoice_id": 4813,
"expected_fact": "paid",
},
{
"name": "missing_invoice",
"input": "Check invoice 9999.",
"expected_tool": "get_invoice_status",
"expected_invoice_id": 9999,
"expected_fact": "not_found",
},
]

Notice what we’ve created.

For each test, we know:

INPUT
user request
EXPECTED BEHAVIOR
which tool should be used
which invoice should be requested
EXPECTED FACT
what the answer should communicate

This is a tiny evaluation dataset.

And it’s already more useful than running a few demonstrations and deciding the agent seems intelligent.


9. Test behavior, not just the final answer

Section titled “9. Test behavior, not just the final answer”

This is one of the most important parts of the lesson.

Imagine the agent answers:

Invoice 4812 is unpaid.

Correct!

But what actually happened?

MODEL:
guessed invoice status
TOOL CALLS:
none

The answer happened to be right.

Should that pass?

No.

Our application requires current invoice information.

The correct behavior is:

MODEL:
requests get_invoice_status(4812)
PYTHON:
retrieves current status
MODEL:
answers using the result

We need to evaluate the process, not just the final text.

This is particularly important for agents that work with:

payments
customer records
permissions
private data
medical information
financial decisions
publishing

A correct-looking answer is not necessarily the result of a correct process.


For our evals, let’s capture which tools were requested.

Initialize this list once inside run_agent(), before the for loop, so it survives every model turn:

observed_calls = []

Every time the model requests a tool:

try:
arguments = json.loads(call.arguments)
except json.JSONDecodeError:
arguments = None
observed_calls.append(
{
"name": call.name,
"arguments": arguments,
}
)

For this exercise, we know the tool arguments contain only invoice IDs.

If the JSON itself is malformed, the captured arguments are None. The Lesson 6 execution wrapper still receives the original call and returns its controlled invalid_arguments result. Capturing an eval trace should not crash before that validation boundary runs.

In a real system, capture a safe, redacted representation instead of arbitrary tool arguments.

Then change the successful final return from:

return answer

to:

return {
"answer": answer,
"tool_calls": observed_calls,
"steps": step + 1,
}

Now the calling program can inspect both the answer and the execution path.

For example:

result = run_agent(
"Has invoice 4812 been paid?"
)

Conceptual result:

{
"answer": "Invoice 4812 is unpaid.",
"tool_calls": [
{
"name": "get_invoice_status",
"arguments": {
"invoice_id": 4812
}
}
],
"steps": 2,
}

Much easier to test.

This is an explicit return-shape change: the initial logging example returns a string; the evaluation version returns a dictionary. The evaluators below expect that dictionary. tool_calls records model requests; the execution events and application state establish whether an operation actually ran or succeeded.


Now we can check whether the agent requested the correct tool with the correct arguments.

def evaluate_tool_use(result: dict, test_case: dict) -> bool:
expected_call = {
"name": test_case["expected_tool"],
"arguments": {
"invoice_id": test_case["expected_invoice_id"]
},
}
return expected_call in result["tool_calls"]

That’s ordinary Python.

No LLM is needed to run this checker on an existing result. It checks only that the expected request appeared. It does not use expected_fact, check successful execution, establish answer accuracy, or reject extra requests; the next checks and human review cover those separate questions.

Then:

for test_case in TEST_CASES:
result = run_agent(test_case["input"])
passed = evaluate_tool_use(
result,
test_case,
)
print(
test_case["name"],
"PASS" if passed else "FAIL",
)

Example output:

unpaid_invoice PASS
paid_invoice PASS
missing_invoice FAIL

These results are illustrative.

We haven’t actually executed the tests.

And notice an important limitation: this function only checks whether the expected call occurred. It doesn’t verify that no unnecessary or unsafe calls occurred.

That would be another useful check.

For example:

def evaluate_no_forbidden_tools(result: dict) -> bool:
forbidden = {
"send_email",
"issue_refund",
"delete_customer",
}
called = {
call["name"]
for call in result["tool_calls"]
}
return not (called & forbidden)

Now we’re testing a safety property.


12. But how do we evaluate the answer itself?

Section titled “12. But how do we evaluate the answer itself?”

Consider:

EXPECTED:
Invoice 4812 is unpaid.
ACTUAL:
Invoice 4812 has an outstanding balance.

Those mean essentially the same thing.

A simple string comparison would fail.

We could check:

"unpaid" in result["answer"].lower()

But that creates problems too.

Consider:

Invoice 4812 is not unpaid.

A simplistic keyword check would find "unpaid".

The wording is wrong for our expected meaning, but the test might pass.

This brings us to a key distinction between two kinds of evaluation.

Use Python to check facts that have precise answers:

Was the correct tool called?
Were the arguments correct?
Was a prohibited tool called?
Did the run exceed its step limit?
Did the application record a successful operation?

These are excellent candidates for automated tests.

Questions such as:

Was the explanation accurate?
Did the agent answer the user's actual question?
Did it clearly communicate uncertainty?
Was the recommendation appropriate?

are harder.

You can evaluate them using manually reviewed examples, structured expected outcomes, or an LLM-based evaluator.

But an LLM-based evaluator is itself imperfect and introduces additional inference cost.

For this course, I would start with deterministic checks plus a small amount of human review.

That’s enough to discover a surprising number of problems.


Let’s keep it simple.

We want output like:

AGENT EVALUATION
Test cases: 20
Correct tool selection: 18/20
Forbidden tool calls: 0
Step-limit failures: 1
Final answers requiring review: 3

Those report figures are hypothetical, not measured results. We can calculate one basic tool-selection rate with the following loop; it does not calculate the other report lines:

total = len(TEST_CASES)
passed = 0
for test_case in TEST_CASES:
result = run_agent(test_case["input"])
if evaluate_tool_use(result, test_case):
passed += 1
rate = passed / total
print(f"Tool selection accuracy: {rate:.1%}")

This is not a comprehensive measure of agent quality.

It measures one specific property.

That’s good.

Prefer several meaningful measurements over one vague “agent quality score.”

For example:

Metric What it measures
Task completion rate Did the result satisfy the task’s acceptance criteria?
Correct tool selection Did it choose appropriate capabilities?
Argument accuracy Did it pass the correct IDs and values?
Unauthorized action rate Did unsafe actions occur?
Average model calls How many model interactions were needed?
Average duration How long did runs take?
Token usage How much model capacity did runs consume?

Each metric tells you something different.


14. The most valuable eval cases are often failures

Section titled “14. The most valuable eval cases are often failures”

It’s tempting to build tests containing only normal requests.

Find invoice 4812.
Find invoice 4813.
Find invoice 4814.

Those are useful.

But the most revealing cases often look like this:

Find invoice that doesn't exist.
Find invoice without providing an ID.
Ask for information belonging to another user.
Request a tool that is unavailable.
Provide conflicting instructions.
Ask the agent to ignore a permission rule.
Cause the same tool to return an error repeatedly.

Why?

Because real users don’t behave like carefully curated demonstrations.

And agent reliability is often determined by how the system handles ambiguity and failure.

For our invoice assistant, I’d want at least these five scenarios:

Scenario Expected behavior
Valid invoice Look up and report status
Unknown invoice Report not found
Missing invoice ID Ask for clarification
Unauthorized invoice Deny access
Database unavailable Report inability to verify, not invent status

That is a much more useful initial eval set than five slight variations of the same successful question.


15. Evals become especially important when you change something

Section titled “15. Evals become especially important when you change something”

Imagine your agent works reasonably well.

Then you improve the system prompt.

Originally:

Use tools when current information is required.

You change it to:

Be efficient. Minimize unnecessary tool calls.

Sounds reasonable.

But perhaps the model now skips necessary invoice lookups.

You might have improved latency while making the agent less reliable.

Without evals, you wouldn’t know.

A disciplined development cycle looks like:

BASELINE
↓
CHANGE PROMPT OR TOOL
↓
RUN SAME EVAL SET
↓
COMPARE RESULTS
↓
KEEP OR REVERT CHANGE

This should look familiar.

It’s essentially regression testing.

You’re testing whether a change broke behavior that previously worked.

The important difference is that model behavior can vary between runs, so one successful evaluation is not always conclusive.

For critical scenarios, run multiple trials and inspect the failure rate.


Traditional unit testing might look like:

def test_invoice_lookup():
result = get_invoice_status(4812)
assert result["data"]["status"] == "unpaid"

That’s testing your Python function.

Excellent.

Keep doing that.

But an agent eval asks something different:

Given a user question:
Does the model choose get_invoice_status?
Does it supply the correct invoice ID?
Does Python execute the tool safely?
Does the model use the returned information correctly?
Does the workflow terminate?

An agent is a system.

You need to test the system’s behavior, not merely its individual Python functions.

Ideally, you’ll have both:

UNIT TESTS
deterministic Python components
AGENT EVALS
model-driven decisions and complete workflows

For tests involving side effects, use a test database and fake email/payment services. Do not let evaluation runs perform real customer-facing actions.


Here’s where everything connects.

Suppose an eval fails:

Test:
missing_invoice
Expected:
Report invoice not found.
Actual:
Agent exceeded step limit.

Without observability, you know only:

FAIL

With a trace, you see:

STEP 1
get_invoice_status(9999)
result: invoice_not_found
STEP 2
get_invoice_status(9999)
result: invoice_not_found
STEP 3
get_invoice_status(9999)
result: invoice_not_found
...

Now the problem is obvious.

The model keeps repeating an operation that cannot succeed.

You might fix it by:

  • improving the tool description;
  • making the error response clearer;
  • adding a Python rule that blocks duplicate unsuccessful lookups;
  • teaching the agent to report missing records instead of retrying indefinitely.

And then you run the eval again.

That’s the actual improvement cycle.

EVAL FINDS FAILURE
↓
TRACE EXPLAINS FAILURE
↓
ENGINEER CHANGES SYSTEM
↓
EVAL RUNS AGAIN

This is far more useful than asking the model whether it thinks it did a good job.


For a small independent operator building a handful of practical agents, I would start with this setup:

project/
agent.py
tools.py
evals.py
test_cases.json
agent_runs.jsonl

Nothing else.

agent.py contains the loop.

tools.py contains ordinary Python capabilities.

evals.py runs the evaluation cases.

test_cases.json contains representative tasks and expected behavior.

agent_runs.jsonl stores execution events.

You don’t need an observability SaaS or evaluation platform to start.

As your system grows, those tools may become worthwhile.

But the underlying engineering practices will remain exactly the same.


Imagine an agent that helps manage a content website.

It has three tools:

search_articles(query)
create_draft(title, content)
publish_article(article_id)

Your application requires human approval before publishing.

You want to evaluate this request:

Find our existing articles about customer-support features, create a new article on a related topic, and publish it.

Design the expected behavior.

Don’t think about code yet.

Think about what an eval should verify.

The expected sequence is:

1. Agent searches existing content.
2. Agent proposes or creates a draft.
3. Agent requests publication.
4. Python checks publishing permission.
5. Python creates a pending approval.
6. Agent pauses.
7. No article is published without approval.

Your eval should verify at least:

{
"search_used": True,
"draft_created": True,
"approval_requested": True,
"published_without_approval": False,
}

Notice which property matters most.

"published_without_approval": False

That should be a hard requirement.

Even if the article is beautifully written, the agent fails the evaluation if it bypasses approval.

This illustrates an important principle:

Not all evaluation failures are equally serious.

An awkwardly worded answer may be acceptable.

An unauthorized publication is not.

Your evaluation criteria should reflect those differences.


Several problems appear once you move beyond a handful of local tests.

You test ten normal questions and get ten correct answers.

Then real users ask ambiguous, incomplete, or adversarial questions.

Your impressive test score doesn’t generalize.

Solution: Include realistic failures, missing data, permissions, and edge cases.

2. You evaluate the final answer but ignore the process

Section titled “2. You evaluate the final answer but ignore the process”

The agent gives the correct invoice status without looking it up.

That happens to work today.

Tomorrow it invents a different invoice status.

Solution: Evaluate tool selection and execution, not just final text.

Yesterday:

invoice 4812 = unpaid

Today:

invoice 4812 = paid

Your eval expects unpaid and starts failing.

Solution: Run evaluations against fixed fixtures or an isolated test database. Record the dataset version.

4. You accidentally evaluate against production systems

Section titled “4. You accidentally evaluate against production systems”

Your test suite asks an agent to issue a refund.

The agent does exactly that.

To a real customer.

Solution: Mock side-effecting tools, use isolated environments, and enforce the same approval and permission boundaries during tests.

Your trace captures full customer messages, API responses, and database records.

Now your logs contain sensitive data.

Solution: Record only the fields needed for debugging. Redact or omit sensitive values and control log access and retention.

The same prompt produces slightly different model behavior.

One run passes.

Another fails.

Solution: Run important scenarios multiple times and measure failure rates. Pin model and configuration versions where possible.

7. You improve one metric while damaging another

Section titled “7. You improve one metric while damaging another”

You reduce model calls from four to two.

Excellent.

But task accuracy falls from 95% to 75%.

Not excellent.

Solution: Evaluate quality, safety, latency, and cost together rather than optimizing one metric in isolation.

8. Your logs say success when the operation actually failed

Section titled “8. Your logs say success when the operation actually failed”

The model replies:

Your article was published.

But the publishing API returned an error.

Solution: Record authoritative tool outcomes. Evaluate the actual application state, not merely the model’s final answer.


Let’s update our agent architecture.

USER TASK
↓
┌────────────┐
│ AGENT │
│ LOOP │
└──────┬─────┘
│
┌─────────┴──────────┐
│ │
↓ ↓
MODEL CALL TOOL EXECUTION
│ │
└─────────┬──────────┘
│
↓
EVENT LOGGING
│
↓
┌────────────┐
│ TRACE │
│ what ran? │
└──────┬─────┘
│
↓
┌────────────┐
│ EVALS │
│ correct? │
└──────┬─────┘
│
↓
ENGINEER
│
improves
│
└────────→ AGENT

This gives us something we didn’t have before:

A feedback loop for improving the software itself.

The agent loop decides what actions to take.

The evaluation loop helps the engineer determine whether the agent is behaving as intended.

Those are two different loops.

And it’s the engineer who remains responsible for making changes to the system.


After eight lessons, our mental model is:

MODEL = reasoning
STRUCTURED OUTPUT = contracts
TOOLS = capabilities
PYTHON = authority
AGENT LOOP = controlled autonomy
STATE = application truth
CONTEXT = model-visible information
RETRIES = controlled recovery
PERMISSIONS = enforceable boundaries
APPROVAL = human authorization for a specific action
OBSERVABILITY = what happened
EVALS = whether it was correct

I want you to remember two things from today’s lesson.

First: An agent that works in a demonstration is not necessarily a reliable agent.

Reliability requires testing representative tasks, especially failure scenarios.

Second: Evals without observability tell you something failed. Observability helps you understand why.

And perhaps most importantly, we’re still using ordinary software-engineering principles.

Logging.

Tests.

Assertions.

Fixtures.

Metrics.

Regression testing.

All familiar concepts.

We’re simply applying them to a system where some decisions are made by a probabilistic model instead of deterministic Python code.

That’s precisely why these practices matter.


So far, we’ve built one agent with several tools.

But imagine a workflow involving two different responsibilities:

RESEARCH
↓
WRITING

Could we assign them to two separate agents?

Yes.

But what does that actually mean technically?

Next we’ll build two cooperating agents using plain Python and direct API calls.

We’ll answer:

  • What makes something a second agent rather than another function?
  • How do agents pass work to each other?
  • How do we define the contract between them?
  • Who controls the overall workflow?
  • How do we prevent unnecessary complexity?

And we’ll keep it deliberately simple.

No multi-agent framework.

No message broker.

No graph engine.

Just two model-driven components coordinated by normal Python.

Then, in Lesson 10, we’ll challenge the entire design:

When is the second agent unnecessary?

That’s an especially important question, because understanding how to build multi-agent systems is useful.

Knowing when not to build one is even more useful.