Lesson 5 of 7 · About 3 minutes
Test the result, then improve it
Choose checks that measure the goal and use a failure to revise the design.
01 / Learn the idea
How you know it worked
An evaluation is a test with an input and a rule for judging success. Check the actual outcome, not just the agent's claim that it finished. A room-planning agent should produce a plan that meets the capacity and time requirements. Clear writing alone cannot show that those requirements were met.[1]
A review step can send specific feedback into a revision loop. Choose a stopping condition and a limit on attempts. Editing an instruction or adding feedback changes the next input; it does not automatically train the model or guarantee that the revision will pass.[2][3]
Design a tiny test set for our club helper: 20 students, 21 students, and a room with unknown capacity. A helper that passes the first case might fail the others. Keep some fresh cases for after a change, so you check whether your fix handles more than the example that inspired it.
Original school example
A boundary test
A room holds 20. Test a group of 19, a group of 20, and a group of 21. The first two fit; the third does not. This exposes an error that an easy 10-student example could miss.
It shows success on that case. Vary the inputs and, for live models, repeat trials because outputs can vary.
02 / Explore the flow · Guided simulation
A better score—or a broken objective?
A six-slot computer lab needs 20 devices in slots 2, 3, and 5. Each powered device uses one fictional energy unit per slot. Choose what the evaluator checks.
Original fictional scenario. Decisions are authored; calculations and checks run in your browser. No live AI or external action.
↳ A failed check can send the flow back to another action—or to a human.
Step 1 of 5 · Visible system event
Goal
Reduce simulated energy while serving all required classes.
Inspect the visible trace (1 events)
- Goal
Reduce simulated energy while serving all required classes.
This log shows authored actions, tool results, and checks. It is not a model’s private reasoning.
03 / Check your understanding
A room agent works for 10 students. What is the most useful next evaluation?
Where these ideas come from
Primary sources, explained in our own words. Research findings and engineering guidance are different kinds of evidence; neither is a guarantee that an agent will succeed.
Supports: Explicit success criteria; Checking outcomes and repeated trials.
Limit: Practical evaluation guidance; passing a limited test set does not establish general competence.
Supports: Review and revision loops; Multi-agent coordination tradeoffs.
Limit: Architecture guidance, not a controlled comparison proving that more agents are better.
Supports: Prompt changes versus parameter updates.
Limit: Used only for the distinction between prompting and training, not its broader performance statements.
Stemtiq original lesson · Sources checked September 12, 2026 · Editorial review pending. School examples and guided simulations are original simplifications, not replications of the cited systems. No research affiliation or endorsement is implied.