What you’ll make work
Choose a discount rule, build a small set of boundary tests, and use executed results to verify a repair. Plain-English controls make code optional.
New to AI agents? Start with seven short foundation lessons on models, tools, feedback, evidence, and boundaries. Then return here to apply the ideas.
Each run connects a goal to a tool result, a check, and a next action. You change a meaningful setting, watch what changes, then apply the idea to a new case.
- Goal
- Apply 20% off an entire order of at least 10 tickets.
- Tools
- Read the requirement, inspect the function, and execute tests.
- Check
- Compare required and actual totals at 9, 10, and 11 tickets.
- Stop
- Finish when the selected checks pass, including the zero-ticket case.
This is a guided simulation: its authored controller selects from a bounded set of actions, while its tools execute calculations and checks. A live model would choose actions dynamically.
Original worked example · fictional scenario
One ticket count exposes a $20 bug
In this fictional club, tickets cost $10 each. An order of 10 or more gets 20% off its entire total. Inputs are whole numbers from 0 to 100, and calculations use integer cents. The starting rule mistakenly discounts only more than 10 tickets.
| Tickets | Required total | Starting function | After repair |
|---|---|---|---|
| 9 | $90.00 | $90.00 · passes | $90.00 · passes |
| 10 | $80.00 | $100.00 · fails | $80.00 · passes |
| 11 | $88.00 | $88.00 · passes | $88.00 · passes |
| 0 | $0.00 | $0.00 · passes | $0.00 · passes |
run_tests identifies a mismatch at 10 tickets: 10,000 cents returned, 8,000 cents required. The neighboring cases pass.
What happens nextThe failed result leads to an allowed repair: change “more than 10” to “at least 10.” The agent reruns the selected tests before finishing.
A boundary is the point where a rule changes. Testing just below, at, and just above it distinguishes “more than” from “at least.” Changing the rule to “at least 9” would fix the 10-ticket example but break 9 tickets. The neighboring test catches that new error. Passing this set verifies the listed behaviors; it does not prove every possible property of a production system.
Common mistakes worth catching
Accepting “fixed” as evidence
Read the executed totals. A success message is a claim; a checked result shows whether the requirement was met.
Testing only easy examples
Five and 15 tickets both miss this particular bug. Exactly 10 is where the starting function differs from the rule.
Fixing one case and breaking its neighbor
Keep a case below the boundary and the existing zero-ticket case. A repair should preserve previously correct behavior.
What to take with you
- Translate a requirement into expected outputs.
- Test where the rule changes, including both neighbors.
- Use failed tool output to guide a repair, then run the checks again.
Finishing the activity gives you a record of the configuration you changed, the case you tested, and the result you observed. The new case checks your understanding separately from the guided run.
A few good questions
Do I have to write code?
No. You choose plain-English rules and test cases. An optional code view shows the same bounded function that the tool executes.
What does a test actually check?
It gives the function an input and compares the returned total with the total required by the promotion. A mismatch is a failing test.
Have I built a production coding agent?
You configured and tested a simulated coding agent. Its allowed repairs and tools are intentionally small so you can inspect exactly what changed.
Research behind this lesson
Anthropic · January 9, 2026
Checks observable outcomes and tool traces with executable evaluations, rather than relying on an agent's claim of success. Demystifying evals for AI agents ↗
Engineering guidance, not a controlled learning study. Our ticket calculator is an original bounded exercise.
This school scenario is a simplified simulation. The research connection explains the method that inspired the activity; it does not establish this lesson’s effectiveness or imply an endorsement.
Content version: 2026-09-12.1 · Prepared September 12, 2026
Editorial review pending · No completed specialist review or review date recorded.