SHORT COURSE 4 OF 6 · INTERMEDIATE · 4 GUIDED LESSONS
Test, debug, and control the work.
Useful behavior includes stopping at the right time. Bind approval to a particular action, write checks that can reject a confident answer, handle failed reads, and inspect repeated work in the actual tool trace.
What happens when a tool fails, a result is wrong, or an action needs permission?
THE BUILD YOU ARE WORKING TOWARD
An approval gate, a behavior checker, a bounded retry policy, and a per-run cache with observable call counts.
Require approval for the exact proposal being acted on.
Check evidence and outcomes independently of a proposed confidence score.
Distinguish temporary failure, an unsuitable result, and repeated work.
Each lesson follows the same rhythm: understand three small ideas, edit and run the code, then explain the result. Take one lesson at a time.
In your browser, JavaScript and local tools really execute against inspectable inputs. Model decisions and classroom records are authored examples. The optional SDK project runs separately on your computer with its own setup.
Compare requests [A, A, C, A] with [A, C]. Keep the catalog unchanged and inspect the output and trace of each run.
Explain why the output lengths differ while both need only two distinct reads.
Show a changed proposal that requires a fresh review.
Explain which failed status can be retried and where retries stop.
Use these lessons’ experiment inputs and lab notebooks to record your evidence. This is a self-review prompt; only the lesson questions and code cases contribute to the progress shown above.
Research into practiceThe ideas behind this short course
Original teaching informed by primary research and official engineering guidance. Our exercises isolate mechanisms so you can test them; they do not reproduce a research system or its reported performance.
Anthropic · 9 January 2026
Demystifying evals for AI agents
Define tasks, trials, and graders; inspect both execution records and final outcomes; repeat trials when model behavior varies.
A score depends on its cases and grading rules. Repeating a deterministic classroom case does not measure the variability of a live model.
OpenAI · Reviewed 13 September 2026 · undated documentation
Guardrails and human review
Distinguish automatic checks from approval decisions, pause sensitive tool requests, retain state, and resume after an application approves or rejects them.
Model-generated approval text is not authorization. Resume examples that automatically approve a request do not establish that a person reviewed it.
Amazon Builders’ Library · Marc Brooker · PDF copyright 2019 · reviewed 13 September 2026
Timeouts, retries, and backoff with jitter
Handle transient failures with timeouts and bounded retries, consider backoff and jitter, and establish whether repeating an operation is safe.
Retries can amplify overload or duplicate side effects. Three immediate classroom attempts are an exercise rule, not a universal policy for real services.
Observe your agent applications on Amazon Bedrock AgentCore Observability
Use instrumented traces and operational measurements to inspect execution, intermediate outputs, duration, errors, and reported model usage.
Monitoring does not enforce a budget or make an answer correct. Tool calls, tokens, elapsed time, and money are different quantities; simulated models provide no real token bill.
Your JavaScript really runs. The model decisions and school data are authored simulations, so you can learn without an API key. Every workspace also includes a separate real SDK example to explore next. Passing the lab’s cases is practice, not proof that an agent is ready for real-world use.