Enrollments Closing Soon for the Agentic AI Program.

Book A Free Session with the Mentor Now
Skip to content

SHORT COURSE 4 OF 6 · INTERMEDIATE · 4 GUIDED LESSONS

Test, debug, and control the work.

Useful behavior includes stopping at the right time. Bind approval to a particular action, write checks that can reject a confident answer, handle failed reads, and inspect repeated work in the actual tool trace.

What happens when a tool fails, a result is wrong, or an action needs permission?

YOUR FIRST STEP

Ask before changing something

Apply least privilege and check explicit approval at the tool that would cause a side effect.

Start this course
0 / 4 lessons complete
Checking your saved progress…

Your four lessons

  1. 1Your next small stepAsk before changing somethingApply least privilege and check explicit approval at the tool that would cause a side effect.
  2. 2Watch · try · explainTest behavior, not confidenceBuild representative and held-out cases; distinguish code checks, human review, and fallible model graders.
  3. 3Watch · try · explainFail clearly and recover carefullyClassify errors, bound retries, and distinguish a retryable read from a potentially duplicated write.
  4. 4Watch · try · explainSee where time and calls wentCompare correctness, duration, and call counts; use token and cost metrics only when a real runtime reports them.
What you’ll buildThe skills these four lessons connect

An approval gate, a behavior checker, a bounded retry policy, and a per-run cache with observable call counts.

  • Require approval for the exact proposal being acted on.
  • Check evidence and outcomes independently of a proposed confidence score.
  • Distinguish temporary failure, an unsuitable result, and repeated work.
Before you startHelpful background and how the code works

Use sourced evidence, explicit unknowns, and a controlled tool loop.

Find a useful piece of informationMake a claim you can supportGive the agent the right contextData is not an instruction

Watch a short video, try the code, then explain what happened. You can pause, turn on captions, or read the lesson instead.

Your JavaScript and local tools really run. Model choices and classroom records are prepared examples, so no API key is needed for these browser labs. The optional SDK project runs separately on your computer with its own setup.

Your course checkpointAfter the four lessons, connect what you learned

Compare requests [A, A, C, A] with [A, C]. Keep the catalog unchanged and inspect the output and trace of each run.

  • Explain why the output lengths differ while both need only two distinct reads.
  • Show a changed proposal that requires a fresh review.
  • Explain which failed status can be retried and where retries stop.

Use your experiment inputs and lab notebooks as evidence. This is a self-review: only lesson questions and checked code cases count toward your progress.

See where time and calls wentAsk before changing somethingFail clearly and recover carefully
Record and download your findings
Research into practiceThe ideas behind this short course

Original teaching informed by primary research and official engineering guidance. Our exercises isolate mechanisms so you can test them; they do not reproduce a research system or its reported performance.

Anthropic · 9 January 2026

Demystifying evals for AI agents

Define tasks, trials, and graders; inspect both execution records and final outcomes; repeat trials when model behavior varies.

A score depends on its cases and grading rules. Repeating a deterministic classroom case does not measure the variability of a live model.

OpenAI · Reviewed 13 September 2026 · undated documentation

Guardrails and human review

Distinguish automatic checks from approval decisions, pause sensitive tool requests, retain state, and resume after an application approves or rejects them.

Model-generated approval text is not authorization. Resume examples that automatically approve a request do not establish that a person reviewed it.

Amazon Builders’ Library · Marc Brooker · PDF copyright 2019 · reviewed 13 September 2026

Timeouts, retries, and backoff with jitter

Handle transient failures with timeouts and bounded retries, consider backoff and jitter, and establish whether repeating an operation is safe.

Retries can amplify overload or duplicate side effects. Three immediate classroom attempts are an exercise rule, not a universal policy for real services.

Amazon Builders’ Library · Reviewed 13 September 2026 · undated guidance

Caching challenges and strategies

Reusing observations can reduce dependency calls; cache scope, freshness, and failure behavior need deliberate design.

A room ID is a sufficient classroom cache key only while the task and room facts remain unchanged. Caching does not itself refresh stale information.

AWS · Amazon Bedrock AgentCore · Reviewed 13 September 2026 · undated documentation

Observe your agent applications on Amazon Bedrock AgentCore Observability

Use instrumented traces and operational measurements to inspect execution, intermediate outputs, duration, errors, and reported model usage.

Monitoring does not enforce a budget or make an answer correct. Tool calls, tokens, elapsed time, and money are different quantities; simulated models provide no real token bill.

Want to build with an instructor?Explore the workshop and longer program

Your JavaScript really runs. The model decisions and school data are authored simulations, so you can learn without an API key. Every workspace also includes a separate real SDK example to explore next. Passing the lab’s cases is practice, not proof that an agent is ready for real-world use.