Start from one person's problem. Build a complete small workflow, compare authored run records under the same rules, and present the evidence behind your claims. Then apply these habits to a project in your chosen major.
Can you show what your agent does, where it fails, and why its design fits your chosen field?
THE BUILD YOU ARE WORKING TOWARD
A scoped project brief, a runnable catalog-backed preview, a comparison of authored run records, and an honest project report you can download with your notebook.
Define the user, allowed actions, success criteria, and a simple baseline.
Keep missing evidence and failed execution visible when comparing versions.
Document the code, environment, checks, results, and remaining limits.
Each lesson follows the same rhythm: understand three small ideas, edit and run the code, then explain the result. Take one lesson at a time.
In your browser, JavaScript and local tools really execute against inspectable inputs. Model decisions and classroom records are authored examples. The optional SDK project runs separately on your computer with its own setup.
Give a candidate version one improvement and one regression. In the report exercise, remove a version field and keep an errored run in the input.
Show why equal total scores can hide a new failure.
Keep errors in the attempted-case count and missing metadata explicit.
Use your notebook to explain one success, one failure, and the next experiment for a major-specific project.
Use these lessons’ experiment inputs and lab notebooks to record your evidence. This is a self-review prompt; only the lesson questions and code cases contribute to the progress shown above.
Research into practiceThe ideas behind this short course
Original teaching informed by primary research and official engineering guidance. Our exercises isolate mechanisms so you can test them; they do not reproduce a research system or its reported performance.
Anthropic · 9 January 2026
Demystifying evals for AI agents
Define tasks, trials, and graders; inspect both execution records and final outcomes; repeat trials when model behavior varies.
A score depends on its cases and grading rules. Repeating a deterministic classroom case does not measure the variability of a live model.
Jimenez et al. · SWE-bench research team · ICLR 2024 · original benchmark
SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
Evaluate patches against repository issues and executable tests using a reproducible harness. Inspect the actual code change and its tested behavior.
Benchmark variants cover different tasks. Passing a repair case does not establish general coding ability or that a patch meets every unstated requirement.
Zhou et al. · Carnegie Mellon University · 2023 preprint · ICLR 2024
WebArena: A Realistic Web Environment for Building Autonomous Agents
Evaluate browser agents in reproducible website environments, checking functional outcomes rather than requiring one exact sequence of actions.
This research benchmark does not represent every website or permission boundary. Classroom examples are not WebArena results, and historical model scores are not current rankings.
NVIDIA · Version 1.8 observed · reviewed 13 September 2026
Agent Evaluation in NVIDIA NeMo Agent Toolkit
Run curated cases, inspect generated answers and intermediate steps, retain effective configurations, and enable profiling when operational measurements are needed.
Different artifacts require different configuration. Scores depend on cases and graders; timing measurements do not establish correctness, and model-based grading is fallible.
Observe your agent applications on Amazon Bedrock AgentCore Observability
Use instrumented traces and operational measurements to inspect execution, intermediate outputs, duration, errors, and reported model usage.
Monitoring does not enforce a budget or make an answer correct. Tool calls, tokens, elapsed time, and money are different quantities; simulated models provide no real token bill.
Your JavaScript really runs. The model decisions and school data are authored simulations, so you can learn without an API key. Every workspace also includes a separate real SDK example to explore next. Passing the lab’s cases is practice, not proof that an agent is ready for real-world use.