Use these notes to review a concept or read at your own pace. The interactive workspace above adds predictions, editable code, and actual run results.
IDEA 1
The planner supplies a candidate, not a verdict
A planner proposes a way to reach a goal. Its output might be a room ID, a sequence of tasks, or a piece of code. That proposal is something to inspect. Fluent wording and a confidence claim are not evidence that the plan meets your requirements.
In this lesson, propose returns an authored sequence of room IDs. There is no language model behind it. Using a predictable planner lets you focus on the controller: does it check a bad suggestion before accepting it?
Separate the two jobs
- Propose. The fixture suggests room A.
- Read evidence. The independent record says A is unavailable.
- Decide. The controller rejects A even if it is cheap.
A scripted proposal and a factual check are different observations.
A confident wrong answer
const proposal = "A";
const record = { available: false, cost: 5 };
// Cheap does not mean available.The proposal names an option. Only the record supplies the facts needed to test it.
Think it through: The planner says it already checked everything. What should the controller do?
IDEA 2
The evaluator needs checking too
An evaluator turns requirements into checks. Here a room must exist, be available, fit the students, and stay within budget. AND makes every condition necessary. But the checker can contain a bug: using capacity > students wrongly rejects an exact fit. Test a known valid candidate and a deliberately invalid one before trusting its score.
Google DeepMind's AlphaEvolve evaluates candidate programs with executable checks and scores during evolutionary search. The useful connection is a checkable objective, not a guarantee that every objective captures what people need. Our room verifier uses simple comparisons. A second model repeating the planner's unsupported claim would not provide the same independent evidence.
Four gates
- Exists. An unknown room ID is not a valid option.
- Feasible. Check availability and capacity.
- Affordable. Check the requested budget without raising it.
The same gates apply to every proposal.
Exact boundaries count
const fits = 20 >= 20; // true
const affordable = 40 <= 40; // trueA room exactly meeting the capacity and budget limits is valid. Strict greater-than or less-than would incorrectly exclude it.
Think it through: A checker rejects a room with 20 seats for exactly 20 students. What should you inspect first?
IDEA 3
Revision needs a stopping rule
Tree of Thoughts explores alternative intermediate candidates and uses heuristic evaluations to decide which branches to pursue. A heuristic is a useful estimate, not proof: it can discard a promising branch. That research motivates comparing options under a budget; this two-proposal scripted lab does not reproduce its model search or reported results.
Set a maximum number of attempts and return a meaningful no-choice result when exhausted. Save the candidate IDs checked, keep the user's requirements fixed, and distinguish “nothing passed in these attempts” from “no solution exists.” This trace records observable actions, not a model's private reasoning. A looping planner cannot spend unlimited calls looking for a nicer answer.
Bounded search
- Attempt one. Inspect A and reject an unavailable option.
- Attempt two. Inspect C under exactly the same conditions.
- Stop. Return a verified choice or null with checked IDs.
The lab checks at most two authored proposals, not every possible room.
A bound you can read
for (let attempt = 0; attempt < 2; attempt++) {
// propose, inspect, accept or continue
}The loop has a fixed upper bound. A no-choice result means no inspected proposal passed within this budget.
Think it through: Two proposals fail and the attempt budget is two. What is honest?
Put it into practice
Verify at most two authored proposals against independent room records.
- Run the starter on the closed-room case and inspect its accepted choice.
- Replace the permissive valid flag with all four constraints.
- Run every case, including the one with no acceptable proposal.
- Change a proposed ID to an unknown value and inspect the null record.
Your next experiment: Make the second proposal exceed the budget. Does the controller preserve the user's limit?
Treat a proposal as a candidate, test the verifier as well, and bound any revision loop.
Key terms
- Candidate
- A proposed option that has not yet earned acceptance.
- Verifier
- A fallible check that tests a proposal against evidence or requirements.
- Heuristic
- A practical estimate used to guide a search without guaranteeing the best choice.
- Attempt budget
- The maximum number of proposals a run may inspect.
Sources and scope
Original Stemtiq teaching, reviewed 2026-09-14. The named researchers and organizations do not endorse this course. Classroom cases are authored exercises, not published findings.
Google DeepMind: AlphaEvolve: A Gemini-powered coding agent for designing advanced algorithms
14 May 2025 · research-system article
Propose programs, execute and score candidates, then use evaluation feedback and a program database to guide later proposals.
This research-system description is not a classroom SDK. A bounded practice loop is an adaptation, not a reproduction of AlphaEvolve or its scientific results.
Anthropic: Demystifying evals for AI agents
9 January 2026
Define tasks, trials, and graders; inspect both execution records and final outcomes; repeat trials when model behavior varies.
A score depends on its cases and grading rules. Repeating a deterministic classroom case does not measure the variability of a live model.
Yao et al. · NeurIPS: Tree of Thoughts: Deliberate Problem Solving with Large Language Models
17 May 2023 · v2 revised 3 December 2023 · reviewed 14 September 2026
Generate and evaluate alternative intermediate candidates, then explore them with bounded search and optional backtracking.
Model-generated evaluations are heuristics and can reject a useful branch. Results on the paper's tasks do not guarantee that broader search improves every task or makes a classroom controller a reasoning model.