October workshops are openBuild a Search AI Agent$99 early bird

Enroll now
Skip to content

PHASE 5 · LESSON 18 OF 24 · 3 SMALL IDEAS + A GUIDED LAB

A proposal must earn acceptance

Your planner confidently picks the cheapest room. The independent room record says it is closed.

You will learn to: Separate a proposal from verification, revise using unchanged requirements, and stop after a bounded number of attempts.

Preparing your lesson and this browser’s progress…
Read the complete lessonAll the ideas in one place · works without the editor

Use these notes to review a concept or read at your own pace. The interactive workspace above adds predictions, editable code, and actual run results.

IDEA 1

The planner supplies a candidate, not a verdict

A planner proposes a way to reach a goal. Its output might be a room ID, a sequence of tasks, or a piece of code. That proposal is something to inspect. Fluent wording and a confidence claim are not evidence that the plan meets your requirements.

In this lesson, propose returns an authored sequence of room IDs. There is no language model behind it. Using a predictable planner lets you focus on the controller: does it check a bad suggestion before accepting it?

Separate the two jobs

  1. Propose. The fixture suggests room A.
  2. Read evidence. The independent record says A is unavailable.
  3. Decide. The controller rejects A even if it is cheap.

A scripted proposal and a factual check are different observations.

A confident wrong answer

const proposal = "A";
const record = { available: false, cost: 5 };
// Cheap does not mean available.

The proposal names an option. Only the record supplies the facts needed to test it.

Think it through: The planner says it already checked everything. What should the controller do?

IDEA 2

The evaluator needs checking too

An evaluator turns requirements into checks. Here a room must exist, be available, fit the students, and stay within budget. AND makes every condition necessary. But the checker can contain a bug: using capacity > students wrongly rejects an exact fit. Test a known valid candidate and a deliberately invalid one before trusting its score.

Google DeepMind's AlphaEvolve evaluates candidate programs with executable checks and scores during evolutionary search. The useful connection is a checkable objective, not a guarantee that every objective captures what people need. Our room verifier uses simple comparisons. A second model repeating the planner's unsupported claim would not provide the same independent evidence.

Four gates

  1. Exists. An unknown room ID is not a valid option.
  2. Feasible. Check availability and capacity.
  3. Affordable. Check the requested budget without raising it.

The same gates apply to every proposal.

Exact boundaries count

const fits = 20 >= 20; // true
const affordable = 40 <= 40; // true

A room exactly meeting the capacity and budget limits is valid. Strict greater-than or less-than would incorrectly exclude it.

Think it through: A checker rejects a room with 20 seats for exactly 20 students. What should you inspect first?

IDEA 3

Revision needs a stopping rule

Tree of Thoughts explores alternative intermediate candidates and uses heuristic evaluations to decide which branches to pursue. A heuristic is a useful estimate, not proof: it can discard a promising branch. That research motivates comparing options under a budget; this two-proposal scripted lab does not reproduce its model search or reported results.

Set a maximum number of attempts and return a meaningful no-choice result when exhausted. Save the candidate IDs checked, keep the user's requirements fixed, and distinguish “nothing passed in these attempts” from “no solution exists.” This trace records observable actions, not a model's private reasoning. A looping planner cannot spend unlimited calls looking for a nicer answer.

Bounded search

  1. Attempt one. Inspect A and reject an unavailable option.
  2. Attempt two. Inspect C under exactly the same conditions.
  3. Stop. Return a verified choice or null with checked IDs.

The lab checks at most two authored proposals, not every possible room.

A bound you can read

for (let attempt = 0; attempt < 2; attempt++) {
  // propose, inspect, accept or continue
}

The loop has a fixed upper bound. A no-choice result means no inspected proposal passed within this budget.

Think it through: Two proposals fail and the attempt budget is two. What is honest?

Put it into practice

Verify at most two authored proposals against independent room records.

  1. Run the starter on the closed-room case and inspect its accepted choice.
  2. Replace the permissive valid flag with all four constraints.
  3. Run every case, including the one with no acceptable proposal.
  4. Change a proposed ID to an unknown value and inspect the null record.

Your next experiment: Make the second proposal exceed the budget. Does the controller preserve the user's limit?

Treat a proposal as a candidate, test the verifier as well, and bound any revision loop.

Key terms

Candidate
A proposed option that has not yet earned acceptance.
Verifier
A fallible check that tests a proposal against evidence or requirements.
Heuristic
A practical estimate used to guide a search without guaranteeing the best choice.
Attempt budget
The maximum number of proposals a run may inspect.

Sources and scope

Original Stemtiq teaching, reviewed 2026-09-14. The named researchers and organizations do not endorse this course. Classroom cases are authored exercises, not published findings.

Google DeepMind: AlphaEvolve: A Gemini-powered coding agent for designing advanced algorithms

14 May 2025 · research-system article

Propose programs, execute and score candidates, then use evaluation feedback and a program database to guide later proposals.

This research-system description is not a classroom SDK. A bounded practice loop is an adaptation, not a reproduction of AlphaEvolve or its scientific results.

Anthropic: Demystifying evals for AI agents

9 January 2026

Define tasks, trials, and graders; inspect both execution records and final outcomes; repeat trials when model behavior varies.

A score depends on its cases and grading rules. Repeating a deterministic classroom case does not measure the variability of a live model.

Yao et al. · NeurIPS: Tree of Thoughts: Deliberate Problem Solving with Large Language Models

17 May 2023 · v2 revised 3 December 2023 · reviewed 14 September 2026

Generate and evaluate alternative intermediate candidates, then explore them with bounded search and optional backtracking.

Model-generated evaluations are heuristics and can reject a useful branch. Results on the paper's tasks do not guarantee that broader search improves every task or makes a classroom controller a reasoning model.

Your JavaScript really runs. The model decisions and school data are authored simulations, so you can learn without an API key. Every workspace also includes a separate real SDK example to explore next. Passing the lab’s cases is practice, not proof that an agent is ready for real-world use.