Use these notes to review a concept or read at your own pace. The interactive workspace above adds predictions, editable code, and actual run results.
IDEA 1
A retrieved passage is a candidate
Evidence is information that bears on a particular claim. A dated record about room A cannot automatically support a claim about room B. In this exercise each fictional record names a subject, a value, a date, and a source ID. Those fields make the support check visible instead of asking a mysterious 'truth tool' to decide.
A source can be relevant and still disagree with another relevant source. Keep disagreement visible. Do not silently choose the record with the more confident wording or the value you hoped to find. First gather records for the requested subject; then compare what they actually state.
Match the subject first
- Claim. We want the capacity of room A.
- Records. A notice about B is not evidence about A.
- Relevant set. Compare only records about the requested subject.
Follow this authored example, then test the idea in the lab.
Read the example
const relevant = records.filter(
record => record.subject === input.subject
);This fixture uses exact subject IDs. Real claims may need a more detailed, evaluated matching method.
Think it through: A record says room B holds 30 people. Does it support 'room A holds 30'?
IDEA 2
Match a claim to its source
A useful citation lets another person recover the record that supports the answer. Preserve both its ID and date. A date tells the reader when the statement was recorded; it does not automatically establish that it is current. In this lab we deliberately stop when relevant records disagree instead of applying an unstated 'newest always wins' rule.
If all inspected records for a subject agree on the value, return that value with their source references. The program is reporting agreement within this collection. It is not certifying the real world. Our school records are invented, and the only facts the checker evaluates are the supplied fields.
Keep the evidence attached
- Record. The fact and provenance arrive together.
- Compare. Check other records for the same subject.
- Answer. Return the value and source references together.
Follow this authored example, then test the idea in the lab.
Read the example
const sources = relevant.map(r => ({
id: r.id, date: r.date
}));Keeping the date alongside the source makes the scope of your evidence inspectable.
Think it through: Two relevant records agree. What does a source-linked answer establish?
IDEA 3
Say what the evidence cannot settle
Missing information and contradictory information are different problems. With no relevant record, the answer is unknown because the collection has no evidence for the subject. With two different values for the same subject, the answer is unresolved because the evidence conflicts. Returning null alone would hide which problem occurred.
Use an explicit status such as unknown, conflict, or supported. That small vocabulary helps the next part of a program choose a sensible response. An unknown might trigger a request for the missing record. A conflict might require someone to check a source version. Neither should be disguised as a confident answer.
Three honest outcomes
- No records. No inspected evidence about this subject.
- Different values. Relevant records disagree.
- Agreement. Return the supported value with its record dates.
Follow this authored example, then test the idea in the lab.
Read the example
if (relevant.length === 0) return { status: "unknown", sources: [] };
if (!relevant.every(r => r.value === relevant[0].value))
return { status: "conflict", sources };The status explains why no single value is returned. It is not an admission of a broken program.
Think it through: Room A has one record saying 24 seats and another saying 28. No version policy is supplied. What should you return?
Put it into practice
Report supported, conflicting, or missing evidence for a requested subject.
- Read the selected case and predict its expected result.
- Run the starter once. Use the failed check and tool trace to locate the missing rule.
- Insert the explained snippet at the TODO, then run the case again.
- Test all three cases. Change the experiment input and explain whether the same rule still works.
Your next experiment: Give both A records the same value, then change one. Explain why the status changes while the source list stays.
Support is a relationship between a specific claim and specific evidence—not the appearance of a citation.
Key terms
- Provenance
- Where a record came from and how it can be identified.
- Conflict
- Relevant records state incompatible values.
- Unknown
- The available evidence does not establish a result.
Sources and scope
Original Stemtiq teaching, reviewed 2026-09-14. The named researchers and organizations do not endorse this course. Classroom cases are authored exercises, not published findings.
Lewis et al. · NeurIPS: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
NeurIPS 2020
Combine information retrieval with generation so a model can use external passages while producing an answer.
The paper uses a trained neural retrieval and generation architecture. A keyword lookup illustrates retrieval but neither reproduces that architecture nor proves a retrieved claim true.
Anthropic: Demystifying evals for AI agents
9 January 2026
Define tasks, trials, and graders; inspect both execution records and final outcomes; repeat trials when model behavior varies.
A score depends on its cases and grading rules. Repeating a deterministic classroom case does not measure the variability of a live model.