AI × COMPUTER SCIENCE
Explore AI in Computer Science
AI software can classify data, generate code, or call tools. The input, expected behavior, and evaluation determine how you should build each system.
Compare three types of system, examine how SWE-bench tests repository repairs, and consider a classifier or code-review project.
Select a step to see what happens.
Inputs and task
Define the intended behavior
Specify the input, expected output, and a simple baseline. Keep evaluation cases separate from the examples used to build the system.
Model or tool
Inspect how the result was produced
A classifier predicts a label; a generative model proposes material; an agent can call tools. Each system needs a suitable evaluation.
Tests and review
Check behavior beyond one example
Run the relevant checks and inspect failures. Generated code still needs a developer to understand and validate its behavior.
Schematic software workflow. Checks shown here are illustrative.
THE BASICS
A model is one part of a system
Imagine sorting messages for a school club. Rules could look for a phrase. A classifier could learn categories from labeled examples. A generative model could draft a reply. An agent could select a lookup tool. Each approach has different inputs, outputs, and ways to fail.
Computer science examines how information is represented and processed. In an AI application, inspect the model together with its input data, surrounding code, tool permissions, and tests.
Using a coding assistant, building an application around a model, and training a model require different preparation. Programming and test design support application work; model training adds data preparation and statistical evaluation. Original research requires a further understanding of methods and prior work.
APPLICATIONS
Which kinds of AI systems can you build?
Follow the information through each task: the input, the model’s contribution, and the person responsible for checking the result.
Documented machine-learning example
Predicting a category from measured features
A system needs to assign a category to a new observation rather than remember examples it already saw.
- The material
- The scikit-learn evaluation guide uses iris measurements and species labels to explain training and held-out evaluation.
- AI’s role
- A classifier learns a mapping from features to labels. Its predictions can then be compared with labels in data withheld from training.
- The human decision
- Choose meaningful features, check the data split, and compare held-out predictions with a simple baseline. Examine which categories account for the errors.
The limitRepeatedly choosing model settings against the test set leaks information into evaluation. The documentation distinguishes model selection from a final held-out test.[1]
Published code-generation research
Generating a candidate function from a specification
Different implementations can satisfy the same programming requirement. Evaluation needs to check behavior under specified inputs.
- The material
- The HumanEval paper evaluates generated Python functions using written task descriptions and associated tests.
- AI’s role
- A language model proposes code. Execution-based evaluation checks aspects of the function's behavior rather than whether it resembles one reference solution.
- The human decision
- Read the requirement, understand the proposed algorithm, and inspect edge cases before trusting the implementation. Keep generated code inside an appropriate isolated environment.
The limitPassing a finite test suite is evidence about those checks. It does not establish every requirement, security property, or behavior in an unfamiliar application.[2][5]
Published agent research and an educational workflow possibility
Looking up information through permitted tools
A model answering from its existing context may need information that an external resource can provide.
- The material
- ReAct research combines generated planning text, task-specific actions, and observations; its question-answering experiments use a Wikipedia interface.
- AI’s role
- A tool-using system can select an available action, read its result, and continue. This is one kind of AI system, alongside classifiers and generators.
- The human decision
- For a student workflow, define the allowed tools, preserve source links, limit retries, and review the evidence behind the answer.
The limitInspect the observations returned by each tool and the sources supporting the conclusion. Keep permissions restricted to what the task requires.[3]
CASE STUDY
How SWE-bench evaluates a repository repair
A repository repair must work in the surrounding software.
The original SWE-bench paper builds tasks from GitHub issues and corresponding changes in Python repositories. A system receives an issue description and a repository state, then proposes a patch. It must locate relevant code and work within an existing project rather than fill an isolated function.[4]
Its evaluation checks tests that should change from failing to passing and tests that should continue passing. A task is considered resolved when the required groups pass. That second group matters: a repair that fixes one example while breaking established behavior is incomplete.[4]
The benchmark evaluates selected repositories and test suites. Security, usability, and requirements outside those tests may need other checks. Report the evaluated task and test conditions with the result.
Follow the process Open a step to see what happens.
Inspect
Issue description and relevant repository context.
Propose
A change addressing the specified behavior.
Check
The repair and previously working behavior.
A repair report should connect the issue, proposed change, newly passing tests, and regression checks. That record lets a reviewer examine both the fix and its effect on existing behavior.[4]
SKILLS & STUDY
What to learn first
- Read and explain code
- Learn variables, functions, collections, control flow, and debugging. Explain what an implementation does in your own words before extending it.
- Reason about data
- Understand labels, features, class balance, and the difference between a training example and an evaluation case. Trace where each input came from.
- Design checks
- Translate a requirement into observable behavior. Include ordinary cases, boundaries, invalid inputs, and comparisons with a simple baseline.
- Build maintainable systems
- Use version control, document dependencies, and define permissions. NIST's AI-focused SSDF profile places security practices throughout development rather than at the final demonstration.
PROJECT IDEAS
Projects with tests you can explain
Choose a question, inspect the requirements, and keep a brief for later. Each proposal includes a way to check the work.
Save an idea in this browser or copy its brief to discuss with a mentor.
No account neededA small classifier with an error report
Does a simple model improve on a stated baseline for a clearly defined classification task?
- Intended output
- A reproducible analysis with a fixed split, comparison table, confusion matrix, and discussion of mistaken predictions.
Skills, sources and checks
- Starting knowledge
- Python functions and arrays, proportions, and supervised-learning basics. Understand the evaluation before adding model complexity.
- Material to consider
- The iris example documented by scikit-learn, or an appropriately licensed non-personal dataset with clear feature and label definitions.
- How to judge the result
- Select settings without consulting final test labels. Compare against a majority-class baseline and inspect errors by category.
A proposed small-dataset project; its results do not transfer automatically to unrelated data.
A code-review workflow that shows its evidence
Can an assistant identify a seeded issue and support its suggestion with relevant code and a meaningful check?
- Intended output
- A proposed patch, file-linked explanation, review checklist, and before-and-after test evidence from a prepared repository.
Skills, sources and checks
- Starting knowledge
- Reading a small codebase, writing tests, and basic Git. A mentor can help scope the repository and allowed tools.
- Material to consider
- A synthetic or permissioned educational repository containing one small, documented issue. Exclude credentials and private user records.
- How to judge the result
- Check the targeted behavior, existing passing tests, unsupported findings, and whether the explanation points to the right evidence.
A proposed workflow with human review; no changes to real repositories.
Use permitted data and code. Keep permissions narrow; distinguish suggested changes from approved changes.
MENTOR SUPPORT
Build and evaluate a code-review agent
The STEMtiq Agentic AI Scholars Program has a Computer Science pathway centered on a Software Engineering & Code Review Agent. Its stated workflow uses a prepared repository, a proposed patch, approved sandboxed checks, and evidence for human review.
Discuss your coding experience, the repository, and the evaluation plan with a mentor. The pathway develops one agent-building workflow; training classifiers or larger models requires separate preparation.
5 weeks · 25 live hours · Grades 9-12
COMMON QUESTIONS
What students ask
Is every AI system an agent?
No. A classifier predicts a category and a generative model produces candidate material. An agent connects model decisions with available actions or tools. You can learn useful AI concepts without building an agent or giving a model permission to act.
How much coding should I learn first?
Begin with small programs, functions, data structures, and debugging. Classifier analysis adds arrays and evaluation concepts; repository work adds tests and version control. Training a large model is unnecessary for either proposed project.
Why keep a test set separate?
You need evidence about examples that did not guide model selection. Repeatedly changing a model after inspecting test results turns that set into part of development and weakens the final comparison. Reserve independent cases for the final evaluation.[1]
If generated code passes tests, is it finished?
Passing tests supports the behaviors those tests examine. Review the requirement, edge cases, dependencies, and permissions as well. Benchmark tests cover specified behaviors; security and maintainability may require additional review.[2][5]
What can I show if I used an existing model?
Show your question, code, sources, baseline, evaluation cases, and revisions. Identify the model or library you used and explain your own decisions.
Start with one software behavior.
Choose a small repository issue and describe the behavior that should change. A mentor can help define the allowed tools and the tests needed to evaluate the proposed repair.
Sources & further reading
- [1]scikit-learn: Cross-validation: evaluating estimator performance (Current technical documentation)Return to guide ↑
- [2]
- [3]
- [4]Jimenez et al. · ICLR: SWE-bench: Can Language Models Resolve Real-World GitHub Issues? (ICLR 2024; revised November 2024)Return to guide ↑
- [5]
Sources inform this guide; their organizations do not endorse STEMtiq. Project ideas and diagrams are original editorial explanations.