On July 2 and 3, the newest repo in Andrej Karpathy's teaching lineage became smaller and stricter. Nanochat deleted a report system, removed its bundled web UI, cut a Hugging Face dependency path, and unified two optimizers that had already diverged enough to cause a bug. Then it added 427 lines of adversarial, optimizer, task, tokenizer, and execution-sandbox tests.
That sequence—subtract, consolidate, add an oracle, then measure—is the clearest current explanation of Karpathy's educational open-source factory. The product is not a pile of tiny frameworks. It is a series of textbook editions written as executable source, each moving the boundary of what a learner is allowed to treat as magic.
The latest lesson begins with deletion
Nanochat is not literally small anymore. Its own README maps a complete train-to-chat system: tokenization, distributed data loading, pretraining, supervised finetuning, evaluation, inference, tool execution, checkpointing, and conversation. The project turns model depth into a main complexity dial and derives many surrounding choices from it.
But minimalism here is not a line-count stunt. The July cleanup says generated UI is cheap compared with a forkable research core. The optimizer consolidation says duplicate concepts become bugs. A follow-up commit moved FLOP and KV-cache accounting into the GPT model and deleted duplicate benchmark formulas. The lesson is spatial: a concept should live where a reader can find its authority.
The new tests make the subtraction credible. Removing surfaces without an answer key would only make the repo prettier. Commit f8a85a5 added checks around execution isolation, optimizer behavior, tasks, and tokenization—the failure boundaries most likely to disappear behind a successful training demo.
The boundary of magic keeps moving
The same editing method appears across the portfolio. Micrograd begins with scalar values. Each addition and multiplication creates a node in a dynamic graph; each node carries a local backward rule; a topological walk applies the chain rule. The implementation is intentionally too low-level for useful tensor work and exactly low-level enough to make reverse-mode autodiff visible.
Makemore holds the data problem still—generate more name-like words—while the model changes underneath it. Counts, multilayer perceptrons, recurrent networks, convolutional structures, and Transformers become comparable chapters rather than unrelated demos. Its source explicitly removes pretrained loading, dropout, and weight-decay machinery that the chosen teaching scale does not need.
MinGPT moves the boundary to the Transformer itself. Its causal attention class could call a higher-level PyTorch attention module, but the file keeps query, key, value projections, masking, softmax, and head reassembly explicit. The code comment gives away the editorial intent: there is “nothing too scary here.”
Llama2.c then moves below PyTorch inference. The C loop spells out QKV projection, rotary position encoding, attention, SwiGLU, residual connections, and final logits. The README's “700-line” description is historical—the current file is longer—but the durable decision remains: hard-code the architecture, avoid dependencies, and keep a complete inference path available for inspection.
Llm.c moves the boundary again, into training. Its readable CPU implementation sits beside a faster CUDA path and a library of kernels that progresses from simple versions to optimized ones. Nanochat widens the frame one final time: not only the model and trainer, but the path from raw data to an evaluated model a reader can talk to.
Tiny code still needs a large answer key
The most important recurring choice is easy to miss because it is not minimal: the repos keep an external oracle.
Micrograd's engine tests repeat scalar expressions in PyTorch, then compare both forward values and backward gradients. MinGPT's Hugging Face import test loads the same GPT-2 weights into both implementations and requires matching logits, generated token IDs, and decoded output.
Llm.c removes Python from the training loop without removing Python from the correctness argument. Its C test consumes debug state written by the PyTorch reference and checks logits, loss, gradient groups, and an optimization trajectory. The small implementation is not presented as self-evidently right. It earns the right to teach by agreeing with a system readers already trust.
This is the portfolio's most transferable move. A reference implementation should minimize the concept being explained, not the evidence needed to establish correctness. Clarity without an oracle is only confidence.
A successful chapter becomes hard to edit
The repositories also expose a maintenance problem peculiar to teaching software: success can freeze the lesson.
Karpathy's minGPT transition note says notebooks, blogs, courses, and books had begun linking into the repository. A breaking cleanup would not only affect users; it would invalidate references scattered across other people's explanations. The response was to semi-archive that chapter and move active development into nanoGPT.
In November 2025, nanoGPT made the same move explicit: the repo is old and deprecated, points readers toward nanochat, and remains online for posterity. This is not ordinary framework versioning. A textbook edition must sometimes stop changing so the page numbers still mean something.
The public lineage is careful rather than linear. Llama2.c says it adapted nanoGPT's training side and added a C inference engine. Llm.c calls its parallel PyTorch implementation a modified nanoGPT. Nanochat describes itself as a wider successor, but not every file or use case descends cleanly from the older repos. The continuity is an editorial method: preserve the old boundary, choose a new one, and rebuild around the next question.
Minimal is a complexity budget
That method also explains how these projects accept community work without letting every useful feature into the core.
Llama2.c's public guidance favors small, localized, broadly useful changes and points specialized platform work toward forks. Llm.c asks ports to live in separate linked repositories so the main project can remain C/CUDA, while its complexity policy explicitly weighs speed against dependencies and added code. Experimental complexity can live in dev/; the reference route keeps a tighter budget.
This is not solitary work disguised as a personal portfolio. The repositories carry substantial contributions, and nanochat's README credits leaderboard collaborators and community management. The distinctive skill is stewardship of the teaching boundary: deciding which contribution clarifies the chapter, which belongs on the fast path, and which deserves a separate book.
What builders can borrow
Most teams do not need to publish seven machine-learning repositories. They can still copy the architecture of the curriculum.
- Name the boundary. Decide exactly which concept the reader must see and which dependency may remain trusted for this edition.
- Keep an oracle. Compare the readable path with a mature implementation, reference state, fixture, or reproducible trajectory.
- Separate reference from fast. Let the simplest path explain the mechanism and the optimized path establish a practical upper bound.
- Give complexity an address. Put experiments, ports, UI, and specialized integrations where they cannot obscure the concept at the center.
- Let a chapter end. When compatibility with tutorials and downstream explanations makes the code impossible to reshape, preserve it and start the next edition honestly.
Those rules are useful far beyond ML. An agent runtime, compiler, database, or API client can expose one mechanism, prove it against an oracle, and resist becoming the universal framework its popularity invites.
Limits
The source trail does not prove that fewer lines always teach better. Makemore removes guardrails that would be irresponsible at a different scale. NanoGPT documents a data-domain gap in its reproduction work. Llama2.c trades generality for a readable hard-coded path. Nanochat's affordable training claims assume high-end GPU hardware, and parts of its scaling logic are candidly described as assumptions rather than settled theory.
Nor is this one author's untouched code. Contributors improved kernels, portability, testing, documentation, and evaluation across the portfolio. The defensible claim is narrower and more useful: Karpathy's public repositories repeatedly treat code as an edited learning surface. The factory works when each new edition makes one more layer inspectable—and when deletion protects the reader from everything that does not belong in that lesson.