karpathy /autoresearch

AI agents running research on single-GPU nanochat training automatically

★ 100,086⑂ 13,777👁 732⑃ 408● 116▤ 0
PythonCreated 2026-03-06
+43
Daily
+442
Weekly
100,086
stars
13,777
forks
732
watchers
408
prs
116
issues
0
releases

README

autoresearch

!teaser

One day, frontier AI research used to be done by meat computers in between eating, sleeping, having other fun, and synchronizing once in a while using sound wave interconnect in the ritual of "group meeting". That era is long gone. Research is now entirely the domain of autonomous swarms of AI agents running across compute cluster megastructures in the skies. The agents claim that we are now in the 10,205th generation of the code base, in any case no one could tell if that's right or wrong as the "code" is now a self-modifying binary that has grown beyond human comprehension. This repo is the story of how it all began. -@karpathy, March 2026.

The idea: give an AI agent a small but real LLM training setup and let it experiment autonomously overnight. It modifies the code, trains for 5 minutes, checks if the result improved, keeps or discards, and repeats. You wake up in the morning to a log of experiments and (hopefully) a better model. The training code here is a simplified single-GPU implementation of nanochat. The core idea is that you're not touching any of the Python files like you normally would as a researcher. Instead, you are programming the program.md Markdown files that provide context to the AI agents and set up your autonomous research org. The default program.md in this repo is intentionally kept as a bare bones baseline, though it's obvious how one would iterate on it over time to find the "research org code" that achieves the fastest research progress, how you'd add more agents to the mix, etc. A bit more context on this project is here in this tweet and this tweet.

 How it works

The repo is deliberately kept small and only really has three files that matter:

- prepare.py — fixed constants, one-time data prep (downloads training data, trains a BPE tokenizer), and runtime utilities (dataloader, evaluation). Not modified.
- train.py — the single file the agent edits. Contains the full GPT model, optimizer (Muon + AdamW), and training loop. Everything is fair game: architecture, hyperparameters, optimizer, batch size, etc. This file is edited and iterated on by the agent.
- program.md — baseline instructions for one agent. Point your agent here and let it go. This file is edited and iterated on by the human.

By design, training runs for a fixed 5-minute time budget (wall clock, excluding startup/compilation), regardless of the details of your compute. The metric is val_bpb (validation bits per byte) — lower is better, and vocab-size-independent so architectural changes are fairly compared.

If you are new to neural networks, this "Dummy's Guide" looks pretty good for a lot more context.

 Quick start

Requirements: A single NVIDIA GPU (tested on H100), Python 3.10+, uv.

bash

 1. Install uv project manager (if you don't already have it)
curl -LsSf https://astral.sh/uv/install.sh | sh

 2. Install dependencies
uv sync

 3. Download data and train tokenizer (one-time, ~2 min)
uv run prepare.py

 4. Manually 

Top issues

Discussions

News & coverage (20)

Hacker NewsAutoresearch: Agents researching on single-GPU nanochat training automatically▲ 208 · 58 comments · 2026-03-07Hacker NewsShow HN: My AI agents bully each other to prevent context drift▲ 9 · 0 comments · 2026-05-09Hacker NewsKarpathy: Autoresearch▲ 4 · 1 comments · 2026-03-10Hacker NewsShow HN: Karpathy's autoresearch applied to debugging – two open-source skills▲ 3 · 1 comments · 2026-03-16Hacker NewsShow HN: AI agents who prevent context drift through gossip▲ 1 · 0 comments · 2026-05-11irblog.prnewswire.comEvoMap Open-Sources AutoResearch, Giving AI Agents a Way to Test Their Own Research Ideas2026-09-01marktechpostGuide to Loop Engineering: How ‘autoresearch’ and ‘Bilevel Autoresearch’ Turn AI Agents Into Autonomous Machine Learning ML Research Loops2026-07-12techtimesKarpathy’s Autoresearch Loop Is Spreading Fast: Shopify’s 53% Speed Claim Still Unmerged, Flagged as Overfit2026-05-19HackerI Let Karpathy's AutoResearch Agent Run Overnight!2026-04-11Geeky GadgetsWhy Self-Improving AI Is the Next Big Shift in Software Development2026-04-09
10 moreGeeky GadgetsCut Manual AI Training Time With the Karpathy AutoResearch Framework2026-03-30NextBigFutureAndrej Karpathy on Code Agents, AutoResearch and the Self Improvement Loopy Era of AI2026-03-20Yahoo FinanceYounet.ai Highlights Researgency.ai Collaboration with Kala Bio (KALA) as New "AutoResearch" Milestone Signals the Rise of Autonomous Research Agents2026-03-11blockchainAutoresearch by Andrej Karpathy: Latest Agentic Research Workflow Guide and 5 Business Use Cases2026-03-09mlqAndrej Karpathy Releases Open-Source AutoResearch Tool for Autonomous AI Experiments2026-03-09xHow to deploy autoresearch:2026-03-08VentureBeatAndrej Karpathy's new open source 'autoresearch' lets you run hundreds of AI experiments a night — with revolutionary implications2026-03-08Unisba MediaKarpathy's "autoresearch" Broke The Internet Lamine Yamal (p1CF8BOssT)2026-11-07Kingy AIKarpathy Autoresearch Guide: Setup, Loop & val_bpb2026-03-09Quantum ZeitgeistAndrej Karpathy AI’s Iterative Self-Improvement Of Code2026-03-08

Snapshot: 2026-11-16T09:23:14Z · Data license Apache-2.0 · JSON