MakazhanAlpamys /Soup
Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.
★ 10,083⑂ 1,665👁 59▤ 183
+0
Daily
+0
Weekly
+1,470
Monthly
10,083
stars
1,665
forks
59
watchers
–
prs
–
issues
183
releases
README
🌍 English | Türkçe | العربية | 日本語 | 中文
Soup
Fine-tune and post-train LLMs in one command. No SSH, no config hell.
Website ·
Quick Start ·
Web UI ·
Config ·
Docs ·
Commands ·
Models ·
Discord ·
Telegram ·
Product Hunt
---
Soup turns the pain of LLM fine-tuning into a simple workflow. One config, one command, done.
bash
pip install "soup-cli[train]" add [train] to fine-tune; bare soup-cli is the light CLI
soup init --template chat
soup train
Fine-tune an 8B model on a 4 GB laptop GPU. Layer streaming keeps the frozen base out of
VRAM and feeds it to the GPU one decoder layer at a time. Measured on an RTX 3050 Laptop 4 GB:
Llama-3.1-8B-Instruct + NF4 at 119.6 tok/s at 952 MHz, 3.32 GB peak (pre-repair) — bit-exact
against a normal resident run, and reproduced independently on an H100 at 113.00 tok/s in the same
3.32 GB. (Both figures were measured on v0.72.2, before the v0.73.0 correctness repair that cost
−4.8% at 32B. The post-repair run, on another RTX 3050 Laptop 4 GB (@umran666, 1103), reads
208.6 tok/s at 1935–1957 MHz in a 2.40 GB peak. That is 59% of its same-session GEMM ceiling
against 68%: about 14% slower per unit clock, so the raw 1.74x gain is the clock, not a speedup.
Large-layer streaming also changed the memory pattern between the two rows, so even the fraction
is not like-for-like — the record.) Opt-in (stream_layers: true)
and still BETA —
how it works ·
all measurements · paper ·
check it yourself on a free Colab T4 (caps the process to
4 GB, then asserts a streamed model is bit-identical to a normal one)
Llama-3.1-8B-Instruct + NF4, LoRA, batch 1, seq 512 on an RTX 3050 Laptop 4 GB — 3.32 GB peak, 119.6 tok/s at 952 MHz (pre-repair, v0.72.2; post-331: 208.6 tok/s at 1935 MHz in 2.40 GB, not directly comparable, @umran666, 1103). Full video (90s)
Why Soup?
Training LLMs is still painful. Even experienced teams spend 30-50% of their time fighting
infrastructure instead of improving models. Soup fixes that.
- Zero SSH. Never SSH into a broken GPU box again.
- One config. A simple YAML file is all you need.
- Auto everything. Batch size, GPU detection, quantization — handled.
- Works locally. Train on your own GPU with QLoRA. No cloud required.
What's New
v0.77.2 — a cut of main after v0.77.1: 159 pull requests, 153 of them from 30 people outside
the maintainer. Most of it is fixes. It also adds features and two rules that can refuse what
v0.77.1 accepted, so read the
v0.77.2 release notes before
upgrading a server or a task: sft config. Coming from 0.76.x, run soup doctor --config soup.yaml
first and read the v0.77.0 release notes.
- soup profile --model answers "will it fit?" without a soup.yaml. It prices the run
from the model's config.json, shaped by --seq, --batch, --method, --lora-r,
--optimizer and --gradient-checkpointing;