A hierarchical inference engine for frontier MoE models: hot experts on the GPU (CUDA, Vulkan), warm ones in RAM, cold ones streamed from NVMe. Written in C, runs on the hardware you already own.
★ 51,314⑂ 6,198👁 405⑃ 2,109● 651▤ 21
+107
Daily
+2,116
Weekly
+9,840
Monthly
51,314
stars
6,198
forks
405
watchers
2,109
prs
651
issues
21
releases
README
Website · Discord · English · 简体中文 · 繁體中文 · Italiano · 日本語 · Bahasa Indonesia · Português (Brasil) Tiny engine, immense model. colibri runs very large open models on the machine you already have. A mixture-of-experts model of hundreds of billions of parameters uses only a small part of itself for each token, so colibri keeps that part in RAM and reads the rest, the experts, from the disk when the model asks for them. Pure C, one file per model family, no GPU required. Thirteen engines run today. Ten are for language models: GLM-5.2/5.3 (which also runs Mistral-Large-4), GLM-5.3-Flash, Inkling, Kimi K3, DeepSeek V4 Flash, DeepSeek V4.1 Flash, MiMo-V2.6 Flash (and Pro), Qwen3.8-Flash-Next, Qwen3.6 (which also runs Qwen3-Coder, the dense Qwen3.8-27B, Flint, Muse Spark Open and LFM3-VL) and OLMoE. One draws pictures: Qwen-Image-2.1. Two answer decisions: Laya and GLiNER2.5-Decide, with a third decision model, Clef, on the Qwen3.6 engine. The language models also run from the GGUF files already published, read as they are (GGUF), on the CPU, on any GPU through Vulkan and on NVIDIA cards through CUDA. Which one for my machine $ ./coli chat colibri v2.1.0 · GLM-5.2 · 744B MoE · int4 · streaming CPU ✓ ready in 32s · resident 9.9 GB › ciao! ◆ Ciao! Come posso aiutarti oggi? colibri is free and open source. If it has earned a place on your machine, a coffee helps pay for the next engine and the next speedup. Get started in one step You need a computer with 8 GB of RAM at the very least (16 GB or more is better), 22 GB free on the disk for the smallest model, and an internet connection. A graphics card is optional. Windows 1. On this page click Code, then Download ZIP, and unzip it. 2. Double-click START-HERE.bat in the unzipped folder. If Python is missing, it offers to install it for you. Linux (Ubuntu and Debian; other distributions have the same packages under their own names) bash sudo apt install git python3 build-essential git clone https://github.com/JustVugg/colibri cd colibri ./start-here.sh macOS (with Homebrew) bash xcode-select --install brew install libomp git python git clone https://github.com/JustVugg/colibri cd colibri ./start-here.sh You answer one question, which model, and Enter takes the recommendation. Then the setup: 1. looks at your machine: RAM, free disk, CPU and GPUs; 2. recommends a model that fits: the part of the model that always stays in RAM, plus a minimum cache of experts, must fit in your RAM, and the download on your disk; 3. gets the engine: it builds it for your machine when a compiler is there, otherwise it downloads the prebuilt one, which runs on the CPU and, on Linux and Windows, on a Vulkan GPU too. It builds for your GPU when that pays: CUDA for an NVIDIA card on Linux when the CUDA toolkit is installed, otherwise Vulkan. On a discrete GPU it always does; on an integrated GPU, which shares the CPU's RAM, only for the models measured faster there
Top issues
#1405[Feature]: Provide the necessary Windows coli_cuda.dll files or update instructions to build them20 comments · 2026-09-09#594[Feature]: Lossless entropy compression for streamed expert traffic (transport layer, any container)18 comments · 2026-07-24#1852[Feature]: Deb or pkg16 comments · 2026-10-03#2757[HIP][Qwen3.8] Lower non-MTP decode throughput on pinned dev vs main on Radeon AI PRO R970012 comments · 2026-11-11#708[Experiment] expert-transition-history placement policy vs gate-momentum — controlled A/B for hypothesis #112 comments · 2026-07-30#441[Performance]: PILOT prefetch runs at NVMe queue-depth 1, and DIRECT=1 re-reads WILLNEED-prefetched pages12 comments · 2026-07-19#2583[Bug]: qwen36 Vulkan degradation when all experts are resident.11 comments · 2026-11-04#689[Bug]: deep speculative verify batches are not token-exact vs CPU on CUDA (near-tie flips + lower draft acceptance)11 comments · 2026-07-29
Discussions
#2830How to resume a chat after interruption ?0 comments · 2026-11-14#2807How to control number of CPU cores used?1 comments · 2026-11-13#2806Interesting answer0 comments · 2026-11-13#2795Kimi k3.1 pro full0 comments · 2026-11-12#2776🏆 colibri earned Repocharts Silver: Top gainer in C0 comments · 2026-11-12#2649AVX2 Not detected on a CPU with AVX2 (E5-2697A V4 Xeon)1 comments · 2026-11-06#2644Checking the internet for information.1 comments · 2026-11-06#2553PS5 port?0 comments · 2026-11-02
Latest release
colibri v2.1.0 v2.1.0 2026-10-31Huge open models on the machine you already have: on the CPU, on any GPU through Vulkan, on NVIDIA cards through CUDA. Pure C, one file per model family. 387 pull requests since v2.0.0, 208 of them from 35 contributors. GGUF files now run natively on every engine, read exactly as published. Four new models arrive, Mistral-Large-4 (1T) among them. CUDA gets device chains of its own and a routed-expert tier for every engine. Qwen3.8-Flash-Next runs a 256k context at the speed of 64k, an agent's turns read only what is new, and there is a new chat UI. Highlights - GGUF on every engine.
相关报道 (4)
News & coverage (9)
Hacker NewsShow HN: Getting GLM 5.2 running on my slow computer▲ 937 · 240 comments · 2026-07-09Hacker NewsBrio mode in Colibri: Scoring a closed set instead of generating▲ 1 · 1 comments · 2026-09-21Hacker NewsCall for probes: building the GLM-5.2 Expert Atlas▲ 1 · 0 comments · 2026-08-11xJustVugg on X: "colibri is #1 on @github today! What a milestone. Thank you to every developer, contributor and user who helped us get here. This is your achievement too."2026-09-15alphamatch.aiColibri: The Revolutionary AI Engine Running 744B-Parameter Models on Just 25GB RAM2026-07-10GIGAZINEA new inference engine called 'Colibrì' has emerged that can run the massive AI 'GLM-5.2,' with 744 billion parameters, on a regular PC with 25GB of memory.2026-07-10TechTargetWhat repos are trending on GitHub?2026-09-21Pasquale PillitteriColibrì: GLM-5.2, a 744-Billion-Parameter Model on 25 GB of RAM2026-07-12Tom's HardwareColibrì proof-of-concept gains frontier-level 1.5-TB AI model — novel approach runs on only 25GB of RAM and shows promise for local AI setups2026-07-11