A hierarchical inference engine for frontier MoE models: hot experts on the GPU (CUDA, Vulkan), warm ones in RAM, cold ones streamed from NVMe. Written in C, runs on the hardware you already own.

★ 51,314⑂ 6,198👁 405⑃ 2,109● 651▤ 21
CApache-2.0Created 2026-07-01justvugg.github.io/colibri
Download: colibri v2.1.0colibri-2.1.0 ↓2.1.02026-10-31
+107
Daily
+2,116
Weekly
+9,840
Monthly
51,314
stars
6,198
forks
405
watchers
2,109
prs
651
issues
21
releases

README

Website ·
  Discord ·
  English · 简体中文 · 繁體中文 · Italiano · 日本語 · Bahasa Indonesia · Português (Brasil)


Tiny engine, immense model. colibri runs very large open models on the
machine you already have. A mixture-of-experts model of hundreds of billions of
parameters uses only a small part of itself for each token, so colibri keeps
that part in RAM and reads the rest, the experts, from the disk when the model
asks for them. Pure C, one file per model family, no GPU required.

Thirteen engines run today. Ten are for language models: GLM-5.2/5.3 (which
also runs Mistral-Large-4), GLM-5.3-Flash, Inkling, Kimi K3,
DeepSeek V4 Flash, DeepSeek V4.1 Flash, MiMo-V2.6 Flash (and Pro),
Qwen3.8-Flash-Next, Qwen3.6 (which also runs Qwen3-Coder, the dense
Qwen3.8-27B, Flint, Muse Spark Open and LFM3-VL) and OLMoE. One
draws pictures: Qwen-Image-2.1. Two answer decisions: Laya and
GLiNER2.5-Decide, with a third decision model, Clef, on the Qwen3.6
engine. The language models also run from the GGUF files already published,
read as they are (GGUF), on the CPU, on any GPU through Vulkan
and on NVIDIA cards through CUDA. Which one for my machine


$ ./coli chat
  colibri v2.1.0 · GLM-5.2 · 744B MoE · int4 · streaming CPU
  ✓ ready in 32s · resident 9.9 GB
  › ciao!
  ◆ Ciao! Come posso aiutarti oggi?



  
  
  colibri is free and open source. If it has earned a place on your machine, a coffee helps pay for the next engine and the next speedup.





 Get started in one step

You need a computer with 8 GB of RAM at the very least (16 GB or more is
better), 22 GB free on the disk for the smallest model, and an internet
connection. A graphics card is optional.

Windows

1. On this page click Code, then Download ZIP, and unzip it.
2. Double-click START-HERE.bat in the unzipped folder. If Python is
   missing, it offers to install it for you.

Linux (Ubuntu and Debian; other distributions have the same packages under
their own names)

bash
sudo apt install git python3 build-essential
git clone https://github.com/JustVugg/colibri
cd colibri
./start-here.sh


macOS (with Homebrew)

bash
xcode-select --install
brew install libomp git python
git clone https://github.com/JustVugg/colibri
cd colibri
./start-here.sh


You answer one question, which model, and Enter takes the recommendation. Then
the setup:

1. looks at your machine: RAM, free disk, CPU and GPUs;
2. recommends a model that fits: the part of the model that always stays in
   RAM, plus a minimum cache of experts, must fit in your RAM, and the download
   on your disk;
3. gets the engine: it builds it for your machine when a compiler is there,
   otherwise it downloads the prebuilt one, which runs on the CPU and, on Linux
   and Windows, on a Vulkan GPU too. It builds
   for your GPU when that pays: CUDA for an NVIDIA card on Linux when the CUDA toolkit is installed,
   otherwise Vulkan. On a discrete GPU it always does; on an integrated GPU,
   which shares the CPU's RAM, only for the models measured faster there

Top issues

Discussions

Latest release

colibri v2.1.0 v2.1.0 2026-10-31
Huge open models on the machine you already have: on the CPU, on any GPU through Vulkan, on NVIDIA cards through CUDA. Pure C, one file per model family.

387 pull requests since v2.0.0, 208 of them from 35 contributors. GGUF files now run natively on every engine, read exactly as published. Four new models arrive, Mistral-Large-4 (1T) among them. CUDA gets device chains of its own and a routed-expert tier for every engine. Qwen3.8-Flash-Next runs a 256k context at the speed of 64k, an agent's turns read only what is new, and there is a new chat UI.


  


 Highlights

- GGUF on every engine. 

Snapshot: 2026-11-16T09:23:14Z · Data license Apache-2.0 · JSON