Device-native foundation models: 145B on a laptop, 35B on a phone. Latent MoE, SSD-streamed weights, a router that predicts loads a step ahead. Mac, iPhone, Android, Windows.

★ 11,746⑂ 1,856👁 850⑃ 110● 47▤ 3
C++Apache-2.0Created 2026-09-08edge0.ai/
Download: 0.2.00.2.0 ↓0.2.02026-11-11
+95
Daily
+2,570
Weekly
11,746
stars
1,856
forks
850
watchers
110
prs
47
issues
3
releases

README

Edge0

An open-source streaming MoE inference framework



Think big. Run here.Sparse. By design.



macOS · iOS · Android · Linux · WindowsOne program for every Edge0 model

Project

    


Latest release

   


Old release

   


English · 中文 · 日本語 · Español · Français

News · About · Models · Quality · Benchmark · Getting started



 News








More updates






Full release notes

- [2026-11-13] We released edge0-145b and edge0-120b, latent mixture-of-experts models derived from Qwen3.8-Flash-Next for a single NVIDIA GPU on Linux or Windows. They run on edge0-strata, which edge0-infer serve and edge0-infer chat start for them; see the model page. Weights on Hugging Face: Edge0-145B-q3k · Edge0-145B-NVFP4 · Edge0-120B-q3k · Edge0-120B-NVFP4.
- [2026-11-04] We released the unified inference engine edge0-infer: built on GGML, one engine and one native C++ API run the same model on macOS, iOS, Android and Windows, with a CLI, an OpenAI-compatible local server, and chat app examples for desktop, Android and iOS. The first release supports edge0-35b; edge0-8b follows in a later release. It replaces the Python framework and the per-platform engines; the previous code is kept at tag pre-edge0-infer.
- [2026-09-30] We released the edge0 inference engines for four platforms — iOS, macOS, Android and Windows (source at tag pre-edge0-infer).
- [2026-09-16] Our technical report is on arXiv: The Other Half of the Memory Wall: Serving 35B MoEs from SSD with Trained Routing Prediction.
- [2026-09-08] Initial open-source release of edge0, together with both model tiers — Edge0-35B-A3B-preview and Edge0-8B-A1B-preview — on Hugging Face and ModelScope.






Previous release

→ 35B / 8B release guide — model downloads, architecture, benchmarks, supported platforms and setup instructions.



 About

Edge0 is an open-source streaming MoE inference framework for running large sparse models on consumer hardware.

Use it through the command line, a local API server, or the Edge0 Chat app.

The latest release brings 145B and 120B latent-MoE models to a single NVIDIA GPU or an Apple Silicon Mac.

Trained natively at 4 bits, both models are available in Q3_K and NVFP4 for local agent workloads.

edge0-infer provides shared serve and chat commands and selects the engine from the model folder. The latest models run on edge0-strata, a fork of Strata; the previous 35B release runs on the GGML-based native engine.

 Platforms

Support depends on the model and its engine.

| Platform | edge0-145b, edge0-120b (edge0-strata) | edge0-35b (native engine) |
| --- | --- | --- |
| Linux | NVIDIA GPU (CUDA 13) | CPU |
| Windows | NVIDIA GPU (CUDA 13) | CPU, Vulkan |
| macOS (Apple Silicon) | Metal | Metal, CPU |
| iOS | — | Metal |
| Android | — | CPU |

Use the 145B / 120B setup for the latest release. The previous-release guide covers native backends, platform-specific builds and chat examples. The 8B checkpoint is available; native-engine support is planned.

 Models

145B / 1

Top issues

Discussions

Latest release

0.2.0 v0.2.0 2026-11-11
Windows and macOS. The Windows packages are built from 3203c39 on main, the macOS package from f833336 on main. Installation and troubleshooting: see README.md inside each package.

 What's Changed

- Edge0 145B and Edge0 120B. Two latent-MoE models for a single NVIDIA GPU, run by edge0-strata.
- New inference engine, edge0-infer. A single on-device MoE engine in C++ on GGML, with one native API, replaces the Python framework and the four separate platform apps. It includes a CLI (run, chat, bench, serve, info), an OpenAI-compatible local server and the Edge0 Chat desktop app. This release sup

Snapshot: 2026-11-16T09:23:14Z · Data license Apache-2.0 · JSON