Device-native foundation models: 145B on a laptop, 35B on a phone. Latent MoE, SSD-streamed weights, a router that predicts loads a step ahead. Mac, iPhone, Android, Windows.
★ 11,746⑂ 1,856👁 850⑃ 110● 47▤ 3
+95
Daily
+2,570
Weekly
11,746
stars
1,856
forks
850
watchers
110
prs
47
issues
3
releases
README
Edge0
An open-source streaming MoE inference framework
Think big. Run here.Sparse. By design.
macOS · iOS · Android · Linux · WindowsOne program for every Edge0 model
Project
Latest release
Old release
English · 中文 · 日本語 · Español · Français
News · About · Models · Quality · Benchmark · Getting started
News
More updates
Full release notes
- [2026-11-13] We released edge0-145b and edge0-120b, latent mixture-of-experts models derived from Qwen3.8-Flash-Next for a single NVIDIA GPU on Linux or Windows. They run on edge0-strata, which edge0-infer serve and edge0-infer chat start for them; see the model page. Weights on Hugging Face: Edge0-145B-q3k · Edge0-145B-NVFP4 · Edge0-120B-q3k · Edge0-120B-NVFP4.
- [2026-11-04] We released the unified inference engine edge0-infer: built on GGML, one engine and one native C++ API run the same model on macOS, iOS, Android and Windows, with a CLI, an OpenAI-compatible local server, and chat app examples for desktop, Android and iOS. The first release supports edge0-35b; edge0-8b follows in a later release. It replaces the Python framework and the per-platform engines; the previous code is kept at tag pre-edge0-infer.
- [2026-09-30] We released the edge0 inference engines for four platforms — iOS, macOS, Android and Windows (source at tag pre-edge0-infer).
- [2026-09-16] Our technical report is on arXiv: The Other Half of the Memory Wall: Serving 35B MoEs from SSD with Trained Routing Prediction.
- [2026-09-08] Initial open-source release of edge0, together with both model tiers — Edge0-35B-A3B-preview and Edge0-8B-A1B-preview — on Hugging Face and ModelScope.
Previous release
→ 35B / 8B release guide — model downloads, architecture, benchmarks, supported platforms and setup instructions.
About
Edge0 is an open-source streaming MoE inference framework for running large sparse models on consumer hardware.
Use it through the command line, a local API server, or the Edge0 Chat app.
The latest release brings 145B and 120B latent-MoE models to a single NVIDIA GPU or an Apple Silicon Mac.
Trained natively at 4 bits, both models are available in Q3_K and NVFP4 for local agent workloads.
edge0-infer provides shared serve and chat commands and selects the engine from the model folder. The latest models run on edge0-strata, a fork of Strata; the previous 35B release runs on the GGML-based native engine.
Platforms
Support depends on the model and its engine.
| Platform | edge0-145b, edge0-120b (edge0-strata) | edge0-35b (native engine) |
| --- | --- | --- |
| Linux | NVIDIA GPU (CUDA 13) | CPU |
| Windows | NVIDIA GPU (CUDA 13) | CPU, Vulkan |
| macOS (Apple Silicon) | Metal | Metal, CPU |
| iOS | — | Metal |
| Android | — | CPU |
Use the 145B / 120B setup for the latest release. The previous-release guide covers native backends, platform-specific builds and chat examples. The 8B checkpoint is available; native-engine support is planned.
Models
145B / 1Top issues
#201Support for Apple Silicon (Metal) for Edge0 120B / 145B5 comments · 2026-11-13#197Support for AMD GPUs (Vulkan or ROCm/HIP) for Edge0 120B / 145B5 comments · 2026-11-12#205[Engine] Layer split over two CUDA sm_86 cards: stage 2 never gets prompt rows, stall at layer 1 of a small tail chunk4 comments · 2026-11-13#209Docs and reachability feedback from a from-source Windows build (fb0c9b6)2 comments · 2026-11-13#208Performance report: edge0-145b / RTX 4090 24 GB / 64 GB RAM / Windows (from-source fb0c9b6)2 comments · 2026-11-13#107CUDA support timeline2 comments · 2026-09-16#233[Engine] Helper caches and --peer-device give wrong output on latent packs (Edge0-145B NVFP4, sm_86)1 comments · 2026-11-15#217Serving fails with ModuleNotFoundError: No module named 'jinja2.sandbox'1 comments · 2026-11-14
Discussions
#220Edge0-145B Q3_K on an RTX 3090 (24 GB, PCIe Gen 2): community bench at 24 / 16 GiB, plus why 12 GiB doesn't fit0 comments · 2026-11-14#219Edge0 145B Q3_K on Apple M5 Max (MacBook Pro, 128 GB): speed results on Metal0 comments · 2026-11-14#198Withdrawn2 comments · 2026-11-13#195distributed inference in a local cluster2 comments · 2026-11-12#154how to add model with the released Android app?1 comments · 2026-10-28#142📊 Hardware Benchmarks — run edge0 on your device & share results1 comments · 2026-10-15#138Welcome to edge0 Discussions!0 comments · 2026-10-15
Latest release
0.2.0 v0.2.0 2026-11-11Windows and macOS. The Windows packages are built from 3203c39 on main, the macOS package from f833336 on main. Installation and troubleshooting: see README.md inside each package. What's Changed - Edge0 145B and Edge0 120B. Two latent-MoE models for a single NVIDIA GPU, run by edge0-strata. - New inference engine, edge0-infer. A single on-device MoE engine in C++ on GGML, with one native API, replaces the Python framework and the four separate platform apps. It includes a CLI (run, chat, bench, serve, info), an OpenAI-compatible local server and the Edge0 Chat desktop app. This release sup
相关报道 (3)
News & coverage (7)
Hacker NewsA 35B language model running on an iPhone using only 1–2.5 GB of peak memory▲ 2 · 0 comments · 2026-09-10MarkTechPostEdge0 Releases Audio8-TTS-Realistic, a 0.6B Voice Cloning TTS Model2026-10-28noteEdge0-AI/Edge0: A Detailed Explanation of This Week's GitHub Trends!2026-10-04Geeky GadgetsEdge0 Framework Runs Qwen 35B AI Model from Your SSD2026-09-17xEdge0 Runs 35B AI Model on iPhone with Just 2.5 GB Memory2026-09-09huggingfaceEdge0-8b-a1b Preview2026-09-09YouTubeRun 35B Model on Phone Under 3GB Memory with Edge02026-09-11