7,403
stars
482
forks
35
watchers
115
prs
87
issues
25
releases
README
TurboFieldfare Gemma 4 26B-A4B inference in about 2 GB of RAM A custom Swift + Metal runtime for any Apple Silicon Mac, even the 8 GB ones. Quick start · Local server · Benchmarks · Contribute results · How it works · Experiments · References !TurboFieldfare Mac app with local chat history, LaTeX rendering, and run statistics Memory got expensive. So I gave a 26-billion-parameter model a ~2 GB budget. TurboFieldfare runs the instruction-tuned Gemma 4 26B-A4B without loading the entire 14.3 GB model into memory. It keeps the shared 1.35 GB core and FP16 KV cache in memory, then streams only the experts needed for each token from SSD. This is what lets the model run on Macs with 8 GB of RAM. The runtime, streaming installer, CLI, and native Mac app are written in Swift and Metal. TurboFieldfare is model-specific rather than a wrapper around MLX or llama.cpp. The curated experiment record summarizes 103 measured results across kernels, caching, I/O, prefill, and decode. Try it bash git clone https://github.com/drumih/turbo-fieldfare.git cd turbo-fieldfare swift build -c release .build/release/TurboFieldfareMac On the first run, Swift Package Manager downloads and builds the Swift packages required by the tokenizer. The complete release build includes the foreground Mac app and its sibling decode-service executable. When the app opens, choose Download Model and let TurboFieldfare fetch and repack the pinned model (about 15 GB). Once it is ready, choose Load Model, type your prompt, and press Generate. At a glance | Metric | Value | | --------------- | ------------------------------------------------------------------------------------------------------------------------ | | Model | Gemma 4 26B-A4B IT, 26B total parameters, about 3.88B active per token | | Weights | MLX affine 4-bit, group 64; 8-bit router; 4-bit shared and routed experts | | Memory | ~2 GB of weights and 4K KV cache | | Storage | About 14.3 GB for the text model, plus about 1.1 GB for the optional image pack | | Hardware | Apple Silicon Mac; 8 GB of RAM | | Platform | macOS 26, Metal 4, Swift 6.2 | | M2 measured decode | 5.5–6.8 tok/s on an 8 GB M2 MacBook Air, expert prefetch on | | M5 measured decode | 35.3–39.2 tok/s on a 24 GB M5 Pro, expert prefetch off | | Community Reports | Here | The measured result is a reference point, not a performance ceiling. Prom
Top issues
#18[Feature]: Claude code integration7 comments · 2026-07-29#44[Feature]: implement other MOE family models5 comments · 2026-08-01#73[Bug]: Unable to trigger prompt caching in OpenAI-compatible local server4 comments · 2026-08-03#84[Bug]: tool-call parse fails with GemmaToolCallParserError.malformed after a long generation3 comments · 2026-08-04#121[Bug]: module 'SwiftUIMacros' not found2 comments · 2026-08-11#54[Feature]: Different model for higher ram models2 comments · 2026-08-02#48[Feature]: Link a community Homebrew tap for the CLI tools and Mac app2 comments · 2026-08-01#47[Bug]: the dictation button in the prompt field does not function2 comments · 2026-08-01
Latest release
0.10.0 0.10.0 2026-11-14Speculative expert prefetch Expert prefetch can speed up generation on devices with less RAM. - Enable Expert prefetch in the Mac app, then reload the model. The setting is saved. - CLI and server users can enable it with --early-expert-read router. - Improved reader cleanup during cancellation, errors, and model reuse. - Updated benchmarks and experiment documentation with M2 and M5 Pro results. Thanks to @maxslamdunk for the expert-prefetch implementation in 187.