Neywa-Labs /mlx-vlm
MLX-VLM is a package for inference and fine-tuning of Vision Language Models (VLMs) on your Mac using MLX.
★ 5,848⑂ 893👁 35⑃ 1,750● 929▤ 93
+2
Daily
5,848
stars
893
forks
35
watchers
1,750
prs
929
issues
93
releases
README
MLX-VLM
Run and fine-tune multimodal models on Apple Silicon. Text, images, video, and audio in — any out. Built on MLX.
Installation ·
Documentation ·
Releases
macOS · Apple silicon
Built by Neywa Labs, the local AI company behind Nativ.
Table of Contents
- Installation
- Usage
- Command Line Interface (CLI)
- Thinking Budget
- Speculative Decoding
- DFlash, DFlash2, and DSpark
- Bundled native heads
- Gemma 4 MTP
- Gemma 4 EAGLE-3
- MiniMax M3 EAGLE-3
- Chat UI with Gradio
- Python Script
- Server (FastAPI)
- Continuous Batching
- Automatic Prefix Caching (APC)
- KV Cache Quantization
- 1-bit Affine Inference
- Activation Quantization (CUDA)
- Multi-Image Chat Support
- Supported Models
- Usage Examples
- Model-Specific Documentation
- Vision Feature Caching
- TurboQuant KV Cache
- Distributed Inference
- Fine-tuning
Model-Specific Documentation
Some models have detailed documentation with prompt formats, examples, and best practices:
| Model | Documentation |
|-------|---------------|
| LFM3 / LFM3-VL | Docs |
| LFM3-Audio | Docs |
| LFM3-Omni | Docs |
| DeepSeek-OCR | Docs |
| DeepSeek-OCR-2 | Docs |
| Unlimited-OCR | Docs |
| DOTS-OCR | Docs |
| DOTS-MOCR | Docs |
| ERNIE 5.5 Flash Base (text) | Docs |
| ERNIE 4.5 VL | Docs |
| GLM-OCR | Docs |
| Phi-4 Reasoning Vision | Docs |
| MiniCPM-o | Docs |
| PaddleOCR-VL | Docs |
| OXR | Docs |
| Phi-4 Multimodal | Docs |
| MolmoPoint | Docs |
| LocateAnything | Docs |
| Moondream2 | Docs |
| Moondream3 | Docs |
| EmbeddingGemma 2 | Docs |
| Gemma 4 | Docs |
| Muse Spark Open | Docs |
| MiniMax M3 / M3.1 Flash | Docs |
| MiniMax-H3-Image | Docs |
| Ling 3.1 Flash | Docs |
| Falcon-OCR | Docs |
| IndicOCR | Docs |
| PP-DocLayoutV3 | Docs |
| Granite Vision 3.2 | Docs |
| Granite Vision (4.0, 4.1) | Docs |
| MiniCPM-OCR 1.0 | Docs |
| MiniCPM-OCR Jev | Docs |
| MiniCPM-V 4.7 | Docs |
| MiniCPM-V 4.6 | Docs |
| GLiNER2.5 | Docs |
| BERT | Docs |
| LLaVA-OneVision | Docs |
| K2-Horizon | Docs |
| Z1T-0 | Docs |
| Spark-X2.5 | Docs |
| Ternary Bonsai 2 | Docs |
| Flint | Docs |
| Ripple | Docs |
| Nemotron 3.5 Super VL / Bolt / Nano Omni | Docs |
| Idol (image to 3D Gaussian splats) | Docs |
Installation
The easiest way to get started is to install the mlx-vlm package using pip:
sh
pip install -U mlx-vlm
The Gradio chat UI needs an extra dependency that is not
part of the base install:
sh
pip install -U 'mlx-vlm[ui]'
Quote the package name so that shells which expand square brackets, such as
zsh, do not treat [ui] as a glob pattern.
Agent Skills
This repo ships an agent-skills bundle under skills/ for common MLX-VLM workflows — usage, conversion, development, and support. Skills load into a coding agent (Claude Code, Codex, Gemini) so it follows the right project conventions instead of guessing.
| Skill | Description |
|-------|-------------|
| cli-inference | Run and debug cTop issues
#39Models to port to MLX-VLM74 comments · 2024-06-11#999Server clears Metal cache after every request, destroying KV prefix cache and forcing full re-prefill on every turn7 comments · 2026-04-09#286Does MLX-VLM support fine-tuning of vision tower layers?7 comments · 2025-04-04#1781Server: add backend-aware hotword and custom vocabulary support for audio transcriptions6 comments · 2026-08-03#2254glm5_next: remaining KDA decode kernel-launch overhead after #2127 (spec + measurements for a fused/compiled step)5 comments · 2026-09-14#1925Qwen3.8-MLX vision NOT work with base64 data-URI5 comments · 2026-08-16#1629Redesign APC around pluggable cache components for custom attention and hybrid models5 comments · 2026-07-18#1562APC + kv-bits: staggered continuous-batching join crashes (last-layer type mismatch)5 comments · 2026-07-10
Discussions
#2456AREX-2 (Qwen3.8-27B fine-tune) converted with 0.7.4: works well at 5/6/8-bit, plain 4-bit loops at temperature 01 comments · 2026-10-07#2165SmolVLA on Apple Silicon: an MLX runtime using mlx-vlm’s SmolVLM as an architecture reference1 comments · 2026-09-05#2026EVIE-Preview-4.5B on Apple Silicon: MLX Q4 visual retrieval0 comments · 2026-08-26#1364MCP to help OSS devs understand Blaizzy/mlx-vlm0 comments · 2026-06-12#1290KVarN: 3-5x more context, throughput above FP16, and FP16-level accuracy. from Huawei is used to back their Qwen workload. would be nice to add here0 comments · 2026-06-04#1258v0.6.01 comments · 2026-06-01#1025Does mlx vlm respect to transformers generation config when generating response?0 comments · 2026-04-15#932Memory issues when fine-tuning Qwen2-VL with LoRA on 16GB M1 Max - any optimization tips?2 comments · 2026-04-05
Latest release
v0.7.12 v0.7.12 2026-11-04What's Changed Lower MOSS-Image peak memory during VAE decode by @felix-ab in https://github.com/Neywa-Labs/mlx-vlm/pull/2615 Fix LFM3 thinking leakage by initializing response parsers with the prompt by @Blaizzy in https://github.com/Neywa-Labs/mlx-vlm/pull/2621 Add Raon-1 Mini support by @Blaizzy in https://github.com/Neywa-Labs/mlx-vlm/pull/2622 Support text-only ASR with diarized_json by @Blaizzy in https://github.com/Neywa-Labs/mlx-vlm/pull/2626 Send SSE heartbeats during idle OpenAI streams by @Blaizzy in https://github.com/Neywa-Labs/mlx-vlm/pull/2633 Add native Mage-VL-Cowo