Neywa-Labs /mlx-vlm

MLX-VLM is a package for inference and fine-tuning of Vision Language Models (VLMs) on your Mac using MLX.

★ 5,848⑂ 893👁 35⑃ 1,750● 929▤ 93
PythonMITCreated 2024-04-16neywa-labs.github.io/mlx-vlm/
Download: v0.7.12v0.7.12 ↓0.7.122026-11-04
+2
Daily
5,848
stars
893
forks
35
watchers
1,750
prs
929
issues
93
releases

README

MLX-VLM




  Run and fine-tune multimodal models on Apple Silicon. Text, images, video, and audio in — any out. Built on MLX.



  Installation ·
  Documentation ·
  Releases



  macOS · Apple silicon



  Built by Neywa Labs, the local AI company behind Nativ.



  
  
  
  


 Table of Contents
- Installation
- Usage
  - Command Line Interface (CLI)
    - Thinking Budget
  - Speculative Decoding
    - DFlash, DFlash2, and DSpark
    - Bundled native heads
    - Gemma 4 MTP
    - Gemma 4 EAGLE-3
    - MiniMax M3 EAGLE-3
  - Chat UI with Gradio
  - Python Script
  - Server (FastAPI)
    - Continuous Batching
    - Automatic Prefix Caching (APC)
    - KV Cache Quantization
- 1-bit Affine Inference
- Activation Quantization (CUDA)
- Multi-Image Chat Support
  - Supported Models
  - Usage Examples
- Model-Specific Documentation
- Vision Feature Caching
- TurboQuant KV Cache
- Distributed Inference
- Fine-tuning

 Model-Specific Documentation

Some models have detailed documentation with prompt formats, examples, and best practices:

| Model | Documentation |
|-------|---------------|
| LFM3 / LFM3-VL | Docs |
| LFM3-Audio | Docs |
| LFM3-Omni | Docs |
| DeepSeek-OCR | Docs |
| DeepSeek-OCR-2 | Docs |
| Unlimited-OCR | Docs |
| DOTS-OCR | Docs |
| DOTS-MOCR | Docs |
| ERNIE 5.5 Flash Base (text) | Docs |
| ERNIE 4.5 VL | Docs |
| GLM-OCR | Docs |
| Phi-4 Reasoning Vision | Docs |
| MiniCPM-o | Docs |
| PaddleOCR-VL | Docs |
| OXR | Docs |
| Phi-4 Multimodal | Docs |
| MolmoPoint | Docs |
| LocateAnything | Docs |
| Moondream2 | Docs |
| Moondream3 | Docs |
| EmbeddingGemma 2 | Docs |
| Gemma 4 | Docs |
| Muse Spark Open | Docs |
| MiniMax M3 / M3.1 Flash | Docs |
| MiniMax-H3-Image | Docs |
| Ling 3.1 Flash | Docs |
| Falcon-OCR | Docs |
| IndicOCR | Docs |
| PP-DocLayoutV3 | Docs |
| Granite Vision 3.2 | Docs |
| Granite Vision (4.0, 4.1) | Docs |
| MiniCPM-OCR 1.0 | Docs |
| MiniCPM-OCR Jev | Docs |
| MiniCPM-V 4.7 | Docs |
| MiniCPM-V 4.6 | Docs |
| GLiNER2.5 | Docs |
| BERT | Docs |
| LLaVA-OneVision | Docs |
| K2-Horizon | Docs |
| Z1T-0 | Docs |
| Spark-X2.5 | Docs |
| Ternary Bonsai 2 | Docs |
| Flint | Docs |
| Ripple | Docs |
| Nemotron 3.5 Super VL / Bolt / Nano Omni | Docs |
| Idol (image to 3D Gaussian splats) | Docs |

 Installation

The easiest way to get started is to install the mlx-vlm package using pip:

sh
pip install -U mlx-vlm


The Gradio chat UI needs an extra dependency that is not
part of the base install:

sh
pip install -U 'mlx-vlm[ui]'


Quote the package name so that shells which expand square brackets, such as
zsh, do not treat [ui] as a glob pattern.

 Agent Skills

This repo ships an agent-skills bundle under skills/ for common MLX-VLM workflows — usage, conversion, development, and support. Skills load into a coding agent (Claude Code, Codex, Gemini) so it follows the right project conventions instead of guessing.

| Skill | Description |
|-------|-------------|
| cli-inference | Run and debug c

Top issues

Discussions

Latest release

v0.7.12 v0.7.12 2026-11-04
What's Changed
 Lower MOSS-Image peak memory during VAE decode by @felix-ab in https://github.com/Neywa-Labs/mlx-vlm/pull/2615
 Fix LFM3 thinking leakage by initializing response parsers with the prompt by @Blaizzy in https://github.com/Neywa-Labs/mlx-vlm/pull/2621
 Add Raon-1 Mini support by @Blaizzy in https://github.com/Neywa-Labs/mlx-vlm/pull/2622
 Support text-only ASR with diarized_json by @Blaizzy in https://github.com/Neywa-Labs/mlx-vlm/pull/2626
 Send SSE heartbeats during idle OpenAI streams by @Blaizzy in https://github.com/Neywa-Labs/mlx-vlm/pull/2633
 Add native Mage-VL-Cowo

Snapshot: 2026-11-16T09:23:14Z · Data license Apache-2.0 · JSON