bytedance /Dolphin
The official repo for “Dolphin: Document Image Parsing via Heterogeneous Anchor Prompting”, ACL, 2025.
★ 9,068⑂ 771👁 70⑃ 32● 142▤ 0
More from this author
+1
Daily
9,068
stars
771
forks
70
watchers
32
prs
142
issues
0
releases
README
Dolphin: Document Image Parsing via Heterogeneous Anchor Prompting
Dolphin-v2 is an enhanced universal document parsing model that substantially improves upon the original Dolphin. It seamlessly handles any document type—whether digital-born or photographed—through a document-type-aware two-stage architecture with scalable anchor prompting.
📑 Overview
Document image parsing is challenging due to diverse document types and complexly intertwined elements such as text paragraphs, figures, formulas, tables, and code blocks. Dolphin-v2 addresses these challenges through a document-type-aware two-stage approach:
1. 🔍 Stage 1: Document type classification (digital vs. photographed) + layout analysis with reading order prediction
2. 🧩 Stage 2: Hybrid parsing strategy - holistic parsing for photographed documents, parallel element-wise parsing for digital documents
Dolphin achieves promising performance across diverse page-level and element-level parsing tasks while ensuring superior efficiency through its lightweight architecture and parallel parsing mechanism.
📅 Changelog
- 🔥 2025.12.12 Released Dolphin-v2 model. Upgraded to 3B parameters with 21-element detection, attribute field extraction, dedicated formula/code parsing, and robust photographed document parsing. (Dolphin-1.5 moved to v1.5 branch)
- 🔥 2025.10.16 Released Dolphin-1.5 model. While maintaining the lightweight 0.3B architecture, this version achieves significant parsing improvements. (Dolphin 1.0 moved to v1.0 branch)
- 🔥 2025.07.10 Released the Fox-Page Benchmark, a manually refined subset of the original Fox dataset. Download via: Baidu Yun | Google Drive.
- 🔥 2025.06.30 Added TensorRT-LLM support for accelerated inference!
- 🔥 2025.06.27 Added vLLM support for accelerated inference!
- 🔥 2025.06.13 Added multi-page PDF document parsing capability.
- 🔥 2025.05.21 Our demo is released at link. Check it out!
- 🔥 2025.05.20 The pretrained model and inference code of Dolphin are released.
- 🔥 2025.05.16 Our paper has been accepted by ACL 2025. Paper link: arXiv.
📈 Performance
Comprehensive evaluation of document parsing on OmniDocBench (v1.5)
Model
Size
Overall&x2191;
TextEdit&x2193;
FormulaCDM&x2191;
TableTEDS&x2191;
TableTEDS-S&x2191;
Read OrderEdit&x2193;
Dolphin
0.3B
74.67
0.125
67.85
68.70
77.77
0.124
Dolphin-1.5
0.3B
85.06
0.085
79.44
84.25
88.06
0.071
Dolphin-v2
3B
89.78
0.054
87.63
87.02
90.48
0.054
🛠️ Installation
1. Clone the repository:
bash
git clone https://github.com/ByteDance/Top issues
#93Failed to inference by vllm server: TypeError: Unknown image model type: vision-encoder-decoder10 comments · 2025-07-01#84Can't reproduce Fox-Page-En Results10 comments · 2025-06-23#22How to use this with vLLM?10 comments · 2025-05-23#39图片文字识别的效果太离谱9 comments · 2025-05-27#62Can you please share a finetuning script8 comments · 2025-06-13#103json输出里面所有label以及命名形式能够提供一下吗6 comments · 2025-07-09#64install on python 3.13.35 comments · 2025-06-15#8demo代码无法在mac电脑上运行5 comments · 2025-05-23