opendatalab /MinerU-HTML
MinerU-HTML: An SLM-powered HTML main content extractor that outputs clean HTML bodies. Perfect for Deep Research Agents, RAG applications, and training data generation.
★ 301⑂ 28👁 5⑃ 13● 24▤ 3
-1
Daily
301
stars
28
forks
5
watchers
13
prs
24
issues
3
releases
README
MinerU-HTML English/中文 MinerU-HTML is an advanced HTML main content extraction tool based on Small Language Models (SLM). It can accurately identify and extract main content from complex web page HTML, automatically filtering out auxiliary elements such as navigation bars, advertisements, and metadata. Try on our website - Mineru-Extractor Welcome to try our online document extraction tool! Supports HTML main content extraction and OCR for various document formats. OR Download the model for local usage - MinerU-HTML-v1.1 —— Now updated to v1.1 ! News - 2026.11.05 🎉 A new Qwen3.5-0.8B-based MinerU-HTML-v1.1 checkpoint is now available! It keeps the compact output format and can be used as a drop-in replacement for the previous release. Welcome to use! - 2026.03.19 🎉 The MinerU-HTML-v1.1 is released! A more efficient and powerful version than v1.0, now integrated with MinerU-Webkit for HTML-to-Markdown/JSON/Txt conversion. Welcome to use! - 2025.12.1 🎉 The AICC dataset is released, welcome to use! AICC dataset contains 7.3T web pages extracted and converted to Markdown format by MinerU-HTML, with cleaner main content and high-quality code, formulas, and tables. - 2025.12.1 🎉 The MinerU-HTML model is released, welcome to use! MinerU-HTML model is a fine-tuned model on Qwen3, with better performance on HTML main content extraction. - 2025.12.1 🎉 The trial page is online, welcome to visit Opendatalab-AICC to try our extraction tool! ✨ Features - 🎯 LLM-Powered Extraction: Uses state-of-the-art language models to intelligently identify main content - 📝 Extensible Output: Integrated with MinerU-Webkit to support efficient conversion of extracted HTML into Markdown, JSON and Txt. - ⚡ Compact Format: Now use compact format for the model instead of the original JSON format for faster inference speed - ⚡ Regex structured output: Now use regex structured output for the model instead of the custom logits processor, so we can use vLLM v1 backend for inference. - 🔄 Complete Processing Pipeline: Includes HTML simplification, prompt construction, LLM inference, result parsing, main content extraction, and other complete processes - 🚀 Multiple Inference Backend Support: Supports three inference backends: VLLM, Transformers, and OpenAI API - 🛡️ Fault Tolerance: Supports fallback mechanism (trafilatura, bypass or empty) to ensure content extraction even when processing fails - 🔌 Modular Design: Clear architecture design, easy to extend and customize - 🧪 Complete Testing: Includes comprehensive unit tests and integration tests Evaluation Results We evaluated MinerU-HTML on the WebMainBench v1.1 benchmark, which contains 7,809 (7,887 for v1.0, we removed some low quality web pages) meticulously annotated web pages along with their corresponding Markdown-formatted main content converted using html2text. This benchmark measures the extraction accuracy of content extractors by computing ROUGE-N scores between the extracted results and ground-truth content.
Top issues
Latest release
v1.1.3-qwen v1.1.3-qwen 2026-11-06Full Changelog: https://github.com/opendatalab/MinerU-HTML/compare/v1.1...v1.1.3-qwen