hiyouga /LlamaFactory

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

★ 76,051⑂ 9,375👁 342⑃ 1,660● 8,056▤ 36
PythonApache-2.0Created 2023-05-28llamafactory.readthedocs.io
Download: v0.9.5: Qwen3.5/3.6, Gemma 4, Transformers v5v0.9.5 ↓0.9.52026-05-30
+9
Daily
+97
Weekly
76,051
stars
9,375
forks
342
watchers
1,660
prs
8,056
issues
36
releases

README

! LlamaFactory




















 Used by Amazon, NVIDIA, Aliyun, etc.

----



 Check our new open-source project —🐧 PenguinHarness: Local-first multi-agent app development platform - create, fine-tune, deploy AI apps automatically!

Follow us and give us a star ⭐: https://github.com/Prism-Shadow/penguin-harness



https://github.com/user-attachments/assets/9b7033e8-f08a-4c3f-bd33-547896664e6e

----



 Supporters ❤️

| Warp, the agentic terminal for developersAvailable for MacOS, Linux, & Windows |   |
| ---- | ---- |

----

 Easily fine-tune 100+ large language models with zero-code CLI and Web UI

!GitHub Trend



👋 Join our WeChat and NPU user groups.

\ English | [中文 \]

Fine-tuning a large language model can be easy as...

https://github.com/user-attachments/assets/3991a3a8-4276-4d30-9cab-4cb0c4b9b99e

Start local training:
- Please refer to usage

Start cloud training:
- Colab (free): https://colab.research.google.com/drive/1eRTPn37ltBbYsISy9Aw2NuI2Aq5CQrD9?usp=sharing
- PAI-DSW (free trial): https://gallery.pai-ml.com//preview/deepLearning/nlp/llama_factory
- AMD GPU Cloud (free credits): https://github.com/AMD-AIM/AMD_Developers_Notebooks/blob/main/en/AMD_developer_LLaMAFactory_note_en.md

Read technical notes:
- Documentation (WIP): https://llamafactory.readthedocs.io/en/latest/
- Documentation (AMD GPU): https://rocm.docs.amd.com/projects/ai-developer-hub/en/latest/notebooks/fine_tune/llama_factory_llama3.html
- Documentation (ASCEND NPU): https://llamafactory.readthedocs.io/en/latest/multibackend/npu/index.html
- Official Blog: https://blog.llamafactory.net/en/

 [!NOTE]
 Except for the above links, all other websites are unauthorized third-party websites. Please carefully use them.

 Table of Contents

- Features
- Blogs
- Changelog
- Supported Models
- Supported Training Approaches
- Provided Datasets
- Requirement
- Getting Started
  - Installation
  - Data Preparation
  - Quickstart
  - Fine-Tuning with LLaMA Board GUI
  - Build Docker
  - Deploy with OpenAI-style API and vLLM
  - Download from ModelScope Hub
  - Download from Modelers Hub
  - Use W&B Logger
  - Use SwanLab Logger
- Projects using LlamaFactory
- License
- Citation
- Acknowledgement

 Features

- Various models: LLaMA, LLaVA, Mistral, Mixtral-MoE, Qwen3, Qwen3-VL, DeepSeek, Gemma, GLM, Phi, etc.
- Integrated methods: (Continuous) pre-training, (multimodal) supervised fine-tuning, reward modeling, PPO, DPO, KTO, ORPO, etc.
- Scalable resources: 16-bit full-tuning, freeze-tuning, LoRA and 2/3/4/5/6/8-bit QLoRA via AQLM/AWQ/GPTQ/LLM.int8/HQQ/EETQ.
- Advanced algorithms: GaLore, BAdam, APOLLO, Adam-mini, Muon, OFT, DoRA, LongLoRA, LLaMA Pro, Mixture-of-Depths, LoRA+, LoftQ and PiSSA.
- Practical tricks: FlashAttention-2, Unsloth, Liger Kernel, KTransformers, RoPE scaling, NEFTune and rsLoRA.
- Wide tasks: Multi-turn dialogue, tool using, image understanding, visual grounding, video recognition, audio understanding, etc.
- Experiment monitors: LlamaBoard, TensorBoard

Top issues

Discussions

Latest release

v0.9.5: Qwen3.5/3.6, Gemma 4, Transformers v5 v0.9.5 2026-05-30
Added primary support for Qwen3.5/Qwen3.6/Gemma4 models and compatibility with Transformers v5.

 What's Changed

 [misc] set dev version by @hiyouga in https://github.com/hiyouga/LlamaFactory/pull/9703
 fix(fp8): add Transformer Engine backend support by @sbhavani in https://github.com/hiyouga/LlamaFactory/pull/9705
 [misc] Compatible with an empty architectures field in config.json by @tangefly in https://github.com/hiyouga/LlamaFactory/pull/9709
 [model] support Youtu-LLM-2B by @isLinXu in https://github.com/hiyouga/LlamaFactory/pull/9707
 [misc] lint by @hiyouga in https://github.c

Snapshot: 2026-11-16T09:23:14Z · Data license Apache-2.0 · JSON