meta-pytorch /torchtune

PyTorch native post-training library

★ 5,817⑂ 764👁 40⑃ 1,868● 1,102▤ 10
PythonBSD-3-ClauseCreated 2023-10-20pytorch.org/torchtune/main/
More from this author
Download: v0.6.1v0.6.1 ↓0.6.12025-04-07
+0
Daily
5,817
stars
764
forks
40
watchers
1,868
prs
1,102
issues
10
releases

README

⚠️ Torchtune is no longer actively maintained: torchtune development wound down in 2025 — see The future of torchtune. Huge thanks to the 150+ contributors who made this library what it was.

 torchtune


!Integration Tests


Overview | Installation | Get Started |  Documentation | Community | Citing torchtune | License

 📣 Recent updates 📣
 May 2025: torchtune has added support for Qwen3 models! Check out all the configs here
 April 2025: Llama4 is now available in torchtune! Try out our full and LoRA finetuning configs here
 February 2025: Multi-node training is officially open for business in torchtune! Full finetune on multiple nodes to take advantage of larger batch sizes and models.
 December 2024: torchtune now supports Llama 3.3 70B! Try it out by following our installation instructions here, then run any of the configs here.
 November 2024: torchtune has released v0.4.0 which includes stable support for exciting features like activation offloading and multimodal QLoRA
 November 2024: torchtune has added Gemma2 to its models!
 October 2024: torchtune added support for Qwen2.5 models - find the configs here
 September 2024: torchtune has support for Llama 3.2 11B Vision, Llama 3.2 3B, and Llama 3.2 1B models! Try them out by following our installation instructions here, then run any of the text configs here or vision configs here.


 

 Overview 📚


torchtune is a PyTorch library for easily authoring, post-training, and experimenting with LLMs. It provides:

- Hackable training recipes for SFT, knowledge distillation, DPO, PPO, GRPO, and quantization-aware training
- Simple PyTorch implementations of popular LLMs like Llama, Gemma, Mistral, Phi, Qwen, and more
- Best-in-class memory efficiency, performance improvements, and scaling, utilizing the latest PyTorch APIs
- YAML configs for easily configuring training, evaluation, quantization or inference recipes

 

 Post-training recipes

torchtune supports the entire post-training lifecycle. A successful post-trained model will likely utilize several of the below methods.

 Supervised Finetuning (SFT)

| Type of Weight Update | 1 Device | 1 Device | 1 Node |
|-----------------------|:--------:|:---------:|:-------:|
| Full                  |    ✅    |     ✅    |   ✅    |
| LoRA/QLoRA            |    ✅    |     ✅    |    ✅    |

Example: tune run lora_finetune_single_device --config llama3_2/3B_lora_single_device 
You can also run e.g. tune ls lora_finetune_single_device for a full list of available configs.

 Knowledge Distillation (KD)

| Type of Weight Update | 1 Device | 1 Device | 1 Node |
|-----------------------|:--------:|:---------:|:-------:|
| Full                  |    ❌    |     ❌    |    ❌    |
| LoRA/QLoRA            |    ✅    |     ✅    |    ❌    |

Example: tune run knowledge_distillation_distributed --config qwen2/1.5B_to_0.5B_KD_lora_distributed 
You can also run e.g. tune ls knowledge_distillation_distributed for a full list of available configs.

 Reinforcement 

Top issues

Discussions

Latest release

v0.6.1 v0.6.1 2025-04-07
v0.6.1

Patch for 2561. 

Full Changelog: https://github.com/pytorch/torchtune/compare/v0.6.0...v0.6.1

Snapshot: 2026-11-16T09:23:14Z · Data license Apache-2.0 · JSON