OpenGVLab /LLaMA-Adapter

[ICLR 2024] Fine-tuning LLaMA to follow Instructions within 1 Hour and 1.2M Parameters

★ 5,911⑂ 376👁 72⑃ 11● 147▤ 3
PythonGPL-3.0Created 2023-03-19
Download: Release LLaMA-Adapter V2.1 modelsv.2.1.0 ↓2.1.02023-10-12
+0
Daily
5,911
stars
376
forks
72
watchers
11
prs
147
issues
3
releases

README

LLaMA-Adapter: Efficient Fine-tuning of LLaMA 🚀

 Announcement: We release LLaMA2-Accessory, an open-source toolkit for pre-training, fine-tuning and deployment of LLMs and mutlimodal LLMs.🔥

Official implementation of 'LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention' and 'LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Model'.

                                                                                                                                           



This repo proposes LLaMA-Adapter (V2), a lightweight adaption method for fine-tuning Instruction-following and Multi-modal LLaMA models 🔥.

Try out the web demo 🤗 of LLaMA-Adapter: , LLaMA-Adapter V2 and ImageBind-LLM.

Join us at Wechat!

 News
- [2024.01.18] LLaMA-Adapter is accepted by ICLR 2024!🎉
- [2023.12.08] We release OneLLM which aligns eight modalities to language using a unified framework.🔥🔥🔥
- [2023.11.11] We release SPHINX, a new multi-modal LLM, which is a huge leap from LLaMa-Adapter-V2.🔥🔥🔥
- [2023.10.11] We realse LLaMA-Adapter V2.1, an improved version of LLaMA-Adapter V2 with stronger multi-modal reasoning performance. Check llama_adapter_v2_multimodal7b for details.
- [2023.08.28] We release quantized LLM with OmniQuant, which is an efficient, accurate, and omnibearing (even extremely low bit) quantization algorithm. Multimodal version is coming soon.🔥🔥🔥
- [2023.07.24] We release LLaMA2-Accessory, an open-source toolkit for pre-training, fine-tuning and deployment of Large Language Models (LLMs) and mutlimodal LLMs. Please check Alpha-VLLM/LLaMA2-Accessory for more details!🔥🔥🔥
- [2023.07.05] We release the pretrain/finetune code of llama_adapter_v2_multimodal7b.
- [2023.07.04] We release the code for reproducing Gorilla by both full finetune and LLaMA-Adapter, please see gorilla/README.md.
- [2023.06.08] We release the demo of ImageBind-LLM.
- [2023.06.06] We release Point-Bind to extend ImageBind with 3D point clouds, which achieves 3D instruction-following capacity for imagebind_LLM.
- [2023.06.05] We support the integration of LLaMA-Adapter (both V1 and V2) and LangChain. Check out the Notebook.
- [2023.05.29] We release the code of ImageBind-LLM at imagebind_LLM.
- [2023.05.23] We release the demos and multi-modal code of LLaMA-Adapter V2!
- [2023.05.05] We release the paper and code of our new work Personalize Segment Anything, which efficiently fine-tunes Segment Anything with 10 seconds, and improves DreamBooth for better text-to-image generation. 
- [2023.04.30] We noticed that GPT-4 evaluation has a strong positional bias in favor of the first response. We will soon update the paper to reveal the position bias. Great thanks to Canwen Xu.
- [2023.04.28] We release LLaMA-Adapter V2, a multi-modal instruction model. Check out our paper, demos and code!
- [2023.03.28]  The paper and training code for LLaMA-Adapter V1 are released. 📌


 Released Models 

| Name                                                         |

Top issues

Latest release

Release LLaMA-Adapter V2.1 models v.2.1.0 2023-10-12
Release LLaMA-Adapter V2.1 multi-modal models

Snapshot: 2026-11-16T09:23:14Z · Data license Apache-2.0 · JSON