xlite-dev /Awesome-LLM-Inference

📚A curated list of Awesome LLM/VLM Inference Papers with Codes: Flash-Attention, Paged-Attention, WINT8/4, Parallelism, etc.🎉

★ 5,558⑂ 447👁 132⑃ 187● 13▤ 45
PythonGPL-3.0Created 2023-08-27
More from this author
Download: v2.6.20v2.6.20 ↓2.6.202025-06-17
+1
Daily
5,558
stars
447
forks
132
watchers
187
prs
13
issues
45
releases

README

📒Introduction
Awesome-LLM-Inference: A curated list of 📙Awesome LLM Inference Papers with Codes. For Awesome Diffusion Inference, please check 📖Awesome-DiT-Inference  !. For CUDA learn notes, please check 📖LeetCUDA  !.

 📖 News 🔥🔥


- [2026/03] Cache-DiT 🎉v1.3.0 release is ready, the major updates including: Ring Attention w/ batched P2P, USP (Hybrid Ring and Ulysses), Hybrid 2D and 3D Parallelism (💥USP + TP),  VAE-P Comm overhead reduce.

!arch

 ©️Citations

BibTeX
@misc{Awesome-LLM-Inference@2024,
  title={Awesome-LLM-Inference: A curated list of Awesome LLM Inference Papers with codes},
  url={https://github.com/xlite-dev/Awesome-LLM-Inference},
  note={Open-source software available at https://github.com/xlite-dev/Awesome-LLM-Inference},
  author={xlite-dev, liyucheng09 etc},
  year={2024}
}


 🎉Awesome LLM Inference Papers with Codes

Awesome LLM Inference for Beginners.pdf: 500 pages, FastServe, FlashAttention 1/2, FlexGen, FP8, LLM.int8(), PagedAttention, RoPE, SmoothQuant, WINT8/4, Continuous Batching, ZeroQuant 1/2/FP, AWQ etc.





 🎉Download All PDFs
bash
python3 download_pdfs.py  The code is generated by Doubao AI





 📖Contents
 📖Trending LLM/VLM Topics🔥🔥🔥
 📖DeepSeek/MLA Topics🔥🔥🔥
 📖Multi-GPUs/Multi-Nodes Parallelism🔥🔥🔥
 📖Disaggregating Prefill and Decoding🔥🔥🔥
 📖LLM Algorithmic/Eval Survey
 📖LLM Train/Inference Framework/Design
 📖Weight/Activation Quantize/Compress🔥
 📖Continuous/In-flight Batching
 📖IO/FLOPs-Aware/Sparse Attention🔥
 📖KV Cache Scheduling/Quantize/Dropping🔥
 📖Prompt/Context Compression🔥
 📖Long Context Attention/KV Cache Optimization🔥🔥
 📖Early-Exit/Intermediate Layer Decoding
 📖Parallel Decoding/Sampling🔥
 📖Structured Prune/KD/Weight Sparse
 📖Mixture-of-Experts(MoE) LLM Inference🔥
 📖CPU/NPU/FPGA/Mobile Inference
 📖Non Transformer Architecture🔥
 📖GEMM/Tensor Cores/WMMA/Parallel
 📖VLM/Position Embed/Others
 📖LLM Inference Applications

 📖Trending LLM/VLM Topics (©️back👆🏻)


|Date|Title|Paper|Code|Recom|
|:---:|:---:|:---:|:---:|:---:|
| 2026.03 | 🔥🔥🔥[OneComp] OneComp: One-Line Revolution for Generative AI Model Compression(@Fujitsu) | [[pdf]](https://arxiv.org/pdf/2603.28845) | [[OneCompression]](https://github.com/FujitsuResearch/OneCompression) ! | ⭐️⭐️ |
| 2025.12 | 🔥🔥[QEP] QEP: Quantization Error Propagation, NeurIPS 2025(@Fujitsu) | [[pdf]](https://openreview.net/pdf?id=a3l3K9khbL) | [[OneCompression]](https://github.com/FujitsuResearch/OneCompression) ! | ⭐️⭐️ |
|2024.04| 🔥🔥🔥[Open-Sora] Open-Sora: Democratizing Efficient Video Production for All(@hpcaitech)|[[docs]](https://github.com/hpcaitech/Open-Sora/blob/main/docs/zh_CN/README.md) | [[Open-Sora]](https://github.com/hpcaitech/Open-Sora) !| ⭐️⭐️ |
|2024.04| 🔥🔥🔥[Open-Sora Plan] Open-Sora Plan: This project aim to reproduce Sora (Open AI T2V model)(@PKU)|[[report]](https://github.com/PKU-YuanGroup/Open-Sora-Plan/blob/main/docs/Report-v1.0.0.md) | [[Open-Sora-Plan]](https://github.com/PKU-YuanGroup/Open-Sora-Plan) !| ⭐️⭐️ |
|2024.05| 🔥🔥🔥[DeepSeek-V2] DeepSeek-V2: 

Top issues

Latest release

v2.6.20 v2.6.20 2025-06-17
What's Changed
 Add 4 papers by @woominsong in https://github.com/xlite-dev/Awesome-LLM-Inference/pull/148
 Update new paper (KVzip) by @Janghyun1230 in https://github.com/xlite-dev/Awesome-LLM-Inference/pull/149
 Add a new paper (GuidedQuant) by @jusjinuk in https://github.com/xlite-dev/Awesome-LLM-Inference/pull/150
 Add STAND by @woominsong in https://github.com/xlite-dev/Awesome-LLM-Inference/pull/151
 Add Inference-Time Hyper-Scaling by @CStanKonrad in https://github.com/xlite-dev/Awesome-LLM-Inference/pull/152

 New Contributors
 @woominsong made their first contribution in https

Snapshot: 2026-11-16T09:23:14Z · Data license Apache-2.0 · JSON