PeterGriffinJin /Search-R1
Search-R1: An Efficient, Scalable RL Training Framework for Reasoning & Search Engine Calling interleaved LLM based on veRL
★ 5,579⑂ 503👁 17⑃ 27● 170▤ 0
+0
Daily
5,579
stars
503
forks
17
watchers
27
prs
170
issues
0
releases
README
Search-R1: Train your LLMs to reason and call a search engine with reinforcement learning
Search-R1 is a reinforcement learning framework for training reasoning and searching (tool-call) interleaved LLMs. --
Search-R1 is a reinforcement learning framework designed for training reasoning-and-searching interleaved LLMs—language models that learn to reason and make tool calls (e.g., to search engines) in a coordinated manner.
DeepSeek-R1(-Zero) with interleaved search engine calling and an opensource RL training-based solution for OpenAI DeepResearch. --
Built upon veRL, Search-R1 extends the ideas of DeepSeek-R1(-Zero) by incorporating interleaved search engine access and provides a fully open-source RL training pipeline. It serves as an alternative and open solution to OpenAI DeepResearch, enabling research and development in tool-augmented LLM reasoning.
We support different RL methods (e.g., PPO, GRPO, reinforce), different LLMs (e.g., llama3, Qwen2.5, etc) and different search engines (e.g., local sparse/dense retrievers and online search engines).
Paper: link1, link2; Model and data: link; Twitter thread: link; Full experiment log: prelim; v0.1; v0.2; v0.3. Details about these logs and methods can be find here.
!single-turn
News
- [2025.10] Search-R1 is featured by Thinking Machines Lab's first product Tinker! Details: Document.
- [2025.7] Search-R1 is supported by SkyRL! Detailed instructions: code, Document.
- [2025.6] Search-R1 is now integrated into the latest version of veRL and can take advantage of its most up-to-date features! Detailed instructions: veRL, English Document, Chinese Document.
- [2025.5] The second paper conducting detailed empirical studies is published with logs: v0.3.
- [2025.4] We support multinode training for 30B+ LLMs!
- [2025.4] We support different search engines including sparse local retriever, dense local retriever with ANN indexing and online search engines!
- [2025.3] The first Search-R1 paper is published with the logs: v0.1; v0.2.
- [2025.2] We opensource Search-R1 codebase with preliminary results.
Links
- Installation
- Quick start
- Preliminary results
- Inference
- Use your own dataset
- Use your own search engine
- Features
- Ackowledge
- Citations
Installation
Search-r1 environment
bash
conda create -n searchr1 python=3.9
conda activate searchr1
install torch [or you can skip this step and let vllm to install the correct version for you]
pip install torch==2.4.0 --index-url https://download.pytorch.org/whl/cu121
install vllm
pip3 install vllm==0.6.3 or you can install 0.5.4, 0.4.2 and 0.3.1
verl
pip install -e .
flash attention 2
pip3 install flash-attn --no-build-isolation
pip install wandb
Retriever environment (optional)
If you would like to call a local retriever as the search engine, you can install the environment as follows. (We recommend using a seperate environment.)
bash
conda create -n retrieverTop issues
#122Self-built data output is abnormal9 comments · 2025-06-25#88Question about GRPO and sample indexing8 comments · 2025-04-25#153Upgrade Search-R1 to support latest vLLM and Verl5 comments · 2025-09-07#142PPO training fails after 50-100 steps usually before it reaches the trainer.save_freq/trainer.test_freq5 comments · 2025-07-25#155Top-k Parameter Set to 3, but 5 Documents Returned for Some Queries During Training4 comments · 2025-09-24#152The training time4 comments · 2025-08-21#67Getting error while running inference3 comments · 2025-04-07#175训练稳定性与信用分配:针对长检索序列实现“跳过检索内容”的 GAE 机制2 comments · 2026-03-07