rllm-org /rllm
Democratizing Reinforcement Learning for LLMs
★ 5,919⑂ 643👁 29⑃ 687● 272▤ 4
+1
Daily
5,919
stars
643
forks
29
watchers
687
prs
272
issues
4
releases
README
Agentic RL on any harness, with any backend, on any benchmark.
rLLM is an open-source framework for training language agents with reinforcement learning. Bring any harness, run it in any sandbox, and switch training backends with one flag — the same agent code drives both eval and training.
Core features
- Any harness. 10+ CLI harnesses (Claude Code, Codex, Terminus-2, mini-swe-agent, opencode, ...) plus Harbor-compatible task dirs. Or wrap your own agent — LangGraph, OpenAI Agents SDK, openai.OpenAI — with @rllm.rollout.
- Any sandbox. Docker, Daytona, Modal, or local — with snapshot + warm-pool acceleration to keep rollouts cheap at training-scale.
- Multiple training backends, one API. verl (distributed multi-GPU), tinker (single-machine), fireworks (Fireworks platform). Switch with one flag.
- 60+ integrated benchmarks. Math, code, MCQ, QA, search, VLM, translation, agentic — Terminal-Bench 2.0, SWE-bench, SkillsBench, AIME, MATH-500, GPQA, and more. rllm eval auto-pulls and runs.
- Multiple training methods. GRPO, REINFORCE, RLOO, SFT, on-policy distillation, and more.
- Battle-tested. State-of-the-art open-source results (DeepScaleR-1.5B, DeepCoder-14B, DeepSWE-32B, FinQA-4B). Adopted by academic labs and industry research teams (see Community Projects below).
Read more on our documentation site.
Installation
rLLM requires Python = 3.11. You can install it either directly via pip or build from source.
bash
uv pip install "rllm @ git+https://github.com/rllm-org/rllm.git"
This installs dependencies for running rllm CLI with the tinker backend (single-machine, Tinker API). For other backends:
bash
Distributed multi-GPU training (verl + vLLM/SGLang)
uv pip install "rllm[verl] @ git+https://github.com/rllm-org/rllm.git"
Fireworks training platform
uv pip install "rllm[fireworks] @ git+https://github.com/rllm-org/rllm.git"
For building from source or Docker, see the installation guide.
Quickstart
Option A: CLI (no code needed)
bash
1. Configure your model provider
rllm model setup
2. Evaluate on a benchmark
rllm eval gsm8k
3. Train with RL
rllm train gsm8k
Option B: Python API
Define a rollout (your agent) and an evaluator (your reward function), then hand them to the trainer:
python
my_flow.py
from openai import OpenAI
import rllm
from rllm.types import AgentConfig, Episode, Task, Trajectory
@rllm.rollout
def solve(task: Task, config: AgentConfig) - Episode:
client = OpenAI(base_url=config.base_url, api_key="EMPTY")
response = client.chat.completions.create(
model=config.model,
messages=[{"role": "user", "content": task.instruction}],
)
answer = response.choices[0].message.content or ""
return Episode(
trajectories=[Trajectory(name="solver", steps=[])],
artifacts={"answer": answer},
)
python
my_evaluator.py
import rllm
from rllm.eval.types import EvalOutput, Signal
from rllm.types import Episode
@rllm.evaluator
def score(task: dict, episode: Top issues
#30oom when update_policy15 comments · 2025-02-15#434Qwen3.5 support13 comments · 2026-03-10#29Extremely Long Execution Time per Step12 comments · 2025-02-15#33Reproduce deepscaler10 comments · 2025-02-17#7great job!have some questions to ask10 comments · 2025-02-12#67Can we train the model on a 4 X 40G gpus or even fewer?9 comments · 2025-03-24#12Reproduce evaluation results on AIME (41% vs 43.1%)9 comments · 2025-02-12#53AssertionError: Expects sharded strategy8 comments · 2025-03-02
Discussions
#937Research question: distinguishing failed evaluation from a genuine zero reward1 comments · 2026-10-17#856Toward a dependable rollout record in rLLM0 comments · 2026-08-14#648MCP to help OSS devs understand rllm-org/rllm0 comments · 2026-06-12#639[Agentflow Refactor] How agent flows, sandboxes, tasks, and evaluators fit together0 comments · 2026-06-10#492Standardized rLLM Message Format: Analysis & Proposal0 comments · 2026-04-06#251rLLM v0.2 release0 comments · 2025-10-16#231rllm Container Image size too big0 comments · 2025-10-02#159Is there a groundtruth solution available for all the deepcoder datasets?2 comments · 2025-07-23
Latest release
rLLM: v0.3.0-pre v0.3.0-pre 2026-04-30What's Changed [feature] add support for on policy distillation by @kylemontgomery1 in https://github.com/rllm-org/rllm/pull/356 [Bugfix] Make sure the tracking logger is now explicitly finished by trainers by @listar2000 in https://github.com/rllm-org/rllm/pull/358 Add Geo3K Tinker training example with VLM support by @BrianChen26 in https://github.com/rllm-org/rllm/pull/357 Support custom metric for SDK by @thwu1 in https://github.com/rllm-org/rllm/pull/362 fix: typo in bibtex by @tongyx361 in https://github.com/rllm-org/rllm/pull/363 Fireworks API returns raw token ids to avoid