README

AReaL: A Large-Scale Asynchronous Reinforcement Learning System



| Paper | Documentation | 中文文档 | Ask DeepWiki | 🤗 Models & Data |
 WeChat (微信) Group |
  




AReaL is a reinforcement learning (RL) infrastructure designed to bridge foundation
model training with modern agent-based applications. It was originally developed by
researchers and engineers from Tsinghua IIIS and the AReaL Team at Ant Group.

Built on a fully asynchronous RL training paradigm, AReaL is optimized for
efficiency and scalability, making it particularly well-suited for training large-scale
reasoning and agentic models.

AReaL’s mission is to make building AI agents accessible, efficient, and
cost-effective for a broad community of developers and researchers.

 Like milk tea - customizable, scalable, and enjoyable - we hope AReaL brings both
 flexibility and delight to your AI development experience. Cheers!

AReaL Highlights

- ⚡ Flexibility: Seamless customization for
  agentic RL and
  online RL training for
  black-box agent applications by simply replacing the base_url.
- 📈 Scalability: Stable fully asynchronous RL training with industry-leading
  speed.
- ✨ Cutting-Edge Performance: State-of-the-art math,
  coding, search, and
  customer service agents.

 📰 News

\[2026/07/01\] 🎉 We release AReaL 2.0, a major architectural milestone. This
release refactors AReaL into a microservice architecture with independent
training, inference,
agent, and weight-update
services, and ships the Hermes online RL loop and
end-to-end SWE RL training examples. Check out our
technical report.

\[2026/06/17\] 🔬 Introducing KPop — bidirectional binary KL divergence token
masking. Configured via rejection_sampling.metric=binary_kl. Also adding an
IcePop config (importance-ratio-based token masking).
Check out gsm8k_kpop.yaml and
gsm8k_icepop.yaml to get started!


📋 Previous Releases

\[2026/04/23\] 🚀 We’re excited to release our integration with
Scaffoldings
for agentic RL training - now live in our
examples! Huge
shoutout to @narutolhy and @WeiHaocheng for making this happen 🙌. The modular design of
the Scaffoldings enables it to achieve a thorough decoupling of agent execution, reward
calculation, and trajectory acquisition. This enables developers to reuse existing
modules when implementing an agentic RL method, allowing them to focus on their own
innovative modules.

\[2026/04/18\] We are thrilled to announce that AReaL's first Community Biweekly
Meeting was successfully held! Thank you to everyone who joined us. Meeting materials
are now available
here. Our next
meeting is scheduled for 2026/05/01 and will also be conducted in Chinese;
English-language meetings will be scheduled in the future. We warmly welcome everyone to
participate! See Community for more details.

\[2026/03/02\] We provide a complete example to train your
own 🦞 OpenClaw agent by simply replacing the base_url and api_key with AReaL's RL
service - no complicated dependencies, no code changes, works with any agentic