Open library
The essential reading of generative AI and agents, open in a clean reading view, with Meow and Standard summaries when available. No account needed to read.
560 papers · 558 with a Meow summary · 333 posts
The selected papers and blogs: the essential papers of generative AI and agents, and the key lab publications and essays, each with Meow and Standard summaries.
John Yang, Kilian Lieret, Carlos E. Jimenez et al.
Despite recent progress in Language Models (LMs) for software engineering, collecting training data remains a significant pain point. Existing datasets are small, with at most $1{,}000$s of training instances from $11$ or fewer GitHub repositories. The procedures to curate such datasets are often complex, necessitatin…
Naman Jain Jaskirat Singh Manish Shetty Liang Zheng Koushik Sen Ion Stoica
Improving open-source models on real-world Swe tasks (solving GitHub issues) faces two key challenges: 1) scalable curation of execution environments to train these models, and 2) optimal scaling of test-time compute. We introduce R2E-Gym, the largest procedurally-curated executable gym environment for training real-w…
Jiayi Pan, Xingyao Wang, Graham Neubig et al.
We present SWE-Gym, the first environment for training software engineering (SWE) agents. SWE-Gym contains 2,438 real-world task instances, each comprising a Python codebase with an executable runtime environment, unit tests, and a task specified in natural language. We use SWE-Gym to train language model based SWE ag…
Guibin Zhang, Hejia Geng, Xiaohang Yu et al.
The emergence of agentic reinforcement learning (Agentic RL) marks a paradigm shift from conventional reinforcement learning applied to large language models (LLM RL), reframing LLMs from passive sequence generators into autonomous, decision-making agents embedded in complex, dynamic worlds. This survey formalizes thi…
Xufang Luo, Yuge Zhang, Zhiyuan He et al.
We present Agent Lightning, a flexible and extensible framework that enables Reinforcement Learning (RL)-based training of Large Language Models (LLMs) for any AI agent. Unlike existing methods that tightly couple RL training with agent or rely on sequence concatenation with masking, Agent Lightning achieves complete…
Jiazhan Feng, Shijue Huang, Xingwei Qu et al.
While reasoning models (e.g., DeepSeek R1) trained with reinforcement learning (RL), excel in textual reasoning, they struggle in scenarios requiring structured problem-solving, such as geometric reasoning, concise computation, or complex equation solving—areas where computational tools like code interpreters (CI) dem…
Chengsong Huang, Zifeng Wang, Rujun Han et al.
LLM agents learn by interacting with environments, yet these environments are hand-built and static: blind to an agent’s weaknesses, and quickly left behind as it improves. While recent environment generation methods attempt to address this, they require domain-specific pipelines, rely on expensive or unreliable verif…
Jinyang Wu , Shuo Yang , Zhengxi Lu, Fan Zhang, Yuhao Shen, Lang Feng,
Large language models are increasingly trained as interactive agents for long-horizon tasks involving multi-turn interaction, tool use, and environment feedback. Outcome-based reinforcement learning (RL) provides a practical optimization paradigm, but its sparse trajectory-level rewards offer limited guidance on inter…
Zhenyu Hou, Yujiang Li, Jie Tang et al.
Reinforcement learning (RL) is becoming increasingly important for post-training large language models (LLMs). Previous RL pipelines for LLMs were mostly synchronous and batch-interleaved, which is inefficient for long-horizon agentic tasks. Recently, asynchronous RL has emerged as a more efficient alternative by upda…
Shuo Yang , Jinyang Wu , Zhengxi Lu, Yuhao Shen, Fan Zhang, Lang Feng,
Outcome-based reinforcement learning provides a stable optimization backbone for language agents, but its sparse trajectory-level rewards provide little guidance on which intermediate decisions should be reinforced or suppressed. On-policy self-distillation offers dense token-level supervision, yet existing skill-cond…
Zhengxi Lu, Zhiyuan Yao, Zhuowen Han, Zi-Han Wang, Jinyang Wu
Reinforcement learning (RL) has emerged as a central paradigm for post-training LLM agents, yet its trajectory-level reward signal provides only coarse supervision for long-horizon interaction. On-Policy Self-Distillation (OPSD) complements RL by introducing dense token-level guidance from a teacher branch augmented w…
Yaorui Shi, Yuxin Chen, Zhengxi Lu et al.
A persistent skill library allows language model agents to reuse successful strategies across tasks. Maintaining such a library requires three coupled capabilities. The agent selects a relevant skill, utilizes it during execution, and distills new skills from experience. Existing methods optimize these capabilities in…
Zhengxi Lu, Zhiyuan Yao, Jinyang Wu, Chengcheng Han, Qi Gu
Agent skills, structured packages of procedural knowledge and executable resources that agents dynamically load at inference time, have become a reliable mechanism for augmenting LLM agents. Yet inference-time skill augmentation is fundamentally limited: retrieval noise introduces irrelevant guidance, injected skill c…
Jenny Zhang, Bingchen Zhao, Wannan Yang et al.
Self-improving AI systems aim to reduce reliance on human engineering by learning to improve their own learning and problem-solving processes. Existing approaches to recursive self-improvement typically rely on fixed, handcrafted meta-level mechanisms, which fundamentally limit how fast such systems can improve. The D…
Yinjie Wang, Xuyang Chen, Xiaolong Jin et al.
Every agent interaction generates a **next-state signal**, namely the user reply, tool output, terminal or GUI state change that follows each action, yet no existing agentic RL system recovers it as a live, online learning source. We present **OpenClaw-RL**, a framework that employs next-state signals to optimize pers…
Zhixia Zhang, Zixuan Huang, Gongxun Li et al.
We introduce **H**eterogeneous **A**gent **C**ollaborative **R**einforcement **L**earning (**HACRL**), a new Reinforcement Learning from Verifiable Reward (RLVR) problem that addresses the inefficiencies of isolated multi-agent on-policy optimization. HACRL enables collaborative optimization with independent execution…
Peng Xia, Jianwen Chen, Hanyang Wang et al.
Large Language Model (LLM) agents have shown stunning results in complex tasks, yet they often operate in isolation, failing to learn from past experiences. Existing memory-based methods primarily store raw trajectories, which are often redundant and noise-heavy. This prevents agents from extracting high-level, reusab…
Dwarkesh Patel
The author’s central question is whether AI can turn scarce, messy work experience into efficient, lasting learning—not just get better at repeatable training tasks.
ben burtenshaw · Hugging Face
OpenEnv is moving to committee coordination and a clearer role as shared RL-environment infrastructure—not a framework for rewards or training loops—so agents can work across tools without one mandated stack.
Brenda Potts · Microsoft Research
Agent Lightning v1.0 trains agents through the same harness used in deployment, avoiding a costly reimplementation inside the RL framework.