SkillOS: Learning Skill Curation for Self-Evolving Agents

Siru Ouyang, Jun Yan, Yanfei Chen, Rujun Han, Zifeng Wang, Bhavana Dalvi Mishra, Rui Meng, Chun-Liang Li

and 8 more authors

Yizhu Jiao, Kaiwen Zha, Maohao Shen, Vishy Tirumalashetty, George Lee, Jiawei Han, Tomas Pfister, Chen-Yu Lee

arXiv:2605.06614v1 · This workspace stays on this version. Check arXiv for newer versions

Affiliations and author notes

Google Cloud AI Research
University of Illinois Urbana-Champaign
Massachusetts Institute of Technology

Some source details need review 5 to check

Original formulas, table values and figures are preserved. Comparison can miss differences; use the original PDF to check details.

Reading view checked: 508 math expressions, 288 rendering failures; 28 of 28 images available.

Compared 134 of 137 source passages against 33 of 33 PDF pages.

The HTML source contains failed formatting commands; these were omitted from the reading view. The original sources are retained. (1)

Some source graphics could not be placed in the reading view. (30)
Some formulas could not be rendered; their original TeX is shown. Check the original PDF. (288)
This long paper was only partially compared; the original sources remain available.

HTML S1_p1_1 reads “long-term proficiency ([He et al., 2026a](#bib.bib51))”; the original PDF, page 1, reads “long-term proficiency (He et al., 2026b).” Check PDF page 1

HTML source S4_SS1_p2_1 cites “DeepMath-103k ([He et al., 2026b]”; the original PDF page 6 shows “DeepMath-103k (He et al., 2026a)”. The PDF contradicts the HTML. Check PDF page 6

28 of 28 figure, table and graphic images captured. Open original PDF

Abstract

LLM-based agents are increasingly deployed to handle streaming tasks, yet they often remain one-off problem solvers that fail to learn from past interactions. Reusable skills distilled from experience provide a natural substrate for self-evolution, where high-quality skill curation serves as the key bottleneck. Existing approaches either rely on manual skill curation, prescribe heuristic skill operations, or train for short-horizon skill adaptation, but still struggle to learn complex long-term curation policies from indirect and delayed feedback. We propose SkillOS, an experience-driven RL training recipe for learning skill curation in self-evolving agents. SkillOS pairs a frozen agent executor that retrieves and applies skills with a trainable skill curator that updates an external SkillRepo from accumulated experience. To provide learning signals for curation, we train on grouped task streams based on skill-relevant task dependencies, where earlier trajectories update the SkillRepo, and later related tasks evaluate these updates. We further design composite rewards to better attribute downstream executor feedback to curation decisions. Across multi-turn agentic tasks and single-turn reasoning tasks, SkillOS consistently outperforms memory-free and strong memory-based baselines in both effectiveness and efficiency, with the learned skill curator generalizing across different executor backbones and task domains. Further analyses show that the learned curator produces more targeted skill use, while the evolving SkillRepo develops richer internal structure and higher-level meta-skills over time.

LLM-based agents (Wang et al., 2024) are increasingly deployed in real-world scenarios, where they must move beyond instantaneous problem-solving toward long-term proficiency (He et al., 2026a). However, the prevailing paradigm of “one-off” task execution limits their utility in streaming settings, where tasks unfold sequentially over time. This makes self-evolution (Fang et al., 2025a; Gao et al., 2025) essential: capable agents should not repeatedly start from scratch, but instead continually accumulate, refine, and reuse experience for future tasks.

A key substrate for self-evolution is procedural memory (Hu et al., 2025; Wu et al., 2025b; Fang et al., 2025b), specifically, reusable skills (Anthropic, 2025b; Wang et al., 2025c) accumulated from past interactions. In real-world streaming settings (Wu et al., 2024), a skill-based self-evolving agent typically follows a closed-loop workflow: for each new task, it selects relevant skills, uses them to guide execution, and updates its skill collection based on the resulting trajectory. This makes skill curation—the extraction of high-quality lessons and their integration into the skill collection—essential for self-evolving agents.

However, existing skill curation works remain limited. Manually curated skills, such as Anthropic’s skills repository (Anthropic, 2025b), demand huge human expertise and cannot scale to the diversity of tasks that agents may encounter. Prompting or heuristic-based methods that dictate memory operations (Xu et al., 2025; Qiu et al., 2025; Zheng et al., 2025) rely on fixed rules and lack downstream performance feedback, preventing them from adapting to the executor’s actual needs. Recent studies explored reinforcement learning (RL) to optimize skill-based agent systems. However, they either focus on teaching agents to use skills (Xia et al., 2026; Tu et al., 2026) or optimize skill operations within a short task stream (Wang et al., 2025a; Ye et al., 2026). This limits the density of learning signals available for curating highly reusable skills and mastering complex management operations such as skill update and deletion, which are essential for robust and scalable long-term self-evolution.

Figure 1: SkillOS pairs a frozen Agent Executor with a trainable Skill Curator. The executor retrieves relevant skills from SkillRepo to act; the curator edits the repo (insert/update/delete) based on the resulting experiences, with Markdown as the skill format.

To tackle this challenge, we propose SkillOS, an experience-driven RL training recipe to learn the capability of skill curation for self-evolving agents. We study skill curation in a modular multi-agent framework in a streaming setting, where a frozen agent executor solves tasks with a skill collection (termed SkillRepo), while a trainable skill curator updates and manages this collection through function calls (Figure 1(a)). We represent skills as Markdown files (Anthropic, 2025b) (Figure 1(b)) managed via file I/O operations similar to an operating system (OS). Our recipe features two core designs. First, we construct each training instance as a group of related tasks. By mimicking test-time streaming settings, it grounds skill curation in long-term utility: skills induced from earlier experiences are evaluated by their ability to improve later related tasks. Second, we design rewards to better attribute environmental feedback to curation decisions, combining task performance with signals for valid function calls, skill quality, and SkillRepo’s compactness. Together, these designs turn delayed and indirect supervision into learning signals for skill curation.

We evaluate SkillOS on both multi-turn agentic tasks and single-turn reasoning tasks. Experiments show that SkillOS consistently outperforms memory-free and strong memory-based methods in both effectiveness and efficiency, with up to $+9.8\%$ relative performance improvement and $-6.0\%$ fewer interaction steps compared to the strongest baseline (Table 1). Our trained skill curator generalizes well across executors and tasks, improving performance even with the Gemini-2.5-Pro executor. Notably, our 8B curator also outperforms Gemini-2.5-Pro when used directly as the curator. Beyond performance gains, our analyses further show that the learned skill curator leads to more targeted and effective skill utilization, while the skills in SkillRepo evolve into more richly structured Markdown files that encode higher-level meta-skills over time. Together, we establish SkillOS as a practical, modular, and experience-driven RL training recipe for building self-evolving agents.

Memory for Self-Evolving Agents. Learning from past experiences as procedural memory (Wu et al., 2025b; Wei et al., 2025; Shen et al., 2026; Hu et al., 2025; Huang et al., 2026; Zhang et al., 2024) is a central mechanism for developing self-evolving agents (Gao et al., 2025; Fang et al., 2025a). The central challenge is to encode interaction histories into reusable and retrievable representations. Case-based representations are the most concrete form in this research line: they store experiences in minimally processed formats, allowing past histories to be replayed directly or reused as in-context exemplars, such as raw trajectories (Zheng et al., 2024; Zhou et al., 2025; Wu et al., 2025a) and abstracted query–response pairs (Zhao et al., 2024; Islam et al., 2024). Another line of work abstracts experiences into higher-level knowledge that is editable, auditable, and composable, reducing reliance on long trajectory replay and improving both cross-task generalization and efficiency. Such strategy-based memory typically consists of reusable workflows (Wang et al., 2025d; Tang et al., 2025), distilled insights (Ouyang et al., 2026; Huang et al., 2025; Ho et al., 2025), and recurring patterns (Yang et al., 2024; Kim et al., 2025). Recently, skills (Wang et al., 2025c; Kuroki et al., 2025; Ling et al., 2026; Li et al., 2026a; Zhang et al., 2026a; Yang et al., 2026; Alzubi et al., 2026; Liang et al., 2026) have emerged as a new agent-native form of memory and an orchestrable capability layer, owing to their modularity and ease of customization. Anthropic conceptualizes each skill as a folder containing instructions, scripts, and supporting resources (Anthropic, 2025a), which has become the most widely adopted design in the current community. Our work follows this design philosophy, simplifying the setting for research purposes by representing each skill as a single Markdown file.

Learning Memory and Skill Curation with RL. Training LLM-based agent systems with memory capabilities using RL has become a growing research direction. One research line targets training for long-context management with predefined operations such as compaction (Zhou et al., 2026; Yu et al., 2026; Wang et al., 2025b). Another interesting area focuses more on memory utilization and management by learning additional memory tool-calls (Yan et al., 2025; Zhang et al., 2025b; Zhang et al., 2025a) or training policies for different stages, such as memory retrieval (Zhang et al., 2026b). More recently, RL has been applied at various stages of agent skill development. Specifically, SkillRL (Xia et al., 2026) and D2Skill (Tu et al., 2026) teach smaller models to use skills curated from powerful LLMs in an iterative manner. ARISE (Li et al., 2026b) trains a shared policy operating both as skill retriever and worker, with heuristics for skill management. Recent studies have begun to train agents for memory or skill curation (Wang et al., 2025a; Ye et al., 2026), but their supervision is mostly restricted to local adaptation within short task streams. This favors immediately useful operations such as skill insertion, while offering limited signal for complex management operations, such as revising outdated skills and deleting harmful ones. SkillOS instead formulates skill curation as a long-horizon, executor-grounded learning problem. We group related tasks into training instances and combine downstream task outcomes with intermediate rewards, turning delayed and indirect feedback into learning signals for skill curation.

In this section, we first formalize the problem setting and introduce the multi-agent modular design of SkillOS. We then detail the RL training recipe designed specifically for training the skill curator.

We consider a streaming test-time setting (Wu et al., 2024), where an LLM-based agent is deployed to solve a sequence of tasks $\mathcal{D}=\{x_{1},x_{2},\dots,x_{T}\}$ that arrive over time. At each time stamp $t$ , the agent must solve the current task $x_{t}$ before observing future tasks, producing an execution trajectory $\xi_{t}=\{o_{1},a_{1},\dots,o_{n},a_{n}\}$ , where $o$ and $a$ denote observations and actions, respectively. This setting naturally captures the challenge of self-evolving agents, where the system must distill useful experience from the trajectories of past interactions to improve performance on future tasks, and become more capable over time. Figure 1(a) presents an overview of the system.

Skill Repository. We maintain an external skill repository $\mathcal{S}_{t}$ at time stamp $t$ , which consists of $N_{t}$ reusable skills $\mathcal{S}_{t}=\{s_{t}^{1},s_{t}^{2},\dots,s_{t}^{N_{t}}\}$ . Following the widely adopted SKILL.md format (Anthropic, 2025b), each skill is represented as a single Markdown file with two components as shown in Figure 1(b): (i) YAML frontmatter, which specifies the skill name and a natural-language description of when the skill should be used, and (ii) Markdown instructions, which describe the executable knowledge, workflows, constraints, and reusable heuristics captured by the skill.

Agent Executor. Given a task $x_{t}$ , a frozen agent executor $\pi_{\mathcal{L}}$ solves the task conditioning on the current environment observation and relevant skills. Specifically, we retrieve a subset of skills $\tilde{\mathcal{S}}_{t}\subseteq\mathcal{S}_{t}$ using BM25 (Robertson and Zaragoza, 2009) for each task $x_{t}$ , and the executor samples actions following $a\sim\pi_{\mathcal{L}}(\cdot\mid x_{t},o_{t},\tilde{\mathcal{S}}_{t})$ .

Skill Curator. After the executor completes task $x_{t}$ , the skill curator $\pi_{\mathcal{S}}$ observes the trajectory $\xi_{t}$ , the self-judged correctness of the answers/interactions $\mathbbm{1}_{\xi_{t}}$ , and a retrieved subset of related skills $\tilde{\mathcal{S}}_{t}$ . It then generates a sequence of structured curation operations $c_{t}=(u_{t}^{1},\dots,u_{t}^{M_{t}})\sim\pi_{\mathcal{S}}(\cdot\mid\xi_{t},\mathbbm{1}_{\xi_{t}},\tilde{\mathcal{S}}_{t})$ , where each operation $u_{t}^{m}$ is one of $\{\hbox to48.81pt{\vbox to8.86pt{\pgfpicture\makeatletter\hbox{\hskip 0.0pt\lower 0.0pt\hbox to0.0pt{\lxSVG@begingroup@{_scopebegin=1} \lxSVG@begingroup@{stroke=#000000} \lxSVG@begingroup@{fill=#000000} \lxSVG@setlinewidth{\the\pgflinewidth}\lxSVG@begingroup@{stroke-width=0.4pt} \lx@inpgf@ignorespaces\nullfont\hbox to0.0pt{{}{}{}{}\lx@inpgf@ignorespaces\lxSVG@begingroup@{_scopebegin=1} {}{}{}{}{}{}{}{}\lxSVG@begingroup@{fill=#404040} \lxSVG@fill@opacity{1.0}\lxSVG@begingroup@{fill-opacity=1.0} {{}{}{{}}}{{}{}{{}}}{}{}{{}{}{{}}}{{}{}{{}}}{}{}{{}{}{{}}}{{}{}{{}}}{}{}{{}{}{{}}}{{}{}{{}}}{}{}\lxSVG@fill\lxSVG@drawpath@unclipped{M 0 1.38 L 0 10.88 C 0 11.64 0.62 12.26 1.38 12.26 L 66.15 12.26 C 66.91 12.26 67.53 11.64 67.53 10.88 L 67.53 1.38 C 67.53 0.62 66.91 0 66.15 0 L 1.38 0 C 0.62 0 0 0.62 0 1.38 Z}{stroke:none} \lx@inpgf@ignorespaces\lxSVG@closescope \lxSVG@begingroup@{_scopebegin=1} {}{}{}{}{}{}{}{}\lxSVG@begingroup@{fill=#FAE6F1} \lxSVG@fill@opacity{1.0}\lxSVG@begingroup@{fill-opacity=1.0} {{}{}{{}}}{{}{}{{}}}{}{}{{}{}{{}}}{{}{}{{}}}{}{}{{}{}{{}}}{{}{}{{}}}{}{}{{}{}{{}}}{{}{}{{}}}{}{}\lxSVG@fill\lxSVG@drawpath@unclipped{M 0 1.38 L 0 10.88 C 0 11.64 0.62 12.26 1.38 12.26 L 66.15 12.26 C 66.91 12.26 67.53 11.64 67.53 10.88 L 67.53 1.38 C 67.53 0.62 66.91 0 66.15 0 L 1.38 0 C 0.62 0 0 0.62 0 1.38 Z}{stroke:none} \lx@inpgf@ignorespaces\lxSVG@closescope \lxSVG@begingroup@{_scopebegin=1} \lxSVG@fill@opacity{1.0}\lxSVG@begingroup@{fill-opacity=1.0} {{{\lx@inpgf@ignorespaces}}{{\lx@inpgf@ignorespaces}}{{\lx@inpgf@ignorespaces}}{{}}{{}}{{}}{{}}\lxSVG@begingroup@{_scopebegin=1} \lxSVG@transformcm{1.0}{0.0}{0.0}{1.0}{4.0pt}{2.0pt}\lxSVG@begingroup@{transform=matrix(1.0 0.0 0.0 1.0 5.53 2.77)} \pgfsys@hbox{58}\lxSVG@closescope }\lxSVG@closescope {\lx@inpgf@ignorespaces}{\lx@inpgf@ignorespaces}{\lx@inpgf@ignorespaces}\hss}\lxSVG@discardpath\lxSVG@closescope \hss}}\lxSVG@closescope\endpgfpicture}},\hbox to53.65pt{\vbox to10.22pt{\pgfpicture\makeatletter\hbox{\hskip 0.0pt\lower 0.0pt\hbox to0.0pt{\lxSVG@begingroup@{_scopebegin=1} \lxSVG@begingroup@{stroke=#000000} \lxSVG@begingroup@{fill=#000000} \lxSVG@setlinewidth{\the\pgflinewidth}\lxSVG@begingroup@{stroke-width=0.4pt} \lx@inpgf@ignorespaces\nullfont\hbox to0.0pt{{}{}{}{}\lx@inpgf@ignorespaces\lxSVG@begingroup@{_scopebegin=1} {}{}{}{}{}{}{}{}\lxSVG@begingroup@{fill=#404040} \lxSVG@fill@opacity{1.0}\lxSVG@begingroup@{fill-opacity=1.0} {{}{}{{}}}{{}{}{{}}}{}{}{{}{}{{}}}{{}{}{{}}}{}{}{{}{}{{}}}{{}{}{{}}}{}{}{{}{}{{}}}{{}{}{{}}}{}{}\lxSVG@fill\lxSVG@drawpath@unclipped{M 0 1.38 L 0 12.76 C 0 13.52 0.62 14.14 1.38 14.14 L 72.85 14.14 C 73.61 14.14 74.23 13.52 74.23 12.76 L 74.23 1.38 C 74.23 0.62 73.61 0 72.85 0 L 1.38 0 C 0.62 0 0 0.62 0 1.38 Z}{stroke:none} \lx@inpgf@ignorespaces\lxSVG@closescope \lxSVG@begingroup@{_scopebegin=1} {}{}{}{}{}{}{}{}\lxSVG@begingroup@{fill=#E3F0E7} \lxSVG@fill@opacity{1.0}\lxSVG@begingroup@{fill-opacity=1.0} {{}{}{{}}}{{}{}{{}}}{}{}{{}{}{{}}}{{}{}{{}}}{}{}{{}{}{{}}}{{}{}{{}}}{}{}{{}{}{{}}}{{}{}{{}}}{}{}\lxSVG@fill\lxSVG@drawpath@unclipped{M 0 1.38 L 0 12.76 C 0 13.52 0.62 14.14 1.38 14.14 L 72.85 14.14 C 73.61 14.14 74.23 13.52 74.23 12.76 L 74.23 1.38 C 74.23 0.62 73.61 0 72.85 0 L 1.38 0 C 0.62 0 0 0.62 0 1.38 Z}{stroke:none} \lx@inpgf@ignorespaces\lxSVG@closescope \lxSVG@begingroup@{_scopebegin=1} \lxSVG@fill@opacity{1.0}\lxSVG@begingroup@{fill-opacity=1.0} {{{\lx@inpgf@ignorespaces}}{{\lx@inpgf@ignorespaces}}{{\lx@inpgf@ignorespaces}}{{}}{{}}{{}}{{}}\lxSVG@begingroup@{_scopebegin=1} \lxSVG@transformcm{1.0}{0.0}{0.0}{1.0}{4.0pt}{3.3611pt}\lxSVG@begingroup@{transform=matrix(1.0 0.0 0.0 1.0 5.53 4.65)} \pgfsys@hbox{58}\lxSVG@closescope }\lxSVG@closescope {\lx@inpgf@ignorespaces}{\lx@inpgf@ignorespaces}{\lx@inpgf@ignorespaces}\hss}\lxSVG@discardpath\lxSVG@closescope \hss}}\lxSVG@closescope\endpgfpicture}},\hbox to49.77pt{\vbox to8.86pt{\pgfpicture\makeatletter\hbox{\hskip 0.0pt\lower 0.0pt\hbox to0.0pt{\lxSVG@begingroup@{_scopebegin=1} \lxSVG@begingroup@{stroke=#000000} \lxSVG@begingroup@{fill=#000000} \lxSVG@setlinewidth{\the\pgflinewidth}\lxSVG@begingroup@{stroke-width=0.4pt} \lx@inpgf@ignorespaces\nullfont\hbox to0.0pt{{}{}{}{}\lx@inpgf@ignorespaces\lxSVG@begingroup@{_scopebegin=1} {}{}{}{}{}{}{}{}\lxSVG@begingroup@{fill=#404040} \lxSVG@fill@opacity{1.0}\lxSVG@begingroup@{fill-opacity=1.0} {{}{}{{}}}{{}{}{{}}}{}{}{{}{}{{}}}{{}{}{{}}}{}{}{{}{}{{}}}{{}{}{{}}}{}{}{{}{}{{}}}{{}{}{{}}}{}{}\lxSVG@fill\lxSVG@drawpath@unclipped{M 0 1.38 L 0 10.88 C 0 11.64 0.62 12.26 1.38 12.26 L 67.48 12.26 C 68.25 12.26 68.87 11.64 68.87 10.88 L 68.87 1.38 C 68.87 0.62 68.25 0 67.48 0 L 1.38 0 C 0.62 0 0 0.62 0 1.38 Z}{stroke:none} \lx@inpgf@ignorespaces\lxSVG@closescope \lxSVG@begingroup@{_scopebegin=1} {}{}{}{}{}{}{}{}\lxSVG@begingroup@{fill=#FDF3E0} \lxSVG@fill@opacity{1.0}\lxSVG@begingroup@{fill-opacity=1.0} {{}{}{{}}}{{}{}{{}}}{}{}{{}{}{{}}}{{}{}{{}}}{}{}{{}{}{{}}}{{}{}{{}}}{}{}{{}{}{{}}}{{}{}{{}}}{}{}\lxSVG@fill\lxSVG@drawpath@unclipped{M 0 1.38 L 0 10.88 C 0 11.64 0.62 12.26 1.38 12.26 L 67.48 12.26 C 68.25 12.26 68.87 11.64 68.87 10.88 L 68.87 1.38 C 68.87 0.62 68.25 0 67.48 0 L 1.38 0 C 0.62 0 0 0.62 0 1.38 Z}{stroke:none} \lx@inpgf@ignorespaces\lxSVG@closescope \lxSVG@begingroup@{_scopebegin=1} \lxSVG@fill@opacity{1.0}\lxSVG@begingroup@{fill-opacity=1.0} {{{\lx@inpgf@ignorespaces}}{{\lx@inpgf@ignorespaces}}{{\lx@inpgf@ignorespaces}}{{}}{{}}{{}}{{}}\lxSVG@begingroup@{_scopebegin=1} \lxSVG@transformcm{1.0}{0.0}{0.0}{1.0}{4.0pt}{2.0pt}\lxSVG@begingroup@{transform=matrix(1.0 0.0 0.0 1.0 5.53 2.77)} \pgfsys@hbox{58}\lxSVG@closescope }\lxSVG@closescope {\lx@inpgf@ignorespaces}{\lx@inpgf@ignorespaces}{\lx@inpgf@ignorespaces}\hss}\lxSVG@discardpath\lxSVG@closescope \hss}}\lxSVG@closescope\endpgfpicture}}\}$ . Each operation is implemented as a function call (detailed signature in Figure 8) that manipulates the skill repository $\mathcal{S}_{t}$ . Applying these operations transforms the repository from $\mathcal{S}_{t}$ to $\mathcal{S}_{t+1}$ as $\mathcal{S}_{t+1}=\textsc{ApplyOps}(\mathcal{S}_{t},c_{t})$ . The updated repository is then used by the executor on subsequent tasks, forming a closed loop between task execution and experience-driven skill evolution.

We optimize the skill curator $\pi_{\mathcal{S}}$ with RL and keep the agent executor $\pi_{\mathcal{L}}$ frozen. The main challenge is indirect and delayed feedback for curation decisions, which is only revealed through $\pi_{\mathcal{L}}$ ’s performance on future relevant tasks. We address this by constructing grouped training instances (§ 3.2.1) and designing a composite reward (§ 3.2.2) that combines future task outcomes with intermediate signals on operation validity, skill quality, and the conciseness of skills. An overview of the training process is shown in Figure 2.

Figure 2: SkillOS training pipeline. Each training step samples a group of related tasks and initializes an empty SkillRepo. $\pi_{\mathcal{S}}$ is optimized with composite rewards, enabling self-evolution.

To provide downstream learning signals for skill curation, we construct each training instance as a group of related tasks that are solved sequentially. Within each group, SkillRepo is updated by the curator $\pi_{\mathcal{s}}$ after each task, allowing skills derived from earlier experiences to be evaluated by whether they help solve related future tasks. This also differs from prior work that focuses on short-horizon transfer (Wang et al., 2025a; Ye et al., 2026), where our grouped formulation exposes the curator to longer skill-evolution trajectories and provides denser feedback for learning complex curation operations.

Concretely, for each task $x_{i}$ in $\mathcal{D}=\{x_{i}\}_{i=1}^{N}$ , we first annotate each instance with a set of skill-relevant attributes. Formally, for each $x_{i}$ , we use Gemini-2.5-Pro (Team, 2025a) to produce a set of tags:

Z_{i}=\{z_{i}^{1},z_{i}^{2},\dots,z_{i}^{|Z_{i}|}\},

where each attribute $z_{i}$ captures a salient aspect of the task $x_{i}$ , such as topic and common pitfalls. For example, in mathematical reasoning, attributes may include labels such as “algebra” or “Fourier transformation”. These attributes serve as proxies for task-relatedness and potential skill dependency.

Based on the annotated attributes, we then partition $\mathcal{D}$ into a collection of $M$ task groups using the similarity of attributes of these data samples:

\mathcal{D}=\{G_{1},G_{2},\dots,G_{M}\},\qquad G_{m}=\{x_{m,1},x_{m,2},\dots,x_{m,|G_{m}|}\},

where all instances within the same group $G_{m}$ exhibit non-trivial dependency in terms of required skills. Detailed description of data processing and grouping algorithms can be found in Appendix B.2.

We employ Grouped Reward Policy Optimization (GRPO Shao et al. (2024)) for its training stability and sample efficiency. The training loop shown in Algorithm 1 optimizes the skill curator policy $\pi_{\mathcal{S}}$ to maximize a composite reward function over the distribution of generated traces. For a task group $G=(x_{1},\dots,x_{|G|})$ , the curator produces a sequence of curation decisions $c=(c_{1},\dots,c_{|G|})$ as the executor proceeds through the group. Each training step, the reward combines four signals:

r\;=\;\underbrace{r^{\text{task}}}_{\text{task outcome}}+\;\lambda_{\mathrm{f}}\underbrace{r^{\text{fc}}}_{\text{function call}}+\;\lambda_{\mathrm{u}}\underbrace{r^{\text{cnt}}}_{\text{content quality}}+\;\lambda_{\mathrm{c}}\underbrace{r^{\text{comp}}}_{\text{compression}}

Task outcome reward. The first task uses an empty SkillRepo, before any curator update occurs. We thus define the task outcome reward as the average success over the remaining tasks as $r^{\text{task}}=\frac{1}{|G|-1}\sum_{i=2}^{|G|}\mathbbm{1}(\xi_{i})$ , which provides executor-grounded signal on downstream performance achieved by the evolving SkillRepo from $\pi_{\mathcal{S}}$ .

Function call reward. The function call reward measures whether the curator produces valid skill operations. For each curation decision $c_{i}$ , let $\mathrm{Valid}(c_{i})$ be the fraction of generated function calls that are valid and successfully executed. We define the function call reward as $r^{\text{fc}}=\frac{1}{|G|}\sum_{i=1}^{|G|}\mathrm{Valid}(c_{i})$ .

Algorithm 1 Training Skill Curator with Task Groups using GRPO

1: for each training step do

G=(x_{1},\dots,x_{|G|})

\mathcal{S}\leftarrow\emptyset

\triangleright

Sample a task group and initialize SkillRepo

3: for task index

i=1,\dots,|G|

\tilde{\mathcal{S}}\leftarrow\textsc{BM25}\!\left(x_{i},\;\mathcal{S}\right)

\triangleright

Retrieve relevant skills

\xi_{i}\leftarrow\textsc{RunTask}\!\left(\tilde{\mathcal{S}},\;\pi_{\mathcal{L}},\;x_{i}\right)

\triangleright

Run inference on frozen executor

c_{i}\sim\pi_{\mathcal{S}}\!\left(\cdot\;\middle|\;\xi_{i},\tilde{\mathcal{S}}\right)

\triangleright

Sample a rollout from skill curator

\mathcal{S}\leftarrow\textsc{ApplyOps}\!\left(\mathcal{S},\;c_{i}\right)

\triangleright

Apply insert/update/delete

8: end for

r\leftarrow\textsc{CalculateReward}(\xi,c)

10:

\textsc{Update}\ \pi_{\mathcal{S}}

\triangleright

Update skill curator using GRPO

11: end for

Compression reward. To discourage verbatim trajectory copying, we reward concise repository updates. Let $\mathcal{S}_{i}$ denote the skill repository after applying $c_{i}$ , and let $\chi_{i}$ denote the curator input context at position $i$ . We define $r^{\text{comp}}=\frac{1}{|G|}\sum_{i=1}^{|G|}\left(1-\frac{|\mathcal{S}_{i}|}{|\chi_{i}|}\right)$ , where $|\mathcal{S}_{i}|$ and $|\chi_{i}|$ denote token lengths. This encourages the curator to distill reusable skills rather than store raw trajectories.

Content quality reward. The content quality reward evaluates whether the curated skills are semantically meaningful and likely to be useful for future tasks. Let $\mathrm{Judge}(c_{i})$ denote the scalar score assigned by an external judge (Qwen3-32B) $c_{i}$ , we compute the reward as $r^{\text{cnt}}=\frac{1}{|G|}\sum_{i=1}^{|G|}\mathrm{Judge}(c_{i})$ .

For each task group $G$ , we sample $N$ independent rollouts of the entire curation sequence from $\pi_{\mathcal{S}}$ . Within each rollout, the executor produces trajectory $\xi_{i}$ using the skill repository $\mathcal{S}^{i}$ resulting from previous curations $c_{<i}$ till task position $i$ with the same training task group, so different rollouts evolve different repository histories. The GRPO advantage is computed as: $A^{n}=r^{n}-\frac{1}{N}\sum_{n^{\prime}=1}^{N}r^{n^{\prime}},$ where $r^{n}$ is the composite reward (Eq. 1) for the $n$ -th rollout. We optimize $\pi_{\mathcal{S}}$ with a clipped surrogate objective over all curation steps $i=1,\ldots,|G|$ :

\mathcal{L}=\mathbb{E}_{n}\!\left[\min\!\left(\rho^{n}\,A^{n},\;\mathrm{clip}\!\left(\rho^{n},\,1{-}\epsilon,\,1{+}\epsilon\right)\!A^{n}\right)\right]

where $\rho^{n}=\pi_{\mathcal{S}}(c^{n}\mid\chi)\,/\,\pi_{\theta_{old}}(c^{n}\mid\chi)$ is the importance ratio. The advantage $A^{n}$ is assigned uniformly to all tokens in $c^{n}$ , and we discard the KL term in GRPO to encourage policy exploration.

We conduct experiments on both multi-turn agentic tasks and single-turn reasoning tasks, in line with prior work (Xia et al., 2026; Wei et al., 2025; Ye et al., 2026). We additionally show that the trained skill curator transfers across agent executors and task domains, highlighting its flexibility and generalizability.

Original Table 1 — Table 1: Experiment results on ALFWorld benchmark. Success rate (SR $\uparrow$ ) and the number of steps (Steps $\downarrow$ ) are reported on 6 subsets with 3 different frozen executors.

Table 1
Methods	Curator	Pick	Look	Clean	Heat	Cool	Pick2	Avg. SR	Steps
$\pi_{\mathcal{S}}$	(35)	(13)	(27)	(16)	(25)	(24)	(140)
Executor $\pi_{\mathcal{L}}$ : Qwen3-8B
No Memory	None	$78.1_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 1.6}}$	$46.2_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 7.7}}$	$33.3_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 13.4}}$	$37.5_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 10.8}}$	$29.3_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 6.1}}$	$47.2_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 6.4}}$	$47.9_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 1.2}}$	21.1
ReasoningBank	Qwen3-8B	$83.8_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 0.0}}$	$48.7_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 7.2}}$	$49.4_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 16.2}}$	$39.6_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 4.4}}$	$41.3_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 8.5}}$	$54.2_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 8.8}}$	$55.7_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 3.1}}$	20.1
MemP	Qwen3-8B	$80.0_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 5.7}}$	$43.6_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 4.4}}$	$24.7_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 4.3}}$	$33.3_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 3.6}}$	$38.7_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 6.1}}$	$48.6_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 6.4}}$	$49.7_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 0.7}}$	21.0
SkillOS-base	Qwen3-8B	$79.0_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 8.7}}$	$41.0_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 4.4}}$	$45.7_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 4.3}}$	$37.5_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 9.5}}$	$38.7_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 4.0}}$	$55.6_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.1}}$	$53.1_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.5}}$	20.4
SkillOS-gemini	Gemini-2.5-Pro	$77.1_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 6.0}}$	$53.8_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 6.1}}$	$37.0_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 6.4}}$	$37.5_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 9.5}}$	$36.0_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 3.2}}$	$50.0_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 6.7}}$	$50.7_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 3.6}}$	20.8
SkillOS	Qwen3-8B	$\textbf{85.7}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 3.3}}$	$\textbf{56.4}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 7.7}}$	$\textbf{54.3}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 8.6}}$	$\textbf{43.8}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 9.5}}$	$\textbf{46.7}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.3}}$	$\textbf{62.5}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 6.4}}$	$\textbf{61.2}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 4.6}}$	18.9
Executor $\pi_{\mathcal{L}}$ : Qwen3-32B
No Memory	None	$80.0_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.9}}$	$69.2_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 0.0}}$	$45.6_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 7.7}}$	$37.5_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 16.5}}$	$42.7_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 6.1}}$	$43.1_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.4}}$	$54.5_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.5}}$	20.3
ReasoningBank	Qwen3-8B	$86.7_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 3.0}}$	$71.8_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 5.4}}$	$50.6_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 6.3}}$	$45.8_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 13.3}}$	$52.0_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 8.9}}$	$51.4_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 5.1}}$	$61.4_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.5}}$	18.7
MemP	Qwen3-8B	$80.0_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.9}}$	$76.9_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 0.0}}$	$44.4_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 7.4}}$	$37.5_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 10.8}}$	$42.7_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.3}}$	$47.2_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 6.4}}$	$55.7_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 3.7}}$	20.0
SkillOS-base	Qwen3-8B	$82.9_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.9}}$	$69.2_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 11.8}}$	$48.1_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.1}}$	$50.0_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 9.7}}$	$48.0_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 14.4}}$	$52.8_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 11.0}}$	$59.8_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 3.0}}$	19.2
SkillOS-gemini	Gemini-2.5-Pro	$\textbf{97.1}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 3.0}}$	$76.9_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 5.4}}$	$55.6_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 6.0}}$	$43.8_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 11.3}}$	$40.0_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 5.7}}$	$54.2_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 4.9}}$	$63.6_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 4.2}}$	18.1
SkillOS	Qwen3-8B	$91.4_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 3.3}}$	$\textbf{76.9}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 4.4}}$	$\textbf{59.3}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 8.6}}$	$\textbf{56.3}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 12.5}}$	$\textbf{57.3}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 10.1}}$	$\textbf{62.5}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 4.2}}$	$\textbf{68.6}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 5.7}}$	17.3
Executor $\pi_{\mathcal{L}}$ : Gemini-2.5-pro
No Memory	None	$90.5_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 3.2}}$	$66.7_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 5.1}}$	$48.1_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 10.2}}$	$39.6_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 17.1}}$	$68_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 7.4}}$	$68.1_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 3.8}}$	$66.4_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.0}}$	17.7
ReasoningBank	Qwen3-8B	$91.4_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 3.4}}$	$61.5_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 4.1}}$	$63.0_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 9.3}}$	$39.6_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 10.3}}$	$70.7_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 3.2}}$	$76.4_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 8.5}}$	$71.4_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.9}}$	16.0
MemP	Qwen3-8B	$95.2_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.1}}$	$\textbf{74.4}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 6.8}}$	$61.7_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 7.6}}$	$56.3_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 12.4}}$	$76.0_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 6.2}}$	$68.1_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 8.5}}$	$74.3_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 3.4}}$	15.2
SkillOS-base	Qwen3-8B	$91.4_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 1.6}}$	$69.2_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 7.7}}$	$56.8_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 5.7}}$	$54.2_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 13.7}}$	$72.0_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 4.0}}$	$66.7_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 11.0}}$	$70.7_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 3.0}}$	16.3
SkillOS-gemini	Gemini-2.5-Pro	$94.3_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 5.7}}$	$69.2_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 0.0}}$	$\textbf{77.8}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 5.7}}$	$\textbf{75.0}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 16.5}}$	$80.0_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 12.2}}$	$66.7_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.4}}$	$79.3_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.6}}$	14.9
SkillOS	Qwen3-8B	$\textbf{95.2}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.9}}$	$71.8_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 7.7}}$	$74.1_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 13.0}}$	$72.9_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 10.1}}$	$\textbf{77.3}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 6.1}}$	$\textbf{77.8}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 10.0}}$	$\textbf{80.2}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 3.1}}$	14.8

We briefly discuss the experiment setup throughout this paper. Full description of datasets, implementations, baselines, and evaluations can be found in Appendix B.

Dataset. For agentic tasks, we conduct experiments on ALFWorld (Shridhar et al., 2021) and WebShop (Yao et al., 2022). ALFWorld is a text-based interactive environment aligned with the ALFRED embodied AI benchmark, where agents must complete household tasks through textual navigation and object manipulation. WebShop simulates an online shopping environment in which agents navigate a realistic web interface to identify and purchase products that satisfy user-specified requirements. For each benchmark, we train SkillOS on its training split where $Z_{i}$ is the default task type annotations, and evaluate on the corresponding test set. In addition to agentic tasks, we also benchmark for single-turn reasoning tasks, including AIME24, AIME25, and GPQA-Diamond (Rein et al., 2024). Training data are constructed from DeepMath-103k (He et al., 2026b), where we randomly sample a subset of 33,000 data points.

Evaluation Configurations. We evaluate all methods across two dimensions, effectiveness and efficiency. For effectiveness, we measure the success rate (SR) and accuracy for agentic tasks and reasoning tasks, respectively. For efficiency, we compute the number of execution steps per agentic task and the number of tokens per reasoning problem, respectively. We compare SkillOS with three categories of baselines: (i) a memory-free agent (No Memory); (ii) existing memory-based methods, including ReasoningBank (Ouyang et al., 2026), which distills reusable insights from past experiences, and MemP (Fang et al., 2025b), which induces procedural memory with advanced memory-management strategies; and (iii) internal variants of our framework, including SkillOS-base, which uses the initial skill curator without RL training, and SkillOS-gemini, which uses Gemini-2.5-Pro to directly perform skill curation instead of learning the curator with RL. All prompts used can be found in Appendix A.

Implementation Details. We use Qwen3-8B (Team, 2025b) as the base model for $\pi_{\mathcal{S}}$ . The frozen executor is also instantiated with Qwen3-8B during training. We train our model using GRPO with a learning rate $1\times 10^{-6}$ , batch size $32$ , and group size $8$ . Training is conducted on 16 H100 GPUs using the verl framework (Sheng et al., 2024). The full training process takes approximately 3 days for ALFWorld, 2.5 days for reasoning tasks, and 5 days for WebShop. For testing, we additionally include Qwen3-32B, Gemini-2.5-Pro (Team, 2025a), and Gemini-3.1-Flash-Lite (Appendix C.1) as executors to evaluate the generalization of SkillOS under different executor scales and architectures. Task outcome signal $\mathbbm{1}_{\xi_{t}}$ is obtained via LLM-as-a-judge with the frozen agent executor (prompt shown in Appendix A). We use ReAct (Yao et al., 2023) for agent execution and CoT (Wei et al., 2022a) for reasoning tasks. For the reward function, we set $\lambda_{f}=1.0$ , $\lambda_{u}=0.1$ , and $\lambda_{c}=0.05$ . We report averaged performance and standard deviation over 3 runs.

Original Table 2 — Table 2: Experiment results on WebShop and single-turn reasoning tasks for 3 different frozen executors. For WebShop, the averaged score, success rate (SR $\uparrow$ ), and the number of steps (Steps $\downarrow$ ) are reported. For reasoning tasks, accuracy (Acc. $\uparrow$ ) is reported on three datasets.

Table 2
Methods	Curator	WebShop	Reasoning
$\pi_{\mathcal{S}}$	Score	SR	Steps	AIME24	AIME25	GPQA	Avg. Acc
Executor $\pi_{\mathcal{L}}$ : Qwen3-8B
No Memory	None	$33.3_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 0.7}}$	$9.8_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 0.5}}$	20.3	$76.0_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 6.9}}$	$71.1_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 10.7}}$	$61.8_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 1.1}}$	$69.6_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 4.7}}$
ReasoningBank	Qwen3-8B	$35.4_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 1.1}}$	$11.4_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 0.9}}$	20.5	$75.4_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 5.0}}$	$73.2_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 10.8}}$	$60.3_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 3.9}}$	$69.6_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.5}}$
MemP	Qwen3-8B	$35.7_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 0.9}}$	$12.0_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 0.5}}$	21.3	$75.6_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 5.1}}$	$71.1_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 5.1}}$	$60.6_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 4.0}}$	$69.1_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 4.0}}$
SkillOS-base	Qwen3-8B	$38.6_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 0.9}}$	$13.6_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 0.8}}$	20.1	$75.6_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 5.1}}$	$71.9_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 6.9}}$	$59.3_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.5}}$	$68.9_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.6}}$
SkillOS-gemini	Gemini-2.5-pro	$38.1_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 1.0}}$	$13.2_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 0.9}}$	19.6	$73.3_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 1.3}}$	$71.3_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 1.9}}$	$57.6_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.8}}$	$67.4_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 0.8}}$
SkillOS	Qwen3-8B	$\textbf{40.6}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 0.7}}$	$\textbf{16.5}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 0.7}}$	19.4	$\textbf{80.0}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 3.3}}$	$\textbf{76.7}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 5.8}}$	$\textbf{64.6}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 1.3}}$	$\textbf{73.8}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 1.8}}$
Executor $\pi_{\mathcal{L}}$ : Qwen3-32B
No Memory	None	$41.5_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 0.5}}$	$12.2_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 0.3}}$	17.0	$81.4_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 1.3}}$	$72.2_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 3.8}}$	$68.4_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.0}}$	$74.0_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 1.9}}$
ReasoningBank	Qwen3-32B	$40.4_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 0.8}}$	$11.2_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 1.1}}$	17.9	$81.1_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 9.6}}$	$75.6_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 5.9}}$	$66.9_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 1.2}}$	$74.9_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.2}}$
MemP	Qwen3-32B	$30.7_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 0.7}}$	$10.1_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 0.6}}$	17.4	$82.2_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 5.1}}$	$76.7_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 0.0}}$	$66.5_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.3}}$	$75.1_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.1}}$
SkillOS-base	Qwen3-8B	$43.4_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 0.8}}$	$12.3_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 1.0}}$	16.8	$80.0_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 3.3}}$	$75.6_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 10.2}}$	$67.7_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 1.5}}$	$74.7_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 3.3}}$
SkillOS-gemini	Gemini-2.5-pro	$45.2_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 1.0}}$	$13.2_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 1.1}}$	16.6	$77.8_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 6.7}}$	$74.4_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 1.9}}$	$66.2_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 0.6}}$	$73.2_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.6}}$
SkillOS	Qwen3-8B	$\textbf{49.2}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 1.2}}$	$\textbf{16.5}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 0.6}}$	15.9	$\textbf{85.6}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 1.9}}$	$\textbf{81.1}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 3.3}}$	$\textbf{72.4}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 3.0}}$	$\textbf{79.7}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 1.6}}$
Executor $\pi_{\mathcal{L}}$ : Gemini-2.5-pro
No Memory	None	$48.6_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 0.3}}$	$38.4_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 0.5}}$	19.5	$85.6_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 1.9}}$	$80.0_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 6.7}}$	$79.9_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 1.5}}$	$81.8_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.8}}$
ReasoningBank	Gemini-2.5-pro	$50.8_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 1.5}}$	$40.2_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 1.3}}$	19.2	$85.6_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 5.1}}$	$84.4_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 6.7}}$	$80.4_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.1}}$	$83.5_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.1}}$
MemP	Gemini-2.5-pro	$51.3_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 1.2}}$	$39.8_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 1.0}}$	19.4	$83.3_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 6.9}}$	$76.7_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 5.8}}$	$81.8_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 3.4}}$	$80.6_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 3.2}}$
SkillOS-base	Qwen3-8B	$52.8_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 1.0}}$	$39.6_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 0.8}}$	19.0	$87.8_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 3.3}}$	$83.3_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 1.9}}$	$82.8_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.7}}$	$84.6_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 1.8}}$
SkillOS-gemini	Gemini-2.5-pro	$54.7_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 1.0}}$	$41.0_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 1.2}}$	17.8	$90.0_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 5.1}}$	$85.6_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 7.7}}$	$80.7_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 5.5}}$	$85.4_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 3.5}}$
SkillOS	Qwen3-8B	$\textbf{56.0}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 0.7}}$	$\textbf{41.3}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 0.8}}$	18.3	$\textbf{92.2}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.4}}$	$\textbf{86.7}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 3.5}}$	$\textbf{86.8}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.1}}$	$\textbf{88.6}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 1.5}}$

Tables 1 and 2 summarize the results for different benchmarks with Qwen3-8B as the skill curator on various agent executors. Based on the results, we have the following observations.

SkillOS achieves strong performance gains across benchmarks. Across all three benchmarks, SkillOS consistently outperforms both memory-free and memory-based baselines, showing that the gains come from learning to manage and evolve skills rather than from maintaining a static collection. On ALFWorld, SkillOS improves the average success rate from 55.7 to 61.2 over the strongest baseline ReasoningBank with Qwen3-8B as the executor; similar trends hold on WebShop and reasoning tasks. Strikingly, our RL-trained 8B curator even surpasses SkillOS-gemini, despite the latter using a far stronger frontier model as the curator, demonstrating that targeted training of a small curator can outweigh raw model scale. The benefits brought by RL training are also compounded with executor capacity, yielding $+9.5$ absolute improvement with Gemini-2.5-Pro versus $+7.9$ with Qwen3-8B for ALFworld, compared with SkillOS-base.

SkillOS is more efficient, requiring fewer interactions and lower execution cost. The gains of SkillOS are accompanied by better efficiency rather than longer trajectories. On ALFWorld, SkillOS reduces the average interaction steps by $2.2$ , $3.0$ , and $3.1$ compared with “no memory” setting with 3 executors, consistently outperforming all memory-based baselines. This trend extends to WebShop, where SkillOS secures higher success rates with fewer environment interactions. These results indicate that the learned skill manager enables the executor to identify procedural shortcuts and bypass redundant exploration. Rather than relying on additional trial-and-error, SkillOS improves performance by distilling experience into direct, actionable expertise that simplifies task execution.

The gains differ between agentic and reasoning tasks, reflecting different forms of reusable skills. A notable trend is that the gains of SkillOS are generally larger on multi-turn agentic benchmarks than on single-turn reasoning tasks. We hypothesize that this difference arises from how reusable skills manifest across task types. Agentic tasks naturally expose procedural regularities, such as action ordering, exploration strategies, recovery behaviors, and environment-specific constraints, which can be repeatedly composed and refined across task streams. Reasoning tasks also benefit from skill curation, but their reusable knowledge often appears at a more abstract level, such as decomposition heuristics, constraint formulation, or verification patterns, rather than as directly reusable action procedures. As a result, SkillOS still improves reasoning performance, while the gains are typically smaller than those observed on agentic benchmarks. We provide a case study demonstrating skills curated for different tasks in Figure 17.

Figure 3: Cross-task generalization results of SkillOS with (a) Qwen3-8B, (b) Qwen3-32B, and (c) Gemini-2.5-Pro as frozen executors. We plot relative improvement with baselines from **least** to **most**.

SkillOS is transferable and remains effective for different agent executors. During training, we use Qwen3-8B as the executor. To test whether SkillOS brings improvement for executors that are not seen in training, we pair the trained skill curator with different executors. As shown in Table 1 and 2, SkillOS consistently improves a wide range of frozen executors across benchmarks, from open-source models (Qwen3-8B, Qwen3-32B) to frontier models (Gemini-2.5-Pro). On ALFWorld, it lifts the average success rate of Qwen3-8B from 47.9 to 61.2 and Gemini-2.5-Pro from 66.4 to 80.2, demonstrating compatibility with executors of varying capacity. Notably, using Gemini-2.5-Pro directly as the curator (SkillOS-gemini) underperforms our trained curator, especially when paired with the smaller Qwen3-8B executor. This highlights a curator-executor mismatch: stronger reasoning ability alone does not guarantee effective skill curation, as frontier-generated skills may be misaligned with the executor’s capacity or usage patterns. By contrast, SkillOS learns executor-grounded curation behaviors through RL, producing skills that better match the downstream agent.

SkillOS delivers consistent performance improvement when generalized to different task domains. Figure 3 shows that the skill curator learned by SkillOS transfers well across different tasks. While training and testing on the same task often gives the strongest gain, most off-diagonal entries still bring performance improvement over baselines, indicating that SkillOS captures reusable skills beyond task-specific heuristics. Specifically, skill curator $\pi_{s}$ learned from reasoning tasks transfer particularly well to the two agentic tasks, likely because they contain more abstract and high-level strategies, such as decomposition, verification, and adaptive planning, which are broadly useful across settings. In contrast, skills learned from WebShop or ALFWorld are more tied to environment-specific knowledge, making them less transferable across tasks.

Beyond performance, we analyze why SkillOS works, focusing on design choices, evolution of curator’s behaviors and contents in SkillRepo, and the role of curated skills in task success. Additional analyses are included in Appendix C.

Original Table 3 — Table 3: Ablation results of reward design on the ALFWorld dataset.

Table 3
Methods	Avg. SR	Steps
SkillOS-GRPO	61.2	18.9
w/o $r^{cnt}$	58.6	20.1
w/o $r^{comp}$	60.0	19.3
w/o grouping	57.3	20.6

We ablate two components of SkillOS: (i) auxiliary rewards in Eq. 1, and (ii) grouped task streams in § 3.2.1. Experiments are conducted on ALFWorld, with Qwen3-8B used as both the curator and executor. As shown in Table 3, removing either reward component hurts performance. Without the content-quality reward, the success rate drops from 61.2 to 58.6, showing the importance of intermediate supervision for guiding skill updates in a pipelined system. Removing the compression reward causes a smaller but consistent drop, suggesting that concise repositories are easier for the executor to use. The most significant degradation comes from using random task sequences (w/o grouping), which lowers the success rate to 57.3. This highlights the importance of training on grouped task streams, in which curation decisions are learned from their downstream impact on related future tasks.

Figure 4: Behaviors of the skill curator w.r.t. skill operations during training.

Behaviors of Skill Curator. To better understand how the behavior of the skill curator evolves during training, we analyze the distribution of its three skill operations from rollouts at different training steps: , , and . Figure 4 plots the proportion of each operation. At the beginning of training, insert overwhelmingly dominates, indicating that the model is primarily focused on populating the skill repository with new knowledge distilled from experience. As training progresses, however, update becomes increasingly frequent, while insert steadily declines. This suggests that the skill curator gradually moves from plain expansion of skills to refining existing skills. Meanwhile, delete remains a relatively small fraction throughout training with a slightly growing trend, showing the effectiveness of rewarding conciseness of SkillRepo. Instead, the dominant form of adaptation is to revise and consolidate previously acquired skills.

Graphic from the paper; its text follows

Figure 5: Evolution dynamics of the curated skills under RL training.

Skill Evolution Dynamics. Beyond aggregate performance, we examine how the skill repository evolves during RL training. We focus on two emergent phenomena: (i) new Markdown sections within individual skills, and (ii) higher-level meta-skills that capture reusable principles across tasks. Figure 5(a) shows that early in training, the curator tends to introduce generic sections such as additional guidance, tips, or recommendations, which often make skills more verbose without substantially improving their operational value. As training progresses, these additions shift toward more actionable structures, such as failure-handling logic and conditional branches that specify when to deviate from the default workflow. This suggests that RL gradually steers the curator from superficial enrichment toward execution-oriented skill refinement. Figure 5(b) further shows that evolution occurs not only within individual skills, but also in the global organization of the repository. Early repositories are dominated by narrow, task-specific skills, whereas later repositories contain a more diverse set of meta-strategy skills covering verification, fallback planning, system search, and strategy adjustment. This indicates that the learned curator does not merely accumulate skills, but progressively expands the repository’s strategic space, shifting it from isolated task-local procedures toward more compositional cross-task control knowledge.

Figure 6: Comparison of skill utilization statistics on ALFWorld.

Attribution of Skill Usage. To better understand whether the gains of SkillOS come from the evolved skills, we analyze how skills are used during evaluation. We consider 4 complementary metrics: (i) skill usage rate, the fraction of examples where the agent invokes at least one skill; (ii) successful skill usage rate, the success rate among examples that use skills; (iii) skill coverage, the fraction of the skill collection that are actually used; and (iv) the average number of skills used per example, which measures the degree of skill reliance. Figure 6 reports results on ALFWorld. Compared with the baseline, SkillOS invokes skills on all evaluation examples and achieves a higher success rate, indicating that the evolved skills contribute directly to task solving. Also, a larger fraction of the skill curated by SkillOS is used, showing that RL training improves the overall utility of the curated SkillRepo. Meanwhile, SkillOS uses fewer skills per example, suggesting that gains come from more precise skill selection rather than more skill context.

We presented SkillOS, an RL training recipe for learning skill curation in self-evolving agents. By decoupling the skill curator from the agent executor, SkillOS enables modular skill curation without retraining the underlying executor. Through grouped task streams and executor-grounded rewards, SkillOS optimizes curation decisions by their downstream impact on future tasks. Across diverse benchmarks and LLM backbones, SkillOS consistently improves both performance and efficiency. Further analyses show that trained skill curation can outperform frontier models’ zero-shot curation ability and generalize across settings, highlighting modular, trained skill curation as a practical path toward agents that self-evolve from experience.

We thank Zilin Xiao, I-Hung Hsu, Zexue He, and members from Google Cloud AI Research for their valuable feedback during the preparation of the paper. Siru was supported by the Molecule Maker Lab Institute: An AI Research Institutes program supported by NSF under Award No. 2019897.

References

Alzubi et al. (2026) S. Alzubi, N. Provenzano, J. Bingham, W. Chen, and T. Vu EvoSkill: automated skill discovery for multi-agent systems. Vol. abs/2603.02766. External Links: Link Cited by: §2.Source
Anthropic (2025a) AnthropicAgent skills(Website) Note: Claude API Docs. Accessed: 2026-04-01 External Links: Link Cited by: §2.Source
Anthropic (2025b) Anthropic Skills. Note: https://github.com/anthropics/skills Cited by: Appendix D, §1, §1, §1, §3.1.Source
Fang et al. (2025a) J. Fang, Y. Peng, X. Zhang, Y. Wang, X. Yi, G. Zhang, Y. Xu, B. Wu, S. Liu, Z. Li, et al. A comprehensive survey of self-evolving ai agents: a new paradigm bridging foundation models and lifelong agentic systems. ArXiv preprint abs/2508.07407. External Links: Link Cited by: §1, §2.Source
Fang et al. (2025b) R. Fang, Y. Liang, X. Wang, J. Wu, S. Qiao, P. Xie, F. Huang, H. Chen, and N. Zhang Memp: exploring agent procedural memory. ArXiv preprint abs/2508.06433. External Links: Link Cited by: §B.3.2, §1, §4.1.Source
Feng et al. (2025) L. Feng, Z. Xue, T. Liu, and B. An Group-in-group policy optimization for llm agent training. ArXiv preprint abs/2505.10978. External Links: Link Cited by: §A.2.Source
Gao et al. (2025) H. Gao, J. Geng, W. Hua, M. Hu, X. Juan, H. Liu, S. Liu, J. Qiu, X. Qi, Y. Wu, et al. A survey of self-evolving agents: what, when, how, and where to evolve on the path to artificial super intelligence. ArXiv preprint abs/2507.21046. External Links: Link Cited by: §1, §2.Source
Guo et al. (2025) D. Guo, D. Yang, H. Zhang, J. Song, P. Wang, Q. Zhu, R. Xu, R. Zhang, S. Ma, X. Bi, et al. Deepseek-r1: incentivizing reasoning capability in llms via reinforcement learning. ArXiv preprint abs/2501.12948. External Links: Link Cited by: §B.3.1.Source
He et al. (2026a) Z. He, Y. Wang, C. Zhi, Y. Hu, T. Chen, L. Yin, Z. Chen, T. A. Wu, S. Ouyang, Z. Wang, et al. MemoryArena: benchmarking agent memory in interdependent multi-session agentic tasks. ArXiv preprint abs/2602.16313. External Links: Link Cited by: §1.Source
He et al. (2026b) Z. He, T. Liang, J. Xu, Q. Liu, X. Chen, Y. Wang, L. Song, D. Yu, Z. Liang, W. Wang, Z. Zhang, R. Wang, Z. Tu, H. Mi, and D. Yu DeepMath-103k: a large-scale, challenging, decontaminated, and verifiable mathematical dataset for advancing reasoning. In The Fourteenth International Conference on Learning Representations, External Links: Link Cited by: §B.2, §B.3.1, §4.1.Source
Ho et al. (2025) M. Ho, C. Si, Z. Feng, F. Yu, Y. Yang, Z. Liu, Z. Hu, and L. Qin ArcMemo: abstract reasoning composition with lifelong LLM memory. ArXiv preprint abs/2509.04439. External Links: Link Cited by: §2.Source
Hu et al. (2025) Y. Hu, S. Liu, Y. Yue, G. Zhang, B. Liu, F. Zhu, J. Lin, H. Guo, S. Dou, Z. Xi, et al. Memory in the age of ai agents. ArXiv preprint abs/2512.13564. External Links: Link Cited by: §1, §2.Source
Huang et al. (2025) T. Huang, K. Basu, I. Abdelaziz, P. Kapanipathi, J. May, and M. Chen R2D2: remembering, replaying and dynamic decision making with a reflective agentic memory. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), W. Che, J. Nabende, E. Shutova, and M. T. Pilehvar (Eds.), Vienna, Austria, pp. 30318–30330. External Links: Document, ISBN 979-8-89176-251-0, Link Cited by: §2.Source Source
Huang et al. (2026) W. Huang, W. Zhang, Y. Liang, Y. Bei, Y. Chen, T. Feng, X. Pan, Z. Tan, Y. Wang, T. Wei, et al. Rethinking memory mechanisms of foundation agents in the second half. ArXiv preprint abs/2602.06052. External Links: Link Cited by: §2.Source
Islam et al. (2024) Md. A. Islam, M. E. Ali, and M. R. Parvez MapCoder: multi-agent code generation for competitive problem solving. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), L. Ku, A. Martins, and V. Srikumar (Eds.), Bangkok, Thailand, pp. 4912–4944. External Links: Document, Link Cited by: §2.Source Source
Kim et al. (2025) N. Kim, K. T. Ong, Y. Hwang, M. Kang, I. Jihn, G. Kim, M. Kim, and J. Yeo PRINCIPLES: synthetic strategy memory for proactive dialogue agents. In Findings of the Association for Computational Linguistics: EMNLP 2025, C. Christodoulopoulos, T. Chakraborty, C. Rose, and V. Peng (Eds.), Suzhou, China, pp. 21329–21368. External Links: Document, ISBN 979-8-89176-335-7, Link Cited by: §2.Source Source
Kuroki et al. (2025) S. Kuroki, T. Nakamura, T. Akiba, and Y. Tang Agent skill acquisition for large language models via cycleQD. In The Thirteenth International Conference on Learning Representations, External Links: Link Cited by: §2.Source
Li et al. (2026a) X. Li, W. Chen, Y. Liu, S. Zheng, X. Chen, Y. He, Y. Li, B. You, H. Shen, J. Sun, S. Wang, Q. Zeng, D. Wang, X. Zhao, Y. Wang, R. B. Chaim, Z. Di, Y. Gao, J. He, Y. He, L. Jing, L. Kong, X. Lan, J. Li, S. Li, Y. Li, Y. Lin, X. Liu, X. Liu, H. Lyu, Z. Ma, B. Wang, R. Wang, T. Wang, W. Ye, Y. Zhang, H. Xing, Y. Xue, S. Dillmann, and H. Lee SkillsBench: benchmarking how well agent skills work across diverse tasks. ArXiv preprint abs/2602.12670. External Links: Link Cited by: §2.Source
Li et al. (2026b) Y. Li, R. Miao, Z. Qi, and T. Lan ARISE: agent reasoning with intrinsic skill evolution in hierarchical reinforcement learning. ArXiv preprint abs/2603.16060. External Links: Link Cited by: §2.Source
Liang et al. (2026) Y. Liang, R. Zhong, H. Xu, C. Jiang, Y. Zhong, R. Fang, J. Gu, S. Deng, Y. Yao, M. Wang, et al. Skillnet: create, evaluate, and connect ai skills. ArXiv preprint abs/2603.04448. External Links: Link Cited by: §2.Source
Ling et al. (2026) G. Ling, S. Zhong, and R. Huang Agent skills: A data-driven analysis of claude skills for extending large language model functionality. ArXiv preprint abs/2602.08004. External Links: Link Cited by: §2.Source
Ouyang et al. (2026) S. Ouyang, J. Yan, I. Hsu, Y. Chen, K. Jiang, Z. Wang, R. Han, L. Le, S. Daruki, X. Tang, V. Tirumalashetty, G. Lee, M. Rofouei, H. Lin, J. Han, C. Lee, and T. Pfister ReasoningBank: scaling agent self-evolving with reasoning memory. In The Fourteenth International Conference on Learning Representations, External Links: Link Cited by: §B.3.2, §2, §4.1.Source
Qiu et al. (2025) J. Qiu, X. Qi, T. Zhang, X. Juan, J. Guo, Y. Lu, Y. Wang, Z. Yao, Q. Ren, X. Jiang, et al. Alita: generalist agent enabling scalable agentic reasoning with minimal predefinition and maximal self-evolution. ArXiv preprint abs/2505.20286. External Links: Link Cited by: §1.Source
Reimers and Gurevych (2019) N. Reimers and I. Gurevych Sentence-BERT: sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), K. Inui, J. Jiang, V. Ng, and X. Wan (Eds.), Hong Kong, China, pp. 3982–3992. External Links: Document, Link Cited by: §B.2.2.Source Source
Rein et al. (2024) D. Rein, B. L. Hou, A. C. Stickland, J. Petty, R. Y. Pang, J. Dirani, J. Michael, and S. R. Bowman GPQA: a graduate-level google-proof q&a benchmark. In First Conference on Language Modeling, External Links: Link Cited by: §B.3.1, §4.1.Source
Robertson and Zaragoza (2009) S. Robertson and H. Zaragoza The probabilistic relevance framework: bm25 and beyond. Vol. 4, Now Publishers Inc. Cited by: §3.1.
Shao et al. (2024) Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, M. Zhang, Y. K. Li, Y. Wu, and D. Guo DeepSeekMath: pushing the limits of mathematical reasoning in open language models. ArXiv preprint abs/2402.03300. External Links: Link Cited by: §3.2.2.Source
Shen et al. (2026) M. Shen, K. Zha, Z. He, Z. Hong, S. Ouyang, J. J. Ryu, P. Sattigeri, S. Diggavi, and G. Wornell Decocted experience improves test-time inference in llm agents. ArXiv preprint abs/2604.04373. External Links: Link Cited by: §2.Source
Sheng et al. (2024) G. Sheng, C. Zhang, Z. Ye, X. Wu, W. Zhang, R. Zhang, Y. Peng, H. Lin, and C. Wu HybridFlow: a flexible and efficient rlhf framework. arXiv preprint arXiv: 2409.19256. Cited by: §4.1.
Shridhar et al. (2021) M. Shridhar, X. Yuan, M. Côté, Y. Bisk, A. Trischler, and M. J. Hausknecht ALFWorld: aligning text and embodied environments for interactive learning. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021, External Links: Link Cited by: §B.3.1, §4.1.Source
Tang et al. (2025) X. Tang, T. Qin, T. Peng, Z. Zhou, D. Shao, T. Du, X. Wei, P. Xia, F. Wu, H. Zhu, G. Zhang, J. Liu, X. Wang, S. Hong, C. Wu, H. Cheng, C. Wang, and W. Zhou Agent KB: leveraging cross-domain experience for agentic problem solving. ArXiv preprint abs/2507.06229. External Links: Link Cited by: §2.Source
Team (2025a) G. Team Gemini 2.5: pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. ArXiv preprint abs/2507.06261. External Links: Link Cited by: §3.2.1, §4.1.Source
Team (2025b) Q. Team Qwen3 technical report. ArXiv preprint abs/2505.09388. External Links: Link Cited by: §4.1.Source
Tu et al. (2026) S. Tu, C. Xu, Q. Zhang, Y. Zhang, X. Lan, L. Li, and D. Zhao Dynamic dual-granularity skill bank for agentic rl. ArXiv preprint abs/2603.28716. External Links: Link Cited by: §1, §2.Source
Wang et al. (2025a) J. Wang, Q. Yan, Y. Wang, Y. Tian, S. S. Mishra, Z. Xu, M. Gandhi, P. Xu, and L. L. Cheong Reinforcement learning for self-improving agent with skill library. ArXiv preprint abs/2512.17102. External Links: Link Cited by: §1, §2, §3.2.1.Source
Wang et al. (2024) L. Wang, C. Ma, X. Feng, Z. Zhang, H. Yang, J. Zhang, Z. Chen, J. Tang, X. Chen, Y. Lin, W. X. Zhao, Z. Wei, and J. Wen A survey on large language model based autonomous agents. Frontiers Comput. Sci. 18 (6), pp. 186345. External Links: Document, Link Cited by: §1.Source Source
Wang et al. (2025b) Y. Wang, R. Takanobu, Z. Liang, Y. Mao, Y. Hu, J. McAuley, and X. Wu Mem-$\{$ $\backslash$alpha$\}$: learning memory construction via reinforcement learning. ArXiv preprint abs/2509.25911. External Links: Link Cited by: §2.Source
Wang et al. (2025c) Z. Z. Wang, A. Gandhi, G. Neubig, and D. Fried Inducing programmatic skills for agentic tasks. In Second Conference on Language Modeling, External Links: Link Cited by: §1, §2.Source
Wang et al. (2025d) Z. Z. Wang, J. Mao, D. Fried, and G. Neubig Agent workflow memory. In Forty-second International Conference on Machine Learning, External Links: Link Cited by: §2.Source
Wei et al. (2022a) J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. H. Chi, Q. V. Le, and D. Zhou Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022, S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (Eds.), External Links: Link Cited by: §4.1.Source
Wei et al. (2022b) J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. H. Chi, Q. V. Le, and D. Zhou Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022, S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (Eds.), External Links: Link Cited by: §A.2.Source
Wei et al. (2025) T. Wei, N. Sachdeva, B. Coleman, Z. He, Y. Bei, X. Ning, M. Ai, Y. Li, J. He, E. H. Chi, et al. Evo-memory: benchmarking llm agent test-time learning with self-evolving memory. ArXiv preprint abs/2511.20857. External Links: Link Cited by: §2, §4.Source
Wu et al. (2024) C. Wu, Z. R. Tam, C. Lin, Y. Chen, and H. Lee StreamBench: towards benchmarking continuous improvement of language agents. In Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024, A. Globersons, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. M. Tomczak, and C. Zhang (Eds.), External Links: Link Cited by: §1, §3.1.Source
Wu et al. (2025a) W. Wu, K. Zhou, R. Yuan, V. Yu, S. Wang, Z. Hu, and B. Huang Auto-scaling continuous memory for gui agent. ArXiv preprint abs/2510.09038. External Links: Link Cited by: §2.Source
Wu et al. (2025b) Y. Wu, S. Liang, C. Zhang, Y. Wang, Y. Zhang, H. Guo, R. Tang, and Y. Liu From human memory to ai memory: a survey on memory mechanisms in the era of llms. ArXiv preprint abs/2504.15965. External Links: Link Cited by: §1, §2.Source
Xia et al. (2026) P. Xia, J. Chen, H. Wang, J. Liu, K. Zeng, Y. Wang, S. Han, Y. Zhou, X. Zhao, H. Chen, et al. SkillRL: evolving agents via recursive skill-augmented reinforcement learning. ArXiv preprint abs/2602.08234. External Links: Link Cited by: §1, §2, §4.Source
Xu et al. (2025) W. Xu, Z. Liang, K. Mei, H. Gao, J. Tan, and Y. Zhang A-mem: agentic memory for LLM agents. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, External Links: Link Cited by: §1.Source
Yan et al. (2025) S. Yan, X. Yang, Z. Huang, E. Nie, Z. Ding, Z. Li, X. Ma, H. Schütze, V. Tresp, and Y. Ma Memory-r1: enhancing large language model agents to manage and utilize memories via reinforcement learning. ArXiv preprint abs/2508.19828. External Links: Link Cited by: §2.Source
Yang et al. (2024) L. Yang, Z. Yu, T. Zhang, S. Cao, M. Xu, W. Zhang, J. E. Gonzalez, and B. Cui Buffer of thoughts: thought-augmented reasoning with large language models. In Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024, A. Globersons, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. M. Tomczak, and C. Zhang (Eds.), External Links: Link Cited by: §2.Source
Yang et al. (2026) Y. Yang, J. Li, Q. Pan, B. Zhan, Y. Cai, L. Du, J. Zhou, K. Chen, Q. Chen, X. Li, B. Zhang, and L. He AutoSkill: experience-driven lifelong learning via skill self-evolution. ArXiv preprint abs/2603.01145. External Links: Link Cited by: §2.Source
Yao et al. (2022) S. Yao, H. Chen, J. Yang, and K. Narasimhan WebShop: towards scalable real-world web interaction with grounded language agents. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022, S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (Eds.), External Links: Link Cited by: §B.3.1, §B.3.3, §4.1.Source
Yao et al. (2023) S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. R. Narasimhan, and Y. Cao ReAct: synergizing reasoning and acting in language models. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023, External Links: Link Cited by: §4.1.Source
Ye et al. (2026) Y. Ye, H. Jiang, F. Jiang, T. Lan, Y. Du, B. Fu, X. Shi, Q. Jia, L. Wang, and W. Luo UMEM: unified memory extraction and management framework for generalizable memory. ArXiv preprint abs/2602.10652. External Links: Link Cited by: §1, §2, §3.2.1, §4.Source
Yu et al. (2026) H. Yu, T. Chen, J. Feng, J. Chen, W. Dai, Q. Yu, Y. Zhang, W. Ma, J. Liu, M. Wang, and H. Zhou MemAgent: reshaping long-context LLM with multi-conv RL-based memory agent. In The Fourteenth International Conference on Learning Representations, External Links: Link Cited by: §2.Source
Zhang et al. (2024) C. Zhang, Y. Jian, Z. Ouyang, and S. Vosoughi Working memory identifies reasoning limits in language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 16896–16922. Cited by: §2.
Zhang et al. (2026a) H. Zhang, Q. Long, J. Bao, T. Feng, W. Zhang, H. Yue, and W. Wang MemSkill: learning and evolving memory skills for self-evolving agents. ArXiv preprint abs/2602.02474. External Links: Link Cited by: §2.Source
Zhang et al. (2026b) S. Zhang, J. Wang, R. Zhou, J. Liao, Y. Feng, Z. Li, Y. Zheng, W. Zhang, Y. Wen, Z. Li, et al. Memrl: self-evolving agents via runtime reinforcement learning on episodic memory. ArXiv preprint abs/2601.03192. External Links: Link Cited by: §2.Source
Zhang et al. (2025a) Y. Zhang, J. Shu, Y. Ma, X. Lin, S. Wu, and J. Sang Memory as action: autonomous context curation for long-horizon agentic tasks. ArXiv preprint abs/2510.12635. External Links: Link Cited by: §2.Source
Zhang et al. (2025b) Z. Zhang, Q. Dai, R. Li, X. Bo, X. Chen, and Z. Dong Learn to memorize: optimizing llm-based agents with adaptive memory framework. ArXiv preprint abs/2508.16629. External Links: Link Cited by: §2.Source
Zhao et al. (2024) A. Zhao, D. Huang, Q. Xu, M. Lin, Y. Liu, and G. Huang ExpeL: LLM agents are experiential learners. In Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence, IAAI 2024, Fourteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2014, February 20-27, 2024, Vancouver, Canada, M. J. Wooldridge, J. G. Dy, and S. Natarajan (Eds.), pp. 19632–19642. External Links: Document, Link Cited by: §2.Source Source
Zheng et al. (2025) B. Zheng, M. Y. Fatemi, X. Jin, Z. Z. Wang, A. Gandhi, Y. Song, Y. Gu, J. Srinivasa, G. Liu, G. Neubig, and Y. Su SkillWeaver: web agents can self-improve by discovering and honing skills. ArXiv preprint abs/2504.07079. External Links: Link Cited by: §1.Source
Zheng et al. (2024) L. Zheng, R. Wang, X. Wang, and B. An Synapse: trajectory-as-exemplar prompting with memory for computer control. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024, External Links: Link Cited by: §2.Source
Zhou et al. (2025) H. Zhou, Y. Chen, S. Guo, X. Yan, K. H. Lee, Z. Wang, K. Y. Lee, G. Zhang, K. Shao, L. Yang, and J. Wang Memento: fine-tuning LLM agents without fine-tuning llms. ArXiv preprint abs/2508.16153. External Links: Link Cited by: §2.Source
Zhou et al. (2026) Z. Zhou, A. Qu, Z. Wu, S. Kim, A. Prakash, D. Rus, B. K. H. Low, and P. P. Liang MEM1: learning to synergize memory and reasoning for efficient long-horizon agents. In The Fourteenth International Conference on Learning Representations, External Links: Link Cited by: §2.Source

Contents of Appendix

In this section, we provide the full prompt templates used throughout different phases and components of our framework.

The following prompt templates demonstrate the input to the skill curator during training processes.

Figure 7: System prompt used for skill curator during training process.

Figure 8: Tool call definition/signature of skill curator in Figure 7.

The following prompts are used for the frozen agent executor. These templates provide the agent with the current task description, a history of previous interactions, and a set of retrieved skills to guide its decision-making process. All prompts explicitly force chain-of-thought (CoT) (Wei et al., 2022b) reasoning.

For agent tasks including ALFWorld and WebShop, we follow GiGPO (Feng et al., 2025) and leverage its environment and prompt setting for inference.

Figure 9: Prompt for ALFWorld agent execution with relevant retrieved skills.

Figure 10: Prompt for WebShop agent execution with relevant retrieved skills.

Figure 11: Prompt for agent execution in reasoning tasks with relevant retrieved skills.

During the RL training process, a reward $r^{cnt}$ is assigned based on an external judge of Qwen3-32B to judge whether the curated skills are semantically meaningful and are likely to be useful for future tasks. We show the prompt to the external judge here.

Figure 12: Prompt for using an external judge to assign a reward score $r^{cnt}$ for generated skill contents.

We present the prompts used to obtain the self-judged correctness signal $\mathbbm{1}_{\xi_{t}}$ for self-evolution via LLM-as-a-judge using the corresponding frozen agent executor as the backbone model in Figures 13, 14 for ALFWorld, reasoning, and WebShop tasks, respectively.

Figure 13: Prompt for LLM-as-a-judge to obtain the correctness signal to the current trajectory in the ALFWorld benchmark.

Figure 14: Prompt for LLM-as-a-judge to obtain the correctness signal for single-turn reasoning problems.

Figure 15: Prompt for LLM-as-a-judge to obtain the correctness signal to the current trajectory for the WebShop benchmark.

We present the choices for all hyperparameters during both the training and inference processes in Table 4 for different tasks.

Original Table 4 — Table 4: Hyperparameters for SkillOS for training and inference settings.

Table 4
Hyperparameter	Value
ALFWorld	WebShop	Reasoning
RL Training
Learning rate	$1\times 10^{-6}$
Batch size	32
KL loss Coef	0.001
Max Prompt Length	16,384
Max Response Length	4,096
GRPO group size	8
Temperature	1.0
Steps	60	50	100
Data Grouping Size	10	10	Random(5,12)
Agent Executor Inference
Top-K skill retrieval	5
Max number of turns	30	30	1
Action history length	3	3	-

In this section, we detail the two-stage pipeline used to turn the raw training set $\mathcal{D}=\{x_{i}\}_{i=1}^{N}$ into the grouped training set $\mathcal{G}=\{G_{j}\}_{j=1}^{M}$ of Section 3.2.1. Stage 1 annotates each instance with a structured set of latent attributes via an LLM annotator (Sec. B.2.1). Stage 2 assembles groups of related tasks by retrieving, filtering, and ranking candidates under a semantic phrase-level similarity (Sec. B.2.2). For training of single-turn reasoning tasks, we instantiate the pipeline on DeepMath-103K (He et al., 2026b), which provides both the raw problems $x_{i}$ and a scalar difficulty score $d_{i}\in\mathbb{R}$ that is reused as a curriculum signal by Stage 2. For multi-turn agentic tasks, we leverage the default task type annotation for each benchmark (e.g., 6 task types in ALFWorld) as they naturally expose a discrete partition of tasks into families that share the same underlying skills, and we can use this partition directly in place of the annotated attribute set $Z_{i}$ .

We implement the attribute set $Z_{i}$ of each instance $x_{i}$ as a tuple of five phrase-lists,

Z_{i}\;=\;\bigl(T_{i},\;S_{i},\;C_{i},\;R_{i},\;P_{i}\bigr),

where $T_{i}$ is the list of high-level topics, $S_{i}$ the required skills or capabilities, $C_{i}$ the underlying mathematical concepts or theorems, $R_{i}$ the applicable heuristic strategies, and $P_{i}$ the common pitfalls. Each dimension is populated by a small set of short phrases (at most five words each). The annotator is instructed to: (i) emit standardized terminology rather than free-form rationales, (ii) omit any content specific to the question text or its final answer, and (iii) use as few phrases per dimension as necessary to characterize the task. We enforce the output schema via structured decoding with a fixed JSON response schema, and query Gemini-2.5-Pro with the highest thinking-budget configuration. The exact annotation instruction is reproduced in Figure 16.

Figure 16: System instruction used to elicit $Z_{i}$ from each task in $\mathcal{D}$ .

Given $\{(x_{i},Z_{i},d_{i})\}_{i=1}^{N}$ , we construct each group $G_{j}=(x_{j,1},\dots,x_{j,n})$ by sampling a seed task and then iteratively appending related tasks. The core primitive is a pair sampler that, given a source $x_{s}$ , returns an admissible successor $x_{t}$ ; longer groups are obtained by iterating this primitive with a growing exclusion set so that instances within a group remain distinct.

Because the annotated phrases come from a large open vocabulary (e.g., “pigeonhole principle” vs. “counting argument”), exact set overlap is unreliable. We therefore score the similarity between any two phrase lists $A$ and $B$ using a soft-Jaccard $\mathrm{SJ}_{\tau}(A,B)$ that combines exact matches with a greedy one-to-one matching between remaining phrases under a sentence-embedding cosine similarity (computed with all-MiniLM-L6-v2 (Reimers and Gurevych, 2019)) above a threshold $\tau$ . We write $m_{\tau}(A,B)$ for the resulting integer matched-pair count, which we use alongside $\mathrm{SJ}_{\tau}$ in the filters below.

For a source $x_{s}$ and candidate $x_{t}$ , we accept the pair only when all of the following hold:

1.
Shared foundation: $m_{\tau}(C_{s},C_{t})\geq\kappa_{C}$ and $m_{\tau}(S_{s},S_{t})\geq\kappa_{S}$ ;
2.
Shared reasoning: $m_{\tau}(R_{s},R_{t})+m_{\tau}(P_{s},P_{t})\geq 1$ ;
3.
Not a near-duplicate: $\mathrm{SJ}_{\tau}(T_{s},T_{t})\leq\theta_{T}$ and the weighted overall similarity $\Omega(x_{s},x_{t})\leq\sigma_{\max}$ ;
4.
Not too unrelated: $\Omega(x_{s},x_{t})\geq\sigma_{\min}$ ;
5.
Progression: $x_{t}$ introduces at least one new concept or skill, i.e. $|C_{t}|>m_{\tau}(C_{s},C_{t})$ or $|S_{t}|>m_{\tau}(S_{s},S_{t})$ ;
6.
Curriculum direction: $d_{t}-d_{s}\geq\delta_{\min}$ .

Here $\Omega$ is a convex combination of per-dimension soft-Jaccard scores across $\{C,S,R,P,T\}$ with weights listed in Table 5. Conditions (1)–(2) ensure genuine reuse of foundational knowledge and reasoning machinery; (3)–(4) place the pair in a useful “related but not redundant” band; (5) guarantees that $x_{t}$ carries something new for the skill curator to compress into the library; and (6) enforces a forward curriculum.

Scoring all $N{-}1$ alternatives per source is prohibitive, so we precompute an inverted index over the dependency fields $\{C,R,P\}$ : for each source $x_{s}$ , the candidate pool consists of tasks that share at least one exact dependency phrase with $x_{s}$ , capped at $K_{\text{inv}}$ entries via uniform subsampling. Routing retrieval through dependency fields rather than topics prevents groups from collapsing onto a single narrow subject. Among the candidates that pass the gate, we select the one that maximizes

s(x_{s},x_{t})\;=\;\sum_{f\in\{C,S,R,P,T\}}w_{f}\,\mathrm{SJ}_{\tau}(f_{s},f_{t})\;+\;\lambda\cdot b(d_{s},d_{t}),

where $b(\cdot)$ is a bounded difficulty bonus that rewards moderate forward steps. If no inverted-index candidate passes the gate, we fall back to a uniform random pool of size $F$ and re-apply the same gate and scoring; this catches pairs whose phrases agree semantically but not lexically. Extensions sourced from the fallback pool are tagged so downstream training can audit or downweight them. The difficulty gap $d_{t}-d_{s}$ is additionally modulated by a randomized curriculum mode $(p_{\uparrow},p_{=},p_{\downarrow})$ ; for our main experiments, we use an almost exclusively forward curriculum, which produced a more stable training signal than mixed curricula.

Table 5 lists all hyperparameters of the Stage 2 pipeline and the values adopted for our main experiments. The weights were tuned on a held-out subset of 200 source tasks by manually inspecting sampled pairs for prerequisite quality; we found the pipeline largely insensitive to small perturbations of the weights but noticeably sensitive to the progression and overall-similarity-band conditions, removing either of which produced markedly more trivial or degenerate pairs.

Original Table 5 — Table 5: Hyperparameters of the Stage 2 grouping pipeline.

Table 5
Symbol	Meaning	Value
—	Phrase encoder	all-MiniLM-L6-v2
$\tau$	Cosine threshold for fuzzy phrase matching	$0.60$
$\kappa_{C}$	Minimum matched concept pairs	$1$
$\kappa_{S}$	Minimum matched skill pairs	$1$
$\theta_{T}$	Maximum topic soft-Jaccard	$0.65$
$\sigma_{\min},\sigma_{\max}$	Overall-similarity band	$0.30,\,0.85$
$\delta_{\min}$	Difficulty-delta floor	$0.0$
$(w_{C},w_{S},w_{R},w_{P},w_{T})$	Dimension weights	$(5,\,4,\,3,\,1,\,2)$
$\lambda$	Difficulty-bonus weight	$1.0$
$(p_{\uparrow},p_{=},p_{\downarrow})$	Mode probabilities	$(0.80,\,0.20,\,0.00)$
$[\Delta_{\min},\Delta_{\max}]$	Gap in easy $\rightarrow$ hard mode	$[0.5,\,3.0]$
$\Delta_{=}$	Maximum $\|d_{t}-d_{s}\|$ in same mode	$0.3$
$K_{\text{inv}}$	Inverted-index subsample cap	$2{,}000$
$F$	Fallback pool size	$200$

In this section, we provide a detailed introduction to all the datasets involved in this paper.

ALFWorld. ALFWorld (Shridhar et al., 2021) is a text-based interactive benchmark that aligns the TextWorld engine with the embodied ALFRED environment, enabling agents to learn high-level household policies through natural-language interaction. The benchmark covers six task types — Pick & Place, Examine in Light, Clean & Place, Heat & Place, Cool & Place, and Pick Two & Place — situated in 120 simulated rooms spanning kitchens, bedrooms, bathrooms, and living rooms. It provides $3,553$ training tasks, together with $140$ valid_seen tasks for the test set. At each step, the agent receives a textual description of its surroundings together with a goal instruction (e.g., "put a hot apple in the fridge") and must issue high-level commands such as go to, take, open, heat, and put.

WebShop WebShop (Yao et al., 2022) is a simulated e-commerce web environment designed to benchmark language agents on realistic, grounded shopping tasks. The environment is populated with 1.18 million real-world products scraped from Amazon and 12,087 crowd-sourced natural-language instructions, partitioned into 10,587 training, 1,000 dev, and 500 test instructions. Given an instruction (e.g., “I’m looking for a quick-release fitness strap band in teal, priced lower than $40.00”), the agent interacts with the environment via two action types — search[query] and click[button] — to locate and purchase a product that matches the specified attributes, type, options, and price. At the end of each episode, a programmatic reward in [0, 1] is computed by comparing the purchased item against the ground-truth product specification. Following the standard evaluation protocol used in prior LLM-agent work, we evaluate on the 500 held-out test instructions.

DeepMath-103K DeepMath-103K (He et al., 2026b) is a large-scale, decontaminated mathematical reasoning dataset containing approximately 103K problems at high difficulty (primarily AoPS Levels 5–9), spanning algebra, calculus, number theory, geometry, probability, and discrete mathematics. Each problem is paired with a verifiable final answer — enabling rule-based RL rewards — together with a difficulty score, topic label, and three DeepSeek-R1 (Guo et al., 2025) chain-of-thought solutions. Specifically, we annotate a subset with around $33,000$ problems, with a final $20,000$ set of grouped training instances.

AIME24 & AIME25. A collection of demanding mathematical problems sourced from the 2024 and 2025 American Invitational Mathematics Examination (AIME), with 30 problems each year. Problems encompass algebra, geometry, number theory, and combinatorics. Created to assess large language models’ sophisticated mathematical reasoning abilities, the dataset presents substantial difficulty, systematic multi-phase solutions, and distinctive answers, establishing it as a robust benchmark for evaluating advanced analytical capabilities.

GPQA. Short for Graduate Level Google-Proof Q $\&$ A Benchmark (Rein et al., 2024), GPQA comprises a collection of demanding text-based multiple choice problems authored by subject specialists in biology, physics, and chemistry, intentionally crafted to be “exceptionally challenging”. We use the “GPQA-Diamond” subset for testing, which has $198$ problems in total.

We compare SkillOS against five representative baselines that span memory-free agents, recent memory-augmented methods, and two internal variants of our own framework. All baselines share the same frozen Agent Executor and are evaluated under identical task suites, retrieval budgets, and decoding settings to isolate the contribution of the memory mechanism.

(i) No Memory. A memory-free baseline in which the Agent Executor solves each task independently, without access to any external memory or cross-task knowledge transfer. Each episode begins from a blank state, and no information is retained across tasks. This baseline establishes a lower bound and isolates the contribution of any form of accumulated experience.

(ii) ReasoningBank (Ouyang et al., 2026). A recent memory-augmented method that distills reusable reasoning insights from past trajectories and stores them as a searchable bank for future tasks. At inference time, relevant insights are retrieved and injected into the executor’s context to guide reasoning. ReasoningBank represents the class of experience-distillation approaches, which emphasize the content of stored knowledge but rely on fixed, heuristic policies for deciding what to write or discard.

(iii) MemP (Fang et al., 2025b). A procedural-memory method that induces reusable procedures from agent experience and applies advanced memory-management strategies — including consolidation, forgetting, and re-indexing — to maintain the memory store over time. MemP represents the class of rule-based memory management approaches, which feature more sophisticated maintenance policies than ReasoningBank but still prescribe curation decisions through hand-designed heuristics rather than learning them from downstream task feedback.

(iv) SkillOS-base. A variant of our framework in which the Skill Curator is instantiated with the same open-source backbone as SkillOS but without any RL fine-tuning, while all other components remain identical to SkillOS. This baseline serves two purposes: (a) it provides a lower-bound reference point that reflects the intrinsic prompting-based curation ability of the open-source backbone prior to optimization, and (b) it isolates the contribution of our GRPO-based training, since SkillOS-base shares exactly the same model architecture, prompting template, and memory interface as SkillOS but forgoes end-to-end optimization against task performance.

(v) SkillOS-gemini. A variant of our framework in which the Skill Curator is instantiated with Gemini-2.5-Pro instead of a trained open-source model, while all other components remain identical to SkillOS. This baseline serves two purposes: (a) it provides a strong closed-source reference point for the upper bound of prompting-based curation, and (b) it isolates the effect of our GRPO-based training, since SkillOS-gemini shares the same prompting template and memory interface as SkillOS but forgoes RL optimization against task performance.

Together, these baselines cover the main design axes along which memory-augmented agents differ from SkillOS: whether memory exists at all (i), how stored knowledge is represented (ii vs. iii), and whether curation decisions are prescribed by heuristics or learned from task feedback (ii and iii vs. SkillOS), as well as whether the curator itself benefits from RL optimization (iv and v vs. SkillOS).

We evaluate SkillOS and all baselines along two complementary axes — task effectiveness and action efficiency — using metrics tailored to each benchmark. Across all benchmarks and methods, every configuration is run with three independent random seeds; we report the mean across seeds, with one standard deviation shown as a subscript (e.g., $85.7_{\pm 1.6}$ ). Within each backbone block of Tables 1 and 2, the best value in each column is highlighted in bold.

Our primary effectiveness metric on both ALFWorld and WebShop. On ALFWorld, SR is the fraction of evaluation episodes in which the agent reaches the goal state within the step budget, yielding a binary $\{0,1\}$ outcome per episode. We report SR both per task category — Pick, Look, Clean, Heat, Cool, and Pick2 — and as a macro-average (Avg. SR) across the six categories, so that categories with fewer tasks are not dominated by larger ones. On WebShop, following (Yao et al., 2022), SR is the fraction of episodes whose final reward equals exactly $1$ , i.e., the purchased product fully matches all specified attributes, options, type, and price constraints.

In addition to SR, WebShop provides a dense per-episode reward in $[0,100]$ that credits partial matches on attributes, options, type, and price even when the purchase is not a perfect match. We report the average score across evaluation episodes as a finer-grained complement to SR: two methods with similar SR may differ substantially in how close their near-misses are to the target product.

Our efficiency metric on ALFWorld and WebShop. Steps is the average number of environment actions the agent issues per episode, computed over all evaluation episodes regardless of success. Failed episodes contribute steps up to their termination point (task completion, max-step cutoff, or early stop). This metric captures a dimension that SR and Score alone cannot: two methods may achieve comparable effectiveness while differing substantially in how efficiently they reach the goal, which has direct implications for inference cost and deployment feasibility.

For the single-turn reasoning datasets — AIME24, AIME25, and GPQA — we report exact-match accuracy: the fraction of questions whose extracted final answer matches the ground truth. For AIME24 and AIME25, we adopt the evaluation protocol from the HuggingFace math_verify $^{1}$ toolkit, which parses the model’s final boxed expression and verifies mathematical equivalence to the reference answer (accounting for equivalent numerical forms, simplifications, and formatting variants). For GPQA, which is a multiple-choice benchmark, we extract the predicted option letter from the model’s response and score it as correct if and only if it exactly matches the ground-truth option. We additionally report an average accuracy (Avg. Acc.) across the three datasets to summarize overall reasoning ability.

All methods share the same frozen Agent Executor, retrieval budget (top- $k$ skills retrieved via BM25), maximum step budget, and decoding temperature within each backbone, so that differences in the reported metrics are attributable to the memory mechanism rather than to confounding inference settings. Unless stated otherwise, all numbers in the main paper are computed on the official held-out evaluation splits of each benchmark.

In addition to the Qwen3-8B/32B and Gemini-2.5-Pro executors used in the main paper, we further evaluate SkillOS on ALFWorld with the more recent Gemini-3.1-Flash-Lite as the frozen Agent Executor, to verify that our gains generalize to newer model families. Results are reported in Table 6.

SkillOS achieves the highest average success rate (73.1%), outperforming the strongest external baseline ReasoningBank (66.0%) by +7.1 points and the No-Memory baseline (61.2%) by +11.9 points, while requiring the fewest interaction steps (15.5 vs. 18.5 for No Memory). The two internal variants reproduce the ordering observed in the main experiments: SkillOS-base reaches only 63.6% — barely above No Memory — confirming that the open-source backbone cannot recover the curation policy through prompting alone, and SkillOS-gemini improves to 71.2% but is still surpassed by SkillOS despite using a much stronger curator backbone. This reinforces our main finding that learning the curator with task-level feedback contributes more than scaling up the curator model. We also note that MemP (58.6%) underperforms even No Memory under this executor, suggesting that hand-designed curation heuristics are brittle when the executor is less capable, whereas the policy learned by SkillOS remains robust. Per-subset, SkillOS wins on four of six subsets, with particularly large margins on Look (84.6% vs. 71.8%) and Cool (68.0% vs. 48.0%); the remaining two subsets are won by SkillOS-gemini (Pick) and ReasoningBank (Heat), on which SkillOS nonetheless remains competitive. Overall, these results confirm that the advantage of SkillOS transfers cleanly to a newer executor family.

Original Table 6 — Table 6: Experiment results on ALFWorld benchmark. Success rate (SR $\uparrow$ ) and the number of steps (Steps $\downarrow$ ) are reported on 6 subsets for Gemini-3.1-Flash-Lite as frozen executor.

Table 6
Methods	Pick	Look	Clean	Heat	Cool	Pick2	Avg. SR	Steps
(35)	(13)	(27)	(16)	(25)	(24)	(140)
No Memory	$85.7_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 0.0}}$	$59.0_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 8.9}}$	$67.9_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 9.3}}$	$25.0_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 6.2}}$	$38.7_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.3}}$	$66.7_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 0.0}}$	$61.2_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.3}}$	18.5
ReasoningBank	$87.6_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 4.4}}$	$71.8_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 4.4}}$	$63.0_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 0.0}}$	$\textbf{52.1}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 14.4}}$	$48.0_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 10.6}}$	$62.5_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 0.0}}$	$66.0_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.7}}$	17.6
MemP	$84.3_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 6.1}}$	$57.7_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 5.4}}$	$63.0_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 0.0}}$	$28.1_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 4.4}}$	$34.0_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.8}}$	$62.5_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 0.0}}$	$58.6_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 1.0}}$	19.3
SkillOS-base	$86.7_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 1.6}}$	$61.5_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 0.0}}$	$66.7_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 0.0}}$	$41.7_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 6.2}}$	$38.7_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 16.0}}$	$68.1_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.4}}$	$63.6_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 3.9}}$	17.7
SkillOS-gemini	$\textbf{96.2}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 1.6}}$	$61.5_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 13.3}}$	$74.1_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 3.7}}$	${31.2}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 12.5}}$	$66.7_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 4.6}}$	$68.1_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.4}}$	$71.2_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.9}}$	16.1
SkillOS	$88.6_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 0.0}}$	$\textbf{84.6}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 13.3}}$	$\textbf{77.8}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 0.0}}$	$37.5_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 17.2}}$	$\textbf{68.0}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 8.0}}$	$\textbf{68.1}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.4}}$	$\textbf{73.1}_{\,{\color[rgb]{0.5,0.5,0.5}\scriptscriptstyle 2.7}}$	15.5

Figure 17: Case studies of curated skills by SkillOS.

Figure 17 presents two representative skills curated by SkillOS that illustrate qualitatively different curation patterns across task types. For agentic tasks (Figure 17(a)), the curator distills a meta-strategy for failure recovery: rather than memorizing a specific object-search trajectory, it abstracts the recovery procedure into a reusable workflow (exhaustive search $\rightarrow$ confirm unavailability $\rightarrow$ identify a substitute $\rightarrow$ proceed with substitute) and explicitly references existing skills, demonstrating compositional curation. For reasoning tasks (Figure 17(b)), the curator captures branching-out reasoning: a single skill on inradius–circumradius–semiperimeter relations encodes multiple solution paths (relating the target distance to either the in/circumradius or the side lengths), each paired with its formula, application, and prerequisite constraints. Together, these examples show that SkillOS learns to produce skills tailored to the structure of the underlying task: procedural and composable for agentic settings, and multi-path with explicit preconditions for reasoning settings, rather than verbatim trajectory copies.

Figure 18: Case study on math-reasoning skill curation. SkillOS-base produces a generic partitioning recipe, while SkillOS curates a concrete and reusable counting framework with explicit constraints, equations, and a worked example.

We further qualitatively compare the skills curated by SkillOS against those produced by the baseline curator. In the math-reasoning case as shown in Figure 18, SkillOS-base outputs only a generic high-level recipe based on partitioning into disjoint sets, without explicit formulas, constraints, or examples. By comparison, SkillOS curates a much more useful skill that provides a concrete counting framework, including explicit constraint formulation, equation setup, and a worked example tailored to the target sub-problem. These examples show that RL-trained skill curation improves not only the correctness of the curated content, but also its specificity and usability, enabling skills to better capture the underlying structure of tasks.

Figure 19: Case studies of how skills curated by SkillOS successfully helped to solve a task in ALFWorld.

Figure 19 illustrates a representative example of how curated skills improve agent behavior in interactive environments. Given the task “look at the CD under the desklamp,” the memory-free baseline fails to infer the correct object–location relation and performs an inefficient search over irrelevant containers, eventually exhausting the step budget. In contrast, SkillOS retrieves a skill that encourages the agent to examine objects under or around light sources when the instruction refers to an object being “under” a lamp. Guided by this reusable strategy, the agent first locates and picks up the CD near the desk area, then moves to the desklamp and inspects the correct target location, completing the task successfully. This case highlights that curated skills do not merely memorize task-specific action sequences; instead, they provide transferable decision guidance that helps the agent focus exploration on semantically relevant objects and locations, reducing unnecessary interactions and improving task success.

Our current implementation relies on a relatively simple keyword-based retrieval mechanism, such as BM25, to retrieve relevant skills from the skill repository. This design choice allows us to isolate the main focus of this work: studying how skills can be curated, updated, and organized through experience-driven learning. However, more advanced retrieval methods, such as dense retrieval, hybrid retrieval, or learned retrievers, may further improve the relevance of retrieved skills and thus lead to stronger downstream performance. We leave the joint optimization of skill curation and skill retrieval to future work.

Following Anthropic’s skill paradigm (Anthropic, 2025b), we instantiate each skill as a single Markdown file that combines a YAML frontmatter and Markdown body. This simplification keeps the curator’s action space tractable, but it discards two affordances of the original SKILL.md format: (i) supporting scripts and external resource files that allow skills to encapsulate executable procedures rather than purely declarative knowledge, and (ii) hierarchical organization in which a top-level skill can reference or compose lower-level sub-skills. As a result, behaviors that are most naturally expressed as runnable code or as compositions of finer-grained primitives must currently be flattened into prose. Extending SkillOS to multi-file, hierarchical, and partially executable skills is a natural next step.

Throughout training, we keep the agent executor $\pi_{\mathcal{L}}$ frozen and optimize only the skill curator $\pi_{\mathcal{S}}$ . This decoupling is deliberate: it isolates the contribution of skill curation, makes the recipe modular across executors, and avoids confounding our analysis with executor-side adaptation. The downside is that the curator can only shape the system’s behavior through what it writes into SkillRepo; any miscalibration between the curated skills and the executor’s idiosyncrasies must be absorbed by the curator alone. Joint or alternating optimization of $\pi_{\mathcal{S}}$ and $\pi_{\mathcal{L}}$ may yield a better-aligned pair, at the cost of executor specificity and substantially higher training cost.

Our work opens several promising directions for future research.

SkillOS currently retrieves relevant skills from SkillRepo through a fixed top- $k$ BM25 lookup, treating retrieval as a static, one-shot operation. As the skill repository grows across thousands of tasks and domains, the bottleneck of self-evolving agents shifts from what to store to how to reliably retrieve and inject the right fragments at each decision step. A natural next step is to replace static retrieval with agentic search: letting the Skill Curator (or a dedicated retrieval agent) actively issue multiple queries, reformulate them based on intermediate evidence, and iteratively decide which skills to surface, cite, or compose for the executor. This reframes memory access as a first-class decision in the agent’s policy rather than a preprocessing step, and opens the door to scaling SkillOS to memory stores orders of magnitude larger than those considered here.

Our current skills are flat Markdown entries, each describing a single reusable pattern. Real agent competence, however, is hierarchical: high-level procedures invoke lower-level sub-skills, which in turn depend on primitive operations. Extending SkillRepo to support hierarchical decomposition — where the curator learns not only to insert, update, and delete skills but also to link, compose, and abstract them — could enable the agent to build increasingly expressive procedural libraries over time. This direction connects naturally to program-synthesis and library-learning literature, and would allow SkillOS to scale to longer-horizon tasks where single-skill retrieval is insufficient.

SkillOS treats memory as a single agent’s private artifact. In many realistic deployments, however, multiple agents operate in parallel (e.g., code review, multi-hop research, collaborative robotics) and could benefit from shared experiential memory. Open questions include how to arbitrate conflicting curation decisions from different agents, how to attribute credit when a shared skill contributes to one agent’s success but another’s failure, and how to preserve specialization while enabling cross-agent transfer. Our GRPO-based curator provides a natural starting point, but extending it to the multi-agent credit-assignment setting is non-trivial and likely to require new algorithmic ideas.

We used LLMs as a general-purpose writing assist tool during the preparation of this submission. Specifically, LLMs were employed for polishing the clarity and readability of text (e.g., refining sentence structure, improving grammar, and shortening overly verbose phrasing). All research ideas, methodology design, experiments, analyses, and final writing decisions were conceived, implemented, and validated solely by the authors.

SkillOS: Learning Skill Curation for Self-Evolving Agents

Abstract

References

Meow summary

The problem

The trick: test skills on later tasks

What the evidence says

Where it stops

Make notes of your own

Abstract

1 Introduction

2 Related Work

3 Methodology

3.1 Streaming Skill Curation with Multi-Agent Modular Design

3.2 Learning Skill Curation with RL

3.2.1 Training Instance Construction

3.2.2 Training Loop and Policy Optimization

4 Experiments

4.1 Setup

4.2 Main Results

4.3 Generalization of SkillOS

5 Analysis

Ablation Studies.

6 Conclusion

7 Acknowledgments

References

Appendix A Prompts

A.1 Prompt for Skill Curator

A.2 Prompt for Agent Executor

A.3 Prompt Used During Training

A.4 Prompt for LLM-as-a-Judge to Obtain Correctness Signals

Appendix B Implementation Details

B.1 Hyperparameters

B.2 Grouping Training Instances

B.2.1 Stage 1: Latent Attribute Annotation

B.2.2 Stage 2: Group Construction

Phrase similarity.

Dependency gate.

Candidate retrieval and scoring.

Hyperparameters.

B.3 Experiment Setup

B.3.1 Datasets

B.3.2 Baselines

B.3.3 Evaluation Metrics

Success Rate (SR ↑\uparrow↑).

WebShop Score (↑\uparrow↑).

Number of Steps (Steps ↓\downarrow↓).

Accuracy (Acc. ↑\uparrow↑) on reasoning benchmarks.

Evaluation protocol.

Appendix C Additional Analyses

C.1 Results on Gemini-3.1-Flash-Lite

C.2 Case Studies

Curated Skills for Different Tasks.

How SkillOS Curates Better Skills Compared to Baselines.

How Curated Skills Help to Solve Tasks Successfully.

Appendix D Limitations

Retrieval Mechanism.

Simplified Skill Representation.

Frozen Agent Executor.

Appendix E Future Research Directions

Agentic Search over Experiential Memory.

Hierarchical and Compositional Skills.

Multi-Agent and Shared Memory.

Appendix F Use of LLMs

Meow summary

The problem

The trick: test skills on later tasks

What the evidence says

Where it stops

Make notes of your own