Reliable Memory from Unstable Growth: Curating Self-Written Agent Playbooks with Randomized Experiments

The memory loop. A self-written memory system lets an agent learn from its own experience. It wraps three steps around the agent:

On a task $x$, the agent acts over many turns, an episode, and earns a success score $y(x,S)\in[0,1]$ when $S$ is served (reported $\times 100$). The value of serving $S$ is its mean success, $V(S)=\mathbb{E}[\,y(x,S)\,]$, and a playbook helps only when $V(S)>V(\emptyset)$. Our two reference points are No playbook ($S=\emptyset$) and the whole playbook (Whole, $S=P$). A growth run grows one playbook from scratch on the training tasks. Runs differ: with the tasks in another order and new sampling, the same procedure writes a different playbook, one that may help or harm.

Goal. In this work, we replace the judge and serve steps by measurement. For a playbook $P$ and a token budget $B$, the best set of lessons to serve is

and with several growth runs, the best of their sets $S^{\star}$, whole playbooks and no playbook. As $V$ is seen only in noisy episodes, we estimate it on practice tasks (Section 3.2) before serving.

Two systems. We study two established systems that write a playbook in different ways, to test whether a finding depends on how memory is written; ReasoningBank’s retrieval by similarity serves as a baseline (Section 6).

ACE. Our main system, ACE (Zhang et al., 2026), grows its playbook lesson by lesson. As the agent solves training tasks, a Reflector reads each trajectory and its feedback and writes new lessons; no lesson is ever removed. The Reflector also judges: it tags the lessons used in each episode as helpful or harmful, a record we can check (Appendix A). ACE serves the whole playbook.

Dynamic Cheatsheet. Our second system (Suzgun et al., 2026) rewrites its whole playbook, a cheatsheet: after each batch of training tasks, one model call reads it and the batch’s trajectories and writes a new one. It judges by itself, without the reward, which attempts succeeded, counts each entry’s successful uses (Appendix A.2) and, like ACE, serves its whole playbook.

Benchmarks. We conduct experiments on three benchmarks. In AppWorld (Trivedi et al., 2024), the agent writes code against the APIs of nine apps. In WebShop (Yao et al., 2022), it finds and buys a described product. In ALFWorld (Shridhar et al., 2021), it reaches a goal in a text household. They are scored by task goal completion, purchase reward and success rate.

Task splits. Each benchmark has three disjoint task sets: training tasks to grow playbooks, practice tasks to test playbooks before serving one, and evaluation tasks for the final score: AppWorld 168, WebShop 500 and ALFWorld 134 (Table 3).

Models and settings. We run three open models: Qwen3.8-27B (Qwen Team, 2026b), Qwen3.5-9B (Qwen Team, 2026a) and Gemma-4-31B (Gemma Team, 2026). A setting is one system, model and benchmark. Per model, ACE has four, two of them on AppWorld, where its Reflector sees whether a task passed (pass/fail) or each unit test’s result (per-test); Dynamic Cheatsheet, which never sees the reward, has three. Without 9B on ALFWorld (it solves almost no game), that makes eleven ACE and eight Dynamic Cheatsheet settings. Each setting has three growth runs.

Blocks. We measure groups of related lessons, as a single lesson rarely moves success beyond the noise of a few hundred episodes. We embed each lesson with all-mpnet-base-v2 (Reimers and Gurevych, 2019), as ACE does to find duplicates, and cluster the embeddings with Ward’s method (Ward, 1963) into $m=8$ blocks $C_{1},\dots,C_{m}$ (the number matters little; Appendix E.7). A block holds lessons on one topic, which tend to help or harm together. For a set of blocks $T\subseteq[m]$, we write $V(T)$ for the value of serving their lessons.

Randomized inclusion. Each practice episode $i$ draws a task $x_{i}$ and includes each block $j$ independently with probability $p=\frac{1}{2}$, $z_{ij}\sim\mathrm{Bernoulli}(p)$; the agent is served the included blocks, $T_{i}=\{\,j:z_{ij}=1\,\}$, and scores $y_{i}$. Each episode is thus both an ordinary run of the agent and one trial of a two-level factorial design over all $m$ blocks, each block on or off (Box et al., 2005; Figure 4, step 1).

What is estimated. A block’s gain can depend on the other blocks served with it, so we define its effect as its average gain over them, which at $p=\frac{1}{2}$ form a uniformly random subset $T\subseteq[m]\setminus\{j\}$:

the Banzhaf value of block $j$ (Banzhaf, 1965): how much the block raises success, averaged over what else is served with it. Random inclusion makes it measurable by a plain comparison of the episodes with and without the block:

The measurement thus needs no assumption on $V$ and few episodes: interactions between blocks are averaged into $\theta_{j}$, not assumed away (a block that adds $g$ only alongside another gets $\theta_{j}=g/2$), and one round of episodes measures every block, where leaving out one block at a time would need $(m+1)/2=4.5$ times as many episodes for the same precision. We compute $\hat{\theta}$ by a ridge regression of success on the inclusions, with one intercept per task to absorb task difficulty, minimizing

with the task intercepts $\alpha$ unpenalized and $\lambda$ chosen by leaving one task out. Unpenalized, this fit is consistent for $\theta_{j}$; under this balanced design, the penalty scales all estimates by about the same factor, keeping signs and order (Appendix D.1).

The curated set uses only the signs and order of $\hat{\theta}$ (Figure 4, step 2):

In Figure 4, the complete_task rules sit in block $C_{2}$, which gets the most negative estimate and is dropped, although its lessons are tagged helpful. The budget is a cap, not a target: a playbook with one helpful block is curated to that block alone. The rule is optimal when block effects add, $V(T)=V(\emptyset)+\sum_{j\in T}\beta_{j}$: then $\theta_{j}=\beta_{j}$, and without a budget the positive blocks maximize $V$ over sets of blocks. Within a budget, largest first is the greedy knapsack solution, exact for blocks of equal size; greedy would rank unequal blocks by effect per token, but dividing a noisy estimate by a small size inflates its noise, so we rank by effect. Blocks that interact can be worth more or less together than their estimates say, and one growth run may hold nothing of value; the last step checks both, scoring whole playbooks and no playbook.

RBC grows $K$ playbooks $P_{1},\dots,P_{K}$, with the training tasks in $K$ orders, and curates each into $\tilde{S}_{(k)}$. It then scores $2K+1$ candidates on fresh practice episodes (Figure 4, step 3), each with its own reason: the curated sets, best when block effects add; the whole playbooks, in case blocks help only together; and no playbook, in case every run harms. It serves the candidate with the highest practice score $\pi(S)$, its mean success on them:

The choice thus reuses no episode of the estimates and touches no evaluation task. Since no playbook ($\emptyset$) is itself a candidate, the choice is safe:

The bound is on true success, which the winning noisy score overstates, and carries over to evaluation tasks as far as they resemble the practice tasks (Section 6.5). The margin shrinks as $1/\sqrt{n}$ but is loose at our sample sizes, about a quarter of the scale (Appendix D.2); in practice the drop is far smaller: on evaluation tasks, RBC never falls more than 0.8 points below no playbook on ACE (Table 1). RSEA’s acceptance rule (Nguyen et al., 2026), which we call Best-Full, grows the same playbooks and serves the best whole one or no playbook by the same $\pi$: it does everything RBC does except curate, so their gap is what curating adds beyond choosing among runs.

Over all 19 settings, RBC averages 66.9 task success against 55.5 with the whole playbook and 54.1 with no playbook, the best of all methods (Table 1). The gain is broad, not carried by a few settings: RBC is above the whole playbook in 15 of the 19 and above both reference points for every model on both systems. Untuned on Dynamic Cheatsheet, RBC lifts success there to 66.5 from 61.3 (whole playbook) and 53.8 (no playbook).

Little harm remains on ACE. The whole playbook falls up to 69.6 points below no playbook, and every baseline, even Best-Full, at least 5.4 below it somewhere; RBC at most 0.8, within evaluation noise. On Dynamic Cheatsheet, it shares Best-Full’s two AppWorld drops (Table 1).

Cutting a playbook by judgement keeps its harm. At the same budget, the system’s own judgement, similarity to the task and random choice score no better than the whole playbook (at most 50.9 on ACE against 51.2, 58.9 on Dynamic Cheatsheet against 61.3), while measuring one growth run reaches 61.9 on ACE (RBC-cut; Table 1); each judge falls at least 60.3 points below no playbook in some ACE setting, the agent’s own tags furthest (72.6). Judgement fails because it does not track what a block does (Figure 5): over 81 blocks, ACE’s tags and the measured effects have a rank correlation of only 0.21, and the blocks that harm most carry more helpful tags than all but the most helpful ones. Judges given an advantage (similarity tuned on evaluation tasks, an LLM curator told each outcome) still stay below RBC-cut, and, from the same episodes, DG-Mem’s subset-Shapley estimates curate worse than RBC’s (Appendix E.2).

Choosing among growth runs, as Best-Full does, already avoids most of the harm; curating adds by keeping the helpful blocks of a harmful playbook. On ACE, RBC scores 67.2 against 66.1 while serving 60% of the whole playbook’s tokens against 78%, leads in 5 of the 6 settings where their choices differ, and by +1.9 on the seven settings not used to fix it. It gains most where all whole playbooks harm: 49.6 against 43.8 on 9B WebShop and 69.8 against 65.3 on Gemma AppWorld (pass/fail).

More runs do not replace curating. RBC is ahead of Best-Full at every number of runs (Figure 6). Choosing needs two more, weaker runs to catch up (66.8 against 67.2) and serves 27% more tokens; at matched growth and practice cost, RBC on two runs and Best-Full on three are level (65.7 against 66.1), and Best-Full serves 23% more.

Each part adds. Curating one run alone scores 61.9 on ACE and choosing alone 62.2; both on one run score 63.1, and over three runs 67.2 (Table 10), with the same order on Dynamic Cheatsheet.

It curates only where curating pays. Dynamic Cheatsheet rewrites its playbook as a whole, so its lessons are less separable, and a curated cheatsheet scores level with the whole one (61.1 against 61.3). RBC detects this on practice tasks and serves a whole playbook in 7 of 8 settings, the same one as Best-Full, so the two are level (66.5 against 66.9): it falls back on choosing.

ACE never removes a lesson, so playbooks keep growing. We grew three runs for up to ten passes on nine settings, leaving out WebShop for 9B and Gemma, since the 27B WebShop playbook barely grew (Table 2). On AppWorld with per-test feedback, whole playbooks then stop helping (46.6 against 78.4 with no playbook for 27B, 17.7 against 19.8 for 9B). Choosing can then only fall back on no playbook or a weak run, while RBC still finds the helpful blocks: 84.9 against 78.4 for Best-Full on 27B and 32.3 against 21.2 on 9B. Where a whole playbook is best, RBC serves the same one as Best-Full; in the other 6 settings it serves a curated set, at most 44% of the tokens, and scores 4.7 higher on average. Its lead of 3.1 over Best-Full thus comes entirely from curating, not from choosing.

Lessons are general rules, such as how to end a task, so choosing on practice tasks carries over to evaluation tasks when the two are alike: practice success ranks the seven candidates of a setting much as evaluation success does (median Spearman correlation 0.89), and RBC picks the best candidate on evaluation in 6 of the 11 ACE settings and one within half a point of it in 7. Where it misses, the tasks differ: AppWorld’s practice tasks come from another split. This largely explains the two Dynamic Cheatsheet AppWorld settings that stay below no playbook (9B and Gemma, pass/fail). Choosing on held-out tasks of the evaluation split instead raises RBC by a further 2.2 points on Dynamic Cheatsheet and 0.4 points on ACE (Appendix E.6).

A playbook. Each line of an ACE playbook is one lesson with an id and the agent’s tags, for example:

[sai-00001] helpful=166 harmful=0 :: Always extract dynamic identifiers (e.g., artist_id, track_id) from the output of previous API calls and use them in subsequent requests. Never assume or hardcode IDs unless they are explicitly static and known in advance.

The actor sees the lesson without the tags.

Growth. We run ACE’s batched mode (Zhang et al., 2026). On each batch of training tasks, the actor plays every task with the current playbook; the Reflector reads each trajectory with its feedback and tags the lessons that bore on it as helpful, harmful or neutral; the Curator writes new lessons, which are appended; and near-duplicate lessons are merged. No step removes a lesson, and the whole playbook is served on every call (Whole). Growth runs differ in the order of the training tasks and the actor’s sampling; a run stops once a pass over the training tasks grows the playbook by less than 10%, after at most five passes.

Feedback. On AppWorld, the Reflector sees either whether the task passed (pass/fail) or the result of each unit test (per-test); the two grow different playbooks and count as different settings. On WebShop it sees the reward, on ALFWorld success.

We run Dynamic Cheatsheet’s cumulative mode (Suzgun et al., 2026) with the actors and training tasks of our ACE runs. After each batch of training tasks, one model call reads the current cheatsheet and the batch’s trajectories, judges by itself, without the reward, which attempts succeeded, and writes a new cheatsheet that replaces the old one, with a count of each entry’s successful uses. The cheatsheet is served whole; RBC treats its entries as lessons and is otherwise unchanged.

Table 3 lists the settings and their tasks. Every method chooses on practice tasks only, and practice tasks are never scored. Scores are on 0–100, averaged over seeds per task and then over tasks. RBC was developed on four ACE settings, scored on tasks apart from their evaluation tasks: 27B on AppWorld with pass/fail feedback and on WebShop, and 9B on AppWorld with both feedback types.

Models. Qwen3.8-27B (27B), Qwen3.5-9B (9B) and Gemma-4-31B (Gemma) play every role in their own settings, with thinking off. Gemma is evaluated on an FP8 checkpoint (Micikevicius et al., 2022), and its growth runs also differ in precision.

Every budgeted method draws from the same lessons, fills the same budget $B$, and is scored on the same tasks and seeds. Baselines run at their defaults and are never tuned, except the two marked ∗, which are tuned on the evaluation tasks on purpose.

Random. Random lessons until $B$ is full; mean of two orders.

Own. On ACE (Tags), lessons ranked by their share of helpful tags; on Dynamic Cheatsheet, by usage count. This is the cut the system’s own judgement implies.

Top-$k$. The lessons most similar to the task text, as in ReasoningBank (Ouyang et al., 2026).

Tags∗ and Top-$k^{*}$. Per setting, the best of eight tag rules or of five values of $k$, chosen on the evaluation tasks, which can only favour them.

DC$_{self}$ and DC$_{outcome}$. Dynamic Cheatsheet’s curator, run over the fixed lessons on practice episodes to choose the lessons to keep within $B$. DC$_{self}$ judges success by itself; DC$_{outcome}$ is told the task outcome, the signal RBC uses.

Best-Full. RSEA’s acceptance rule (Nguyen et al., 2026) across $K$ runs: of no playbook and the $K$ whole playbooks, the one with the highest practice success, on the same runs and practice episodes as RBC (Algorithm 2).

DG-Mem’s estimates. DG-Mem’s subset-Shapley estimate (Wang et al., 2026), computed per block from the same practice episodes as RBC’s regression on the four 27B settings and served by RBC’s rule; only the estimator differs.

Table 4 lists every harmful block of the four harmful settings with one of its rules. The harm is not scattered: in each setting it comes from one family of rules that sound useful.

AppWorld: one family of rules. On all three AppWorld pass/fail settings the harmful blocks tell the agent to pass an answer to complete_task, the call that ends the task, even when the task expects none. With such a block, the agent answers more of these tasks and fails the check “answers match” more often. Under pass/fail feedback the Reflector never sees the checker and writes rules for an imagined validator; under per-test feedback it sees the failed tests and writes none.

WebShop: a different family. On 9B WebShop the harmful blocks make the agent verify a product again or abandon it, and with block b02 included it buys less often.

The tags. On 27B and Gemma the harmful lessons carry mostly helpful tags and sit near chance in the tag ranking (top-half share). On 9B, by contrast, the Reflector tags its harmful rules mostly as harmful. The rates are unadjusted means over practice episodes.

A playbook that is rewritten as a whole depends on its growth run just as one grown lesson by lesson (Figure 7, which repeats Figure 2 for Dynamic Cheatsheet). Its playbooks help on average (61.3 against 53.8 with no playbook), but the runs of one setting lie up to 37.3 points apart (Gemma ALFWorld), and on 9B WebShop one run helps while another harms. The instability of Section 4 is thus a property of self-written memory, not of one system.

One lemma does most of the work, and the three parts of the proposition follow from it: identification, precision and the effect of the ridge penalty.

Notation. Practice tasks are drawn from $\mathcal{D}$, and $\mu(x,T)$ is the expected success on task $x$ when the blocks in $T\subseteq[m]$ are served, so $V(T)=\mathbb{E}_{x\sim\mathcal{D}}\,\mu(x,T)$. Episodes are i.i.d.: on episode $i$ the task is $x_{i}\sim\mathcal{D}$, the inclusions $z_{i1},\dots,z_{im}$ are independent $\mathrm{Bernoulli}(p)$ variables, independent of $x_{i}$, and $\mathbb{E}[\,y_{i}\mid x_{i},z_{i}\,]=\mu(x_{i},T_{i})$ with $T_{i}=\{j:z_{ij}=1\}$. We write $\tilde{y}=y-\alpha(x)$ for success with the task mean $\alpha(x)=\mathbb{E}[\,y\mid x\,]$ removed, and $\sigma^{2}$ for the largest $\mathrm{Var}(\tilde{y}\mid z_{j}=b)$ over blocks $j$ and $b\in\{0,1\}$. For a block $j$, the other blocks included form a set $T\subseteq[m]\setminus\{j\}$ with probability

Proof. Tasks and inclusions are independent, so given $z_{j}=b$ the other blocks served form $T$ with probability $w_{p}(T)$ and the task is still drawn from $\mathcal{D}$. Averaging over $T$,

and subtracting the two lines gives Eq. (5).

Identification. At $p=\tfrac{1}{2}$ every weight is $2^{-(m-1)}$, the uniform distribution over the subsets of the other blocks, so by Eq. (2)

whatever $V$ is. The regression estimates the same quantity. By the Frisch–Waugh–Lovell theorem, and since the inclusions are independent of the task and of one another, the coefficient on $z_{j}$ in the population fit with task intercepts is $\mathrm{Cov}(z_{j},y)/p(1-p)$. With $\eta_{b}=\mathbb{E}[\,y\mid z_{j}{=}b\,]$,

so the coefficient is $\Delta_{j}=\theta_{j}$. With finitely many tasks the sample fit converges to this fit, so least squares ($\lambda=0$ in Eq. (3)) is consistent for $\theta_{j}$.

Precision. Let $\hat{\Delta}_{j}$ be the mean of $\tilde{y}$ over the $n_{1}$ episodes that include block $j$ minus its mean over the $n_{0}$ that leave it out. Given which episodes include $j$, the $\tilde{y}_{i}$ are independent and the task term cancels, so $\hat{\Delta}_{j}$ is unbiased for $\Delta_{j}=\theta_{j}$, with

as $n_{1}\approx pn$. Since $p(1-p)\leq\tfrac{1}{4}$, this is smallest at $p=\tfrac{1}{2}$: $4\sigma^{2}/n$ for every block, from the same $n$ episodes. Leaving out one block at a time instead runs $m+1$ arms (all blocks, and all but $j$ for each $j$) of $n/(m+1)$ episodes each, so each contrast has variance up to

the factor of the proposition (Box et al., 2005). The argument treats $\alpha$ as known; estimating it from the same episodes does not change the first-order variance when $n$ grows with the tasks fixed.

Ridge keeps signs and order. With the task means removed from $y$ and $z$ ($\tilde{y}$ and $\tilde{Z}$), the unpenalized intercepts drop out of Eq. (3), and the ridge estimate solves

If $\tilde{Z}^{\top}\tilde{Z}=cI$ with $c>0$, then $\hat{\theta}_{\lambda}=\tfrac{c}{c+\lambda}\,\hat{\theta}_{0}$: every estimate is scaled by the same factor in $(0,1]$, so signs and order are kept. Fair inclusion gives $\mathbb{E}[\tilde{Z}^{\top}\tilde{Z}]\approx\tfrac{n}{4}I$, so in a finite round only blocks with close estimates can swap order, and only estimates near zero can change sign.

Proof. Under additivity every bracket in Eq. (5) equals $\beta_{j}$, and the weights sum to one, so $\theta_{j}=\beta_{j}$ and

This sum is largest when $T$ holds every positive term, which proves (a), and, among sets of at most $\lfloor B/c_{0}\rfloor$ blocks, when $T$ holds the largest positive terms, which proves (b).

When effects do not add, $\theta_{j}$ still averages the interactions, but positive blocks can be worth less together than their effects suggest (Appendix E.7).

Let $S_{1},\dots,S_{C}$ be the candidates, and let candidate $c$ be scored on $n$ practice episodes with outcomes $y_{c1},\dots,y_{cn}\in[0,1]$, independent across episodes. Its practice score and expected practice success are

By Hoeffding’s inequality, for each candidate

and by the union bound over the $C$ candidates, all $C$ deviations are at most $\varepsilon$ with probability at least $1-2C\exp(-2n\varepsilon^{2})$, which equals $1-\delta$ for $\varepsilon=\sqrt{\log(2C/\delta)/(2n)}$. On that event, let $\hat{S}$ be the candidate with the highest $\pi$ and $S_{c}$ any candidate. Then

which holds for the best candidate and, since no playbook is one of them, for $S_{c}=\emptyset$.

The bound needs nothing about how the candidates were built, so it holds for any curation rule and any $K$; the price of more candidates is the factor $\log C$. It is loose at our sample sizes: with $C=7$ and the 171 practice episodes per candidate on AppWorld, $2\varepsilon$ at $\delta=0.05$ is about a quarter of the scale. In practice the choice lands much closer, within half a point of the best candidate on evaluation in 7 of the eleven settings (Section 6.5). The guarantee concerns the practice tasks; it says nothing about evaluation tasks that differ from them.

Table 6 averages Table 1 per model, the evidence for Section 6.1’s claim on every model.

An advantage does not make judging catch up with measuring (Table 7; per setting in Table 8, definitions in Appendix A.4). Tuned on the evaluation tasks, or told each task’s outcome, the judges still score below RBC-cut on the same runs and settings. The estimator matters as well: from the same practice episodes and with the same serving rule, DG-Mem’s subset-Shapley estimates give a worse curated playbook than RBC’s regression in three of the four 27B settings (Table 9). Judging can even undo a good choice: cutting the playbook that Best-Full chose to 40% of its tokens by similarity averages 62.6 on ACE and 62.3 on Dynamic Cheatsheet, against 66.1 and 66.9 for Best-Full alone.

Table 5 gives every growth run behind Figure 2 and Section 6. Within one setting, whole playbooks and even their curated sets can lie on both sides of no playbook, which is why RBC lets practice tasks choose among them instead of trusting any single run. Table 10 shows what each part of RBC adds on both systems.

Why choose rather than pool. A natural alternative pools the lessons of the $K$ runs, merges near-duplicates, clusters them into $8K$ blocks and curates once, with each block included with probability $1/K$ (Table 11). It scores below Best-Full on average, and below no playbook on 9B AppWorld pass/fail although the good run was in the pool: a playbook’s value can lie in its lessons together, which choosing among runs keeps and pooling breaks.

RBC spends its cost before serving, and saves at serving time. Curating one run adds one practice round to growth, in which every episode measures all blocks at once (Table 12); RBC repeats growth and the round for each of its $K$ runs and scores its $2K+1$ candidates, 3.3–4.1 times the episodes of curating one run at $K=3$, against 1.8–2.6 times for Best-Full. In tokens (Table 13), which also count the Reflector and Curator calls of growth, RBC with three runs serves fewer tokens per task than Best-Full with any number of runs, and Best-Full needs two more runs, and 17% more growth and practice tokens, to come level in success.

Long growth changes two choices of the growth recipe, both fixed before any result: up to ten passes over the training tasks instead of five, and no early stop. Everything else, including RBC and Best-Full, is unchanged, and each setting gets three new growth runs.

AppWorld’s practice tasks come from its development split and its evaluation tasks from test_normal (Table 3). To test whether this difference causes the misses of Section 6.5, we checked the candidates on tasks of the evaluation split without choosing on the tasks that are scored: each setting’s evaluation tasks are split in half, the candidate is chosen on one half and scored on the other, and the halves are swapped. We average over 50 random halvings and check Best-Full in the same way. Table 14 compares the two checks. Every conclusion of Section 6 holds under both. Checked on the evaluation split, RBC gains more over every reference and moves ahead of Best-Full on Dynamic Cheatsheet as well, and the AppWorld settings that fell below no playbook reach it or come close: the practice split, not the method, explains most of the misses.

Development and tuning. We chose $K=3$ in a pilot on the four development settings, scored on held-out task pools (AppWorld’s test_challenge split and half of WebShop’s practice tasks), never on evaluation tasks. A tuning round on the same settings then moved one choice at a time from this default ($K=3$, $B=40\%$, $m=8$; Table 15), under a rule fixed in advance: a choice moves only if it gains at least one point on the four-setting mean. None did, so the default ran unchanged on every setting.

The cap is part of the method. Serving every block with a positive estimate, instead of the best blocks within 40% of the tokens, scores lower from the same estimates (63.8 against 65.2; Table 16). The loss sits on Gemma ALFWorld and 9B AppWorld per-test. On Gemma ALFWorld, practice and evaluation estimates agree, so the loss comes from blocks with small positive effects that do not add up when served together, not from noise; the cap keeps only the blocks whose effect is clear.

One practice round is enough. When the round is subsampled, the value of the curated set rises up to three quarters of the round and is flat from there (Table 17); a model of the estimates’ variance predicts that a round two or four times larger would add less than the evaluation noise of the comparisons.

The harm is not an artefact of thinking off. We regrew the 9B WebShop playbook with thinking on in every role. With thinking on as with it off, the whole playbook scores below no playbook and the curated set above every tag and retrieval rule (Table 18).

Task success cannot be replaced by an LLM judge. On AppWorld we also estimated block effects from a goal check by the model itself instead of task success. With every judge prompt we tried, including a judge from another model, the set curated on the judge’s estimates failed on 27B and 9B AppWorld pass/fail, in line with the known biases of LLM judges (Zheng et al., 2023; Wang et al., 2024): the measured outcome is what makes RBC work.