Drafting Agents, Not Just Tokens: Speculative Recursive Self-Improvement for Agentic Systems
arXiv:2610.16491v2 · This workspace stays on this version. Check arXiv for newer versions
Some source details need review 25 to check
Original formulas, table values and figures are preserved. Comparison can miss differences; use the original PDF to check details.
Reading view checked: 531 math expressions, 0 rendering failures; 37 of 39 images available.
Compared 137 of 137 source passages against 25 of 25 PDF pages.
- Some figure, table or graphic images are unavailable; the original paper is still accessible. (2)
- Some graphics are laid out differently in the HTML source than in the paper (text or formulas outside their frame); check the original PDF. (21)
- S1_p5_pic1: the arXiv HTML draws text or formulas of this graphic outside its frame and it could not be corrected safely (too-small); the image shows the HTML layout, which differs from the PDF.
- S1_p5_pic2: the arXiv HTML draws text or formulas of this graphic outside its frame and it could not be corrected safely (too-small); the image shows the HTML layout, which differs from the PDF.
- S1_p5_pic3: the arXiv HTML draws text or formulas of this graphic outside its frame and it could not be corrected safely (too-small); the image shows the HTML layout, which differs from the PDF.
- S3_SS1_p2_pic1: the arXiv HTML draws text or formulas of this graphic outside its frame and it could not be corrected safely (too-small); the image shows the HTML layout, which differs from the PDF.
- S3_SS1_p2_pic2: the arXiv HTML draws text or formulas of this graphic outside its frame and it could not be corrected safely (too-small); the image shows the HTML layout, which differs from the PDF.
- S4_SS2_p1_pic1: the arXiv HTML draws text or formulas of this graphic outside its frame and it could not be corrected safely (too-small); the image shows the HTML layout, which differs from the PDF.
- S4_SS2_p1_pic2: the arXiv HTML draws text or formulas of this graphic outside its frame and it could not be corrected safely (too-small); the image shows the HTML layout, which differs from the PDF.
- S4_SS2_p1_pic3: the arXiv HTML draws text or formulas of this graphic outside its frame and it could not be corrected safely (too-small); the image shows the HTML layout, which differs from the PDF.
- S4_SS2_p2_pic1: the arXiv HTML draws text or formulas of this graphic outside its frame and it could not be corrected safely (too-small); the image shows the HTML layout, which differs from the PDF.
- S4_SS3_p1_pic1: the arXiv HTML draws text or formulas of this graphic outside its frame and it could not be corrected safely (too-small); the image shows the HTML layout, which differs from the PDF.
- S4_SS3_p1_pic2: the arXiv HTML draws text or formulas of this graphic outside its frame and it could not be corrected safely (too-small); the image shows the HTML layout, which differs from the PDF.
- S4_SS3_p1_pic3: the arXiv HTML draws text or formulas of this graphic outside its frame and it could not be corrected safely (too-small); the image shows the HTML layout, which differs from the PDF.
- S4_SS4_p1_pic1: the arXiv HTML draws text or formulas of this graphic outside its frame and it could not be corrected safely (too-small); the image shows the HTML layout, which differs from the PDF.
- S4_SS4_p1_pic2: the arXiv HTML draws text or formulas of this graphic outside its frame and it could not be corrected safely (too-small); the image shows the HTML layout, which differs from the PDF.
- S4_SS4_p1_pic3: the arXiv HTML draws text or formulas of this graphic outside its frame and it could not be corrected safely (too-small); the image shows the HTML layout, which differs from the PDF.
- S4_SS5_p1_pic1: the arXiv HTML draws text or formulas of this graphic outside its frame and it could not be corrected safely (too-small); the image shows the HTML layout, which differs from the PDF.
- S4_SS5_p1_pic2: the arXiv HTML draws text or formulas of this graphic outside its frame and it could not be corrected safely (too-small); the image shows the HTML layout, which differs from the PDF.
- S4_SS5_p1_pic3: the arXiv HTML draws text or formulas of this graphic outside its frame and it could not be corrected safely (too-small); the image shows the HTML layout, which differs from the PDF.
- A2_T1: blank, oversized or incomplete source rendering; capture unavailable.
- A3_SS1_p1_pic1: the arXiv HTML draws text or formulas of this graphic outside its frame and it could not be corrected safely (too-small); the image shows the HTML layout, which differs from the PDF.
- A3_SS2_p1_pic1: the arXiv HTML draws text or formulas of this graphic outside its frame and it could not be corrected safely (too-small); the image shows the HTML layout, which differs from the PDF.
- A3_SS2_p1_pic2: the arXiv HTML draws text or formulas of this graphic outside its frame and it could not be corrected safely (too-small); the image shows the HTML layout, which differs from the PDF.
- A3_T3: blank, oversized or incomplete source rendering; capture unavailable.
37 of 39 figure, table and graphic images captured. Open original PDF
Abstract
Improving an agent through successive revisions requires balancing exploration and evaluation under a limited budget. A narrow search can miss useful sequences of edits, while fully evaluating every intermediate version often exhausts the budget before those sequences yield a verified improvement. Inspired by the draft–verify principle of speculative decoding, we propose Speculative Recursive Self-Improvement (SpecRSI), a two-level framework coupling inner-loop agent search with outer-loop improver learning. SpecRSI drafts successors before their predecessors are fully evaluated, while adopting only fully evaluated, admissible versions. In the inner loop, Verification-Aware Tree Search (VATS) allocates the budget between further edits and outstanding checks using the predicted verified return of a continuation. In the outer loop, Group Verification Optimization (GVO) updates the improver from completed searches while keeping the real evaluator fixed. We evaluate SpecRSI on scientific optimization (including active queue management (AQM) for network infrastructure), coding, and general agent-system benchmarks. Under matched evaluation-call budgets, SpecRSI improves scientific terminal gain over Janus by 60.0%–96.3% and area under the budget curve by 32.1%–91.6%, while achieving higher means across all 50 scientific backbone–task pairs and outperforming MetaRSI-v1 by 1.30–2.70 percentage points (pp) in system score across twenty backbones. On instance-level comparisons, it further improves PyBaMM terminal gain by 142.6% and SWE-Pro success by 6.0 pp at 7.0% and 9.2% lower token cost, respectively. These results demonstrate the efficacy of the proposed approach.

Agent systems can improve through successive revisions to their prompts, tools, and workflows, and recursive self-improvement (RSI) extends this goal to the mechanisms that generate subsequent improvements. Recent systems revise agent programs (Zhang et al., 2026a; Wang et al., 2026b), modify the meta-agent that proposes revisions (Zhang et al., 2026b), adapt improvement operators and their scheduling in MetaRSI (Tan et al., 2026), or co-evolve target programs and proxy evaluators to allocate expensive real evaluations in Janus (Liu et al., 2026). Across these approaches, proposing changes and establishing their benefits both consume time and computation. This allocation problem is central to Web infrastructure and agentic systems: in active queue management (AQM), revising a controller to manage queue buildup and delay requires assessing traffic delivery, delay, loss, and fairness across diverse scenarios, consuming substantial simulator resources (Liu et al., 2026; de Almeida et al., 2024). This raises a central question: how should an agent allocate its limited budget between exploring new revisions and verifying their benefits? Useful revisions often support later improvements, yet identifying them requires expensive evaluation; the challenge is turning exploration into verified progress before resources run out (Figure 1).
Two limiting strategies expose this trade-off. Following one fully evaluated lineage gives every decision fresh feedback, but narrows the alternatives and places an evaluation barrier between dependent edits—for instance, when a coding-agent revision adds a file–test map that a successor uses for focused intermediate retesting while retaining the final full suite. Maintaining many lineages explores more broadly, but fully evaluating them all spends budget on branches that are never used. Extra workers cut elapsed time without making evaluation free, and under a fixed budget a wider search may not leave enough computation to finish the most useful branch. While prior systems partially relax this barrier by separating expansion from evaluation or validating in parallel (Wang et al., 2026b; Hu et al., 2026), we focus on drafting successors ahead of their parents’ full evaluation and jointly selecting further revisions and bundles of outstanding checks under the remaining budget.
Allowing successors to be drafted ahead changes where the budget should be spent next. A candidate that looks strong may still need several checks before it can be adopted, whereas a child that repairs an unhelpful parent may need only one remaining check. Evaluating a version solely by its apparent quality misses the cost of making it usable. Under a hard budget, what matters is a continuation—a sequence of further edits and checks from the current state—and the verified improvement it can still deliver with the remaining budget. Because this return must be estimated before paying for the checks that reveal it, scoring must shift from static candidate quality to the marginal verified return of further editing and checking.
Extending speculation from tokens to agent revisions therefore changes the search space, the verification process, and the learning objective. Agent versions are revisable programs whose successors may repair or build on a parent that is not yet admissible. Verification requires paid, complementary external checks with heterogeneous costs and dependencies; several may be jointly necessary for adoption, so a single target-model pass cannot supply this evidence. Progress must consequently be measured by realized verified gains under resource limits, accounting for all incurred search costs, including failed and unfinished work.
Contributions. We introduce SpecRSI, a framework for budgeted speculative agent improvement. Two-Level Speculative Self-Improvement. We decouple speculative exploration from verified adoption: inner searches extend incompletely evaluated versions but adopt only fully evaluated, admissible ones, while the improver remains frozen within each search and learns from complete-search feedback between rounds. Verification-Aware Tree Search (VATS). VATS allocates the remaining budget across drafting and verification: the Solver preserves affordable completion bundles, including individually low-value checks; the Scorer predicts verified gains attainable through further edits and checks within the remaining budget; and the Policy selects the next draft or verification plan. Group Verification Optimization (GVO). GVO assigns credit across independent complete searches from the same start, using verified gains from searches under matched resource caps to provide delayed credit to sampled edits and work choices for the Drafter and Policy, while training the Scorer on realized suffix returns and paid same-state comparisons.

Text in this graphic
1

Text in this graphic
2

Text in this graphic
3
Speculative decoding uses a cheap draft model to propose tokens ahead of a target model (Leviathan et al., 2023). Given prefix , let and :
where indexes the token position, is the sampled draft token, and is its acceptance probability. Evaluating a draft block in parallel, the target model retains the accepted prefix; at the first rejection, it samples a replacement token from the normalized positive part of and discards the remaining drafts, preserving the target distribution. SpecRSI brings this draft–verify separation to agent revisions (Figure 1), where acceptance requires complete admissible evidence instead of token-distribution equivalence (extended related work is in Appendix A).
In recursive self-improvement (RSI), we distinguish an agent version —comprising executable code and persistent prompts, tools, and workflows—from an improver that proposes and selects revisions; updating changes how subsequent rounds generate improvements. Given retained versions and observed history , a proposal–evaluation step within round is
where indexes steps under fixed , is the proposed version, and is its evaluation outcome. A full real evaluation returns , with utility (higher is better) and admissibility indicator . Based on , revises and ; retained stepping stones need not immediately outperform the incumbent.

Accurate benefit evaluation is central to guiding RSI toward better agents under a limited search budget. If benefit evaluation is inaccurate, search either overlooks useful multi-step edits or wastes resources on candidates that never become admissible. Effective allocation must therefore evaluate the verified improvement that continued work can deliver.
Estimating this benefit is non-trivial because a candidate’s current quality does not determine the value of continuing from it, due to two challenges. Benefit evaluation should reflect downstream verified improvement. Intermediate revisions may serve as stepping stones to later gains, but realizing those gains requires subsequent edits and real checks. Benefit evaluation should account for completion costs. Complementary checks, planning, and failed attempts all draw from the same finite resources, so even promising gains may be too costly to verify. Together, these challenges induce a performance–cost trade-off: a worthwhile continuation must offer sufficient verified improvement while remaining affordable to complete. Figure 2 illustrates this trade-off across ten backbones, and Section 4.5 analyzes broader lifecycle expenditure.

Text in this graphic
1

Text in this graphic
2

SpecRSI brings the draft–verify separation of speculative decoding to the RSI loop, allowing successor agent versions to be drafted before their parents are fully evaluated while allocating real checks selectively. Unlike token acceptance in Equation (1), adopting an agent requires complete admissible evidence. Specifically, we refine Equation (2) by organizing retained versions as nodes in a search tree anchored at a complete admissible root , where stores immutable agent versions and records executed edit edges, and by decomposing into paid checks. Each version freezes its prompts, tools, workflows, and permitted persistent state (including memory), with model-parameter edits allowed only under D/H/M permissions. An uneditable verification contract specifies a finite required-item set , execution and admission rules, and constraints; its score and admissibility condition instantiate the utility and indicator from Section 2.2. At any step, state tracks admitted evidence , history , remaining expense budget , remaining attempts , and worker count . In token-accounted runs, and expense charges are measured in k tokens under the registered cap (while end-to-end totals also include overhead outside that cap), and elapsed time is tracked separately. For version and item , indicates whether a valid outcome is present, regardless of whether it passed. The complete admissible set and its best observed gain are
where admissibility is judged under and is the observed adoption-ready gain rather than a learned estimate; because root is eligible, search-time , while held-out gain is evaluated separately. The fixed contract and execution interface constitute the real evaluator: defines valid evidence, VERIFY invokes to run selected real checks, and complete admissible evidence determines and ACCEPT eligibility. By contrast, the learned Scorer for proposed work serves as the score evaluator, predicting continuation gain with optional expense and elapsed-time penalties (Equation (6)) to guide budget allocation without supplying evidence, whereas DRAFT generates successor edits without assigning evaluation scores.
At outer round , a complete admissible agent serves as root . SpecRSI parameterizes the improver as , comprising Drafter parameters , work-selection Policy parameters , continuation-value Scorer parameters , and fixed menu settings . During search collection, all four components are frozen, with scorer snapshot . After independent inner searches (counting distinct search runs rather than shared-tree leaves), GVO updates the learned parameters:
where indexes the searches, each returns an admissible successor and trajectory log , and GVO updates from their union while keeping , , and the outer update rule fixed (Figure 3; see Algorithm S1 and Algs. S2–S3 in Appendix D). Across outer rounds, Reset returns the task root to , whereas Carry retains a complete admissible successor chosen by the registered inter-round selector; both choices operate independently of within-search acceptance and GVO updates, and neither resets the learned improver.
Whereas Janus co-evolves programs with proxy evaluators (Liu et al., 2026) and MetaRSI learns operator schedules (Tan et al., 2026), SpecRSI learns to allocate budget between drafting and verification from complete-search verified returns (Appendix A).
Within each inner search, Verification-Aware Tree Search (VATS) iteratively selects bounded work under expense and attempt caps: either , where indexes existing parent versions and bounds edit depth, or , where specifies verification jobs on existing versions. Either action may repeat until ACCEPT terminates the search. Because complementary checks cannot be scheduled by token-confidence ranking alone, VATS coordinates three modules under the frozen improver: the Solver constructs feasible completion bundles, the Scorer refines verification scope, and the Policy selects the next action before execution updates the state for replanning (Appendix D).
The Solver assembles completion bundles that fit within the remaining budget. For version , let denote missing items and the corresponding outstanding outcomes. A relation reader extracts masks, dependencies, shared setups, and source relations from existing records without running new checks or substituting forecasts for missing evidence (Appendix D). To ensure that versions can reach full admissibility even when individual checks carry little standalone value, the Solver forms bundled options:
where is the feasible work menu at decision , contains retained draft plans (which may extend incomplete parents), and contains feasible verification bundles . Specifically, a mixed-integer linear program (MILP) selects bundles covering target sets and cross-branch unions subject to prerequisites, shared setups, conflicts, and reserved budgets. While feasibility guarantees neither passing check outcomes nor exhaustive tree coverage, original completion seeds remain eligible throughout subsequent refinement.
The Scorer estimates the continuation value of state–work pairs under the frozen procedure (which encompasses edit generation and work selection under fixed menu rules), accounting for subsequent edits, checks, and branch switches:
where is the state after settling the current preparation cost from pre-preparation state (Alg. S2), and is the terminal state. Subsequent expense and elapsed time include later preparation and failed work but exclude sunk preparation at step . Fixed weights convert expense and elapsed time into score-unit penalties ( in cost-only settings; our reference profile sets , relying on resource inputs and hard caps rather than explicit return penalties). By definition, , and Scorer predictions never replace observed or admitted evidence.
Given a feasible bundle , let denote job together with its prerequisites. The frozen scorer snapshot refines by evaluating the marginal gain of adding :
where both terms are evaluated at the same conservative state that reserves the preparation cap. Comparing whole bundles captures cross-check interactions without summing isolated check scores, counting shared jobs and setups only once. Query and extension caps bound refinement overhead while preserving the eligibility of initial feasible seeds (Appendix D).
Once preparation cost is settled, the Policy selects work from the nonempty menu conditioned on observed relations . With temperature-scaled logits , the sampling distribution is
where , is the sampling temperature, and is the uniform-exploration rate. Because preference logits are parameterized separately from value estimates (with the ACCEPT logit fixed to zero), the Policy need not greedily maximize . Search collection samples from , whereas deployment selects the maximum-logit action (Appendix D).
DRAFT. Given draft plan , the Drafter initializes for each parent and sequentially generates edits up to depth using only the context observed prior to that edit:
where is sampled from , executes the edit, and stores the resulting immutable version . The executor runs this bounded plan on actual artifacts: incomplete parents may be extended, provided each edit waits for its immediate parent. Any invalid edit is charged to the budget and terminates that path while preserving its valid prefix.
VERIFY. For requested jobs , either retrieves valid identical records or executes real checks. Our core configuration runs direct checks only, whereas checked reuse imports results after scoped equality checks via the shared backend used by compatible controls. The executor records outcomes and updates the state before VATS replans (Appendix D).
ACCEPT. Admitted records must satisfy identity, scope, field, and provenance requirements, including for failed checks. Upon termination, VATS adopts
returning the root if it attains the maximum, and otherwise the first maximizer under a fixed version order. Only the adopted version requires complete admissible evidence, and the reference rule introduces no hidden completion or adoption margin.
Outer-loop training begins with supervised warm-up and then alternates between frozen-improver search collection and Group Verification Optimization (GVO) updates. Completed trajectories provide two supervision signals derived strictly from observed outcomes under the frozen improver rather than updated Scorer predictions: realized suffix returns train the Scorer (when return or pair records are present), and whole-search advantages train the Drafter and Policy over sampled choices (with empty empirical loss terms contributing zero).
Prior to the first collection round, supervised warm-up initializes by training Drafter to imitate executed legal edits, Policy to imitate legal plan choices, and Scorer to predict observed returns; unexecuted alternatives receive no labels. Throughout training, Solver feasibility rules and menu settings remain fixed, and neither Solver outputs nor environment outcomes carry policy likelihoods.
In each round, we run independent searches from a shared pre-preparation state under frozen , matching the root agent, verification contract, budgets, and worker count. Each search terminates at state without requiring every visited version to be fully evaluated, logging total expense and elapsed time (including initial preparation as well as failed or unfinished work):
where is the net return of search and is its leave-one-out advantage across the group . Here is the best observed complete admissible gain over root (Equation (3)), while and penalize total cost and time in score units; if a search yields no new complete improvement, its gain is zero while incurred costs are still charged. Comparing whole searches eliminates the need for a learned baseline and assigns delayed credit to intermediate edits through their verified descendants, even when those intermediate versions are not themselves admissible.
The Scorer is updated by minimizing a combined regression and pairwise objective, with weight . The regression loss fits realized, undiscounted suffix returns from executed plan starts:
where is the realized net return from executing at paid state , and collects the resulting state–work–return triples under the frozen procedure (including paid alternatives). Here is the remaining observed gain, and measure expense and elapsed time from step to termination; unlike , excludes sunk preparation at step while charging subsequent preparation, so estimates the conditional expectation via mean squared error.
To sharpen relative comparisons between legal plans at the same saved paid state , paid rollouts under the shared continuation policy estimate mean returns and their difference :
where stores matched tuples . While anchors absolute value levels (which pairwise differences alone cannot identify), directly supervises relative plan rankings. Generating these pairs restores paid state , forces a legal initial plan, and continues under the frozen improver; the resulting rollouts train only the Scorer, with all clones and rollouts charged to the budget and logged failures retained as valid data (whereas missing records are never imputed as zero).
The actor parameters are updated over sampled edit tokens and plan choices via , where reinforces choices from searches with positive advantage and regularizes divergence from the fixed post-warm-up actor (Appendix D). Using the PPO-style clipped objective (Schulman et al., 2017) for sampled choice under context and legal support , we have
where indexes the sampled choices in search , denotes the frozen collection parameters, and is the probability ratio between current and collection policies (evaluating Drafter probabilities for edit tokens and Policy probabilities for plan choices, with radius distinct from exploration rate ). Trajectory logs record token contexts, legal supports, and behavior probabilities (which are empty for VERIFY steps) so that numerator and denominator are evaluated under identical contexts, supports, temperatures, masks, and exploration settings. During gradient computation, collection probabilities and advantages are held constant, invalid sampled edits retain their likelihood and cost, and each choice contributes once without search-length normalization. In the single exact-replay step of Alg. S3, , so clipping is inactive and does not bound the update.
Finally, GVO freezes the updated improver for the next collection round; because actor updates shift the continuation distribution, subsequent rounds collect fresh searches to recalibrate the Scorer under the fixed contract , with Drafter and Policy supervision based on complete-search returns (Appendix D).
In this section, we conduct a comprehensive empirical evaluation to demonstrate the efficacy of SpecRSI, centered on the following research questions (RQs):
- 1.
RQ1: Does SpecRSI produce better successor agents? Section 4.2 compares SpecRSI against competitive baselines across tasks and backbones under matched evaluation settings.
- 2.
- 3.
RQ3: Does GVO learn a more effective improver? Section 4.4 compares frozen and GVO-updated improvers from identical starting agents and tracks cumulative gains over successive rounds.
- 4.
RQ4: How do planning capacity and parallelism trade off gain, cost, and latency? Section 4.5 analyzes candidate-plan limits, full-lifecycle token expenditure, and multi-worker execution latency.
Datasets. We evaluate five scientific tasks (PyBaMM (Sulzer et al., 2021), AQM (de Almeida et al., 2024), FDTD (Wu et al., 2024), Reactor (Braniff & Tian, 2024), and IRP (Violi et al., 2024)) and four agent-system benchmarks (Terminal-Bench 2.1 (Merrill et al., 2026), SWE-Pro (Deng et al., 2025), GPQA-hard100 (Rein et al., 2024), and AIME). We further evaluate specialized settings for code (SWE-Pro (Deng et al., 2025), HumanEval-Ext. (Chen et al., 2021), MBPP-Advanced (Austin et al., 2021), FixBench (Li et al., 2025)), prompt (GPQA-hard100 (Rein et al., 2024), MMLU-Pro (Wang et al., 2024), GSM8K-Opt. (Cobbe et al., 2021)), and workflow optimization (PlanBench (Valmeekam et al., 2023), Math (Fei et al., 2026), MBPP (Austin et al., 2021), Game24 (Yao et al., 2023)). Here HumanEval-Ext., MBPP-Advanced, GPQA-hard100, and GSM8K-Opt. are benchmark-derived profiles, FixBench uses Fixbench-RTL, and workflow Math follows EMAS; where a benchmark appears in multiple settings, each experiment specifies its own cohort, starting agent, and verification contract.
Baselines. We benchmark SpecRSI against five groups of baselines: (1) scientific optimization, following Janus’s comparison protocols (Liu et al., 2026): Alg.-Only (AlphaEvolve-style program evolution with a fixed evaluator) (Novikov et al., 2025), Direct-LLM (LAEA-style candidate-quality prediction) (Hao et al., 2024), Embed-GP (a Gaussian-process surrogate on fixed code embeddings) (Snoek et al., 2012), and Janus; (2) system-level RSI (Tan et al., 2026): Initial, Data-RSI, Harness-RSI, Model-RSI, Fixed DMH, a failure-signature Static router, and MetaRSI-v1; (3) code optimization: DGM (Zhang et al., 2026a), SICA (Robeyns et al., 2025), HGM (Wang et al., 2026b), our non-learning Beam-Spec control with speculative decoding (Leviathan et al., 2023), and FlashEvolve (async/optional) (Hu et al., 2026); (4) prompt optimization: MIPROv2 (with/without demonstrations) (Opsahl-Ong et al., 2024), Trace (Cheng et al., 2024), TextGrad (Yuksekgonul et al., 2025), and GEPA/GEPA+Merge (Agrawal et al., 2026); and (5) workflow optimization: Genesis/Genesis+SOP (Fei et al., 2026), AFlow (Zhang et al., 2025), ADAS (Hu et al., 2025), and EMAS (Fei et al., 2026). All comparisons match edit permissions, where MetaRSI-H and SpecRSI-H restrict updates to D+H (prompts, tool workflows, and non-test data) with frozen model weights (Tan et al., 2026, Sec. 5.3), and SpecRSI executes direct checks. In cross-backbone transfer tests (Figure 4), we freeze the shared Qwen-trained improver across all ten scientific and twenty system backbones (nineteen beyond the reference model; see the online reproduction notes), matching starting agents, contracts, seeds, and evaluation caps ( for Janus/SpecRSI in scientific tasks, and D+H splits and search caps for MetaRSI/SpecRSI in system tasks).
| Method | PyBaMM | AQM | FDTD | Reactor | IRP |
|---|---|---|---|---|---|
| Alg.-Only (Novikov et al., 2025) | 0.12±0.02 [1.5pt]7.3±0.1 | 0.09±0.03 [1.5pt]5.1±0.8 | 0.14±0.05 [1.5pt]8.5±0.6 | 0.11±0.05 [1.5pt]6.7±0.2 | 0.10±0.07 [1.5pt]5.8±0.6 |
| Direct-LLM (Hao et al., 2024) | 0.21±0.03 [1.5pt]19.2±0.8 | 0.17±0.01 [1.5pt]14.3±0.5 | 0.23±0.02 [1.5pt]22.6±0.3 | 0.18±0.06 [1.5pt]16.2±0.1 | 0.18±0.05 [1.5pt]13.7±0.3 |
| Embed-GP (Snoek et al., 2012) | 0.29±0.03 [1.5pt]30.5±0.4 | 0.26±0.01 [1.5pt]24.8±0.1 | 0.36±0.03 [1.5pt]36.2±0.9 | 0.29±0.03 [1.5pt]27.4±0.2 | 0.28±0.06 [1.5pt]23.1±0.1 |
| Janus (Liu et al., 2026) | 0.35±0.05 [1.5pt]42.8±0.3 | 0.30±0.03 [1.5pt]34.7±0.9 | 0.40±0.02 [1.5pt]51.4±0.7 | 0.33±0.02 [1.5pt]37.0±0.2 | 0.27±0.04 [1.5pt]22.6±0.1 |
| SpecRSI (ours) | 0.58±0.02 [1.5pt]57.7±0.4 | 0.52±0.03 [1.5pt]46.8±0.1 | 0.64±0.01 [1.5pt]67.9±0.2 | 0.55±0.01 [1.5pt]50.3±0.4 | 0.53±0.03 [1.5pt]43.3±0.1 |
| Method | TB2.1 | SWE-Pro | GPQA | AIME |
|---|---|---|---|---|
| Initial (Tan et al., 2026) | 23.6±0.7 | 10.2±0.3 | 71.2±1.0 | 54.9±0.5 |
| Data-RSI (Tan et al., 2026) | 27.4±0.3 | 12.4±0.8 | 73.8±0.3 | 57.6±0.1 |
| Harness-RSI (Tan et al., 2026) | 29.4±0.6 | 14.8±0.3 | 78.2±0.7 | 63.1±0.8 |
| Model-RSI (Tan et al., 2026) | 27.0±0.3 | 13.9±0.1 | 75.4±0.6 | 59.7±0.2 |
| Fixed DMH (Tan et al., 2026) | 30.3±0.6 | 15.3±0.3 | 79.5±0.2 | 64.2±0.6 |
| Static router (Tan et al., 2026) | 30.8±0.2 | 14.9±0.3 | 79.7±0.9 | 64.0±0.2 |
| MetaRSI-v1 (Tan et al., 2026) | 32.1±0.3 | 19.5±0.8 | 83.8±0.2 | 68.5±0.1 |
| SpecRSI (ours) | 35.6±0.6 | 23.7±0.1 | 87.3±0.3 | 72.9±0.3 |
| MetaRSI-H (Tan et al., 2026) | 28.6±0.5 | 14.2±0.2 | 77.5±0.5 | 61.7±0.2 |
| SpecRSI-H (ours) | 32.1±0.1 | 17.3±0.5 | 80.9±0.5 | 65.6±0.1 |
View original table

Evaluation Metrics. We report terminal gain () and area under the budget curve (AUBC) on scientific tasks following Janus (Liu et al., 2026), alongside task scores and mean inference cost (). Token-accounted searches track initial and remaining budgets and (in k tokens) separately from the evaluation-call limit , which is fixed across scientific methods at for PyBaMM, AQM, FDTD, Reactor, and IRP, respectively. Each call executes one instance’s full real-evaluation suite (counting failed runs, but excluding cached and proxy evaluations). Let denote the held-out score of the complete admissible incumbent selected on validation after calls (with held-out scores never used during search selection), and let . We compute
where is the number of evaluated instances and sums all input and output inference tokens over instance ’s full execution, including failed or unfinished attempts. Thus measures online inference cost in k tokens per task (whereas Figure 2A counts tokens up to the first qualifying outcome or termination, and seed totals or training-token costs are listed separately; Appendix C.1). Scientific gains use dimensionless task scores (Appendix B), and AUBC integrates gain over calls by carrying the final value forward to after early stopping. Task scores are reported as accuracy or success percentages (with absolute differences in percentage points, pp, and relative gains over the named baseline); higher gain, AUBC, and score and lower cost indicate better performance. Unless stated otherwise, we average three searches per training seed and report mean SD across five seed means.
Implementation Details. We implement SpecRSI using Qwen3.5-35B-A3B as the reference backbone, equipping the Drafter with rank-16 LoRA (Hu et al., 2022) and using separate 256-dimensional branches for the Scorer and Policy. Supervised warm-up trains for one epoch using AdamW (Loshchilov & Hutter, 2019) with a learning rate of , followed by two reference GVO collection-and-update rounds with independent complete searches per group (see Appendix B for full settings and Appendix C.2 for the 20-round extension).

In this section, we benchmark SpecRSI across tasks and backbones. Higher scientific gains. Across the five scientific tasks, SpecRSI improves mean terminal gain and call-axis AUBC over Janus by 60.0%–96.3% and 32.1%–91.6%, respectively (Table 1A). Better system-level successors. Under matched D/H/M permissions, SpecRSI outperforms MetaRSI-v1 by 3.5–4.4 pp, while restricting updates to D+H retains a 3.48 pp average advantage (Table 1B). Gains persist across backbones. Transferring the frozen improver across models improves all 50 scientific backbone–task pairs over Janus and outperforms MetaRSI-v1 by 1.30–2.70 pp across all twenty system backbones (Figure 4).

Text in this graphic
1

Text in this graphic
2

Text in this graphic
3


Text in this graphic
4
Benefits extend across optimization settings. On the eleven code, prompt, and workflow benchmarks, SpecRSI improves mean scores by 3.5–6.0 pp over the strongest baseline on each task while reducing inference cost by 3.2%–17.6% (Figure 5; see Appendix C.1 and Figure S1(A,B) for instance-level results).

In this section, we dissect verification scope and planning in VATS on PyBaMM (Figure 6). Bundles improve accepted outcomes. With full-state inputs and draft-ahead disabled, bundled verification raises accepted gain from 4.8 to 6.3 pp and hit rate (the share of 125 searches returning a complete, admissible agent with positive gain) from 84% to 92%, while reducing search cost from 47.96 to 44.99 k tokens. Draft-ahead and bundles yield the best mean outcomes. In the factorial comparison, combining draft-ahead with bundled checks achieves the highest accepted gain (6.02 pp) and hit rate (92%) alongside the lowest cost (47.64 k tokens). Planning improves the gain–cost trade-off. Full SpecRSI achieves the highest mean gain and lowest mean cost across all eight budget caps; at the 70% cap, it reaches 6.00 versus 4.73 pp for additive MILP planning while using 44.76 versus 49.29 k tokens (9.2% less). Under matched inputs, scoring continuation return (CR) also improves gain by 1.32 pp over candidate-quality scoring (CQ) at 7.4% lower search cost (Figure S2A,B; Appendix C.2).

Text in this graphic
1

Text in this graphic
2

Text in this graphic
3

In this section, we evaluate improver learning across starting agents and multi-round trajectories (Figure 7). GVO updates improve search quality. When starting searches either from the original task agent or from the previous round’s output (without resetting the improver), GVO-only updates raise gains from 4.10/4.80 to 4.60/5.68 pp, respectively (Figure 7A). In the matched factorial comparison (Figure 7C), GVO adds 0.80 pp under Greedy search and 1.30 pp under VATS; overall, full SpecRSI outperforms Greedy with a frozen improver by 2.00 pp at 19.5% lower total token cost (including training). Learning helps from the same starting agent. Holding the starting task agent fixed in a paired test, GVO updates increase gain from 5.7 to 6.9 pp (Figure 7A), yielding a paired improvement of 1.20 pp (95% CI , positive across all five seeds; Table S2). Gains accumulate across rounds. Across 20 rounds (comparing frozen before-training improvers against after-training improvers updated each round via GVO or MetaRSI’s rule), cumulative gain reaches 42.1 pp for updated SpecRSI versus 29.6 pp for updated MetaRSI and 21.2 pp for frozen SpecRSI, with positive mean per-round gains throughout rounds 2–20 (Figure 7B; see also frozen-improver transfer in Figure 4B and Appendix C.2).

Text in this graphic
1

Text in this graphic
2

Text in this graphic
3
| Cap | Gain (pp) | Planning cost (k tokens) |
|---|---|---|
| 8 | ||
| 16 | ||
| 32 | ||
| 64 |
| Category | MetaRSI-v1 | SpecRSI |
|---|---|---|
| Acquisition | ||
| Training | ||
| Search | ||
| Reuse checks | ||
| Copies | ||
| Failed work | ||
| Planning | ||
| Total |
| Wkr. | Gain (pp) | Time (s) | Total worker time (worker-s) |
|---|---|---|---|
| 1 | |||
| 2 | |||
| 3 | |||
| 4 | |||
| 5 | |||
| 6 | |||
| 7 | |||
| 8 | |||
| 9 | |||
| 10 | |||
| 12 | |||
| 14 | |||
| 16 | |||
| 18 | |||
| 20 |
View original table

In this section, we examine candidate-plan limits, lifecycle costs, and parallel execution (Table 2). More plans yield diminishing gains. Raising the candidate-plan limit from 8 to 32 increases gain from 5.2 to 6.3 pp, whereas doubling it to 64 adds only 0.2 pp while increasing planning cost from 3.9 to 5.6 k tokens (Table 2A). SpecRSI uses fewer lifecycle tokens. SpecRSI achieves lower mean token usage across all seven lifecycle categories, reducing full-run expenditure (including training and serialized copies) from 112.7 k tokens for MetaRSI-v1 to 106.3 k tokens (; Table 2B). Parallel workers reduce latency but use more worker time. Scaling from one to eight concurrent workers cuts elapsed wall-clock time from 1,287 to 192 s () at the cost of 19.0% more total worker time, while twenty workers further reduce latency to 100 s () with 55.2% more worker time as mean gain rises from 6.3 to 6.7 pp (Table 2C).

Text in this graphic
1

Text in this graphic
2

Text in this graphic
3
In this work, we present SpecRSI, whose two-level loop extends incompletely evaluated parents but adopts only complete, admissible versions. VATS allocates edits and complementary checks by predicted verified continuation return; GVO updates the improver from completed searches under a fixed verification contract. SpecRSI improves terminal-gain and AUBC means over Janus in all 50 scientific backbone–task pairs and exceeds MetaRSI-v1 by 1.30–2.70 pp in score across twenty backbones. Full planning outperforms six planning controls in mean held-out gain and mean total token cost across all eight budget caps. Starting-artifact controls support gains from improver updates beyond retaining stronger task agents. Instance-level comparisons show higher PyBaMM gain and SWE-Pro success with lower token cost. These results show that planning verification and learning the improver jointly improve agent performance and token efficiency.
References
- Agrawal et al. (2026) Lakshya A. Agrawal, Shangyin Tan, Dilara Soylu, Noah Ziems, Rishi Khare, Krista Opsahl-Ong, Arnav Singhvi, Herumb Shandilya, Michael J. Ryan, Meng Jiang, Christopher Potts, Koushik Sen, Alexandros G. Dimakis, Ion Stoica, Dan Klein, Matei Zaharia, and Omar Khattab. GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning. In Proc. ICLR, pp. 8479–8565, 2026.Source
- Austin et al. (2021) Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and Charles Sutton. Program synthesis with large language models, 2021.
- Braniff & Tian (2024) Austin Braniff and Yuhe Tian. A hierarchical multi-parametric programming approach for dynamic risk-based model predictive quality control. Control Engineering Practice, 152:106062, 2024.Source
- Chen et al. (2021) Mark Chen, Jerry Tworek, Heewoo Jun, et al. Evaluating large language models trained on code, 2021.
- Cheng et al. (2024) Ching-An Cheng, Allen Nie, and Adith Swaminathan. Trace is the Next AutoDiff: Generative Optimization with Rich Feedback, Execution Traces, and LLMs. In NeurIPS, volume 37, pp. 71596–71642, 2024.Source
- Cobbe et al. (2021) Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems, 2021.
- de Almeida et al. (2024) Leandro C. de Almeida, Washington Rodrigo Dias da Silva, Thiago C. Tavares, Rafael Pasquini, Chrysa Papagianni, and Fábio L. Verdi. DESiRED—Dynamic, Enhanced, and Smart iRED: A P4-AQM with Deep Reinforcement Learning and In-band Network Telemetry. Computer Networks, 244:110326, 2024.Source
- Deng et al. (2025) Xiang Deng, Jeff Da, Edwin Pan, Yannis Yiming He, Charles Ide, Kanak Garg, Niklas Lauffer, Andrew Park, Nitin Pasari, Chetan Rane, Karmini Sampath, Maya Krishnan, Srivatsa Kundurthy, Sean Hendryx, Zifan Wang, Vijay Bharadwaj, Jeff Holm, Raja Aluri, Chen Bo Calvin Zhang, Noah Jacobson, Bing Liu, and Brad Kenstler. SWE-Bench Pro: Can AI agents solve long-horizon software engineering tasks?, 2025.
- Fei et al. (2026) Chao Fei, Qingyi Si, Kaihua Liang, Yanghua Xiao, Panos Kalnis, and Hongcheng Guo. EMAS: Stabilizing multi-agent system evolution through evidence-guided revision, 2026.
- Hao et al. (2024) Hao Hao, Xiaoqun Zhang, and Aimin Zhou. Large language models as surrogate models in evolutionary algorithms: A preliminary study. Swarm and Evolutionary Computation, 91:101741, 2024.Source
- Hu et al. (2022) Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-Rank Adaptation of Large Language Models. In Proc. ICLR, 2022.Source
- Hu et al. (2025) Shengran Hu, Cong Lu, and Jeff Clune. Automated Design of Agentic Systems. In Proc. ICLR, pp. 21344–21377, 2025.Source
- Hu et al. (2026) Zhengding Hu, Mingge Lu, Zhen Wang, Jixuan Ruan, Chang Chen, Zaifeng Pan, Yue Guan, Ruiyi Wang, Zhongkai Yu, Chao Zhang, and Yufei Ding. FlashEvolve: Accelerating agent self-evolution with asynchronous stage orchestration, 2026.
- Huangfu & Hall (2018) Q. Huangfu and J. A. J. Hall. Parallelizing the Dual Revised Simplex Method. Mathematical Programming Computation, 10(1):119–142, 2018.Source
- Lee et al. (2019) Juho Lee, Yoonho Lee, Jungtaek Kim, Adam Kosiorek, Seungjin Choi, and Yee Whye Teh. Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks. In Proc. 36th ICML, volume 97 of PMLR, pp. 3744–3753, 2019.Source
- Leviathan et al. (2023) Yaniv Leviathan, Matan Kalman, and Yossi Matias. Fast Inference from Transformers via Speculative Decoding. In Proc. ICML, pp. 19274–19286, 2023.Source
- Li et al. (2025) Shijie Li, Weimin Fu, Yifang Zhao, Xiaolong Guo, and Yier Jin. Fixbench-RTL: A Comprehensive Benchmark for Evaluating LLMs on RTL Debugging. In Proc. AsianHOST. IEEE, 2025.Source
- Liu et al. (2026) Ximeng Liu, Qianlong Wang, Yingming Mao, Annan Li, Yatao Li, Shizhen Zhao, Jianmin Wu, Dawei Yin, and Dou Shen. Janus: An Algorithm-Evaluator Co-Evolution Framework for LLM-Driven Discovery under Expensive Evaluation Budgets, 2026.
- Loshchilov & Hutter (2019) Ilya Loshchilov and Frank Hutter. Decoupled Weight Decay Regularization. In Proc. ICLR, 2019.Source
- Merrill et al. (2026) Mike Merrill, Alexander Shaw, Nicholas Carlini, Boxuan Li, Harsh Raj, Ivan Bercovich, Lin Shi, Jeong Shin, Thomas Walshe, E. Kelly Buchanan, Junhong Shen, Guanghao Ye, Haowei Lin, Jason Poulos, Maoyu Wang, Marianna Nezhurina, Di Lu, Orfeas Menis Mastromichalakis, Zhiwei Xu, Zizhao Chen, Yue Liu, Robert Zhang, Leon Liangyu Chen, Anurag Kashyap, Jan-Lucas Uslu, Jeffrey Li, Jianbo Wu, Minghao Yan, Song Bian, Vedang Sharma, Ke Sun, Steven Dillmann, Akshay Anand, Andrew Lanpouthakoun, Bardia Koopah, Changran Hu, Etash Guha, Gabriel Dreiman, Jiacheng Zhu, Karl Krauth, Li Zhong, Niklas Muennighoff, Robert Amanfu, Shangyin Tan, Shreyas Pimpalgaonkar, Tushar Aggarwal, Xiangning Lin, Xin Lan, Xuandong Zhao, Yiqing Liang, Yuanli Wang, Zilong (Ryan) Wang, Changzhi Zhou, David Heineman, Hange Liu, Harsh Trivedi, John Yang, Junhong Lin, Manish Shetty, Michael Yang, Nabil Omi, Negin Raoof, Shanda Li, Terry Yue Zhuo, Wuwei Lin, Yiwei Dai, Yuxin Wang, Wenhao Chai, Shang Zhou, Dariush Wahdany, Ziyu She, Jiaming Hu, Zhikang Dong, Yuxuan Zhu, Sasha Cui, Ahson Saiyed, Arinbjörn Kolbeinsson, Christopher Rytting, Ryan Marten, Yixin Wang, Jenia Jitsev, Alex Dimakis, Andy Konwinski, and Ludwig Schmidt. Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces. In Proc. ICLR, pp. 40903–40986, 2026.Source
- Novikov et al. (2025) Alexander Novikov, Ngân V u, Marvin Eisenberger, Emilien Dupont, Po-Sen Huang, Adam Zsolt Wagner, Sergey Shirobokov, Borislav Kozlovskii, Francisco J. R. Ruiz, Abbas Mehrabian, M. Pawan Kumar, Abigail See, Swarat Chaudhuri, George Holland, Alex Davies, Sebastian Nowozin, Pushmeet Kohli, and Matej Balog. AlphaEvolve: A coding agent for scientific and algorithmic discovery, 2025.
- Opsahl-Ong et al. (2024) Krista Opsahl-Ong, Michael J. Ryan, Josh Purtell, David Broman, Christopher Potts, Matei Zaharia, and Omar Khattab. Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programs. In Proc. EMNLP 2024, pp. 9340–9366. ACL, 2024.Source
- Rein et al. (2024) David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. GPQA: A Graduate-Level Google-Proof Q&A Benchmark. In Proc. 1st COLM, 2024.Source
- Robeyns et al. (2025) Maxime Robeyns, Martin Szummer, and Laurence Aitchison. A Self-Improving Coding Agent, 2025. ICLR Workshop on Scaling Self-Improving Foundation Models (Oral).
- Schulman et al. (2017) John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal Policy Optimization Algorithms, 2017.
- Shao et al. (2024) Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models, 2024.
- Snoek et al. (2012) Jasper Snoek, Hugo Larochelle, and Ryan P. Adams. Practical Bayesian optimization of machine learning algorithms. In NeurIPS, volume 25, pp. 2951–2959, 2012.
- Sulzer et al. (2021) Valentin Sulzer, Scott G. Marquis, Robert Timms, Martin Robinson, and S. Jon Chapman. Python Battery Mathematical Modelling (PyBaMM). Journal of Open Research Software, 9(1):14, 2021.Source
- Tan et al. (2026) Zihan Tan, Leixin Sun, Zitong Shi, Yitao Liu, Jiajun Wu, Nathaniel Brooks, Jiaru Qian, Xiaoran Shang, Suyuan Huang, Yi Ding, Yangxu Liao, Mukai Li, Qiushi Sun, Shudong Liu, Xuankun Rong, Xiaohang Yu, Zhuo Chen, Hejia Geng, Chenxin Li, Aozhou Wang, Zengji Tu, Robert Tang, Yuxin Zhan, Eric Jiang, Yuxin Wu, Jianqing Zhang, Xiao Liang, Fang Wu, Haochi Zhang, Alexander Marlow, and Guancheng Wan. MetaRSI / RSI$^{2}$: A meta-recursive self-improving system for recursive self-improving systems themselves, 2026.
- Valmeekam et al. (2023) Karthik Valmeekam, Matthew Marquez, Alberto Olmo, Sarath Sreedharan, and Subbarao Kambhampati. PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Change. In NeurIPS, 2023.Source
- Violi et al. (2024) Antonio Violi, Annarita De Maio, and Gerarda Fattoruso. Inventory management and delivery of perishable products with stochastic demands and risks consideration. Procedia Computer Science, 232:2941–2949, 2024.Source
- Wang et al. (2026a) Kun Wang, Guibin Zhang, ManKit Ye, Xinyu Deng, Dongxia Wang, Xiaobin Hu, Jinyang Guo, Yang Liu, and Yufei Guo. MAS$^{2}$: Self-Generative, Self-Configuring, Self-Rectifying Multi-Agent Systems. In Proc. ICLR, 2026a.Source
- Wang et al. (2026b) Wenyi Wang, Piotr Piękos, Li Nanbo, Firas Laakom, Yimeng Chen, Mateusz Ostaszewski, Mingchen Zhuge, and Jürgen Schmidhuber. Huxley-Gödel Machine: Human-Level Coding Agent Development by an Approximation of the Optimal Self-Improving Machine. In Proc. ICLR, pp. 80356–80386, 2026b.Source
- Wang et al. (2024) Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, Tianle Li, Max Ku, Kai Wang, Alex Zhuang, Rongqi Fan, Xiang Yue, and Wenhu Chen. MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark. In NeurIPS, 2024.Source
- Wu et al. (2024) Rui Wu, Fengyao Ding, Feng Li, and Yingjie Liu. Inverse-Designed Low-Crosstalk CWDM (De)Multiplexer Assisted by Photonic Crystals. Journal of Lightwave Technology, 42(14):4899–4905, 2024.Source
- Yao et al. (2023) Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of Thoughts: Deliberate Problem Solving with Large Language Models. In NeurIPS, volume 36, pp. 11809–11822, 2023.Source
- Yuksekgonul et al. (2025) Mert Yuksekgonul, Federico Bianchi, Joseph Boen, Sheng Liu, Pan Lu, Zhi Huang, Carlos Guestrin, and James Zou. Optimizing generative AI by backpropagating language model feedback. Nature, 639(8055):609–616, 2025.Source
- Zhang et al. (2026a) Jenny Zhang, Shengran Hu, Cong Lu, Robert Lange, and Jeff Clune. Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents. In Proc. ICLR, pp. 104223–104294, 2026a.Source
- Zhang et al. (2026b) Jenny Zhang, Bingchen Zhao, Wannan Yang, Jakob Foerster, Jeff Clune, Minqi Jiang, Sam Devlin, and Tatiana Shavrina. Hyperagents, 2026b.
- Zhang et al. (2025) Jiayi Zhang, Jinyu Xiang, Zhaoyang Yu, Fengwei Teng, Xionghui Chen, Jiaqi Chen, Mingchen Zhuge, Xin Cheng, Sirui Hong, Jinlin Wang, Bingnan Zheng, Bang Liu, Yuyu Luo, and Chenglin Wu. AFlow: Automating Agentic Workflow Generation. In Proc. ICLR, pp. 34040–34077, 2025.Source
Recursive self-improvement. Recent agent-optimization frameworks—including DGM, SICA, ADAS, AFlow, and GEPA—iteratively refine agent programs, workflows, or prompts (Zhang et al., 2026a; Robeyns et al., 2025; Hu et al., 2025; Zhang et al., 2025; Agrawal et al., 2026). To guide multi-step evolution, HGM expands lineages using descendant outcomes while scheduling evaluations separately (Wang et al., 2026b), Hyperagents directly modifies the meta-agent (Zhang et al., 2026b), MetaRSI adapts improvement operators and their scheduling (Tan et al., 2026), and Janus co-evolves target programs with screening proxies under real-validation gates (Liu et al., 2026). By contrast, SpecRSI learns both successor drafting and verification allocation directly from the net verified returns of complete searches.
Speculative decoding. Standard speculative decoding accelerates generation by verifying cheap draft tokens in parallel with a target model (Leviathan et al., 2023). In agentic settings, FlashEvolve asynchronously overlaps evolution stages (Hu et al., 2026). SpecRSI instead brings speculation to persistent agent revisions: only adopted versions require complete verification, whereas intermediate ancestors may remain partially evaluated.
Learning for agent improvement. Whereas GRPO estimates group-relative advantages across responses to the same prompt (Shao et al., 2024) and MAS trains meta-agents via Collaborative Tree Optimization (Wang et al., 2026a), GVO assigns credit across independent complete searches launched from matched starting states. Its search-level returns reward verified gains net of all incurred costs—including preparation as well as failed or unfinished branches—using realized suffix returns to train the Scorer and whole-search advantages to update the Drafter and Policy.
In this section, we elaborate on the architecture, optimization, benchmark, and accounting settings used in Section 4.1 and Algorithms S1–S3, summarized in Table S1.

Text in this graphic
Implementation and evaluation
Architecture. Frozen BF16 text representations; separate Scorer/Policy branches with typed relations and four-head set attention (Lee et al., 2019). Inputs: observed outcomes, missing checks, remaining resources, proposed work. Warm-up. Component minibatch 1; Drafter/Policy/Scorer targets: executed legal edits/legal choices/realized returns. Unexecuted alternatives receive no targets. VATS. SciPy milp/HiGHS (Huangfu & Hall, 2018). Training/default limits: targets, pair unions, singleton closures / ; uncached Scorer queries, retained plans, accepted extensions/seed / . GVO. : verified gain under hard caps, with cost features and feasibility active (nonzero penalties use a separate profile). joint SGD step/collection; Scorer/actor rates ; . Zero momentum/weight decay; no gradient-norm clipping. Execution. Training: 2,048-token sequences, one H20/scientific domain. Inference: 32,768-token limit, separate H20. Other backbones: frozen transfer via provider APIs. Scientific evaluation: 12 held-out instances per task. PyBaMM v23.5; (dimensionless), (). Reference: fixed unoptimized industry defaults, distinct from root. Acceptance: physical bounds and convergence. AQM. 12 parameters/12 scenarios; . Throughput/link capacity, 95th-percentile delay/initial-controller delay; : loss fraction; : Jain fairness. Legal parameters, stable queues, no sustained packet loss. Other science. Scores normalized to . FDTD: convergence, crosstalk dB. Reactor: convergence, safe temperature/pressure. IRP: feasible constraints, service level . Public AQM. ns-3.44, seven RED/ECN parameters; three TCP NewReno patterns two seeds/case, 8 s/run; 20-Mb/s dumbbell, four flows. Mean utility ; : normalized goodput; : 95th-percentile end-to-end delay (s). Legal parameters and all six outputs required; measured simulator time charged. System tasks. Terminal-Bench 2.1, SWE-Pro v1.0, GPQA-hard100, AIME 2025 I/II (30 problems). Accounting. Table 2: all-round totals/seed, including fine-tuning input/output tokens and serialized snapshots/trajectories. Defaults: PyBaMM, 65-k-token search budget, 32 candidate plans, one H20; worker threads share global state/caches; worker time sums active execution. Figure S2E: forward input/output and forced-continuation tokens; search charged separately, backward computation in GPU time.
In this section, we present instance-level RQ1 evaluations, additional RQ2–RQ3 controls, and qualitative case studies on PyBaMM and SWE-Pro, complementing Sections 4.2–4.4.
In this subsection, we extend RQ1 to paired instances on PyBaMM and SWE-Pro (Figure S1) under the protocols in Table S3. Higher gains accompany lower cost. SpecRSI improves PyBaMM terminal gain by 142.6% over Janus and SWE-Pro resolution rate by 6.0 pp over HGM, while reducing token expenditure by 7.0% and 9.2%, respectively.

Text in this graphic
1

| Comparison | Difference | 95% CI |
|---|---|---|
| PyBaMM gain: vs. Janus† | ||
| SWE-Pro success: vs. HGM† | ||
| Reference-system mean: vs. MetaRSI-v1† | ||
| PyBaMM AUBC: vs. Janus† | ||
| Full Additive MILP (70%)† | ||
| Full state: bundle single | ||
| Same start: updated original |
View original table

In this subsection, we present additional VATS and GVO controls in Figure S2, with protocols detailed in Table S3. Continuation targets and evidence improve search. Under matched inputs, continuation-return scoring (CR) improves gain by 1.32 pp (95% CI ) over complete-evaluation gain scoring (CQ) while using 3.52 k fewer tokens (7.4%), and full evidence yields 6.30 pp versus 3.20–5.80 pp under restricted inputs. Supervision and GVO improve accepted gain. On identical plans, adding pairwise targets raises gain from 5.40 to 6.30 pp at higher training cost; relative to Greedy/frozen, VATS and GVO add 0.70 and 0.80 pp individually and 2.00 pp jointly (95% paired CI , interaction pp, 95% CI ; Figure 7C).

Text in this graphic
1

Text in this graphic
2


Text in this graphic
Evaluation protocols
Instance comparisons. Matched starts/token caps/cases; five paired within-seed means, pointwise 95% intervals . System scores weight four tasks equally. Held-out gain follows Equation (15), retaining negative values. Figure 2. First complete admissible output with (absolute gain if ), or best admissible output at termination. Per-task thresholds; equal task weights; token accounting: Section 4.1. Cohorts. Figure 6 uses separate cohorts for A, B, and C; Figure S2 uses separate cohorts for A–B, C, and D–E. Figure S2A,B. PyBaMM, original/matched: 25/8 searches/seed. Original: CQ omits resource inputs and uses its own stopping rule; CR uses full inputs. Matched: identical network/state inputs, stopping/ACCEPT, training data, and 65-k-token budget; only targets differ. Figure S2C. Eight PyBaMM searches/seed, 65 k tokens; same trained Scorer/Policy masked at test time, with complete Solver inputs and real verification intact. Flags/observations/relations: available/missing; values incl. admissibility/validity; dependency/shared-setup/source-reuse. Hidden/flags-only/values-only/restored/full: . Remaining budget/attempts always visible. Figures 6A,B / 7C. Mean final ACCEPT gain; failures/root returns count as zero in-search gain. Figure 6C accounting follows Appendix D; at 10%, mean total drafting/verification initialization/contract loading planning/Scorer queries/state checks. Figure S2D,E. Same snapshots/trajectories: mean returns with/without pairwise return-difference targets. Ordinary-search: mean targets, acquisition cost matched within 2%. Gain: independent held-out complete searches. Mean-only / +pair / ordinary gain: / / pp; training cost: / / . Figure 7A. Shared : Frozen/GVO-only/All roles update none/Drafter–Policy/all. SFT-only: random adapters/heads, one epoch on full search trajectories, actor cross-entropy + Scorer regression; warm-up uses fewer demonstrations. Figure 7B. Each curve: 20 rounds three searches; updated curves apply one GVO step or native MetaRSI update per round, while frozen curves retain initial improvers. All curves carry the previous best complete admissible output (or unchanged root), reset evidence/budgets, and sum within-seed gains over rounds 2–20. Figure 7C. PyBaMM; matched starts/permissions/contracts/budgets. Greedy scores current candidate gain without continuation scoring, draft-ahead, or bundles. Final frozen improver: eight independent searches/seed after training. Total cost follows the figure caption, including training collections.
In this subsection, we present two representative cases in Tables S4 and S5, separating single-seed execution traces from five-seed summaries. Search costs count input/output and tool-return tokens, excluding simulator or test runtime and separately tracked LoRA training (the online supplement provides an additional D+H csvcut case with frozen model weights).
| Setup. PyBaMM v23.5; shared root and 140 complete-call budget. Calibrates seven NMC–graphite single-particle-model parameters against a 1C discharge curve at C, including diffusivity, SEI resistance, and exchange-current values/SOC dependence. The root uses constant diffusivity/SEI resistance, has maximal mid-SOC voltage error, underestimates low-SOC resistance, and has a 100-fold exchange-current prefactor error. |
|---|
| Score. Appendix B defines scoring and admissibility. Fixed-reference RMSE: mV; root: , RMSE mV. The 15-mV target () is not an admissibility threshold. |
| Janus trace. Calibrates constant diffusivity and then SEI resistance, simulating each step; proxy screening rejects the joint edit after five proxy evaluations and two full 800-s simulations. |
| SpecRSI trace. Drafts piecewise SOC-dependent diffusivity, exponential SOC-dependent SEI resistance, and exchange-current/concentration corrections; one bound check and one final simulation verify all three edits without intermediate full simulations. |
| Janus, five seeds. Scores: ; mean (RMSE equivalent mV), gain , cost . Token shares: simulation interaction/proxy/generation . |
| SpecRSI, five seeds. Scores: ; mean (RMSE equivalent mV), gain , cost . Token shares: simulation interaction/bounds/generation/planning . |
| Analysis. By composing three sequential drafts before full simulation, SpecRSI achieves 119.9% higher mean gain at 6.7% lower token cost. |
View original table

| Setup. Shared root and 3.5-k-token budget under data, harness, and task-model editing. Repairs accumulation and adds --weight for weighted mean/variance, requiring 28 target tests plus clean syntax/lint. The root passes 21/28 (75%), lacks weighting, and incurs -order error on mixed large/small values. |
|---|
| MetaRSI-v1 trace. Replaces summation with Kahan compensation and then attempts weighted means; two boundary-test failures halt that branch after three full pytest runs, leaving the final version at 26/28 (92.9%) with weighting incomplete. |
| SpecRSI trace. Drafts compensated summation, weight parsing, weighted mean/variance, and boundary handling alongside a task-model LoRA update; syntax, precision, weighting-subset, and lightweight post-update checks precede a single full pytest/pyflakes run, passing 28/28. |
| Search cost. MetaRSI-v1: 3.185; SpecRSI: 2.974 (). Full-test interaction/generation/other search inference account for and , respectively, with SpecRSI spending 14% on staged checks. |
| Analysis. SpecRSI completes both features with fewer full test runs and lower token cost, jointly verifying code and adapter updates before adoption. |
View original table

In this section, we state the execution, MILP, and training rules supplementing Section 3 and Algorithms S1–S3 (with hyperparameters in Appendix B).
Writing , we define
where the post-warm-up reference remains fixed, detached, and strictly positive on the shared legal support, with both distributions evaluated under identical logged contexts, temperatures, masks, and exploration transforms.
Deployment runs a single frozen search, and forced continuations trigger no outer-loop updates. Disjoint preparation and execution ledgers charge all failed or unfinished work without imputing missing records as zero labels. At each step, the menu retains ACCEPT, feasible seeds, available edits, singleton checks, and affordable completions regardless of their isolated scores; every non-ACCEPT action consumes at least one attempt, leaving ACCEPT as the sole option when no other work is feasible, and inconsistent caps are rejected. Preparation tracks actual and reserved budget charges alongside total expense and cap (); in Figure 6C, setting means caps drafting and verification while separately capped preparation enters total cost, with cache hits requiring matched state, scorer snapshot, and plan.
Each job specifies covered keys , reserved expense , attempts , and a setup group with cost . Using binary decision variables for jobs and setups under target set and planning budget , the Solver solves
Only independently validated feasible covers seed the search (valid timeout incumbents need not be optimal); our implementation uses SciPy/HiGHS with round-robin targets and an independently seeded reservoir prior to observing outcomes.