Evolving Agents in the Dark: Retrospective Harness Optimization via Self-Preference

Harness optimization improves an agent by editing the prompts, program parameters, or workflow code that surround a fixed model. One line optimizes prompts or pipeline parameters against a labeled metric, spanning LLM-as-optimizer search (Yang et al., 2023), declarative pipeline compilation (Khattab et al., 2023), textual-gradient updates (Yuksekgonul et al., 2024), reflective prompt evolution (Agrawal et al., 2025), and cross-model prompt evolution (Wu et al., 2026). A more agentic line lets a meta-agent rewrite the agent’s own code, where ADAS searches the space of agentic system designs (Hu et al., 2024), Meta-Harness searches over harness code using the execution traces and scores of prior candidates (Lee et al., 2026), and M⋆ evolves the memory system as an executable program, scoring candidate memory programs on a labeled validation set (Pan et al., 2026a). Although these methods differ in the surface they edit, all of them steer the search with a labeled validation metric. RHO departs from this paradigm, requiring no validation feedback and improving the harness in a single retrospective pass over unlabeled past trajectories.

A second line improves agents from their own past experience, using the agent’s self-judgment over trajectories in place of ground-truth labels. Dynamic Cheatsheet maintains a self-curated memory of reusable strategies and code snippets at test time (Suzgun et al., 2025), while ReasoningBank distills generalizable reasoning strategies from self-judged successes and failures (Ouyang et al., 2025). MemMA coordinates the memory cycle with multiple agents and repairs its memory bank against self-generated probe questions (Lin et al., 2026), and Sleep-time Compute precomputes useful context offline before queries arrive (Lin et al., 2025). Concurrent to our work, SkillOS instead trains a skill curator with reinforcement learning from outcome and judge rewards, updating a skill repository from accumulated experience (Ouyang et al., 2026). These methods enrich an agent’s stored memory, context, or skill list while leaving the rest of the harness untouched. RHO instead optimizes the full harness, including executable tools and instructions, rather than memory alone. Appendix A gives a detailed comparison with related work.

Given a large set of past trajectories, we need to extract the most critical signals to guide the harness optimization. Optimizing the harness on every individual trajectory is computationally prohibitive, and it further risks diluting important signals with trivial ones. To address this, we first select a coreset $\mathcal{D}_{\mathrm{core}}$ from the full set $\mathcal{D}$ to represent the trajectories that require optimization the most, echoing trajectory-based data valuation that scores training samples by their influence on optimization (Xu et al., 2025). Specifically, we require the coreset to capture both challenging and diverse scenarios. This requirement encourages our optimization to cover a wide range of failure modes when addressing the most difficult problems. To accomplish this, we introduce a Determinantal Point Process (DPP) kernel (Kulesza and Taskar, 2012) to rank all past trajectories by difficulty while satisfying a diversity constraint. In practice, we employ a language model judge to analyze every trajectory $\tau_{i}$ and extract a difficulty score $r_{i}$ alongside a textual description. This description details the specific challenges of the problem and potential failure modes. We then compute the embedding of this description and use the cosine similarity between embeddings as the similarity metric $S_{i,j}$ for any two trajectories $\tau_{i}$ and $\tau_{j}$. By considering both the difficulty scores $r$ and the trajectory similarity matrix $S$, we construct an L-ensemble kernel matrix

where $\widetilde{r}_{i}$ is a scaled version of the trajectory’s difficulty score $r_{i}$:

With this kernel function $L$, DPP selects a subset $Y$ with probability proportional to the kernel determinant $\det(L_{Y})$, using parameter $\theta$ to adjust the relative importance of difficulty and diversity via $\alpha$. With $\theta=1$, the trajectories are ranked purely by difficulty and $\theta=0$ (uniform weights) ranked purely by similarity diversity. Using $\theta=0.7$, we select $k$ trajectories into a coreset $\mathcal{D}_{\mathrm{core}}$ that covers difficult, diverse failure modes for later stages.

Inspired by previous work that computes advantages relative to a group of sampled rollouts, using the group as a baseline in place of a learned value model (Shao et al., 2024), we generate a set of trajectories by running $G$ parallel agent solves on each coreset task. Subsequently, the agent compares these group trajectories to identify underperforming runs. The agent then uses contrastive signals within the group to formulate instructions for optimizing the harness. Specifically, we perform this self-preference analysis along two dimensions.

These $\mathrm{rank}_{\mathrm{val}}$ and $\mathrm{rank}_{\mathrm{con}}$ analyses yield structured evaluations in JSON format, and for each task their union forms the improvement instruction $I_{t}=\mathrm{rank}_{\mathrm{val}}\bigl(t,\{\tau_{t,g}\}\bigr)\cup\mathrm{rank}_{\mathrm{con}}\bigl(t,\{\tau_{t,g}\}\bigr)$. As a result, we merge $\{I_{t}\}$ across all tasks in the coreset to form the final harness improvement instructions.

After obtaining the improvement instructions, we optimize the harness by providing these instructions to the agent. However, as observed in prior studies on agent evolution (Agrawal et al., 2025; Hu et al., 2024; Lee et al., 2026), harness optimization is inherently stochastic and may not reliably improve performance even with valid input signals. To mitigate this limitation, we sample harness proposals in parallel and filter them using agent self-preference. This selection is designed to favor candidates whose improvements generalize to future tasks. Specifically, we execute $N$ parallel optimization calls to generate $N$ candidate harnesses, denoted as $h_{1}$ to $h_{N}$. Following this step, we use these candidates to obtain $N$ sets of new trajectories on the $k$ coreset tasks. For every coreset task, we then compute an agent preference score by ranking the new trajectory from each candidate harness against the old trajectory from the original harness. We aggregate these scores across the coreset to determine the relative advantage score of each candidate:

where $\tau_{t}^{(0)}$ is the original harness trajectory for task $t$. Finally, we return the candidate harness with the maximum relative advantage to replace the original one. We accept this update only if its score is strictly greater than zero ($S_{j}>0$).

We compare RHO against three competitive harness optimization methods that do not require validation feedback. For the baselines, Dynamic Cheatsheet maintains a running record of useful facts and procedures (Suzgun et al., 2025). ReasoningBank stores reusable reasoning patterns and retrieves the top-$k$ relevant entries at inference time (Ouyang et al., 2025). Similarly, Sleep-time Compute preprocesses past traces offline into compact notes, which are then prepended to the agent’s context (Lin et al., 2025). We adapt each baseline to our datasets and agent setting while holding the total agent-call budget approximately fixed to ensure a fair comparison. Detailed adaptation procedures are provided in Appendix F.

As Table 1 shows, RHO delivers consistent improvements across all three benchmarks, whereas the baselines do not. Most notably, we achieve an absolute improvement of 19% on SWE-Bench Pro without relying on any validation-based grading. We attribute this advantage to the more flexible harness optimization that RHO enables. Specifically, the agent can create new tools, skills, and instructions for the harness, whereas previous methods focus predominantly on memory systems or text-based skills. Furthermore, the use of self-preference may contribute to the consistency of these harness improvements. In contrast, the performance gains of the baseline methods tend to be smaller and vary across different datasets. In the next section (Section 5.2), we examine how RHO modifies the harness to improve the agent.

Figure 3 summarizes and interprets the new harness contents generated after RHO optimization. In our work, the harness is materialized as a directory containing markdown files for instructions and skills, as well as executable scripts for tools.

Across all three benchmarks, RHO adds multiple new skills and tools to the harness. Many of these new additions address typical failure modes encountered by the original harness. For example, in SWE-Bench Pro, the agent learns that the Go toolchain resides at a non-standard location outside the default path. It also discovers that Python cache directories must be stripped before producing the final diff, as failing to do so often prevents patches from applying cleanly. To address these, the agent adds a new check_build_and_lint tool that locates non-standard toolchains and flags the generated artifacts that must be kept out of the patch, fixing the diff-hygiene procedures the original trajectories repeatedly missed. These examples illustrate how RHO identifies useful tools and skills across diverse scenarios by analyzing past failures.

We next compare RHO against Meta-Harness (Lee et al., 2026). Meta-Harness is a validation-feedback optimizer: a proposer emits harness edits, each candidate is graded on a labeled search set with the ground-truth grader, and the search keeps the harnesses that score best on that set. To maintain a fair comparison, we use the same Codex agent as the Meta-Harness proposer and solver. Unlike RHO, Meta-Harness requires held-out labels, and because it evolves over multiple rounds, it consumes more agent calls. Therefore, we evaluate it at a single round to match our compute budget, as well as in an extended-budget setting of ten rounds.

At the matched single-round budget, Meta-Harness selects its best candidate using validation scores but achieves only a 0.62 pass rate on SWE-Bench Pro. This is substantially lower than the 0.78 pass rate achieved by RHO. Scaling Meta-Harness to a 10-round setting increases its performance ceiling to 0.80 on SWE-Bench Pro. However, this higher performance requires roughly three times the optimization-phase compute of RHO (320 vs. 103 agent calls) and, more importantly, still depends on held-out labels that RHO does not use.

Although RHO creates new skills and tools for the agent, it is not immediately apparent through which mechanisms these updates enable the agent to perform better on future tasks. To examine this, we visualize the frequency of tool calls and the cumulative success rate with respect to the number of steps taken by the agent. Specifically, Figure 4 plots the cumulative fraction of held-out tasks resolved within a given number of agent steps, and it shows how the action mix of the agent changes over time. We observe that the performance improvements across all three datasets primarily originate from higher success rates on tasks requiring long horizons. In contrast, the gains concentrate on long-horizon tasks rather than those completed in fewer steps, most clearly on SWE-Bench Pro. Furthermore, the optimization process changes the working patterns of the agent. Consequently, the optimized agent shifts toward relying more heavily on specific types of actions. For example, on SWE-Bench Pro, the agent verifies its work much more frequently. This proactive verification appears to account for a large portion of the performance gains on long-horizon tasks. On Terminal-Bench 2 and GAIA-2, the agent increases its accuracy by actively applying newly developed tools. The newly added tools are invoked on nearly every held-out task, and on GAIA-2 the new environment helper runs about twenty times per task. Terminal-Bench 2 is an instructive exception—there the generated scripts are never executed, and the gain instead comes from the procedural playbook written into the instructions, showing that RHO adapts the harness surface to each environment rather than always adding executable tools. Complementary to our action-level statistics, token-level attribution over long reasoning chains offers a finer-grained lens on long-horizon agent behavior (Pan et al., 2026b).

We investigate how coreset selection influences the optimization process. To this end, we compare our DPP-based selection against several ablation strategies. These variants include selecting tasks solely by difficulty ($\theta=1$), selecting tasks purely to maximize coverage ($\theta=0$), and sampling trajectories randomly. In addition, we measure the final performance of the optimized harness under each selection strategy. As the t-SNE projection of task embeddings in Figure 5 reveals, selecting tasks exclusively by difficulty causes the chosen samples to cluster in a narrow region of the task distribution. This clustering occurs because the language model judges certain types of tasks as inherently more difficult, and consequently it fails to include other task types in the coreset. As a result, this strategy yields no meaningful performance improvement after optimization. Similarly, optimizing solely for coverage also produces suboptimal results. In contrast, random sampling can occasionally select a trajectory that proves useful for optimization. These findings suggest that a coreset selection strategy balancing both difficulty and diversity is vital for providing the proper signals to guide harness optimization.

Because RHO operates without ground-truth labels for reference, we evaluate whether its optimization outputs are consistent across runs. This consistency is closely tied to the overall effectiveness of our best-of-$N$ harness proposal. We examine whether the selection strategy reliably identifies the candidate harness that performs best on downstream tasks. In this experiment, we measure the test scores of all three generated candidate harnesses rather than just the most preferred one. Here Table 3 shows that the generated harnesses exhibit only moderate variance. Notably, even the lowest-scoring candidate still meaningfully improves agent performance over the baseline. Furthermore, the best-of-$N$ selection prevents the deployment of poorly performing harnesses. Specifically, the chosen harness scores higher than the worst candidate across all three benchmarks. At the same time, the most preferred harness does not invariably coincide with the highest-scoring candidate on the test set, though the selection consistently avoids the worst candidate. Comparing each candidate’s preference score $S_{j}$ with its held-out pass rate further shows that self-preference is a conservative selector rather than a perfect ranker: on SWE-Bench Pro a less-preferred candidate in fact reached 0.85, above the chosen one’s 0.78. What the mechanism reliably guarantees is safety rather than perfect ranking. Because an update is deployed only when the agent prefers it over the current harness ($S_{j}>0$), the deployed harness never fell below the vanilla baseline on any benchmark; every candidate beats vanilla on SWE-Bench Pro and GAIA-2, and on Terminal-Bench 2 the worst candidate matches vanilla (0.71) while the chosen one reaches 0.76.

We analyze the contribution of the two signals extracted during retrospective analysis, namely self-validation and self-consistency. We compare these explicit signals against a more direct approach that provides raw trajectories during optimization and skips the explicit retrospective analysis step. In addition, we study the individual contribution of each diagnostic signal to the final performance.

To investigate the necessity of the group rollout and diagnostic stages, we conduct an ablation study. Specifically, we remove the self-validation and self-consistency signals independently, and we subsequently rerun the optimization and evaluation procedures. We also introduce a raw trajectory baseline. This baseline bypasses the separate ranking analysis. Instead, it provides the original trajectories directly to the optimization step, asking the agent to analyze the trajectory and propose improvements in a single pass. Table 4 shows that removing either the self-consistency or the self-validation signal consistently degrades final performance across benchmarks. This result indicates that both signals are highly important for optimizing the harness. Additionally, full diagnosis outperforms the simplified raw-trajectory baseline on all three benchmarks. The raw-trajectory baseline grants the proposal agent the same editable surface as RHO—tools, skills, and instructions—so the gap to full diagnosis (0.60 vs. 0.78 on SWE-Bench Pro) reflects the value of the structured retrospective analysis rather than a broader editing surface. This result suggests that although single-pass trajectory analysis is feasible, the explicit self-validation and self-consistency signals are essential rather than incidental, yielding more reliable improvements across datasets. Finally, Table 4 is a controlled ablation that removes one diagnosis cue while holding the rest of the pipeline on a fixed path ($n{=}1$), so it measures the marginal contribution of each cue; it should not be read as the end-to-end effect of disabling a signal. To measure run-to-run robustness, we additionally reran the full pipeline end to end twice per ablated strategy, with independently generated candidate sets: every rerun configuration stayed within $\pm$0.01–0.03 of its mean ($0.79\pm 0.01$ without self-consistency and $0.815\pm 0.025$ without self-validation on SWE-Bench Pro, $n{=}2$), and in no rerun did performance drop to the controlled-ablation level. With $n{=}2$ this is indicative rather than conclusive, but it shows that the best-of-$N$ proposal and the remaining signal compensate when a cue is missing, so removing one component does not destabilize the pipeline.

Every $\mathrm{solve}(h,t)$ call materializes the harness and the task into a fresh workspace at harness/ and task/ and hands the agent the wrapper instructions below. The wrapper is task-agnostic, in that the harness directory carries all task-shaping guidance, and the agent is told only how to read the workspace and how to deliver a final answer. The same wrapper is reused for the baseline rollout and for every candidate-harness rollout so that $\mathrm{solve}$ is the only varying input.

The difficulty judge produces the score $r_{i}\in[0,10]$ and the abstract fingerprint $\phi_{i}$ that drive the DPP coreset selector of §4.1. It sees the task description together with a length-bounded digest of one short prior trajectory under the current harness. The digest is truncated head/tail to a fixed token budget, and any commands that read the task’s expected-answer files are scrubbed before the digest is shown to the judge. The judge is asked to keep the fingerprint in task-agnostic structural vocabulary so that fingerprints from different codebases remain comparable under cosine similarity.

{query} is the natural-language task description. {trajectory_digest} is the scrubbed, head/tail-truncated digest of one prior trajectory for the same task. This difficulty value is the score $r_{i}\in[0,10]$ that enters the DPP kernel, where it is normalized by $r_{i}/10$ as in §4.1, and the fingerprint is embedded to a unit vector $x_{i}$ that defines the similarity matrix $S=XX^{\top}$.

The diagnosis prompt implements $I_{t}=\mathrm{rank}_{\mathrm{val}}(t,\{\tau_{g}\}_{g=1}^{G})\cup\mathrm{rank}_{\mathrm{con}}(t,\{\tau_{g}\}_{g=1}^{G})$ (§4.2). The workspace presents the agent with the original task, the shared harness used by every rollout, and $G$ rollout directories. The agent executes a five-step workflow, comprising per-trajectory inspection, failure-mode analysis (self-validation), cross-trajectory disagreement analysis (self-consistency), a single high-level harness improvement direction, and a severity score that doubles as a soft attention weight downstream. The structured JSON output binds each field to a fixed slot so $\mathrm{optimize}$ can attend by severity.

The trajectory directories carry the full event stream, the agent’s final message, and any workspace diff produced by the rollout. The $-$self-validation and $-$self-consistency ablations in Table 4 remove the corresponding step and the corresponding output field, and the surrounding scaffolding is left intact.

The optimization prompt implements $h_{j}=\mathrm{optimize}(h_{0},\{I_{t}\}_{t\in\mathcal{D}_{\mathrm{core}}})$ (§4.3). The editor agent is given write access to a fresh copy of the harness and a directory of per-task diagnoses sorted by severity. The instruction frames severity as a soft attention weight, requires cross-task pattern matching before any edit, and discourages task-specific hardcoded fixes. Each of the $N$ candidates is sampled independently with the same prompt, and randomness from the underlying agent supplies the diversity.

Each diagnoses/task_XXXX/ subdirectory holds the serialized diagnosis JSON rendered as Markdown alongside the original task prompt. Subdirectories are indexed in descending severity so the agent encounters the most consequential diagnoses first.

The ranking prompt implements $\mathrm{rank}(t,\tau_{a},\tau_{b})\in[-10,10]$ used in Best-of-$N$ acceptance (§4.3 and Algorithm 1). The evaluator sees the task, the two harnesses, and the two trajectories. The candidate trajectory is presented as trajectory_A and the baseline as trajectory_B, and the orchestrator negates the returned integer so the scalar score is oriented as baseline $\to$ candidate regardless of presentation order. Presenting the candidate first reduces a later-option preference bias we observed in pilot runs. The rubric below uses an integer scale in $[-10,10]$, and downstream only the sign and relative magnitude of the score are used (§3).

The candidate score $S_{j}$ in Algorithm 1 averages this integer (with the orientation flip described above) over the coreset $\mathcal{D}_{\mathrm{core}}$, and the candidate is accepted only when $S_{j}>0$, otherwise the harness remains at $h_{0}$.

A harness is a directory of files with no fixed schema, where prose instructions, executable scripts, and structured configuration sit side by side. At every operator call the harness is materialized as a subdirectory of the agent’s working directory, and the agent reads it the same way it reads task files rather than as a system-prompt prefix. The $\mathrm{solve}$ operator mounts the harness read-only by convention, while $\mathrm{optimize}$ mounts a fresh copy with write access and accepts whatever the agent leaves behind on the filesystem. Harnesses are compared by content, so if $\mathrm{optimize}$ returns a harness identical to the input, the candidate is treated as a no-op and dropped before evaluation. This keeps the harness surface tool-agnostic and lets the optimizer extend any of the three file kinds without a representation change.

The same backbone executes $\mathrm{solve}$, the diagnosis analysis ($\mathrm{rank}_{\mathrm{val}}$/$\mathrm{rank}_{\mathrm{con}}$), $\mathrm{optimize}$, and $\mathrm{rank}$. Role separation is achieved by workspace contents rather than by changing the backbone, where each operator runs in a fresh workspace that contains only the inputs it should see. $\mathrm{solve}$ sees the task files and the current harness. The diagnosis analysis sees the task and the group of trajectories for that task. $\mathrm{optimize}$ sees the harness directory, the diagnosis instructions, and the trajectory rollouts that motivated them. Finally, $\mathrm{rank}$ sees the task, two trajectories, and both candidate harness directories side by side. Per-role ablation therefore amounts to swapping prompts and inputs, not models, which keeps the operators directly comparable.

$\mathrm{rank}$ is invoked once per (task, candidate) pair. We present the candidate trajectory first and the baseline trajectory second, then negate the parsed scalar. This swap is a standard mitigation for position bias in pairwise judges, applied without per-call order randomization. The judge is constrained to return a single integer in $[-10,10]$ with a one-sentence rationale, and we do not retry. Any parse or execution failure deterministically yields zero. This makes $\mathrm{rank}$ a strict pessimist, since silent failures pull the mean toward the rejection threshold rather than away from it.

The diagnosis analysis consumes the full group of $G$ rollouts for one task and produces a single textual instruction $I$ with a severity weight, whereas $\mathrm{rank}$ is strictly pairwise. We do not collapse a group into a Bradley–Terry score for evaluation. For each candidate, we pair the candidate’s rollout of a task against a fixed baseline trajectory drawn from the original group, and we hold this baseline constant across every candidate so the comparison stays anchored to the same reference. The consequence is that $G>1$ enters the pipeline as a diagnostic device only, since it sharpens the instruction $I$ through cross-trajectory inconsistency but does not vote on which candidate wins.

$\mathrm{optimize}$ is a code-agent invocation, not a constrained text rewriter. It sees the materialized harness as a filesystem and may add, remove, or modify any file inside it. We hand it the diagnosis instructions sorted by descending severity, with severity exposed as a soft attention weight in $[0,1]$ rather than a hard priority queue, so the agent can still merge or skip suggestions that conflict. We do not parse a diff out of the agent’s final message, and the new harness state is recovered from the directory itself when the call returns. This keeps the action space identical to the representation, where anything that fits in a directory is a valid edit.

An update is accepted only when the best candidate’s mean pairwise score over the coreset is strictly positive, and a mean of zero is rejected. Before the gate, a candidate is dropped if the optimizer failed, the call timed out, or the produced harness is identical to the input. If every candidate is dropped, or every surviving candidate scores $S_{j}\leq 0$, RHO leaves the harness unchanged. This no-update behavior is part of the result rather than a tooling failure. In Figure 5, selectors whose coreset fails to expose useful weaknesses (pure coverage, pure difficulty) leave held-out accuracy near the Vanilla Codex baseline.

We persist the input harness id, along with the trajectories produced by each operator ($\mathrm{solve}$, the diagnosis analysis, $\mathrm{optimize}$, and $\mathrm{rank}$) identified by id. We further persist the diagnosis instruction, the candidate harness ids together with their diffs against the input, the per-candidate pairwise scores, the mean pairwise score, and the accept flag. Every stored trajectory carries its full event stream, final message, workspace diff, and wall-clock time. These records are what makes downstream audit, re-grading, and ablation possible without re-running the agent.

SWE-Bench Pro (Deng et al., 2025) is the long-horizon software-engineering benchmark in our suite. Tasks resolve only when a multi-file patch passes the upstream test set, and failure modes are therefore concrete and traceable to the harness’s repository conventions and build commands.

Source. We load the test split of ScaleAI/SWE-bench_Pro from Hugging Face. Evaluator scripts and per-instance Docker images come from the official scaleapi/SWE-bench_Pro-os repository, pinned to a fixed commit.$^{1}$

Split. Rows are ordered by the SHA-256 hash of (seed, instance_id) with the seed fixed. The first 100 form the training pool and the next 100 the held-out test pool, and the remaining rows are unused. We report on the held-out test pool.

Solve interface. Each task materializes a prompt file describing the issue plus a fresh clone of the upstream repository at its base commit, mounted in the workspace. The agent edits repository files in place, and no extra tools are injected.

Grading. We extract the agent’s patch by re-applying its workspace edits to a fresh checkout and taking git diff --binary. If the workspace path fails, we fall back to scanning the agent’s final message for fenced diff blocks. Binary hunks and auto-generated paths (dependency, cache, and build directories) are stripped before scoring. The patch is then applied inside the official per-instance Docker image distributed by the SWE-Bench Pro authors, and the official run and parser scripts are invoked. A task passes iff every FAIL_TO_PASS and PASS_TO_PASS test resolves correctly. The Docker wall-clock budget is one hour per task.

Terminal-Bench 2 (Merrill et al., 2026) contains executable command-line tasks. Failure is a missed reward, not a missed convention, so the harness mostly buys consistency in how the agent inspects state and chains commands.

Source. We use the upstream Terminal-Bench 2 repository at a fixed commit$^{2}$, yielding 89 tasks.

Split. Tasks are hash-ordered with the same seeded scheme as SWE-Bench Pro. The first 30 form the training pool, the remaining 59 the held-out pool.

Solve interface. Each task spins up a fresh Docker container whose image is declared by the task. Containers carry a cleanup label and per-task CPU and memory limits. The agent’s host workspace is bind-mounted inside the container. The solve prompt instructs the agent to author shell scripts on the host and execute them inside the container. A wall-clock watchdog terminates the container when the task’s declared agent timeout elapses.

Grading. We run the task’s upstream test suite inside the still-running container under its declared verifier timeout. The verifier writes a reward of 0 or 1 to a known path. A task passes iff the reward equals 1. We apply no difficulty filter, so held-out numbers span the upstream difficulty mix.

Data integrity. The solve prompt instructs the agent not to read the test directory, the upstream solution, or the verifier log. This is a prompt-level convention, not a sandbox restriction. We document it for transparency, since a sufficiently adversarial agent could read the verifier and game the reward. We rely on the held-out pool and Table 1’s cross-benchmark consistency to detect such behavior, and we have not observed it.

GAIA-2 (Froger et al., 2026) differs from the two coding benchmarks in that the environment evolves independently of the agent. The harness layer therefore has to encode behavior under partial observation and asynchronous events, not just stable build conventions.

Source. We load the validation split of the GAIA-2 release on Hugging Face under the mini configuration, yielding 200 scenarios. Each scenario ships an asynchronous event stream and an upstream write-action verifier.

Split. Scenarios are hash-ordered with the same seeded scheme. The first 100 form the training pool, and the remainder, 100, form the held-out pool. We report on the held-out slice.

Solve interface. Each scenario’s workspace contains a task prompt, a tool dispatcher, and a tool catalog. The agent invokes tools through a single shell command per call. The dispatcher relays each call to a sidecar process running the upstream environment, which advances simulated time and replays scheduled events. The scenario’s ground-truth state is held entirely by the sidecar, and the agent never reads it.

Grading. When the agent finishes, the sidecar invokes the scenario’s upstream verifier. The judge LLM is the same Codex gpt-5.5 backbone we use elsewhere, and routing defaults to the same Azure Foundry endpoint. A scenario passes iff the upstream verifier reports success.

Environment modifications. Two changes to the upstream environment affect reported numbers and must be disclosed. First, the upstream cap of one send_message_to_user call per turn is raised to four, since the original cap penalizes verbose agents that would otherwise complete the task correctly. Second, three optional judge-relaxation switches (event filtering, relaxed per-app UI judging, and trivial filesystem-read filtering) are available as ablation axes but are all disabled in the experiments reported in Table 1.

Dynamic Cheatsheet curates a running list of reusable facts and procedures harvested from past trajectories.

Persistence layer. A single markdown file lives inside the harness, structured as <memory_item> blocks containing a description, a worked example, and a usage count.

Offline phase. A curator agent iterates over the selected training tasks in order. For each task it reads the current cheatsheet plus the solve transcript and rewrites the cheatsheet in place.

Online consumption. The cheatsheet is part of the harness, and the solver reads it as part of normal harness lookup. No additional prompt prefix is injected.

Faithfulness. Upstream Dynamic Cheatsheet updates the cheatsheet after every task, so the next task’s solver sees the update. Our default configuration runs solves for all selected training tasks before invoking the curator, then commits one cheatsheet. The single-task configuration that matches the upstream per-example update is available as a sensitivity setting, while Table 1 uses the default.

ReasoningBank stores reusable reasoning patterns extracted from past trajectories and retrieves them on demand at inference time.

Persistence layer. A JSONL bank of items, each with title, description, body, and a precomputed embedding. The bank lives outside the harness directory because solve-time access is by similarity retrieval, not by harness materialization.

Offline phase. For each selected training task we solve it, ask the judge whether the trajectory was successful, then run an extraction prompt that distills reusable reasoning patterns from the trajectory. Extracted items are appended to the bank, and embeddings are computed locally.

Online consumption. At each held-out solve, we encode the task description with the same encoder, retrieve the top-$n$ items by cosine similarity, render them as a memory preamble prepended to the solve instructions. Top-$n$ is fixed at 1.

Faithfulness. We replace the upstream Gemini embedding with BAAI/bge-large-en-v1.5 (Xiao et al., 2023), a 1024-dimensional locally executed sentence encoder, so that every baseline that uses retrieval shares the same encoder, dimensionality, and storage backend. This removes a confound between memory-bank quality and embedding-provider quality.

Sleep-time Compute preprocesses past traces into compact notes that are prepended to the agent’s context at inference time, following the Letta implementation (Packer et al., 2023).

Persistence layer. A set of bounded markdown memory blocks stored inside the harness, edited with structured edit tools (insert, replace, rethink, finish) that mirror upstream Letta semantics.

Offline phase. A sleep-time agent iterates over the selected training tasks. For each task it reads the task’s prompt and trajectories, then issues a sequence of memory-edit tool calls to update the harness’s memory blocks. The system prompt is the upstream Letta system prompt, used verbatim.

Online consumption. The solver reads the memory blocks as part of the harness, the same way it reads any harness file.

Scope. Edits accumulate across tasks, and the sleep-time agent works against a single evolving memory state task after task. We do not reset memory between training tasks.

Meta-Harness is the validation-feedback reference, where a meta-agent rewrites the harness, and the rewrites are scored against a labeled validation set.

Persistence layer. A sequence of candidate harness directories paired with a search history recording per-candidate validation pass rates.

Offline phase. An outer loop alternates between a proposer that reads the search history and emits a new harness, and an evaluator that grades the proposed harness on the search-task set using the dataset’s ground-truth grader. Our re-implementation’s default budget is 20 outer iterations $\times$ 3 candidates per iteration $\times$ 2 solve trials per task, far above the budget we give RHO; upstream reports evaluating roughly 60 candidate harnesses over 20 iterations.

Validation-feedback footprint. The proposer reads each candidate’s per-task scores, mean score, and pass rate from the history, and the grader signal directly shapes the next proposal. This is the hallmark of the validation-feedback family and the axis along which Meta-Harness is incomparable to the trajectory-only baselines.

Matched-budget configuration. For Table 2 we run the outer loop for one iteration with 3 candidates and 1 solve trial per task, so the candidate count matches RHO’s $N=3$. The validation-grade cost is what Table 2 reports.

Shared configuration. All four baselines and RHO start from the same empty harness for a given dataset, use the same coreset of selected training tasks, and are graded against the same held-out split with the same grader. The only deliberate axis of variation is what the offline phase persists and how the solver consumes it. Meta-Harness additionally consumes validation grades, and this is the axis Table 2 measures.

Every method dispatches two kinds of model calls during the offline phase, plus the same per-task solve at evaluation time. We count them separately because they go through different backends and have different unit costs.

Codex (agent) invocations. One Codex CLI subprocess that runs to completion is one agent invocation. Each invocation persists a trajectory directory recording the role (one of solve, diagnose, optimize, rank), the harness id, the task id, the wall-clock duration, and the exit status. We obtain the per-method totals by counting these directories.

Auxiliary LLM calls. A single direct API call to the same gpt-5.5 backbone, issued outside the Codex CLI. Only ReasoningBank uses these in the offline phase, namely two per training task, for success judgment and memory-item extraction. Coreset selection also issues one auxiliary LLM call per candidate task before any method starts. This cost is shared across methods at matched coreset and is reported once at the end of this appendix.

Embedding calls. ReasoningBank and DPP coreset selection both call BAAI/bge-large-en-v1.5 (Xiao et al., 2023) for fingerprint or query embedding. We execute the encoder locally, so these calls do not hit a remote endpoint and are not aggregated with the LLM totals.

Table 7 breaks down the optimization phase on SWE-Bench Pro ($k=10$ train tasks, $M_{\mathrm{test}}=100$ held-out tasks) into agent invocations by role. Terminal-Bench 2 differs only in $M_{\mathrm{test}}=59$, while GAIA-2 matches SWE-Bench Pro.

The table is a literal accounting of how each method spends its offline budget. ReasoningBank issues one solve per training task and never proposes a candidate harness, so the offline phase costs only $k=10$ extra Codex invocations on top of the held-out evaluation, and its $2k$ auxiliary LLM calls are the success judge and the memory-item extractor. Dynamic Cheatsheet and Sleep-time Compute both roll out each training task $G=3$ times to surface variance, then run an optimizer agent. Dynamic Cheatsheet invokes a single curator over all $k$ tasks ($N{=}1$ candidate), while Sleep-time Compute runs one edit pass per task per restart (3 restarts $\times$ $k$ tasks $=30$ optimize calls). Both then commit each unique candidate harness through the after-solve and pairwise-rank steps shared with RHO. RHO’s footprint differs from Sleep-time Compute’s only by trading $30-3=27$ per-task optimize calls for $10$ diagnose calls plus the $3$ candidate-level optimize calls in the best-of-$N$ proposer, and the after-solve and rank phases are identical at $N=3$ candidates.

Table 8 reports two complementary wall-clock measurements per (method, dataset) cell. $\Sigma_{\mathrm{codex}}$ is the sum of per-invocation wall-clock over every agent invocation in the run, and it is the cost of running the same workload serially on one agent process. End-to-end is the elapsed time from the run’s start to its completion, under our infrastructure setting of 10 concurrent agent calls and 10 concurrent graders. The ratio between the two columns reflects how much of the workload was successfully parallelized, where the trajectory-only methods reach $5$–$10\times$ speedup, while ReasoningBank reaches $3.5$–$4.5\times$ because its offline phase is sequential by construction (each training task’s solve must commit to memory before the next task’s retrieval). Meta-Harness is a validation-feedback rather than a trajectory-only method, so we report its budget separately in §G.4, not here.

Two observations are worth stating explicitly. First, per-invocation cost varies by more than $2\times$ across datasets (GAIA-2 averages around $150$ s per invocation, Terminal-Bench 2 around $300$ s, SWE-Bench Pro around $400$ s), so an agent-call count is informative only within a dataset, and cost cannot be compared across datasets by summing invocation counts alone. Second, at matched coreset and $N=3$, RHO is within $\pm 4\%$ of Sleep-time Compute on every dataset, more expensive than Dynamic Cheatsheet (which uses $N=1$), and substantially more expensive than ReasoningBank (which performs no group rollout, no candidate proposal, no pairwise rank).

Meta-Harness has a qualitatively larger compute footprint because validation grades feed back into the proposer. At its upstream default ($I=20$ outer iterations, $C=3$ candidates per iteration, $T=2$ solve trials per task) the offline phase alone would cost $I(1+CM_{\mathrm{search}}T)+M_{\mathrm{search}}T\approx 1{,}210$ agent invocations on SWE-Bench Pro — nearly twelve times RHO’s optimization-phase count of $103$. The matched-budget configuration we report in Table 2 reduces this to $I=1$, $C=3$, $T=1$, amounting to $M_{\mathrm{search}}T+I(1+CM_{\mathrm{search}}T)=10+31=41$ optimization-phase invocations; this is the configuration whose held-out pass rate ($0.62$ on SWE-Bench Pro) is reported alongside RHO’s in the main text. The 10-round configuration reported in the same table keeps $C=3$, $T=1$ but raises $I$ to $10$, giving $10+10\cdot 31=320$ optimization-phase invocations ($3.1\times$ RHO’s optimization-phase count of $103$). These counts exclude the shared $M_{\mathrm{test}}=100$ held-out evaluation, which every method pays once regardless of optimizer. Because Meta-Harness is a validation-feedback rather than a trajectory-only method, it is not included in the wall-clock comparison of Table 8.

Coreset selection runs once before any method’s offline phase begins. For SWE-Bench Pro this consumes $100$ auxiliary LLM calls (one difficulty judgment per candidate task) plus one batched embedding call over the same $100$ fingerprints, after which the selected $k=10$ task ids are persisted and reused verbatim by every baseline. This cost is paid once per dataset and is not attributed to any individual method.