EnvHarness: Awakening Static Worlds for Agent Learning

An agent harness (Anthropic, 2025; Anthropic, 2026b; OpenAI, 2026) is the software layer (execution loops, tool registries, and context management) (Meng et al., 2026) that wraps a LLM to form an autonomous agent ($\text{Agent}=\text{Model}+\text{Harness}$). It adds new capabilities without changing the model weights. We apply the same idea to the other side of the agent-environment loop. We define EnvHarness as a programmable layer that wraps an existing, static environment and turns it into a customizable one ($\text{Customized Env}=\text{Static Env}+\text{{EnvHarness} }$). EnvHarness achieves this customization by modifying the information flow through the standard interface, leaving the underlying environment completely untouched. Analogous to the tools and memory of an agent harness, EnvHarness is assembled from modular plug-in components that tailor the environment to specific training needs like isolating a specific skill, extending a task’s horizon, or adjusting task difficulty.

The mapping $w$ of Eq. (1) defines a general interface, and any transformation that follows this interface is a valid EnvHarness component. In this work, we introduce three concrete types of components, chosen to cover three fundamental modes of environment customization, and we expect more to follow. Each type is specified by its own parameters, overriding standard environment interface methods such as reset or step. Figure 3 compares the three component types with the original interface, highlighting the specific method each component overrides. We detail the software implementation of this interface protocol and class architecture in Appendix C. Throughout this subsection, we illustrate the components on one ALFWorld task, “put a clean mug on the desk”, where the default instance leaves the mug in the open and ends immediately upon placement.

Given a base environment $E$ supporting a set of base tasks, and a target policy agent $\pi$, our objective is to automatically generate a modified environment $E^{\prime}$ tailored to a specific task $t$, exposing the unique flaws of $\pi$ to facilitate targeted policy improvement. A single EnvHarness component is itself policy-agnostic, as Eq. (1) defines $w$ as a transformation of the environment alone, allowing the same component to be applied without modification to any policy. Conversely, the selection and parameterization of these components must be conditioned on both the base task $t$ and the observed behavior of the policy $\pi$. We therefore introduce the task-policy-conditioned map $\mathcal{H}$:

where each $w_{i}$ is a customized EnvHarness component designed to wrap the base environment $E$ and expose the critical weaknesses of $\pi$ on task $t$. Rather than inspecting internal model weights, these components treat the agent as a black box and operate solely on its outputs to generate a steady, corrective training signal.

We introduce EnvRigger to realize the task-policy-conditioned map $\mathcal{H}$ of Eq. (6). EnvRigger runs $\pi$ in the environment on task $t$, analyzes the resulting trajectories, writes specific EnvHarness components to customize the environment for this task, and validates the customized environment using fresh policy rollouts. Candidates from customized environments that provide an appropriate learning signal are accepted, while the EnvHarness components of unsuccessful candidates are rejected or revised. Figure 4 illustrates this complete workflow, in which EnvRigger operates systematically through four distinct stages: Observe, Diagnose, Write, and Validate. To ensure that the initial state mutations introduced by Stage can be reliably reproduced, we assume the base environment supports deterministic resets during the validation phase.

In this section, we mainly focus on the skill-based learning paradigm, where skills are extracted from environments to improve agent capabilities. We also evaluate the compatibility and performance of our framework under the online reinforcement learning paradigm, which is presented as part of our broader analysis in Section 5.

Table 2 and Table 3 summarize the primary results across five benchmarks. Based on these evaluations, we present the following key observations.

Beyond skill-based learning, we investigate whether EnvHarness-customized environments can provide active training signals in online reinforcement learning. We perform this analysis on ALFWorld and WebShop (Yao et al., 2022a) using Qwen3-8B-base (Yang et al., 2025) as the policy, optimized via Group Relative Policy Optimization (GRPO) (Shao et al., 2024); training details are provided in Appendix F.1. We train two distinct policies: one trained solely on the original static environments, and one trained entirely on environments reshaped by EnvHarness, evaluating both on the same held-out instances. Table 4 reports the results. Training on EnvHarness environments consistently improves policy performance, outperforming the baseline trained solely on original environments on three out of four metrics. Specifically, on ALFWorld, EnvHarness Envs achieves an in-distribution success rate of 87.9 compared to 81.4 for original environments. On WebShop, it achieves a higher score of 79.2 (versus 75.6) and success rate of 67.4 (versus 66.0). Although there is a slight, negligible trade-off in the ALFWorld OOD success rate (88.8 versus 89.6), the overall results underscore a fundamental advantage: the reshaped environments are not merely auxiliary data but provide a highly effective, independent optimization signal for online policy learning.

Real-world applications often require agents to operate over extended horizons. The Chain component addresses this by joining two randomly paired base environments into a single, extended episode. To isolate its effect, this pairing operates independently of the autonomous EnvRigger loop. We evaluate the extracted skills on standard single-environment test instances, reporting Success Rate (SR) and Average Steps (AS) in Table 5.

Skills from Chain environments yield substantial efficiency gains, reducing AS from 53.58 to 41.96. Their standalone SR (49.63) is marginally below the 49.88 baseline. This aligns with their stringent training condition—where success requires solving both halves—which prioritizes long-term goal preservation over short-task maximization. Combining both skill sets (Stage/Contract + Chain) achieves the best of both worlds: the highest SR (54.30) and excellent efficiency (43.12 AS), demonstrating highly complementary behaviors. Representative skills are in Appendix F.3.

Environment scaling evaluates performance as the available training environments expand, comparing three allocation strategies under an identical budget: EnvHarness environments, unmodified benchmark environments, and SWE-smith generated environments. Holding the policy model, environment budget, and skill retrieval protocol fixed, each batch of 50 environments yields one skill bank (alternating between 2 and 3 skills per bank, totaling 15 skills at 300 environments). Crucially, while both baselines draw environment batches independently of the learner, EnvHarness synthesizes each batch specifically targeting the policy equipped with previously accumulated skills, enabling the environments and the policy to co-evolve. As shown in Figure 5, EnvHarness climbs from 47.67 to 54.79 (a 7.12-point gain) and maintains an upward trajectory at 300 environments. In contrast, the same budget yields only 52.13 on original environments and 50.37 on generated ones. This performance gap confirms that targeting the learner’s current capability boundary is fundamentally more effective than unconditioned environment scaling. Representative skills from each round are provided in Appendix F.2.

To evaluate generalizability, we test four distinct models on SWE-bench Verified: Gemini 3.1 Flash-Lite, Qwen3.6 27B (Qwen Team, 2026), Gemini 3.5 Flash, and Claude Sonnet 4.6 (Anthropic, 2026a). These span open-weight and proprietary architectures across a wide capability spectrum. In each setting, we use the same model backbone for both the target policy and the EnvRigger to keep the setup consistent, while the extraction pipeline and protocol remain unchanged.

Figure 6 shows the results. EnvHarness skills outperform real environment skills on all four policies, by 2.7 to 3.7 absolute points, even though the skill-free success rates span a broad range from 30.7 to 67.2. The size of this gain is largely independent of how strong the underlying policy is: the customization loop neither breaks down on the weakest model nor saturates on the strongest, and the same pipeline, prompts, and acceptance criteria are used throughout. What the policy’s capability level appears to change is the content of the diagnoses rather than the applicability of the loop. We note that skills of either kind help the two weakest policies most relative to no skills at all (+9.3 and +11.1 points for EnvHarness, and +6.1 and +7.4 for unmodified environments, versus under 5.5 points for the two strongest).

While the EnvRigger loop autonomously identifies training targets via behavioral diagnosis in standard settings, the same machinery can readily accept explicit, user-defined constraints. We evaluate two classes of constraints. The first is a quantitative target (success rate or average steps) for an objective performance metric in Appendix G. The second constraint targets capability weaknesses described in natural language. For example, given the weakness below, the EnvRigger generates a Contract that rejects code submissions unless tests are run. This forces the agent to verify its fixes. From the resulting trajectories, we distill the skill shown below. Rather than overfitting to a single task, the skill combines a general principle with actionable steps, perfectly matching our requirement for a skill (Yang et al., 2026b).

Environment scaling has emerged as a research direction that supplies agents with more environments to learn from (Huang et al., 2025b; Xi et al., 2025). Existing efforts scale environments in various forms, including simulating environments and their feedback with an LLM (Guo et al., 2025; Zala et al., 2024; Wang et al., 2025), up to world models that simulate or synthesize whole families of agentic environments (Zuo et al., 2026; Wang et al., 2026), synthesizing executable environments programmatically (Chae et al., 2026; Song et al., 2026; Tang et al., 2026; Sun et al., 2026; Dong et al., 2026), and synthesizing new task instances inside an existing benchmark (Yang et al., 2026a; Pan et al., 2024). Beyond producing more tasks, another line adapts what the environment presents to the learner, from curriculum generation in reinforcement learning (Jiang et al., 2021; Dennis et al., 2020; Wang et al., 2019; Liu et al., 2026) to hand-designed corrective feedback and reward shaping (Lu et al., 2025). Different from previous works that rely on benchmark-specific pipelines or hand-designed curricula, EnvHarness reshapes an existing environment through one interface shared across benchmarks, conditions the reshaping on the diagnosed weaknesses of the current policy, and leaves tasks and verifiers untouched.

Self-evolving agents improve themselves from their own experience without additional human supervision (Fang et al., 2025). Existing efforts evolve different parts of the agent, including prompts and reflections (Shinn et al., 2023; Madaan et al., 2023), skill and workflow libraries (Wang et al., 2023; Wang et al., 2024; Xia et al., 2026a; Yang et al., 2026b; Xia et al., 2026b; Huang et al., 2026b), experience memory distilled from past trajectories (Zhao et al., 2024; Ouyang et al., 2025), the model weights through self-generated rewards or self-proposed tasks (Yuan et al., 2024; Huang et al., 2025a; Zhao et al., 2026; Xia et al., 2025; He et al., 2025; Huang et al., 2026a), and recently the agent harness itself, rewritten and tested around a frozen model (Lee et al., 2026b). Different from these methods that evolve the agent while the world it learns from stays fixed, EnvHarness reshapes the environment itself against the diagnosed weaknesses of a frozen policy.

The released code predates the terminology of this paper. The three component types appear there as the classes Setups, Rules, and Link, and a candidate emitted by the designer carries the fields rules_code and in_env_actions. Table 6 maps the two, and the rest of this appendix uses the names of the paper.

The prompt below is the part shared by every benchmark. Each benchmark appends a short block of its own detailing the tools its bridge exposes, the fields of its env_state view, and its domain-specific constraints; those blocks are in the code release.

At the base of the framework is ActionableEnv, an abstract class that fixes the contract every environment must satisfy. It consists of two groups of methods. The first group is a Gymnasium-style interaction loop with typed, validated data contracts. reset(seed, options) initializes an episode. step(action) consumes an Action (a tool name plus JSON-serializable keyword arguments) and returns an EnvResponse, a Pydantic wrapper around the Gymnasium 5-tuple (observation, reward, terminated, truncated, info). evaluate() returns a terminal EvaluationResult, and observe() returns a fresh Observation of the current state. observe() is deliberately separated from reset(): a component may mutate the environment after reset returns but before the policy acts, and observe() lets the outer layer re-read the world without paying for another reset. Finally, get_env_state() exposes a runtime-safe view of the internal state. This view consists of plain data with no Docker handles, browser pages, or sockets, and it is the only state that component hooks are permitted to read. This restriction is what makes component code portable: the same hook that runs against an in-memory puzzle also runs against a containerized repository, because neither ever touches the runtime beneath the state view.

The second group makes persistence environment-owned. save_state() returns a JSON-serializable dictionary, and the classmethod from_state(dict) reconstructs an instance from it. The contract intentionally does not prescribe what to save. Pure in-memory environments serialize their full live state, while environments whose runtime cannot be cheaply cloned (containers, browsers, game engines) save only their reset arguments and accept that restoration is valid at episode boundaries. Each concrete class registers a stable string tag through a registry decorator, so saved stacks can be reconstructed without embedding import paths in checkpoint files.

The interface also declares optional capabilities with safe defaults: a per-step dense reward hook step_reward(step_info) (non-fatal by contract, where exceptions are recorded but never episode-terminating), a notify_replay_complete() callback that lets an environment rewind per-episode bookkeeping after a state-preparation replay, task enumeration via list_tasks(), and close() for releasing external runtime resources.

A Bridge is a benchmark’s direct implementation of ActionableEnv, and it is the only layer in the system aware of the underlying runtime. We implement seven Bridges including four distinct runtime classes: a pure in-memory arithmetic game (Toy24), a text-adventure engine (ALFWorld via TextWorld), per-instance Docker containers (SWE-bench, OfficeQA and spreadsheetBench, where each step is a stateless docker exec against the task repository), and a Playwright-driven browser (WebArena via BrowserGym and Webshop). Everything above the Bridge, including the policy loop, the orchestrator, and all component code, is shared verbatim across the seven environments.

Each Bridge declares its action space as a tool_registry of typed tools whose signatures are introspected into function-calling schemas for the policy prompt. The registry serves two roles that the design decouples: schema generation is universal, while dispatch through it is optional. Toy24 routes step() through the registry because its state is the runtime. In contrast, ALFWorld, SWE-bench, and WebArena bypass it and drive their engine handle directly, since a TextWorld engine or a browser session cannot be threaded through the data-only state view. Bridges likewise choose their own persistence granularity along the two patterns above (full snapshot for Toy24, and reset-arguments-only for the three heavy runtimes) and publish a human-readable env_state_schema() describing the fields that component hooks may read. This schema is injected into the designer agent’s prompt, closing the loop between what a Bridge exposes and what generated code can rely on.

Each environment carries one EnvHarness, an ordered stack of components over its Bridge (Figure 7, instance view). In code the stack is realized by the decorator pattern. EnvHarness is the abstract base every component derives from, and an EnvHarness is an ActionableEnv that wraps another ActionableEnv. Its default implementation delegates every interface method to the inner environment, so a concrete component overrides only the methods on the axes it affects. Because the wrapped object may itself carry components, they stack arbitrarily, meaning $\texttt{Rules}(\texttt{Setups}(\texttt{Toy24Bridge}))$ is again an ActionableEnv, and the policy interacting with the outermost layer cannot observe how many components sit beneath it. Persistence is layered accordingly: each component serializes only its own state, and a checkpoint records the environment plus an ordered list of components (innermost first), which the loader rebuilds inward-out by passing each reconstructed component as the inner field of the next. The three component types of Section 2.2 ship as the classes Setups, Rules, and Link (Figure 7, bottom right).

By default, the Link operator concatenates environments in sequence. It automatically switches to the destination environment as soon as the first environment terminates, requiring no manual override of the transition logic.

The composition logic can evaluate the success or failure of the first task upon termination, and dynamically route the agent to a harder task (such as AdvancedEnv) or an easier one (such as RemedialEnv).

The transition point is fully controlled by the harness. This allows switching the agent to a different environment immediately when a specific condition is met, without waiting for the current task to officially end.

Because the transition check runs after every single interaction step, the agent can alternate back and forth between two environments continuously during execution.

Table 7 lists the training and evaluation splits. Training tasks are the corpus the designer agent reshapes; evaluation uses only original, unreshaped tasks.

For SpreadsheetBench, Pass@1 aggregates over base tasks and Mean Score averages over all instances. On ALFWorld, In-Dist and OOD refer to the benchmark’s own seen and unseen evaluation splits, which differ in whether the task’s object–receptacle configuration appeared during training; we do not construct these splits ourselves. Skill extraction uses the same model as the designer and the policy.

GenEnv (Guo et al., 2025) generates new tasks with an environment-simulator model that keeps task difficulty at the edge of the agent’s current ability. VeriEnv (Chae et al., 2026) clones websites into executable synthetic environments whose rewards are checked programmatically. SWE-smith (Yang et al., 2026a) synthesizes new repository-level task instances. We run each with the same seed tasks and the same model as EnvHarness, and each produces the same number of environments as EnvHarness does, so differences come from the generation strategy rather than the data, the model, or the amount of generation. Generation pipelines are benchmark-specific because they must reach into an environment’s internals and construct verifiers, which is why each baseline covers a single benchmark and none exists for the office domain. For every baseline, skill extraction, retrieval, and the policy model are identical to ours, and only the environment the skills come from differs.

Table 8 collects the settings of the EnvRigger. The same values are used on every benchmark.

We evaluated the effectiveness of EnvHarness environments in a Reinforcement Learning (RL) setting. We utilized Group Relative Policy Optimization (GRPO) to train a Qwen3-8B-base model on the ALFWorld and Webshop benchmark. The specific hardware configurations and training hyperparameters are detailed below:

Each round of the loop diagnoses flaws the previous policy did not yet show, so the skills extracted per round shift in focus, from basic test invocation and file editing, to keeping the test loop alive when the runner itself breaks, to interpreter resolution and pre-edit navigation. The gains shrink round over round (Figure 5) as the remaining flaws grow more local. Every skill below traces to a specific accepted component. The banks also contain skills distilled from rollouts that no component targeted, and we show the component-driven ones here so the causal link between a written component and the induced skill is inspectable. Each component’s code follows its skill in an amber box, together with the axis of Eq. (3) it exercises. Import lines are omitted from the listings.

Skills extracted from Chain environments concern behaviors that only appear when tasks are joined, such as managing a shared step budget and reorienting after a mid-episode task switch. We list representative examples below.

Table 9 lists the success rate and the average episode length for all four policy models under the protocol of Section 5. The success rates move in one direction, while the average steps reveal three different regimes. Qwen3.6 27B runs 69.8 steps bare, the longest of any model, and skills nearly halve this to 37.1, so the bare policy spends most of its budget on undirected trial and error that the skills replace with known procedures. Gemini 3.1 Flash-Lite shows the reverse. The bare policy gives up early at 36.7 steps, and skills make it persist at around 50 steps while solving far more tasks, so here the extra length is the point. Claude Sonnet 4.6 barely moves, from 29.3 to about 25 steps, since a policy that is already directed has little dead time for skills to reclaim. Gemini 3.5 Flash sits between these regimes and is the one model where EnvHarness skills improve both metrics at once against both baselines, more resolved issues in visibly shorter episodes.

Two further points follow from the same numbers. First, the gains of EnvHarness over the original-environment skills do not come from simply running longer. The episode lengths of the two skill sources are within a step of each other on Flash-Lite and Sonnet, EnvHarness is over five steps shorter on Flash, and only on Qwen does it spend more, 3.7 extra steps for 3.7 extra points. Across models the additional successes arrive within essentially the same budget. Second, average steps alone is not a quality signal. A short episode can mean an efficient solution, as for Sonnet, or an early surrender, as for bare Flash-Lite, and the two cases sit at nearly the same step count. Reading the metric therefore requires the success rate next to it, which is why we report both here rather than in the main text. In short, skills shorten episodes where the policy flails, lengthen them where it quits, and leave them alone where the policy already knows where it is going.

To test whether the skills transfer beyond the task types they were learned on, we run a leave-one-out evaluation on ALFWorld. Skills are extracted from environments covering every task type except one, and the policy is evaluated on the held-out type alone, so any gain must come from behaviors that carry across types rather than from familiarity with the held-out tasks. As Table 10 shows, skills from EnvHarness environments outperform skills from the original environments on four of the six types and by 3.1 points on average, with the largest gain of 16.4 points on clean and one regression of 8.7 points on heat. Reshaped environments push the policy off its memorized routines during extraction, so the resulting skills encode behaviors that apply across task types instead of recipes tied to a single one.

Table 11 decomposes token consumption into Design Tokens and Rollout Tokens. EnvHarness spends far more on design than the single-pass baselines (1.46M vs. 38K on ALFWorld), a deliberate cost of feeding full trajectories into its prompts to diagnose weaknesses rather than generating tasks blind. Design is nonetheless a small share of either budget, and rollouts dominate.

Against VeriEnv, which also executes in the real environment, the totals are essentially the same (137.3M vs. 137.8M), so the gains of Section 4 do not come from outspending the baseline. GenEnv’s total is 3.5$\times$ lower, but its rollouts are simulated rather than executed, and that saving buys the hallucinated transitions and drifting success signals discussed in Appendix. The extra cost is thus the cost of grounding: at equal grounding EnvHarness matches the baseline’s footprint, and where it spends more, it spends on real execution against a trusted verifier.

Beyond skill quality, we ask whether EnvHarness can steer an environment so that an objective, quantitative metric lands in a prescribed range. We run the loop on 100 ALFWorld tasks under two such metrics, measuring each task with $K=10$ rollouts: per-task success rate (SR), targeted to $[0.4,0.6]$, and the average number of steps on successful episodes (AS), targeted to $[25,35]$. In both cases EnvHarness reshapes a task in whichever direction its baseline value requires, tightening it when the value sits above the band and scaffolding it when the value sits below. Table 12 reports the fraction of tasks landing inside the band. SR is the more tractable target: the original tasks are strongly bimodal, with most either always solved or never solved, and EnvHarness compresses them into the middle of the range, raising in-band coverage from 6.0% to 80.0% and moving mean SR from 0.74 to 0.48. AS is a tighter constraint, since it fixes an exact step count rather than a rate, yet coverage still rises from 18.0% to 53.0%. A single interface therefore suffices to calibrate an environment against an explicit, measurable objective, without any access to its internals.

For each case we hand the designer one sentence naming a capability weakness. The designer writes components that make the weakness fatal inside an ordinary benchmark task, runs the policy in the reshaped environment, and one skill is distilled from the resulting trajectories. Table 13 gives an overview, and the rest of this appendix lists the remaining cases in full, the generated component next to the skill it produced. The SWE-bench verification case appears in the main text. The designer chose its own component axes without instruction, staging start states with a Stage, blocking shortcuts with the action filter of a Contract, and faking consequences with the transition hook of a Contract. Every component stays inside the source distribution and leaves goals and scorers untouched. Code is abridged for space. The policy and the designer are Gemini 3.1 Flash-Lite on ALFWorld and WebArena and Gemini 3.5 Flash on SWE-bench Verified.

EnvHarness builds each environment through an iterative loop in which a designer agent proposes, executes, and revises a candidate harness. A weaker designer needs more iterations to reach a harness that passes validation, and each iteration requires rolling out the environment, so producing a pool of high-quality environments can consume substantial time and inference compute. This cost is paid once per environment rather than per training episode, and we expect it to shrink as designer agents improve.

EnvHarness assumes a reset/step interface over textual actions and observations. The binding constraint is reset: a Stage must place the environment into a chosen initial state and a Chain must return it to a known state between subtasks, both of which presuppose that the environment can be restored rather than only advanced. This excludes environments backed by a live service or any other non-resettable backend, such as an agent acting on a real user account where a sent email or a placed order cannot be undone, or a physical robot whose surroundings do not return to their initial configuration between episodes.

A Chain composes subtasks by concatenation and verifies the result through the verifiers of its parts. This is what allows every reshaped task to inherit trusted, human-built verification, but it leaves a Chain with no notion of whether the composed subtasks are semantically related, and no way to express workflows with branching or shared intermediate state. Semantic composition would require both a measure of compatibility between subtasks and a verifier defined over the composed objective.

Stage, Contract, and Chain are a first set of components, not a closed one. The agent harness has grown well beyond its initial pieces, and we expect the same for environments: components that inject stochasticity or partial observability, that expose auxiliary feedback channels, or that place several agents in a shared environment would each extend what a frozen benchmark can be reshaped into, while keeping the same reset/step interface.

EnvHarness currently operates over textual actions and observations. Extending it to visual, GUI-driven, or embodied environments would test whether the wrapping abstraction survives when observations are no longer symbolic, and would require components that can specify and verify states that are not expressible as text.

A Chain composes subtasks by concatenation and verifies the result through the verifiers of its parts. This is what allows every reshaped task to inherit trusted, human-built verification, but it also bounds what a Chain can be. The control-flow mechanism can route between sub-environments in richer ways, yet only serial concatenation admits a composite verifier: each leg terminates on its own and contributes a verdict, so the composite is their conjunction. Under branching or interleaving there is no such pair of verdicts to combine, and a Chain has no notion of whether its subtasks are semantically related in the first place. Semantic composition would therefore require both a measure of compatibility between subtasks and a verifier defined over the composed objective, not merely richer control flow.