Where is Physical RSI capability from? An Observational Scaling Law of LLM-as-policy

What allows an agent to turn physical feedback into a better policy? Perception helps it interpret outcomes, spatial reasoning relates observations to actions [31], and coding and harness use allow it to implement and test revisions. These capabilities may play different roles. We ask whether strengths behind a good first attempt also explain improvement after feedback.

We introduce PhysRSI-eval to study these questions under a common protocol (Figure 2). Six systems revise persistent code, tools and notes across 18 manipulation tasks, with frozen model weights and a fixed physical interaction budget. Gemini 4 is reported separately on ten tasks.

An exploratory comparison of four public capability profiles on the 18 tasks associates spatial reasoning most strongly with initial performance, and perception with attained performance and recovery (Figure 4). This pattern motivates the Physical Barrel Hypothesis: the capability that limits an agent may depend on which aspect of improvement we measure. Starting well and learning from failure may require different strengths. These associations suggest candidate bottlenecks, not causal requirements.

An endpoint score alone cannot quantify physical RSI. It says little about when gains occur or what they cost. Repeated attempts can also increase the chance of success without useful revision. Our five-axis metric separates initial performance, gain timing, attained score, physical budget savings and recovery from failure (Section 3.4). These axes reveal different strengths. Opus 5.5 attains a higher score than GPT-6.1 Sol, but has lower gain-timing and budget-savings scores. Higher scores need not mean improving earlier or using fewer physical interactions.

With these outcomes defined, we study observational scaling across model capabilities and physical interaction budgets. Following Ruan et al. 34, we first ask whether public capability scores can predict the relative order of held-out systems, in attained performance and gain timing (Section 5.2).

We then ask whether early physical progress predicts the gains from a larger interaction budget. This connects physical RSI to the study of learning curves [19, 12], with the interaction round as our budget variable. Calibrating the improvement shape of a reference model to a few early rounds forecasts the rest of a related model’s trajectory (Figure 8). The two analyses address complementary questions: what improvement to expect from a model’s capabilities, and how much further it may progress after its first attempts. Both remain exploratory.

Does the agent acquire a reusable procedure, or adapt to its practice scene? A rising score alone cannot answer this question. We first remove history to test whether retained experience helps beyond repeated attempts. We then freeze workspaces on unseen layouts and transfer them across tasks.

In a six-task Astra ablation, removing history lowers late-round success from 41.7% to 8.3% (Table 8). With further revision disabled, frozen workspace files support success in 25 of 60 trials on unseen layouts (Table 9). Three cross-task cases also reach first success earlier when initialized from other tasks’ workspaces (Figure 9). These results support a role for retained experience and show that some artifacts remain useful beyond the scenes where they were refined. Scene-specific gains and useful transfer can coexist. Explaining which retained skills or memories enable transfer connects this question back to the origins of physical RSI.

Our contributions follow this progression:

LLMs can act as robot policies by writing programs over perception and control tools [22]. Feedback lets them revise plans or reward code [39, 14, 26]. Recent benchmarks evaluate these policies through controlled robot interfaces [7, 24, 5, 57, 52], while others give coding agents extended simulator access and assess the resulting programs [35, 25]. These evaluations typically summarize outcomes per episode, across independent trials or within a time budget. A reported Astra case, where a self-written fix succeeded once and failed on repetition, highlights the gap between finding a solution and retaining reliable behavior [9]. PhysRSI-eval tracks judged physical attempts connected by a persistent workspace, measuring both progress and physical cost under a fixed budget.

Agents can accumulate skills and reflections without updating their model weights [43, 37, 27]. Recent benchmarks examine whether changes to the harness or retained experience improve later performance [13, 46, 38, 58]. Producing an update and benefiting from it are distinct questions [23]. Related designs test memory, transfer and gains relative to cost [36, 49, 10, 17, 20, 47]. AI-R&D benchmarks address research engineering under time or compute budgets [45, 1, 40, 32, 4]. These designs motivate separating initial ability, improvement and cost, and testing retained experience against repeated attempts. PhysRSI-eval brings these questions to physical interaction. Native task scores evaluate each attempt, while workspace interventions test what the agent retains. Clearing the workspace tests whether accumulated experience helps; freezing and transferring it test whether its benefits extend to new layouts and tasks.

Public benchmarks provide candidate measures of perception [8, 54, 33], spatial reasoning [2, 50, 6, 16], coding and tool use [18, 44, 15, 28, 48, 53], and broader reasoning and work performance [30, 42, 29]. Embodied benchmarks and VLA models address planning and physical reasoning [51, 21, 11, 56, 41], but do not track policy revision from graded attempts. Observational scaling relates benchmark-derived capability factors to agent performance across models [34]. We predict the relative order of physical improvement, in both extent and timing, from public scores. We also forecast later gains from early physical performance. With few closed systems and no training-compute data, these analyses remain observational and exploratory.

A strong first attempt and useful revision are different abilities. We therefore score a first-attempt baseline separately and follow subsequent attempts under a fixed budget. Per-round scores show when gains occur; charged physical steps show what they cost. Reaching a high score, reaching it early and using fewer interactions are separate outcomes.

Success alone does not show whether the agent has accumulated useful experience. Code, tools and notes must remain accessible for intervention. Clearing the workspace tests whether retained experience helps; freezing it on unseen layouts tests whether it remains useful; transferring it to new tasks tests whether it can support a different goal. We report these tests separately from the main profiles in Section 6.

Model weights stay fixed, while skills, tools, policy code and memory can change in a persistent workspace. Each revision is tested through a judged physical attempt (Figure 2). This connects workspace changes to an observed outcome and a physical cost.

K0 measures the starting point without a persistent workspace. Each task runs three times through the shared robot tools, and its scores are averaged. At K1, the agent receives a Harness workspace that is empty except for the host’s read-only tools. Each round executes one physical attempt and returns the native score, success flag and failure reason. The agent then revises its workspace with robot actions disabled and commits it for the next round. The main evaluation allows at most $H=17$ refinement rounds per task and stops at the first native success.

Both stages share the robot interface. K1 also introduces workspace tools and program execution, so the K0-to-K1 gain mixes interface changes with improvement. We therefore study the Harness rounds as the self-improvement segment. Each round allows 4200 s of reasoning; revision takes up to 1800 s for GPT systems and 600 s for Claude systems (Table 3).

For aggregation, successful tasks retain score 1 after stopping, without further execution. Unsuccessful tasks retain their current scores, including regressions. Extra attempts enter the physical budget but add no scoring rounds. Nine GPT-5.6 Sol Harness entries summarize two observed runs. The three-repeat K0 protocol applies to all six systems.

The task suite must expose both immediate competence and room for improvement. We evaluate the loop on 18 RoboDojo dual-arm manipulation tasks. They cover language-conditioned arrangement, stacking, container placement, insertion, pouring, pushing and dynamic picking (Figures 3 and 2). The suite spans RoboDojo’s five official task categories (Precision, Long-Horizon, Memory, Open and Generalization) and samples three tasks from each of six difficulty strata. The strata are assigned from the official RoboDojo success rates [3]: the 18 tasks are ranked by success rate and grouped three per stratum, from D1 (success rate 60–68.7%) to D6 (success rate 0–0.4%). This balances coverage across difficulty levels. The strata are fixed before the main evaluation and are not derived from our refinement outcomes. All 18 tasks, including object packing, receive equal weight. Each retains its native score in $[0,1]$, success criterion and step limit of 300–1300 steps at 25 Hz. The main evaluation uses layout 0 and seed 0. Partial scores expose progress before the first success.

We evaluate six systems: GPT-6 Astra, GPT-6.1 Sol, GPT-6 Sol, GPT-5.6 Sol, Claude Opus 5.5 and Claude Sonnet 5.5. Each system comprises a model and its runtime configuration, which can also affect performance (Table 3). In a separate experiment without supplementary hints, Astra solves 13 tasks, compared with 15 in the main evaluation. Results from these conditions are reported separately.

Gemini 4 is reported in a separate partial run covering ten tasks through K20, with six solved. Its reported mean scores are 0.050 at K0 and 0.48 at K10. Its five-axis profile is computed on these ten tasks (Tables 1 and 5). Because the task set differs from the main 18-task protocol, it is excluded from the benchmark-based association and prediction analyses.

To study how physical RSI scales, we need more than an endpoint score. The five-axis profile separates the starting point from gain timing, attained score, physical budget savings and recovery from failure. Each axis measures a different aspect of improvement.

Let $q_{k}$ be the task-mean score at round $k$, with $q_{0}$ computed from three-run K0 means. Let $C_{k}$ be the fraction of tasks solved in Harness rounds K1–K$k$.

Strong coding scores alone do not ensure physical improvement. GPT-5.6 Sol exceeds GPT-6 Astra on LiveBench code generation and completion, yet solves one of the 18 tasks, whereas Astra solves 15.

The exploratory analysis uses six systems on all 18 tasks. We group 17 public benchmark scores into Spatial, Harness, Perception and Coding (Table 6), then compare their ranks with A1–A5 (Figure 4).

Spatial reasoning has the strongest observed association with A1 ($\rho=0.94$) and A2/A4 ($\rho=0.83$). Perception has the strongest association with A3 and A5 ($\rho=0.94$). Coding has the weakest association with every axis ($\rho\leq 0.37$).

The pattern motivates the Physical Barrel Hypothesis: different aspects of improvement may have different capability bottlenecks. Spatial reasoning may matter most for starting well, while perception may matter more for interpreting failures and recovering from them. Under this hypothesis, strong coding cannot always compensate for a weakness elsewhere. Its weak associations do not imply that coding is unnecessary; all systems use code here.

These associations identify candidate bottlenecks that targeted capability interventions can test directly (Section 12).

These candidate bottlenecks lead to the second question: how can we quantify physical RSI and its scaling with model capabilities and interaction budget?

Opus has a higher Success score than GPT-6.1 Sol (0.744 versus 0.703), but lower Efficiency (0.609 versus 0.822). On each system’s solved tasks, Opus saves less of the budget (0.579 versus 0.808), and the gap persists on the ten tasks both systems solve. Higher attained scores can come with later gains and greater physical costs (Tables 1 and 5).

Across the six systems, task-mean scores rise from 0.01–0.37 at K0 to 0.10–0.84 by the end of evaluation. They solve 15, 12, 7, 1, 13 and 11 of 18 tasks, respectively: Astra, GPT-6.1 Sol, GPT-6 Sol, GPT-5.6 Sol, Opus and Sonnet. Astra leads on all five axes. The two Claude systems pair relatively high attained scores with middling gain timing, while the GPT Sol family spans a much wider range. GPT-5.6 Sol remains low on Success and Improvement; its Budget score is conditional on one solved task. The profile exposes these trade-offs without a single ranking.

How much do public benchmark averages reveal about these differences? Across the four GPT systems, mean scores over 17 public benchmarks span 54.1–78.1, while the equally weighted physical profile mean, $100\times\sum_{j=1}^{5}A_{j}/5$, spans 13.9–73.4 (Figure 6). The ranges are 24 and 59.5 points on different scales. GPT-5.6 Sol stays near zero cumulative success for 16 rounds: more attempts do not close every capability gap within this budget.

The Physical Barrel Hypothesis offers one explanation. Useful improvement may require a minimum level of a needed capability, even when other capabilities are strong. The rapid contraction region in Figure 6 illustrates this threshold hypothesis and motivates predicting physical outcomes from capability profiles rather than a public benchmark average.

Can public capability scores anticipate physical improvement before the full evaluation is run? This is the cross-model question of observational scaling. Beyond correlating capabilities with outcomes, a predictor must place an unseen system using only the relationships observed in other systems.

We ask how far a system improves and when its gains emerge. Let $\bar{q}_{k}=\max_{1\leq j\leq k}q_{j}$ be the best mean score reached by round $k$. For the first question, ranking systems by the equal-weight average of their four capability profiles reproduces the observed order of $\bar{q}_{17}$, the best score reached by K17. For the second, we define the half-gain round $T_{50}$ as the first round at which $\bar{q}_{k}$ reaches half of its gain from K1 to K17. For every pair of systems, a kernel regressor fitted on the capability profiles of the remaining systems predicts which of the two reaches $T_{50}$ first, and all pairwise orderings are recovered (Figure 7). Across the six systems, public capability profiles carry information about both the extent and the timing of physical improvement.

Once physical interaction begins, can early progress predict further gains? The six systems reach their high-score windows at different times. The first round reaching 90% of peak gain, $\tau_{90}$, is K9 for Astra, K7 for GPT-6.1 Sol, K15 for Opus, K13 for Sonnet, and K17 for GPT-6 Sol and GPT-5.6 Sol. Forecasting later rounds tests whether a short trajectory contains useful information about continued improvement.

A related model may share the shape of its improvement curve. We fit a Weibull growth template $g_{A}$ to Astra’s K1–K17 mean score and forecast GPT-6.1 Sol as $\hat{q}(k)=u+(v-u)\,g_{A}(\rho(k-1))$. The start $u$, plateau $v$ and relative speed $\rho$ are calibrated on Sol’s first $K_{i}$ rounds only, with gain and speed regularized toward the template (Figure 8). With K1–K5 observed, the transferred curve forecasts the remaining rounds through K17 with mean absolute error 0.019, compared with 0.112 for holding the last observed score. A reference trajectory can predict a related model’s later performance from fewer rounds.

These analyses address the amount and timing of improvement. We next ask what the gains represent: adaptation to the refinement setting, or experience that remains useful elsewhere?

The Astra ablation varies which information from earlier rounds reaches the agent: the workspace, grader feedback, images and action records (Table 7). It covers six tasks with two chains each; layouts change every round over K1–K6, and execution continues after success. The full loop keeps all four channels, whereas the no-history condition keeps none, so its six rounds are independent attempts. Late-round success over K4–K6 is 15/36 (41.7%) for the full loop and 3/36 (8.3%) without history (Table 8). This supports a role for accumulated information under this protocol.

The remaining conditions withhold individual channels. Withholding grader feedback causes the largest drop, to 5/36 (13.9%), and withholding earlier-round images yields 12/36 (33.3%). Keeping only the workspace and grader feedback yields 17/36 (47.2%), so past images and action records add little once these two channels are kept. The no-history control supports a role for retained experience, but leaves open whether that experience is specific to the refinement setting.

To test whether the stored artifacts work beyond practice, we freeze Astra workspaces after K6 and test each on five unseen layouts, with no further revision or outcome feedback (Table 9). Across 12 sources, workspace files alone succeed in 25/60 tests (41.7%). Adding prior raw records gives 23/60 (38.3%). The stored artifacts remain useful under layout changes, while these point estimates show no benefit from exposing the raw history at test time.

Fixed-layout source workspaces reach 27/60 (45.0%) on held-out layouts, compared with the reported 14/16 (88%) on their own layout. The different denominators prevent a matched paired interpretation. Still, the comparison suggests that scene-specific gains and transferable experience can coexist. These tests establish some cross-scene usefulness for Astra on this task subset; they do not establish generalization across all systems.

Traces from five models suggest two forms of reusable experience: executable routines and textual operating knowledge. Sonnet 5.5 and Opus 5.5 increasingly encode control in self-written tools, which generate over 96% of their actions in later rounds. GPT-6 Sol also builds a substantial tool library, but tools generate 67% of actions while the model generates 33% directly. These shares measure tool reliance.

These tool libraries grow across rounds, combining tools created from scratch with those derived from earlier implementations. We use tool lineage to identify reuse across rounds. Direct use, modification and derivation of an existing tool all count as reuse. A directed relation A → B indicates that tool B is derived from tool A. Tracing these relations across rounds reveals how later tools build on earlier implementations. Among the three models with substantial tool libraries, Opus exhibits broader reuse and lower concentration. Its lineage reuse rate is 0.347, compared with 0.167 for Sonnet and 0.086 for GPT-6 Sol. The top 10% of reused lineages account for 48% of subsequent calls in Opus, compared with 76% in Sonnet and 83% in GPT-6 Sol. Together, these patterns suggest that Opus accumulates a broader reusable toolbox, while reuse in Sonnet and GPT-6 Sol remains concentrated in a smaller subset of tools (Figure 10).

Astra and GPT-6.1 Sol follow a different pattern. Approximately 99% of their actions remain directly generated by the model. They use only a very small number of tools for verification, but their reasoning time decreases across rounds. File-access traces show repeated use of memory, skills and policy notes to recover successful actions and subtask solutions, diagnose failures and guide revisions. These observations suggest that both executable routines and retained operating knowledge can reduce repeated reasoning.

We further find that the workspace and generalization analyses align with the Physical RSI Barrel Hypothesis across its four capability factors: spatial reasoning, perception, coding and harness operation. Spatial tools account for 14.5% of Sonnet’s tool calls and 13.7% of Opus’s, with 408 of 2,819 calls and 183 of 1,332 calls, respectively. Astra makes only seven tool calls, including one to such a tool, or 14%. Astra’s minimal reliance on these modules suggests stronger intrinsic perceptual and spatial capabilities. Sonnet and Opus repeatedly reuse these modules across tasks, suggesting that they rely more on tools to compensate for limitations in these capabilities.

During cross-task transfer, agents also consult memory and skills from semantically similar tasks. In Organize Table, Opus combines a parameterized move tool from Pack Objects Into Box with a memory from Store Tools In Toolbox: “the PITCHED 30 deg grasp… reaches the +y half…” The tool supplies an operation, while the memory identifies a grasp orientation that extends its reach. This shows how a reusable procedure and knowledge of when it works can jointly support transfer (Figure 9).

The transfer traces suggest that performance gains depend on experience relevant to the target task’s bottleneck and the agent’s ability to assess its applicability from failure evidence. Semantic similarity helps identify candidate experience; failure evidence guides whether to reuse, adapt or discard it. The teaser illustrates this mechanism through the reuse of a closed-loop grasp-and-transfer routine from stacking blocks to covering blocks with a cup, and an insertion routine from key insertion to charger plugging.

An action sets a target pose for each end-effector, within 5 cm of translation and 0.35 rad of rotation of its current pose per call, executed for 1–5 control steps at 25 Hz; each call returns a state JSON and three camera images. Per round, the reasoning budget is 4200 s and the revision window after grading is 1800 s (GPT systems) or 600 s (Claude systems). The physical budget per task is $17\,L_{t}$, that is, 17 full-step-limit attempts.