Closing the Lab-to-Sto… | arXiv Physical AI Research Summary

1. Key Themes

Systems Integration Over Architectural Novelty

The paper's core thesis is that the gap between benchmark performance and reliable real-world deployment for Vision-Language-Action (VLA) robots is primarily a systems integration challenge, not an architectural one. The authors demonstrate that careful data design and targeted post-training can turn a failing policy into a competent one without modifying the underlying foundation model. As stated in the abstract: "Our results suggest that bridging the lab-to-store gap is primarily a systems integration challenge rather than an architectural one: careful data design and targeted post-training can transform a policy that fails under naive fine-tuning into a competent real-world system using only a single GPU."

The DEED Post-Training Recipe

The authors introduce DEED (Data-Efficient Post-Training and Experience-Driven Learning), a pipeline that includes control-frequency alignment, data curation, visual highlighting, and reduced VLA dependence. The practical impact of this recipe is substantial: it took a naive fine-tuned GR00T model from a 0% success rate to a 32% success rate on a real-world supermarket restocking task. The paper notes in Section III-B: "Reaching a deployable policy required the combined design decisions of the Data-Efficient recipe, not fine-tuning alone: the reduced action space, added wrist cameras, matched control frequencies, curated demonstrations, and output smoothing together take the policy from non-functional to competent."

Experience-Driven Learning via RECAP Adaptation

The framework adapts the RECAP (RL with Experience and Corrections via Advantage-conditioned Policies) method to GR00T's decoupled architecture using a text-based advantage prefix and a vision-language value function. This allows the robot to learn from its own autonomous rollouts and human corrections. A single iteration of this experience-driven refinement improved the success rate from 32% to 42% and reduced execution time. Section III-B states: "Building on this policy, a single Experience-Driven iteration improves both success rate and efficiency, raising success to 42% (21/50) while reducing per-bag time to 22.37 s."

Latent-Space In/Out-Distribution Analysis

DEED includes a tool for measuring how far a robot's current state deviates from its training data using the policy's own latent space. This provides a model-agnostic way to monitor deployment reliability and diagnose performance drops. Section II-C explains: "States are compared in the latent space of the fine-tuned GR00T state encoder, the representation the policy itself consumes when producing actions... a state flagged as distant is one the policy is genuinely likely to mishandle, making the measure a predictor of behavioral reliability rather than an abstract geometric quantity."

2. Contrarian Perspectives

Naive Fine-Tuning of Foundation Models Fails in the Real World

Despite the hype around generalist VLA foundation models like GR00T, simply fine-tuning them on task-specific data is insufficient for real-world deployment. The paper shows that a naive supervised fine-tuning (SFT) approach on the GR00T N1.6 model yielded a 0% success rate on a retail restocking task. Section III-B notes: "Despite the embodiment-specific initialization, this naive SFT is unable to perform the restocking task: without the DE engineering improvements it does not reliably grasp or place the bags, yielding a 0% success rate."

More Experience-Driven Iterations Do Not Always Compound

While reinforcement learning from experience is often assumed to improve performance iteratively, the authors found that a second iteration of their RECAP adaptation actually degraded performance, dropping the success rate from 42% to 22%. This was due to distributional drift as self-generated rollouts dominated the training set. Section III-B states: "A second iteration does not continue this trend: success drops to 22% (11/50) despite a further small reduction in execution time (21.09 s). We hypothesize this degradation stems from distributional effects during iterative refinement..."

Foundation Models Shouldn't Control Everything End-to-End Yet

The paper challenges the notion that VLA models should handle all aspects of control end-to-end. For discrete actions like opening and closing a gripper, the authors recommend using a binary hand controller rather than continuous VLA control, as current foundation models lack the high-frequency, fine-grained adaptation needed for physical interactions. Section II-A advises: "For tasks reducing primarily to opening and closing the hand, we recommend controlling the hand as a gripper via a binary signal rather than learning continuous, fine-grained joint control."

3. Companies Identified

HIVE Robots

Description: A robotics company based in Denmark focused on deploying humanoid robots.
Why relevant: The authors are affiliated with HIVE Robots, and the paper is motivated by an ongoing industrial effort to deploy humanoid restocking in an operating supermarket.
Quotes: "1 HIVE Robots, Denmark"

Unitree

Description: A robotics hardware manufacturer.
Why relevant: The experiments were conducted using the Unitree G1-Edu humanoid robot with Dex-3 hands.
Quotes: "evaluated on a supermarket chip-restocking task using a Unitree G1-Edu humanoid robot" (Abstract)

NVIDIA

Description: A technology company known for GPUs and AI hardware/software.
Why relevant: The GR00T N1.6 foundation model used in the paper is an NVIDIA model, and training was done on a single NVIDIA RTX 5090 workstation.
Quotes: "All models are initialized from the publicly available GR00T-N1.6-G1-PnPAppleToPlate checkpoint... run on a single NVIDIA RTX 5090 workstation" (Section III-A)

Physical Intelligence

Description: A robotics AI company.
Why relevant: The authors adapted the RECAP recipe, originally introduced by Physical Intelligence with their π*0.6 model, to the GR00T architecture.
Quotes: "we adapt the RECAP (RL with Experience and Corrections via Advantage-conditioned Policies) recipe, originally introduced with π*0.6" (Section II-B)

Intel

Description: A technology company manufacturing computing hardware.
Why relevant: The robot's visual input was provided by Intel RealSense cameras (D435i and D405).
Quotes: "one head-mounted camera (Intel RealSense D435i) and two wrist-mounted cameras (RealSense D405)" (Section III-A)

4. People Identified

Roger Sala Sisó

Lab/Institution: HIVE Robots & Technical University of Denmark
Why notable: Co-author and equal contributor to the research on deploying VLA humanoids in retail settings.
Quotes: "Roger Sala Sisó 1,2,∗"

Tiago Silvério

Lab/Institution: HIVE Robots
Why notable: Co-author and equal contributor, focused on the systems-level approach to bridging the lab-to-store gap.
Quotes: "Tiago Silvério 1,∗"

Jakob Sand

Lab/Institution: HIVE Robots
Why notable: Co-author involved in the industrial effort to deploy humanoid restocking.
Quotes: "Jakob Sand 1"

Tran Nguyen Le

Lab/Institution: Technical University of Denmark
Why notable: Co-author contributing to the academic and research aspects of the framework.
Quotes: "Tran Nguyen Le 2"

5. Operating Insights

Match Control and Recording Frequencies

A critical early decision in VLA deployment is establishing a clear frequency hierarchy between cameras, teleoperation, recording, and inference control. Recording or acting faster than the camera leads to duplicated observations and unreliable behavior. Section II-A states: "Recording and inference control frequencies must match, since the policy learns the temporal semantics of the action space at whatever rate it was recorded at, and dispatching actions at any other rate at inference time is a mismatch the model was never trained to handle."

Curate Data to Include Recoveries, But Trim Failures

When collecting demonstration data, it is essential to retain incidental recoveries so the robot learns how to correct itself when prediction errors compound. However, failed segments should be trimmed to prevent the model from learning bad patterns. Section II-A advises: "A policy trained purely on clean successes has no learned behavior for returning to the task once prediction errors compound and drift it into poorly represented states; we trim only the failed segment of an episode and keep the recovery, letting the policy learn how to respond when it departs from normal execution."

Use Continuous Episodes for Multi-Subtask Demonstrations

For tasks involving multiple subtasks, segmented demonstrations can cause failures at transition points because the model expects episode boundaries. Recording continuous episodes that span full subtask transitions simplifies data collection and improves performance. Section II-A notes: "Rather than the common strategy of segmenting long-horizon demonstrations with overlapping boundaries, we recommend recording continuous episodes that span full subtask transitions, which also simplifies data collection."

6. Overlooked Insights

Reset Behavior Suppression in Refined Policies

A structural finding from the experience-driven learning stage is that refined policies tend to stop after a single placement rather than returning to their start configuration. This occurs because the terminal completion state carries a higher value than the intermediate poses of a reset, causing reset transitions to receive low or negative advantage and be suppressed during training. Section III-C explains: "refined policies tend to stop after a single placement rather than return to their start configuration, since the terminal completion state carries higher value than the intermediate poses of a reset, so reset transitions receive low or negative advantage and are suppressed during training."

Value Function Design Constraints

The design of the value function is critical, and certain choices can significantly degrade performance. The authors found that encoding the advantage as an additional state dimension significantly hurt performance, likely because it violated the zero-padded state convention established during pretraining. Section III-B states: "Encoding the advantage as an additional state dimension significantly degraded performance, likely because it violated the zero-padded state convention established during pretraining."