One Policy, Many… | arXiv Physical AI Research Summary

1. Key Themes

Camera-Centric Geometry as a Universal Action Interface

The core breakthrough is reframing what a robot policy should predict. Instead of outputting robot-specific commands (joint angles, end-effector deltas, gripper states), UCAG-P predicts the 3D camera-frame trajectory of two semantic anchors: the wrist/end-effector point (p0) and the grasp center (p1). This representation is embodiment-agnostic — a human hand, a single-arm robot, and a humanoid all produce observable wrist and grasp motion in camera coordinates. The paper states: "the most stable common structure across embodied demonstrations lies not in the low-level controller, but in the camera-observable geometry of manipulation" (Section 1). A separate geometry-conditioned translator then converts these shared predictions into executable commands for whichever robot you want to deploy on, using that robot's Jacobians, camera-to-base transform, and current state (Section 4.3, Equation 3).

Direct Integration of Human Video Without Retargeting

Nearly 37% of the training corpus (2,340 hours) comes from egocentric human hand videos — VITRA, EgoDex, and EgoVerse — containing zero robot commands. Instead of the standard pipeline of retargeting human motion to robot kinematics or synthesizing robot videos from human data, UCAG-P extracts wrist and grasp-center keypoints from human video (using MediaPipe) and treats them as just another embodiment in the shared camera-centric space. The paper explicitly states: "human demonstrations are directly treated as a distinct embodiment within the shared action space, allowing human-hand motion directly to supervise policy learning, eliminating the need for explicit human-to-robot video synthesis or retargeting" (Section 1, bullet 2). This is a massive data scaling unlock — any egocentric manipulation video becomes usable training data.

Single Checkpoint Competitive Across Single-Arm, Bimanual, and Humanoid

One model, one checkpoint, no benchmark-specific fine-tuning. The results: 98.3% on LIBERO (single-arm), 88.7% and 89.2% on RoboTwin Easy and Hard (bimanual), 82.0% zero-shot on LIBERO-Plus (distribution shift), and 62.0% on RoboCasa GR-1 (humanoid with arms, hands, and waist control). Table 3 shows UCAG-P outperforming the generalist Qwen-VLA across all four benchmarks while remaining competitive with specialist models that are fine-tuned per benchmark. On LIBERO-Plus distribution shift (Table 4), UCAG-P achieves 92.8% robustness to robot-state perturbations and 98.9% on lighting changes — both zero-shot.

Three-Stage Decoupled Training Pipeline

The training is split into three stages (Section 4.4, Figure 6): Stage 1 trains the VLM backbone and motion head on all data with camera-centric supervision (200K steps, 128 H20 GPUs); Stage 2 trains only the action translator on robot data using ground-truth trajectories (10K steps, 8 H20 GPUs); Stage 3 jointly optimizes both on the mixed robot-human corpus (10K steps, 64 H20 GPUs). This staged approach isolates motion prediction from action translation, preventing upstream motion errors from corrupting translator training.

6,373 Hours of Heterogeneous Training Data

The corpus spans 11 datasets, 1,020,672 episodes, and 9 embodiments (Table 1). Simulation data dominates at 59.11% (3,768 hours), human hand video contributes 36.71% (2,340 hours), and real-robot data is 4.18% (266 hours). The largest single source is InternData-MultiRobot at 3,649 hours of simulated multi-robot trajectories. Notably, real-robot data is a small fraction — the system leans heavily on simulation and human video, with real-robot data primarily for calibration and validation.

2. Contrarian Perspectives

Human-to-Robot Retargeting Is Unnecessary — and Possibly Harmful

The dominant approach in the industry (used by companies like Physical Intelligence and many academic labs) is to either retarget human motion to robot kinematics before training, or to generate synthetic robot videos from human demonstrations. UCAG-P argues this is both computationally wasteful and conceptually wrong. Retargeting forces you to assume a target robot morphology before training, locking the data to one embodiment. By instead representing human motion as camera-centric anchor trajectories, the same human video can supervise policies for any robot. The paper states: "completely bypassing the need for computationally expensive robot-video inpainting, video editing, or explicit action retargeting as a prerequisite" (Section 1). The real-world bread-pickup task (Figure 7) validates this: UCAG-P achieves 60% success on human-to-robot transfer using only MediaPipe keypoints, compared to 20% for π0.5.

The Shared Policy Target Should Not Be a Robot Action

Most VLA models (π0, OpenVLA, GR00T N1, RT-2) predict robot-native commands as the policy output — joint angles, end-effector deltas, or action tokens. UCAG-P argues this is the wrong abstraction layer for cross-embodiment learning. The paper states: "A single low-level command parameterization therefore cannot serve as the shared prediction target for all data sources" (Section 4.1). Even Qwen-RobotManip, which aligns robot data through a unified camera-frame end-effector space, "remains centered on robot end effectors and does not directly capture human grasp behavior" (Section 1). By making camera-observable geometry the shared target and pushing robot-specific execution to a separate translator, the base policy learns transferable manipulation geometry that isn't contaminated by any single robot's control interface.

Specialist Fine-Tuning Is Not Worth the Operational Cost

Every major competitor in Table 3 — π0, GR00T N1, Being-H0.7, ABot-M0, ZR-0 — is a specialist model fine-tuned per benchmark. UCAG-P matches or exceeds most of them with a single checkpoint and no benchmark-specific training. On RoboTwin Hard, UCAG-P's 89.2% beats π0.5's 76.8% by 12.4 points. On LIBERO-Plus, UCAG-P's 82.0% zero-shot beats ABot-M0's 80.5% (which was post-trained on standard LIBERO). The implication: maintaining separate fine-tuned checkpoints per robot, task, or benchmark may be an operational burden that doesn't pay off if your base representation is good enough.

3. Companies Identified

Xiaomi — The paper is authored by Xiaomi's Embodied Intelligence Team. This signals Xiaomi is investing seriously in generalist robot policies, not just consumer robots. The UCAG-P project page and code are hosted publicly, suggesting they are building an ecosystem play. Their real-world evaluation uses Piper robots, which appear to be Xiaomi's internal robotic platform.

Physical Intelligence (π0, π0.5) — Referenced as baselines [4, 43]. π0.5 is directly compared on real-world Piper robot tasks (Figure 7), where UCAG-P outperforms it on bread pickup (60% vs 20%), drawer opening (90% vs 85%), and bowl stacking (75% vs 65%). Physical Intelligence's approach of specialist fine-tuning is positioned as the competing paradigm.

NVIDIA (GR00T N1.6) — Referenced as a baseline [39]. GR00T N1.6 scores 97.2% on LIBERO and 49.9% on RoboCasa GR-1 as a specialist, compared to UCAG-P's 98.3% and 62.0% as a generalist. NVIDIA's foundation model for humanoid robots is directly outperformed on humanoid tasks by Xiaomi's generalist approach.

Alibaba/Qwen Team (Qwen-VLA, Qwen3-VL) — The VLM backbone is Qwen3-VL-4B-Instruct [2], and Qwen-VLA [58] is the generalist baseline that UCAG-P outperforms across all benchmarks. The Qwen-RobotManip work [48] is cited as prior art that UCAG-P improves upon by extending beyond robot end-effectors to include human hand data.

Google DeepMind (RT-2, OpenVLA) — Referenced as foundational VLA work [5, 19]. OpenVLA is a baseline on LIBERO-Plus (Table 4), scoring only 15.6% average zero-shot, highlighting the generalization gap that UCAG-P addresses.

Various startups and labs — Being-H0.5/H0.7 (Peking University), ABot-M0, ZR-0, JoyAI-RA, DeMaVLA, LingBot-VLA, MolmoAct2, OASIS, StarVLA-OFT are all referenced as specialist baselines in Table 3, representing the competitive landscape of VLA models.

4. People Identified

Shaoqing Xu — Xiaomi Embodied Intelligence Team, core contributor and project lead. Also appears on multiple other Xiaomi robotics papers (VilTA, SpaAct, Think Before Go, FPC-VLA, OneVL), indicating he is a key figure in Xiaomi's embodied AI research program.

Fang Li — Xiaomi Embodied Intelligence Team, equal core contribution. Appears across multiple Xiaomi papers including autonomous driving VLA work (AdaThinkDrive, Last-VLA), suggesting cross-domain expertise between driving and manipulation.

Zhixiang Duan — Xiaomi Embodied Intelligence Team, core contributor and project lead. Co-author on FPC-VLA, indicating involvement in failure prediction and correction for robot policies.

Zhi-Xin Yang — University of Macau, corresponding author. His lab is the academic partner, suggesting an industry-academia collaboration model that Xiaomi is using for embodied intelligence research.

Yuechen Luo — Xiaomi Embodied Intelligence Team, contributor. Appears on multiple Xiaomi autonomous driving and robotics papers, again showing cross-domain VLA expertise.

5. Operating Insights

The Data Strategy: Human Video Is Your Cheapest Scaling Lever

The training corpus reveals a striking data economics story. Real-robot data costs the most to collect but contributes only 4.18% of total hours (266 hours). Simulation provides 59.11% (3,768 hours) at lower cost. But the biggest untapped resource is human video at 36.71% (2,340 hours) — and this is the cheapest to acquire at scale. If you are building a robot company, the implication is clear: your data acquisition strategy should aggressively leverage egocentric human manipulation video. UCAG-P demonstrates that with the right action representation, this data requires no retargeting, no robot-video synthesis, and no assumption about target morphology. The paper notes that human data "supplements robot trajectories with diverse object interactions and embodiment-independent motion cues" (Section 3.1). The real-world bread-pickup task (60% success from pure human-to-robot transfer, Figure 7) validates that this isn't just a training trick — it produces deployable policies.

Adding a New Robot Only Requires Training the Translator, Not the Base Policy

The decoupled architecture has a direct operational implication for fleet management. If you deploy UCAG-P on a new robot, you don't need to retrain the entire VLA backbone. The shared motion head (which predicts camera-centric geometry) transfers as-is. You only need to train the geometry-conditioned action translator (Stage 2), which is lightweight — 10K steps on 8 H20 GPUs (Table A.1) versus 200K steps on 128 H20 GPUs for the base policy (Stage 1). The translator needs ground-truth camera-centric trajectories, robot state, calibration, and executable command labels — all available from standard robot demonstrations. This means the marginal cost of supporting a new robot platform is roughly 1/200th of the base training cost.

Calibration Quality Is a Hard Dependency — Plan for It

The paper's limitations section is explicit: "UCAG-P relies on geometric information that must be estimated or calibrated reliably. Errors in camera calibration, depth estimation, embodiment kinematics, or MediaPipe hand-keypoint localization can propagate into the camera-centric target and the downstream translator" (Section 6). This means your deployment pipeline needs robust camera calibration, reliable depth estimation, and accurate forward kinematics. If any of these are noisy — which they often are in real-world deployments — the entire action chain degrades. The failure cases in Appendix D confirm this: representative failures include errors in anchor detection and trajectory prediction propagating to action translation. For a CTO, this means investing in calibration infrastructure is not optional — it is a prerequisite for this architecture to work.

6. Overlooked Insights

The 80-Dimensional Sparse Command Layout Is a Quiet Engineering Win

Buried in Appendix A.3 (Table A.3) is an 80-dimensional command layout that packs left arm, left end-effector, left hand, right arm, right end-effector, right hand, waist, and mobile base into fixed-width slots. Each robot only activates the blocks it needs — LIBERO uses slots 0-9 and 20-29 (left arm + gripper), RoboTwin uses 0-9, 20-39, 50-59 (both arms + grippers), RoboCasa GR-1 uses 0-9, 20-39, 50-69 (both arms + hands + waist). Inactive dimensions are masked from the loss and never sent to the controller. This means you can train a single-arm robot, a bimanual robot, and a humanoid in the same batch without their action spaces interfering with each other. For a company building a robot fleet with mixed morphologies, this eliminates the need for separate training runs per robot type — a significant infrastructure cost saving that isn't highlighted in the main text.

VLM Context Boosts Humanoid Performance by 3.7 Points

The ablation in Table B.2 shows that adding attention-pooled VLM features to the action head improves RoboCasa GR-1 success from 58.3% to 62.0%. This means the visual-language context from the backbone provides complementary information that pure geometric translation misses — particularly important for humanoid robots where the action space is more complex (arms, hands, and waist). The implication: if you are building a humanoid robot policy, do not strip visual-language context from your action prediction module. The 3.7-point gap is the difference between second place and fourth place in the RoboCasa GR-1 leaderboard.

Articulated Objects Remain a Hard Failure Mode

In the full RoboTwin task-level results (Table B.5), UCAG-P scores only 11% (Clean) and 13% (Randomized) on OpenMicrowave — dramatically lower than competitors like π0.5 (34%/77%) or Motus (95%/91%). This is despite UCAG-P achieving 88-89% average on RoboTwin. The paper doesn't discuss this in the main text, but it reveals that camera-centric geometry may struggle with articulated objects where the manipulation requires understanding joint constraints, not just wrist/grasp trajectory. For companies deploying robots in kitchen or household environments, this is a critical limitation to monitor — the aggregate numbers hide task-specific failure modes that could block deployment.