Xiaomi-Robotics-1… | arXiv Physical AI Research Summary

1. Key Themes

Pre-Training on Non-Robot Data Transfers to Real Robot Performance

The central contribution is demonstrating that massive-scale pre-training on UMI (handheld gripper) data — not robot teleoperation data — produces manipulation capabilities that transfer to actual robots after post-training. The model is pre-trained on over 100,000 hours of real-world manipulation trajectories collected via UMI devices across households, commercial premises, industrial sites, offices, and outdoor spaces (Section 2.2.1, Figure 3). After post-training on ~10K hours of cross-embodiment robot data, the model achieves 75% out-of-the-box success rate across four tasks in unseen environments, compared to 26% for a baseline without action pre-training (Section 3.2.1, Figure 8). This validates the thesis that you can decouple scalable data collection from expensive robot teleoperation.

Scaling Laws Hold for Physical AI — But Data Is the Bottleneck, Not Model Size

The paper provides systematic evidence that VLA models scale with both data and model size, but the two axes are not equal. In data-scaling experiments, validation action error decreases monotonically with more data, while smaller data subsets overfit (Section 3.1, Figure 5). In model-scaling experiments across 2B, 5B, and 10B parameter variants, improvements are "less pronounced than those observed across different data scales" (Section 3.1). The authors explicitly state: "model capacity at the billions-parameter scale may already be sufficient to capture the current dataset's distribution, thereby making data volume the primary bottleneck for further generalization" (Section 3.1). Critically, pre-training scaling behavior transfers to post-training: the model pre-trained on 100% of data achieves 75% out-of-the-box success vs. 53% with only 12.5% of data (Section 3.2.1, Figure 8).

Auto-Labeling Pipeline Replaces Human Annotation at 100K-Hour Scale

A major operational contribution is the scalable auto-labeling pipeline that annotated the entire 100K+ hour corpus in roughly two weeks. The pipeline uses Qwen3.5-27B to caption scene state transitions for fixed-length trajectory segments, with a producer-consumer architecture that decouples clip segmentation from captioning (Section 2.2.1). This is distinct from traditional approaches requiring "manual segmentation by task semantics and language annotations — a labor-intensive process that becomes prohibitive at this scale" (Section 1). The labels describe state transitions (e.g., "the gripper moves toward the cup and grasps it") rather than imperative instructions, which the model learns to map to task commands during post-training.

State-of-the-Art Across Four Simulation Benchmarks

Xiaomi-Robotics-1 achieves SOTA on RoboCasa (74.5%), RoboCasa365 (57.4% vs. previous best of 46.6%), VLABench (59.1% average SR), and RoboDojo (20.07 average score vs. previous best of 13.07) (Section 3.4, Tables 2-5). On RoboCasa365's most challenging split — Composite-Unseen tasks that test zero-shot recomposition of learned skills — it achieves 32.1% success, nearly quadrupling the previous best of 7.9% (Table 3). This suggests the model learns composable primitives rather than memorizing task-specific policies.

Data-Efficient Fine-Tuning on Complex Dexterous Tasks

With less than 10 hours of fine-tuning data per task on average, the model achieves 75% average success rate across four challenging novel tasks (phone packing with bimanual coordination, laundry loading as long-horizon mobile manipulation, printer refilling with deformable paper sheets, and box packing), outperforming π0.5 which achieves 40% (Section 3.3, Figure 10). On printer refilling specifically — a task involving highly deformable objects — Xiaomi-Robotics-1 improves from 20% (best baseline) to 70% success in the low-data setting (Section 3.3).

2. Contrarian Perspectives

Model Scaling Is Not the Primary Lever — Data Volume Is

Most well-funded robotics companies (Figure AI, 1X, Apptronik) are racing to build larger models. This paper's evidence suggests that at the billions-parameter scale, model size yields diminishing returns relative to data scale. The authors found that "the performance gap among different model sizes are less pronounced than those observed across different data scales" and that "model capacity at the billions-parameter scale may already be sufficient to capture the current dataset's distribution, thereby making data volume the primary bottleneck" (Section 3.1). The practical implication: a company with 10B parameters and 100K hours of diverse data will outperform one with 50B parameters and 10K hours. Capital should flow toward data infrastructure, not just compute.

You Don't Need Robots to Collect Robot Training Data

The dominant paradigm in robotics data collection is teleoperation — slow, costly, and hardware-bound. This paper challenges that by pre-training entirely on UMI handheld gripper data (no robots involved) and showing the capabilities transfer after post-training. The authors state: "robotics is hindered by a unique bottleneck of data. The dominant data collection paradigm, real-robot teleoperation, is slow, costly, and hardware-bound, making it difficult to scale" (Section 1). Their approach — 100K+ hours collected with $200 handheld grippers — suggests that companies investing heavily in teleoperation fleets may be building an unnecessarily expensive data pipeline. The key insight is that manipulation primitives (grasping, reaching, placing) are embodiment-agnostic at the pre-training stage, and only post-training needs robot-specific data.

State-Transition Descriptions Are Better Pre-Training Targets Than Task Instructions

Most VLA models are trained on imperative instructions ("pick up the cup"). This paper pre-trains on descriptive state-transition language ("the gripper approaches the red cup and closes around the handle") and only switches to imperative instructions during post-training. The authors argue this provides "rich and precise conditioning for action learning" (Abstract) and that post-training "effectively maps descriptive state transition understanding into actionable task prompts" (Section 1). This two-stage approach — learn what happens in the world first, then learn to follow commands — mirrors the LLM pre-training/SFT paradigm more faithfully than current VLA approaches that train on instructions from the start.

3. Companies Identified

Xiaomi Robotics — The authors and developers of Xiaomi-Robotics-1. They have previously released Xiaomi-Robotics-0 (referenced as [8]). They are positioning themselves as a major player in Physical AI with open-source releases ("Code and model checkpoints will be released"). Their competitive advantage is access to massive data collection infrastructure and the ability to deploy on their own hardware.

Physical Intelligence — Referenced extensively as the creator of π0, π0.5, π0.6*, and π0.7. Their π0.5 model is the primary baseline for downstream fine-tuning comparison, where Xiaomi-Robotics-1 outperforms it significantly (75% vs. 40% success rate with <10 hours of fine-tuning data). Their OpenPi fine-tuning protocol is used. This positions Xiaomi as a direct competitor to Physical Intelligence in the foundation model space.

NVIDIA — Referenced as creators of GR00T N1 (generalist humanoid robot foundation model) and Cosmos Policy/Cosmos 3 (world models). GR00T N1.6 appears as a baseline in RoboCasa (66.2%) and RoboCasa365 (21.9%) benchmarks, both significantly outperformed by Xiaomi-Robotics-1 (74.5% and 57.4% respectively). NVIDIA's role as both a compute provider and model developer in this space is notable.

Alibaba/Qwen Team — Qwen3-VL is the VLM backbone used in Xiaomi-Robotics-1's architecture (Section 2.1), and Qwen3.5-27B is used for the auto-labeling pipeline (Section 2.2.1). This makes Alibaba's models critical infrastructure for Xiaomi's robotics stack. The Qwen-robotmanip model also appears as a baseline in RoboCasa365 (35.9%).

Google DeepMind — Referenced via RT-1 and RT-2 (robotics transformer models) and Gemini Robotics. RT-1 data is included in the post-training dataset. Gemini Robotics is referenced in related work.

Galaxea — Referenced as creator of GalaxeaVLA (G0), which appears as a baseline in RoboDojo with relatively low scores (5.82 average score vs. Xiaomi's 20.07).

4. People Identified

Jun Guo, Piaopiao Jin, Jason Li, Peiyan Li, Yingyan Li, Futeng Liu, Wanli Peng, Optimus Qin, Yifei Su, Nan Sun (Core Contributors) — Xiaomi Robotics Team. These are the lead researchers building Xiaomi's robotics foundation model stack, including both Xiaomi-Robotics-0 and Xiaomi-Robotics-1. Their work represents one of the most well-resourced VLA efforts, with access to 100K+ hours of proprietary data.

Chelsea Finn (Physical Intelligence / Stanford) — Referenced across multiple works (π0, π0.5, π0.6*, π0.7, OpenVLA, FAST). Her group's work on flow-matching action generation and VLA fine-tuning protocols directly informs Xiaomi-Robotics-1's architecture and training recipe. The fact that Xiaomi-Robotics-1 significantly outperforms π0.5 in fine-tuning benchmarks is a notable competitive signal.

Shuran Song (Stanford / Physical Intelligence) — Co-creator of UMI (Universal Manipulation Interface), which is the data collection backbone for Xiaomi-Robotics-1's 100K+ hour pre-training dataset. Also co-creator of Diffusion Policy and DROID dataset. Her work on scalable data collection is foundational to this paper's approach.

Russ Tedrake (MIT / Toyota Research Institute) — Co-creator of UMI and referenced in multiple works. His contributions to manipulation interface design enable the data scaling strategy this paper demonstrates.

Sergey Levine (UC Berkeley / Physical Intelligence) — Referenced across multiple works including training strategies for embodied reasoning and DROID dataset. His work on robot foundation models and reinforcement learning underpins much of the field.

5. Operating Insights

The UMI + Auto-Labeling Pipeline Is a Replicable Data Strategy

The most operationally significant finding is that 100K+ hours of manipulation data can be collected and labeled without robots and without human annotators. The UMI devices are handheld grippers with cameras that anyone can use — no robot hardware required. The auto-labeling pipeline uses a VLM (Qwen3.5-27B) to caption state transitions for fixed-length clips, with a producer-consumer architecture that processed the entire corpus in two weeks (Section 2.2.1). A CTO building a robotics company should seriously evaluate whether their data collection strategy should prioritize UMI-style collection (cheap, scalable, diverse) over teleoperation (expensive, slow, narrow). The key architectural decision is pre-training on state-transition descriptions and post-training on imperative instructions — this two-stage approach is what makes the UMI data useful for robot control.

Fine-Tuning Data Requirements Are Dramatically Lower Than Expected

The paper shows that with a strong pre-trained foundation model, complex dexterous tasks can be learned with less than 10 hours of task-specific data — including bimanual coordination (phone packing), long-horizon mobile manipulation (laundry loading), and deformable object handling (printer refilling) (Section 3.3, Figure 10). This has direct implications for deployment economics: if your foundation model is strong enough, the marginal cost of adding a new task is ~10 hours of teleoperation rather than hundreds. Companies should invest in the foundation model first, then amortize data collection across many downstream tasks. The progress metric (Table 6) is also worth adopting — it measures partial task completion across milestones, giving a more granular view of where policies fail than binary success/failure.

Action Space Unification Across Embodiments Is Critical and Non-Obvious

The paper unifies action spaces across UMI grippers, static robot arms, and mobile manipulators by using relative delta end-effector poses and aligning end-effector frame orientations across all platforms (Section 2.2.2). Dimensions corresponding to missing action components are masked during loss computation. This is a practical detail that matters enormously for cross-embodiment training — without it, the model would learn conflicting action representations. Any team building a multi-robot or cross-embodiment policy should adopt this approach.

6. Overlooked Insights

Excluding Action Tokens from DiT Attention Prevents a Shortcut Learning Failure Mode

The paper describes a subtle architectural finding: when the DiT (diffusion transformer) is allowed to attend to the action-related tokens generated by the VLM (via Choice Policies), performance degrades. The authors hypothesize "this arises from a shortcut in which the DiT simply copies the actions generated by the VLM rather than effectively grounding its own generation in the visual and textual context" (Section 2.1). They solve this by excluding action-related tokens from the DiT's attention computation. This is a non-obvious architectural detail that could save teams months of debugging when building multi-component VLA architectures — if your diffusion head is underperforming, check whether it's shortcutting by copying from the VLM rather than grounding in visual context.

The Data Sampling Ratio Reveals Where Value Actually Comes From

During post-training, the sampling ratio is 0.5:0.5:0.5:8.5 for vision-language data, open-source robot data, instruction-labeled UMI data, and in-house robot data respectively (Section 2.2.2). This means in-house robot data is sampled 17x more frequently than open-source robot data (Bridge V2, RT-1, DROID). Despite the field's excitement about open-source datasets, this ratio suggests that proprietary, high-quality, in-domain robot data remains the dominant driver of post-training performance. Open-source data contributes diversity but is heavily diluted. Companies should not rely on open-source datasets as their primary training signal — they are a supplement, not a substitute, for proprietary data collection.