Riemann-1.0: An… | arXiv Physical AI Research Summary

1. Key Themes

Unified Policy and Simulator in a Single Causal Model

Riemann-1.0's core architectural innovation is a fully causal autoregressive formulation where actions are predicted before their visual consequences, mirroring how real robots interact with the world. This means one model serves as both an executable robot policy (observation → action) and an action-conditioned visual world simulator (action → future observation). The paper states: "This fully causal formulation naturally aligns with the interaction process of real-world robots, enabling a single model to simultaneously function as both an executable robot policy and a multi-embodiment action-conditioned visual world simulator" (Section 1). This is architecturally distinct from competitors like DreamZero (joint denoising), LingBot-VA (video-first prediction), and FastWAM (decoupled DiTs), as illustrated in Figure 3.

Progressive Pretraining from Human Video to Robot Control

The paper introduces a three-stage curriculum that progressively shifts from weakly supervised human video to executable robot actions, with the action loss weight increasing from λ=0.1 → 0.5 → 0.9. Stage 1 uses a frozen Latent Action Model to extract pseudo-actions from 200K+ hours of unlabeled egocentric human video. Stage 2 aligns these pseudo-actions with real trajectories from UMI demonstrations, robot data, and 3D hand-annotated videos. Stage 3 specializes on robot-only data. The paper explains: "This stage is not intended to learn a deployable robot policy; with λ=0.1, it mainly initializes the visual dynamics backbone and teaches the model how action-like motion tokens correlate with future visual change" (Section 4.1, Stage I).

State-of-the-Art Results Across Simulation and Real-World Benchmarks

Riemann-1.0 achieves 94.3% on RoboTwin2.0, 99.0% on LIBERO, and 62.6% on RoboCasa-365 (outperforming the previous best by 8.4 percentage points). On real-world long-horizon tasks, it achieves 85.0% SR and 94.4% PSR, exceeding the strongest open-source baseline (G0.5) by 15 percentage points in SR (Table 1). Notably, on kitchen organization—a complex long-horizon task—Riemann-1.0 achieves 90.0% SR while the next best baseline (G0.5) achieves only 35.0% (Table 1).

Massive Heterogeneous Data Infrastructure

The model is built on 230K+ hours of embodied experience: 200K+ hours of egocentric human video, 12K+ hours of handheld-gripper/wearable demonstrations, and 20K+ hours of heterogeneous robot trajectories (Section 2.1). The paper argues these sources are complementary, not redundant: "human videos provide scalable interaction knowledge, handheld-gripper demonstrations reduce the embodiment gap between humans and robots, and robot trajectories provide executable control supervision" (Section 2.1).

Strong Compositional and Out-of-Domain Generalization

On held-out tasks not seen during post-training, Riemann-1.0 achieves 65.0% SR on compositional generalization (seen objects, unseen task instructions) and 85.0% SR on out-of-distribution evaluation (unseen objects, unseen instructions), with an overall average of 75.0% across four held-out tasks (Table 2). This suggests the pretrained prior transfers beyond the fine-tuning distribution.

2. Contrarian Perspectives

Video-First World Models Are Architecturally Suboptimal

Most WAM approaches either jointly denoise actions and video (DreamZero), generate video first then infer actions (LingBot-VA), or decouple video and action generation into separate models (FastWAM). Riemann-1.0 argues all of these are flawed: "existing World Action Model formulations—including joint generation, video-first prediction, and decoupled action-video modeling—capture different aspects of robot interaction but fail to jointly model observations, robot states, actions, and world evolution within a unified causal process" (Section 1). The causal action-first ordering matters because it matches real robot interaction: "actions are executed before their visual consequences are observed" (Section 3.2). Video-first approaches also "introduce additional inference latency" (Section 3.1).

Robot Trajectories Alone Cannot Scale Embodied Intelligence

Many robotics companies focus exclusively on collecting robot demonstration data. Riemann-1.0 argues this is fundamentally limiting: "Scaling World Action Models fundamentally depends on scaling embodied experience rather than robot trajectories alone" (Section 2). The paper's data composition is 86% human video, 5% handheld-gripper, and 9% robot trajectories (Figure 1). The argument is that robot data is too expensive and narrow to serve as the primary scaling axis, while human video provides "the broadest diversity of real-world interactions, object-centric manipulation skills, and long-horizon task compositions" (Section 2.1).

You Don't Need Separate Systems for Policy Learning and World Simulation

The conventional approach in robotics separates policy models (fast action prediction) from world models (predicting future observations for planning or evaluation). Riemann-1.0 unifies these: "when deployed as a policy, the model predicts the next action chunk from causal history and uses the real environment observation as the next context; when used as a simulator, the same action chunk conditions the latent head to generate the corresponding future visual observation" (Section 6). This dual interface means companies don't need to maintain separate policy and simulation infrastructure.

3. Companies Identified

Riemann Dynamics

Description: The company that developed Riemann-1.0, listed as the sole institutional affiliation.
Why relevant: This is their flagship model. The paper represents a comprehensive system-level contribution spanning data infrastructure, model architecture, and training methodology. The domain "riemanndynamics.ai" and project website suggest an active commercial entity.
Quote: "Riemann-1.0 achieves state-of-the-art performance across both simulation benchmarks and long-horizon real-world manipulation tasks" (Abstract).

Physical Intelligence (π0.5)

Description: Developer of the π0.5 vision-language-action model, a major open-source baseline.
Why relevant: π0.5 is one of the primary competitors benchmarked across both simulation (RoboTwin2.0, LIBERO) and real-world tasks. It underperforms Riemann-1.0 significantly on real-world tasks (36.25% avg SR vs. 85.0%) and on RoboCasa-365 (not directly compared but implied through baseline rankings).
Quote: "On long-horizon real-world manipulation tasks, Riemann-1.0 achieves a Success Rate (SR) of 85.0%... exceeding the strongest open-source baseline by 15 percentage points in SR" (Abstract). Table 1 shows π0.5 at 36.25% average SR.

Galaxea AI (G0.5)

Description: Developer of G0.5, described as "one autoregressive stream for robot reasoning and action." Also provides the Galaxea Open-World Dataset.
Why relevant: G0.5 is the strongest real-world baseline (70.0% avg SR) and competitive on RoboTwin2.0 (93.3%) and LIBERO (98.9%). The 15-point SR gap on real-world tasks is the paper's headline result.
Quote: Table 1 shows G0.5 at 70.00% average SR vs. Riemann-1.0 at 85.00%.

AgiBot World Team

Description: Developer of "AgiBot World Colosseo," a large-scale manipulation platform and dataset.
Why relevant: Referenced as a robot trajectory data source in the related work. Represents the broader ecosystem of large-scale robot data providers that Riemann-1.0's data infrastructure is designed to incorporate.
Quote: Listed among robot trajectory datasets that "provide direct action-state supervision but are fragmented across embodiments, camera layouts, and control conventions" (Section 6).

4. People Identified

Haofeng Sun (Equal Contribution, First Author)

Lab/Institution: Riemann Dynamics
Why notable: Lead author and equal contributor on the paper. As a founding researcher at Riemann Dynamics, likely a key architect of both the model and the company's technical strategy.
Quote: Co-authored the full system description; no individual quote attributed.

Yang Liu (Project Lead and Corresponding Author)

Lab/Institution: Riemann Dynamics
Why notable: Listed as project lead and corresponding author (†), indicating primary responsibility for the research direction. In the Physical AI landscape, project leads at companies building foundation models are typically the CTO or VP of Research.
Quote: No individual quote; listed as corresponding author.

Yangguang Li (Project Lead and Corresponding Author)

Lab/Institution: Riemann Dynamics
Why notable: Co-project lead and corresponding author. The dual project lead structure suggests a collaborative leadership model for the research team.
Quote: No individual quote; listed as corresponding author.

5. Operating Insights

The Three-Stage Curriculum Is a Practical Recipe for Data-Efficient Deployment

The progressive pretraining recipe (λ=0.1 → 0.5 → 0.9) provides a concrete template for companies trying to leverage cheap human video data alongside expensive robot data. Stage 1 uses a frozen Latent Action Model to extract pseudo-actions from unlabeled video—this is the key mechanism for scaling beyond robot data. The action loss weight schedule is explicitly designed to "first teach broad visual dynamics with LAM-derived latent actions, then align the model with real multi-embodiment trajectories, and finally specialize it for robot execution" (Section 4, Figure 7). For a CTO, this means you can front-load pretraining on cheap human video and reserve expensive robot data for the final specialization stage.

Real-World Deployment Requires Only 3 Hours of Teleoperation Per Task

The post-training data collection is remarkably lean: 15 demonstrations (approximately 3 hours) per task across four tasks, aggregated into a single generalist model. The paper states: "For each real-world task, we collect three hours of teleoperated demonstrations. Instead of training task-specialized individual models, we aggregate all collected demonstration data to jointly fine-tune a single generalist model" (Section 4.2). This is a critical data-efficiency signal: the heavy lifting is done in pretraining, and post-training is lightweight. The action loss weight is increased to 0.95 in post-training "which improves action execution performance" (Section 4.2).

Multi-Embodiment Training Requires Careful Action Space Engineering

Riemann-1.0 handles heterogeneous robots by padding actions to fixed dimensions with validity masks, using embodiment-specific linear projections and prediction heads, and normalizing per-embodiment. Critically, "inactive padded regions are kept at zero for the clean sample, injected noise, noisy sample, and velocity target. This avoids a train-test mismatch in which padded channels would contain random Gaussian noise during training but zeros during deployment" (Section 3.2, Loss masking). This is a non-obvious implementation detail that would cause silent performance degradation if missed.

6. Overlooked Insights

The Latent Action Model Bootstrap Is the Key to Unlocking Unlabeled Video

The LAM architecture is described in detail but its strategic significance is buried. The LAM is trained as a VAE over adjacent video frames with a 32-dimensional latent action bottleneck, and the posterior mean (not a sample) is used as the deterministic pseudo-action. The paper notes: "The KL term is deliberately weak: its role is to regularize the latent distribution without collapsing the transition code into an overly generic prior" (Section 4.1, Stage I). This means the quality of pseudo-action supervision depends critically on the LAM's bottleneck design—too much regularization collapses the signal, too little produces noisy actions. Companies attempting to replicate this approach should treat LAM design as a first-class engineering problem, not an afterthought.

The Data Pipeline Is Six Stages of Heavy Engineering

Figure 2 reveals a six-stage data processing pipeline: visual preprocessing (fisheye correction, orientation), instruction annotation and action segmentation (hierarchical VLM-based), quality filtering (four-level semantic classification), 3D hand motion annotation (MANO-based reconstruction with VGGT camera pose estimation), geometric filtering, and semantic-aware data balancing. For egocentric human video alone, this involves hand detection, tracking, 3D reconstruction, camera pose estimation, and temporal smoothing. The paper notes that "directly combining these heterogeneous sources would result in inconsistent supervision and unstable optimization" (Section 2.2). The data infrastructure—not the model architecture—may be the true moat here, and it represents substantial engineering investment that is difficult to replicate quickly.