TourPhysics: Bringing… | arXiv Physical AI Research Summary

1. Key Themes

Physics Simulator as the Source of Truth, Video Generator as the Renderer

TourPhysics fundamentally restructures how video world models work: instead of letting a learned video model implicitly predict physics, a deterministic simulator computes the complete physical and camera trajectory first, and only then does the video generator render an observation of that fixed trajectory. The paper states: "For each action, the simulator computes a finite physical and camera trajectory before the corresponding observation is generated" (Abstract). This means the video model can never alter the physical state — it only supplies appearance. The result: TourPhysics achieves camera trajectory adherence (ATE) of 0.0233 vs. 0.0560 for Sora-2 and 0.1031 for LingBot-Cam, and object motion ADE of 0.0297 vs. 0.0721 for Sora-2 (Table 2). For a robotics company, this means the generated visual predictions are actually constrained by real physics, not just plausible-looking motion.

Persistent Stateful World Model from a Single Image

Unlike most video generators that produce a fixed-length clip and forget everything, TourPhysics maintains an explicit world state across multiple action-observation cycles. The paper describes: "TourPhysics maintains an explicit world state across multiple action–observation cycles. Each cycle consists of a simulator-defined physical and camera trajectory followed by observation synthesis over that fixed trajectory segment" (Section 1). This is initialized from a single RGB image plus a declarative configuration specifying objects, materials, forces, and camera actions. The system supports rigid bodies, cloth, elastic, granular, and fluid materials through RBD, MPM, and PBD solvers (Figure 3). This is architecturally closer to how a robot actually operates — perceiving a scene, forming a hypothesis, acting, and updating its world model.

Quality-Gated Atomic Commits Prevent Hallucinated State Corruption

TourPhysics implements a transactional commit mechanism: each generated observation window must pass a quality gate before its terminal state and appearance evidence are published to the persistent world. The paper specifies: "A quality gate either commits the terminal state and the appearance evidence together or leaves the persistent state unchanged. Generated observations may extend appearance knowledge, but they cannot modify the committed dynamics or the simulator geometry" (Section 1). The gate checks for black frames, frozen motion, boundary discontinuities, and media contract compliance (Section 4.6, Eq. 21). If it fails, the candidate is discarded entirely and the system retries with a new sampling seed while keeping the same physical trajectory. This is a database-style atomicity guarantee applied to world model state — a design pattern directly relevant to any system where visual hallucinations must not corrupt physical reasoning.

Long-Horizon Appearance Consistency via Geometry-Routed Memory

The paper addresses a critical failure mode: when a camera revisits a surface after many intervening actions, finite temporal caches lose the original appearance. TourPhysics uses a "reference-anchored memory" that retrieves accepted static appearance through geometric cross-view correspondence. The mechanism is a bounded residual: "When no valid correspondence exists, the model follows its native reference-conditioned path" (Abstract). Table 4 shows the persistent sink cache reduces end-to-start drift from 26.33 to 11.17, and appearance memory reduces historical-region error by 8.25% on development identities and 2.39% on held-out identities. The memory only stores static, high-confidence regions — moving or deforming objects are excluded (Section 4.5, Eq. 14).

2. Contrarian Perspectives

Generated Pixels Should Never Write Back to Physical State

Most interactive world models (Genie, Oasis, GameGen-X, LingBot) let the video generator's output implicitly define the next state. TourPhysics explicitly rejects this: "Generated observations may extend appearance knowledge, but they cannot modify the committed dynamics or the simulator geometry" (Section 1). The paper argues this is the root cause of why current video world models "lose physical or spatial consistency over long horizons" — because "generated appearance is allowed to stand in for geometric evidence or physical state" (Section 1). Most robotics companies building learned simulators or world models would disagree with this strict separation, as it limits the video model's role to pure rendering and requires a separate physics engine.

A Single Image Plus a Declarative Configuration Is Sufficient for a Controllable Physical World

The paper argues that rather than requiring multi-view capture, depth sensors, or scene scanning, a single RGB image combined with a user-provided (or VLM-proposed) declarative configuration is enough to initialize a persistent, manipulable physical scene. The configuration specifies "controllable objects, material families, boundary conditions, interventions, and camera actions" (Section 1). The paper acknowledges this is underdetermined — "An image constrains visible appearance but leaves occluded geometry, scale, material parameters, and future dynamics underdetermined" (Section 1) — but argues the declarative configuration resolves enough ambiguity to build "one persistent, controllable physical hypothesis." This challenges the assumption that high-fidelity physical simulation requires dense 3D scanning.

Simulator RGB Should Not Replace Generated Pixels

Even though the simulator produces rendered frames, TourPhysics explicitly refuses to use them as direct pixel output: "Simulator-derived signals may provide conditioning, geometric, and evaluation information, but simulator RGB is not used as a direct substitute for generated pixels" (Section 3.2). The paper warns: "This distinction prevents a successful-looking simulator frame from silently becoming the generated observation and keeps the observation branch from acquiring physical authority" (Section 4.5). This is contrarian because many hybrid systems blend simulator renderings with generated content — TourPhysics argues this blurs the ownership of state and creates a path for visual artifacts to corrupt the physical model.

3. Companies Identified

OpenAI, Creator of Sora-2, used as a baseline video generator. Sora-2 achieves camera ATE of 0.0560 and object ADE of 0.0721, but TourPhysics outperforms it significantly (0.0233 and 0.0297 respectively, per Table 2). Sora-2's native duration is 12.2 seconds vs. TourPhysics's 25.1 seconds (Table 1). Relevant as a benchmark for what pure learned video priors can achieve without explicit physics.

Google DeepMind, Creator of Genie 2, referenced as an interactive world model that "synthesize[s] observations conditioned on learned actions or compact state representations" (Section 2.1). The paper notes such systems "generally do not distinguish observation from intervention within their state updates" — a direct critique of DeepMind's approach.

Decart, Creator of Oasis, referenced as an interactive world model (Section 2.1, ref [13]). Same critique applies: learned world models without explicit physics state ownership.

Robbyant Team (LingBot), Creators of LingBot-Cam and LingBot-Act, open-source interactive world model variants used as primary baselines. LingBot-Cam achieves camera ATE of 0.1031 and object ADE of 0.0833; LingBot-Act achieves 0.1235 and 0.0834 (Table 2). Both are significantly outperformed by TourPhysics.

Team Wan (Alibaba), Creators of Wan2.1-I2V, a general image-to-video generator used as baseline. Achieves camera ATE of 0.1436 and object ADE of 0.0774 (Table 2). Native duration only 10.7 seconds (Table 1).

minWM Team, Creators of minWM-Wan, an interactive video world model framework. Achieves camera ATE of 0.1092 and object ADE of 0.1051 (Table 2). Native duration 39.8 seconds but with weakest object tracking (S@0.05 of only 9.46%).

Genesis Authors, Creators of the Genesis physics engine, which TourPhysics uses as its GPU-accelerated multi-physics backend (Section A.2, ref [79]). Genesis handles RBD, MPM, and PBD simulation. This is a critical infrastructure dependency — TourPhysics's physical fidelity depends on Genesis's solver quality.

Meta (SAM 3 / SAM 3D), Creators of SAM 3 and SAM 3D Objects, used for instance segmentation and 3D mesh lifting from the single input image (Section 4.2). These are upstream dependencies for scene initialization.

4. People Identified

Xin Zhang, Fudan University / TeleAI intern, Co-first author. Developed the system during an internship at China Telecom's Institute of Artificial Intelligence. Corresponding author mark indicates leadership role.

Yabo Chen, TeleAI (Institute of Artificial Intelligence, China Telecom), Co-first author. Equal contribution with Zhang. Based at China Telecom's AI research institute, indicating significant corporate investment in physical AI from a major telecommunications company.

Xuelong Li, TeleAI (Institute of Artificial Intelligence, China Telecom), Corresponding author. A prominent figure in computer vision; his lab is producing a stream of physical AI work including PhysOmni (ACM Multimedia 2026), TeleWorld, and related world model papers. The lab appears to be building a comprehensive physical AI stack from scene understanding to interactive world models.

Feng Xu, Fudan University, Corresponding author. Represents the academic partnership side of this industry-academia collaboration.

Haibin Huang, TeleAI, Co-author. Likely involved in the video generation and appearance memory components given the system's emphasis on visual synthesis quality.

Chi Zhang, TeleAI, Co-author. The TeleAI group is systematically publishing in the physical AI space — this paper extends their PhysOmni work, and related publications include TeleWorld (a 4D world model) and several autoregressive video generation papers.

5. Operating Insights

Separate Physical State from Visual Appearance in Your Architecture

The single most actionable architectural pattern in this paper is the strict separation between simulator state (which owns physics, geometry, and trajectory) and the video generator (which only renders appearance). The paper demonstrates that when generated pixels cannot write back to physical state, you get dramatically better trajectory adherence: 98.26% correct translation direction vs. 89.58% for the next best (minWM-Wan), and 93% correct rotation direction vs. 61% for LingBot-Act (Table 2). For any team building a world model for robotics, this means your physics engine should compute the full trajectory before your visual model touches it, and the visual model's output should never be allowed to modify the physical state — only extend appearance knowledge through a gated commit.

Use a Declarative Configuration to Resolve Single-Image Ambiguity

The paper shows that a single RGB image is sufficient to initialize a manipulable physical scene if you pair it with a declarative configuration specifying materials, forces, and boundary conditions. The system uses Qwen2.5-VL-72B to optionally propose this configuration from the image and object crops, then validates it against a material/force whitelist and freezes it before simulation (Section A.2). For a robotics company, this means you can bootstrap a physical world model from a single camera frame plus a structured text description — no multi-view scanning required. The trade-off is that the physical hypothesis is only as good as the configuration, and occluded geometry remains underdetermined.

Quality Gates on Generated Observations Are Essential for Long-Horizon Reliability

The quality gate is simple but effective: it checks for black frames (≤10% of pixels), frozen motion (≤25% of adjacent frame changes below 0.2), boundary discontinuities (overlap error ≤4.0, boundary-motion ratio ≤3.0), and media contract compliance (exactly 81 decodable frames at 832×480) (Section 4.6, Eq. 21). These are cheap, deterministic checks on decoded 8-bit RGB — no learned quality model required. The gate prevents a single bad generation from corrupting the persistent state, which would compound over long horizons. Any team building autonomous systems with learned perception should consider similar transactional commit patterns for their world state.

6. Overlooked Insights

The Depth Representation Split Has Subtle but Critical Implications

TourPhysics maintains three separate depth representations per frame: simulator-consistent metric depth (Dgeo), normalized base depth for the generator (Dbase), and fused generator-facing depth (Dgen) that incorporates accepted tail evidence (Section 4.4, Eq. 11-12). The eligibility test for depth completion is deliberately fail-closed with strict thresholds: trusted overlap ≥0.20, Spearman agreement ≥0.70, model-space NRMSE ≤0.08, metric-space NRMSE ≤0.12 (Section 4.4). When any test fails, the weight is set to zero and the base control is preserved exactly. This means the system degrades gracefully — it never uses uncertain depth to condition generation. For deployment, this implies that in novel scenes where depth estimation is unreliable, the system falls back to simulator-only geometry rather than risking hallucinated depth, which is the safer failure mode for physical interaction.

The VLM Only Participates in Initialization, Never Online

The paper mentions that Qwen2.5-VL-72B-Instruct is used to propose the declarative configuration from the image and masked crops, but critically: "The resulting declaration is frozen before the first trajectory simulation, so the VLM participates only in initialization" (Section A.2). This is a deliberate design choice — the VLM proposes materials and forces, but once frozen, the physics engine has full authority. This means the system's physical reliability does not depend on VLM reasoning during execution, only on the simulator. The VLM's errors are bounded to the initialization phase and can be manually corrected before the first action. This is an important separation for safety-critical applications where you cannot afford runtime language model hallucinations to affect physical predictions.