N0-TWAM: Scaling… | arXiv Physical AI Research Summary

1. Key Themes

First Tactile-Native World-Action Model at Scale

N0-TWAM is the first world-action model that predicts future touch jointly with future vision, rather than treating tactile data as a side input. It was pre-trained on "tens of thousands hours of real-robot data… spanning six embodiments and 450 tasks" (§1). The model predicts three coupled streams — what the robot will see, what it will feel, and the action it will take — in a single forward pass. This is a meaningful departure from both VLA models (which regress actions from current observations with no future prediction) and existing world-action models (which predict only future video). The paper states: "What is missing is a model that predicts touch together with vision, under the same objective and at the same causal step, and a policy that reads its actions from that jointly predicted future" (§1).

Dual-Pathway Tactile Design: Predict and Observe

Touch plays two distinct roles in the architecture: (1) a predicted future tactile stream that the model generates as a foresight target alongside video, and (2) an observed current tactile reading that conditions action generation in real time. The paper explains: "the predicted tactile expert co-denoises future contact with future video, and the causal cascade lets the action expert read this committed tactile future before producing the next action… In parallel, the observed pathway injects the current force-space tactile representation directly into the action expert" (§4.3). Ablations confirm both pathways are load-bearing: removing predicted foresight drops UniVTAC success from 84.5% to 71.8%, and removing observed conditioning drops it to 70.5% (§4.5, Figure 9).

Asymmetric Architecture for Real-Time Inference at 7B Parameters

The Mixture-of-Transformers design keeps the video expert at full width (d=3072, ~5B params) while the action and tactile experts run slim (d=1024, ~1B each), totaling 7.16B parameters — roughly half of an all-full-width design (~15B). This is not just a training efficiency play: at inference, "once a chunk's video and tactile are predicted, their attention keys and values are cached, so each action-denoising step re-runs only the lightweight action expert rather than the full-width video expert" (§2.1). The paper also notes that FP8 or NVFP4 precision on linear layers "reduces the memory footprint and matrix-multiplication cost of the dominant video expert while having a negligible effect on the action output" (§2.3.2).

Tactile-Punctuated Long-Horizon Execution

The paper introduces a system where tactile contact events (grasp onset, release, slip, insertion seating) segment long-horizon demonstrations into sub-tasks during training and schedule execution stages at inference. "The predicted future tactile signal triggers the advance to the next sub-task, and the corresponding release or contact-onset event in the observed tactile stream then confirms it before the switch is committed" (§2.3.3). This addresses a real problem: "a single episode-level instruction does not tell the model which stage of the task the robot is currently in" and "the policy stalls, repeats a step, or skips ahead" (§2.3.3). The same tactile signal also verifies task success: "a gripper can close without loading any force, so a failed grasp shows an aperture dip but no tactile rise" (§2.3.3, Figure 5).

Strong Empirical Results on Contact-Rich Tasks

N0-TWAM achieves 84.5% on the UniVTAC simulation benchmark (17 points above the strongest baseline InternVLA-A1 at 67.1%), 49.4% on the in-house NeoSim suite (ahead of π0.5 at 45.8%), and 46.3% average across eight real-robot tasks (vs. 30.0% for π0.5, 21.9% for LingBot-VA, and 14.4% for FastWAM) (§4.3). The margin is largest on tasks where "success hinges on a hidden contact state a camera cannot resolve": Bottle Standing (70% vs. 20% for best baseline), Socket Plugging (70% vs. 60%), and Board Wiping (55% vs. 40%) (§4.3, Figure 7).

2. Contrarian Perspectives

Vision-Only World-Action Models Are Insufficient for Contact-Rich Tasks

The paper provides evidence that predicting future video alone — without touch — is not enough for contact-rich manipulation. On the UniVTAC benchmark, "the vision-only world-action baselines trail even the VLA policies here (LingBot-VA 31.4, FastWAM 48.0, GigaWorld-Policy 16.5): predicting the future scene is not enough for tasks defined by contact" (§4.3). This challenges the current trend in the field where major world-action model efforts (FastWAM, Metis, Efficient-WAM) focus on efficiency optimizations for video prediction but do not incorporate touch. The implication: companies building manipulation policies on video-only world models may hit a ceiling on tasks where the deciding variable is fingertip pressure, incipient slip, or contact seating.

Touch Should Be Predicted, Not Just Consumed as Input

Most tactile-aware policies in the literature treat touch as a reactive input channel — the policy reads current tactile readings and maps them to actions. N0-TWAM argues this is insufficient: "A policy that is to act well in these regimes therefore needs two things that pure visuomotor regression does not naturally provide: access to touch, and the ability to anticipate how the immediate future of an interaction will unfold rather than reacting only to the present frame" (§1). The paper identifies three partial paths in prior work — tactile policies that consume but never predict touch, separate frozen tactile predictors bolted onto policies, and touch-aware models that gate or mask tactile tokens — and argues all fail to make touch a first-class predicted modality (§1). The ablation evidence supports this: the predicted tactile pathway alone (without observed touch) still contributes meaningfully, and removing it degrades performance (§4.5).

Isolate Tactile Capacity in Weights, Not Attention

Concurrent touch-aware world-action models protect their visual stream by "gating or masking tactile tokens to protect the visual stream" (§1). N0-TWAM takes the opposite approach: each modality gets its own expert weights, but all experts share a single self-attention layer, so "vision and touch stay fully attentive to one another" (§1). The rationale: "tactile statistics are sparse and event-driven, and concurrent touch-aware world models report that injecting such tokens into a visual dynamics model degrades video and action prediction. Isolating touch at the weight level, rather than restricting it in attention, lets vision and touch stay mutually attentive while keeping separate capacity" (§2.2.1). This is a design philosophy disagreement with approaches that treat touch as a perturbation risk to the visual stream.

3. Companies Identified

NeoteAI (with Fudan TEAI Team)

Description: The organization behind N0-TWAM, NeoForce, NeoData, NeoSim, and NeoReal benchmarks. They have built a full stack: tactile sensors (InTac S1), force representations (NeoForce), large-scale datasets (NeoData, 30,000+ hours), simulation (NeoSim), and the world-action model itself.
Why relevant: This appears to be a serious, well-funded effort building a vertically integrated tactile robotics stack. The paper references their own data collection system, sensors, simulation suite, and benchmarks. The codebase and model checkpoints will be publicly released.
Quote: "We pre-train N0-TWAM at large scale with visuo-tactile joint training over tactile-rich demonstrations, spanning six embodiments and 450 tasks" (Abstract).

Physical Intelligence (π0.5)

Description: Creators of π0.5, a vision-language-action model with open-world generalization. Used as the strongest VLA baseline throughout the paper.
Why relevant: π0.5 is the primary competitor on real-robot tasks. It wins on placement-dominated tasks (Cup Stacking 50% vs. 45%, Bag Packing 20% vs. 15%) but loses significantly on contact-critical tasks. The paper positions N0-TWAM's tactile advantage as complementary to π0.5's semantic priors: "its large vision–language pretraining carries broad object and semantic priors, so it adapts to new instances best" on unseen objects (§4.4).
Quote: "π0.5 is the strongest baseline. As a large vision-language-action policy it brings broad semantic priors and dependable visually guided placement… but it regresses actions directly from the current frame, with no model of the next moment and no touch, so it can neither anticipate nor feel a contact event" (§4.3).

Shanghai Xinzhi Embodied Intelligence Technology Co., Ltd. / NeoteAI

Description: Manufacturer of the InTac S1 visuotactile sensor used in all real-robot experiments.
Why relevant: The hardware-software co-design is notable — the same organization produces the sensor, the force representation, the dataset, and the model. This vertical integration could be a competitive moat or a dependency risk depending on perspective.
Quote: "All real-robot experiments use the InTac S1 visuotactile sensor (Shanghai Xinzhi Embodied Intelligence Technology Co., Ltd. / NeoteAI)" (§4.2).

Google DeepMind

Description: Gemini 3.5 Flash is used to auto-generate sub-task instructions for segmented demonstration clips.
Why relevant: Shows that even a tactile-native robotics stack still relies on LLMs for language labeling of training data. The pipeline uses "Gemini 3.5 Flash to generate a short sub-task instruction for each clip, with human spot checks" (§2.3.3).

NVIDIA

Description: Training infrastructure provider. Pre-training runs on 128 NVIDIA H800 GPUs under FSDP2. NeoForce pre-training uses 8 NVIDIA A100 GPUs. The paper also references NVFP4 precision on Blackwell-class hardware for deployment.
Why relevant: The compute footprint is significant but not extreme — 128 H800s for 30,000 steps is accessible to well-funded startups. The Blackwell FP4 reference suggests they are already thinking about edge deployment cost optimization.
Quote: "Training runs on 128 NVIDIA H800 GPUs under FSDP2 with bf16 mixed precision" (§4.1).

Xiaomi

Description: Xiaomi-Robotics-0, an open-sourced VLA model with real-time execution, is used as a baseline.
Why relevant: Appears as a competitor in the VLA space, evaluated on UniVTAC (54.3% average) and NeoSim (23.4% average).
Quote: Table 2 and Table 3 show Xiaomi-Robotics-0 results.

Wan2.2 (referenced as video VAE backbone)

Description: The Wan2.2-TI2V-5B video-diffusion transformer family provides the pretrained video expert that N0-TWAM's backbone is warm-started from (via LingBot-VA).
Why relevant: Demonstrates that the tactile world model is built on top of an existing video generation foundation, not trained from scratch. The video expert inherits "a strong visual and dynamics prior" (§2.1).

4. People Identified

Yiran Qin

Lab/Institution: NeoteAI / Fudan TEAI
Why notable: Project Lead. Also listed in Pre-Training contributors. Appears to be one of the senior figures driving the N0-TWAM program and the broader N0-Foundation effort.
Quote: Listed under "Project Lead" and "Pre-Training" in the Contributors section.

Shunlin Lu

Lab/Institution: NeoteAI / Fudan TEAI
Why notable: Project Lead and Pre-Training contributor. The N0-Foundation reference [56] is attributed to the NeoteAI Team and Fudan TEAI Team, suggesting this group has been building the tactile intelligence stack (NeoForce, NeoData, NeoSim, NeoReal) as a coordinated program.
Quote: Listed under "Project Lead" and "Pre-Training."

Zuxuan Wu

Lab/Institution: Fudan University (Fudan TEAI Team)
Why notable: Project Lead, representing the academic side of the collaboration. The Fudan TEAI (Tactile Embodied AI) lab appears to be a significant academic group in tactile robotics.
Quote: Listed under "Project Lead."

Li Kang

Lab/Institution: NeoteAI / Fudan TEAI
Why notable: The most ubiquitous contributor — listed in Pre-Training, Post-Training (Simulation), Post-Training (Real Robot), and Data Processing. Likely the technical lead implementing the core system across all stages.
Quote: Appears in four contributor categories.

Shihao Zhao, Daoguo Dong

Lab/Institution: NeoteAI
Why notable: Project Leads, likely representing the industry/commercial side of the collaboration.
Quote: Listed under "Project Lead."

5. Operating Insights

Tactile Is Not Optional for Contact-Rich Manipulation — and the Data Requirements Are Substantial

The paper makes a clear case that if your robot needs to seat a screw, peel a cup off a stack, or close a gripper on a soft object, vision-only policies will hit a wall. The pre-training corpus required 30,000+ hours of real-robot data with synchronized per-finger tactile across six embodiments and 450 tasks (§3.1). The ablation shows pre-training scale is "the largest single factor": reducing to 20% of the corpus drops UniVTAC success from 84.5% to 65.4% (§4.5, Figure 9). For a CTO evaluating whether to invest in tactile sensing: the payoff is real (46.3% vs. 30.0% real-robot success over the strongest VLA), but the data collection infrastructure is a moat — you need synchronized tactile streams at scale, and most existing robot datasets do not have them.

Graceful Degradation via Tactile-Condition Dropout

The training pipeline randomly drops tactile conditioning with probability 0.1 by simply not passing any tactile tensor, which means "the model learns the marginal, vision-only policy alongside the tactile-conditional one, so if a sensor fails or is absent at deployment the policy degrades to its vision-only self instead of reacting to an out-of-distribution zero input" (§4.1). This is a practical deployment insight: the same model can run on robots with or without tactile sensors, and the generalization results confirm it — under visual perturbation (lighting/background changes), N0-TWAM holds at 45% success while π0.5 drops to 25% and LingBot-VA to 30% (§4.4, Table 4). The tactile pathway also exposes a classifier-free-guidance knob at inference to amplify touch sensitivity, though the paper leaves this at w=1 in all reported results.

Delta vs. Absolute Action Representation Is a Task-Specific Decision

The default unified action space uses chunk-anchored delta end-effector poses for cross-embodiment transfer, but the paper found that "on alignment-sensitive tasks the absolute end-effector pose can outperform the delta target… a delta is measured from the moving chunk-start anchor, so small per-chunk errors accumulate and the target is never tied to a fixed location" (§4.6). On four alignment-sensitive tasks, absolute poses achieved 82.5% average vs. 50.0% for delta (Table 5). The practical guidance: use delta for pre-training and cross-embodiment generalization, switch to absolute for task-specific post-training when sub-centimeter placement matters.

6. Overlooked Insights

Sim-to-Real Tactile Transfer Works When the Rendering Looks Real

A buried finding in §4.6 has significant implications for sim-to-real transfer of tactile policies. The paper created a more realistic gel-rendered tactile image (silvery, speckled, with sensor noise) to replace the simulator's clean color-blob output. When this realistic rendering was fed through NeoForce — a force encoder pretrained on real tactile sensors — success on UniVTAC rose from 82.4% to 88.1%, and on single-arm NeoSim from 58.5% to 64.8% (Table 6). The paper states: "once the simulated touch looks real, a force encoder pretrained on real touch transfers into it" (§4.6). This means the gap between simulated and real tactile sensing is not fundamental — it is a rendering problem. Companies investing in tactile simulation should prioritize photorealistic gel rendering over more accurate contact physics.

Staged Training Prevents Tactile Shortcut Learning

The decision to disable the observed tactile pathway during pre-training is a subtle but important design choice. The paper explains: "giving the policy the true current contact during pre-training would let it rely on that signal and under-train the foresight it is meant to complement" (§2.2.2). The observed pathway is only activated during post-training, where it is "grafted onto a pretrained model without perturbing it" through zero-initialized cross-attention (Eq. 7). This means the model first learns to predict future contact from large-scale data (building a contact dynamics prior), and only then learns to react to current contact. A company training tactile policies from scratch with both pathways enabled simultaneously would likely produce a model that shortcuts to reactive touch and never develops predictive contact reasoning — a failure mode that would be invisible in benchmarks but costly in deployment.