Humanoid Everyday: A… | arXiv Physical AI Research Summary

Bottom Line Up Front: This paper is a dataset and infrastructure paper, not a breakthrough in robot capabilities. But it matters strategically: it establishes the first large-scale, multimodal, task-diverse benchmark for full-body humanoid manipulation, exposes hard failure modes in every major policy architecture, and provides a cloud evaluation platform that could become the de facto standard for comparing humanoid policies. For investors and builders, the key signal is in what didn't work — and why.

1. Key Themes

The Humanoid Data Gap Is Real and Structurally Different From Arm Robot Data

Every major dataset in robot learning — Open X-Embodiment, BridgeData, DROID — was built around stationary robot arms. This paper argues that humanoid data is categorically different: it requires full-body coordination across 28 degrees of freedom, locomotion-integrated manipulation, and human-robot interaction in uncontrolled environments. The authors collected 10,300 trajectories across 260 tasks and 7 categories (Basic Manipulation, Deformable, Articulated, Tool Use, High-Precision, Human-Robot Interaction, Loco-Manipulation), spanning indoor and outdoor environments with multimodal sensing including RGB, depth, LiDAR, tactile, and IMU.

"Existing datasets mainly target stationary arms or mobile platforms with simple grippers and wheeled bases; even egocentric efforts like Humanoid Policy emphasize repetitive tasks with limited locomotion." (Section I)

This isn't just a gap in volume — it's a gap in task topology. Operators building humanoid fleets for logistics, healthcare, or retail can't rely on arm-robot datasets to pre-train their policies.

Every Major Policy Architecture Fails Badly on Full-Body Humanoid Tasks

The benchmark results are the most operationally important finding in the paper. Seven policies — Diffusion Policy, 3D Diffusion Policy, ACT, OpenVLA, π₀-FAST, π₀.5, and GR00T N1.5 — were evaluated across all seven task categories with 10 trials each. The best-performing model, NVIDIA's GR00T N1.5, achieved only 51% average success. On the hardest tasks, results were near zero across the board.

"All end-to-end imitation policies struggle in humanoid manipulation tasks due to the high-dimensional action space in our dataset (which is 28 DoFs in total)." (Section V-A)

"In the 'Insert rose into vase' task, nearly all policies achieve a 0% success rate: while many can lift the rose, they consistently fail to insert its thin stem into the vase, suggesting that current models lack fine-grained visuospatial perception." (Section V-A)

For anyone evaluating "is this policy ready for deployment," these results are a cold shower — and a calibration tool.

Humanoid-Specific Pretraining Provides Measurable Lift for VLA Models

The paper tests whether pretraining on Humanoid Everyday before task-specific fine-tuning improves performance over direct fine-tuning. The answer is yes, consistently across VLA models. This matters because it validates the "foundation model + humanoid pretraining + task-specific fine-tune" pipeline that companies like Physical Intelligence, NVIDIA, and Fourier are betting on.

"Finetuning on Humanoid Everyday prior to task-specific adaptation consistently improves the performance of VLA models. This suggests that exposure to diverse humanoid behaviors provides a useful prior that facilitates learning downstream manipulation tasks." (Section V-B)

The implication: proprietary humanoid datasets are a durable moat. Companies that collect diverse, high-quality humanoid data early will compound that advantage through better pretraining priors.

A Cloud Evaluation Platform for Real Humanoid Hardware Is a New Infrastructure Primitive

The paper introduces what they claim is the first cloud-based policy evaluation system that executes policies on real physical humanoid robots — not simulators. Researchers upload their policy, specify a server IP/port, and receive real-time sensory data back for inference. The system ran continuously for 100+ minutes with only three human interventions.

"Unlike existing cloud evaluation systems that focus on robotic arms or simulated agents, ours is the first platform designed for humanoid, which aims to lower the barrier to fair comparison and fosters collaborative progress in humanoid robotics." (Section I)

This is the infrastructure play that could matter long-term — similar to how MLCommons benchmarks shaped LLM development.

2. Contrarian Perspectives

Larger Models Don't Solve the Core Problem — Action Space Dimensionality Does

Conventional wisdom in robotics AI currently is "scale the model, scale the data, results follow." This paper pushes back with specific failure modes. OpenVLA, a large VLA, often "fails to produce meaningful motions" at 30Hz because it doesn't compress the action space. π₀-FAST's DCT tokenizer breaks down on 28-DoF humanoid actions, producing "excessive token numbers, incorrect output tokens, and decoding errors that sometimes cause the robot to halt during inference."

"OpenVLA does not compress the action space and thus when trained on high-frequency 30 Hz data, it often fails to produce meaningful motions... π₀-FAST employs a DCT-based tokenizer for action compression, but the high-dimensional humanoid actions are not well represented." (Section V-A)

The contrarian implication: the architecture choices that work for 6-DoF arms may be fundamentally ill-suited for humanoids. Companies deploying large VLAs on humanoids without rethinking action representation may be building on sand.

3D Point Cloud Inputs Hurt Performance in the Tasks That Matter Most

There is strong enthusiasm in robotics research for 3D representations — depth cameras, point clouds, and spatial embeddings are widely considered a step up from RGB-only inputs. This paper shows that for loco-manipulation tasks — arguably the highest-value humanoid use case — 3D inputs are worse than RGB.

"In Loco-Manipulation tasks where the robot must move and the point cloud undergoes large frame-to-frame changes, 3D-based inputs become less reliable than RGB images, leading to failure cases." (Section V-A)

This challenges teams investing heavily in spatial AI perception stacks for mobile humanoids. The signal isn't "3D is bad" — it's "3D is brittle when the robot's own motion contaminates the scene representation," which is exactly the regime humanoids operate in most of the time.

Teleoperation System Design Is a Competitive Variable, Not a Commodity

Most teams treat the teleoperation pipeline as plumbing. This paper treats it as a first-class engineering problem and quantifies the results: their redesigned multi-processing pipeline reduced control delay from 500ms to 2ms and halved data collection time compared to the official Unitree teleoperation system.

"Our pipeline halves data collection time compared to the official Unitree teleoperation system, while the control delay decreases from 500 ms to 2 ms." (Section III-B)

At scale, a 2x improvement in collection throughput with near-zero latency fundamentally changes the economics of humanoid data collection. Teams treating teleoperation as off-the-shelf infrastructure are leaving significant efficiency on the table.

3. Companies Identified

Unitree Robotics

Description: Chinese humanoid robot manufacturer
Why relevant: Both hardware platforms used in the dataset (G1 and H1 robots) are Unitree products. The paper's custom teleoperation pipeline was built as a re-engineering of Unitree's official teleoperation library.
Quote: "We gather data with two Unitree humanoid robots: the 29-degrees of freedom G1 with 7-DoF three-fingered dexterous hands (Dex3-1) and the 27-DoF H1 with 6-DoF INSPIRE hands." (Section III-A)

NVIDIA (GR00T N1.5)

Description: Semiconductor and AI company; developer of GR00T humanoid foundation model
Why relevant: GR00T N1.5 was the top-performing policy in the benchmark at 51% average success rate, directly validating NVIDIA's strategy of large-scale humanoid pretraining.
Quote: "GR00T N1.5 achieves the strongest performance overall, largely due to its extensive pretraining across multiple large-scale humanoid datasets, which provides a powerful prior well-suited for our diverse manipulation tasks." (Section V-A)

Physical Intelligence (π₀ and π₀.5)

Description: Robotics AI startup focused on foundation models for dexterous manipulation
Why relevant: Both π₀-FAST and π₀.5 were evaluated as benchmark policies. π₀.5 showed smoother humanoid actions but still overfitted to trajectories; π₀-FAST showed architectural failures under 28-DoF action spaces.
Quote: "Although π₀.5 produces smoother humanoid actions compared to the previous two models, it still tends to overfit to trajectories, ignoring visual feedback." (Section V-A)

Toyota Research Institute

Description: R&D arm of Toyota Motor Corporation focused on robotics and AI
Why relevant: TRI is listed as a funder and institutional collaborator (two TRI researchers are co-authors), indicating strategic interest in humanoid manipulation datasets.
Quote: "The USC Geometry, Vision, and Learning Lab acknowledges generous supports from Toyota Research Institute." (Section VII)

Fourier Intelligence (ActionNet)

Description: Chinese humanoid robot company
Why relevant: Referenced as a comparable humanoid dataset effort (ActionNet); represents the competitive landscape for humanoid data collection at scale.
Quote: "Fourier ActionNet Team, 'Actionnet: A dataset for dexterous bimanual manipulation,' 2025." (Reference [8])

Apple

Description: Consumer technology company
Why relevant: Apple Vision Pro is the teleoperation interface used for capturing wrist and finger keypoints in data collection — a non-obvious deployment of consumer AR hardware as a robotics data collection tool.
Quote: "The operator wears an Apple Vision Pro to capture wrist and finger keypoints using its cameras on the bottom." (Section III-A)

AgiBot

Description: Chinese robotics company
Why relevant: AgiBot World Colosseo is cited as a comparable real-world humanoid data collection platform, representing the competitive landscape.
Quote: Reference [9]: "Agibot World Colosseo: A large-scale manipulation platform for scalable and intelligent embodied systems." (Reference [9])

4. People Identified

Yue Wang

Lab/Institution: USC Geometry, Vision, and Learning Lab (University of Southern California)
Why notable: Co-senior advisor on this paper; PI of the USC GVL Lab. Active in 3D perception and robot learning; also co-authors on related humanoid pose control work.
Quote: "Yue Wang is also supported by a Powell Research Award." (Section VII)

Jiageng Mao

Lab/Institution: USC GVL Lab / University of Southern California
Why notable: Co-senior advisor; previously worked on universal humanoid pose control (cited as reference [4]). Bridging perception and humanoid control.
Quote: Equal advising credit alongside Yue Wang. (Author list)

Zhenyu Zhao and Hongyi Jing

Lab/Institution: USC GVL Lab
Why notable: Equal first authors; led the data collection pipeline engineering and policy evaluation experiments.
Quote: "Equal contribution." (Author list)

Sergey Zakharov and Vitor Guizilini

Lab/Institution: Toyota Research Institute
Why notable: TRI researchers embedded in an academic humanoid dataset paper — signals TRI's interest in using large-scale humanoid data for downstream robotics development.
Quote: Listed as co-authors from Toyota Research Institute. (Author list)

5. Operating Insights

Don't Deploy VLA Models on Humanoids Without Solving Action Representation First

The benchmark reveals a specific, actionable failure mode: VLA models trained on high-frequency (30Hz) humanoid data with 28-DoF action spaces either produce meaningless motions (OpenVLA), generate decoding errors mid-execution (π₀-FAST), or overfit to demonstrated trajectories without adapting to visual feedback (π₀.5, ACT). Teams evaluating VLA models for humanoid deployment should stress-test action representation — specifically, how the model handles high-dimensional, high-frequency action sequences — before committing to an architecture.

"OpenVLA does not compress the action space and thus when trained on high-frequency 30 Hz data, it often fails to produce meaningful motions. Downsampling to 2 Hz alleviates this issue, but the resulting behaviors appear less smooth." (Section V-A)

The practical implication: if you're running Unitree G1/H1 or similar 20+ DoF platforms, verify your policy's action tokenization strategy handles the full DoF count without degradation. This is a pre-deployment checklist item, not just a research question.

Treat Teleoperation Infrastructure as a Data Quality Problem, Not a Logistics Problem

500ms control latency in the default Unitree teleoperation stack means the operator is flying blind on a half-second delay — systematically degrading data quality and operator throughput. The USC team cut this to 2ms through multi-process architecture with shared memory buffers, halving total collection time. For any team running humanoid data collection at scale, the ROI on engineering the teleoperation stack is enormous.

"Our pipeline halves data collection time compared to the official Unitree teleoperation system, while the control delay decreases from 500 ms to 2 ms." (Section III-B)

This isn't just about efficiency. Low-latency teleoperation produces qualitatively better demonstrations — smoother trajectories, more accurate insertion tasks, cleaner bimanual coordination. Garbage-in-garbage-out applies here: bad teleoperation infrastructure produces bad training data regardless of model scale.

6. Overlooked Insights

The Cloud Evaluation Platform's Biggest Limitation Reveals the Industry's Hardest Unsolved Problem

Buried in the Discussion section is an admission that undercuts the platform's current utility: the evaluation system does not support automatic scene resetting. Every failed trial requires a human to manually reset the environment because "current imitation learning policies are not sufficiently robust for humanoids to recover the environments without human assistance."

"Our cloud-based evaluation system does not yet support automatic scene resetting, as current imitation learning policies are not sufficiently robust for humanoids to recover the environments without human assistance." (Section VI)

This is not a minor inconvenience — it's a direct indicator of where the field actually stands on robustness. The inability to autonomously reset after failure means current policies have near-zero fault recovery capability. For operators planning humanoid deployments in warehouses, hospitals, or homes, fault recovery and graceful degradation need to be first-class design requirements, not afterthoughts. Any company claiming "production-ready" humanoid manipulation should be asked directly: what is your scene recovery rate after policy failure?

Tactile Data Is Collected But Not Evaluated — That's a Signal Worth Tracking

The dataset includes tactile sensor data from the G1 robot's Dex3-1 hands, but none of the seven evaluated policies in the benchmark appear to use tactile inputs. This is an unclaimed opportunity: the dataset infrastructure for tactile-aware policy learning exists and is publicly available, but no policy in the benchmark leverages it.

"The G1's Dex3-1 hands possess tactile sensors, which further enhances the multimodality of our dataset." (Section III-A); "Each episode includes RGB video, depth maps, LiDAR, tactile feedback, and natural language task descriptions." (Section III-C)

For teams working on contact-rich manipulation — insertion tasks, deformable object handling, handover — this dataset may be the only publicly available large-scale humanoid tactile dataset. The 0% success rate on the vase insertion task across all policies, combined with available tactile data that no policy is using, points directly at where the next performance breakthrough likely lives.