Carnegie Mellon University

“Shi is a rising figure in robot learning and control theory, with prior work on flow-based policies.”

“Simchowitz is known for theoretical work in online learning and control”

“Sreenath's lab focuses on control theory and robotics, particularly bipedal locomotion and manipulation.”

“Demonstrations are collected via teleoperation using a Meta VR headset.”

“JAMB simultaneously predicts the future 3D motion of points in the scene (3D point tracks). By generating both actions and future motion together in a shared neural network, they can correct and refine each other.”

“Across 16 simulation tasks, JAMB achieves an average success rate of 83.4%, outperforming the strongest baseline by 23.9 percentage points.”

“Wenkai Li2 wenkail@andrew.cmu.edu... 2 Carnegie Mellon University”

“allowing a larger reasoning budget generally yields marked gains, especially on group, line, V, iip, gris, iV, diag_line, rect, and iL”

“The paper's core contribution is demonstrating that a VLM can be trained to produce free-form natural language reasoning before issuing high-level instructions to a frozen low-level robot policy”

“Shreyas Kowshik - Lab/Institution: Carnegie Mellon University - Equal-contribution first authors”

“Max Simchowitz - Lab/Institution: Carnegie Mellon University (CMU Maxlab) - Co-senior author with equal advising”

“Aviral Kumar - Lab/Institution: Carnegie Mellon University (CMU AIRe Lab) - Co-senior author with equal advising”

“representations from the pretrained π0.5 backbone enable rapid improvement throughout training”

“On LIBERO-Long, MiDAS achieves 91.2% average success from one demo, compared to 33.5% for behavior cloning alone and 0% for zero-shot deployment of the base policy”

“To the best of our knowledge, this is the first demonstration of reliable robot policy adaptation from a single task demonstration”

“Sreyas Venkataraman - Lab/Institution: Carnegie Mellon University - Equal-contribution first authors”

“πR²: Reactive Real-time Flow Policies”

“The paper splits conditioning into 'a slow channel (vision and language features from the VLM and image/text encoders) and a fast channel (proprioception, fresh every tick)' (Section 3.2).”

“Train-Time RTC, the strongest baseline, commits to a stale plan and its middle-finger force overshoots to ~120 N, 'crushing the book,' while πR² 'modulates its grip from live force feedback and stops near ~50 N, gripping just enough to reorient the book' (Section 4.2, Reactivity analysis).”

“WEAVER achieves a Pearson correlation of ρ=0.870 between simulated and real-world success rates... finetuning on synthetic data closely matches that on real data, with only a 4% average performance gap”

“YW and AB were partially supported by the National Science Foundation (NSF) award [#2246447] and NSF CAREER award [#2441014]”

“Gokul Swamy, Carnegie Mellon University, Why notable: Senior co-author with broad expertise in imitation learning and robot policy optimization. Also co-author on the SAILOR paper (robust imitation via search). Swamy's lab appears to be a hub for work at the intersection of world models, imitation learning, and test-time planning.”

“Also first author on a 2025 RSS paper on VLM-in-the-loop policy steering via latent alignment — directly adjacent work on using learned representations for robot planning.”

“Tony Tao | Carnegie Mellon University | Equal-contribution first author, part of the CMU core team. Contributed to both FACTR and FACTR 2, indicating sustained focus on force-aware manipulation.”

“NEXT (Neural External Torque Estimation) trains a small LSTM on 10 minutes of free-motion data — no contact, no labels, no force sensor required — and achieves external torque estimation accuracy of 0.547 ± 0.348 Nm L1 error during contact on a Franka arm”

“achieves external torque estimation accuracy of 0.547 ± 0.348 Nm L1 error during contact on a Franka arm, compared to 4.395 Nm for FILIC and 1.471 Nm for a disturbance observer. That's an 87.6% error reduction vs. the next best learning-based method”

“We evaluate this capability on platforms spanning low-cost arms such as the Piper ($2,500) and high-end arms such as the Franka ($30,000)”

“Deepak Pathak | Carnegie Mellon University | Senior PI on the paper and a central figure in Physical AI at CMU. Pathak's lab has produced a consistent line of work on dexterous manipulation, LEAP Hand, and now force-aware policy learning.”

“Ruslan Salakhutdinov | Carnegie Mellon University | Co-author and CMU ML department head. His involvement signals institutional depth and connects this work to broader generative modeling and representation learning research threads feeding into physical AI.”

“Correspondence to: Jason Jingzhou Liu, liujason@cmu.edu”

“Steven Oh | Carnegie Mellon University / Waseda University | Equal-contribution first author with a joint appointment, bringing hardware and contact dynamics expertise from Waseda.”

“Kenneth Shaw | Carnegie Mellon University | Co-author, known for LEAP Hand development. His presence connects FACTR 2 to the dexterous manipulation hardware thread — LEGO assembly experiments use LEAP Hand V2.”

“Aaron Johnson — CMU robotics faculty with cross-institutional appointment at TU Munich's IAS. CMU's robotics program has strong industry pipeline to Boston Dynamics, Agility Robotics, and the Pittsburgh robotics corridor.”

“SONIC [23] scales to 100M frames with an MLP controller, yet MLP capacity saturates as data grows.”

“The software implementation required '9.2 minutes and 420 lines of code across 3 files, with zero human intervention during code generation.'”

“Guanya Shi holds concurrent appointments as an Assistant Professor at Carnegie Mellon University and as an Amazon Scholar.”

“NVIDIA's humanoid robot foundation model is the second VLA fine-tuned and deployed through RIO, specifically on the Unitree G1... GR00T achieves 95% success on humanoid pick-box with 150 demonstrations (Table III).”

“Total software integration required '9.2 minutes and 420 lines of code across 3 files, with zero human intervention during code generation.'”

“Using only 50 teleoperated demonstrations collected via RIO, π₀.₅ achieves 95% success on place-can and 92.5% on shirt folding (Table III).”

“RIO reaches 130.3 ms end-to-end observation-to-action latency versus 581.2 ms for LeRobot... Lower pipeline latency directly raises the effective control frequency, which is what makes the same stack viable across policy classes from the slower VLA rollouts to the high-frequency DP and RL policies.”

“CMU researchers open-sourced a middleware framework that cuts robot-to-robot porting time from weeks to under an hour, directly attacking the infrastructure tax that's slowing down every VLA deployment team in the field.”

“Arthur Fender Coelho Bucker — Carnegie Mellon University. CMU robotics researcher contributing to the core framework architecture. Worth tracking as a future founder or key hire in the robotics infrastructure space.”

“Pablo Ortega-Kral & Eliot Xing — Carnegie Mellon University (equal first authors). Lead architects of RIO. Their work sits at the intersection of systems engineering and robot learning.”

“Raunaq Bhirangi / Abhinav Gupta / Tess Hellebrekers (CMU/Meta) — Referenced for ReSkin [2], AnySkin [3], and 'All the Feels' [1] — competing/complementary tactile sensing platforms.”

“EmbodiedMidtrain applied to InternVL3.5-1B (1.1B parameters, 1.0M/4.1M/4.1M training samples) scores 3.714 average task length on Calvin ABC-D, surpassing both OpenVLA at 7.7B (2.548) and π₀ at 3.1B (3.509), while using training budgets roughly 6-25x smaller (Table 1).”

“Zhanqiu Guo — Language Technologies Institute, Carnegie Mellon University. Co-author on the core methodology.”

“Chenyan Xiong — Language Technologies Institute, Carnegie Mellon University. Senior author. Known for work on information retrieval and representation learning. His involvement signals that the proximity estimation methodology draws on well-established principles from information retrieval and data selection for LLM pretraining — the paper explicitly cites the DSIR framework (Xie et al., 2023) from Percy Liang's group at Stanford.”

“MMD distances are generally smaller within the VLM group and within the VLA group than across the two groups, quantitatively confirming a clear distributional mismatch (Section 3).”

“EmbodiedMidtrain inserts a lightweight 'alignment' step between VLM pretraining and VLA fine-tuning that costs a fraction of normal training compute but consistently delivers performance competitive with models 3–8x larger.”