FACTR 2: Learning… | arXiv Physical AI Research Summary

Why Should You Care?

Force sensing has been a silent gating factor for contact-rich manipulation at scale. Tasks like assembly, insertion, screwing, and deformable object handling require force feedback — but deploying it has meant either buying expensive arms (Franka at ~$30K) or retrofitting hardware that can cost more than the arm itself. This paper removes that constraint entirely with a software-only solution that takes 10 minutes of data and 1 minute to train.

1. Key Themes

Force Sensing Is Now a Software Problem, Not a Hardware Problem

The central claim: you no longer need dedicated force-torque sensors to get force feedback quality comparable to sensor-equipped platforms. NEXT (Neural External Torque Estimation) trains a small LSTM on 10 minutes of free-motion data — no contact, no labels, no force sensor required — and achieves external torque estimation accuracy of 0.547 ± 0.348 Nm L1 error during contact on a Franka arm, compared to 4.395 Nm for FILIC and 1.471 Nm for a disturbance observer. That's an 87.6% error reduction vs. the next best learning-based method (Section 7, Table 1). Critically, in free-space, NEXT is quieter than the Franka's own dedicated sensor (0.414 vs. 0.449 Nm L1), because it learns hardware-specific noise characteristics directly from the robot itself.

Contact-Rich Policy Failures Are Concentrated in a Predictable, Addressable Window

The paper surfaces a non-obvious insight about where policies actually fail: not during free-space motion, but in the 1-second pre-contact window — the moment before the robot touches anything. "Policy failures concentrate not in free-space motion, but in the brief pre-contact intervals requiring precise alignment as well as in contact phases requiring controlled interaction and fine corrections" (Section 1). FIRST exploits this by simply over-sampling those trajectory segments during behavior cloning training. Across five long-horizon tasks, upsampling pre-contact data alone (5× weight relative to free-motion) achieves average task progress of 0.818 vs. 0.670 for upsampling contact data alone (Table 2) — meaning the moment before touch matters more than the touch itself.

A 17%+ Performance Gain from a Resampling Schedule Alone

FIRST is not a new architecture. It is a resampling strategy applied to existing behavior cloning. Using flow-matching policies identical across conditions, FIRST outperforms prior force-aware policies by over 17% in task progress across five real-world tasks (Abstract). The gain comes entirely from reweighting which demonstrations get sampled during training — no new data, no architectural changes. This is operationally significant: it means existing demonstration datasets can be retroactively improved using NEXT-derived phase labels, even without rerunning teleoperation.

The Method Generalizes Across a 12× Price Range of Hardware

The system was validated on the AgileX Piper ($2,500), the YAM arm, the Nero arm, and the Franka ($30,000+). On the Piper — a low-cost arm with no force sensing — NEXT achieves 0.018 ± 0.012 Nm L1 error in free-space (Appendix A.6, Table 5), outperforming both physics-model-based baselines by more than 10×. This validates that the method is not tuned to high-end hardware; it is designed to democratize force awareness across commodity platforms.

Pre-Contact Up-Sampling Can Be Decoupled from Force as a Policy Input

In a buried but strategically important ablation, the paper shows FIRST improves policy performance even when force estimates are not fed to the policy at inference time — NEXT is used only to label training segments offline. "Phase-based upsampling improves over standard behavior cloning... showing that NEXT-derived phase labels are useful even when force estimates are used only at training time for data re-sampling" (Appendix D.5, Table 9). This means teams without real-time force pipelines can still benefit from the data curation insight alone.

2. Contrarian Perspectives

Pre-Contact Alignment Matters More Than In-Contact Behavior

Conventional wisdom in contact-rich manipulation focuses heavily on compliant control and force regulation during contact. This paper argues the opposite: the moment before contact is the higher-leverage intervention point. Table 2 shows that upsampling contact-only phases hurts or provides minimal benefit (average task progress 0.670), while upsampling pre-contact-only phases delivers the highest gains (0.818). "Upsampling pre-contact samples achieves the highest average task progress, suggesting that the moments before contact are especially important as they determine reaching the correct pose and alignment" (Section 7). For teams investing in force-compliant controllers, this suggests the ROI on better pre-contact pose estimation may be higher than better contact-phase control.

Raw Motor Current Is Not a Useful Force Proxy for Policy Learning

Several deployed systems and prior work (e.g., TA-VLA) use raw joint current as a cheap force signal. The paper directly challenges this: "Conditioning on joint current improves performance on NIST Insertion compared to the base ACT policy, but does not improve Cap Screwing... policies conditioned on raw joint current often continue rotating the cap after it is already tightened, leading to over-screwing failures" (Appendix D.6, Table 10). The problem is that raw current mixes free-space actuation effort with contact-induced torque, requiring the policy to implicitly learn the separation that NEXT does explicitly. Attention analysis shows NEXT-conditioned policies develop sharper, event-aligned attention to force signals at key contact moments; current-conditioned policies do not (Figure 13).

Model-Based Force Estimation Is Essentially Obsolete for Low-Cost Arms

The disturbance observer and FILIC (physics-model-based baselines) require accurate URDF parameters, system identification, and clean acceleration estimates. On the Franka — the best case for physics-based methods — the disturbance observer still produces 1.471 Nm contact error vs. NEXT's 0.547 Nm (Table 1). On the Piper, the gap is even larger (NEXT: 0.018 Nm vs. disturbance observer: 0.248 Nm, Table 5). The paper argues this is structural: "model-based external torque estimation is highly sensitive to errors in estimating τ_f... accurately identifying the inertial, gravitational, and friction parameters of a robot is difficult, especially when only motor-side measurements are available" (Section 3). For teams building on low-cost arms where accurate dynamics models are unavailable, this is a strong signal to skip physics-based force estimation entirely.

3. Companies Identified

AgileX Robotics | Low-cost robot arm manufacturer | The AgileX Piper (~$2,500) is the primary low-cost platform evaluated, representing the target deployment class for NEXT and FIRST. The Piper lacks dedicated force sensing, making it the canonical use case. NEXT achieves 0.018 Nm L1 error on Piper free-space, and force-feedback teleoperation user studies are run on the Piper. | "We evaluate this capability on platforms spanning low-cost arms such as the Piper ($2,500) and high-end arms such as the Franka ($30,000)" (Section 1)

Franka Robotics (now part of Agility/acquired) | Premium robot arm manufacturer | Used as the gold-standard reference platform because it has built-in joint torque sensors. NEXT's estimates are benchmarked against Franka's factory-calibrated external torque — and match or exceed it. The cost delta between Franka and Piper ($30K vs. $2.5K) frames the democratization argument. | "Dedicated force-torque (FT) sensors are typically available only on expensive platforms such as Franka, Flexiv, or KUKA arms" (Section 1)

Flexiv | Premium robot arm manufacturer | Cited as an example of high-end platforms with dedicated force sensing, implicitly positioned as hardware whose capabilities FACTR 2 partially replicates in software on lower-cost alternatives. | "Dedicated force-torque (FT) sensors are typically available only on expensive platforms such as Franka, Flexiv, or KUKA arms" (Section 1)

KUKA | Industrial robot manufacturer | Same positioning as Flexiv — cited as a premium force-sensing platform that NEXT's software approach aims to democratize. | Section 1

ATI Industrial Automation | Force/torque sensor manufacturer | Cited as a conventional piezoresistive FT sensor provider. The paper's core argument is that NEXT eliminates the need for products like ATI's Gamma sensor, which the paper describes as "expensive and bulky." | "Conventional piezoresistive FT sensors mounted between the arm and end effector rely on multiple strain gauges attached to precision-machined structures, making them expensive and bulky" (Section 1, Reference 1)

Bota Systems | Force/torque sensor manufacturer | Cited alongside ATI as a conventional force sensing hardware vendor whose products NEXT aims to replace. | Reference 2

YAM (AIRe Lab) | Low-cost robot arm | Evaluated as a third low-cost platform alongside Piper and Nero. Members of the AIRe Lab provided access for evaluation, suggesting collaborative validation across multiple commodity arm architectures. | "We also thank Zheyuan Hu and members of the AIRe Lab for generously providing access to their YAM arms" (Acknowledgments, Figure 1)

4. People Identified

Deepak Pathak | Carnegie Mellon University | Senior PI on the paper and a central figure in Physical AI at CMU. Pathak's lab has produced a consistent line of work on dexterous manipulation, LEAP Hand, and now force-aware policy learning. FACTR 2 extends his group's FACTR (RSS 2025) and LEAP Hand V2 work into accessible hardware. | Co-author, corresponding lab; "LEAP Hand V2: Dexterous, low-cost anthropomorphic hybrid rigid soft hand for robot learning" (Reference 51)

Ruslan Salakhutdinov | Carnegie Mellon University | Co-author and CMU ML department head. His involvement signals institutional depth and connects this work to broader generative modeling and representation learning research threads feeding into physical AI. | Co-author (Abstract)

Jason Jingzhou Liu | Carnegie Mellon University | Equal-contribution first author and correspondence contact. Liu is the primary researcher bridging the FACTR line of work — first author or co-author on both FACTR (RSS 2025) and FACTR 2. Likely the operational lead on the force-sensing research thread at CMU. | "Correspondence to: Jason Jingzhou Liu, liujason@cmu.edu" (Abstract footnote); lead author on FACTR [6]

Steven Oh | Carnegie Mellon University / Waseda University | Equal-contribution first author with a joint appointment, bringing hardware and contact dynamics expertise from Waseda. Co-author on related work on soft wrists for contact-rich manipulation. | Equal contribution (Abstract); "A soft wrist with anisotropic and selectable stiffness for robust robot learning in contact-rich manipulation" (Reference 7)

Tony Tao | Carnegie Mellon University | Equal-contribution first author, part of the CMU core team. Contributed to both FACTR and FACTR 2, indicating sustained focus on force-aware manipulation. | Equal contribution (Abstract)

Kenneth Shaw | Carnegie Mellon University | Co-author, known for LEAP Hand development. His presence connects FACTR 2 to the dexterous manipulation hardware thread — LEGO assembly experiments use LEAP Hand V2. | Co-author; LEAP Hand V2 reference (Reference 51)

5. Operating Insights

NEXT Is a Zero-Cost Upgrade Path for Any Arm Already in Your Fleet

If you are operating commodity arms — whether Piper, WidowX, SO-ARM, or any current-reporting arm — NEXT requires no hardware changes. The onboarding protocol is: (1) run the arm through a range of free-space motions for 10 minutes, (2) train the LSTM for 1 minute on an RTX 3090, (3) deploy at 100 Hz. "In practice, the dataset can be collected from approximately 10 minutes of free-space robot motion, generated either by teleoperation, motion planning, or replay of known free-space trajectories" (Section 4). For teams building teleoperation data collection pipelines, this is an immediate quality upgrade: force-feedback teleoperation on non-sensorized arms now performs comparably to sensorized systems in user studies (Section 7). The marginal cost is compute time, not hardware.

Restructure Your Demonstration Dataset Before Retraining — The ROI Is High

For teams with existing demonstration datasets for contact-rich tasks, FIRST's data resampling insight is directly actionable without collecting new data. Run NEXT on your existing demonstrations to label free-space, pre-contact, and contact phases, then apply 5× upsampling on pre-contact segments during the next training run. The paper shows this alone — without architectural changes — delivers 17%+ task progress improvements. "FIRST modifies only the sampling distribution used for batch construction" (Section 5). The most important tuning decision: use 1:5:1 (free:pre-contact:contact) weighting for most manipulation tasks; use 1:3:3 for tasks with sustained screwing or wrapping contact (Table 8). This is a practical recipe, not a research suggestion.

6. Overlooked Insights

NEXT Must Be Retrained Per Arm — But the Protocol Can Be Productized

A limitation buried in Section 9: "NEXT is robot-specific and may require retraining across arms due to hardware variation." This is operationally significant at scale. If you are managing a fleet of 50 Piper arms across a warehouse, each arm may require its own 10-minute calibration run and 1-minute training pass. The paper does not address inter-unit variation within the same arm model (e.g., whether one Piper's learned model transfers to another Piper). For fleet operators, this means NEXT onboarding is likely a per-unit commissioning step, not a one-time fleet-wide deployment. The silver lining: "these trajectories can be generated automatically through motion planning, in practice they can also be recorded once and replayed on new robot setups to quickly collect training data" (Appendix A.3) — meaning automated calibration pipelines are feasible and should be engineered into deployment infrastructure.

Aggressive Upsampling Hurts — There Is a Specific Saturation Point

The ablation in Appendix D.4 (Figure 12) shows that upsampling magnitude m=5 is approximately optimal, and that larger values degrade performance: "Larger values lead to saturation or degradation, suggesting that overly aggressive up-sampling can over-bias the training distribution toward contact-relevant segments and reduce coverage of the full task trajectory." This is a practical guardrail for teams applying FIRST: more is not better. Additionally, Table 11 shows that weighted regression (modifying the loss function) is worse than weighted sampling (modifying the batch composition) — weighted regression "either degrades performance or provides only limited improvement," while per-batch upsampling consistently improves results. Teams tempted to implement FIRST via loss reweighting (a common default) should use the sampling approach instead.