Teahose.
SIGN IN
NEW HERE — WHAT TEAHOSE DOES
We read the entire AI & tech firehose — so you don't have to.
PODPodcastsAll-In, No Priors, Acquired…
NEWNewslettersStratechery, Newcomer…
PAPPapersPhysical AI research
PHProduct Huntdaily launches
VCInvestor ScoutSequoia, a16z, Benchmark…
CLAUDE DISTILLS →
7 reads, 30 sec each — free, 6 AM ET.
+ a live graph of the companies, people & themes underneath.
HOME/ARXIV PHYSICAL AI RESEARCH/PGX: Physically Grounded Executi…
PAPR
// RESEARCH PAPER
ARXIV PHYSICAL AI RESEARCH

PGX: Physically Grounded Execution for Long-Horizon Humanoid Loco-Manipulation

DATE November 19, 2026SOURCE ARXIV PHYSICAL AI RESEARCHPARTICIPANTS KAI LU, CHENYANG MA, IOANNIS HAVOUTISARXIV 2611.26596
// KEY TAKEAWAYS5 ITEMS
  1. 01Execution Structure, Not Model Scale, Unlocks Long-Horizon Success
  2. 02Physical Predicates as the Universal Interface Between Training, Execution, and Evaluation
  3. 03World-Coordinate Locking Solves the Precision Problem VLAs Can't
  4. 04Recovery and Reverse Transitions Make Failures Non-Fatal
  5. 05Validated on Real Hardware, Not Just Simulation
// SUMMARY

The one-sentence pitch: A training-free execution scaffold — a predicate-gated state machine plus world-frame geometric control — turns a fine-tuned VLA that scores 0–35% on a multi-step humanoid coffee task into an 80% performer without changing a single model weight. The paper is a strong argument that near-term humanoid deployment value lives in the orchestration layer, not in bigger policies.


1. Key Themes

Execution Structure, Not Model Scale, Unlocks Long-Horizon Success

The headline result: on coffee fetching, PGX "improves the task success rate of a fine-tuned VLA from 35% to 80% with unchanged weights" (Sec. I), and lifts flat π0.5 from literally 0% success to 60% (nominal) and 55% (randomized start) — see Table I. The authors' core insight is a division of labor: "the aspects a flat VLA handles least reliably, deciding when and where to act, can often be resolved directly from physical quantities, whereas learning how to execute a skill is precisely where VLAs are most effective" (Sec. I). For anyone building humanoid products, this means the highest-leverage engineering investment may be the executor, not the policy.

Physical Predicates as the Universal Interface Between Training, Execution, and Evaluation

PGX's most elegant design choice is that one rule serves three purposes: "The same predicate determines skill boundaries in demonstrations offline (Sec. III-C), gates transitions online, and records task progress at evaluation" (Sec. III-B). Predicates like grasped combine hand state, wrist–object distance, and lift height "so that closing an empty hand is not mistaken for a grasp" (Eq. 3, Sec. III-B). This eliminates the classic drift between what you train on, what you run, and what you measure — a common source of silent failure in deployed robot stacks.

World-Coordinate Locking Solves the Precision Problem VLAs Can't

A biped's base moves constantly, and "actions represented relative to the moving pelvis frame make precise manipulation difficult, since the action reference changes as the humanoid moves" (Sec. I). WCL anchors stances, contact targets, and base positions in the world frame. The payoff is dramatic: the button-press stage, driven by a world-anchored geometric primitive (WCL-P), "completed 243 of 245 entries into the button stage (99%), against 72–100% when the stage is left to the VLA" (Sec. IV-E). Station keeping cut p90 base drift under sustained disturbance from 54 cm to 9 cm (Fig. 5).

Recovery and Reverse Transitions Make Failures Non-Fatal

The state machine doesn't just sequence stages — it recovers from them. "When the guard fails, the executor restores the stance and requests a fresh action chunk, without releasing a held object" and for stage confusion, "the machine allows reverse transitions: if during press the mug is grasped again while the device has not been triggered, the executor returns to place" (Sec. III-B). This retry/reverse machinery is why PGX policies survive mid-task errors that terminate flat policies.

Validated on Real Hardware, Not Just Simulation

PGX ran on a real Unitree G1 with Dex-3 hands performing the full coffee task — walk, grasp, place, wait, re-grasp, deliver (Fig. 8) — with the stage graph and predicates "transferred from simulation" (Sec. IV-F). Per-stage completion (Fig. 7) shows grasping transfers best, with gaps concentrated "at the two contact-rich stages, placing and re-grasping," which the authors attribute to "the precision of the fine-tuned skills on the real hand rather than the execution structure" (Sec. IV-F).


2. Contrarian Perspectives

"Don't Put the Planner in the Network" — Against the End-to-End Everything Trend

Most of the field is racing to fold temporal reasoning into the VLA itself (phase-aware masking, explicit memory, embodied chain-of-thought). PGX explicitly rejects this: rather than "folding temporal structure into the policy itself," PGX "switches skills on physical events rather than inferred semantics" (Secs. I, II-B). The evidence is hard to argue with — the FSM alone (no spatial layer, no retraining) lifted π0.5's mug-lift rate to 90–95% of episodes (Table I, FSM-only rows). Companies betting purely on monolithic end-to-end policies for long-horizon work should note that a classical FSM beat their approach for free.

Prompt-Switching Planners Are Not Enough

This is a direct challenge to the popular "LLM/VLM planner picks the sub-task prompt" architecture (SayCan, Hi Robot, Code-as-Policies style stacks). The authors found that "on the π0.5 training frames we inspected, replacing the stage instruction by a mismatched one changes the output little... This also suggests that a planner that only switches the prompt would not by itself recover this gain" (Sec. IV-D). The gain comes from the executor re-establishing start conditions and controlling the base — not from telling the VLA what to do. If your architecture is a language planner over a flat VLA, this paper says you're missing the mechanism that actually matters.

Some Jobs Should Be Taken Away From the VLA Entirely

The stage-wise training ablation is humbling for the "more learning" instinct: "Stage-wise training, in contrast, is a secondary and model-dependent factor: for π0.5 it changes success by at most 5 points, and for GR00T the flat weights are the stronger skill model on coffee" (Sec. IV-D). And for millimeter-precision contacts, the VLA loses to a geometric primitive outright — the button press hit 99% with the primitive vs. 72–100% under VLA control (Sec. IV-E). The contrarian read: knowing when to hand a sub-task to classical control is a core competency, not a retreat from AI.


3. Companies Identified

Physical Intelligence

  • Description: Developer of π0 and π0.5 VLA foundation models; one of the two skill backbones fine-tuned in this paper.
  • Why relevant: PGX's results quantify both the power and the ceiling of π0.5 as a skill engine — flat π0.5 scored 0% on coffee fetching, but inside PGX reached 70% success (Table I). This is simultaneously a validation of the backbone and a signal that Physical Intelligence's models need external orchestration for long-horizon loco-manipulation today.
  • Quote: "We fine-tuned the state-of-the-art foundational VLA models π0.5 and GR00T N1.6 on collected human demonstrations" (Sec. I).

NVIDIA

  • Description: Provider of GR00T N1.6 (humanoid VLA), Isaac Lab Arena (simulation), GR00T-WBC (whole-body controller), and Dex-3 hand hardware — essentially the full stack around this work.
  • Why relevant: GR00T N1.6 was the stronger backbone (80% coffee success inside PGX vs. 70% for π0.5), and the real robot ran "the decoupled GR00T-WBC policy" for locomotion (Sec. IV-A.2). NVIDIA's ecosystem is becoming the default substrate for academic humanoid research, which compounds its platform position.
  • Quote: "Simulation experiments are performed in Isaac Lab Arena [40] with a Unitree G1 humanoid, Dex-3 hands, and the corresponding whole-body control stack" (Sec. IV-A.1).

Unitree

  • Description: Manufacturer of the G1 humanoid robot used for both simulation and real-world experiments.
  • Why relevant: The G1 is the de facto research humanoid; PGX demonstrates a complete loco-manipulation pipeline (walking, grasping, machine operation, delivery) on it. Real-robot validation on commodity hardware strengthens the G1's case for pilot deployments.
  • Quote: "We deploy PGX on a G1 with a transferred stage graph and predicates from simulation" (Sec. IV-F).

Vicon

  • Description: Motion-capture vendor; the real-robot predicates "read Vicon and joint states" (Sec. IV-F, Fig. 8 caption).
  • Why relevant: A quiet but important caveat — the real-world demo depends on external motion capture for object and base poses. PGX's predicate gating currently needs state estimation that isn't fully onboard, which is a real gap for commercial deployment.

Meta (Quest)

  • Description: VR teleoperation hardware used to collect demonstrations — 50 Quest-teleoperated demos for sim coffee, 44 for the real robot (Sec. IV-A.1).
  • Why relevant: Confirms VR teleop as the standard data-collection path for humanoid VLAs; relevant to anyone sizing data-collection costs.

4. People Identified

Kai Lu

  • Lab/Institution: Oxford Robotics Institute, University of Oxford (lead author).
  • Why notable: First author on PGX and previously on KitchenVLA (iterative vision-language corrections for robot execution, cited as [38]) — a consistent research thread on making learned policies reliable in structured execution. Someone to watch in the "orchestration layer" niche of Physical AI.
  • Quote: Corresponding author; the paper's framing that "physical structure governs task progression and spatial precision, allowing the VLA to focus on transferable manipulation dynamics" (Sec. I) is his thesis.

Chenyang Ma

  • Lab/Institution: Department of Computer Science, University of Oxford.
  • Why notable: Also first author of FurnitureVLA (long-horizon bimanual furniture assembly, cited as [13]) — another long-horizon VLA effort. His trajectory shows the field's leading researchers repeatedly hitting the same wall: monolithic VLAs fail at long-horizon sequencing, motivating structured execution.
  • Quote: Co-author; the FurnitureVLA citation is used to support the observation that "visually similar observations from different stages can lead to incorrect skill execution" (Sec. I).

Ioannis Havoutis

  • Lab/Institution: Director, Oxford Robotics Institute, University of Oxford.
  • Why notable: Leads one of the world's premier robotics institutes; his group's pivot toward hybrid classical-learned execution for humanoids is a meaningful signal about where credible academic robotics thinks the deployment bottleneck is.

5. Operating Insights

Budget Your Engineering for the Executor, Not Just the Policy

The ablation ladder (Table III) is the most operationally useful result in the paper: moving a flat model into the PGX executor — same weights, same task-level prompt — took π0.5 from 0% to 60% and GR00T from 35% to 80% on coffee. Stage-wise retraining moved things by ≤5 points, and actually hurt GR00T (80% → 35%). If you're a CTO allocating headcount between fine-tuning experiments and execution infrastructure, this paper says the executor is where the compounding returns are. It also means you can swap in "whichever model has the stronger local skill... without retraining a stage model" (Sec. IV-D) — backbone-agnostic orchestration protects you from model-vendor lock-in.

Mask the VLA's Base Commands During Manipulation

A specific, immediately actionable finding: "a flat policy's base commands are the main source of drift in our trajectories" (Sec. III-D.2). PGX masks the base-velocity channel whenever the VLA is manipulating and lets the executor own station keeping. If your humanoid or mobile-manipulator stack lets the learned policy command the base while the arms work, you are likely accumulating exactly this drift failure mode. Related: the executor also waits "for the stopping transient to decay" and sets a compatible arm posture before each skill (Sec. III-C) — start-state discipline is cheap and high-yield.

Robustness to Start Poses Comes From Object-Relative Stances, Not More Data

Under randomized initial poses (±0.30 m, ±30° — conditions "the flat policies have never seen"), PGX's mug-placement success barely moved because "the object-relative stance and station keeping re-establish the same start state before every grasp" (Sec. IV-C). The strategic implication: generalization to deployment variability can be bought with geometric re-anchoring rather than data collection, and each retry refreshes the goal "so that a placement error does not carry over as a start-state error" (Sec. III-D.1).


6. Overlooked Insights

The Latency Tax Is Real and Unquantified

Buried in the conclusion: "PGX achieves higher success rates yet costs longer execution time for stage transitions" (Sec. V). For commercial deployments — think warehouse throughput or service-robot task cycles — a framework that trades wall-clock time for reliability changes unit economics. The paper never quantifies the slowdown, which is a gap any operator evaluating this approach should probe. The 40–110 second time budgets per task (Sec. IV-A.4) already suggest these are not fast cycles.

The Predicate Layer Is Surprisingly Noise-Tolerant — and Surprisingly Dependent on External State

Two findings that cut in opposite directions and deserve more attention than they get. On the positive side: "on the mug task, per-episode object-position biases of 2, 3, and 5 cm with per-frame jitter keep success at 75–90%" (Sec. IV-E) — the hysteresis and hold-time design means you don't need perfect perception for gating. On the cautionary side, the real-robot predicates read Vicon motion capture (Sec. IV-F), so the demonstrated system's grounding depends on external localization. The unspoken engineering question for any PGX-style product: can your onboard state estimation (object pose, grasp evidence, device states) deliver predicate-quality signals without mocap? That's the actual gap between this paper and a deployable coffee robot — and it's a perception/state-estimation problem, not a VLA problem.