PAC-MAN: Perception-Aware CBF-RL for Whole-Body Safety in Humanoid Dodgeball

1. Key Themes

Perception-Aware Safety Co-Design

The paper demonstrates that the mathematical safety constraints used to train a robot must be explicitly matched to the quality of the robot's sensors. If a robot has perfect knowledge of an incoming object's position, it can use complex, whole-body safety filters. If it only has a fixed, noisy depth camera, it must use a simpler safety structure or it will actually perform worse. The authors state that "usable barrier structure depends on perceptual observability" (Abstract).

Zero-Shot Sim-to-Real Transfer for Reactive Evasion

The researchers successfully trained a policy in simulation and deployed it directly to a physical humanoid robot without any fine-tuning. The robot was able to dodge incoming balls thrown by hand, achieving a 95% success rate using only its onboard cameras and joint sensors. The paper notes, "We therefore deploy a lightweight Link-CBF policy zero-shot on the Unitree G1 in the real world, where it tolerates imperfect perception, succeeds on 95% of throws" (Abstract).

Whole-Body Collision Avoidance

Rather than just protecting the robot's core or torso, the system explicitly tracks and protects every individual limb (arms, legs, head) from being hit. This is a much harder problem because a robot might save its torso but get hit in the arm. The paper formulates this as a comprehensive safe set: "The corresponding whole-body collision-free set is C = ⋂i {hi ≥ 0}" (Section III-B).

2. Contrarian Perspectives

Stronger Safety Constraints Can Hurt Performance

Conventional wisdom in control theory suggests that adding more rigorous safety barriers (like a Joint-CBF) will always yield a safer system. This paper argues the opposite when perception is limited: forcing a robot to learn a complex safety constraint that it cannot accurately perceive leads to worse behavior. The authors explain, "We theorize that the Joint-CBF gap comes from perception: the policy is asked to internalize a whole-body barrier that it cannot perceive accurately enough, and it learns worse evasions instead" (Section IV-A).

Fixed Onboard Cameras Are Sufficient for Sub-Second Reactions

Many robotics companies invest heavily in active gaze tracking (gimbals that follow objects) or external motion capture systems for fast, reactive tasks. This paper shows that a simple, rigidly mounted head camera is enough for a humanoid to dodge a ball thrown at it, provided the software is designed correctly. The authors conclude, "The fixed onboard camera alone is therefore adequate for evasion; what perception changes is which safety structure works" (Section IV-A).

3. Companies Identified

Unitree

Description: Manufacturer of the G1 humanoid robot. Why relevant: The G1 serves as the physical hardware platform for the real-world deployment of the dodgeball policy. The paper states, "We deploy the fixed-camera Link-CBF policy on the Unitree G1 with a head-mounted ZED Mini" (Section IV-B).

The Dow Chemical Company

Description: Chemical manufacturing corporation. Why relevant: Funded the research. The paper notes, "This work is supported by The Dow Chemical Company project #227027AW" (Title page).

Technology Innovation Institute

Description: Applied research institute. Why relevant: Co-funded the research. The paper notes the work was supported "in part by Technology Innovation Institute" (Title page).

4. People Identified

Lizhi Yang

Lab/Institution: California Institute of Technology (Caltech) Why notable: Lead author of the paper, driving the implementation of the perception-aware CBF-RL framework on the Unitree G1 hardware. (Title page)

Aaron D. Ames

Lab/Institution: California Institute of Technology (Caltech) Why notable: A leading figure in control theory and robotics, particularly known for his foundational work on Control Barrier Functions (CBFs). His lab bridges rigorous control theory with modern reinforcement learning for physical deployment. (Title page)

5. Operating Insights

Match Safety Enforcement to Sensor Fidelity

CTOs and heads of engineering should not blindly apply the most advanced safety filters if their robot's perception stack cannot support it. If your robot relies on fixed, low-resolution cameras, use lighter safety guidance during training rather than complex joint-space projections. The paper explicitly states, "the design implication is that the strength of the safety structure must be matched to the information available" (Section IV-A).

Use Asymmetric Actor-Critic for Sim-to-Real Transfer

When training robots in simulation, give the "brain" (the value function) access to perfect, simulated ground-truth data (like exact ball position and velocity), but force the "muscles" (the policy) to only look at realistic sensor data. This allows the system to learn efficiently without becoming dependent on data it won't have in the real world. The authors note, "Training follows the asymmetric actor-critic paradigm: the value function Vϕ(st) observes a privileged state st... These privileged signals support training and barrier computation but are discarded at deployment" (Section III-A).

6. Overlooked Insights

The Necessity of Motion Priors for Sparse Rewards

Simply telling a robot "don't get hit" is not enough to generate useful, human-like movement; the robot might find bizarre, unstable ways to avoid a ball. By using an adversarial motion prior (AMP) trained on human dodging clips, the robot learns to crouch, lean, and sidestep naturally. The authors explain, "For dodgeball, a motion prior is attractive because the reward for 'not being hit' is too semantically sparse and geometric to specify how a humanoid should move" (Section I-B).

Click-to-Track Generalization for Different Objects

The deployed vision system uses a tracker initialized by a single human click, meaning the robot does not need to be retrained to dodge different types of objects. This allows the same policy to dodge a foam ball or a soccer ball just by changing the initial tracking point. The paper highlights, "an EfficientTAM track, initialized from a clicked foreground point, segments the ball in RGB at ~60 Hz... This also gives us the ability to dodge different balls since the tracker can be initialized on any object of interest" (Section IV-B).