Yilun Du

“The paper's core contribution is showing that one video model checkpoint can serve as both a forward dynamics simulator (given robot motion, predict scene response) and an inverse model (given desired object motion, synthesize robot behavior). This is achieved by expressing actions as masked pixel-space trajectories rather than low-dimensional action vectors.”

“The model trained only on forward examples (robot motion → scene) generalizes zero-shot to the inverse direction (object motion → robot), which the authors note was unexpected”

“A Single Model Can Serve as Simulator, Evaluator, and Policy. The forward model serves as a simulator for planning and policy evaluation; the inverse model serves as a policy by generating robot motion from desired outcomes”

“Harvard researchers built a controller that lets robots trained on lightweight objects immediately handle much heavier ones — without force/torque sensors, without retraining, and without extra demonstrations.”

“the cerebellum builds internal models of the body and external objects based on prior experience, and utilizes these models to generate appropriate feedforward forces to counteract predictable disturbances such as object gravity.”

“Du is an emerging force in robot learning, with connections to diffusion policy work (co-authored with Chi et al.) and broader generative model applications to robotics.”