GORDON: Graph-based Object-centric Rewards for Decomposition of Long-Horizon Manipulation

1. Key Themes

Automatic Subtask Discovery Eliminates Manual Engineering

The most significant operational achievement of GORDON is its ability to automatically break down complex, multi-step (long-horizon) tasks into smaller, manageable subtasks without human annotation. Instead of requiring an engineer to manually define when one step ends and another begins, the system analyzes the temporal profile of its own learned reward to find natural transition points. The paper notes that "candidate subtask boundaries can be recovered directly from the learned full-task reward and used to define simpler subtask-specific policy-learning problems" (Section IV-A). Figure 6 visually demonstrates this, showing that the system's inferred subtask endings closely align with the actual ground-truth task transitions.

Robustness to Visual Distractors via Object-Centric Graphs

Many robotic learning systems fail when deployed in new environments because they rely on raw pixel data, making them sensitive to background changes, lighting, or object textures. GORDON bypasses this by converting visual scenes into graphs where nodes are objects and edges are spatial relations. The authors prove this robustness by testing the system under "altered scene and object colors," finding that while pixel-based methods degrade significantly, "object-centric methods are unaffected... because their rewards are computed from object identities and geometric relations rather than raw pixels" (Section IV-B, Table III).

Action-Free Video Demos as a Scalable Supervision Source

Data collection is a massive bottleneck for robotics companies. Traditionally, training a robot requires recording the exact motor commands (actions) of a teleoperator. GORDON learns purely from "action-free video demonstrations"—meaning you can just point a camera at a human or another robot completing a task, and the system learns the reward function from the video alone. The paper states these demos "are especially attractive because they do not require access to expert actions and costly annotation" (Section I).

Masking Robot Motion for Semantic Task Alignment

A subtle but critical issue in robot learning is that the robot's own movement dominates the visual frame, distracting the learning algorithm from the actual task progress. GORDON introduces an "activity-aware weighted pooling mechanism" that explicitly masks the robot from the graph representation. The authors find that "masking robot-dominated motion not only improves semantic alignment but also reduces sensitivity to noisy robot geometry in the reward representation" (Section IV-B, Table III).

2. Contrarian Perspectives

Learned Rewards Can Outperform Hand-Engineered Oracle Rewards

The conventional wisdom in robotics is that hand-crafted rewards using privileged simulator state data (the "oracle") represent the ceiling of performance. GORDON challenges this by beating the oracle. The paper reports achieving an average success rate of 74.4% on long-horizon tasks, which is "approximately +25 p.p. vs. oracle" (Abstract). The authors argue that manually engineered single-task rewards fail to provide "sufficient stage-wise guidance for full long-horizon completion" (Section IV-A), whereas their automatic decomposition creates more targeted supervision.

Language Models Are Not Necessary for Task Decomposition

A major trend in Physical AI is using Large Language Models (LLMs) or Vision-Language Models (VLMs) to decompose long-horizon tasks into steps. GORDON argues against this, showing that learning task progress directly from object-centric demonstrations is more precise and transferable. The paper explicitly states that their method achieves strong results "without relying on externally specified reward stages or language-generated decomposition" (Section IV-A). They outperform RoboHorizon (an LLM-assisted baseline) by 35.3 percentage points (Section I).

Raw Pixel-Based Reward Learning is Too Brittle for Real Robotics

While representation learning from raw pixels (like the XIRL baseline) is popular in academic research, GORDON's results suggest it is practically useless for complex tasks. The paper shows that pixel-based methods completely fail on long-horizon tasks, achieving a "0.0±0.0" success rate across all four long-horizon benchmarks (Table I). Furthermore, under visual appearance shifts, the pixel-based reward discrimination ratio degrades from 0.898 to 1.759, whereas GORDON remains perfectly stable at 0.629 (Section IV-B, Table III).

3. Companies Identified

ManiSkill3

Description: A GPU-parallelized robotics simulation and rendering benchmark. Why relevant: GORDON uses ManiSkill3 to implement its long-horizon benchmarks. The paper highlights its operational advantage, noting that ManiSkill3 reduces training time from nearly 8 hours in RLBench to under 1 hour, making it a highly efficient platform for rapid iteration (Section IV-B).

RLBench

Description: A robot learning benchmark and learning environment. Why relevant: Serves as the original basis for the 3D manipulation tasks used in the paper. The authors re-implemented RLBench tasks into ManiSkill3 to achieve faster, parallelized training (Section IV).

RoboHorizon

Description: A baseline method that uses LLM-assisted task decomposition and multi-view world models for long-horizon manipulation. Why relevant: Represents the competing approach of using language models for task decomposition. GORDON outperforms it by 35.3 percentage points, challenging the viability of LLM-based decomposition for precise manipulation (Section IV-A, Table I).

4. People Identified

Andrea Protopapa

Lab/Institution: Politecnico di Torino Why notable: Lead author of the paper. The project page is hosted on their GitHub, indicating primary development of the GORDON framework. Quotes: "we propose GORDON, a graph-based object-centric reward learning framework that learns dense rewards from action-free video demonstrations" (Abstract).

Georgia Chalvatzaki

Lab/Institution: Interactive Robot Perception & Learning (PEARL) Lab, TU Darmstadt / Hessian.AI / Robotics Institute Germany (RIG) Why notable: Co-author and a notable figure in robot perception. Her involvement signals strong cross-institutional collaboration between Italian engineering and German robotics AI hubs. Quotes: (Co-authored the paper, contributing to the methodology and analysis).

Giuseppe Averta

Lab/Institution: Politecnico di Torino Why notable: Co-author and corresponding senior figure from Politecnico di Torino. Quotes: (Co-authored the paper, contributing to the overarching strategy of object-centric reward learning).

5. Operating Insights

Prioritize Object-Centric State Representations Over Raw Vision for RL Rewards

CTOs and heads of engineering should structure their perception pipelines to output object graphs (bounding boxes, semantic classes, spatial relations) rather than feeding raw pixels into reward functions. The paper demonstrates that graph-based representations are immune to visual distractors and background changes, drastically improving the transferability of learned policies from simulation to the real world (Section IV-B, Table III).

Leverage Action-Free Video Demos to Cut Data Collection Costs

Collecting teleoperation data with precise joint actions is expensive and slow. GORDON proves that simple video recordings of a task being completed (even by a human, without recording robot actions) are sufficient to learn dense, effective rewards. Operators can scale their demonstration datasets much faster by decoupling data collection from robot teleoperation (Abstract, Section I).

Use ManiSkill3 for Faster Iteration Cycles

For teams evaluating simulation environments, the paper provides a concrete data point on the efficiency of ManiSkill3. By utilizing 32 parallel environments, training time for a manipulation task was reduced from 7 hours and 54 minutes (in RLBench) to just 27 minutes. This allows engineering teams to test reward functions and policies an order of magnitude faster (Section IV-B).

6. Overlooked Insights

The Importance of Persistent Object Relevance

A buried but highly practical detail is how GORDON handles "active" objects. Once an object is moved or interacted with, the system keeps it marked as "active" for the rest of the episode. The authors note this "reflects the intuition that objects involved in earlier stages often remain relevant for representing completed progress" (Section III-A). This prevents the system from forgetting that a previously manipulated object exists, a common failure mode in sequential manipulation.

Terminal Bonus via Learned Classifier is Critical for Contact-Rich Tasks

The dense, distance-based reward alone is not enough for precise, contact-rich tasks. The paper reveals in an ablation study that while the dense reward can solve simple subtasks (like opening a box), it completely fails on placement tasks (like putting shoes in a box). To solve this, they had to train a lightweight MLP classifier to act as a "success detector" that provides a terminal bonus. Without this learned bonus, the composed policy success rate on PutShoesInBox was 0%, but with it, the success rate jumped to 81.5% (Section IV-B, Table IV).