Chronos Policy: Learning Robot Policies on Physical Time

1. Key Themes

Making Physical Time Explicit in Robot Policies

The paper introduces a policy formulation that conditions on observation capture times and action execution times, rather than relying on sequence position as a proxy for time. This is critical because real-world robots experience variable sensor rates, delays, and inference latency. As stated in Section 1: "This assumes a consistent relationship between sequence position and elapsed time; when that relationship changes, the policy cannot tell how much time has passed between observations or how far in the future an action will execute."

Time-Aware Rotary Positional Encodings (RoPE)

The method uses physical timestamps as rotary coordinates in attention, keeping token content unchanged while making attention depend on elapsed physical time. This approach is more robust than adding time embeddings directly to token features. Section 4.3 notes: "On Pivoting, Chronos-RoPE outperforms Chronos-AddTime in every condition and improves the perturbed-condition average from 33.5% to 55.3%."

Training with Temporal Randomization

By randomly varying observation rates, delivery delays, and action chunk start times during training, the policy is exposed to deployment-time timing variations. This also enables learning from asynchronous, multi-rate data. Section 3.3 states: "Together, these two randomizations expose the policy to temporal settings that may occur at deployment. They also allow different measurements from the recorded trajectory to be used across training examples, including measurements that a fixed-rate downsampling scheme would otherwise discard."

Robustness to Timing Mismatch in Real-World Deployment

Chronos Policy improves performance on dynamic tasks (e.g., grasping moving objects) and multisensory tasks (e.g., contact-rich manipulation with vision, proprioception, and force/torque), while preserving nominal performance on quasi-static tasks. Figure 5 shows that in real-world evaluation, "Chronos Policy achieves the highest task score in every evaluated condition across all three tasks."

2. Contrarian Perspectives

Sequence Position is Not a Sufficient Proxy for Time

Most imitation learning methods convert demonstrations into fixed-rate sequences. This paper argues that this assumption breaks in the real world where sensors operate at different rates and latencies, and inference takes time. Section 1 states: "This assumes a consistent relationship between sequence position and elapsed time; when that relationship changes, the policy cannot tell how much time has passed between observations or how far in the future an action will execute."

Data Augmentation via Temporal Randomization is Insufficient Without Explicit Time

One might think that simply randomizing timing during training (data augmentation) would make a standard policy robust. The paper shows that without explicit timestamps, this randomization introduces ambiguity, especially in multisensory tasks, actually reducing performance. Section 4.3 notes: "The same strategy does not generalize to multisensory Pivoting. There, temporal randomization reduces the perturbed-condition average from 48.4% to 21.6%. Without timestamps, the policy cannot tell what timing produced each sampled history..."

Adding Time to Token Features is Inferior to Modulating Attention

A common approach to incorporating continuous variables is to encode them with Fourier features and add them to token embeddings. Chronos Policy shows that keeping token content and time separate, using time only in positional encoding (RoPE), is more robust. Section 4.3 states: "On Pivoting, Chronos-RoPE outperforms Chronos-AddTime in every condition and improves the perturbed-condition average from 33.5% to 55.3%."

3. Companies Identified

Samsung

Description: Multinational electronics and technology company. Why relevant: The research was supported by Samsung, indicating their interest in advancing physical AI and robotics capabilities. Quotes: "This work was supported in part by... Samsung..." (Section 6: Acknowledgment)

4. People Identified

Changhao Wang

Lab/Institution: Stanford University (REALab) Why notable: First author of the paper, developing methods to make robot policies robust to real-world timing variations. Quotes: "We introduce Chronos Policy, which makes physical time explicit by conditioning on observation capture times and desired action execution times." (Abstract)

Jaden Clark

Lab/Institution: Stanford University Why notable: Co-author, contributing to the development of time-aware policies for multisensory manipulation. Quotes: (Co-authored the paper, listed as a primary contributor).

Shuran Song

Lab/Institution: Stanford University Why notable: Senior author and head of REALab, a prominent researcher in physical AI, robotics, and manipulation. Her lab focuses on building robots that can operate in unstructured environments. Quotes: (Senior author, oversaw the research).

5. Operating Insights

Design for Asynchronous, Multi-Rate Sensor Fusion

When building robots with heterogeneous sensors (e.g., cameras at 20Hz, force/torque at 90Hz), do not force them onto a fixed temporal grid. Instead, pass the physical capture timestamps to the policy network. Section 3.1 states: "For multimodal sensing, each modality simply keeps its own capture times rather than being aligned to a common sampling grid."

Estimate Inference Latency at Deployment

At deployment, the exact inference latency is not known in advance. The system should estimate it from previously completed requests and use that estimate to set the action execution times. Section 3.4 notes: "On the real robot, the current inference time is not known in advance, so we estimate dt from previously completed requests."

Train with Temporal Randomization

To make policies robust to deployment-time timing variations, randomly vary observation rates, delivery delays, and action start delays during training. This acts as both data augmentation and a way to use measurements that fixed-rate downsampling would discard. Section 3.3 states: "Together, these two randomizations expose the policy to temporal settings that may occur at deployment. They also allow different measurements from the recorded trajectory to be used across training examples, including measurements that a fixed-rate downsampling scheme would otherwise discard."

6. Overlooked Insights

Temporal Randomization Can Hurt Performance Without Explicit Time

While temporal randomization sounds like a good idea for robustness, applying it to a standard Diffusion Policy without explicit timestamps actually degrades performance, especially on multisensory tasks, because the policy cannot disambiguate the timing. Section 4.3 notes: "Adding temporal randomization to Diffusion Policy reduces performance in every condition, including nominal success from 61.6% to 29.8%."

Improved Generalization to Out-of-Distribution Objects

In the real-world Pivoting task, Chronos Policy showed improved generalization to out-of-distribution objects (box, sideways configuration) compared to baselines, particularly under timing perturbations. Table 9 shows that for the Box (OOD) under the Combined timing condition, Chronos Policy achieved a 60.0% task score (3/0/2 full/partial/failure) compared to Diffusion Policy's 40.0% (2/0/3).