Efficient Vision-Language-Action Management and Serving for Robot Factories

1. Key Themes

Intra-GPU Stage Disaggregation for VLA Models

Vision-Language-Action (VLA) models consist of two distinct stages: a Vision-Language Model (VLM) stage that is compute-intensive, and an Action Diffusion Transformer (ADiT) stage that is memory-intensive. Prior serving systems attempted to disaggregate these stages across separate GPUs, which left most compute resources underutilized. Robion introduces "Intra-GPU Stage Disaggregation," running both stages concurrently within a single GPU using two asynchronous streams. As the paper states in Section 4.1.1, "We disaggregate VLM and ADiT stages within GPU to enable separate optimizations for each, overlap their executions for independent requests, and improve GPU utilization." This allows the system to exploit the complementary resource demands of the two stages without the communication overhead of moving data between GPUs.

Dynamic SM Partitioning via Locksteps

Simply running the VLM and ADiT stages on separate streams does not guarantee concurrent execution, as the larger VLM kernels can saturate all available Streaming Multiprocessors (SMs) on the GPU, blocking the smaller ADiT kernels. Robion solves this by restricting the number of SMs the VLM stream can occupy, ensuring ADiT always finds available SMs. To avoid the performance degradation of preempting a running stage to change SM limits, Robion executes the stages in synchronized "locksteps." As detailed in Section 4.1.2, "We enable dynamic SM restriction on VLM stream without stage preemption, while leveraging CUDA graphs by executing VLM and ADiT stages as a sequence of locksteps... At the start of lockstep 𝑘, we dequeue up to a max_batch requests, and admit them as a fixed batch size 𝑏𝑘 to the VLM stage, while ADiT stage concurrently processes the (𝑘-1)-th batch."

Multi-Model Co-Location and Traffic Control

Unlike large language models (LLMs) that can have hundreds of billions of parameters, VLA models are relatively small, typically 1-5 billion parameters, requiring only about 10GB of memory. This means multiple VLA models can be co-located on a single high-end GPU (which often has 80-120GB of memory). Robion shares a single VLM stream and a single ADiT stream across all co-located models. Furthermore, its management engine uses an "intelligent traffic controller" that formulates robot-to-GPU assignments as an integer program. According to Section 4.2.2, this "concentrates each model’s requests onto a single model copy on a GPU to increase per-model batching, while bounding each GPU’s load to prevent SLO violations."

Edge Server Deployment for Robot Factories

The paper makes a strong case against on-robot inference for VLA models. High-end GPUs are too heavy, costly, and power-hungry to integrate directly onto robots. Instead, the authors advocate for a local edge server architecture where a single server with multiple high-end GPUs serves inference requests for an entire fleet of robots. As stated in Section 2.2, "edge server offloading reduces inference latency, extends battery life, improves cost efficiency, and allows a single server to serve multiple robots at high utilization, being a practical deployment target for VLA inference." In their evaluation, Robion served up to 64 robots on a single 4-GPU server while maintaining 98% SLO attainment (Abstract).

2. Contrarian Perspectives

Existing Multi-Stage Serving Systems Fail for VLA

Conventional wisdom in AI serving often favors stage disaggregation across separate processes or GPUs to optimize throughput. However, the paper argues that existing systems like vLLM-Omni are fundamentally ill-suited for VLA models. These systems incur a "buffer inspection cost" to check for newly arrived requests at every diffusion step, which is negligible for large models but dominant for VLA models whose stages execute in only a few milliseconds. As the paper notes in Section 2.5, "in vLLM-Omni, the most widely used serving system with stage disaggregation, this buffer inspection cost accounts for 30-35% of total VLA inference latency." Consequently, a monolithic approach without stage disaggregation actually outperforms vLLM-Omni, challenging the assumption that process-level disaggregation is always better.

On-Robot Compute is Impractical for VLA

A common trend in edge AI is to push compute as close to the sensor as possible, often onto the device itself. This paper directly challenges that for robotics, arguing that on-robot inference is impractical for VLA models. The authors point out in Section 1 that "on-robot inference substantially increases power consumption, reducing robot battery life and operational duration by up to 45% [18], resulting in battery replacements and higher operating costs." Furthermore, the weight and cost of high-end GPUs can compromise robot stability and be prohibitive. The contrarian view is that robots should remain "dumb" clients offloading inference to a local edge server over a low-latency network like WiFi 7.

Inter-GPU Disaggregation is Suboptimal

While splitting model stages across multiple GPUs is a standard technique for large models, the paper shows this is highly inefficient for VLA models due to their low GPU utilization. The authors profiled the 𝜋0 model and found that the VLM stage only achieves 52% average tensor core utilization, while the ADiT stage achieves only 15% (Section 2.5). Placing these stages on separate GPUs leaves most compute resources underutilized. The paper demonstrates in Section 5.4 that Robion's intra-GPU disaggregation outperforms all inter-GPU schemes, serving 72 robots within 98% SLO attainment versus 60 for the best-performing inter-GPU scheme.

3. Companies Identified

• BMW: Automotive manufacturer. Relevant because they are "incorporating robotic fleets across manufacturing facilities and warehouses, where robots perform assembly-line tasks" (Section 1).
• Amazon: E-commerce and logistics giant. Relevant because they are deploying robotic fleets in their warehouses (Section 1).
• DHL: Logistics company. Relevant because they are using robotic fleets in their warehouses (Section 1).
• NVIDIA: GPU and AI hardware/software provider. Relevant because their hardware (Jetson AGX Thor, RTX 6000 Pro, H100) and software (CUDA streams, Green Contexts) are used throughout the paper. The paper also references their GR00T N1.5 VLA model (Section 5.1).
• Figure AI: Robotics company. Relevant because their "F.03 arrives at BMW" (Section 1).
• Agility Robotics: Robotics company. Relevant because they "broadens relationship with Amazon" (Section 1).
• Boston Dynamics: Robotics company. Relevant because their "Stretch at DHL" (Section 1).
• Microsoft: Technology company. Relevant because they published a technical report on mobile robotic manipulation workloads cited in the paper (Section 2.2).
• Xiaomi: Technology company. Relevant because they developed the Xiaomi-Robotics-0 and Xiaomi-Robotics-1 VLA models used in the evaluation (Section 5.1).
• Physical Intelligence (PI): AI robotics company. Relevant because they developed the 𝜋0 and 𝜋0.5 VLA models used in the evaluation (Section 5.1).

4. People Identified

• Dionysios Adamopoulos, Nattapol Chanpaisit, Basel Fakhri, Christina Giannoula: Max Planck Institute for Software Systems. The authors of the paper who designed and evaluated the Robion system.
• Kevin Black, Chelsea Finn, Sergey Levine: Physical Intelligence. Key researchers behind the 𝜋0 and 𝜋0.5 VLA models, which are central to the paper's evaluation.
• Linxi "Jim" Fan, Dieter Fox, Jan Kautz, Yuke Zhu: NVIDIA. Key researchers behind the GR00T N1.5 VLA model, another model used in the evaluation.
• Remi Cadene, Thomas Wolf: Hugging Face / SmolVLA. Researchers behind the SmolVLA model, used in the evaluation.
• Wenqi Jiang, Christos Kozyrakis: Max Planck Institute / NVIDIA. Researchers behind ROSA and VLA-Perf, prior works that Robion builds upon and cites.
• Rohan Bansal, Danfei Xu: Researchers behind the Armory scheduler, a prior work on request scheduling for robot factories.
• Yinwei Dai, Ravi Netravali: Researchers behind Kairos, a prior work on scalable serving for physical AI.

5. Operating Insights

Design for the Edge Server, Not the Robot

CTOs and heads of engineering should plan their robotic deployments around a local edge server architecture rather than attempting to integrate high-end GPUs onto robots. The paper provides compelling evidence that on-robot inference is a losing proposition: it reduces battery life by up to 45% and adds prohibitive weight and cost (Section 1). By offloading to an edge server, a single high-end GPU can serve dozens of robots. The authors note in Section 5.2 that "given the cost of Jetson AGX Thor and RTX 6000 Pro is ∼5.5k USD and ∼16k USD, respectively, Robion’s serving capability translates to ∼6.8× cost reductions in hardware equipment."

Co-locate Multiple VLA Models on a Single GPU

VLA models are surprisingly small, typically 1-5 billion parameters, requiring only about 10GB of memory. Dedicating an entire 80-120GB GPU to a single VLA model is a massive waste of resources. Operators should co-locate multiple VLA models on the same GPU to maximize utilization and cost-effectiveness. Robion's design allows a single shared VLM stream and ADiT stream to be reused across all co-located models, and its traffic controller ensures that batching is maximized without overloading the GPU (Section 4.1.3, 4.2.2).

Avoid Continuous Batching for Latency-Critical VLA

Continuous batching, a staple of LLM serving systems, is counterproductive for VLA models. The overhead of inspecting for new requests at every diffusion step is too high relative to the millisecond-scale execution time of VLA stages. Engineers building VLA serving systems should use fixed-size stage batching instead, admitting requests only at the start of a stage to eliminate inspection and packing overheads during execution (Section 4.1.1).

6. Overlooked Insights

Cloud Deployment is Suboptimal for VLA

While cloud deployment is often seen as a flexible alternative to on-premise infrastructure, the paper reveals that it is a poor fit for latency-critical VLA inference. In Section 5.5, the authors state, "cloud deployment achieves substantially lower SLO-AT than edge deployment due to higher network latencies: at 24 robots per-model, cloud deployment achieves 23.6% and 18.4% lower SLO-AT on the 4×RTX 6000 Pro and 4×H100 server, respectively." This implies that companies deploying VLA models at scale will need to invest in local edge infrastructure rather than relying on cloud GPUs.

Initialization Overhead is Acceptable

Robion requires a one-time initialization phase that includes graph capture, profiling, benchmarking, and solver execution. For a complex setup with 8 models on a 4-GPU server, this process takes about 1 hour and 7 minutes (Section 4.3). However, this is a one-time cost: profiling and benchmarking only rerun when the model set or hardware changes, and the solver only reruns when robot populations change. This is an acceptable trade-off for a system that can serve 64 robots with 98% SLO attainment, and operators should not be deterred by this initialization overhead.