Beyond Vision-Language-Action Models: Adapting, Steering, and Accelerating Generalist Robot Policies

July 2026

Beyond Vision-Language-Action Models: Adapting, Steering, and Accelerating Generalist Robot Policies

Authors:

Sungjae Park

Abstract:

Generalist robot policies, vision-language-action models that pair a large pretrained vision-language backbone with a diffusion or flow-matching action head, are increasingly capable, yet hard to deploy in the real world. Three gaps separate such a policy from a deployable one: a data gap (adapting to a new task still demands task-specific teleoperation data), an inference gap (the policy samples its action distribution with no control over how conservative or diverse the result is), and an architecture gap (a large policy is too slow to replan often, so it must run its action chunks open-loop and cannot react mid-motion). This thesis argues that these gaps can be closed not by training larger models on more robot data, but by changing how a pretrained policy generates its actions, with little or no additional training.

We develop three methods. DemoDiffusion (data) imitates a single human demonstration instead of collecting teleoperation data: it retargets the human hand motion into a coarse robot trajectory, then uses a frozen generalist diffusion policy to refine it into plausible robot actions. It needs no task-specific or paired human-robot data, and succeeds even where the base policy fails outright. Temporal Score Rescaling (inference) rescales the learned score/flow at inference to draw from a sharper or broader distribution than the model was trained on; it is training-free, works with any diffusion or flow model, and improves image generation, depth, pose, and protein design alongside robot policies. πR² (architecture, inference) builds on diffusion forcing to split conditioning into a fast proprioceptive channel and a slow, asynchronously updated vision-language channel, so the policy reacts to fresh proprioception while tolerating stale vision. A latency-adaptive schedule lets a single model handle varying inference latency and emit actions in a single denoising step, making a large policy reactive and real-time, several times faster than the original.

Together, these methods adapt, steer, and accelerate a pretrained policy, taking vision-language-action models beyond what they can do as trained and toward real-world deployment.
@mastersthesis{Park-2026-88328,
author = {Sungjae Park},
title = {Beyond Vision-Language-Action Models: Adapting, Steering, and Accelerating Generalist Robot Policies},
year = {2026},
month = {July},
school = {Carnegie Mellon University},
address = {Pittsburgh, PA},
number = {CMU-RI-TR-26-73},
keywords = {vision-language-action model, generalist robot policy, diffusion policy, flow matching, imitation learning, real-time control, temperature sampling},
}
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.