July
2026
Hierarchical Manipulation Policies: Adapting to Unseen Objects and Discovering Sub-goals
Authors:
Abstract:
A robot that manipulates one object well may still fail on the next. Generalizing across diverse objects and tasks is hard because such objects vary widely in geometry, articulation, and interaction dynamics. Hierarchical policies offer a powerful approach, where a high-level policy predicts sub-goal end-effector poses and a low-level policy generates the actions to reach them. However, dominant approaches leave the high-level policy brittle to unseen objects and dependent on deterministic sub-goal heuristics that many tasks cannot provide. This thesis asks the following question. \emph{How should sub-goals be defined, represented, and communicated from the high-level to the low-level policy so that a single hierarchical policy generalizes across diverse objects and tasks?} We address this question through two complementary projects, both building on a prior hierarchical policy that grounds 3D sub-goal prediction in the observed scene.
We first present demonstration-conditioned learning for adapting to out-of-distribution objects. Rather than fine-tuning, the pre-trained policy is conditioned on a single additional demonstration provided at test time, without any weight updates. We show that reasoning about the demonstration and the current observation jointly in 3D outperforms compressing the demonstration into a latent embedding, and that a single human hand demonstration, captured in two RGB-D keyframes, can replace a teleoperated robot trajectory. In simulation, a single demonstration improves normalized opening performance on previously low-performing objects by 12.4\%. On two real robot platforms, it increases grasp success from 74.0\% to 86.6\% on tabletop objects and from 33.0\% to 83.3\% on mobile-manipulator objects in real kitchens and lounges.
We then present an uncertainty-aware hierarchical framework for tasks where sub-goals cannot be deterministically defined. Common heuristics, such as gripper open/close transitions or near-zero end-effector velocity, provide no signal for non-prehensile tasks such as pushing, sliding, or manipulating levers and handles without a discrete grasp event. Near a sub-goal transition, nearly identical observations map to abruptly different goals, forcing a unimodal high-level policy to place an arbitrary decision boundary whose errors cause goal averaging, premature switching, or stalling. Our framework instead derives candidate sub-goals through Bayesian online changepoint detection, represents the high-level prediction as a mixture model over candidate sub-goals whose weights shift smoothly through transitions, and conditions the low-level policy on this distribution through weighted, goal-aware cross-attention.
Together, the two projects support a common conclusion, that the sub-goal interface determines how far a hierarchical policy generalizes. Grounding sub-goals in the observed 3D scene enables adaptation to unseen objects from a single demonstration, and representing sub-goals as distributions rather than points extends hierarchical policies to tasks that cannot be deterministically segmented.
We first present demonstration-conditioned learning for adapting to out-of-distribution objects. Rather than fine-tuning, the pre-trained policy is conditioned on a single additional demonstration provided at test time, without any weight updates. We show that reasoning about the demonstration and the current observation jointly in 3D outperforms compressing the demonstration into a latent embedding, and that a single human hand demonstration, captured in two RGB-D keyframes, can replace a teleoperated robot trajectory. In simulation, a single demonstration improves normalized opening performance on previously low-performing objects by 12.4\%. On two real robot platforms, it increases grasp success from 74.0\% to 86.6\% on tabletop objects and from 33.0\% to 83.3\% on mobile-manipulator objects in real kitchens and lounges.
We then present an uncertainty-aware hierarchical framework for tasks where sub-goals cannot be deterministically defined. Common heuristics, such as gripper open/close transitions or near-zero end-effector velocity, provide no signal for non-prehensile tasks such as pushing, sliding, or manipulating levers and handles without a discrete grasp event. Near a sub-goal transition, nearly identical observations map to abruptly different goals, forcing a unimodal high-level policy to place an arbitrary decision boundary whose errors cause goal averaging, premature switching, or stalling. Our framework instead derives candidate sub-goals through Bayesian online changepoint detection, represents the high-level prediction as a mixture model over candidate sub-goals whose weights shift smoothly through transitions, and conditions the low-level policy on this distribution through weighted, goal-aware cross-attention.
Together, the two projects support a common conclusion, that the sub-goal interface determines how far a hierarchical policy generalizes. Grounding sub-goals in the observed 3D scene enables adaptation to unseen objects from a single demonstration, and representing sub-goals as distributions rather than points extends hierarchical policies to tasks that cannot be deterministically segmented.