Despite recent advances in policy learning, long-horizon manipulation remains difficult because learned policies must avoid compounding errors while preserving future feasibility. While search-based planning can explicitly reason over future consequences, it becomes expensive in high-dimensional continuous action spaces. Classical Task and Motion Planning methods address this by introducing symbolic objects, relations, and abstractions for discretization, but these representations are difficult to define reliably from partial real-world observations. This thesis aims to address this gap by asking: How can learning and search be integrated directly over continuous actions? We study this question in multi-object rearrangement tasks, such as packing a constrained shelf or organizing dinnerware on a table, where the robot must execute multiple pick-and-place actions while preserving downstream task feasibility.

We first introduce SPOT (Search over Point cloud Object Transformations), a system that combines search with learning at test time. SPOT formulates object rearrangement as A* search over object-wise SE(3) transformations from partially observed point clouds. Learned suggesters guide which object to move and where to move it, while a learned model-deviation estimator biases search toward executable transitions. We then introduce DISCO (Distilled Tree Search over Continuous actions), a system that combines search with learning at train time. DISCO uses progressive-widening Monte Carlo Tree Search (MCTS) with a learned policy and value function in GPU-parallelized simulation. MCTS-generated plans are distilled back into the policy-value model, enabling direct execution without search at test time, and improving action proposals in subsequent searches. Finally, this thesis shows that learning and search can be combined in complementary ways: learned models make continuous-action search possible at test time (SPOT), while search at train time can extend learned models to longer-horizon tasks (DISCO).