Manipulation in agricultural and unstructured environments requires acting on objects
that are occluded, variably shaped, and in clutter. Such tasks are contact-rich
and long-horizon: the robot must push aside foliage to reveal targets and reason
about latent or hidden state during interaction. Most deployed systems treat contact
as a hazard and often rely on vision alone, which limits deployment from real-world
field settings. This thesis adopts a different perspective: contact is a strategy for obtaining
information where vision cannot provide. Touch can reveal additional spatial,
temporal, and semantic cues, along with other task relevant latent states.
Sensing alone, however, is insufficient. Under partial observability or POMDP
settings, identical current observation can arise from different hidden states, and endto-
end imitation policies often blend conflicting demonstrations or lose track of earlier
interactions as the horizon grows. This thesis argues that augmenting vision with
contact sensing, and conditioning learned policies on a structured representation of
task-relevant hidden states (inferred from sequential contacts), enables reliable manipulation
where vision alone cannot disambiguate hidden states. In POMDP settings,
we show that structured representations of task-relevant hidden states outperform
end-to-end approaches conditioned on raw observation history.
The dissertation develops this argument in three movements. It first establishes
the limits of vision-only manipulation through a precise contact-rich agricultural task,
where occlusion makes sub-centimeter alignment unreliable without physical interaction.
It then shows that vibrotactile sensing via arrays of contact microphones recovers
both the spatial location and the semantic identity of contacts, complementing
visual priors in occluded canopies. Finally, it demonstrates that conditioning learned
policies on structured hidden-state representations, whether a Bayesian belief over
continuous variables or a discrete stage variable resolving temporal ambiguity, yields
history-aware behavior on contact-rich, long-horizon tasks that end-to-end policies
cannot reliably solve.