Explore and Exploit: Learning Policies for Efficient and Coordinated Active Search

May 2026

Explore and Exploit: Learning Policies for Efficient and Coordinated Active Search

Authors:

Arsh Verma

Abstract:

Robotic search requires deciding where to look next as evidence arrives, while every motion command spends travel distance and mission time. This thesis studies that decision problem under a shared Bayesian active-search formulation and develops learned policies for real-time single-robot field execution and simulated coordination across heterogeneous teams.

The first contribution is a path-aware single-robot policy. We introduce a path-integral expert that scores complete shortest-path routes by their accumulated expected information gain, then amortize that expensive expert into a Graph Attention Network policy by behavior cloning. The resulting policy achieves up to a 254x speedup over the path-integral expert and an 11x speedup over a one-step greedy expert, transfers without retraining across the tested map geometries and object densities, and yields 1.10x to at least 2.86x faster mission completion against a strong NATS baseline in field tests on an autonomous ground vehicle in a 75,000-square-meter forested environment.

The second contribution extends the same shared-belief interface from single-robot search to coordinated teams. We propose Q-learning for Uncertainty-Guided Efficient Search Teams (QUEST), a multi-agent Q-learning framework that trains a related graph-policy family but optimizes a per-decision Bellman target over downstream belief, robot positions, team coverage, and remaining budget. The formulation covers single robots, homogeneous UGV teams, and heterogeneous UAV–UGV teams.

In a pooled evaluation of 1,000 episodes, QUEST reaches a 0.910 F-score with four UGVs versus 0.875 for the strongest learned baseline—a lead that grows with team size. Furthermore, the policy maintains its lead under communication outages and two simultaneous mid-mission robot failures without retraining, and in out-of-distribution UAV–UGV compositions, it improves the early-budget F-score (0.759 vs. 0.728) while using 10% less path length and 12% less duplicate coverage. A map-structure analysis shows that gains on the four-map suite concentrate on constrained, branching maps under binding budgets, while a broader map sweep establishes that algebraic connectivity and modularity are descriptive lenses rather than predictors of performance.

All team results are simulation studies; the field evidence in this thesis is the single-robot deployment. Together, these policies connect real-world field deployment with coordinated team search that generalizes across composition, communication, and partial failure under one shared-belief active-search formulation.
@mastersthesis{Verma-2026-88354,
author = {Arsh Verma},
title = {Explore and Exploit: Learning Policies for Efficient and Coordinated Active Search},
year = {2026},
month = {May},
school = {Carnegie Mellon University},
address = {Pittsburgh, PA},
number = {CMU-RI-TR-26-52},
keywords = {Active Search, Reinforcement Learning, Imitation Learning, Graph Neural Networks, Multi-Robot Systems, Informative Path Planning, Field Robotics},
}
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.