Abstract:
Resilient spatial perception is fundamental for autonomous robots operating in
complex real-world environments. While modern simultaneous localization and
mapping (SLAM) systems achieve high accuracy under controlled conditions,
their performance often degrades when sensing assumptions break due to
environmental degradation, sensor noise, or unexpected dynamics. Visual
sensors are sensitive to lighting variation, motion blur, and occlusion,
while LiDAR sensors may suffer from geometric degeneracy in feature-sparse
environments. These failures can cause rapid error accumulation and lead to
catastrophic localization drift.
This dissertation studies \emph{resilient spatial perception}: the ability of
robotic systems to maintain reliable motion estimation despite sensing
degradation and system uncertainty. The central insight of this work is that
resilient SLAM systems should prioritize \emph{internal continuity} through
inertial sensing while using external observations as adaptive correction
signals. In this paradigm, the inertial measurement unit (IMU) provides
continuous motion awareness, while visual and LiDAR measurements refine the
estimate when reliable observations are available. In other words,
Resilient SLAM = Internal Continuity + External Correction.
Guided by this principle, the dissertation develops a series of techniques
that collectively enable resilient state estimation. First, Super Odometry
introduces a unified IMU-centric sensor fusion framework capable of
integrating heterogeneous sensing modalities. Second, SuperLoc and MSO
introduce uncertainty-aware methods that detect and handle geometric and
visual degradation before catastrophic failure occurs. Third, TartanIMU and
Super Odometry~2.0 enable hierarchical self-recovery by combining adaptive
sensor fusion with learning-based inertial motion estimation when external
sensing becomes unreliable.
To evaluate resilient SLAM systems under realistic conditions, this
dissertation also introduces the SubT-MRS dataset, a large-scale multi-degraded
dataset collected in challenging subterranean environments featuring darkness,
dust, smoke, long corridors, and mixed indoor–outdoor scenarios.
Extensive experiments across these environments demonstrate significant
improvements in robustness under severe sensor degradation and aggressive
robot motion.
Together, these contributions establish a unified framework for resilient
state estimation that enables robots to detect, adapt, and recover from
perception failures. Beyond motion estimation, accurate odometry also
enables persistent scene understanding. SuperMap, a real-time
open-vocabulary 4D spatio-temporal mapping system, builds on the resilient
odometry framework to maintain stable object identities, detect scene
changes, and construct queryable 4D scene graphs that support
language-guided robot navigation in dynamic environments.
complex real-world environments. While modern simultaneous localization and
mapping (SLAM) systems achieve high accuracy under controlled conditions,
their performance often degrades when sensing assumptions break due to
environmental degradation, sensor noise, or unexpected dynamics. Visual
sensors are sensitive to lighting variation, motion blur, and occlusion,
while LiDAR sensors may suffer from geometric degeneracy in feature-sparse
environments. These failures can cause rapid error accumulation and lead to
catastrophic localization drift.
This dissertation studies \emph{resilient spatial perception}: the ability of
robotic systems to maintain reliable motion estimation despite sensing
degradation and system uncertainty. The central insight of this work is that
resilient SLAM systems should prioritize \emph{internal continuity} through
inertial sensing while using external observations as adaptive correction
signals. In this paradigm, the inertial measurement unit (IMU) provides
continuous motion awareness, while visual and LiDAR measurements refine the
estimate when reliable observations are available. In other words,
Resilient SLAM = Internal Continuity + External Correction.
Guided by this principle, the dissertation develops a series of techniques
that collectively enable resilient state estimation. First, Super Odometry
introduces a unified IMU-centric sensor fusion framework capable of
integrating heterogeneous sensing modalities. Second, SuperLoc and MSO
introduce uncertainty-aware methods that detect and handle geometric and
visual degradation before catastrophic failure occurs. Third, TartanIMU and
Super Odometry~2.0 enable hierarchical self-recovery by combining adaptive
sensor fusion with learning-based inertial motion estimation when external
sensing becomes unreliable.
To evaluate resilient SLAM systems under realistic conditions, this
dissertation also introduces the SubT-MRS dataset, a large-scale multi-degraded
dataset collected in challenging subterranean environments featuring darkness,
dust, smoke, long corridors, and mixed indoor–outdoor scenarios.
Extensive experiments across these environments demonstrate significant
improvements in robustness under severe sensor degradation and aggressive
robot motion.
Together, these contributions establish a unified framework for resilient
state estimation that enables robots to detect, adapt, and recover from
perception failures. Beyond motion estimation, accurate odometry also
enables persistent scene understanding. SuperMap, a real-time
open-vocabulary 4D spatio-temporal mapping system, builds on the resilient
odometry framework to maintain stable object identities, detect scene
changes, and construct queryable 4D scene graphs that support
language-guided robot navigation in dynamic environments.
Notes:
copied = false, 2000);
">
@phdthesis{Zhao-2026-88268,
author = {Shibo Zhao},
title = {Resilient Spatial Perception for Autonomous Robots},
year = {2026},
month = {April},
school = {Carnegie Mellon University},
address = {Pittsburgh, PA},
number = {CMU-RI-TR-26-26},
keywords = {Resilient SLAM, Sensor Degradation, IMU-centric, Search and Rescue},
}
author = {Shibo Zhao},
title = {Resilient Spatial Perception for Autonomous Robots},
year = {2026},
month = {April},
school = {Carnegie Mellon University},
address = {Pittsburgh, PA},
number = {CMU-RI-TR-26-26},
keywords = {Resilient SLAM, Sensor Degradation, IMU-centric, Search and Rescue},
}