Abstract:
Event cameras are high-speed, high-dynamic-range sensors whose microsecond
temporal resolution and 120+ dB dynamic range make them
essential for agile perception and SLAM systems operating in challenging
lighting conditions. Despite this potential, reliable deployment of event
cameras remains elusive due to two practical barriers: the hardware setup
process requires careful lens selection, calibration, and bias parameter
tuning that is poorly documented compared to conventional cameras; and
existing event datasets are too small and insufficiently diverse to train
generalizable perception models.
This thesis addresses both barriers in the context of deploying event cameras
on the TartanStar multi-modal sensor payload, an existing platform
developed by collaborators at CMU; neither TartanStar nor TartanAir-V2
are contributions of this thesis. First, we document a systematic
deployment pipeline for the Prophesee EVK4, covering lens and filter
selection, intrinsic calibration using both the Metavision SDK and
E2Calib, and a characterization of the key bias parameters and their
application-specific tuning. Second, we find that state-of-the-art optical
flow models trained on existing real-world event datasets fail to generalize
to new environments, motivating the need for more diverse training
data. We propose that synthetic event data can serve as an effective
pre-training substrate and demonstrate this using TartanAir-V2’s native
event camera modality, which renders high-fidelity event streams across
50+ diverse environments without requiring external simulation tools.
Training E-RAFT on this synthetic data followed by fine-tuning on real
data improves optical flow performance over fine-tuning on real data alone
on both the DSEC and MVSEC benchmarks, confirming that
TartanAir-V2’s event data can bridge the real-world generalization gap.
Together, these contributions lay the foundation for reliable deployment
of event-based perception systems.
temporal resolution and 120+ dB dynamic range make them
essential for agile perception and SLAM systems operating in challenging
lighting conditions. Despite this potential, reliable deployment of event
cameras remains elusive due to two practical barriers: the hardware setup
process requires careful lens selection, calibration, and bias parameter
tuning that is poorly documented compared to conventional cameras; and
existing event datasets are too small and insufficiently diverse to train
generalizable perception models.
This thesis addresses both barriers in the context of deploying event cameras
on the TartanStar multi-modal sensor payload, an existing platform
developed by collaborators at CMU; neither TartanStar nor TartanAir-V2
are contributions of this thesis. First, we document a systematic
deployment pipeline for the Prophesee EVK4, covering lens and filter
selection, intrinsic calibration using both the Metavision SDK and
E2Calib, and a characterization of the key bias parameters and their
application-specific tuning. Second, we find that state-of-the-art optical
flow models trained on existing real-world event datasets fail to generalize
to new environments, motivating the need for more diverse training
data. We propose that synthetic event data can serve as an effective
pre-training substrate and demonstrate this using TartanAir-V2’s native
event camera modality, which renders high-fidelity event streams across
50+ diverse environments without requiring external simulation tools.
Training E-RAFT on this synthetic data followed by fine-tuning on real
data improves optical flow performance over fine-tuning on real data alone
on both the DSEC and MVSEC benchmarks, confirming that
TartanAir-V2’s event data can bridge the real-world generalization gap.
Together, these contributions lay the foundation for reliable deployment
of event-based perception systems.