Rapid prototyping systems demand both adaptability to new components and precision in physical interaction. In this thesis, we explore how geometry can be used in two complementary ways: as a prompt for detecting novel industrial objects, and as a leverage for fine-grained perception in precision tasks.

CAD-Prompted SAM3 enables instance segmentation directly from geometric specification, supporting detection of unseen objects without object-specific training. In CAD-Prompted Manipulation, we extend it to a zero-shot robotic manipulation pipeline by integrating it with pose estimation and grasp generation. Eye-in-Finger incorporates geometric reasoning with tool-integrated sensing, enabling industrial-level perception accuracy using inexpensive commercial hardware.

Together, these works demonstrate how geometry-guided perception provides a scalable approach for rapid prototyping, enabling systems to both adapt to newly introduced components and achieve the precision required for reliable assembly.