July
2026
Reference-Prompted Instance Segmentation for Growing Retail Catalogs
Authors:
Abstract:
Retail and warehouse perception systems work against a catalog that
never stops changing: a segmenter deployed today will eventually be
asked to find products that did not exist when it was trained. A closed-
set segmenter can only grow its vocabulary by widening its classification
head and fine-tuning it, which risks the classes it already handled, or
by retraining on the whole accumulated catalog, which costs more with
every product added. This thesis proposes reference-prompted segmen-
tation, which specifies the target product with a reference prompt, a
fronto-parallel view of the product, rather than a class label or a text de-
scription, so that a product’s identity enters the model as an input rather
than as a slot in its output. We fine-tune a reference-prompted segmenter
one new product at a time — twenty-seven sequential steps growing a
three-product catalog to thirty — using Learning without Forgetting,
and compare it against a closed-set segmenter given the identical recipe,
budget, and data. Our method retains its earlier products substantially
better than the closed-set baseline. Every dataset in this thesis comes
from isaac_datagen, a standalone Isaac Sim synthetic-data engine of-
fered as a reusable contribution in its own right. Under this protocol and
at this scale, the reference-prompted segmenter retains 5× the panoptic
quality of an equally-trained closed-set segmenter after sequentially ab-
sorbing 30 products (0.275 vs. 0.054 PQ), letting a perception system
keep pace with a growing catalog without ever retraining on all of it.
never stops changing: a segmenter deployed today will eventually be
asked to find products that did not exist when it was trained. A closed-
set segmenter can only grow its vocabulary by widening its classification
head and fine-tuning it, which risks the classes it already handled, or
by retraining on the whole accumulated catalog, which costs more with
every product added. This thesis proposes reference-prompted segmen-
tation, which specifies the target product with a reference prompt, a
fronto-parallel view of the product, rather than a class label or a text de-
scription, so that a product’s identity enters the model as an input rather
than as a slot in its output. We fine-tune a reference-prompted segmenter
one new product at a time — twenty-seven sequential steps growing a
three-product catalog to thirty — using Learning without Forgetting,
and compare it against a closed-set segmenter given the identical recipe,
budget, and data. Our method retains its earlier products substantially
better than the closed-set baseline. Every dataset in this thesis comes
from isaac_datagen, a standalone Isaac Sim synthetic-data engine of-
fered as a reusable contribution in its own right. Under this protocol and
at this scale, the reference-prompted segmenter retains 5× the panoptic
quality of an equally-trained closed-set segmenter after sequentially ab-
sorbing 30 products (0.275 vs. 0.054 PQ), letting a perception system
keep pace with a growing catalog without ever retraining on all of it.
Notes:
copied = false, 2000);
">
@mastersthesis{Ke-2026-88338,
author = {Jeffrey Ke},
title = {Reference-Prompted Instance Segmentation for Growing Retail Catalogs},
year = {2026},
month = {July},
school = {Carnegie Mellon University},
address = {Pittsburgh, PA},
number = {CMU-RI-TR-92},
keywords = {instance segmentation, retail, reference-prompting, fine-tuning, less forgetting},
}
author = {Jeffrey Ke},
title = {Reference-Prompted Instance Segmentation for Growing Retail Catalogs},
year = {2026},
month = {July},
school = {Carnegie Mellon University},
address = {Pittsburgh, PA},
number = {CMU-RI-TR-92},
keywords = {instance segmentation, retail, reference-prompting, fine-tuning, less forgetting},
}