Abstract:
Curiosity allows animals and humans to learn through interaction without explicit instruction. Rather than chasing an external reward, a curious agent builds a model of its world and explores through it. This thesis asks how principles of natural curiosity can be translated into embodied agents, and what world models are needed as bodies, action spaces, and environments become more complex. We study this progression across three settings of increasing embodied complexity in simulation, using animal behavior and neural dynamics as both inspiration and evaluation targets.
We first study futility-induced passivity in larval zebrafish, in which an animal whose swimming no longer moves the visual world becomes passive before trying again, a transition carried by a neural--glial circuit that accumulates evidence of futility. We introduce 3M-Progress, a model-memory objective that compares an online world model with a frozen memory of normal action consequences, and train an embodied virtual zebrafish using intrinsic motivation alone. The agent reproduces the active--passive cycling observed in the animal more faithfully than standard curiosity objectives, and its latent dynamics recapitulate the fast neuronal and slow glial timescales of the biological circuit.
We then extend this perspective to walking Drosophila, whose articulated body and richer action repertoire pose a more complex embodied setting, where the question shifts to which action is being prepared. A simulated fly, controlled by a recurrent high-level agent over a fixed low-level central pattern generator, walks on a spherical treadmill that mirrors experiments in which a premotor region predicts a spontaneous turn seconds before it begins. The environment and pipeline are complete, and the remaining experiment tests whether the agent's recurrent activity anticipates future turn direction before movement.
Finally, we ask whether this approach extends to larger, more open-ended environments closer to those in which human curiosity operates, using a social navigation task from egocentric vision as the intended pretraining stage for a harder object-interaction setting. Preliminary results reveal important limitations: a world-model agent does not learn even the navigation task that a model-free agent solves, improving at predicting its observations while its behavior does not, a signature of objective mismatch.
Together, these projects treat world models as the substrate through which curiosity is expressed and assessed. Purpose-built models can reproduce biologically observed behavior and its internal dynamics in a controlled setting, while extending curiosity to richer settings depends on whether the underlying agent can first learn the environment reliably: a result in the zebrafish, an in-progress extension in the fly, and a diagnostic limit in the open-ended case.
We first study futility-induced passivity in larval zebrafish, in which an animal whose swimming no longer moves the visual world becomes passive before trying again, a transition carried by a neural--glial circuit that accumulates evidence of futility. We introduce 3M-Progress, a model-memory objective that compares an online world model with a frozen memory of normal action consequences, and train an embodied virtual zebrafish using intrinsic motivation alone. The agent reproduces the active--passive cycling observed in the animal more faithfully than standard curiosity objectives, and its latent dynamics recapitulate the fast neuronal and slow glial timescales of the biological circuit.
We then extend this perspective to walking Drosophila, whose articulated body and richer action repertoire pose a more complex embodied setting, where the question shifts to which action is being prepared. A simulated fly, controlled by a recurrent high-level agent over a fixed low-level central pattern generator, walks on a spherical treadmill that mirrors experiments in which a premotor region predicts a spontaneous turn seconds before it begins. The environment and pipeline are complete, and the remaining experiment tests whether the agent's recurrent activity anticipates future turn direction before movement.
Finally, we ask whether this approach extends to larger, more open-ended environments closer to those in which human curiosity operates, using a social navigation task from egocentric vision as the intended pretraining stage for a harder object-interaction setting. Preliminary results reveal important limitations: a world-model agent does not learn even the navigation task that a model-free agent solves, improving at predicting its observations while its behavior does not, a signature of objective mismatch.
Together, these projects treat world models as the substrate through which curiosity is expressed and assessed. Purpose-built models can reproduce biologically observed behavior and its internal dynamics in a controlled setting, while extending curiosity to richer settings depends on whether the underlying agent can first learn the environment reliably: a result in the zebrafish, an in-progress extension in the fly, and a diagnostic limit in the open-ended case.