Brain-Aligned Tactile Representations for Dexterous Robot Learning

June 2026

Brain-Aligned Tactile Representations for Dexterous Robot Learning

Authors:

Trinity Chung

Abstract:

Touch is the essential sensory modality through which animals and robots physically negotiate the world. While much of robotic touch focuses on the capabilities of currently available tactile hardware, this thesis asks a more general question: what forms of tactile processing and representation could allow robots to approach the dexterity of animals? This thesis argues that simulated force and torque provide a privileged representation of touch: one that is biologically grounded, computationally scalable, and effective for dexterous behavior.

For biological touch, the rodent vibrissal pathway provides a tractable model system for studying tactile processing. We train task-optimized temporal networks on realistic whisker force/torque sequences to identify shapes and compare their internal representations against recordings from rats' barrel cortex. We find that convolutional recurrent models (ConvRNNs) align very closely with neural data, and contrastive self-supervision with tactile-specific augmentations matches supervised alignment as a label-free proxy.

In robotic touch, we test whether the same class of representation is useful for dexterous behavior. We develop a GPU-parallel tactile sensor simulator that exposes a family of tactile abstractions under one interface, from binary contact and per-taxel force/torque to marker displacement and temperature, fast enough to serve as the front-end for dexterous reinforcement learning. For in-hand manipulation tasks, we find that sensor placement matters more than sensor type or resolution, that whole-hand coverage closes most of the gap to a privileged teacher. Across 3 dexterous tasks, per-taxel force/torque emerges as the most useful and robust observation.

Together, these results argue that force and torque are not merely convenient signals to simulate, but a biologically grounded and behaviorally effective representation of touch. With recurrence and broad spatial coverage, simulated force and torque provide a common representational substrate for understanding tactile computation in brains and building tactile intelligence in robots.

Notes:

@mastersthesis{Chung-2026-88314,
author = {Trinity Chung},
title = {Brain-Aligned Tactile Representations for Dexterous Robot Learning},
year = {2026},
month = {June},
school = {Carnegie Mellon University},
address = {Pittsburgh, PA},
number = {CMU-RI-TR-26-65},
keywords = {Tactile Simulation, NeuroAI, Dexterous Hands},
}
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.