Improved Speech Reading through a Free-Parts Representation

July 2005

Improved Speech Reading through a Free-Parts Representation

Authors:

S. Lucey and P. Lucey

Abstract:

Motivated by the success of free-parts based representations in face recognition [1] we have attempted to address some of the problems associated with applying such a philosophy to the task of speaker-independent automatic speech read-ing. Hitherto, a major problem with canonical area-based approaches in automatic speech reading is the intrinsic lack of training observations due to the visual speech modality's low sample rate and large variability in appearance. We believe a free-parts representation can overcome many of these limitations due to its natural ability to generalize by producing many observations from a single mouth image, whilst still preserving the ability to discriminate between various visual-speech units. This approach additionally re-quires a modification to traditional techniques employed for the estimation of hidden Markov Models (HMMs), whose resultant models we currently refer to as free-parts HMMs (FP-HMMs). Results will be presented on the CUAVE audio-visual speech database.
@conference{Lucey-2005-121082,
author = {S. Lucey And P. Lucey},
title = {Improved Speech Reading through a Free-Parts Representation},
booktitle = {Proceedings of Auditory-Visual Speech Processing (AVSP '05)},
year = {2005},
month = {July},
pages = {85 - 86},
}
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.