Smart Sight: A Tourist Assistant System

October 1999

Smart Sight: A Tourist Assistant System

Authors:

Jie Yang, Weiyi Yang, Matthias Denecke, and Alex Waibel

Abstract:

In this paper, we present our efforts towards developing an intelligent tourist system. The system is equipped with a unique combination of sensors and software. The hardware includes two computers, a GPS receiver, a lapel microphone plus an earphone, a video camera and a head-mounted display. This combination includes a multimodal interface to take advantage of speech and gesture input to provide assistance for a tourist. The software supports natural language processing, speech recognition, machine translation, handwriting recognition and multimodal fusion. A vision module is trained to locate and read written language, is able to adapt to to new environments, and is able to interpret intentions offered by the user such as a spoken clarification or pointing gesture. We illustrate the applications of the system using two examples.
@conference{Yang-1999-15035,
author = {Jie Yang And Weiyi Yang And Matthias Denecke And Alex Waibel},
title = {Smart Sight: A Tourist Assistant System},
booktitle = {Proceedings of 3rd International Symposium on Wearable Computers (ISWC '99)},
year = {1999},
month = {October},
pages = {73 - 78},
}
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.