Evaluating Tracking Accuracy of an Automatic Reading Tutor

August 2011

Evaluating Tracking Accuracy of an Automatic Reading Tutor

Authors:

Morten Højfeldt Rasmussen, Jack Mostow, Zheng-Hua Tan, Børge Lindberg, and Yuanpeng Li

Abstract:

In automatic reading tutoring, tracking is the process of automatically following a reader through a given target text. When developing tracking algorithms, a measure of the tracking accuracy–how often a spoken word is aligned to the right target text word position–is needed in order to evaluate performance and compare different algorithms. This paper presents a framework for determining the observed tracking error rate. The proposed framework is used to evaluate three tracking strategies: A) follow the reader to whichever word he/she jumps to in the text, B) follow the reader monotonically from left to right ignoring word skips and regressions (going back to a previous text word), and C) the same as B but allowing isolated word skips. Observed tracking error rate for each of the three tracking strategies is: A: 53%, B: 56%, and C: 47%, on 1883 utterances from 25 children.
@workshop{Rasmussen-2011-122077,
author = {Morten Højfeldt Rasmussen And Jack Mostow And Zheng-Hua Tan And Børge Lindberg And Yuanpeng Li},
title = {Evaluating Tracking Accuracy of an Automatic Reading Tutor},
booktitle = {Proceedings of ISCA International Workshop on Speech and Language Technology in Education (SLaTE '11)},
year = {2011},
month = {August},
pages = {17 - 20},
}
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.