WWW pages of 3rd European Master School on Language and Speech

Improving Evaluation of ASR devices using the Relative Information Loss and whether Stochastic Edit Probabilities Improve the Evaluation Result

Viktoria Maier
(University of Sheffield)

Word error rate (i.e. WER) is the standard evaluation tool in the field. However, especially under noisy conditions with its high error rate, the WER faces problems, which may become substantial to its results. This presentation will introduce another form of evaluation measure based in the concepts of Information Theory. This measure, called the RIL, is defined as H(X|Y)/H(X). It eliminates a whole range of problems of the WER, but also introduces new problems. These problems could be solved by supplying additional information to the RIL. A new performance measure, called the WIL, is featuring an approximation of the possible solution.