1. Ch 5b: Discriminative Training (temporal model) 14.2.2002 Ilkka Aho

2. Abbreviations MCE = Minimum Classification Error MMI = Maximum Mutual Information STLVQ = Shift-Tolerant Learning Vector Quantization TDNN = Time-Delay Neural Network HMM = Hidden Markov Model DP = Dynamic Programming DTW = Dynamic Time Warping GPD = Generalized Probabilistic Descent PBMEC = Prototype-Based Minimum Error Classifier

3. Basics Prototype-based methods use class representatives (sample or an average of samples) to classify new patterns The MCE framework is used for discriminative training (also MMI is possible) A central concern is the design or learning of prototypes that will yield good classification performance

4. STLVQ for Speech Recognition LVQ algorithm in its basic form is a method for static pattern recognition STLVQ handles a stream of dynamically varying patterns (fig. 1.) STLVQ is much simpler than TDNN model, but yielded very good results on the same phoneme recognition tasks

5. Figure 1. STLVQ system architecture.

6. Limitations and Strengths of STLVQ STLVQ assumes only a single phoneme as an input token Training and testing datasets are obtained from manually labeled speech databases How to extend the phoneme recognition to word or sentence recognition? LVQ is applied locally

7. Expanding the Scope of LVQ for Speech Recognition Representation of longer speech sequences such as entire utterances Global optimization Application to continuos speech recognition A need for some kind of time warping or normalization How to merge the discriminative power of LVQ with the sequential modeling abilities of HMMs? Two methods: LVQ-HMM (fig. 2.) and HMM-LVQ (fig. 3.)

8. Figure 2. LVQ-HMM architecture.

9. Figure 3. HMM-LVQ architecture.

10. MCE Interpretation of LVQ A prototype-based implementation of the MCE framework The LVQ classification rule is based on the Euclidean distance between a pattern vector and each category's reference vectors The category of the nearest reference vector is given as the classification decision Figures 4, 5 and 6 demonstrate the smoothness of MCE loss

11. Figure 4. Average empirical loss measured over 10 samples from a one- dimensional, two class classification problem. The ideal zero-one loss is used in calculating the overall loss.

12. Figure 5. Now a sigmoidal MCE loss, a = 0.1, is used in calculating the overall loss.

13. Figure 6. The same situation as in the figure 5. except a = 1.0 now.

14. Prototype-based Methods Using DP DP is used to find the path through a grid of local matches between prototype and test sample frames that has the best overall score When calculating the reference distance between the input utterance and the reference utterance it is more practical to use the top path or the top few paths than every single DP path possible Nonlinear compressing and stretching prototypes DTW is a specific application of DP techniques to speech processing

15. MCE-Trained Prototypes and DTW The idea is to define the MCE loss in terms of a discriminant function that reflects the structure of a straightforward DTW-based recognizer The loss function have to be continous and differentiable that some gradient-based optimization technique (for example GPD) can be used to minimize the overall loss Also the loss function have to reflect classification performance Good results in the Bell Labs E-set task and in phoneme recognition tasks

16. PBMEC PBMEC models prototypes at a finer grain than MCE-trained DTW PBMEC prototypes are modeled within phonetic or subphonetic states Word models are formed by connecting different states together Multi-state PBMEC (fig. 7.) The discriminant function for a category is defined as the final accumulated score of the best DP path for that category (fig. 8.) MCE-GPD update rule for PBMEC pulls the nearest reference vectors for the correct category closer to the input and pushes the nearest reference vectors for the incorrect category away MCE-GPD in the context of speech recognition using phoneme models (fig. 9.)

17. Figure 7. Multi-state PBMEC architecture.

18. Figure 8. Final DP score.

19. Figure 9. DP segmentations for the words ”aida” and ”taira”.

20. HMM design based on MCE The prototype-like nature of HMMs The MCE framework can be applied to HMMs in a very same way that in the case of the PBMEC model HMM state likelihood and discriminant function MCE misclassification measure and loss Calculating of MCE Gradient for HMMs There are a very large number of applications of MCE-trained HMMs Some of the best context-independent results have been reported for the Texas Instruments-Massachusetts Institute of Technology database

21. Homework Question STLVQ Prototype-based DP (DTW technique) HMM design based on MCE Explain the main differencies between following methods in speech recognition: