1. Discriminative Syntactic Language Modeling for Speech Recognition Michael Collins, Brian Roark Murat, Saraclar MIT CSAIL, OGI/OHSU, Bogazici University 報告者：郝柏翰 2013/02/26 43rd Annual Meeting of the Association for Computational Linguistics

2. Outline Introduction Parse Tree Feature Experiments Conclusion 2

3. Introduction 3 1)SLM in my mind 2)p-value

4. Introduction Word n-gram models have been extremely successful as language models (LMs) for speech recognition. However, modeling long-span dependency can help the language model better predict words. This paper describes a method for incorporating syntactic features into the language model, for reranking approach, using discriminative parameter estimation techniques. 4

5. Introduction Our approach differs from previous work in a couple of important respects. 1)through the feature vector representations we can essentially incorporate arbitrary sources of information from the string or parse tree into the model. We would argue that our method allows considerably more flexibility in terms of the choice of features in the model. 2)second contrast between our work and previous work, is in the use of discriminative parameter estimation techniques. 5

6. Parameter Estimation 6 We are reporting results using just the perceptron algorithm. This has allowed us to explore more of the potential feature space than we would have been able to do using the more costly GCLM estimation techniques.

7. Parse Tree Feature Figure 2 shows a Penn Treebank style parse tree that is of the sort produced by the parser. 7

8. Parse Tree Feature Sequences derived from a parse tree 1)POS-tag sequence: we/PRP helped/VBD her/PRP paint/VB the/DT house/NN 2)Shallow parse tag sequence: we/NP b helped/VP b her/NP b paint/VP b the/NP b house/NP c 3)Shallow parse tag plus POS tag: we/PRP-NP b helped/VBD-VP b her/PRP-NP b paint/VB-VP b the/DT-NP b house/NN-NP c 4)Shallow category with lexical head sequence: we/NP helped/VP her/NP paint/VP house/NP 8

9. P : Parent Node HC : Head Child C : non-head Child 9 + : right - : left 1 : adjacent 2: non-adjacent

10. Experiments The training set consists of 297580 transcribed utterances (3297579 words). The oracle score for the 1000-best lists was 16.7%. 10 n-gram perceptron partition into 28 sets

11. Experiments 11

12. Experiments 12 This yielded 35.2% WER, a reduction of 0.3% absolute over what was achieved with just n-grams, which is significant at p < 0.001, reaching a total reduction of 1.2% over the baseline recognizer.

13. Conclusion The results presented in this paper are a first step in examining the potential utility of syntactic features for discriminative language modeling for speech recognition. The best of which gave a small but significant improvement beyond what was provided by the n-gram features. Future work will include a further investigation of parser- derived features. In addition, we plan to explore the alternative parameter estimation methods 13