1. THE CYK PARSING METHOD (2) Cornelia Kempa Carla Parra Escartín WS 2006-2007

2. Tübingen, 4th December 2006 2 AGENDA 1. Building the recognition table 2. Undoing the effect of the CNF transformation 2.1. Overview 2.2. Adding removed elements 2.3. Parsing 2.3.1. Methodology 2.3.2. Left-most derivation 3. A short retrospective of CYK 4. Chart Parsing

3. Tübingen, 4th December 2006 3 AGENDA 1. Building the recognition table 2. Undoing the effect of the CNF transformation 2.1. Overview 2.2. Adding removed elements 2.3. Parsing 2.3.1. Methodology 2.3.2. Left-most derivation 3. A short retrospective of CYK 4. Chart Parsing

4. Tübingen, 4th December 2006 4 2. Undoing the effect of the CNF transformation 2.1. Overview (1)

5. Tübingen, 4th December 2006 5 2. Undoing the effect of the CNF transformation 2.1. Overview (2)

6. Tübingen, 4th December 2006 6 2. Undoing the effect of the CNF transformation 2.1. Overview (3)

7. Tübingen, 4th December 2006 7 AGENDA 1. Building the recognition table 2. Undoing the effect of the CNF transformation 2.1. Overview 2.2. Adding removed elements 2.3. Parsing 2.3.1. Methodology 2.3.2. Left-most derivation 3. A short retrospective of CYK 4. Chart Parsing

8. Tübingen, 4th December 2006 8 2.2. Adding removed elements (1) 2. Undoing the effect of the CNF transformation

9. Tübingen, 4th December 2006 9 CYK does not require all non-terminals reachable. Rule added!!! + 2.2. Adding removed elements (2) 2. Undoing the effect of the CNF transformation

10. Tübingen, 4th December 2006 10 New line 2. Undoing the effect of the CNF transformation 2.2. Adding removed elements (3)

11. Tübingen, 4th December 2006 11 Set R si, 0  non-terminals that can be considered as possibly occurring just in front of symbol z i Set R sn +1,0  non-terminals that can occur at the end of the sentence. 2.2. Adding removed elements (4) 2. Undoing the effect of the CNF transformation

12. Tübingen, 4th December 2006 12 AGENDA 1. Building the recognition table 2. Undoing the effect of the CNF transformation 2.1. Overview 2.2. Adding removed elements 2.3. Parsing 2.3.1. Methodology 2.3.2. Left-most derivation 3. A short retrospective of CYK 4. Chart Parsing

13. Tübingen, 4th December 2006 13 Derivation starts with start-symbol S. Is A 1 A 2 ···A m a right hand side of S?  Does A 1 A 2 ···A m derive S 1,n ?  We check this starting with A 1  2 cases: 2. Undoing the effect of the CNF transformation 2.3. Parsing 2.3.1. Methodology (1)

14. Tübingen, 4th December 2006 14 A) A 1 is a terminal symbol  it must be the first symbol of s 1,n  then check if A 2 ···A m derives s 2,n-1 B) A1 is a non-terminal symbol  it must be a member of R s1,k  then check if A 2 ···A m derives s 2,k+1,n-k 2. Undoing the effect of the CNF transformation 2.3. Parsing 2.3.1. Methodology (2)

15. Tübingen, 4th December 2006 15 Now: determined whether the rule is applicable  if so, which parts of the rule derive which substrings.  Next step: determine how the substrings can be derived. 2. Undoing the effect of the CNF transformation 2.3. Parsing 2.3.1. Methodology (3)

16. Tübingen, 4th December 2006 16 AGENDA 1. Building the recognition table 2. Undoing the effect of the CNF transformation 2.1. Overview 2.2. Adding removed elements 2.3. Parsing 2.3.1. Methodology 2.3.2. Left-most derivation 3. A short retrospective of CYK 4. Chart Parsing

17. Tübingen, 4th December 2006 17 2. Undoing the effect of the CNF transformation 2.3. Parsing 2.3.2. Left-most derivation (1)

18. Tübingen, 4th December 2006 18 2. Undoing the effect of the CNF transformation 2.3. Parsing 2.3.2. Left-most derivation (2)

19. Tübingen, 4th December 2006 19 This is what we saw so far: If we continue... The derivation is: 2. Undoing the effect of the CNF transformation 2.3. Parsing 2.3.2. Left-most derivation (3)

20. Tübingen, 4th December 2006 20 2. Undoing the effect of the CNF transformation 2.3. Parsing 2.3.2. Left-most derivation (4)

21. Tübingen, 4th December 2006 21 AGENDA 1. Building the recognition table 2. Undoing the effect of the CNF transformation 2.1. Overview 2.2. Adding removed elements 2.3. Parsing 2.3.1. Methodology 2.3.2. Left-most derivation 3. A short retrospective of CYK 4. Chart Parsing

22. Tübingen, 4th December 2006 22 First: recognition table using the original grammar. Then: transforming grammar to CNF. 3. A short retrospective of CYK (1)

23. Tübingen, 4th December 2006 23 For efficiency, CNF is a bit too restrictive Disadvantage of CNF: Resulting recognition table lacks the information we need to construct a derivation using the original grammar! 3. A short retrospective of CYK (2)

24. Tübingen, 4th December 2006 24 In the transformation process, some non-terminals were thrown away (non-productive) Missing information could be added. 3. A short retrospective of CYK (3)

25. Tübingen, 4th December 2006 25 Result: almost the same recognition table. Extra information on non-terminals Obtained in a simpler and much more efficient way. 3. A short retrospective of CYK (2)

26. Tübingen, 4th December 2006 26 AGENDA 1. Building the recognition table 2. Undoing the effect of the CNF transformation 2.1. Overview 2.2. Adding removed elements 2.3. Parsing 2.3.1. Methodology 2.3.2. Left-most derivation 3. A short retrospective of CYK 4. Chart Parsing

27. Tübingen, 4th December 2006 27 CYK algorithm also known as chart parsing. Both techniques  variants (some of them identical among the techniques) Difference: implementation. 4. Chart parsing (1)

28. Tübingen, 4th December 2006 28 A chart is just a recognition table. 4. Chart parsing (2)

29. Tübingen, 4th December 2006 29 FINAL OVERVIEW (1) 1. Building the recognition table 2. Possible Derivations 3. Undoing the effect of the CNF transformation. 4. A short retrospective of CYK 5. Chart Parsing

30. Tübingen, 4th December 2006 30 PARSING METHODS Non-directional methods Top-downBottom-up Unger ParserCYK Parser FINAL OVERVIEW (2)

31. THANK YOU VERY MUCH