1. Chart Parsing and Augmenting Grammars CSE-391: Artificial Intelligence University of Pennsylvania Matt Huenerfauth March 2005

2. Homework 6 Due Friday, April 1, 2005. (Two weeks.) You will write/modify some grammar rules. You will enact some parsing algorithms by hand. You will use the chart-parsing Python code from the textbook (we’ll learn about chart parsing today).

3. Last time: Our grammar S  NP VPVP  V VP  V NP NP  pronoun NP  nounV  verb NP  det adj nounV  aux verb NP  det noun NON-TERMINAL vs. terminal…

4. Last time: Two Parsing Approaches Top Down Parsing –Start with the top-level rule for S. –Blindly try all possible rules to build the sub pieces of the original S rule. Recurse. –When you back up, forget structures you built. Bottom Up Parsing –Start by considering the part of speech of all the words. Build all the possible ‘local’ pieces of structure, and slowly build larger pieces.

5. Last time: Top-down parse [The, old, can, can, hold, the, water] S → NP VP Consider NP… NP → pronoun? pronoun? fail NP → noun? noun? fail NP → det adj noun? det? the adj? old noun? Can Succeed. What about the VP?

6. Last time: Top-down parse [can, hold, the, water] Consider VP… V → verb? verb? fail V → aux verb? aux? can verb? hold succeed fail [the, water]

7. Bottom Up Parsing To illustrate bottom-up parsing, we will draw a chart diagram. For a sentence with k words, we will start with k+1 nodes in the diagram. We put edges between adjacent nodes labeled with input words & part of speech. theoldcan holdthewater detadj noun aux verb noun verb noun det noun verb

8. Drawing an edge… As we apply grammar rules to build a parse, we draw more edges: –An edge has a starting and a stopping point. This represents what input words it covers. –An edge has a label. In the bottom-up algorithm, it looks like this: VP  V NP –This means we just found a V and a NP, and we have combined them to build VP. –We’ll see an example of this on the next slide.

9. Bottom Up Parsing theoldcan holdthewater detadj noun aux verb noun verb noun det noun verb NP  det noun VP  V NP V  aux verb NP  det nounNP  det adj noun VP  V NP VP  VP NP S  NP VP NP  noun V  verb NP  noun V  verb NP  noun V  verb NP  noun V  verb

10. Bottom Up Parsing theoldcan holdthewater detadj noun aux verb noun verb noun det noun verb V  aux verbNP  det nounNP  det adj noun VP  VP NP S  NP VP

11. Three Parsing Algorithms Top Down Parsing –Tries lots of rules, does a lot of backtracking. –Has to re-build structures that it forgets about. –Feels a lot like uninformed search. Bottom Up Parsing –Starts with the small pieces of structure, but it builds everything it can. Don’t need all of it. –Most of it not used in the final complete parse. Top-Down Chart Parsing

12. (Top Down) Chart Parsing Advantages: –Only build those pieces of structure that we actually need to parse the entire sentence. –Remember pieces of structure that we build in a chart so that if we have to backtrack, we don’t have to build those sub-structures all over again.

13. Add one more rule… The first thing we do is add one more rule to our grammar. Our goal will be to produce a parse of the whole sentence that produces an S’: S’  S So, this will be the first rule we’ll try to process with the algorithm…

14. A New Kind of Edge Instead of just drawing edges in our chart diagram for complete structures that we find, now we’ll also add edges that represent incomplete structures that we want to find… For example, an edge labeled: VP  V NP could be used to represent a point in the algorithm where we really want to build a VP, and we have already found a V but have not yet found a NP to complete it. Why would we want such a thing? We’ll see… thecan fall detaux verb noun verb noun

15. How to Start thecan fall detaux verb noun verb noun S’  S Our goal is to build an edge covering the whole sentence labeled S’. So, we first add an edge that covers no words at the start of the sentence. Since we haven’t found any S yet in order to fulfill the S’ rule, then we have to put the dot in front of the S. So, now our mission is to find an S in this sentence so that we can produce an S’. In other words, when we add edges to the chart that contain dots, then we know we are looking for the thing to the right of the dot.

16. The Algorithm When we add an edge to the diagram, we give it a blue label. After we “process” the edge, then the label turns black. We will process edges in the order they are added to the diagram. Process edges like a “queue” (breadth-first feel). Could also use “stack” but this is less traditional. We really want all parses, not just one. How to we “process” an edge? It depends on where the dot is. –Before a NON-TERMINAL? –Before a terminal? –At the end?

17. PREDICT future rules… If the dot is before a NON-TERMINAL. We look up all the rules in the grammar that can produce that NON-TERMINAL. –We add all of these rules to the diagram as edges which cover no words. –We put a dot in front of the first item after the arrow. –Remember: All new edges have blue labels.

18. Dot before NON-TERMINAL: “Predict” thecan fall detaux verb noun verb noun S’  S (1)

19. thecan fall detaux verb noun verb noun S’  S S  NP VP Dot before NON-TERMINAL: “Predict” (1) (2)

20. thecan fall detaux verb noun verb noun S’  S S  NP VP Dot before NON-TERMINAL: “Predict” (1) (2)

21. thecan fall detaux verb noun verb noun S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun Dot before NON-TERMINAL: “Predict” (1) (2) (3) (4) (5) (6)

22. thecan fall detaux verb noun verb noun S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun Dot before NON-TERMINAL: “Predict” (1) (2) (3) (4) (5) (6)

23. SCAN input words… When the dot is before a terminal… Check the next word of input to see if any of its parts of speech matches the terminal symbol. If there is a match, then we: spread the rule to the right and we move the dot to the right as well.

24. thecan fall detaux verb noun verb noun S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun Dot before terminal: “Scan” (1) (2) (3) (4) (5) (6)

25. thecan fall detaux verb noun verb noun S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun Dot before terminal: “Scan” (1) (2) (3) (4) (5) (6)

26. thecan fall detaux verb noun verb noun S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun Dot before terminal: “Scan” (1) (2) (3) (4) (5) (6)

27. thecan fall detaux verb noun verb noun S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun Dot before terminal: “Scan” NP  det noun (1) (2) (3) (4) (5) (6) (7)

28. thecan fall detaux verb noun verb noun S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun Dot before terminal: “Scan” NP  det noun (1) (2) (3) (4) (5) (6) (7)

29. thecan fall detaux verb noun verb noun S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun Dot before terminal: “Scan” NP  det noun NP  det adj noun (1) (2) (3) (4) (5) (6) (7) (8)

30. thecan fall detaux verb noun verb noun S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun Dot before terminal: “Scan” NP  det noun NP  det adj noun (1) (2) (3) (4) (5) (6) (7) (8)

31. thecan fall detaux verb noun verb noun S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun Dot before terminal: “Scan” NP  det noun NP  det adj noun NP  det noun (1) (2) (3) (4) (5) (6) (7) (8) (9)

32. thecan fall detaux verb noun verb noun S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun Dot before terminal: “Scan” NP  det noun NP  det adj noun NP  det noun (1) (2) (3) (4) (5) (6) (7) (8) (9)

33. thecan fall detaux verb noun verb noun S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun Dot before terminal: “Scan” NP  det noun NP  det adj noun NP  det noun (1) (2) (3) (4) (5) (6) (7) (8) (9)

34. Edge Extension When the dot is at the end of the label… Assume current edge is labeled: X  … –Look at all of the edges that ended where the current edge began. –If any of these preceding edges have a label of the form: …  … X … –Then spread this preceding edge across the width of the current edge, and label it: …  … X … (move the dot to the right)

35. thecan fall detaux verb noun verb noun S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun Dot at end: “Edge Extension” NP  det noun NP  det adj noun NP  det noun (1) (2) (3) (4) (5) (6) (7) (8) (9)

36. thecan fall detaux verb noun verb noun S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun Dot at end: “Edge Extension” NP  det noun NP  det adj noun NP  det noun (1) (2) (3) (4) (5) (6) (7) (8) (9)

37. thecan fall detaux verb noun verb noun S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun Dot at end: “Edge Extension” NP  det noun NP  det adj noun NP  det noun S  NP VP (1) (2) (3) (4) (5) (6) (7) (8) (9) (10)

38. thecan fall detaux verb noun verb noun S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun Dot at end: “Edge Extension” NP  det noun NP  det adj noun NP  det noun S  NP VP (1) (2) (3) (4) (5) (6) (7) (8) (9) (10)

39. thecan fall detaux verb noun verb noun S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun Dot at end: “Edge Extension” NP  det noun NP  det adj noun NP  det noun S  NP VP (1) (2) (3) (4) (5) (6) (7) (8) (9) (10)

40. Remember… Process one edge at a time in the order they were added to the diagram. (queue) If the dot is just before: –a NON-TERMINAL: Then “predict.” (grab rules from the grammar) –a terminal: Then “scan.” (compare to POS of next word) –the end of the label: Then “edge extend.” (check preceding edges) Don’t add an edge with same exact label and same exact span as a previous one.

41. thecan fall detaux verb noun verb noun NP  det noun NP  det adj noun NP  det noun S  NP VP (7) (8) (9) (10) S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun (1) (2) (3) (4) (5) (6)

42. thecan fall detaux verb noun verb noun S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun NP  det noun NP  det adj noun NP  det noun S  NP VP (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) VP  V VP  V NP (11) (12)

43. thecan fall detaux verb noun verb noun S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun NP  det noun NP  det adj noun NP  det noun S  NP VP (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) VP  V VP  V NP (11) (12)

44. thecan fall detaux verb noun verb noun S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun NP  det noun NP  det adj noun NP  det noun S  NP VP (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) VP  V VP  V NP V  verb V  aux verb (11) (12) (13) (14)

45. thecan fall detaux verb noun verb noun S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun NP  det noun NP  det adj noun NP  det noun S  NP VP (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) VP  V VP  V NP V  verb V  aux verb (11) (12) (13) (14)

46. thecan fall detaux verb noun verb noun S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun NP  det noun NP  det adj noun NP  det noun S  NP VP (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) VP  V VP  V NP V  verb V  aux verb (11) (12) (13) (14)

47. thecan fall detaux verb noun verb noun S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun NP  det noun NP  det adj noun NP  det noun S  NP VP (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) VP  V VP  V NP V  verb V  aux verb (11) (12) (13) (14) V  verb (15)

48. thecan fall detaux verb noun verb noun S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun NP  det noun NP  det adj noun NP  det noun S  NP VP (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) VP  V VP  V NP V  verb V  aux verb (11) (12) (13) (14) V  verb (15)

49. thecan fall detaux verb noun verb noun S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun NP  det noun NP  det adj noun NP  det noun S  NP VP (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) VP  V VP  V NP V  verb V  aux verb (11) (12) (13) (14) V  verb V  aux verb (15) (16)

50. thecan fall detaux verb noun verb noun S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun NP  det noun NP  det adj noun NP  det noun S  NP VP (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) VP  V VP  V NP V  verb V  aux verb (11) (12) (13) (14) V  verb V  aux verb (15) (16)

51. thecan fall detaux verb noun verb noun S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun NP  det noun NP  det adj noun NP  det noun S  NP VP (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) VP  V VP  V NP V  verb V  aux verb (11) (12) (13) (14) V  verb V  aux verb (15) (16)

52. thecan fall detaux verb noun verb noun S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun NP  det noun NP  det adj noun NP  det noun S  NP VP (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) VP  V VP  V NP V  verb V  aux verb (11) (12) (13) (14) V  verb V  aux verb VP  V VP  V NP (15) (16) (17) (18)

53. thecan fall detaux verb noun verb noun S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun NP  det noun NP  det adj noun NP  det noun S  NP VP (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) VP  V VP  V NP V  verb V  aux verb (11) (12) (13) (14) V  verb V  aux verb VP  V VP  V NP (15) (16) (17) (18)

54. thecan fall detaux verb noun verb noun S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun NP  det noun NP  det adj noun NP  det noun S  NP VP (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) VP  V VP  V NP V  verb V  aux verb (11) (12) (13) (14) V  verb V  aux verb VP  V VP  V NP (15) (16) (17) (18) V  aux verb (19)

55. thecan fall detaux verb noun verb noun S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun NP  det noun NP  det adj noun NP  det noun S  NP VP (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) VP  V VP  V NP V  verb V  aux verb (11) (12) (13) (14) V  verb V  aux verb VP  V VP  V NP (15) (16) (17) (18) V  aux verb (19)

56. thecan fall detaux verb noun verb noun S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun NP  det noun NP  det adj noun NP  det noun S  NP VP (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) VP  V VP  V NP V  verb V  aux verb (11) (12) (13) (14) V  verb V  aux verb VP  V VP  V NP (15) (16) (17) (18) V  aux verb (19) S  NP VP (20)

57. thecan fall detaux verb noun verb noun S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun NP  det noun NP  det adj noun NP  det noun S  NP VP (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) VP  V VP  V NP V  verb V  aux verb (11) (12) (13) (14) V  verb V  aux verb VP  V VP  V NP (15) (16) (17) (18) V  aux verb (19) S  NP VP (20)

58. thecan fall detaux verb noun verb noun S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun NP  det noun NP  det adj noun NP  det noun S  NP VP (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) VP  V VP  V NP V  verb V  aux verb (11) (12) (13) (14) V  verb V  aux verb VP  V VP  V NP (15) (16) (17) (18) V  aux verb (19) S  NP VP (20) NP  pro NP  noun NP  det noun NP  det adj noun (21) (22) (23) (24)

59. thecan fall detaux verb noun verb noun S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun NP  det noun NP  det adj noun NP  det noun S  NP VP (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) VP  V VP  V NP V  verb V  aux verb (11) (12) (13) (14) V  verb V  aux verb VP  V VP  V NP (15) (16) (17) (18) V  aux verb (19) S  NP VP (20) NP  pro NP  noun NP  det noun NP  det adj noun (21) (22) (23) (24)

60. thecan fall detaux verb noun verb noun S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun NP  det noun NP  det adj noun NP  det noun S  NP VP (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) VP  V VP  V NP V  verb V  aux verb (11) (12) (13) (14) V  verb V  aux verb VP  V VP  V NP (15) (16) (17) (18) V  aux verb (19) S  NP VP (20) NP  pro NP  noun NP  det noun NP  det adj noun (21) (22) (23) (24)

61. thecan fall detaux verb noun verb noun S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun NP  det noun NP  det adj noun NP  det noun S  NP VP (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) VP  V VP  V NP V  verb V  aux verb (11) (12) (13) (14) V  verb V  aux verb VP  V VP  V NP (15) (16) (17) (18) V  aux verb (19) S  NP VP (20) NP  pro NP  noun NP  det noun NP  det adj noun (21) (22) (23) (24) VP  V VP  V NP (25) (26)

62. thecan fall detaux verb noun verb noun S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun NP  det noun NP  det adj noun NP  det noun S  NP VP (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) VP  V VP  V NP V  verb V  aux verb (11) (12) (13) (14) V  verb V  aux verb VP  V VP  V NP (15) (16) (17) (18) V  aux verb (19) S  NP VP (20) NP  pro NP  noun NP  det noun NP  det adj noun (21) (22) (23) (24) VP  V VP  V NP (25) (26)

63. thecan fall detaux verb noun verb noun S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun NP  det noun NP  det adj noun NP  det noun S  NP VP (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) VP  V VP  V NP V  verb V  aux verb (11) (12) (13) (14) V  verb V  aux verb VP  V VP  V NP (15) (16) (17) (18) V  aux verb (19) S  NP VP (20) NP  pro NP  noun NP  det noun NP  det adj noun (21) (22) (23) (24) VP  V VP  V NP (25) (26)

64. thecan fall detaux verb noun verb noun S’  S S  NP VP NP  pro NP  noun NP  det noun NP  det adj noun NP  det noun NP  det adj noun NP  det noun S  NP VP (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) VP  V VP  V NP V  verb V  aux verb (11) (12) (13) (14) V  verb V  aux verb VP  V VP  V NP (15) (16) (17) (18) V  aux verb (19) S  NP VP (20) NP  pro NP  noun NP  det noun NP  det adj noun (21) (22) (23) (24) VP  V VP  V NP (25) (26) We’ll stop here for now, but you would continue until you found a S’ edge that covered all the words.

65. Hybrid Approach This use of dots helps us understand what we are looking for – it gives this parsing algorithm its top-down feel. –In other words, we are guided to look for only those syntactic structures that will help build a parse for the whole sentence. The use of a chart diagram gives this algorithm a bottom-up feel (we used a diagram in the bottom-up algorithm).

66. Augmenting a Grammar

67. How Do You Modify a Grammar What if we want to extend a grammar to include a new linguistic structure? S  NP VPVP  V VP  V NP NP  pronoun NP  nounV  verb NP  det adj nounV  aux verb NP  det noun

68. Adding Adverbs How would we augment this grammar to add adverbs and adverb phrases? We’d like to allow sentences like: The can can fall quickly. The can can fall quickly and quietly. The can can quickly fall. Quickly the can can fall.

69. Adding Adverbs How would we augment this grammar to add adverbs and adverb phrases? We’d like to allow sentences like: The can can fall quickly. The can can fall quickly and quietly. The can can quickly fall. Quickly the can can fall.

70. Modify the lexicon… Let’s add the new words we need… –Quickly, quietly, and. What part of speech are they? –Adverb: quickly –Adverb: quietly –Conjunction: and

71. Modify Grammar (1) We’d like to allow single adverbs and adverbs combined with ‘and’ to occur in sentences… –So we add a new kind of NON-TERMINAL to the grammar: an adverb phrase. ADVP ADVP  adv ADVP  adv conj adv –This defines the internal structure of an adverb phrase.

72. Modify Grammar (2) Now that we have specified the internal structure of an adverb phrase, we should add rules to specify how to position adverb phrases within entire sentences. Where do we want to allow adverb phrases to go? –Start of sentence. –End of sentence. –Between aux and verb.

73. Beginning / End of Sentences Modify the rules for a sentence… S  NP VP S  ADVP NP VP S  NP VP ADVP But how to we position adverb phrases between aux’s and verb’s ?

74. Beginning / End of Sentences Modify the rules for a sentence… S  NP VP S  ADVP NP VP S  NP VP ADVP But how to we position adverb phrases between aux’s and verb’s ?

75. Between AUX and VERB We have to edit a different rule to allow adverbs between Auxiliaries and Verbs. V  verb V  aux verb V  aux ADVP verb In between aux and verb…

76. Our New Grammar S  NP VPVP  V S  ADVP NP VPVP  V NP S  NP VP ADVP V  verb NP  pronoun V  aux verb NP  noun V  aux ADVP verb NP  det adj noun NP  det nounADVP  adv ADVP  adv conj adv

77. Our New Lexicon Same entries as before, plus… adv(quickly) adv(quietly) conj(and)

78. Making Changes in Python import nlp.* En = Grammar('En', Rules( S = 'NP VP | ADVP NP VP | NP VP ADVP', NP = 'Pronoun | Noun | Det Noun | Det Adj Noun', VP = 'V | V NP', ADVP = “adv | adv conj adv”, V = 'Verb | Aux Verb | Aux ADVP Verb‘), Lexicon( Noun = "old | can | fall | hold | water", Aux = "can", Verb = "can | hold | fall | water", Adj = "old", Adv = "quickly | happily", Conj = "and", Pronoun = "me | you | I | it", Det = "the | a | an" )) Ec = Chart(En)