Parsing with Context Free Grammars CSC 9010 Natural Language Processing Paula Matuszek and Mary-Angela Papalaskari This slide set was adapted from: Jim.

Slides:



Advertisements
Similar presentations
March 1, 2009Dr. Muhammed Al_mulhem1 ICS482 Parsing Chapter 13 Muhammed Al-Mulhem March 1, 2009.
Advertisements

Basic Parsing with Context-Free Grammars CS 4705 Julia Hirschberg 1 Some slides adapted from Kathy McKeown and Dan Jurafsky.
Natural Language Processing - Parsing 1 - Language, Syntax, Parsing Problems in Parsing Ambiguity, Attachment / Binding Bottom vs. Top Down Parsing.
May 2006CLINT-LN Parsing1 Computational Linguistics Introduction Approaches to Parsing.
PARSING WITH CONTEXT-FREE GRAMMARS
Parsing with Context Free Grammars Reading: Chap 13, Jurafsky & Martin
March 1, 2009 Dr. Muhammed Al-Mulhem 1 ICS 482 Natural Language Processing Probabilistic Context Free Grammars (Chapter 14) Muhammed Al-Mulhem March 1,
CKY Parsing Ling 571 Deep Processing Techniques for NLP January 12, 2011.
Artificial Intelligence 2004 Natural Language Processing - Syntax and Parsing - Language, Syntax, Parsing Problems in Parsing Ambiguity, Attachment.
1 Earley Algorithm Chapter 13.4 October 2009 Lecture #9.
 Christel Kemke /08 COMP 4060 Natural Language Processing PARSING.
Parsing context-free grammars Context-free grammars specify structure, not process. There are many different ways to parse input in accordance with a given.
Parsing with CFG Ling 571 Fei Xia Week 2: 10/4-10/6/05.
Earley’s algorithm Earley’s algorithm employs the dynamic programming technique to address the weaknesses of general top-down parsing. Dynamic programming.
Basic Parsing with Context- Free Grammars 1 Some slides adapted from Julia Hirschberg and Dan Jurafsky.
Syntactic Parsing with CFGs CMSC 723: Computational Linguistics I ― Session #7 Jimmy Lin The iSchool University of Maryland Wednesday, October 14, 2009.
Artificial Intelligence 2004 Natural Language Processing - Syntax and Parsing - Language Syntax Parsing.
Parsing SLP Chapter 13. 7/2/2015 Speech and Language Processing - Jurafsky and Martin 2 Outline  Parsing with CFGs  Bottom-up, top-down  CKY parsing.
Basic Parsing with Context- Free Grammars 1 Some slides adapted from Julia Hirschberg and Dan Jurafsky.
1 Basic Parsing with Context Free Grammars Chapter 13 September/October 2012 Lecture 6.
11 CS 388: Natural Language Processing: Syntactic Parsing Raymond J. Mooney University of Texas at Austin.
PARSING David Kauchak CS457 – Fall 2011 some slides adapted from Ray Mooney.
1 Basic Parsing with Context- Free Grammars Slides adapted from Dan Jurafsky and Julia Hirschberg.
Intro to NLP - J. Eisner1 Earley’s Algorithm (1970) Nice combo of our parsing ideas so far:  no restrictions on the form of the grammar:  A.
1 CPE 480 Natural Language Processing Lecture 5: Parser Asst. Prof. Nuttanart Facundes, Ph.D.
CS 4705 Parsing More Efficiently and Accurately. Review Top-Down vs. Bottom-Up Parsers Left-corner table provides more efficient look- ahead Left recursion.
中文信息处理 Chinese NLP Lecture 9.
1 CKY and Earley Algorithms Chapter 13 October 2012 Lecture #8.
LINGUISTICA GENERALE E COMPUTAZIONALE ANALISI SINTATTICA (PARSING)
10. Parsing with Context-free Grammars -Speech and Language Processing- 발표자 : 정영임 발표일 :
Chapter 13: Parsing with Context-Free Grammars Heshaam Faili University of Tehran.
May 2006CLINT-LN Parsing1 Computational Linguistics Introduction Parsing with Context Free Grammars.
October 2005csa3180: Parsing Algorithms 11 CSA350: NLP Algorithms Sentence Parsing I The Parsing Problem Parsing as Search Top Down/Bottom Up Parsing Strategies.
Parsing I: Earley Parser CMSC Natural Language Processing May 1, 2003.
PARSING David Kauchak CS159 – Spring 2011 some slides adapted from Ray Mooney.
6/2/2016CPSC503 Winter CPSC 503 Computational Linguistics Lecture 9 Giuseppe Carenini.
Lecture 1, 7/21/2005Natural Language Processing1 CS60057 Speech &Natural Language Processing Autumn 2007 Lecture August 2007.
1 Chart Parsing Allen ’ s Chapter 3 J & M ’ s Chapter 10.
Chart Parsing and Augmenting Grammars CSE-391: Artificial Intelligence University of Pennsylvania Matt Huenerfauth March 2005.
Sentence Parsing Parsing 3 Dynamic Programming. Jan 2009 Speech and Language Processing - Jurafsky and Martin 2 Acknowledgement  Lecture based on  Jurafsky.
Natural Language - General
PARSING 2 David Kauchak CS159 – Spring 2011 some slides adapted from Ray Mooney.
NLP. Introduction to NLP Motivation –A lot of the work is repeated –Caching intermediate results improves the complexity Dynamic programming –Building.
Quick Speech Synthesis CMSC Natural Language Processing April 29, 2003.
CS 4705 Lecture 10 The Earley Algorithm. Review Top-Down vs. Bottom-Up Parsers –Both generate too many useless trees –Combine the two to avoid over-generation:
csa3050: Parsing Algorithms 11 CSA350: NLP Algorithms Parsing Algorithms 1 Top Down Bottom-Up Left Corner.
CS 4705 Lecture 7 Parsing with Context-Free Grammars.
Natural Language Processing Lecture 15—10/15/2015 Jim Martin.
Instructor: Nick Cercone CSEB - 1 Parsing and Context Free Grammars Parsers, Top Down, Bottom Up, Left Corner, Earley.
October 2005CSA3180: Parsing Algorithms 21 CSA3050: NLP Algorithms Parsing Algorithms 2 Problems with DFTD Parser Earley Parsing Algorithm.
PARSING David Kauchak CS159 – Fall Admin Assignment 3 Quiz #1  High: 36  Average: 33 (92%)  Median: 33.5 (93%)
Speech and Language Processing SLP Chapter 13 Parsing.
Parsing with Context Free Grammars. Slide 1 Outline Why should you care? Parsing Top-Down Parsing Bottom-Up Parsing Bottom-Up Space (an example) Top -
CSC 594 Topics in AI – Natural Language Processing
Parsing Recommended Reading: Ch th Jurafsky & Martin 2nd edition
CS60057 Speech &Natural Language Processing
Basic Parsing with Context Free Grammars Chapter 13
Parsing Recommended Reading: Ch th Jurafsky & Martin 2nd edition
CSCI 5832 Natural Language Processing
CPSC 503 Computational Linguistics
CSCI 5832 Natural Language Processing
Lecture 14: Grammar and Parsing (II) November 11, 2004 Dan Jurafsky
Natural Language - General
Earley’s Algorithm (1970) Nice combo of our parsing ideas so far:
Parsing and More Parsing
CSCI 5832 Natural Language Processing
CPSC 503 Computational Linguistics
CSA2050 Introduction to Computational Linguistics
David Kauchak CS159 – Spring 2019
NLP.
Presentation transcript:

Parsing with Context Free Grammars CSC 9010 Natural Language Processing Paula Matuszek and Mary-Angela Papalaskari This slide set was adapted from: Jim Martin (after Dan Jurafsky) from U. Colorado Rada Mihalcea, University of North Texas Robert Berwick, MIT Bonnie Dorr, University of Maryland

Slide 1 Parsing Mapping from strings to structured representation Parsing with CFGs refers to the task of assigning correct trees to input strings Correct here means a tree that covers all and only the elements of the input and has an S at the top It doesn’t actually mean that the system can select the correct tree from among the possible trees As with everything of interest, parsing involves a search which involves the making of choices We’ll start with some basic methods before moving on to more complex ones

Slide 1 Programming languages max = min = grade; // Read and process the rest of the grades while (grade >= 0) { count++; sum += grade; if (grade > max) max = grade; else if (grade < min) min = grade; System.out.print ("Enter the next grade (-1 to quit): "); grade = Keyboard.readInt (); } Easy to parse Designed that way!

Slide 1 Natural Languages max = min = grade; // Read and process the rest of the grades while (grade >= 0) count++; sum += grade; if (grade > max) max = grade; else if (grade < min) min = grade; System.out.print ("Enter the next grade (-1 to quit): "); grade = Keyboard.readInt ();} No {} ( ) [ ] to indicate scope and precedence Lots of overloading (arity varies) Grammar isn’t known in advance! Context-free grammar is not the best formalism

Slide 1 Some assumptions.. You have all the words already in some buffer The input isn’t pos tagged We won’t worry about morphological analysis All the words are known

Slide 1 Top-Down Parsing Since we’re trying to find trees rooted with an S (Sentences) start with the rules that give us an S. Then work your way down from there to the words.

Slide 1 Top Down Space

Slide 1 Bottom-Up Parsing Of course, we also want trees that cover the input words. So start with trees that link up with the words in the right way. Then work your way up from there.

Slide 1 Bottom-Up Space

Slide 1 Top-Down VS. Bottom-Up Top-down – Only searches for trees that can be answers – But suggests trees that are not consistent with the words – Guarantees that tree starts with S as root – Does not guarantee that tree will match input words Bottom-up – Only forms trees consistent with the words – Suggest trees that make no sense globally – Guarantees that tree matches input words – Does not guarantee that parse tree will lead to S as a root Combine the advantages of the two by doing a search constrained from both sides (top and bottom)

Slide 1 Top-Down, Depth-First, Left-to-Right Search

Slide 1 Example (cont’d)

Slide 1 Example (cont’d) flight

Slide 1 Example (cont’d) flight

Slide 1 Bottom-Up Filtering

Slide 1 Possible Problem: Left-Recursion What happens in the following situation S -> NP VP S -> Aux NP VP NP -> NP PP NP -> Det Nominal … With the sentence starting with Did the flight…

Slide 1 Solution: Rule Ordering S -> Aux NP VP S -> NP VP NP -> Det Nominal NP -> NP PP The key for the NP is that you want the recursive option after any base case.

Slide 1 Avoiding Repeated Work Parsing is hard, and slow. It’s wasteful to redo stuff over and over and over. Consider an attempt to top-down parse the following as an NP A flight from Indianapolis to Houston on TWA

Slide 1 flight

Slide 1 flight

Slide 1

Dynamic Programming We need a method that fills a table with partial results that – Does not do (avoidable) repeated work – Does not fall prey to left-recursion – Solves an exponential problem in (approximately) polynomial time

Slide 1 Earley Parsing Fills a table in a single sweep over the input words Table is length N+1; N is number of words Table entries represent Completed constituents and their locations In-progress constituents Predicted constituents

Slide 1 States The table-entries are called states and are represented with dotted- rules. S -> · VPA VP is predicted NP -> Det · NominalAn NP is in progress VP -> V NP · A VP has been found

Slide 1 States/Locations It would be nice to know where these things are in the input so… S -> · VP [0,0]A VP is predicted at the start of the sentence NP -> Det · Nominal[1,2]An NP is in progress; the Det goes from 1 to 2 VP -> V NP · [0,3]A VP has been found starting at 0 and ending at 3

Slide 1 Graphically

Slide 1 Earley As with most dynamic programming approaches, the answer is found by looking in the table in the right place. In this case, there should be an S state in the final column that spans from 0 to n+1 and is complete. If that’s the case you’re done. – S – α · [0,n+1] So sweep through the table from 0 to n+1… – New predicted states are created by states in current chart – New incomplete states are created by advancing existing states as new constituents are discovered – New complete states are created in the same way.

Slide 1 Earley More specifically… –Predict all the states you can upfront –Read a word – Extend states based on matches – Add new predictions – Go to 2 –Look at N+1 to see if you have a winner

Slide 1 Earley and Left Recursion So Earley solves the left-recursion problem without having to alter the grammar or artificially limiting the search. – Never place a state into the chart that’s already there – Copy states before advancing them S -> NP VP NP -> NP PP The first rule predicts S -> · NP VP [0,0]that adds NP -> · NP PP [0,0] stops there since adding any subsequent prediction would be fruitless When a state gets advanced make a copy and leave the original alone – Say we haveNP -> · NP PP [0,0] – We find an NP from 0 to 2 so we create NP -> NP · PP [0,2] – But we leave the original state as is

Slide 1 Predictor Given a state With a non-terminal to right of dot That is not a part-of-speech category Create a new state for each expansion of the non-terminal Place these new states into same chart entry as generated state, beginning and ending where generating state ends. So predictor looking at: S ->. VP [0,0] results in: VP ->. Verb [0,0] VP ->. Verb NP [0,0]

Slide 1 Scanner Given a state With a non-terminal to right of dot That is a part-of-speech category If the next word in the input matches this part-of-speech – Create a new state with dot moved over the non-terminal – insert in next chart entry So scanner looking at: VP ->. Verb NP [0,0] If the next word, “book”, can be a verb, add new state: VP -> Verb. NP [0,1] Add this state to chart entry following current one Note: Earley algorithm uses top-down input to disambiguate POS! Only POS predicted by some state can get added to chart!

Slide 1 Completer Applied to a state when its dot has reached right end of role. Parser has discovered a category over some span of input. Find and advance all previous states that were looking for this category copy state, move dot insert in current chart entry Given: NP -> Det Nominal. [1,3] VP -> Verb. NP [0,1] Add VP -> Verb NP. [0,3]

Slide 1 Earley: how do we know we are done? Find an S state in the final column that spans from 0 to n+1 and is complete. S –> α · [0,n+1]

Slide 1 Earley So sweep through the table from 0 to n+1… New predicted states are created by starting top-down from S New incomplete states are created by advancing existing states as new constituents are discovered New complete states are created in the same way.

Slide 1 Earley More specifically… Predict all the states you can upfront Read a word Extend states based on matches Add new predictions Go to 2 Look at N+1 to see if you have a winner

Slide 1 Example Book that flight We should find… an S from 0 to 3 that is a completed state…

Slide 1 Example (cont’d)

Slide 1 Example (cont’d)

Slide 1 Example (cont’d)

Slide 1 A simple example Chart[0] γ →.S [0,0] (dummy start state) S →.NP VP [0,0 ] (predictor) NP →.N [0,0 ] (predictor) Chart[1] N → I. [0,1 ] (scan) NP → N. [0,1 ] (completer) S → NP. VP [0,1 ] (completer) VP →. V NP [1,1 ] (predictor) Chart[2] V → saw [1,2 ] (scan) VP → V. NP [1,2 ] (complete) NP →. N [2,2 ] (predict) Chart[3] NP → N. [2,3 ] (scan) NP → N. [2,3 ] (completer) VP → V NP. [1,3 ] (completer) S → NP VP. [0,3 ] (completer) Grammar: S → NP VP NP → N VP → V NP Lexicon: N→ I | saw | Mary V→ saw Input: I saw Mary Sentence accepted

Slide 1 What is it? What kind of parser did we just describe (trick question). Earley parser… yes Not a parser – a recognizer The presence of an S state with the right attributes in the right place indicates a successful recognition. But no parse tree… no parser That’s how we solve (not) an exponential problem in polynomial time

Slide 1 Converting Earley from Recognizer to Parser With the addition of a few pointers we have a parser Augment the “Completer” to point to where we came from.

Slide 1 Augmenting the chart with structural information S8 S9 S10 S11 S13 S12 S8 S9 S8

Slide 1 Retrieving Parse Trees from Chart All the possible parses for an input are in the table We just need to read off all the backpointers from every complete S in the last column of the table Find all the S -> X. [0,N+1] Follow the structural traces from the Completer Of course, this won’t be polynomial time, since there could be an exponential number of trees So we can at least represent ambiguity efficiently

Slide 1 Earley and Left Recursion Earley solves the left-recursion problem without having to alter the grammar or artificially limiting the search. Never place a state into the chart that’s already there Copy states before advancing them

Slide 1 Earley and Left Recursion: 1 S -> NP VP NP -> NP PP Predictor, given first rule: S -> · NP VP [0,0] Predicts: NP -> · NP PP [0,0] stops there since predicting same again would be redundant

Slide 1 Earley and Left Recursion: 2 When a state gets advanced make a copy and leave the original alone… Say we haveNP -> · NP PP [0,0] We find an NP from 0 to 2 so we create NP -> NP · PP [0,2] But we leave the original state as is

Slide 1 Dynamic Programming Approaches Earley Top-down, no filtering, no restriction on grammar form CYK Bottom-up, no filtering, grammars restricted to Chomsky-Normal Form (CNF) Details are not important... Bottom-up vs. top-down With or without filters With restrictions on grammar form or not