Presentation is loading. Please wait.

Presentation is loading. Please wait.

CPSC 422, Lecture 27Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 27 Nov, 16, 2015.

Published byJessica Cannon Modified over 8 years ago

Similar presentations

Presentation on theme: "CPSC 422, Lecture 27Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 27 Nov, 16, 2015."— Presentation transcript:

1 CPSC 422, Lecture 27Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 27 Nov, 16, 2015

2 CPSC 422, Lecture 27 2 NLP: Knowledge-Formalisms Map (including probabilistic formalisms) Logical formalisms (First-Order Logics, Prob. Logics) Rule systems (and prob. versions) (e.g., (Prob.) Context-Free Grammars) State Machines (and prob. versions) (Finite State Automata,Finite State Transducers, Markov Models) Morphology Syntax Pragmatics Discourse and Dialogue Semantics AI planners (Task decomposition, MDP Markov Decision Processes)

3 CPSC 422, Lecture 273 Lecture Overview Recap English Syntax and Parsing Key Problem with parsing: Ambiguity Probabilistic Context Free Grammars (PCFG) Treebanks and Grammar Learning

4 CPSC 422, Lecture 27 4 Key Constituents: Examples Noun phrases (NP) Verb phrases (VP) Prepositional phrases (PP) Adjective phrases(AP) Sentences (S) (Det) N (PP) the cat on the table (Qual) V (NP) never eat a cat (Deg) P (NP) almost inthe net (Deg) A (PP) very happy about it (NP) (-)(VP) a mouse --ate it (Specifier) X (Complement)

5 Context Free Grammar (CFG) 4-tuple (non-term., term., productions, start) (N, , P, S) P is a set of rules A  ; A  N,  (  N)* CPSC 422, Lecture 275

6 6 CFG Example LexiconGrammar with example phrases

7 CPSC 422, Lecture 277 Derivations as Trees flight Nominal

8 CPSC 422, Lecture 278 Journal of the American Medical Informatics Association, 2005, Improved Identification of Noun Phrases in Clinical Radiology Reports Using a High-Performance Statistical Natural Language Parser Augmented with the UMLS Specialist Lexicon Example of relatively complex parse tree

9 CPSC 422, Lecture 279 Lecture Overview Recap English Syntax and Parsing Key Problem with parsing: Ambiguity Probabilistic Context Free Grammars (PCFG) Treebanks and Grammar Learning

10 CPSC 422, Lecture 2710 Structural Ambiguity (Ex. 1) “I shot an elephant in my pajamas” VP -> V NP ; NP -> NP PP VP -> V NP PP

11 CPSC 422, Lecture 2711 Structural Ambiguity (Ex.2) “I saw Mary passing by cs2” (ROOT (S (NP (PRP I)) (VP (VBD saw) (S (NP (NNP Mary)) (VP (VBG passing) (PP (IN by) (NP (NNP cs2))))))) (ROOT (S (NP (PRP I)) (VP (VBD saw) (NP (NNP Mary)) (S (VP (VBG passing) (PP (IN by) (NP (NNP cs2))))))) “I saw Mary passing by cs2”

12 CPSC 422, Lecture 2712 Structural Ambiguity (Ex. 3) Coordination “new student and profs”

13 CPSC 422, Lecture 2713 Structural Ambiguity (Ex. 4) NP-bracketing “French language teacher”

14 CPSC 422, Lecture 2714 Lecture Overview Recap English Syntax and Parsing Key Problem with parsing: Ambiguity Probabilistic Context Free Grammars (PCFG) Treebanks and Grammar Learning (acquiring the probabilities) Intro to Parsing PCFG

15 CPSC 422, Lecture 2715 Probabilistic CFGs (PCFGs) Each grammar rule is augmented with a conditional probability GOAL: assign a probability to parse trees and to sentences If these are all the rules for VP and.55 is the VP -> Verb.55 VP-> Verb NP.40 VP-> Verb NP NP ?? What ?? should be ? A. 1 C..05 B. 0 D. None of the above

16 CPSC 422, Lecture 2716 Probabilistic CFGs (PCFGs) Each grammar rule is augmented with a conditional probability Formal Def: 5-tuple (N, , P, S,D) The expansions for a given non-terminal sum to 1 VP -> Verb.55 VP-> Verb NP.40 VP-> Verb NP NP.05 GOAL: assign a probability to parse trees and to sentences

17 CPSC 422, Lecture 2717 Sample PCFG

18 CPSC 422, Lecture 2718 PCFGs are used to…. Estimate Prob. of parse tree Estimate Prob. of a sentence A. Sum of the probs of all the rules applied B. Product of the probs of all the rules applied A. Sum of the probs of all the parse trees B. Product of the probs of all the parse trees

19 CPSC 422, Lecture 2719 PCFGs are used to…. Estimate Prob. of parse tree Estimate Prob. to sentences

20 CPSC 422, Lecture 2720 Example

21 CPSC 422, Lecture 2721 Lecture Overview Recap English Syntax and Parsing Key Problem with parsing: Ambiguity Probabilistic Context Free Grammars (PCFG) Treebanks and Grammar Learning (acquiring the probabilities)

22 CPSC 422, Lecture 2722 Treebanks DEF. corpora in which each sentence has been paired with a parse tree These are generally created –Parse collection with parser –human annotators revise each parse Requires detailed annotation guidelines –POS tagset –Grammar –instructions for how to deal with particular grammatical constructions.

23 CPSC 422, Lecture 2723 Penn Treebank Penn TreeBank is a widely used treebank.  Most well known is the Wall Street Journal section of the Penn TreeBank.  1 M words from the 1987- 1989 Wall Street Journal.

24 CPSC 422, Lecture 2724 Treebank Grammars Treebanks implicitly define a grammar. Simply take the local rules that make up the sub-trees in all the trees in the collection if decent size corpus, you’ll have a grammar with decent coverage.

25 CPSC 422, Lecture 2725 Treebank Grammars Such grammars tend to contain lots of rules…. For example, the Penn Treebank has 4500 different rules for VPs! Among them...

26 CPSC 422, Lecture 2726 Treebank Grammars Such grammars tend to be very flat due to the fact that they tend to avoid recursion. –To ease the annotators burden For example, the Penn Treebank has 4500 different rules for VPs! Among them...

27 CPSC 422, Lecture 2727 Heads in Trees Finding heads in treebank trees is a task that arises frequently in many applications. –Particularly important in statistical parsing We can visualize this task by annotating the nodes of a parse tree with the heads of each corresponding node.

28 CPSC 422, Lecture 2728 Lexically Decorated Tree

29 CPSC 422, Lecture 2729 Head Finding The standard way to do head finding is to use a simple set of tree traversal rules specific to each non-terminal in the grammar. Each rule in the PCFG specifies where the head of the expanded non-terminal should be found

30 CPSC 422, Lecture 2730 Noun Phrases

31 CPSC 422, Lecture 2731 Acquiring Grammars and Probabilities Manually parsed text corpora (e.g., PennTreebank) Ex: if the NP -> ART ADJ NOUN rule is used 50 times and all NP rules are used 5000 times, then the rule’s probability is … Grammar: read it off the parse trees Ex: if an NP contains an ART, ADJ, and NOUN then we create the rule NP -> ART ADJ NOUN. Probabilities:

32 CPSC 422, Lecture 2732

33 CPSC 322, Lecture 19 Learning Goals for today’s class You can: Provide a formal definition of a PCFG Apply a PCFG to compute the probability of a parse tree of a sentence as well as the probability of a sentence Describe the content of a treebank Describe the process to identify a head of a syntactic constituent Compute the probability distribution of a PCFG from a treebank

34 Next class on Wed Parsing Probabilistic CFG: CKY parsing PCFG in practice: Modeling Structural and Lexical Dependencies CPSC 422, Lecture 2734 Assignment-3 due on Fri Assignment-4 out same day

Download ppt "CPSC 422, Lecture 27Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 27 Nov, 16, 2015."

Similar presentations

Mrach 1, 2009Dr. Muhammed Al-Mulhem1 ICS482 Formal Grammars Chapter 12 Muhammed Al-Mulhem March 1, 2009.

Mrach 1, 2009Dr. Muhammed Al-Mulhem1 ICS482 Formal Grammars Chapter 12 Muhammed Al-Mulhem March 1, 2009.
Probabilistic and Lexicalized Parsing CS 4705. Probabilistic CFGs: PCFGs Weighted CFGs –Attach weights to rules of CFG –Compute weights of derivations.

Probabilistic and Lexicalized Parsing CS Probabilistic CFGs: PCFGs Weighted CFGs –Attach weights to rules of CFG –Compute weights of derivations.
10. Lexicalized and Probabilistic Parsing -Speech and Language Processing- 발표자 : 정영임 발표일 : 2007. 10. 6.

10. Lexicalized and Probabilistic Parsing -Speech and Language Processing- 발표자 : 정영임 발표일 :
March 1, 2009 Dr. Muhammed Al-Mulhem 1 ICS 482 Natural Language Processing Probabilistic Context Free Grammars (Chapter 14) Muhammed Al-Mulhem March 1,

March 1, 2009 Dr. Muhammed Al-Mulhem 1 ICS 482 Natural Language Processing Probabilistic Context Free Grammars (Chapter 14) Muhammed Al-Mulhem March 1,
For Monday Read Chapter 23, sections 3-4 Homework –Chapter 23, exercises 1, 6, 14, 19 –Do them in order. Do NOT read ahead.

For Monday Read Chapter 23, sections 3-4 Homework –Chapter 23, exercises 1, 6, 14, 19 –Do them in order. Do NOT read ahead.
PCFG Parsing, Evaluation, & Improvements Ling 571 Deep Processing Techniques for NLP January 24, 2011.

PCFG Parsing, Evaluation, & Improvements Ling 571 Deep Processing Techniques for NLP January 24, 2011.
Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 27

Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 27
6/9/2015CPSC503 Winter 20091 CPSC 503 Computational Linguistics Lecture 11 Giuseppe Carenini.

6/9/2015CPSC503 Winter CPSC 503 Computational Linguistics Lecture 11 Giuseppe Carenini.
Albert Gatt LIN3022 Natural Language Processing Lecture 8.

Albert Gatt LIN3022 Natural Language Processing Lecture 8.
Features and Unification

Features and Unification
Introduction to Syntax, with Part-of-Speech Tagging Owen Rambow September 17 & 19.

Introduction to Syntax, with Part-of-Speech Tagging Owen Rambow September 17 & 19.
Syntax and Context-Free Grammars CMSC 723: Computational Linguistics I ― Session #6 Jimmy Lin The iSchool University of Maryland Wednesday, October 7,

Syntax and Context-Free Grammars CMSC 723: Computational Linguistics I ― Session #6 Jimmy Lin The iSchool University of Maryland Wednesday, October 7,
1 CONTEXT-FREE GRAMMARS. NLE 2 Syntactic analysis (Parsing) S NPVP ATNNSVBD NP AT NNthechildrenate thecake.

1 CONTEXT-FREE GRAMMARS. NLE 2 Syntactic analysis (Parsing) S NPVP ATNNSVBD NP AT NNthechildrenate thecake.
74.419 Artificial Intelligence 2004 Natural Language Processing - Syntax and Parsing - Language Syntax Parsing.

Artificial Intelligence 2004 Natural Language Processing - Syntax and Parsing - Language Syntax Parsing.
CPSC 503 Computational Linguistics

CPSC 503 Computational Linguistics
9/8/20151 Natural Language Processing Lecture Notes 1.

9/8/20151 Natural Language Processing Lecture Notes 1.
Context Free Grammars Reading: Chap 12-13, Jurafsky & Martin This slide set was adapted from J. Martin and Rada Mihalcea.

Context Free Grammars Reading: Chap 12-13, Jurafsky & Martin This slide set was adapted from J. Martin and Rada Mihalcea.
Speech and Language Processing Lecture 12—02/24/2015 Susan W. Brown.

Speech and Language Processing Lecture 12—02/24/2015 Susan W. Brown.
BİL711 Natural Language Processing1 Statistical Parse Disambiguation Problem: –How do we disambiguate among a set of parses of a given sentence? –We want.

BİL711 Natural Language Processing1 Statistical Parse Disambiguation Problem: –How do we disambiguate among a set of parses of a given sentence? –We want.
Probabilistic Parsing Reading: Chap 14, Jurafsky & Martin This slide set was adapted from J. Martin, U. Colorado Instructor: Paul Tarau, based on Rada.

Probabilistic Parsing Reading: Chap 14, Jurafsky & Martin This slide set was adapted from J. Martin, U. Colorado Instructor: Paul Tarau, based on Rada.

Similar presentations

Ads by Google