Presentation is loading. Please wait.

Presentation is loading. Please wait.

EMNLP’01 19/11/2001 ML: Classical methods from AI –Decision-Tree induction –Exemplar-based Learning –Rule Induction –T ransformation B ased E rror D riven.

Published byBaldwin Hawkins Modified over 8 years ago

Similar presentations

Presentation on theme: "EMNLP’01 19/11/2001 ML: Classical methods from AI –Decision-Tree induction –Exemplar-based Learning –Rule Induction –T ransformation B ased E rror D riven."— Presentation transcript:

1 EMNLP’01 19/11/2001 ML: Classical methods from AI –Decision-Tree induction –Exemplar-based Learning –Rule Induction –T ransformation B ased E rror D riven L earning ML: Classical methods from AI –Decision-Tree induction –Exemplar-based Learning –Rule Induction –T ransformation B ased E rror D riven L earning

2 EMNLP’01 19/11/2001 TBEDL Transformation-Based Error-Driven Learning (Brill 92,93,95) The learning algorithm is a mistake-driven greedy procedure that iteratively acquires a set of transformation rules Firstly, unannotated text is passed through an initial-state annotator Then, at each step the algorithm adds the transformation rule that best repairs the current errors The learning algorithm is a mistake-driven greedy procedure that iteratively acquires a set of transformation rules Firstly, unannotated text is passed through an initial-state annotator Then, at each step the algorithm adds the transformation rule that best repairs the current errors

3 EMNLP’01 19/11/2001 TBEDL Concrete rules are acquired by instantiation of a predefined set of template rules: conjunction_of_conditions transformation When annotating a new text, all the transformation rules are applied in order of generation Concrete rules are acquired by instantiation of a predefined set of template rules: conjunction_of_conditions transformation When annotating a new text, all the transformation rules are applied in order of generation Transformation-Based Error-Driven Learning (Brill 92,93,95)

4 EMNLP’01 19/11/2001 Unnanotated Text Annotated Text Rules “Truth” Initial State Learner TRAINING TBEDL Transformation-Based Error-Driven Learning (Brill 92,93,95)

5 EMNLP’01 19/11/2001 TB(ED)L Applied to POS Tagging (Brill 92,93,94,95) Initial_State_Annotator = Most_Frequent Label Three types of templates –Non lexicalized conditions –Lexicalized patterns –Morphological conditions for dealing with unknown words Initial_State_Annotator = Most_Frequent Label Three types of templates –Non lexicalized conditions –Lexicalized patterns –Morphological conditions for dealing with unknown words TBEDL

6 EMNLP’01 19/11/2001 TB(ED)L Applied to POS Tagging (Brill 92,93,94,95) Non-lexicalized conditions: TBEDL

7 EMNLP’01 19/11/2001 TBEDL First implementation

8 EMNLP’01 19/11/2001 TB(ED)L Applied to POS Tagging (Brill 92,93,94,95) Non-lexicalized conditions: best rules acquired TBEDL

9 EMNLP’01 19/11/2001 TB(ED)L Applied to POS Tagging (Brill 92,93,94,95) Lexicalized patterns: TBEDL

10 EMNLP’01 19/11/2001 TB(ED)L Applied to POS Tagging (Brill 92,93,94,95) Lexicalized patterns: TBEDL – as /IN tall /JJ as /IN – We do ’nt eat / We did ’nt usually drink

11 EMNLP’01 19/11/2001 TB(ED)L Applied to POS Tagging (Brill 92,93,94,95) Morphological conditions for dealing with unknown words: TBEDL

12 EMNLP’01 19/11/2001 TBEDL TB(ED)L Applied to POS Tagging (Brill 92,93,94,95) Unknown words: best rules acquired

13 EMNLP’01 19/11/2001 Tested on 600 Kw of the Wall Street annotated corpus –Number of transformation rules: <500 –Accuracy: 97.0% - 97.2% (with no unknown words) The accuracy of a HMM trigram tagger is achieved using only 86 transformation rules 96.6% considering unknown words (82.2%) Tested on 600 Kw of the Wall Street annotated corpus –Number of transformation rules: <500 –Accuracy: 97.0% - 97.2% (with no unknown words) The accuracy of a HMM trigram tagger is achieved using only 86 transformation rules 96.6% considering unknown words (82.2%) TBEDL TB(ED)L Applied to POS Tagging (Brill 92,93,94,95)

14 EMNLP’01 19/11/2001 TBEDL TB(ED)L Applied to POS Tagging (Brill 92,93,94,95)

15 EMNLP’01 19/11/2001 TB(ED)L and NLP POS Tagging (Brill 92,94a,95; Roche & Schabes 95; Aone & Hausman 96) PP-attachment disambiguation (Brill & Resnik, 1994) Grammar induction and Parsing (Brill, 1993) Context-sensitive Spelling Correction (Mangu & Brill, 1996) Word Sense Disambiguation (Dini et al., 1998) Dialogue Act Tagging (Samuel et al., 1998a,1998b) Semantic Role Labeling (Higgins, 2004; Williams et al., 2004; CoNLL-2004) POS Tagging (Brill 92,94a,95; Roche & Schabes 95; Aone & Hausman 96) PP-attachment disambiguation (Brill & Resnik, 1994) Grammar induction and Parsing (Brill, 1993) Context-sensitive Spelling Correction (Mangu & Brill, 1996) Word Sense Disambiguation (Dini et al., 1998) Dialogue Act Tagging (Samuel et al., 1998a,1998b) Semantic Role Labeling (Higgins, 2004; Williams et al., 2004; CoNLL-2004) TBEDL

16 EMNLP’01 19/11/2001 TB(ED)L: Main Drawback TBEDL Computational cost –Memory & Time (specially on Training) Some proposals –Ramshaw & Marcus (1994) –LazyTBL (Samuel 98)  -TBL (Lager 99) –ICA (Hepple 00) –FastTBL (Ngai & Florian, 01) Computational cost –Memory & Time (specially on Training) Some proposals –Ramshaw & Marcus (1994) –LazyTBL (Samuel 98)  -TBL (Lager 99) –ICA (Hepple 00) –FastTBL (Ngai & Florian, 01)

17 EMNLP’01 19/11/2001 Extensions: LazyTBEDL (Samuel 98) Uses Brill’s TB(ED)L algorithm Applies Monte Carlo strategy to randomly sample from the space of rules, rather than exhaustively analyzing all possible rules The memory and time costs of the TB(ED)L algorithm are drastically reduced without compromising accuracy on unseen data Application to Dialogue Act Tagging –Accuracy results: 75.5% over state-of-the-art systems Uses Brill’s TB(ED)L algorithm Applies Monte Carlo strategy to randomly sample from the space of rules, rather than exhaustively analyzing all possible rules The memory and time costs of the TB(ED)L algorithm are drastically reduced without compromising accuracy on unseen data Application to Dialogue Act Tagging –Accuracy results: 75.5% over state-of-the-art systems TBEDL

18 EMNLP’01 19/11/2001 TBEDL Extensions: LazyTBEDL (Samuel 98)

19 EMNLP’01 19/11/2001 TBEDL Extensions: LazyTBEDL (Samuel 98)

20 EMNLP’01 19/11/2001 TBEDL Extensions: LazyTBEDL (Samuel 98)

21 EMNLP’01 19/11/2001 TBEDL Extensions: LazyTBEDL (Samuel 98)

22 EMNLP’01 19/11/2001 TBEDL Extensions: LazyTBEDL (Samuel 98)

23 EMNLP’01 19/11/2001 TBEDL Extensions: FastTBEDL (Ngai & Florian 01)

24 EMNLP’01 19/11/2001 TBEDL Extensions: FastTBEDL (Ngai & Florian 01) Software available at: http://nlp.cs.jhu.edu/rflorian/fntbl Software available at: http://nlp.cs.jhu.edu/rflorian/fntbl

25 EMNLP’01 19/11/2001 TB(ED)L: Summary Advantages –General, simple and understandable modeling –Provides a very compact set of interpretable transformation rules –High accuracy in many NLP applications Advantages –General, simple and understandable modeling –Provides a very compact set of interpretable transformation rules –High accuracy in many NLP applications TBEDL Drawbacks –Computational cost: high memory and time requirements. But some efficient variants of TBL have been proposed (fastTBL) –Sequential application of rules Drawbacks –Computational cost: high memory and time requirements. But some efficient variants of TBL have been proposed (fastTBL) –Sequential application of rules

26 EMNLP’01 19/11/2001 TB(ED)L: Summary Others –A transformation list is a processor and not a classifier –A comparison between Decision Trees and Transformation lists can be found in (Brill, 1995) Others –A transformation list is a processor and not a classifier –A comparison between Decision Trees and Transformation lists can be found in (Brill, 1995) TBEDL

Download ppt "EMNLP’01 19/11/2001 ML: Classical methods from AI –Decision-Tree induction –Exemplar-based Learning –Rule Induction –T ransformation B ased E rror D riven."

Similar presentations

Three Basic Problems Compute the probability of a text: P m (W 1,N ) Compute maximum probability tag sequence: arg max T 1,N P m (T 1,N | W 1,N ) Compute.

Three Basic Problems Compute the probability of a text: P m (W 1,N ) Compute maximum probability tag sequence: arg max T 1,N P m (T 1,N | W 1,N ) Compute.
School of something FACULTY OF OTHER School of Computing FACULTY OF ENGINEERING Chunking: Shallow Parsing Eric Atwell, Language Research Group.

School of something FACULTY OF OTHER School of Computing FACULTY OF ENGINEERING Chunking: Shallow Parsing Eric Atwell, Language Research Group.
Machine Learning Approaches to the Analysis of Large Corpora : A Survey Xunlei Rose Hu and Eric Atwell University of Leeds.

Machine Learning Approaches to the Analysis of Large Corpora : A Survey Xunlei Rose Hu and Eric Atwell University of Leeds.
Three Basic Problems 1.Compute the probability of a text (observation) language modeling – evaluate alternative texts and models P m (W 1,N ) 2.Compute.

Three Basic Problems 1.Compute the probability of a text (observation) language modeling – evaluate alternative texts and models P m (W 1,N ) 2.Compute.
CS460/IT632 Natural Language Processing/Language Technology for the Web Lecture 2 (06/01/06) Prof. Pushpak Bhattacharyya IIT Bombay Part of Speech (PoS)

CS460/IT632 Natural Language Processing/Language Technology for the Web Lecture 2 (06/01/06) Prof. Pushpak Bhattacharyya IIT Bombay Part of Speech (PoS)
Part of Speech Tagging Importance Resolving ambiguities by assigning lower probabilities to words that don’t fit Applying to language grammatical rules.

Part of Speech Tagging Importance Resolving ambiguities by assigning lower probabilities to words that don’t fit Applying to language grammatical rules.
Probabilistic Detection of Context-Sensitive Spelling Errors Johnny Bigert Royal Institute of Technology, Sweden

Probabilistic Detection of Context-Sensitive Spelling Errors Johnny Bigert Royal Institute of Technology, Sweden
Learning with Probabilistic Features for Improved Pipeline Models Razvan C. Bunescu Electrical Engineering and Computer Science Ohio University Athens,

Learning with Probabilistic Features for Improved Pipeline Models Razvan C. Bunescu Electrical Engineering and Computer Science Ohio University Athens,
74.793 NLP and Speech Course Review. Morphological Analyzer Lexicon Part-of-Speech (POS) Tagging Grammar Rules Parser thethe – determiner Det NP → Det.

NLP and Speech Course Review. Morphological Analyzer Lexicon Part-of-Speech (POS) Tagging Grammar Rules Parser thethe – determiner Det NP → Det.
Part II. Statistical NLP Advanced Artificial Intelligence Part of Speech Tagging Wolfram Burgard, Luc De Raedt, Bernhard Nebel, Lars Schmidt-Thieme Most.

Part II. Statistical NLP Advanced Artificial Intelligence Part of Speech Tagging Wolfram Burgard, Luc De Raedt, Bernhard Nebel, Lars Schmidt-Thieme Most.
Shallow Processing: Summary Shallow Processing Techniques for NLP Ling570 December 7, 2011.

Shallow Processing: Summary Shallow Processing Techniques for NLP Ling570 December 7, 2011.
Part-of-speech Tagging cs224n Final project Spring, 2008 Tim Lai.

Part-of-speech Tagging cs224n Final project Spring, 2008 Tim Lai.
Ch 10 Part-of-Speech Tagging Edited from: L. Venkata Subramaniam February 28, 2002.

Ch 10 Part-of-Speech Tagging Edited from: L. Venkata Subramaniam February 28, 2002.
CS 47051 Catching Up CS 4705. 2 Porter Stemmer Porter Stemmer (1980) Used for tasks in which you only care about the stem –IR, modeling given/new distinction,

CS Catching Up CS Porter Stemmer Porter Stemmer (1980) Used for tasks in which you only care about the stem –IR, modeling given/new distinction,
Tagging – more details Reading: D Jurafsky & J H Martin (2000) Speech and Language Processing, Ch 8 R Dale et al (2000) Handbook of Natural Language Processing,

Tagging – more details Reading: D Jurafsky & J H Martin (2000) Speech and Language Processing, Ch 8 R Dale et al (2000) Handbook of Natural Language Processing,
Introduction to CL Session 1: 7/08/2011. What is computational linguistics? Processing natural language text by computers  for practical applications.

Introduction to CL Session 1: 7/08/2011. What is computational linguistics? Processing natural language text by computers  for practical applications.
Transformation-based error- driven learning (TBL) LING 572 Fei Xia 1/19/06.

Transformation-based error- driven learning (TBL) LING 572 Fei Xia 1/19/06.
Boosting Applied to Tagging and PP Attachment By Aviad Barzilai.

Boosting Applied to Tagging and PP Attachment By Aviad Barzilai.
Resources Primary resources – Lexicons, structured vocabularies – Grammars (in widest sense) – Corpora – Treebanks Secondary resources – Designed for a.

Resources Primary resources – Lexicons, structured vocabularies – Grammars (in widest sense) – Corpora – Treebanks Secondary resources – Designed for a.
Statistical techniques in NLP Vasileios Hatzivassiloglou University of Texas at Dallas.

Statistical techniques in NLP Vasileios Hatzivassiloglou University of Texas at Dallas.

Similar presentations

Ads by Google