Download presentation

Presentation is loading. Please wait.

Published bySamir Stephens Modified about 1 year ago

1
Prototype-Driven Grammar Induction Aria Haghighi and Dan Klein Computer Science Division University of California Berkeley

2
Grammar Induction DT NN VBD DT NN IN NN The screen was a sea of red

3
First Attempt DT NN VBD DT NN IN NN The screen was a sea of red

4
Central Questions How do we specify what we want to learn? How do we fix observed errors? What’s an NP? That’s not quite it!

5
Experimental Set-up Binary Grammar { X 1, X 2, … X n } plus POS tags Data WSJ-10 [7k sentences] Evaluate on Labeled F 1 Grammar Upper Bound: 86.1 XjXj X i XkXk

6
Experiment Roadmap Unconstrained Induction Need bracket constraint! Gold Bracket Induction Prototypes and Similarity CCM Bracket Induction PhrasePrototype NPDT NN PPIN DT NN

7
Unconstrained PCFG Induction Learn PCFG with EM Inside-Outside Algorithm Lari & Young [93] Results 0 i j n (Inside) (Outside)

8
Gold Brackets Periera & Schables [93] Result Constrained PCFG Induction

9
Encoding Knowledge What’s an NP? Semi-Supervised Learning

10
Encoding Knowledge What’s an NP? For instance, DT NN JJ NNS NNP Prototype Learning

11
Add Prototypes Manually constructed Grammar Induction Experiments

12
How to use prototypes? ? ? ? ? DT The NN koala VBD sat IN in DT the NN tree ¦¦ ? NP PP VP S

13
How to use prototypes? NP PP VP DT The NN koala VBD sat IN in DT the NN tree ¦ ¦ S JJ hungry ?NP

14
Distributional Similarity Context Distribution (DT JJ NN) = { ¦ __ VBD : 0.3, VBD __ ¦ : 0.2, IN __ VBD: 0.1, ….. } Similarity (DT NN) (DT JJ NN) (JJ NNS) (NNP NNP) NP

15
Distributional Similarity Prototype Approximation (NP) ¼ Uniform ( (DT NN), (JJ NNS), (NNP NNP) ) Prototype Similarity Feature span(DT JJ NN) emits proto=NP span(MD NNS) emits proto=NONE

16
Prototype CFG + Model NP PP VP DT The NN koala VBD sat IN in DT the NN tree ¦ ¦ S JJ hungry NP P (DT NP | NP) P (proto=NP | NP)

17
Prototype CFG+ Induction Experimental Set-Up BLIPP corpus Gold Brackets Results

18
Summary So Far Bracket constraint and prototypes give good performance!

19
Constituent-Context Model

20
Product Model Different Aspects of Syntax CCM : Yield and Context properties CFG : Hierarchical properties Intersected EM [Klein 2005] Encourages mass on trees compatible with CCM and CFG

21
Grammar Induction Experiments Intersected CFG and CCM No prototypes Results

22
Grammar Induction Experiments Intersected CFG + and CCM Add Prototypes Results

23
Reacting to Errors Possessive NPs Our Tree Correct Tree

24
Reacting to Errors Add Prototype: NP-POS NN POS New Analysis

25
Error Analysis Modal VPs Our TreeCorrect Tree

26
Reacting to Errors Add Prototype: VP-INF VB NN New Analysis

27
Fixing Errors Supplement Prototypes NP-POS and VP-INF Results

28
Results Summary

29
Conclusion Prototype-Driven Learning Flexible Weakly Supervised Framework Merged distributional clustering techniques with supervised structured models

30
Thank You!

31
Unconstrained PCFG Induction Binary Grammar { X 1, X 2, … X n } Learn PCFG with EM Inside-Outside Algorithm Lari & Young [93] 0 i j n (Inside) (Outside) V XjXj X i XkXk N XkXk XjXj V N

Similar presentations

© 2016 SlidePlayer.com Inc.

All rights reserved.

Ads by Google