Presentation is loading. Please wait.

Presentation is loading. Please wait.

Prototype-Driven Grammar Induction Aria Haghighi and Dan Klein Computer Science Division University of California Berkeley.

Similar presentations


Presentation on theme: "Prototype-Driven Grammar Induction Aria Haghighi and Dan Klein Computer Science Division University of California Berkeley."— Presentation transcript:

1 Prototype-Driven Grammar Induction Aria Haghighi and Dan Klein Computer Science Division University of California Berkeley

2 Grammar Induction DT NN VBD DT NN IN NN The screen was a sea of red

3 First Attempt DT NN VBD DT NN IN NN The screen was a sea of red

4 Central Questions How do we specify what we want to learn? How do we fix observed errors? What’s an NP? That’s not quite it!

5 Experimental Set-up Binary Grammar { X 1, X 2, … X n } plus POS tags Data WSJ-10 [7k sentences] Evaluate on Labeled F 1 Grammar Upper Bound: 86.1 XjXj X i XkXk

6 Experiment Roadmap Unconstrained Induction Need bracket constraint! Gold Bracket Induction Prototypes and Similarity CCM Bracket Induction PhrasePrototype NPDT NN PPIN DT NN

7 Unconstrained PCFG Induction Learn PCFG with EM Inside-Outside Algorithm Lari & Young [93] Results 0 i j n  (Inside)  (Outside)

8 Gold Brackets Periera & Schables [93] Result Constrained PCFG Induction

9 Encoding Knowledge What’s an NP? Semi-Supervised Learning

10 Encoding Knowledge What’s an NP? For instance, DT NN JJ NNS NNP Prototype Learning

11 Add Prototypes Manually constructed Grammar Induction Experiments

12 How to use prototypes? ? ? ? ? DT The NN koala VBD sat IN in DT the NN tree ¦¦ ? NP PP VP S

13 How to use prototypes? NP PP VP DT The NN koala VBD sat IN in DT the NN tree ¦ ¦ S JJ hungry ?NP

14 Distributional Similarity Context Distribution  (DT JJ NN) = { ¦ __ VBD : 0.3, VBD __ ¦ : 0.2, IN __ VBD: 0.1, ….. } Similarity  (DT NN)  (DT JJ NN)  (JJ NNS)  (NNP NNP) NP

15 Distributional Similarity Prototype Approximation  (NP) ¼ Uniform (  (DT NN),  (JJ NNS),  (NNP NNP) ) Prototype Similarity Feature span(DT JJ NN) emits proto=NP span(MD NNS) emits proto=NONE

16 Prototype CFG + Model NP PP VP DT The NN koala VBD sat IN in DT the NN tree ¦ ¦ S JJ hungry NP P (DT NP | NP) P (proto=NP | NP)

17 Prototype CFG+ Induction Experimental Set-Up BLIPP corpus Gold Brackets Results

18 Summary So Far Bracket constraint and prototypes give good performance!

19 Constituent-Context Model

20 Product Model Different Aspects of Syntax CCM : Yield and Context properties CFG : Hierarchical properties Intersected EM [Klein 2005] Encourages mass on trees compatible with CCM and CFG

21 Grammar Induction Experiments Intersected CFG and CCM No prototypes Results

22 Grammar Induction Experiments Intersected CFG + and CCM Add Prototypes Results

23 Reacting to Errors Possessive NPs Our Tree Correct Tree

24 Reacting to Errors Add Prototype: NP-POS NN POS New Analysis

25 Error Analysis Modal VPs Our TreeCorrect Tree

26 Reacting to Errors Add Prototype: VP-INF VB NN New Analysis

27 Fixing Errors Supplement Prototypes NP-POS and VP-INF Results

28 Results Summary

29 Conclusion Prototype-Driven Learning Flexible Weakly Supervised Framework Merged distributional clustering techniques with supervised structured models

30 Thank You!

31 Unconstrained PCFG Induction Binary Grammar { X 1, X 2, … X n } Learn PCFG with EM Inside-Outside Algorithm Lari & Young [93] 0 i j n  (Inside)  (Outside) V XjXj X i XkXk N XkXk XjXj V N


Download ppt "Prototype-Driven Grammar Induction Aria Haghighi and Dan Klein Computer Science Division University of California Berkeley."

Similar presentations


Ads by Google