Presentation is loading. Please wait.

Presentation is loading. Please wait.

Improved Inference for Unlexicalized Parsing Slav Petrov and Dan Klein.

Similar presentations


Presentation on theme: "Improved Inference for Unlexicalized Parsing Slav Petrov and Dan Klein."— Presentation transcript:

1 Improved Inference for Unlexicalized Parsing Slav Petrov and Dan Klein

2 Unlexicalized Parsing Hierarchical, adaptive refinement: 1,140 Nonterminal symbols1621min Parsing time 531,200 Rewrites [Petrov et al. ‘06] 91.2 F1 score on Dev Set (1600 sentences) DT 1 DT 2 DT 3 DT 4 DT 5 DT 6 DT 7 DT 8 DT 1 DT 2 DT 3 DT 4 DT 1 DT DT 2

3 1621 min

4 Coarse-to-Fine Parsing [Goodman ‘97, Charniak&Johnson ‘05] Coarse grammar NP … VP NP-dog NP-cat NP-apple VP-run NP-eat… Refined grammar … Treebank Parse Prune NP-17 NP-12 NP-1 VP-6 VP-31… Refined grammar … Parse

5 Prune? For each chart item X[i,j], compute posterior probability: …QPNPVP… coarse: refined: E.g. consider the span 5 to 12: < threshold

6 1621 min 111 min (no search error)

7 [Charniak et al. ‘06] NP … VP NP-dog NP-cat NP-apple VP-run NP-eat… Refined grammar … X A,B,.. Multilevel Coarse-to-Fine Parsing Add more rounds of pre-parsing Grammars coarser than X-bar ??? ?

8 Hierarchical Pruning Consider again the span 5 to 12: …QPNPVP… coarse: split in two: …QP1QP2NP1NP2VP1VP2… …QP1 QP3QP4NP1NP2NP3NP4VP1VP2VP3VP4… split in four: split in eight: ……………………………………………

9 Intermediate Grammars X-Bar= G 0 G= G1G2G3G4G5G6G1G2G3G4G5G6 Learning DT 1 DT 2 DT 3 DT 4 DT 5 DT 6 DT 7 DT 8 DT 1 DT 2 DT 3 DT 4 DT 1 DT DT 2

10 1621 min 111 min 35 min (no search error)

11 State Drift (DT tag) some this That these Thatthissome the these thissome that Thatthissome the these thissome that …………………………………………some thesethisThatThisthat EM

12 G1G2G3G4G5G6G1G2G3G4G5G6 Learning G1G2G3G4G5G6G1G2G3G4G5G6 Projected Grammars X-Bar= G 0 G= Projection  i 0(G)1(G)2(G)3(G)4(G)5(G)0(G)1(G)2(G)3(G)4(G)5(G) G

13 Estimating Projected Grammars Nonterminals? Nonterminals in G NP 1 VP 1 VP 0 S0S0 S1S1 NP 0 Nonterminals in  (G) VP S NP Projection  Easy:

14 Rules in G Rules in  (G) Estimating Projected Grammars Rules? S 1  NP 1 VP S 1  NP 1 VP S 1  NP 2 VP S 1  NP 2 VP S 2  NP 1 VP S 2  NP 1 VP S 2  NP 2 VP S 2  NP 2 VP S  NP VP ? ???

15 Treebank Estimating Projected Grammars [Corazza & Satta ‘06] Rules in  (G) S  NP VP Rules in G S1  NP1 VP S1  NP1 VP S1  NP2 VP S1  NP2 VP S2  NP1 VP S2  NP1 VP S2  NP2 VP S2  NP2 VP Infinite tree distribution … … 0.56 Estimating Grammars

16 Calculating Expectations  Nonterminals:  c k (X) : expected counts up to depth k  Converges within 25 iterations (few seconds)  Rules:

17 1621 min 111 min 35 min 15 min (no search error)

18 G1G2G3G4G5G6G1G2G3G4G5G6 Learning Parsing times X-Bar= G 0 G= 60 % 12 % 7 % 6 % 5 % 4 %

19 Bracket Posteriors (after G 0 )

20 Bracket Posteriors (after G 1 )

21 Bracket Posteriors (Movie)(Final Chart)

22 Bracket Posteriors (Best Tree)

23 Parse Selection Computing most likely unsplit tree is NP-hard:  Settle for best derivation.  Rerank n-best list.  Use alternative objective function. Parses: -2 Derivations:

24 Parse Risk Minimization  Expected loss according to our beliefs:  T T : true tree  T P : predicted tree  L : loss function (0/1, precision, recall, F1) [Titov & Henderson ‘06]  Use n-best candidate list and approximate expectation with samples.

25 Reranking Results ObjectivePrecisionRecallF1Exact BEST DERIVATION Viterbi Derivation Exact (non-sampled) Exact/F1 (oracle) RERANKING Precision (sampled) Recall (sampled) F1 (sampled) Exact (sampled)

26 Dynamic Programming [Matsuzaki et al. ‘05] Approximate posterior parse distribution à la [Goodman ‘98] Maximize number of expected correct rules

27 ObjectivePrecisionRecallF1Exact BEST DERIVATION Viterbi Derivation DYNAMIC PROGRAMMING Variational Max-Rule-Sum Max-Rule-Product Dynamic Programming Results

28 Final Results (Efficiency)  Berkeley Parser:  15 min  91.2 F-score  Implemented in Java  Charniak & Johnson ‘05 Parser  19 min  90.7 F-score  Implemented in C

29 Final Results (Accuracy) ≤ 40 words F1 all F1 ENG Charniak&Johnson ‘05 (generative) This Work Charniak&Johnson ‘05 (reranked) GER Dubey ‘ This Work CHN Chiang et al. ‘ This Work

30 Conclusions  Hierarchical coarse-to-fine inference  Projections  Marginalization  Multi-lingual unlexicalized parsing

31 Thank You! Parser available at


Download ppt "Improved Inference for Unlexicalized Parsing Slav Petrov and Dan Klein."

Similar presentations


Ads by Google