1. July 11, 2010ACL 2010 Tutorial, Uppsala, Sweden1 Tree-based and Forest-based Translation Yang LiuLiang Huang Institute of Computing Technology Chinese Academy of Sciences Information Sciences Institute University of Southern California

2. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 2 Outline Part 1: Tree-based Translation  Overview and Motivation  Tree-to-String Model and Decoding  Tree-to-String Rule Extraction  Language Model-Integrated Decoding: Cube Pruning Part 2: Forest-based Translation  Packed Forest  Forest-based Decoding  Forest-based Rule Extraction Part 3: Extensions  Tree-to-Tree Translation  Tree Sequence-based Translation  Joint Parsing and Translation Part 4: Conclusion

3. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 3 Natural Languages are Different I love you By Google Translate 我爱你 Je t'aime Ich liebe dich Eu te amo Te quiero Я люблю тебя わたしは、あなたを愛しています 당신을 사랑합니다 من شما را دوست دارم Jag älskar dig Ti amo Miluji tě Ik hou van je Tôi yêu bạn אני אוהב אותך ผมรักคุณ

4. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 4 Translation is Hard! connocting poopie HELP ONESELF TERMINATING MACHINE

5. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 5 Machine Translation 布什与沙龙举行了会谈 bushiyushalongjuxinglehuitan BushheldatalkwithSharon

6. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 6 Word-based MT bushiyushalongjuxinglehuitan BushheldatalkwithSharon (Brown et al., 1993)

7. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 7 Phrase-based MT bushiyushalongjuxinglehuitan BushheldatalkwithSharon (Koehn et al., 2003; Och and Ney, 2004)

8. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 8 Hierarchical Phrase-based MT bushiyushalongjuxinglehuitan BushheldatalkwithSharon (Chiang, 2005; Chiang, 2007) X1X1 X1X1 X2X2 X2X2 X3X3 X3X3 X 3 ->(X 1 yu shalong X 2, X 1 X 2 with Sharon)

9. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 9 Syntax-based MT bushiyushalongjuxinglehuitan BushheldatalkwithSharon NNPVBDDTNNINNNP NP PP VP NP S (Yamda and Knight, 2001; Galley et al., 2006; Shen et al., 2008)

10. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 10 Motivation Human Translation  Understand the source sentence  Generate the target sentence Compiling  Parse input program into a syntax tree  Generate code in machine language

11. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 11 Syntax-Directed Translation for Compiling Input: y:=3*x+z Parsing: := id+ * constid (y) (3)(x) (z) (Irons, 1961; Lewis and Stearns, 1968; Aho and Ullman., 1972)

12. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 12 Motivation Human Translation  Understand the source sentence  Generate the target sentence Compiling  Parse input program into a syntax tree  Generate code in machine language Machine Translation  Parse the source sentence into a tree  Recursively transfer the tree into the target language

13. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 13 Syntax-Directed Translation for MT Input: bushi yu shalong juxing le huitan Parsing: bushiyushalongjuxinglehuitan NPBP VSASNPB PPVPB VP IP (Huang et al., 2006)

14. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 14 Outline Part 1: Tree-based Translation  Overview and Motivation  Tree-to-String Model and Decoding  Tree-to-String Rule Extraction  Language Model-Integrated Decoding: Cube Pruning Part 2: Forest-based Translation  Packed Forest  Forest-based Decoding  Forest-based Rule Extraction Part 3: Extensions  Tree-to-Tree Translation  Tree Sequence-based Translation  Joint Parsing and Translation Part 4: Conclusion

15. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 15 Tree-to-String Translation Recursive rewrite by pattern-matching bushiyushalongjuxinglehuitan NPBP VSASNPB PPVPB VP IP X 1 :NPBX 2 :VP X1X1 X2X2 (Liu et al., 2006; Huang et al., 2006)

16. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 16 Tree-to-String Translation Recursive rewrite by pattern-matching bushiyushalongjuxinglehuitan NPBP VSASNPB PPVPB VP (Liu et al., 2006; Huang et al., 2006) bushi NPB Bush

17. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 17 Tree-to-String Translation Recursive rewrite by pattern-matching Bushyushalongjuxinglehuitan PNPBVSASNPB PPVPB VP (Liu et al., 2006; Huang et al., 2006) VP X 1 :PPX 2 :VPB X2X2 X1X1

18. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 18 Tree-to-String Translation Recursive rewrite by pattern-matching Bush yushalong PNPB PP juxinglehuitan VSASNPB VPB (Liu et al., 2006; Huang et al., 2006) VPB VSASX 1 :NPB lejuxing heldaX1X1

19. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 19 Tree-to-String Translation Recursive rewrite by pattern-matching Bush yushalong PNPB PP heldahuitan NPB (Liu et al., 2006; Huang et al., 2006) huitan NPB talk

20. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 20 Tree-to-String Translation Recursive rewrite by pattern-matching Bush yushalong PNPB PP heldatalk (Liu et al., 2006; Huang et al., 2006) yu PX 1 :NPB PP withX1X1

21. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 21 Tree-to-String Translation Recursive rewrite by pattern-matching Bushwithshalong NPB heldatalk (Liu et al., 2006; Huang et al., 2006) shalong NPB Sharon

22. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 22 Tree-to-String Translation Recursive rewrite by pattern-matching BushwithSharonheldatalk (Liu et al., 2006; Huang et al., 2006) Syntax-directed translation (e.g., Irons, 1961) Tree transducer (e.g., Knight and Graehl, 2005) Synchronous grammar (e.g., Eisner, 2003) … Tree-to-string translation

23. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 23 Expressive Power PP PNPB yushalong withSharon phrasal translationnon-constituent phrasenon-contiguous phrase VPB VSAS X 1 : NPB lejuxing heldaX1X1 PP P LCP dang X 1 : IP LC hou whenX1X1 word omission QP X 1 :CDCLP ben X1X1 multilevel re-ordering IP X 1 :NPVP X 2 : IP X 3 :VPB X1X1 X3X3 X2X2 lexicalized re-ordering NP DNPX 2 :NP X 1 : NP DEG X2X2 ofX1X1 de (Knight and Graehl, 2005)

24. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 24 Outline Part 1: Tree-based Translation  Overview and Motivation  Tree-to-String Model and Decoding  Tree-to-String Rule Extraction  Language Model-Integrated Decoding: Cube Pruning Part 2: Forest-based Translation  Packed Forest  Forest-based Decoding  Forest-based Rule Extraction Part 3: Extensions  Tree-to-Tree Translation  Tree Sequence-based Translation  Joint Parsing and Translation Part 4: Conclusion

25. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 25 NPBASVSNPBP Tree-to-String Rule Extraction Compute target spans BushheldatalkwithSharon bushiyushalongjuxinglehuitan “Bush”“with”“Sharon”“held” “talk” “with Sharon”“held a talk” “held … Sharon” “Bush … Sharon” NPB VPB VP IP PP (Galley et al., 2004)

26. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 26 NPBASVSNPBP Tree-to-String Rule Extraction Find admissible nodes BushheldatalkwithSharon bushiyushalongjuxinglehuitan “Bush”“with”“Sharon”“held” “talk” “with Sharon”“held a talk” “held … Sharon” “Bush … Sharon” NPB VPB VP IP PP (Galley et al., 2004)

27. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 27 NPBASVSNPBP Tree-to-String Rule Extraction Extract minimal rules BushheldatalkwithSharon bushiyushalongjuxinglehuitan “Bush”“with”“Sharon”“held” “talk” “with Sharon”“held a talk” “held … Sharon” “Bush … Sharon” NPB VPB VP IP PP (Galley et al., 2004) bushi NPB Bush

28. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 28 NPBASVSNPBP Tree-to-String Rule Extraction Extract minimal rules BushheldatalkwithSharon bushiyushalongjuxinglehuitan “Bush”“with”“Sharon”“held” “talk” “with Sharon”“held a talk” “held … Sharon” “Bush … Sharon” NPB VPB VP IP PP (Galley et al., 2004) VP X 1 :PPX 2 :VPB X2X2 X1X1

29. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 29 NPBASVSNPBP Tree-to-String Rule Extraction Extract minimal rules BushheldatalkwithSharon bushiyushalongjuxinglehuitan “Bush”“with”“Sharon”“held” “talk” “with Sharon”“held a talk” “held … Sharon” “Bush … Sharon” NPB VPB VP IP PP (Galley et al., 2004) VPB VSASX 1 :NPB lejuxing heldaX1X1

30. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 30 Tree-to-String Rule Extraction Get composed rules VPB VSASX 1 :NPB lejuxing heldaX1X1 huitan NPB talk VPB VSASNPB lejuxing helda huitan talk += (Galley et al., 2006) tree substitution

31. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 31 Outline Part 1: Tree-based Translation  Overview and Motivation  Tree-to-String Model and Decoding  Tree-to-String Rule Extraction  Language Model-Integrated Decoding: Cube Pruning Part 2: Forest-based Translation  Packed Forest  Forest-based Decoding  Forest-based Rule Extraction Part 3: Extensions  Tree-to-Tree Translation  Tree Sequence-based Translation  Joint Parsing and Translation Part 4: Conclusion

32. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 32 Bottom-up Decoding bushiyushalongjuxinglehuitan NPBP VSASNPB PPVPB VP IP bushi NPB Bush (Liu et al., 2006; Huang et al., 2006)

33. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 33 Bottom-up Decoding bushiyushalongjuxinglehuitan NPBP VSASNPB PPVPB VP IP Bush shalong NPB Sharon (Liu et al., 2006; Huang et al., 2006)

34. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 34 Bottom-up Decoding bushiyushalongjuxinglehuitan NPBP VSASNPB PPVPB VP IP Bush Sharon huitan NPB talk (Liu et al., 2006; Huang et al., 2006)

35. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 35 Bottom-up Decoding bushiyushalongjuxinglehuitan NPBP VSASNPB PPVPB VP IP Bush Sharontalk yu PX 1 :NPB PP withX1X1 with Sharon (Liu et al., 2006; Huang et al., 2006)

36. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 36 Bottom-up Decoding bushiyushalongjuxinglehuitan NPBP VSASNPB PPVPB VP IP Bush Sharontalk with Sharon VPB VSASX 1 :NPB lejuxing heldaX1X1 held a talk (Liu et al., 2006; Huang et al., 2006)

37. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 37 Bottom-up Decoding bushiyushalongjuxinglehuitan NPBP VSASNPB PPVPB VP IP Bush Sharontalk with Sharon held a talk VP X 1 :PPX 2 :VPB X2X2 X1X1 held a talk with Sharon (Liu et al., 2006; Huang et al., 2006)

38. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 38 Bottom-up Decoding bushiyushalongjuxinglehuitan NPBP VSASNPB PPVPB VP IP Bush Sharontalk with Sharon held a talk held a talk with Sharon IP X 1 :NPBX 2 :VP X1X1 X2X2 Bush held a talk with Sharon (Liu et al., 2006; Huang et al., 2006)

39. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 39 Beam Search bushiyushalongjuxinglehuitan NPBP VSASNPB PPVPB VP IP (Liu et al., 2006; Huang et al., 2006)

40. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 40 Exhaustive Search VP X 1 :PPX 2 :VPB X2X2 X1X1 held a talk held talks hold a talk hold talks VPB 3,6 PP 1,3 VP 1,6 held a talk with Sharon held a talk and Sharon held talks with Sharon held talks and Sharon … with Sharon and Sharon Sharon with Sharon and

41. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 41 Update Bigram LM Probability with Sharonheld a talk p1=p(“with’’) * p(“Sharon”|”with”)p2=p(“held’’) * p(“a”|”held”)*p(“talk”|”a”) with Sharon held a talk held a talk with Sharon p1*p2*p(“held”|”Sharon”)/p(“held”) p1*p2*p(“with”|”talk”)/p(“with”) Only boundary words are used to update LM probability!

42. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 42 Exhaustive Search with a Bigram Language Model VP X 1 :PPX 2 :VPB X2X2 X1X1 held * talk held * talks hold * talk hold * talks VPB 3,6 PP 1,3 VP 1,6 held * Sharon hold * Sharon held * with hold * with … with * Sharon and * Sharon Sharon * with Sharon * and

43. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 43 Monotonicity VPB 3,6 PP 1,3 VP 1,6 1.03.04.0 1.02.04.05.0 1.12.14.15.1 2.06.0 held * talk held * talks hold * talk hold * talks with * Sharon and * SharonSharon * withSharon * and 6.5 7.5 7.6 8.5 3.54.56.57.510.0 3.05.0 VPB 3,6 PP 1,3 (Huang and Chiang, 2005, 2007; Chiang, 2007) monotonic

44. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 44 Non-Monotonicity VPB 3,6 PP 1,3 VP 1,6 1.03.04.0 1.02.0 + 0.54.0 + 2.05.0 + 4.0 1.12.1 + 0.34.1 + 1.55.1 + 3.5 2.06.0 + 4.0 held * talk held * talks hold * talk hold * talks with * Sharon and * SharonSharon * withSharon * and 6.5 7.5 + 4.0 7.6 + 3.0 8.5 + 4.0 3.54.5 + 0.36.5 + 1.57.5 + 3.510 + 3.5 3.0 + 0.55.0 + 2.0 VPB 3,6 PP 1,3 (Huang and Chiang, 2005, 2007; Chiang, 2007) log(p(with|talk))- log(p(with)) LM introduces non-monotonicity

45. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 45 Cube Pruning 1.03.04.0 1.02.56.09.0 1.12.45.68.6 2.010.0 held * talk held * talks hold * talk hold * talks with * Sharon and * SharonSharon * withSharon * and 6.5 11.5 10.6 12.5 3.54.88.011.013.5 3.57.0 VPB 3,6 PP 1,3 queue 4-best (Huang and Chiang, 2005, 2007; Chiang, 2007)

46. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 46 Cube Pruning 1.03.04.0 1.02.56.09.0 1.12.45.68.6 2.010.0 held * talk held * talks hold * talk hold * talks with * Sharon and * SharonSharon * withSharon * and 6.5 11.5 10.6 12.5 3.54.88.011.013.5 3.57.0 VPB 3,6 PP 1,3 queue 4-best 2.5 (Huang and Chiang, 2005, 2007; Chiang, 2007)

47. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 47 Cube Pruning 1.03.04.0 1.02.56.09.0 1.12.45.68.6 2.010.0 held * talk held * talks hold * talk hold * talks with * Sharon and * SharonSharon * withSharon * and 6.5 11.5 10.6 12.5 3.54.88.011.013.5 3.57.0 VPB 3,6 PP 1,3 queue 4-best2.5 2.46.0 (Huang and Chiang, 2005, 2007; Chiang, 2007)

48. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 48 Cube Pruning 1.03.04.0 1.02.56.09.0 1.12.45.68.6 2.010.0 held * talk held * talks hold * talk hold * talks with * Sharon and * SharonSharon * withSharon * and 6.5 11.5 10.6 12.5 3.54.88.011.013.5 3.57.0 VPB 3,6 PP 1,3 queue 4-best2.4 3.55.6 2.5 6.0 (Huang and Chiang, 2005, 2007; Chiang, 2007)

49. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 49 Cube Pruning 1.03.04.0 1.02.56.09.0 1.12.45.68.6 2.010.0 held * talk held * talks hold * talk hold * talks with * Sharon and * SharonSharon * withSharon * and 6.5 11.5 10.6 12.5 3.54.88.011.013.5 3.57.0 VPB 3,6 PP 1,3 queue 4-best2.4 4.8 2.5 5.6 3.5 (Huang and Chiang, 2005, 2007; Chiang, 2007) 6.07.0

50. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 50 Cube Pruning 1.03.04.0 1.02.56.09.0 1.12.45.68.6 2.010.0 held * talk held * talks hold * talk hold * talks with * Sharon and * SharonSharon * withSharon * and 6.5 11.5 10.6 12.5 3.54.88.011.013.5 3.57.0 VPB 3,6 PP 1,3 queue 4-best2.44.82.5 5.6 3.5 (Huang and Chiang, 2005, 2007; Chiang, 2007) 6.07.0

51. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 51 Cube Pruning within Rule Group VPB VSASX 1 :NPB lejuxing heldaX1X1 VPB VSASX 1 :NPB lejuxing heldX1X1 VPB VSASX 1 :NPB lejuxing holdaX1X1 1.0 1.4 2.0 1.02.02.5 2.1 3.2 3.1 5.0 4.0 6.0 3.7 5.0 4.7 heldaX1X1 X1X1 holdaX1X1 talktalksmeeting Group rules that have the same LHS NPB 5,6 (Huang and Chiang, 2005, 2007; Chiang, 2007)

52. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 52 Cube Pruning within Node VP 1,6 PP 1,3 VPB 3,6 NPB 2,3 NPB 5,6 NPB 2,3 VPB 3,6 process all rules simultaneously! significant savings of computation (Huang and Chiang, 2005, 2007; Chiang, 2007)

53. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 53 Outline Part 1: Tree-based Translation  Overview and Motivation  Tree-to-String Model and Decoding  Tree-to-String Rule Extraction  Language Model-Integrated Decoding: Cube Pruning Part 2: Forest-based Translation  Packed Forest  Forest-based Decoding  Forest-based Rule Extraction Part 3: Extensions  Tree-to-Tree Translation  Tree Sequence-based Translation  Joint Parsing and Translation Part 4: Conclusion

54. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 54 Syntactic Ambiguity bushiyu shalongjuxing lehuitan NPBP VSASNPB PPVPB VP IP bushiyu shalongjuxing lehuitan NPBCCNPBVSASNPB NPVPB IP withand ``Bush held a talk with Sharon’’ ``Bush and Sharon held a talk’’ It is important to choose a correct tree for producing a good translation!

55. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 55 Parsing Mistake Propagation parsing mistakes potentially introduce translation mistakes! (Quirk and Corston-Oliver, 2006) parsetranslate source target stringtree string

56. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 56 1-best Trees => n-best Trees? bushiyu shalongjuxing lehuitan NPBP VSASNPB PPVPB VP IP bushiyu shalongjuxing lehuitan NPBCCNPBVSASNPB NPVPB IP Very few variations among the n-best trees!

57. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 57 Packed Forest bushiyushalongjuxinglehuitan NPB 0,1 CC 1,2 NPB 2,3 VS 3,4 AS 4,5 NPB 5,6 PP 1,3 VPB 3,6 VP 1,6 IP 0,6 P 1,2 NP 0,3 (Billot and Lang, 1989; Klein and Manning, 2001; Huang and Chiang, 2005)

58. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 58 Outline Part 1: Tree-based Translation  Overview and Motivation  Tree-to-String Model and Decoding  Tree-to-String Rule Extraction  Language Model-Integrated Decoding: Cube Pruning Part 2: Forest-based Translation  Packed Forest  Forest-based Decoding  Forest-based Rule Extraction Part 3: Extensions  Tree-to-Tree Translation  Tree Sequence-based Translation  Joint Parsing and Translation Part 4: Conclusion

59. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 59 Pattern Matching on Forest bushiyushalongjuxinglehuitan NPB 0,1 CC 1,2 NPB 2,3 VS 3,4 AS 4,5 NPB 5,6 PP 1,3 VPB 3,6 VP 1,6 IP 0,6 P 1,2 NP 0,3 IP NPX 3 :VPB X 1 :NPBCCX 2 :NPB yu X1X1 X3X3 withX2X2 (Mi et al., 2008)

60. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 60 Translation Forest (Mi et al., 2008) bushiyushalongjuxinglehuitan NPB 0,1 CC 1,2 NPB 2,3 VS 3,4 AS 4,5 NPB 5,6 PP 1,3 VPB 3,6 VP 1,6 IP 0,6 P 1,2 NP 0,3 bushi NPB Bush NPB 0,1

61. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 61 Translation Forest (Mi et al., 2008) bushiyushalongjuxinglehuitan NPB 0,1 CC 1,2 NPB 2,3 VS 3,4 AS 4,5 NPB 5,6 PP 1,3 VPB 3,6 VP 1,6 IP 0,6 P 1,2 NP 0,3 bushi NPB Bush NPB 0,1 shalong NPB Sharon NPB 2,3

62. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 62 Translation Forest (Mi et al., 2008) bushiyushalongjuxinglehuitan NPB 0,1 CC 1,2 NPB 2,3 VS 3,4 AS 4,5 NPB 5,6 PP 1,3 VPB 3,6 VP 1,6 IP 0,6 P 1,2 NP 0,3 bushi NPB Bush NPB 0,1 shalong NPB Sharon NPB 2,3 huitan NPB talk NPB 5,6

63. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 63 Translation Forest (Mi et al., 2008) bushiyushalongjuxinglehuitan NPB 0,1 CC 1,2 NPB 2,3 VS 3,4 AS 4,5 NPB 5,6 PP 1,3 VPB 3,6 VP 1,6 IP 0,6 P 1,2 NP 0,3 bushi NPB Bush NPB 0,1 shalong NPB Sharon NPB 2,3 huitan NPB talk NPB 5,6 VPB VSASX 1 :NPB lejuxing heldaX1X1 VPB 3,6

64. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 64 Translation Forest (Mi et al., 2008) bushiyushalongjuxinglehuitan NPB 0,1 CC 1,2 NPB 2,3 VS 3,4 AS 4,5 NPB 5,6 PP 1,3 VPB 3,6 VP 1,6 IP 0,6 P 1,2 NP 0,3 bushi NPB Bush NPB 0,1 shalong NPB Sharon NPB 2,3 huitan NPB talk NPB 5,6 VPB VSASX 1 :NPB lejuxing heldaX1X1 VPB 3,6 IP NP X 3 : VPB X 1 : NPB CC X 2 : NPB yu X1X1 X3X3 withX2X2 IP 0,6 ``Bush’’``Sharon’’``talk’’ ``held a talk’’ ``Bush held a talk with Sharon’’

65. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 65 N-best Trees Vs. Forest (Mi et al., 2008)

66. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 66 Forest as Virtual ∞ -best list How often is the ith-best tree picked by the decoder? (Mi et al., 2008)

67. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 67 Outline Part 1: Tree-based Translation  Overview and Motivation  Tree-to-String Model and Decoding  Tree-to-String Rule Extraction  Language Model-Integrated Decoding: Cube Pruning Part 2: Forest-based Translation  Packed Forest  Forest-based Decoding  Forest-based Rule Extraction Part 3: Extensions  Tree-to-Tree Translation  Tree Sequence-based Translation  Joint Parsing and Translation Part 4: Conclusion

68. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 68 Forest-based Rule Extraction bushiyushalongjuxinglehuitan NPB 0,1 CC 1,2 NPB 2,3 VS 3,4 AS 4,5 NPB 5,6 PP 1,3 VPB 3,6 VP 1,6 IP 0,6 P 1,2 NP 0,3 Compute target spans BushheldatalkwithSharon “Bush”“with” “Sharon”“held” “talk” “Bush □ with Sharon” “with Sharon”“held … talk” “held … Sharon” “Bush … Sharon” (Mi and Huang, 2008)

69. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 69 Forest-based Rule Extraction bushiyushalongjuxinglehuitan NPB 0,1 CC 1,2 NPB 2,3 VS 3,4 AS 4,5 NPB 5,6 PP 1,3 VPB 3,6 VP 1,6 IP 0,6 P 1,2 NP 0,3 Compute admissible nodes BushheldatalkwithSharon “Bush”“with” “Sharon”“held” “talk” “Bush □ with Sharon” “with Sharon”“held … talk” “held … Sharon” “Bush … Sharon” (Mi and Huang, 2008)

70. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 70 Forest-based Rule Extraction bushiyushalongjuxinglehuitan NPB 0,1 CC 1,2 NPB 2,3 VS 3,4 AS 4,5 NPB 5,6 PP 1,3 VPB 3,6 VP 1,6 IP 0,6 P 1,2 NP 0,3 Extract minimal rules BushheldatalkwithSharon “Bush”“with” “Sharon”“held” “talk” “Bush □ with Sharon” “with Sharon”“held … talk” “held … Sharon” “Bush … Sharon” bushi NPB Bush (Mi and Huang, 2008)

71. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 71 Forest-based Rule Extraction bushiyushalongjuxinglehuitan NPB 0,1 CC 1,2 NPB 2,3 VS 3,4 AS 4,5 NPB 5,6 PP 1,3 VPB 3,6 VP 1,6 IP 0,6 P 1,2 NP 0,3 Extract minimal rules BushheldatalkwithSharon “Bush”“with” “Sharon”“held” “talk” “Bush □ with Sharon” “with Sharon”“held … talk” “held … Sharon” “Bush … Sharon” VP X 1 :PPX 2 :VPB X2X2 X1X1 (Mi and Huang, 2008)

72. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 72 Forest-based Rule Extraction bushiyushalongjuxinglehuitan NPB 0,1 CC 1,2 NPB 2,3 VS 3,4 AS 4,5 NPB 5,6 PP 1,3 VPB 3,6 VP 1,6 IP 0,6 P 1,2 NP 0,3 Extract minimal rules BushheldatalkwithSharon “Bush”“with” “Sharon”“held” “talk” “Bush □ with Sharon” “with Sharon”“held … talk” “held … Sharon” “Bush … Sharon” VPB VSASX 1 :NPB lejuxing heldaX1X1 (Mi and Huang, 2008)

73. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 73 Forest-based Rule Extraction bushiyushalongjuxinglehuitan NPB 0,1 CC 1,2 NPB 2,3 VS 3,4 AS 4,5 NPB 5,6 PP 1,3 VPB 3,6 VP 1,6 IP 0,6 P 1,2 NP 0,3 Extract minimal rules BushheldatalkwithSharon “Bush”“with” “Sharon”“held” “talk” “Bush □ with Sharon” “with Sharon”“held … talk” “held … Sharon” “Bush … Sharon” IP X 1 :NPBX 2 :VP X1X1 X2X2 (Mi and Huang, 2008)

74. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 74 Forest-based Rule Extraction bushiyushalongjuxinglehuitan NPB 0,1 CC 1,2 NPB 2,3 VS 3,4 AS 4,5 NPB 5,6 PP 1,3 VPB 3,6 VP 1,6 IP 0,6 P 1,2 NP 0,3 Extract minimal rules BushheldatalkwithSharon “Bush”“with” “Sharon”“held” “talk” “Bush □ with Sharon” “with Sharon”“held … talk” “held … Sharon” “Bush … Sharon” IP NPX 4 :VPB X 1 :NPBX 2 :CCX 3 :NPB X1X1 X3X3 X4X4 X2X2 (Mi and Huang, 2008)

75. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 75 Rule Probabilities and Rule Count How often does a rule occur in training examples?

76. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 76 Fractional Count bushiyushalongjuxinglehuitan NPB 0,1 CC 1,2 NPB 2,3 VS 3,4 AS 4,5 NPB 5,6 PP 1,3 VPB 3,6 VP 1,6 IP 0,6 P 1,2 NP 0,3 BushheldatalkwithSharon “Bush”“with” “Sharon”“held” “talk” “Bush □ with Sharon” “with Sharon”“held … talk” “held … Sharon” “Bush … Sharon” VP X 1 :PPX 2 :VPB X2X2 X1X1 (Mi and Huang, 2008) Q: What ‘s the count of this rule on this training example?

77. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 77 Fractional Count bushiyushalongjuxinglehuitan NPB 0,1 CC 1,2 NPB 2,3 VS 3,4 AS 4,5 NPB 5,6 PP 1,3 VPB 3,6 VP 1,6 IP 0,6 P 1,2 NP 0,3 BushheldatalkwithSharon “Bush”“with” “Sharon”“held” “talk” “Bush □ with Sharon” “with Sharon” “held … talk” “held … Sharon” “Bush … Sharon” (Mi and Huang, 2008)

78. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 78 Results 1-best treeforest 1-best tree0.25600.2674 forest0.26790.2816 decoding rule extraction (Mi and Huang, 2008)

79. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 79 Outline Part 1: Tree-based Translation  Overview and Motivation  Tree-to-String Model and Decoding  Tree-to-String Rule Extraction  Language Model-Integrated Decoding: Cube Pruning Part 2: Forest-based Translation  Packed Forest  Forest-based Decoding  Forest-based Rule Extraction Part 3: Extensions  Tree-to-Tree Translation  Tree Sequence-based Translation  Joint Parsing and Translation Part 4: Conclusion

80. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 80 Tree-to-Tree Translation Recursive rewrite by pattern-matching bushiyu shalongjuxing lehuitan NPBP VSASNPB PPVPB VP IP NP VP S IP X 1 :NPBX 2 :VP S X 1 :NPX 2 :VP (Eisner 2003, Zhang, 2007)

81. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 81 Tree-to-Tree Translation Recursive rewrite by pattern-matching bushiyu shalongjuxing lehuitan NPBP VSASNPB PPVPB VP NP VP S NP Bush NNP bushi NPB Bush NNP NP (Eisner 2003, Zhang, 2007)

82. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 82 Tree-to-Tree Translation Recursive rewrite by pattern-matching yu shalongjuxing lehuitan PNPBVSASNPB PPVPB VP NP VP S NP Bush NNP juxing le VSASX 2 :NPB X 1 :PPVPB VP helda VBDDTX 2 :NN NP X 1 :PP VP helda VBDDTNN NP PP (Eisner 2003, Zhang, 2007)

83. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 83 Tree-to-Tree Translation Recursive rewrite by pattern-matching yu shalong huitan PNPB PP NP VP S NP Bush NNP helda VBDDTNN NP PP yu shalong PNPB PP withSharon INNNP NP PP withSharon INNNP NP (Eisner 2003, Zhang, 2007)

84. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 84 Tree-to-Tree Translation Recursive rewrite by pattern-matching huitan NPB NP VP S NP Bush NNP helda VBDDTNN NP PP withSharon INNNP NP huitan NPB talk NN talk (Eisner 2003, Zhang, 2007)

85. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 85 Tree-to-Tree Translation Recursive rewrite by pattern-matching NP VP S NP Bush NNP helda VBDDTNN NP PP withSharon INNNP NP talk (Eisner 2003, Zhang, 2007)

86. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 86 Tree-to-Tree Rule Extraction Find admissible node pairs bushiyu shalongjuxing lehuitan NPBP VSASNPB PPVPB VP IP BushheldatalkwithSharon NNPVBDDTNNINNNP NP PP VP S (Zhang, 2007, Liu et al., 2009a)

87. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 87 Tree-to-Tree Rule Extraction Extract minimal rules bushiyu shalongjuxing lehuitan NPBP VSASNPB PPVPB VP IP BushheldatalkwithSharon NNPVBDDTNNINNNP NP PP VP S bushi NPB Bush NNP NP (Zhang, 2007, Liu et al., 2009a)

88. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 88 Tree-to-Tree Rule Extraction Extract minimal rules bushiyu shalongjuxing lehuitan NPBP VSASNPB PPVPB VP IP BushheldatalkwithSharon NNPVBDDTNNINNNP NP PP VP S juxing le VSASX 2 :NPB X 1 :PPVPB VP helda VBDDTX 2 :NN NP X 1 :PP VP (Zhang, 2007, Liu et al., 2009a)

89. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 89 Tree-to-Tree Rule Extraction Extract minimal rules bushiyu shalongjuxing lehuitan NPBP VSASNPB PPVPB VP IP BushheldatalkwithSharon NNPVBDDTNNINNNP NP PP VP S IP X 1 :NPBX 2 :VP S X 1 :NPX 2 :VP (Zhang, 2007, Liu et al., 2009a)

90. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 90 Tree-to-Tree Rule Extraction Get composed rules juxing le VSASX 2 :NPB X 1 :PPVPB VP helda VBDDTX 2 :NN NP X 1 :PP VP huitan NPB talk NN juxing le VSASNPB X 1 :PPVPB VP helda VBDDTNN NP X 1 :PP VP huitan talk += (Zhang, 2007, Liu et al., 2009a)

91. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 91 Challenges Tree-to-tree translation is over-constrained  Poorest rule coverage  Suffers from parsing mistake propagation on both sides Recent advances  Use tree sequence (Zhang et al., 2008)  Use packed forest (Liu et al., 2009a)  Fuzzy extraction and decoding (Chiang, 2010)

92. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 92 Outline Part 1: Tree-based Translation  Overview and Motivation  Tree-to-String Model and Decoding  Tree-to-String Rule Extraction  Language Model-Integrated Decoding: Cube Pruning Part 2: Forest-based Translation  Packed Forest  Forest-based Decoding  Forest-based Rule Extraction Part 3: Extensions  Tree-to-Tree Translation  Tree Sequence-based Translation  Joint Parsing and Translation Part 4: Conclusion

93. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 93 Non-Constituent Phrase Pairs bushiyushalongjuxinglehuitan NPBCCNPBVSASNPB NPVPB IP BushheldatalkwithSharon (Marcu et al., 2006)

94. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 94 Non-Constituent Phrase Pairs bushiyushalongjuxinglehuitan BushheldatalkwithSharon NNPVBDDTNNINNNP NP PP VP S (Marcu et al., 2006)

95. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 95 Non-Constituent Phrase Pairs bushiyu shalongjuxing lehuitan NPBP VSASNPB PPVPB VP IP BushheldatalkwithSharon NNPVBDDTNNINNNP NP PP VP S (Marcu et al., 2006)

96. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 96 Rule Coverage bushiyu shalongjuxing lehuitan NPBP VSASNPB PPVPB VP IP BushheldatalkwithSharon NNPVBDDTNNINNNP NP PP VP S (bushi, Bush) (yu, with) (shalong, Sharon) (huitan, talk) (yu shalong, with Sharon) (juxing le, held) (juxing … huitan, held … talk) (yu … huitan, held … Sharon) (bushi … huitan, Bush … Sharon) s2st2ss2tt2t √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ × × × × phrase pair 100%89% 78%

97. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 97 Rule Coverage modelhumanautomatic string-to-string100% tree-to-string78%75% string-to-tree76%72% tree-to-tree68%60% Results from (Chiang, 2010)

98. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 98 Solutions Extend to larger rules bushiyushalongjuxinglehuitan NPBCCNPBVSASNPB NPVPB IP BushheldatalkwithSharon VPB VSASX 1 :NPB lejuxing heldaX1X1 (Galley et al., 2006)

99. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 99 Solutions Add pseudo nodes bushiyushalongjuxinglehuitan NPBCCNPBVSASNPB NPVPB IP BushheldatalkwithSharon VPB X 1 :*VPB_*NPBX 2 :NPB X1X1 aX2X2 *VPB_*NPB VSAS lejuxing held (Marcu et al., 2006)

100. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 100 Solutions Use tree sequences bushiyushalongjuxinglehuitan NPBCCNPBVSASNPB NPVPB IP BushheldatalkwithSharon VPB X 1 :VSX 2 :NPB X1X1 aX2X2 VSAS lejuxing held X 1 :AS (Liu et al., 2007; Zhang et al., 2008)

101. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 101 Tree-Sequence + Forest systeminputruleBLEU Mosesstringstring-to-string25.7 tree-to-string tree tree-to-string26.1 tree-sequence-to-string27.0 forest tree-to-string27.7 tree-sequence-to-string28.8 Results from (Zhang et al., 2009)

102. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 102 Other Solutions Re-structure syntax-trees (Wang et al., 2007) Offer more trees (Mi and Huang, 2008) Re-align syntax trees and strings (May and Knight, 2007) Well-formed dependency structures (Shen et al., 2008) Gibbs sampling (Cohn and Blunsom, 2009) Joint decoding (Liu et al., 2009b)

103. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 103 Outline Part 1: Tree-based Translation  Overview and Motivation  Tree-to-String Model and Decoding  Tree-to-String Rule Extraction  Language Model-Integrated Decoding: Cube Pruning Part 2: Forest-based Translation  Packed Forest  Forest-based Decoding  Forest-based Rule Extraction Part 3: Extensions  Tree-to-Tree Translation  Tree Sequence-based Translation  Joint Parsing and Translation Part 4: Conclusion

104. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 104 Separate Parsing and Translation parsetranslate source target stringtree/forest string Separate grammar for parsing and translation decoding is fast!

105. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 105 Joint Parsing and Translation parse + translate source target stringtree/forest string Its search space is larger than tree/forest It is a translator as well as a parser Parsing interacts with translation (Liu and Liu, 2010)

106. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 106 Tree-to-String Translation as Parsing bushiyushalongjuxinglehuitan bushi NPB Bush NPB Bush (Liu and Liu, 2010)

107. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 107 Tree-to-String Translation as Parsing bushiyushalongjuxinglehuitan yu P with NPBP withBush (Liu and Liu, 2010)

108. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 108 Tree-to-String Translation as Parsing bushiyushalongjuxinglehuitan shalong NPB Sharon NPBP withBush NPB Sharon (Liu and Liu, 2010)

109. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 109 Tree-to-String Translation as Parsing bushiyushalongjuxinglehuitan NPB talk NPBP withBush NPB Sharon NPB talk (Liu and Liu, 2010)

110. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 110 Tree-to-String Translation as Parsing bushiyushalongjuxinglehuitan NPBP Bush NPB withSharon NPB talk X 1 :P PP X 1 X 2 X 2 :NPB PP (Liu and Liu, 2010)

111. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 111 Tree-to-String Translation as Parsing bushiyushalongjuxinglehuitan NPBP Bush NPB with Sharon NPB talk X 1 :P PP X 1 X 2 X 2 :NPB PP (Liu and Liu, 2010)

112. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 112 Tree-to-String Translation as Parsing bushiyushalongjuxinglehuitan NPBP Bush NPB with Sharon NPB talk PP VPB VSASX 1 :NPB lejuxing heldaX1X1 VSAS VPB (Liu and Liu, 2010)

113. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 113 Tree-to-String Translation as Parsing bushiyushalongjuxinglehuitan NPBP Bush NPB with Sharon NPB PP VPB VSASX 1 :NPB lejuxing heldaX1X1 VSAS VPB heldatalk (Liu and Liu, 2010)

114. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 114 Tree-to-String Translation as Parsing bushiyushalongjuxinglehuitan NPBP Bush NPB PP VSAS VPB with Sharonheldatalk X 1 :PP VP X 2 X 1 X 2 :VPB VP (Liu and Liu, 2010)

115. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 115 Tree-to-String Translation as Parsing bushiyushalongjuxinglehuitan NPBP Bush NPB PP VSAS VPB X 1 :PP VP X 2 X 1 X 2 :VPB VP heldatalkwithSharon (Liu and Liu, 2010)

116. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 116 Tree-to-String Translation as Parsing bushiyushalongjuxinglehuitan NPBP PP VSAS VPB X 1 :NPB IP X 1 X 2 X 2 :VP VP BushheldatalkwithSharon IP (Liu and Liu, 2010)

117. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 117 Tree-to-String Translation as Parsing bushiyushalongjuxinglehuitan NPBP PP VSAS VPB X 1 :NPB IP X 1 X 2 X 2 :VP VP heldatalkwithSharon IP Bush (Liu and Liu, 2010)

118. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 118 Translation Evaluation algorithminput parsing model rulesBLEUtime matching treenone1.2M29.80.56 forestPCFG1.9M31.69.49 parsingstring none 7.7M 32.051.41 PCFG32.455.52 Lex32.689.35 PCFG+Lex32.791.72 (Liu and Liu, 2010)

119. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 119 Parsing Evaluation parsing modelF1time none62.723.9 PCFG65.424.7 Lex79.848.8 PCFG + Lex80.650.4 (Liu and Liu, 2010)

120. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 120 Results on Tree-to-Tree taskextractionrulesfeaturesBLEU Chinese string-to-string440M1K23.7 tree-to-tree50M5K23.9 Arabic string-to-string790M1K48.9 tree-to-tree38M5K47.5 Results from (Chiang, 2010)

121. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 121 Outline Part 1: Tree-based Translation  Overview and Motivation  Tree-to-String Model and Decoding  Tree-to-String Rule Extraction  Language Model-Integrated Decoding: Cube Pruning Part 2: Forest-based Translation  Packed Forest  Forest-based Decoding  Forest-based Rule Extraction Part 3: Extensions  Tree-to-Tree Translation  Tree Sequence-based Translation  Joint Parsing and Translation Part 4: Conclusion

122. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 122 Conclusion Statistical machine translation  Word-based  Phrase-based  Syntax-based String-to-String String-to-Tree Tree-to-String Tree-to-Tree flat hierarchical

123. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 123 Conclusion Tree-based translation  Pros: simplicity, faster decoding, expressive grammar, no need for binarization  Cons: commits to 1-best tree Forest-based translation  Compromise between tree-based and string-based, combining the advantages of both Fast decoding, but does not commit to 1-best trees Significant improvement of translation performance over tree-based

124. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 124 Thank you 谢谢 大家 VVPN VP

125. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 125 Bibliography Alfred V. Aho and Jeffrey D. Ullman. 1972. The Theory of Parsing, Translation, and Compiling, volume I: parsing. Prentice Hall, Englewood Cliffs, New Jersey. Peter F. Brown, Stephan A. Della Pietra, Vincent J. Della Pietra, and Robert L. Mercer. 1993. The mathematics of statistical machine translation: Parameter estimation. Computational Linguistics, 19(2): 263-311. Sylvie Billot and Bernard Lang. 1989. The structure of shared forests in ambiguous parsing. In Proceedings of ACL 1989. David Chiang, 2005. A hierarchical phrase-based model for statistical machine translation. In Proceedings of ACL 2005. David Chiang, 2007. Hierarchical phrase-based translation. Computational Linguistics, 33(2): 201-228. David Chiang, 2010. Learning to translate with source and target syntax. In Proceedings of ACL 2010. Trevor Cohn and Phil Blunsom. 2009. A bayesian model for syntax-directed tree to string grammar induction. In Proceedings of EMNLP 2009. Michel Galley, Mark Hopkins, Kevin Knight, and Daniel Marcu. 2004. What’s in a translation rule? In Proceedings of HLT-NAACL 2004. Michel Galley, Jonathan Graehl, Kevin Knight, Daniel Marcu, Steve DeNeefe, Wei Wang, and Ignacio Thayer. 2006. Scalable inference and training of context- rich syntactic translation models. In Proceedings of COLING-ACL 2006.

126. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 126 Bibliography Liang Huang and David Chiang. 2005. Better k-best parsing. In Proceedings of IWPT 2005. Liang Huang and David Chiang. 2007. Forest rescoring: Faster decoding with integrated language models. In Proceedings of ACL 2007. Liang Huang, Kevin Knight, and Aravind Joshi. 2006. Statistical syntax-directed translation with extended domain of locality. In Proceedings of AMTA 2006. Liang Huang. 2008. Forest reranking: Discriminative parsing with non-local features. In Proceedings. of ACL-HLT 2008. E. T. Irons. 1961. A syntax-directed compiler for ALGOL 60. Comm. ACM, 4(1): 51-55. Kevin Knight and Jonathan Graehl. 2005. An overview of probabilistic tree transducers for natural language processing. In Proceedings of CICLing 2005. Dan Klein and Christopher D. Manning. 2001. Parsing and hypergraphs. In Proceedings of IWPT 2001. Philipp Koehn, Franz Och, and Daniel Marcu. 2003. Statistical phrase-based translation In Proceedings of HLT-NAACL 2003. P. M. Lewis and R. E. Stearns. 1968. Syntax-directed transduction. Journal of the ACM, 15(3): 465-488. Yang Liu, Qun Liu, and Shouxun Lin. 2006. Tree-to-string alignment template for statistical machine translation. In Proceedings of COLING-ACL 2006.

127. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 127 Bibliography Yang Liu, Yun Huang, Qun Liu, and Shouxun Lin. 2007. Forest-to-string statistical translation rules. In Proceedings of ACL 2007. Yang Liu, Yajuan Lu, and Qun Liu. 2009a. Improving tree-to-tree translation with packed forests. In Proceedings of ACL-IJCNLP 2009. Yang Liu, Haitao Mi, Yang Feng, and Qun Liu. 2009b. Joint decoding with multiple translation models. In Proceeding of ACL-IJCNLP 2009. Yang Liu and Qun Liu. 2010. Joint parsing and translation. Submitted to COLING 2010. Daniel Marcu, Wei Wang, Abdessamad Echihabi, and Kevin Knight. 2006. SPMT: Statistical machine translation with syntactified target language phrases. In Proceedings of EMNLP 2006. Jonathan May and Kevin Knight. 2007. Syntax re-alignment models for machine translation. In Proceedings of EMNLP 2007. Haitao Mi and Liang Huang. 2008. Forest-based translation rule extraction. In Proceedings of EMNLP 2008. Haitao Mi, Liang Huang, and Qun Liu. 2008. Forest-based translation. In Proceedings of ACL-HLT 2008. Franz Och and Hermann Ney. 2004. The alignment template approach to statistical machine translation. Computational Linguistics, 30(4): 417-449.

128. July 11, 2010 ACL 2010 Tutorial, Uppsala, Sweden 128 Bibliography Libin Shen, Jinxi Xu, and Ralph Weischedel. 2008. A new string-to-dependency machine translation algorithm with a target dependency language model. In Proceedings of ACL-HLT 2008. Ashish Venugopal, Andreas Zollmann, Noah Smith, and Stephan Vogel. 2008. Wider pipelines: n-best alignments and parses in mt training. In Proceedings of AMTA 2008. Wei Wang, Kevin Knight, and Daniel Marcu. 2007. Binarizing syntax trees to improve syntax-based machine translation accuracy. In Proceedings of EMNLP 2007. Kenji Yamada and Kevin Knight. 2001. A syntax-based statistical machine translation model. In Proceedings of ACL 2001. Hui Zhang, Min Zhang, Haizhou Li, Aiti Aw, and Chew Lin Tan. 2009. Forest- based tree sequence to string translation model. In Proceedings of ACL-IJCNLP 2009. Min Zhang, Hongfei Jiang, Aiti Aw, Jun Sun, Sheng Li, and Chew Lin Tan. 2007. A tree-to-tree alignment-based model for statistical machine translation. In Proceedings of MT Summit 2007. Min Zhang, Hongfei Jiang, Aiti Aw, Haizhou Li, Chew Lin Tan, and Sheng Li. 2008. A tree sequence alignment-based tree-to-tree translation model. In Proceedings of ACL-HLT 2008.