1. Introduction to MT Ling 580 Fei Xia Week 1: 1/03/06

2. Outline Course overview Introduction to MT –Major challenges –Major approaches –Evaluation of MT systems Overview of word-based SMT

3. Course overview

4. General info Course website: –Syllabus (incl. slides and papers): updated every week. –Message board –ESubmit Office hour: Fri: 10:30am-12:30pm. Prerequisites: –Ling570 and Ling571. –Programming: C or C++, Perl is a plus. –Introduction to probability and statistics

5. Expectations Reading: –Papers are online –Finish reading before class. Bring your questions to class. Grade: –Leading discussion (1-2 papers): 50% –Project: 40% –Class participation: 10% –No quizzes, exams

6. Leading discussion Indicate your choice via EPost by Jan 8. You might want to read related papers. Make slides with PowerPoint. Email me your slides by 3:30am on the Monday before your presentation. Present the paper in class and lead the discussion: 40-50 minutes.

7. Project Details will be available soon. Project presentation: 3/7/06 Final report: due on 3/12/06 Pongo account will be ready soon.

8. Introduction to MT

9. A brief history of MT (Based on work by John Hutchins) Before the computer: In the mid 1930s, a French- Armenian Georges Artsrouni and a Russian Petr Troyanskii applied for patents for ‘translating machines’. The pioneers (1947-1954): the first public MT demo was given in 1954 (by IBM and Georgetown University). The decade of optimism (1954-1966): ALPAC (Automatic Language Processing Advisory Committee) report in 1966: "there is no immediate or predictable prospect of useful machine translation."

10. A brief history of MT (cont) The aftermath of the ALPAC report (1966- 1980): a virtual end to MT research The 1980s: Interlingua, example-based MT The 1990s: Statistical MT The 2000s: Hybrid MT

11. Where are we now? Huge potential/need due to the internet, globalization and international politics. Quick development time due to SMT, the availability of parallel data and computers. Translation is reasonable for language pairs with a large amount of resource. Start to include more “minor” languages.

12. What is MT good for? Rough translation: web data Computer-aided human translation Translation for limited domain Cross-lingual IR Machine is better than human in: –Speed: much faster than humans –Memory: can easily memorize millions of word/phrase translations. –Manpower: machines are much cheaper than humans –Fast learner: it takes minutes or hours to build a new system. Erasable memory –Never complain, never get tired, …

13. Major challenges in MT

14. Translation is hard Novels Word play, jokes, puns, hidden messages Concept gaps: go Greek, bei fen Other constraints: lyrics, dubbing, poem, …

15. Major challenges Getting the right words: –Choosing the correct root form –Getting the correct inflected form –Inserting “spontaneous” words Putting the words in the correct order: –Word order: SVO vs. SOV, … –Unique constructions: –Divergence

16. Lexical choice Homonymy/Polysemy: bank, run Concept gap: no corresponding concepts in another language: go Greek, go Dutch, fen sui, lame duck, … Coding (Concept  lexeme mapping) differences: –More distinction in one language: e.g., kinship vocabulary. –Different division of conceptual space:

17. Choosing the appropriate inflection Inflection: gender, number, case, tense, … Ex: –Number: Ch-Eng: all the concrete nouns: ch_book  book, books –Gender: Eng-Fr: all the adjectives –Case: Eng-Korean: all the arguments –Tense: Ch-Eng: all the verbs: ch_buy  buy, bought, will buy

18. Inserting spontaneous words Function words: –Determiners: Ch-Eng: ch_book  a book, the book, the books, books –Prepositions: Ch-Eng: … ch_November  … in November –Relative pronouns: Ch-Eng: … ch_buy ch_book de ch_person  the person who bought /book/ –Possessive pronouns: Ch-Eng: ch_he ch_raise ch_hand  He raised his hand(s) –Conjunction: Eng-Ch: Although S1, S2  ch_although S1, ch_but S2 –…–…

19. Inserting spontaneous words (cont) Content words: –Dropped argument: Ch-Eng: ch_buy le ma  Has Subj bought Obj? –Chinese First name: Eng-Ch: Jiang …  ch_Jiang ch_Zemin … –Abbreviation, Acronyms: Ch-Eng: ch_12 ch_big  the 12 th National Congress of the CPC (Communist Party of China) – …

20. Major challenges Getting the right words: –Choosing the correct root form –Getting the correct inflected form –Inserting “spontaneous” words Putting the words in the correct order: –Word order: SVO vs. SOV, … –Unique construction: –Structural divergence

21. Word order SVO, SOV, VSO, … VP + PP  PP VP VP + AdvP  AdvP + VP Adj + N  N + Adj NP + PP  PP NP NP + S  S NP P + NP  NP + P

22. “Unique” Constructions Overt wh-movement: Eng-Ch: –Eng: Why do you think that he came yesterday? –Ch: you why think he yesterday come ASP? –Ch: you think he yesterday why come? Ba-construction: Ch-Eng –She ba homework finish ASP  She finished her homework. –He ba wall dig ASP CL hole  He digged a hole in the wall. –She ba orange peel ASP skin  She peeled the orange’s skin.

23. Translation divergences Source and target parse trees (dependency trees) are not identical. Example: I like Mary  S: Marta me gusta a mi (‘Mary pleases me’) More discussion next time.

24. Major approaches

25. How humans do translation? Learn a foreign language: –Memorize word translations –Learn some patterns: –Exercise: Passive activity: read, listen Active activity: write, speak Translation: –Understand the sentence –Clarify or ask for help (optional) –Translate the sentence Training stage Decoding stage Translation lexicon Templates, transfer rules Parsing, semantics analysis? Interactive MT? Word-level? Phrase-level? Generate from meaning? Reinforced learning? Reranking?

26. What kinds of resources are available to MT? Translation lexicon: –Bilingual dictionary Templates, transfer rules: –Grammar books Parallel data, comparable data Thesaurus, WordNet, FrameNet, … NLP tools: tokenizer, morph analyzer, parser, …  More resources for major languages, less for “minor” languages.

27. Major approaches Transfer-based Interlingua Example-based (EBMT) Statistical MT (SMT) Hybrid approach

28. The MT triangle word Word Meaning Transfer-based Phrase-based SMT, EBMT Word-based SMT, EBMT (interlingua) Analysis Synthesis

29. Transfer-based MT Analysis, transfer, generation: 1.Parse the source sentence 2.Transform the parse tree with transfer rules 3.Translate source words 4.Get the target sentence from the tree Resources required: –Source parser –A translation lexicon –A set of transfer rules An example: Mary bought a book yesterday.

30. Transfer-based MT (cont) Parsing: linguistically motivated grammar or formal grammar? Transfer: –context-free rules? A path on a dependency tree? –Apply at most one rule at each level? –How are rules created? Translating words: word-to-word translation? Generation: using LM or other additional knowledge? How to create the needed resources automatically?

31. Interlingua For n languages, we need n(n-1) MT systems. Interlingua uses a language-independent representation. Conceptually, Interlingua is elegant: we only need n analyzers, and n generators. Resource needed: –A language-independent representation –Sophisticated analyzers –Sophisticated generators

32. Interlingua (cont) Questions: –Does language-independent meaning representation really exist? If so, what does it look like? –It requires deep analysis: how to get such an analyzer: e.g., semantic analysis –It requires non-trivial generation: How is that done? –It forces disambiguation at various levels: lexical, syntactic, semantic, discourse levels. –It cannot take advantage of similarities between a particular language pair.

33. Example-based MT Basic idea: translate a sentence by using the closest match in parallel data. First proposed by Nagao (1981). Ex: –Training data: w1 w2 w3 w4  w1’ w2’ w3’ w4’ w5 w6 w7  w5’ w6’ w7’ w8 w9  w8’ w9’ –Test sent: w1 w2 w6 w7 w9  w1’ w2’ w6’ w7’ w9’

34. EMBT (cont) Types of EBMT: –Lexical (shallow) –Morphological / POS analysis –Parse-tree based (deep) Types of data required by EBMT systems: –Parallel text –Bilingual dictionary –Thesaurus for computing semantic similarity –Syntactic parser, dependency parser, etc.

35. EBMT (cont) Word alignment: using dictionary and heuristics  exact match Generalization: –Clusters: dates, numbers, colors, shapes, etc. –Clusters can be built by hand or learned automatically. Ex: –Exact match: 12 players met in Paris last Tuesday  12 Spieler trafen sich letzen Dienstag in Paris –Templates: $num players met in $city $time  $num Spieler trafen sich $time in $city

36. Statistical MT Basic idea: learn all the parameters from parallel data. Major types: –Word-based –Phrase-based Strengths: –Easy to build, and it requires no human knowledge –Good performance when a large amount of training data is available. Weaknesses: –How to express linguistic generalization?

37. Comparison of resource requirement Transfer- based InterlinguaEBMTSMT dictionary+++ Transfer rules + parser+++ (?) semantic analyzer + parallel data++ othersUniversal representation thesaurus

38. Hybrid MT Basic idea: combine strengths of different approaches: –Syntax-based: generalization at syntactic level –Interlingua: conceptually elegant –EBMT: memorizing translation of n-grams; generalization at various level. –SMT: fully automatic; using LM; optimizing some objective functions. Types of hybrid HT: –Borrowing concepts/methods: SMT from EBMT: phrase-based SMT; Alignment templates EBMT from SMT: automatically learned translation lexicon Transfer-based from SMT: automatically learned translation lexicon, transfer rules; using LM … –Using two MTs in a pipeline: Using transfer-based MT as a preprocessor of SMT –Using multiple MTs in parallel, then adding a re-ranker.

39. Evaluation of MT

40. Evaluation Unlike many NLP tasks (e.g., tagging, chunking, parsing, IE, pronoun resolution), there is no single gold standard for MT. Human evaluation: accuracy, fluency, … –Problem: expensive, slow, subjective, non-reusable. Automatic measures: –Edit distance –Word error rate (WER), Position-independent WER (PER) –Simple string accuracy (SSA), Generation string accuracy (GSA) –BLEU

41. Edit distance The Edit distance (a.k.a. Levenshtein distance) is defined as the minimal cost of transforming str1 into str2, using three operations (substitution, insertion, deletion). Use DP and the complexity is O(m*n).

42. WER, PER, and SSA WER (word error rate) is edit distance, divided by |Ref|. PER (position-independent WER): same as WER but disregards word ordering SSA (Simple string accuracy) = 1 - WER Previous example: –Sys: w1 w2 w3 w4 –Ref: w1 w3 w2 –Edit distance = 2 –WER=2/3 –PER=1/3 –SSA=1/3

43. Generation string accuracy (GSA) Example: Ref: w1 w2 w3 w4 Sys: w2 w3 w4 w1 Del=1, Ins=1  SSA=1/2 Move=1, Del=0, Ins=0  GSA=3/4

44. BLEU Proposal by Papineni et. al. (2002) Most widely used in MT community. BLEU is a weighted average of n-gram precision (p n ) between system output and all references, multiplied by a brevity penalty (BP).

45. N-gram precision N-gram precision: the percent of n-grams in the system output that are correct. Clipping: –Sys: the the the the the the –Ref: the cat sat on the mat –Unigram precision: –Max_Ref_count: the max number of times a ngram occurs in any single reference translation.

46. N-gram precision i.e. the percent of n-grams in the system output that are correct (after clipping).

47. Brevity Penalty For each sent s i in system output, find closest matching reference r i (in terms of length). Longer system output is already penalized by the n-gram precision measure.

48. An example Sys: The cat was on the mat Ref1: The cat sat on a mat Ref2: There was a cat on the mat Assuming N=3 p 1 =5/6, p 2 =3/5, p 3 =1/4, BP=1  BLEU=0.50 What if N=4?

49. Summary Course overview Major challenges in MT –Choose the right words (root form, inflection, spontaneous words) –Put them in right positions (word order, unique constructions, divergences)

50. Summary (cont) Major approaches –Transfer-based MT –Interlingua –Example-based MT –Statistical MT –Hybrid MT Evaluation of MT systems –Edit distance –WER, PER, SSA, GSA –BLEU

51. Additional slides

52. Translation divergences (based on Bonnie Dorr’s work) Thematic divergence: I like Mary  S: Marta me gusta a mi (‘Mary pleases me’) Promotional divergence: John usually goes home  S: Juan suele ira casa (‘John tends to go home’) Demotional divergence: I like eating  G: Ich esse gern (“I eat likingly) Structural divergence: John entered the house  S: Juan entro en la casa (‘John entered in the house’)

53. Translation divergences (cont) Conflational divergence: I stabbed John  S: Yo le di punaladas a Juan (‘I gave knife- wounds to John’) Categorial divergence: I am hungry  G: Ich habe Hunger (‘I have hunger’) Lexical divergence: John broke into the room  S: Juan forzo la entrada al cuarto (‘John forced the entry to the room’)

54. Calculating edit distance D(0, 0) = 0 D(i, 0) = delCost * i D(0, j) = insCost * j D(i+1, j+1) = min( D(i,j) + sub, D(i+1, j) + insCost, D(i, j+1) + delCost) sub = 0 if str1[i+1]=str2[j+1] = subCost otherwise

55. An example Sys: w1 w2 w3 w4 Ref: w1 w3 w2 All three costs are 1. Edit distance=2 0123 1012 2111 3212 4322 w1 w3 w2 w1 w2 w3 w4