1. META-NET is partially supported by the DG INFSO of the European Commission through the NoE project T4ME of the Seventh Framework Programme. Grant Agreement No.: 249119 Dimitris Mavroeidis Institute for Language and Speech Processing Greece Divide and Translate: Using Document Classification to improve Statistical Machine Translation

2. Date Divide and TranslateTitle of Presentation 2 Language Models Counts n-grams Models a coprus Compare with documents outside the corpus Calculate perplexity to assess similarity 2

3. Date Divide and TranslateTitle of Presentation 3 EUROVOC A multi-lingual thesaurus (eurovoc.europa.eu) Created by the European Commission Used primarily for indexing purposes 21 top-level domains: 04 POLITICS 08 INTERNATIONAL RELATIONS … GEOGRAPHY INTERNATIONAL ORGANIZATIONS Date 3

4. Divide and TranslateTitle of Presentation 4 Classification using Language Models Existing corpora with domain labels Train one LM per domain Calculate perplexity of incoming document against all LMs The lowest perplexity is an indication that the text belongs to the corresponding domain Lets see if thats true…

5. Date Divide and TranslateTitle of Presentation 5 Context in Machine Translation: Research Corpus: JRC-Acquis (legal documents of the EU) Labeled with one or more EUROVOC terms Each term belongs to one of the 21 domains Each document can be classified to one of these domains Date 5

6. Divide and TranslateTitle of Presentation 6 Corpus Date 6 Language ISO code No of texts Text body SignaturesAnnexes Total No words (text + signatures + annexes): Total No words Total No characters Average No words Total No words bg11384161408191045226711417.8521700751411461232425506 cs21438228432791489729811065.5572253001676373346832312 da23624314596272134681351331.6826297861685521350944626 de23541320598922327486751361.8725421491632761150929652 el23184364537492395835431572.3729735741645968055887003 en23545345883832106920591469.0331987661775076155537910 es23573389261612380167561651.334902041971624362132608 et23541246216251927007041045.913360511499574840953424 fi23284248830122121789641068.6726777981254717140107981 fr23627391004992347582901654.9130210131997892062100432 hu22801286023802138046141254.4425294881505649646188364 it23472357646702306770131523.7231207971833153557217002 lt23379269377731994382581152.2224365851501848444392842 lv22906275925141964520511204.616731241543796944703607 mt10545209269091289067481984.5313360421562061137883562 nl23564352651612319635391496.5730395801846711556771856 pl23478297130032144640261265.5725131411702739349253537 pt23505372216682274994181583.5630343081935022759606203 ro65739186947605373011397.685142961118584220887085 ro19211308322121826312771604.92--- 30832212 sk21943267926371799204341221.0132278521619054646211035 sl20642277023051786517671342.0431031931683771747643215 sv20243294330371990044011453.9925757711496538446974192 Total46379263621605042889623481387.23603688933589990111055583954

7. Date Divide and TranslateTitle of Presentation 7 Partitioning JRC-Acquis For each document Create 21 counters For each EUROVOC term in the document Find top-level domains Add 1 to the corresponding counter Find the domain with the most counts Classify document Date 7

8. Divide and TranslateTitle of Presentation 8 Partitioning JRC-Acquis Date 8

9. Divide and TranslateTitle of Presentation 9 Classes info Date 9 DomainDescriptionLinesWords 04POLITICS25.197722.635 08INTERNATIONAL RELATIONS69.0571.832.100 10EUROPEAN COMMUNITIES164.2234.310.702 12LAW31.885917.510 16ECONOMICS35.2071.114.052 20TRADE100.3062.485.814 24FINANCE86.4402.015.943 28SOCIAL QUESTIONS47.3431.387.419 32EDUCATION AND COMMUNICATIONS22.313701.761 36SCIENCE23203 40BUSINESS AND COMPETITION51.7011.388.133 44EMPLOYMENT AND WORKING CONDITIONS17.644473.979 48TRANSPORT26.696733.191 52ENVIRONMENT39.4481.141.313 56AGRICULTURE, FORESTRY AND FISHERIES88.1652.235.480 60AGRI-FOODSTUFFS65.3311.449.969 64PRODUCTION, TECHNOLOGY AND RESEARCH50.6411.313.695 66ENERGY7.604226.486 68INDUSTRY32.8741.115.379 72GEOGRAPHY88.1192.348.373 76INTERNATIONAL ORGANISATIONS9.666271.810

10. Date Divide and TranslateTitle of Presentation 10 Experimentation Trained Domain-specific and Generic Language Models SRILM N-gram order: 3 Smoothing: Knesser-Ney Trained Domain and Generic Translation Models MOSES Maximum sentence length: 120 Tuning (MERT) on each domain Evaluate whether the selected domain corresponds to the best BLEU score for the respective translation Date 10

11. Date Divide and TranslateTitle of Presentation 11 Creating individual domains 11

12. Date Divide and TranslateTitle of Presentation 12 Creating generic domain 12

13. Date Divide and TranslateTitle of Presentation 13 Results 13 Domain 04-Politics DOMAINPERPLEXITY BLEU SCORE 04-Politics64.5520.5416 10-European Com.80.9990.3417 08-Int. Relations99.2080.3383 28-Social Q.102.7140.3491 52-Environment107.1590.3502 12-Law107.1750.3580 20-Trade109.0920.3206 32-Education117.6730.3607 64-Pr.Tech.&Res.117.8970.3282 24-Finance120.4970.3233 56-Agriculture122.9300.3120 40-Business123.1950.3213 44-Employment123.8240.3423 16-Economics126.7870.3395 72-Geography127.0580.3186 48-Transport129.1260.3269 76-Intl. Organ.140.1260.3374 60-Agri-Foodstuffs149.1980.2985 68-Industry152.3720.3216 ALL0.3574 Domain 08-International Relations DOMAINPERPLEXITY BLEU SCORE 08-Int. Relations47.8840.5777 10-European Com.77.8430.3683 16-Economics112.0170.3706 28-Social Q.114.5960.3645 24-Finance115.0430.3425 56-Agriculture116.4360.3392 72-Geography116.7370.3435 20-Trade117.7060.3344 04-Politics118.2310.3801 32-Education119.8150.3789 52-Environment123.4250.3687 12-Law124.2390.3661 40-Business127.6540.3411 64-Pr.Tech.&Res.127.7860.3401 60-Agri-Foodstuffs129.1500.3171 44-Employment132.6150.3501 48-Transport135.5040.3349 76-Intl. Organ.139.7570.3578 68-Industry147.7440.3384 ALL0.3876

14. Date Divide and TranslateTitle of Presentation 14 Results (2) 14 Domain 10-European Communities DOMAINPERPLEXITY BLEU SCORE 10-European Com.47.6360.5766 24-Finance104.8410.3493 08-Int. Relations107.7980.3560 28-Social Q.118.8120.3651 04-Politics125.3230.3813 44-Employment126.7370.3628 32-Education126.9940.3868 12-Law127.5380.3716 52-Environment128.7890.3663 40-Business129.2830.3466 16-Economics129.3510.3692 72-Geography132.7490.3433 64-Pr.Tech.&Res.132.9630.3415 20-Trade135.0060.3228 48-Transport140.2000.3416 56-Agriculture141.0780.3283 76-Intl. Organ.147.8710.3629 60-Agri-Foodstuffs162.8440.3074 68-Industry165.7350.3386 ALL0.3906 Domain 12-Law DOMAINPERPLEXITY BLEU SCORE 12-Law48.3860.5654 10-European Com.72.4440.3715 28-Social Q.95.1770.3792 04-Politics104.7350.3879 08-Int. Relations107.9700.3559 44-Employment108.8370.3791 32-Education110.8420.3875 52-Environment110.8460.3686 24-Finance116.5780.3350 20-Trade119.0540.3280 40-Business121.1220.3417 72-Geography121.9040.3394 64-Pr.Tech.&Res.123.3830.3433 48-Transport127.2280.3431 16-Economics127.5810.3602 56-Agriculture131.9670.3298 76-Intl. Organ.140.4080.3616 60-Agri-Foodstuffs153.1120.3077 68-Industry153.1980.3417 ALL0.3772

15. Date Divide and TranslateTitle of Presentation 15 Results (3) 15 Domain 16-Economics DOMAINPERPLEXITY BLEU SCORE 16-Economics75.6950.5961 10-European Com.129.6860.4388 72-Geography135.2650.4461 40-Business144.8450.4309 08-Int. Relations148.0290.4339 24-Finance156.5980.4249 32-Education157.2890.4505 28-Social Q.158.5740.4383 52-Environment162.2460.4333 44-Employment165.8680.4383 12-Law170.4350.4421 04-Politics173.1940.4403 48-Transport173.6470.4084 56-Agriculture183.2420.4078 76-Intl. Organ.187.6930.4303 68-Industry192.0260.4349 64-Pr.Tech.&Res.198.2530.3772 20-Trade214.3340.3978 60-Agri-Foodstuffs222.2290.3898 ALL0.4436 Domain 20-Trade DOMAINPERPLEXITY BLEU SCORE 20-Trade20.3210.6993 56-Agriculture45.3610.4859 60-Agri-Foodstuffs45.9520.4812 72-Geography47.8820.4803 10-European Com.62.0000.4446 08-Int. Relations63.6730.4482 28-Social Q.64.4890.4655 68-Industry65.6170.4709 32-Education69.4790.4915 64-Pr.Tech.&Res.69.5710.4349 52-Environment70.2960.4713 24-Finance72.6450.4377 48-Transport73.5950.4518 40-Business74.4290.4477 12-Law74.6710.4597 04-Politics76.7750.4854 76-Intl. Organ.77.8860.4565 16-Economics80.9770.4516 44-Employment81.8580.4387 ALL0.5039

16. Date Divide and TranslateTitle of Presentation 16 Results (4) 16 Domain 24-Finance DOMAINPERPLEXITY BLEU SCORE 24-Finance36.6680.6569 10-European Com.74.6780.4421 08-Int. Relations118.0260.4142 16-Economics142.6990.4374 40-Business147.0900.4054 72-Geography148.0290.4112 32-Education149.3870.4372 44-Employment163.2000.4142 64-Pr.Tech.&Res.164.1240.4019 20-Trade165.8520.4001 12-Law166.8160.4201 04-Politics168.3820.4366 56-Agriculture170.8290.3989 76-Intl. Organ.172.8220.4213 28-Social Q.172.8430.4112 52-Environment173.0450.4210 48-Transport174.1510.3963 60-Agri-Foodstuffs178.2710.3885 68-Industry192.7230.4046 ALL0.4616 Domain 28-Docial Questions DOMAINPERPLEXITY BLEU SCORE 28-Social Q.45.5060.5800 10-European Com.83.1980.3615 56-Agriculture88.3340.3658 52-Environment94.8530.3802 12-Law101.5340.3766 20-Trade102.3800.3348 64-Pr.Tech.&Res.105.2920.3493 32-Education107.7020.3763 04-Politics108.8770.3731 72-Geography110.1760.3408 08-Int. Relations110.4000.3439 44-Employment110.6730.3570 48-Transport110.8210.3496 16-Economics118.7920.3607 60-Agri-Foodstuffs119.3930.3238 40-Business121.5800.3331 24-Finance123.9920.3329 76-Intl. Organ.126.6790.3578 68-Industry133.2150.3429 ALL0.3829

17. Date Divide and TranslateTitle of Presentation 17 Results (5) 17 Domain 32-Education DOMAINPERPLEXITY BLEU SCORE 32-Education60.4010.5981 10-European Com.90.8410.3913 28-Social Q.110.0990.3982 08-Int. Relations115.8800.3767 12-Law116.4220.4022 40-Business118.7940.3727 20-Trade122.8410.3516 24-Finance123.0940.3688 52-Environment124.1250.3901 16-Economics125.1420.3958 04-Politics126.6350.3944 48-Transport126.9200.3609 44-Employment128.3390.3856 64-Pr.Tech.&Res.128.4410.3560 72-Geography129.5390.3660 56-Agriculture130.6430.3607 76-Intl. Organ.140.7280.3821 68-Industry142.5990.3802 60-Agri-Foodstuffs159.3740.3396 ALL0.4078 Domain 36-Science DOMAINPERPLEXITY BLEU SCORE 52-Environment33.3510.5087 28-Social Q.37.4180.4982 56-Agriculture39.4080.4730 10-European Com.46.2910.4521 20-Trade50.3370.4257 64-Pr.Tech.&Res.53.3610.4612 32-Education56.5370.4883 48-Transport57.8820.4533 44-Employment59.1830.4594 04-Politics63.0280.4702 12-Law63.0380.4625 60-Agri-Foodstuffs65.5440.4162 08-Int. Relations66.6780.4284 68-Industry67.8450.4405 16-Economics68.1140.4487 72-Geography70.1390.4210 24-Finance70.5780.4244 40-Business72.0250.4204 76-Intl. Organ.75.0360.4439 ALL0.4748

18. Date Divide and TranslateTitle of Presentation 18 Results (6) 18 Domain 40-Business DOMAINPERPLEXITY BLEU SCORE 40-Business40.7790.6513 68-Industry85.6710.4733 72-Geography91.3110.4539 32-Education93.3510.4779 10-European Com.94.1100.4389 48-Transport96.4840.4385 24-Finance99.1330.4173 08-Int. Relations100.3930.4303 20-Trade106.6330.4177 56-Agriculture113.2210.4397 28-Social Q.115.2520.4437 16-Economics121.2430.4446 52-Environment127.5200.4433 12-Law130.3590.4406 60-Agri-Foodstuffs132.7880.4056 44-Employment134.3000.4316 04-Politics138.1320.4502 76-Intl. Organ.140.9200.4453 64-Pr.Tech.&Res.144.0330.4191 ALL0.4702 Domain 44-Employment DOMAINPERPLEXITY BLEU SCORE 44-Employment54.7770.5926 10-European Com.76.0710.3809 28-Social Q.105.7450.3703 12-Law106.5530.3962 08-Int. Relations122.3860.3555 16-Economics122.8300.3748 24-Finance123.3790.3435 32-Education125.0960.3842 04-Politics126.0090.3769 52-Environment128.8330.3583 40-Business130.5350.3448 20-Trade132.3610.3251 64-Pr.Tech.&Res.133.6460.3439 48-Transport135.7270.3469 72-Geography137.4610.3371 56-Agriculture149.7230.3235 76-Intl. Organ.154.3310.3605 68-Industry167.2100.3455 60-Agri-Foodstuffs167.9480.3027 ALL0.3828

19. Date Divide and TranslateTitle of Presentation 19 Results (7) 19 Domain 48-Transport DOMAINPERPLEXITY BLEU SCORE 48-Transport40.0960.6298 10-European Com.86.4380.4016 52-Environment98.2840.4192 72-Geography101.5680.3959 64-Pr.Tech.&Res.101.9500.3867 56-Agriculture104.3980.3884 20-Trade105.4520.3877 32-Education106.4830.4273 40-Business106.6390.3932 28-Social Q.108.9480.4082 68-Industry109.6500.4048 08-Int. Relations114.1090.3918 12-Law116.4920.4119 04-Politics119.3690.4129 44-Employment121.4570.3934 24-Finance123.0020.3754 60-Agri-Foodstuffs124.0950.3588 16-Economics126.6910.4028 76-Intl. Organ.134.0550.3971 ALL0.4146 Domain 52-Environment DOMAINPERPLEXITY BLEU SCORE 52-Environment40.5550.5984 56-Agriculture77.6550.3926 28-Social Q.79.2100.4077 10-European Com.80.6050.3817 64-Pr.Tech.&Res.89.5420.3792 20-Trade96.9190.3632 48-Transport98.6080.3801 04-Politics99.1140.4109 12-Law101.4570.3959 32-Education105.3140.4008 08-Int. Relations106.8480.3674 44-Employment110.0230.3775 72-Geography112.3220.3593 60-Agri-Foodstuffs112.4240.3503 24-Finance113.8870.3579 40-Business114.2970.3581 16-Economics116.1470.3849 76-Intl. Organ.119.1610.3795 68-Industry121.0890.3738 ALL0.4042

20. Date Divide and TranslateTitle of Presentation 20 Results (8) 20 Domain 56-Agriculture DOMAINPERPLEXITY BLEU SCORE 56-Agriculture21.3110.6965 20-Trade44.6980.4638 72-Geography48.2440.4590 60-Agri-Foodstuffs49.5600.4625 28-Social Q.50.6230.4868 52-Environment52.3100.4904 10-European Com.59.1050.4466 08-Int. Relations64.7480.4358 32-Education65.4390.4849 64-Pr.Tech.&Res.66.4920.4389 48-Transport66.5130.4456 68-Industry67.5780.4598 40-Business69.3210.4388 16-Economics72.9590.4522 24-Finance73.6140.4380 12-Law73.6410.4598 44-Employment76.6960.4419 04-Politics77.2010.4675 76-Intl. Organ.80.8330.4484 ALL0.5064 Domain 60-Agri-Foodstuffs DOMAINPERPLEXITY BLEU SCORE 60-Agri-Foodstuffs12.4680.7513 20-Trade29.1110.5532 56-Agriculture30.0730.5746 72-Geography45.9960.5234 28-Social Q.51.4050.5243 24-Finance51.5360.5133 10-European Com.52.4930.5113 64-Pr.Tech.&Res.54.6390.5027 52-Environment56.1740.5387 08-Int. Relations56.4080.4983 32-Education58.5350.5440 16-Economics58.8630.5084 68-Industry59.5660.5078 48-Transport61.0180.4891 40-Business62.1230.5010 12-Law63.7080.4981 44-Employment63.9300.4958 76-Intl. Organ.64.3220.5015 04-Politics69.6110.5371 ALL0.5743

21. Date Divide and TranslateTitle of Presentation 21 Results (9) 21 Domain 64-Production Tech. & Research DOMAINPERPLEXITY BLEU SCORE 64-Pr.Tech.&Res.26.9890.6789 76-Intl. Organ.89.6070.4546 20-Trade104.5230.4013 10-European Com.114.2790.4193 28-Social Q.126.3680.4230 52-Environment129.4760.4418 32-Education142.3290.4366 48-Transport148.6740.4065 68-Industry148.9290.4061 56-Agriculture161.7900.3989 12-Law165.3510.4174 44-Employment166.4130.4176 08-Int. Relations168.4700.4022 24-Finance170.6740.3868 04-Politics178.8780.4266 40-Business180.6870.3964 60-Agri-Foodstuffs180.7180.3729 16-Economics196.4370.4090 72-Geography199.3320.3891 ALL0.4533 Domain 68-Energy DOMAINPERPLEXITY BLEU SCORE 10-European Com.102.9090.3678 52-Environment103.2780.3915 28-Social Q.120.3410.3704 08-Int. Relations121.4070.3597 40-Business124.3970.3611 24-Finance128.0220.3494 32-Education128.2020.3891 48-Transport128.2280.3560 64-Pr.Tech.&Res.133.9420.3493 04-Politics135.1440.3786 20-Trade139.1090.3354 16-Economics140.0300.3678 12-Law140.2220.3628 56-Agriculture140.6890.3396 44-Employment143.2050.3585 72-Geography143.5190.3481 76-Intl. Organ.147.3000.3774 68-Industry156.3700.3600 60-Agri-Foodstuffs173.9010.3178 ALL0.3837

22. Date Divide and TranslateTitle of Presentation 22 Results (10) 22 Domain 68-Industry DOMAINPERPLEXITY BLEU SCORE 68-Industry33.5090.6569 72-Geography58.3450.4409 40-Business72.7900.4251 56-Agriculture76.1110.4146 20-Trade82.7860.4018 32-Education101.9200.4335 48-Transport101.9440.3987 60-Agri-Foodstuffs110.2460.3659 28-Social Q.124.2500.3936 10-European Com.124.8790.3714 08-Intl. Relations132.2180.3813 64-Pr.Tech.&Res.150.9970.3471 52-Environment152.6840.3849 76-Intl. Organ.153.5760.3882 24-Finance153.8040.3494 16-Economics156.4070.3899 12-Law157.6060.3774 04-Politics164.1560.3909 44-Employment178.3740.3605 ALL0.4343 Domain 72-Geography DOMAINPERPLEXITY BLEU SCORE 72-Geography34.8780.6754 40-Business84.7970.4423 56-Agriculture87.2460.4358 68-Industry90.4830.4725 10-European Com.93.2570.4326 20-Trade95.6140.4255 08-Intl. Relations109.1820.4285 16-Economics110.3110.4484 32-Education110.5280.4652 28-Social Q.111.0680.4384 48-Transport115.7850.4186 60-Agri-Foodstuffs117.0860.4031 52-Environment121.1410.4360 24-Finance121.1500.4085 12-Law129.0670.4290 76-Intl. Organ.134.8590.4385 44-Employment137.6540.4172 04-Politics142.6440.4390 64-Pr.Tech.&Res.148.2830.3877 ALL0.4793

23. Date Divide and TranslateTitle of Presentation 23 Results (11) 23 Domain 76-International Organisations DOMAINPERPLEXITY BLEU SCORE 76-Intl. Organ.52.1570.6045 64-Pr.Tech.&Res.76.6720.3963 10-European Com.115.1340.3852 20-Trade125.7310.3522 28-Social Q.127.7640.3801 52-Environment140.5420.3858 32-Education141.6390.3904 8-Intl. Relations155.0900.3745 68-Industry162.3520.3628 12-Law163.2300.3757 40-Business163.7920.3585 48-Transport167.6860.3655 4-Politics170.7620.3785 72-Geography171.2540.3632 24-Finance173.3960.3423 16-Economics178.3790.3691 44-Employment178.3880.3656 56-Agriculture180.1530.3460 60-Agri-Foodstuffs203.2560.3229 ALL0.4019

24. Date Divide and TranslateTitle of Presentation 24 Future work Better classification algorithms (for new documents) Application on other corpora (e.g. Europarl, SE-Times) More language pairs Add the domain-specific corpus multiple times Different approaches on partitioning JRC-Acquis Date 24