Presentation is loading. Please wait.

Presentation is loading. Please wait.

TrebleCLEF Evaluation package and language resources CLEF Evaluation Packages Nicolas Moreau / Khalid Choukri - ELDA.

Similar presentations


Presentation on theme: "TrebleCLEF Evaluation package and language resources CLEF Evaluation Packages Nicolas Moreau / Khalid Choukri - ELDA."— Presentation transcript:

1 TrebleCLEF Evaluation package and language resources CLEF Evaluation Packages Nicolas Moreau / Khalid Choukri - ELDA

2 TrebleCLEF and LRs  CLEF evaluation campaigns produced large volumes of valuable data  A major objective of TrebleCLEF: make CLEF collections publicly available  ELDA responsible for:  Inventory of CLEF resources  IPR issues  Production of CLEF Evaluation Packages

3 Evaluation Packages CLEF test suites consist of:  Multilingual document collections  Sets of topics  Sets of relevance assessments  Scoring tools  Participant results and related publications  Step-by-step documentation on how to perform a system evaluation Reusable test suites for system benchmarking

4 CLEF Packages Already distributed (ELRA/ELDA catalogue):  CLEF (AdHoc on news + domain specific)‏ Produced or being produced:  CLEF AdHoc/News => Preliminary version  CLEF AdHoc/TEL 2008  CLEF QA CLEF QAST  CLEF Domain specific  GeoCLEF Still in discussion:  ImageCLEF, VideoCLEF  WebCLEF, CLEF-IP, and others...‏

5 Distribution and Pricing Policy Distribution  ELRA/ELDA catalogue:  Restricted to evaluation use Pricing policy  Low prices for academics  Cover the management of data (catalogue management, DVD shipping, etc.)‏


Download ppt "TrebleCLEF Evaluation package and language resources CLEF Evaluation Packages Nicolas Moreau / Khalid Choukri - ELDA."

Similar presentations


Ads by Google