Presentation on theme: "TrebleCLEF Evaluation package and language resources CLEF Evaluation Packages Nicolas Moreau / Khalid Choukri - ELDA."— Presentation transcript:
TrebleCLEF Evaluation package and language resources CLEF Evaluation Packages Nicolas Moreau / Khalid Choukri - ELDA
TrebleCLEF and LRs CLEF evaluation campaigns produced large volumes of valuable data A major objective of TrebleCLEF: make CLEF collections publicly available ELDA responsible for: Inventory of CLEF resources IPR issues Production of CLEF Evaluation Packages
Evaluation Packages CLEF test suites consist of: Multilingual document collections Sets of topics Sets of relevance assessments Scoring tools Participant results and related publications Step-by-step documentation on how to perform a system evaluation Reusable test suites for system benchmarking
CLEF Packages Already distributed (ELRA/ELDA catalogue): CLEF (AdHoc on news + domain specific) Produced or being produced: CLEF AdHoc/News => Preliminary version CLEF AdHoc/TEL 2008 CLEF QA CLEF QAST CLEF Domain specific GeoCLEF Still in discussion: ImageCLEF, VideoCLEF WebCLEF, CLEF-IP, and others...
Distribution and Pricing Policy Distribution ELRA/ELDA catalogue: Restricted to evaluation use Pricing policy Low prices for academics Cover the management of data (catalogue management, DVD shipping, etc.)