Download presentation
Presentation is loading. Please wait.
Published byBuck Adams Modified over 8 years ago
1
The TREC-9 Adaptive Filtering track (Coordinators: David Hull and Stephen Robertson) Stephen Robertson Microsoft Research Cambridge ser@microsoft.com
2
Routing and filtering at TREC Routing task (from TREC-1): Text topics Training set (documents with relevance judgements) Test set (new documents without relevance judgements) Time is implicit (training set is history, test set is future)
3
Routing and filtering at TREC Adaptive filtering (from TREC-6): Text topics Time-stamped documents: switch on document stream at time 0 (little or no history) Any incoming document may be sent to user… … and any relevance judgement on that document may be used to modify the query… … but new query only applies to later documents
4
Routing and filtering at TREC TREC-9 track: Approx 350,000 Medline documents, covering approx 4¾ years (~6000 per month) Topic set OHSU: 63 Ohsumed queries Text topics with relevance judgements Topic set MSH: 4903 MeSH headings Scope notes with NLM assignments Topic set MSH-SMP: sample of 500 of these
5
Structure of the TREC-9 tasks Training set: first 9 months’ data (ohsumed.87) Test set: remaining 4 years’ data (ohsumed.88-91) 4 tasks Adaptive filtering Batch-adaptive filtering Batch filtering (no adaptation) Routing
6
Structure of the TREC-9 tasks Adaptive filtering initialization: Text topic + 2 relevant documents from training set (actually 2 for OHSU, 4 for MSH) idfs from training set can be used Batch filtering/routing initialization: Text topic + any use of training set (including relevance judgements) Adaptation (adaptive and batch-adaptive) : Use judgements on documents previously returned
7
Measures Utility: Score (credit) x for a reldoc, debit y for a non-rel doc retrieved TREC-9 utility: credit 2, debit 1 T9U: Utility but with minimum (-100 or -400) Precision-oriented measure: Target number of docs retrieved over whole period (=50 for TREC-9)
Similar presentations
© 2024 SlidePlayer.com Inc.
All rights reserved.