Testbeds Salim Roukos IBM T. J. Watson Research Center 9/11/02

IR Research Testbed Accelerate IR research on massive data 200M pages (2TB), 100ks queries, 10ks users Create an entity that will facilitate data collection of users performing search tasks Real users doing real tasks in context State of the art search capabilities Infrastructure project -- 2 elements: Publicly available data on captured user interactions Software environment for adding functionality

IR Research Testbed Phase I: Create entity to manage/coordinate with a few research groups What to collect, copyright, privacy, subject, … Initial data collection via proxy to Google Develop spidering, common logging facilities Phase II: Build testbed, Open to arbitrary research groups collect data from thousands of users make data available Pilot new data collection experiments on testbed Phase III: Integrate new components for new experiments

It ’ s not about Language Models It ’ s about Modeling … Language