1. AQUAINT R&D Program Phase I Kickoff Workshop WELCOME

2. AQUAINT Kickoff – 3 December 2001 2 AQUAINT: User Testbed / System Integration Pull together best available system components emerging from AQUAINT Program research efforts –Couple AQUAINT components with existing GOTS and COTS software Develop end-to-end AQUAINT prototype(s) aimed at specific Operational QA environments Government-led effort: –Directly Linked into Sponsoring Agency’s Technology Insertion Organizations –Close, working relationship with working Analysts –Provide external system development support –Mitre/Bedford will lead External System Integration / Testbed efforts –Plan to also utilize additional external researchers as Consultants / Advisors

3. AQUAINT R&D Program Phase I Kickoff Workshop Breakout Sessions Dr. John D. Prange AQUAINT Program Director JPrange@nsa.gov 301-688-7092 http://www.ic-arda.org 3 December 2001

4. AQUAINT Kickoff – 3 December 2001 4 Breakout Sessions AQUAINT Evaluation – Both Sessions –Ellen Voorhees & Donna Harman, NIST User Testbed/System Integration Issues – Both Sessions –Marc Light & John Berger, Mitre Data Requirements – 1 st Session –Richard Tong, Tarragon Consulting Sharability of Data, Components, & Resources – 2 nd Session –John Donelan, CIA NRRC Program Committee Meeting – 1 st Session –Penny Lehtola, ARDA

5. AQUAINT Kickoff – 3 December 2001 5 AQUAINT: Evaluation Issues Build upon highly successful TREC Q&A Track Evaluations -- NIST has lead and is currently developing a Phased Evaluation Plan tied to AQUAINT Program Plans Cooperate to maximum extent possible with DARPA’s RKF (Rapid Knowledge Formation) Program Evaluation Efforts First Session: Special Emphasis on Knowledge-Based Systems Second Session: Special Emphasis on Dialogue Systems

6. AQUAINT Kickoff – 3 December 2001 6 AQUAINT: User Testbed / System Integration Pull together best available system components emerging from AQUAINT Program research efforts –Couple AQUAINT components with existing GOTS and COTS software Develop end-to-end AQUAINT prototype(s) aimed at specific Operational QA environments Government-led effort: –Directly Linked into Sponsoring Agency’s Technology Insertion Organizations –Close, working relationship with working Analysts –Provide external system development support –Mitre/Bedford will lead External System Integration / Testbed efforts –Plan to also utilize additional external researchers as Consultants / Advisors

7. AQUAINT Kickoff – 3 December 2001 7 AQUAINT: Data Requirements Issues Start by Using Existing Data Collections –NIST’s TREC Text Corpora –Linguistic Data Consortium (LDC) Human Language Corpora (e.g. TDT, Switchboard, Call Home, Call Friend Corpora) –Existing Knowledge Bases and Other Structured Databases Future Data Collection & Annotation and Question/Answer Key Development will be a major effort –Current Activities to secure major new collections –Ideas for other future collections (e.g. Multiple Genre?) Will likely use combined efforts of NIST and LDC

8. AQUAINT Kickoff – 3 December 2001 8 AQUAINT: Sharability of Data, Components, & Resources Focus is on sharability across the individual projects What Data, Components and other Resources are individual projects willing to share within the AQUAINT community? Similarly what Data, Components and other Resources are individual projects looking for? Data annotations, taxonomy classifications, and/or other locally created/generated data/resources –Do we need to work toward common “structures” or “standards”? –To what extent are these sharable?

9. AQUAINT Kickoff – 3 December 2001 9 Room Assignments Session 1: 9:00-10:15am –Evaluation Main Workshop Room –User Testbed Room N3.365 –Data Requirements Room N3.555 –NRRC Program Comm. Room N3.770 Session 2: 10:45-12 noon –Evaluation Main Workshop Room –User Testbed Room N3.365 –Sharability Room N3.555

10. AQUAINT Kickoff – 3 December 2001 10 Contact Information Dr. John Prange, AQUAINT Program Director ARDA Web Pages:http://www.ic-arda.org Emailarda@nsa.gov JPrange@nsa.gov Phones:301-688-7092 800-276-3747 301-688-7410 (Fax) Mailing:ARDA Room 12A69 NBP#1 STE 6644 9800 Savage Road Fort Meade, MD 20755-6644