1. Wrap-up Dr. John D. Prange AQUAINT Program Manager jprange@nsa.gov 301-688-7092 http://www.ic-arda.org

2. 2 Administrative Deadline of Revised Briefings / Papers / Charts –Friday 21 June 2002 –ARDA will cut and distribute final workshop CD to all registered workshop participants –Email files: Paul Matthews: pmmatth@nsa.gov Update information on “List of Attendees” –Registration Desk –Email updates: Paul Matthews: pmmatth@nsa.gov Government Attendees: –Tax Exempt Form available at registration desk when you are checking out –Must pay bill with your government issued credit card

3. 3 Additional TREC Newspaper/Newswire Collection –Newly assembled collection of English newswire text that spans the period from June, 1998 through September 2000, inclusive. –Drawn from available sources: New York Times newswire, Associated Press Wordstream English newswire, and others. –Collection should include at least 3 gigabytes of data, to be published in compressed form on a set of 2 CD-ROMs –Will be available through NIST for AQUAINT Program and TREC Program Evaluation participants and through LDC (Linguistic Data Consortium) for all others AQUAINT: Newly Acquired Data Resources

4. 4 Acquiring AQUAINT TREC Data Project PI should send email to Ellen Voorhees, NIST with cc to John Prange, ARDA requesting this data set. ellen.voorhees@nist.gov jprange@nsa.gov Ellen Voorhees will email you a Data Rights/Use Form that Project PI executes and return completed form to NIST via postal mail or Fax form to number provided by Ellen. Ellen sends AQUAINT TREC Data CD’s to Project PI

5. 5 Acquiring CNS Data Contract with CNS is being process Once contract is in place, set of data will be released by CNS to the AQUAINT Program Some reformatting of the auxiliary resources will be needed CD’s will be cut Release procedures will be published to all AQUAINT Program PI’s – Anticipate that they will be similar to that of the AQUAINT TREC Data Anticipate that contract with CNS will be awarded during next 6-8 weeks.

6. 6 Exemplar Intelligence Reporting Data Requests have come up in several discussions to see/review samples of Intelligence Reporting Data Internet site including information accumulated during the DoD Investigation of “Gulf War Illnesses” www.gulflink.osd.mil Includes Intelligence reports All information declassified and released by Presidential order

7. 7 Schedule of Future AQUAINT Program Phase I Workshops Phase I 12-Month Workshop 3-5 December 2002 (Tues-Thurs) (REVISED DATES) Crystal City Marriott 1999 Jefferson Davis Highway Arlington, VA 22202 Phase I 18-Month Workshop 10-12 June 2003(Tues-Thurs) Hotel TBD San Diego, CA Phase I 24-Month Workshop Week of 1-5 December 2003 NIST Gaithersburg, MD

8. 8 External Question Answering Workshops Mining Answers from Texts and Knowledge Bases Workshop –AAAI Spring Symposium, 25-27 March 2002 –Sanda Harabagiu, SMU & Vinay Chaudhri, SRI; Workshop Co-Chairs –http://www.cs.utdallas.edu/~sanda/stanford.html Workshop on Question Answering: Strategy and Resources –LREC-2002; 28 May 2002 –Mark Maybury, MITRE; Workshop Chair –http://www.lrec-conf.org/lrec-2002 Workshop on New Directions in Question Answering –AAAI Spring Symposium, 24-26 March 2003 –Mark Maybury, MITRE; Workshop Chair –http://www.aaai.org/Symposia/Spring/spring-symposia.html

9. 9 Related External Conferences HLT (Human Language Technology) Conferences –Sponsored by NAACL & Government Funders DARPA, NSF, and ARDA; SIGIR and ISCA represented on HLT Advisory Committee –HLT-2002San Diego, CA 24-27 March 2002 http://www.hlt2002.org –HLT-2003Univ. of Alberta, Edmonton, Alberta, Canada 31 May - 4 June 2003 http://www.hlt03.org Text REtrieval Conferences (TREC) Question Answering (QA) Track –TREC managed by NIST; Co-Sponsored by DARPA and ARDA –TREC-2002NIST, Gaithersburg, MD 19-22 November 2002 QA Track: Main Task; List Task; Context Task http://trec.nist.gov

10. 10 What to look for in next 6 Months Sustaining interaction and involvement of analysts Continuing encouragement for self-organized interactions within AQUAINT community Establishing mechanism for soliciting, receiving and distributing shareable resources Increased emphasis on test bed Establishing and running multiple pilot AQUAINT evaluations + continued planning on others Begin exploration of ways to investigate, study, evaluate more human factors, usability elements associated with AQUAINT systems/components

11. 11 Thank You ARDA Staff... –For all of the behind the scenes work –Especially Paul Matthews and Susan Martin Kelcy Allwein & the Assembled Team of Analysts... –For the AQUAINT Scenario Breakout Sessions Donna Harman, Ellen Voorhees, Jean Scholtz... –For all their efforts on AQUAINT Evaluation AQUAINT Contractors & Other Presenters... –Excellent Briefings –Excellent Summary Write-ups –Great Demos and Posters External Guests... –Your Active Participation has had a positive impact

12. 12 Good-Bye Safe Travels Home