NRRC Summer ‘02 Workshop Proposal Re-use November 30, 2001 MITRE Marc Light and Abraham Ittycheriah

Page 2 Copyright © 2001 The MITRE Corporation. All rights reserved. Problem A QA system should accumulate information derived in the process of answering questions. Such information should enable the system to better answer future questions.  What information should be stored?  How should it be stored?  How will it be used?

Page 3 Copyright © 2001 The MITRE Corporation. All rights reserved. Motivating Examples  What are some medicines that treat anthrax? The FDA has approved Cipro (ciprofloxacin), tetracyclines, including doxycycline, and penicillins to treat anthrax.  What are some of the side effects of anthrax medicines? The Physician’s Desk Reference reports that of 2,799 patients who took Cipro during clinical investigations, 16.5 percent had adverse reactions that were possibly or probably related to the drug. The most frequently reported reactions; diarrhea, vomiting, abdominal discomfort, headache, ….  Who manufactures anthrax medicine? -Cipro is produced in the U.S. by the german pharmaceutical company Bayer AG. Text from

Page 4 Copyright © 2001 The MITRE Corporation. All rights reserved. Motivating Examples  What is anthrax? Anthrax is an infectious disease caused by the spore-forming bacteria called Bacillus anthracis. Infection in humans most often involves the skin (cutaneous anthrax)  Where can anthrax be obtained? Los Alamos National labs has stored samples of the B. anthracis spores.  Which countries have anthrax cultures? In this NOVA episode, they mentioned the American Type Culture Collection (ATCC), which had made the embarrassing mistake of selling B. anthracis cultures to Iraq.

Page 5 Copyright © 2001 The MITRE Corporation. All rights reserved. Approach  Collect questions and answers such as the previous examples from a small number of domains (e.g., epidemiology, terrorism, nuclear proliferation, etc.)  Find and Classify different types of possible re-use  Build a system that performs subset of the re-use types in a small number of domains  Evaluate by comparing system performance (e.g., MRR and task completion time) when “reuse” is turned off vs. on

Page 6 Copyright © 2001 The MITRE Corporation. All rights reserved. QA System Information Retrieval Question Analysis Documents Question Answer candidate search Answer candidate ranking Answer candidates Ranked answer candidates Answer construction answer strings Re-Use KB Structured representation for question and documents, i.e., entity typing (coarse- and fine- grained), pred-arg structure, geo-temporal tagging, coreference, predicate normalization, etc. Static KB (WordNet, Cyc, etc.)

Page 7 Copyright © 2001 The MITRE Corporation. All rights reserved. Domain and Data sets 5 problems from one domain each requiring questions Examples: -Characterize the victims of the Anthrax attack.  Time, location, name, age, occupation, type of infection, and eventual-outcome for each reported incident. -Characterize the biological weapon  Location, quality of spores, contents of letter, wording of letter, postmark of letter, etc.  Characterize the possible sources of such a weapon  Location, owner, type of spore produced

Page 8 Copyright © 2001 The MITRE Corporation. All rights reserved. Proposed Team Membership  Team Co-leads: -Marc Light (MITRE) and Abraham Ittycheriah (IBM)  Full time members -Nancy McCrackey (Syracuse Uni) -A member of the MetaCarta R&D team and/or A member of the Cycorp R&D team  Part time members -Benjamin Wellner (MITRE) -William Woods (Sun)  Visiting participants - John Frank (MetaCarta) - Liz Liddy (Syracuse Uni) -Salim Roukus (IBM)  Guest speakers -Eugene Charniak (Brown Uni) -Dragomir Radev (UofMich) -Stephen Kosslyn (Harvard)

Page 9 Copyright © 2001 The MITRE Corporation. All rights reserved. Work Plan  Prior to workshop begin: -Pick domains and collect questions and documents -Develop problems for development and testing -Develop QA system based on IBM, Syracuse, and MITRE systems and components  First 4 weeks (May) -Data analysis wrt re-use -Implement initial re-use modules -Initial evaluation  Interim (June and July) -Refine data analysis and implementation  Second 4 weeks (August) -Experiment and evaluate

Page 10 Copyright © 2001 The MITRE Corporation. All rights reserved. Impact  A better understanding of the types of re-use  A system that exemplifies re-use  A system that grows a knowledge base as it answers questions Total Budget: 411k staff is 341k of total