1. TextMap: An Intelligent Question- Answering Assistant Project Members:Abdessamad Echihabi Ulf Hermjakob Eduard Hovy Soo-Min Kim Kevin Knight Daniel Marcu Deepak Ravichandran

2. Research Foci Increase performance on factoid-based questions –Question/Answer Reformulations –Exploiting the Web –Statistical-based techniques for QA –Learning paraphrases from multilingual corpora Develop algorithms for answering non-factoid question types –Causal questions –Opinion questions –Learning answer templates for questions about events

3. Natural Language Reformulations: A Means for Increasing the Performance of QA Systems

4. Reformulations Question answering task: –Given a question in normal English or other languages, find highly focused answer from text collections and/or the web. Problem: –Sentences containing an answer often do not match the direct wording of a question. Question: Who is the leader of France? –“Henri Hadjenberg, who is the leader of France's Jewish community, endorsed confronting the specter of the Vichy past.” * 100% word overlap, but sentence does not contain answer. –“Bush later met with French President Jacques Chirac.” * 0% word overlap, but sentence contains the correct answer.

5. Question: Who killed John F. Kennedy? –Lee Harvey Oswald killed John F. Kennedy. (easy) –Lee Harvey Oswald assassinated John Fitzgerald Kenedy. (word synonyms, spelling) –Minutes after President Kennedy had been killed by Lee Harvey Oswald... (tense, syntax) –Kennedy assassin Lee Harvey Oswald (cross-part-of- speech synonyms, syntax) –Lee Harvey Oswald took the life of John F. Kennedy (phrasal synonyms) –John F. Kennedy's death was caused by two gun shots fired by Lee Harvey Oswald. Degrees of formulation mismatch

6. Semantic reformulations Questions: How much does an AK-47 cost? Sentence 1: An AK-47 mag costs $5. Sentence 2: Then he went and bought a Romanian AK-47 for about $250. :anchor-pattern "SOMETHING_1 costs MONETARY_QUANTITY_2." :is-equivalent-to "the price of SOMETHING_1 is MONETARY_QUANTITY_2." :is-equivalent-to "SOMETHING_1 is on sale for MONETARY_QUANTITY_2." :can-be-inferred-from "to buy SOMETHING_1 for MONETARY_QUANTITY_2." Semantic reformulation knowledge in form of extended natural language Matching at parse tree level allows for greater level of flexibility –The Magna Carta was signed in 1215. –King John signed the Magna Carta at Runnymeade in 1215. Easy to write and filter Can be used for other natural language applications

7. Syntactic reformulations Identify and normalize interrogative terms Transform question to declarative format(s) Allow more focused search with search engines What year did the Titanic sink? => the Titanic sank When was the Shah of Iran overthrown? => the Shah of Iran was overthrown => overthrew the Shah of Iran How tall is the Eiffel Tower? => the Eiffel Tower is tall => the tall Eiffel Tower

8. Advanced forms of reformulation Inferencing :anchor-pattern "SOMEBODY_1 invented SOMETHING_2." :can-be-inferred-from "SOMEBODY_1 received a patent for SOMETHING_2." :weight 0.8 Q: Who invented the telephone? Alexander Graham Bell received a patent for the telephone. A: Alexander Graham Bell

9. Reformulation as qualitative extension of WordNet Word synonym Phrasal synonym astronaut SOMETHING_1 is DISTANCE_QUANTITY_2 long. cosmonaut length of SOMETHING_1 is DISTANCE_QUANTITY_2. Generalization Phrasal inference lawyer SOMETHING_1 is the capital of LOCATION_2. professional<adult SOMETHING_1 is in LOCATION_2.

10. Reformulations for Question Answering: (NL-based query expansion method) NL questions are automatically mapped into web queries using a range of increasingly sophisticated techniques - Simple - Units - Morphological - Synonym - Spelling - Reformulations/paraphrasing - Cuephrase

11. Mapping NL questions into Web Queries -Simple Preserve quoted terms and quote the smallest NPs Example: What is the longest river in the United States? "longest river" AND "United States" -Units Expand with potential units in answer Example: How tall is Mt. Everest? "Mt. Everest" AND "tall" AND ("foot" OR "feet" OR "miles")

12. Mapping NL questions into Web Queries - Morphological Expand query using morphological variants: When did the Titanic sink? "Titanic" AND ("sink" OR "sank" OR "did sink") - Synonym Expand query using WordNet synonyms: What is the length of border between Ukraine and Russia? ("length" OR "duration" OR "distance") AND ("border" OR "surround" OR "skirt") AND ("Ukraine" OR "Ukrayina") AND ("Russia" OR "Soviet Union" OR "Union of Soviet Socialist Republics") AND ("between" OR "betwixt" OR "'tween") - Spelling Expand query using different spellings of question terms: Where is Al-Qaeda located? ("Al-Qaeda" OR "Al Qaida") AND "located"

13. Mapping NL questions into Web Queries -Reformulations/ paraphrasing Add rephrases of the question's declarative form: What is an atom? "is an atom" "an atom is“ "an atom is one of“ "an atom is defined to be“ "an atom is defined as“ "such as an atom" "called an atom" -Cuephrase Expand the rephrases with cues When did Abraham Lincoln die? "Abraham Lincoln died" "Abraham Lincoln died on" "Abraham Lincoln died in" "Abraham Lincoln died about"

14. Utility of our query expansion/reformulation techniques

15. Answer redundancy specific to our query (re)formulation techniques

16. Search Engine Impact (Download Full Pages)

17. Search Engine Impact (Download only Snippets)

18. Accuracy-Speed Tradeoff

19. Q&A Improvements from Reformulations Date# reforms/questionPerformance (first 100Q in TREC-2002) July 2002No reformulations 28% July 20021.234% (35%) Oct 20023.139%