1. AQUAINT June 2002 Workshop June 2002 Just-in-Time Interactive Question Answering Sanda Harabagiu: PI Language Computer Corporation

2. AQUAINT June 2002 Workshop - JITIQA Just-in-Time Interactive Question Answering  People Sanda Harabagiu, PI Dan Moldovan Larry Daffner John Williams

3. AQUAINT June 2002 Workshop - JITIQA Objective of Research Project  Address the interactive aspect of Q&A systems design and implement a dialog shell that can be used with any Q&A system  Extension of work in factual Q&A with dialog capabilities already tested in call center systems

4. AQUAINT June 2002 Workshop - JITIQA Novelty  Usage of just-in-time citations of other analyst’s similar Q&A sessions: Just-in-time citations are provided by Just-in-Time Information Seeking Agents (JITISA)  JITISA: intelligent agents that proactively search and retrieve information that might be useful without requiring any action from the professional analyst Non-intrusive software agents that monitor the dialog contexts generated when different agents interrogate Q&A systems

5. AQUAINT June 2002 Workshop - JITIQA Tasks Proposed

6. AQUAINT June 2002 Workshop - JITIQA Tasks Proposed

7. AQUAINT June 2002 Workshop - JITIQA Building a Dialog Manager  Wizard-of-Oz Experiments  Extract the features from the Wizard-of-Oz Data  Implement the dialog manager with a strategy that optimizes the features

8. AQUAINT June 2002 Workshop - JITIQA Wizard of Oz Experiments  Year 1: Open-Curtain experiments Main focus: gauge the acceptability of the system-user interaction. General assumption = no actually system exists, develop a detailed conception of the system such that the responses to the user’s questions are known  Year 2: Study annoyance factors

9. AQUAINT June 2002 Workshop - JITIQA Our assumption  Build a “transition network” of questions, answers and follow-up questions Completely unrestricted dialog on the user’s side The user asks any questions Be able to refine the transition network by allowing different turns

10. AQUAINT June 2002 Workshop - JITIQA Open-Curtain Wizard-of-Oz experiment  Start with a complex question (which eventually does not return any acceptable answer) User1 Information Satisfaction User2 Different Perspectives User1 Temporal Perspectives Q1 Q2 Q3 Q2 Q1 Q3 Q1 Q2Q3Qn 10AM11AM2PM

11. AQUAINT June 2002 Workshop - JITIQA

13. Semantic Representations for Interactive Q/A Example: Why is there so much violence on the West Bank now ? Processing: Syntactic Parse - binary dependencies Semantic Representation CAUSE violence West Bank now

14. AQUAINT June 2002 Workshop - JITIQA Novel Representation Frame-like template: Event: template slots: event: violence entity: agent ?? patient ??  location: West Bank Motivation: the answer time: now  May 2, 2002 The same template can be generated by another reformulation of the original question: “What are the causes of the current violence in the West Bank ?” information coming from the answer now

15. AQUAINT June 2002 Workshop - JITIQA

16. Ideal Case  Create a complex “transition network” containing all possible follow-up questions and all possible dialog paths Q2 Q1 Q3

17. AQUAINT June 2002 Workshop - JITIQA

18. Template River Flow Question: What are the causes of the current violence in the West Bank? STATE  EVENT = Template Name Event: violence Entity: Israel Location: West Bank Time: now Answer: Large parts of the West Bank and Gaza remain under Israeli military occupation. New slot: Motivation: Israeli occupation of West Bank Filler is a paraphrase of the answer Template (Question)

19. AQUAINT June 2002 Workshop - JITIQA Follow-up Question Possibilities: 1)-same event -a new slot 2)Relation -slot from Template -other entity/event -… Possible Motivator: Elaboration

20. AQUAINT June 2002 Workshop - JITIQA Color Code 1)Same slot, Same filler yellow blue 2)Same slot, Different filler red River of colors

21. AQUAINT June 2002 Workshop - JITIQA

22. How are templates generated?  2-Phase Process First, templates are proposed by a linguist novice, having a clear goal. (Larry) Second, they are refined by John and Sanda.  Phase I: Larry develops an algorithm of identifying fillers for each template. Sources: Question Parse, Semantic Form Answer Parse, Semantic Form

23. AQUAINT June 2002 Workshop - JITIQA Phase II  Generate template slots automatically (John Williams) Sources: Question Parse, Semantic Form Answer Parse, Semantic Form Dialog annotations, NE Recognizer Dialog motivators and Context modeling

24. AQUAINT June 2002 Workshop - JITIQA Template Generation  Larry has created several types of templates: STATE INVOLVEMENTCONTRAST EVENTRELATIONATTITUDE ATTRIBUTES -definition questions MOTIVATION -causality questions

25. AQUAINT June 2002 Workshop - JITIQA Example  RELATION Templates: Generated by: What is Ariel Sharon’s connection to the Palestinians? How do Israel’s Likud and Labor parties differ on peace with Palestine?  ATTITUDE Templates: What is Ariel Sharon’s position on Palestine? Why is Jerusalem important to Palestinians? What were Ehud Barak’s views on peace with the Palestinians? Why did the Lebanese Phalange hate the Palestinians? How do the Hashemites view the Palestinian Authority?

26. AQUAINT June 2002 Workshop - JITIQA Research Challenges  How many template kinds are sufficient?  How should they be organized?  What is the relationship between template kinds, question classes and dialog context?

27. AQUAINT June 2002 Workshop - JITIQA Novelties 1)Open-Domain Questions 2)No pre-defined task -Task needs to be “guessed” as the dialog evolves 3)A general template algebra needs to be implemented to: a)Deal with templates created in an ad-hoc manner b)Help define the template motivators c)New set of motivators (e.g. elaboration, tangent)

28. AQUAINT June 2002 Workshop - JITIQA Phase III and IV  Phase III: Use two analysts to conduct a dialog starting with the same first question and the same general goal. Study the completeness and efficiency of the dialog motivators as well as the possible overlap.  Phase IV: Allow mixed initiative.

29. AQUAINT June 2002 Workshop - JITIQA Enhance the Q/A Accuracy  Definition questions  Causality questions  Causality-Effect/Involvement questions  Attitude questions

30. AQUAINT June 2002 Workshop - JITIQA Thank You !