Carnegie Mellon School of Computer Science Copyright © 2001, Carnegie Mellon. All Rights Reserved. JAVELIN Project Briefing 1 AQUAINT Phase I Kickoff December 2001 Language Technologies Institute Carnegie Mellon University Eric Nyberg (PI) Jamie Callan Jaime Carbonell Bob Frederking John Lafferty Alon Lavie Teruko Mitamura

Carnegie Mellon School of Computer Science Copyright © 2001, Carnegie Mellon. All Rights Reserved. JAVELIN Project Briefing 2 AQUAINT Phase I Kickoff December 2001 Research Objectives QA as Planning –Create a general QA planning system –How should a QA system represent its chain of reasoning? QA and Auditability –How can we improve a QA system’s ability to justify its steps? –How can we make QA systems open to machine learning?

Carnegie Mellon School of Computer Science Copyright © 2001, Carnegie Mellon. All Rights Reserved. JAVELIN Project Briefing 3 AQUAINT Phase I Kickoff December 2001 Research Objectives [2] Utility-Based Information Fusion –Perceived utility is a function of many different factors –Create and tune utility metrics, e.g.: U = Argmax k [F (Rel(I,Q,T), Nov(I,T,A), Ver(S,Sup(I,S)), Div(S), Cmp(I,A)), Cst(I,A)] I: Info item, Q: Question, S: Source, T: Task context, A: Analyst - relevance - novelty - veracity, support - diversity - comprehensibility - cost

Carnegie Mellon School of Computer Science Copyright © 2001, Carnegie Mellon. All Rights Reserved. JAVELIN Project Briefing 4 AQUAINT Phase I Kickoff December 2001 Research Plan Develop end-to-end system –Architecture, Planner, Repository –Individual QA Modules Evaluation: –English queries –English, Chinese and Japanese documents

Carnegie Mellon School of Computer Science Copyright © 2001, Carnegie Mellon. All Rights Reserved. JAVELIN Project Briefing 5 AQUAINT Phase I Kickoff December 2001 JAVELIN Basic Architecture All objects created or retrieved are stored centrally for reuse Details of module implementation are hidden Planner is independent from the particular QA modules being used

Carnegie Mellon School of Computer Science Copyright © 2001, Carnegie Mellon. All Rights Reserved. JAVELIN Project Briefing 6 AQUAINT Phase I Kickoff December 2001 JAVELIN Data Flow

Carnegie Mellon School of Computer Science Copyright © 2001, Carnegie Mellon. All Rights Reserved. JAVELIN Project Briefing 7 AQUAINT Phase I Kickoff December 2001 Layers & Control Flow Strategic Decision Points

Carnegie Mellon School of Computer Science Copyright © 2001, Carnegie Mellon. All Rights Reserved. JAVELIN Project Briefing 8 AQUAINT Phase I Kickoff December 2001 Question Analyzer Parse question Assign question & answer types, base query terms As appropriate: –Provide syntactic and semantic analysis; translation –Initiate clarification dialog with the user

Carnegie Mellon School of Computer Science Copyright © 2001, Carnegie Mellon. All Rights Reserved. JAVELIN Project Briefing 9 AQUAINT Phase I Kickoff December 2001 Retrieval Strategist Automatic, content-based resource selection Facet-based retrieval –Example: Clinton as president, Clinton as defendant, … Map query terms into a set of query strategies

Carnegie Mellon School of Computer Science Copyright © 2001, Carnegie Mellon. All Rights Reserved. JAVELIN Project Briefing 10 AQUAINT Phase I Kickoff December 2001 Retrieval Tactician Provide a uniform interface to external data sources (“wrapper” layer) Will support relational queries May support queries to knowledge bases (if KB is provided)

Carnegie Mellon School of Computer Science Copyright © 2001, Carnegie Mellon. All Rights Reserved. JAVELIN Project Briefing 11 AQUAINT Phase I Kickoff December 2001 Request Filler A range of parsing and information extraction techniques to produce possible answer passages Learning optimal combination(s) or approaches (time, quality) Is it possible to automatically learn or improve filler patterns?

Carnegie Mellon School of Computer Science Copyright © 2001, Carnegie Mellon. All Rights Reserved. JAVELIN Project Briefing 12 AQUAINT Phase I Kickoff December 2001 Answer Generator Strategic vs. Tactical Generation Compare request fills to derive answer: –Merge /remove duplicates –Detect ambiguity –Reconcile answer (secondary search) –Identify possible contradictions –Apply utility measures

Carnegie Mellon School of Computer Science Copyright © 2001, Carnegie Mellon. All Rights Reserved. JAVELIN Project Briefing 13 AQUAINT Phase I Kickoff December 2001 Planner Specify planning representation –Identify decision points –Represent & manage uncertainty –Model states and operations –Model justification network Find acceptable trade-offs: –Ratio of planning to execution –Answer utility vs. available resources

Carnegie Mellon School of Computer Science Copyright © 2001, Carnegie Mellon. All Rights Reserved. JAVELIN Project Briefing 14 AQUAINT Phase I Kickoff December 2001 Architecture Abstract module APIs –Several strategies for each task –Easy modification, extension of QA modules –Comparative evaluation, ablation studies, … Common object format & data repository –Adapters for Java, C++, XML

Carnegie Mellon School of Computer Science Copyright © 2001, Carnegie Mellon. All Rights Reserved. JAVELIN Project Briefing 15 AQUAINT Phase I Kickoff December 2001 Questions? Contact: Eric Nyberg