Explainable Systems: The Inference Web Approach Paulo Pinheiro da Silva Stanford University In collaboration with Deborah L. McGuinness, Richard E. Fikes, Cynthia Chang, Priyendra Deshwal, Dhyanesh Narayanan, Alyssa Glass, Selene Makarios, Jessica Jenkins, Bill Millar, Eric Hsu and many people from IBM, SRI, ISI, IHMC, U. Toronto, U. Trento, U. Fortaleza, U. Texas Austin, Rutgers U., Maryland U., Batelle, SAIC, UCSF, MIT W3C

Paulo Pinheiro da Silva Overview 1. What are explainable systems and why should we care about them? 2. Inference Web: Enabling Explainable Systems 3. Explainable Systems in Action 4. Explainable Systems 10 years from now

Paulo Pinheiro da Silva Explanation Need Google-2.0, where is Paulos office? 1)Stanford, CA, USA 2) Manchester, UK I need to send Paulo a letter but I dont know his address. I believe Paulo lives in the U.S. So, Stanford, CA, USA. appears to be a possible answer. Google-2.0, why is Paulos address Manchester, UK? [Betty]

Paulo Pinheiro da Silva Explanation in Action Paulo At Manchester, UK Paulo At University of Manchester University of Manchester At Manchester, UK transitivity of At Source: Source usage: May/2002 Source: Source usage: May/2002 OK, Manchester, UK was Paulos address in May, 2002 and we are in 2005 !! Why should I believe these? Why should I believe this? Ill send his letter to Stanford. [Betty]

Paulo Pinheiro da Silva What are Explainable Systems? questionanswer expl. 1 explanation request 1 explanation request n expl. n … question answer explanation request 1 answer explanation 1 expl. 1 answer understanding expl. n … question explanation request 1 explanation request 1 explanation request n … explanation n [Bob]

Paulo Pinheiro da Silva Why should we care about explainable systems? As system users, we often need: To understand systems response To trust systems responses Many explanation concerns are the same as in early systems such as Shortliffes MYCIN [1976] Swartouts XPLAIN [1983]

Paulo Pinheiro da Silva Why should we care about explainable systems even more now? Systems are far more complex than 30 years ago Hybrid and distributed processing, e.g., web services, the Grid Large number of heterogeneous, distributed information sources, e.g., the Web More variation in reliability of information sources, e.g., information extraction Sophisticated information integration methods, e.g., SIMS, TSIMMIS Now we have less understanding (and sometimes less trust) of systems answers and behavior Now we have even more reasons for systems to explain their responses

Paulo Pinheiro da Silva How to Enable Explainable Systems? Which information do I have to generate an explanation? 1 -> ((allof (the played-by of (the instances of Project-Leader)) where (It isa Person)) = (:set *Helen *Jody)) 2 -> (allof (the played-by of (the instances of Project-Leader)) where (It isa Person)) 3 -> (forall (the played-by of (the instances of Project-Leader)) where (It isa Person) It) 4 -> (the played-by of (the instances of Project-Leader)) 5 -> (the instances of Project-Leader) 5 (1) Local value(s): (:set *COGS-Proj- Leader-1 *HI-LITE-ProjectLeader-1 *SKIPR- ProjectLeader-1) 6 -> (:set *COGS-Proj-Leader-1 *HI-LITE- ProjectLeader-1 *SKIPR-ProjectLeader-1) [for (the instances of Project-Leader)] 6 <- (*COGS-Proj-Leader-1 *HI-LITE- ProjectLeader-1 *SKIPR-ProjectLeader-1) [(:set... 5 (2) From inheritance: (:set *COGS- Proj-Leader-1 *HI-LITE-ProjectLeader-1 *SKIPR- ProjectLeader-1) I may have (or may be able to record) data describing how I manipulate information to produce answers! questionanswer expl. 1 explanation request 1 explanation request n expl. n …

Paulo Pinheiro da Silva Explainable System Challenge Explanation TrustUnderstanding Information Manipulation Data The GAP

Paulo Pinheiro da Silva Overview 1. What are explainable systems and why should we care about them? 2. Inference Web: Enabling Explainable Systems 3. Explainable Systems in Action 4. Explainable Systems 10 years from now

Paulo Pinheiro da Silva Requirements for Explainable Systems Information Manipulation Traces hybrid, distributed, portable, shareable, combinable encoding of proof fragments supporting multiple justifications Presentation multiple display formats supporting browsing, visualization, etc. Abstraction understandable summaries Interaction multi-modal mixed initiative options including natural-language and GUI dialogues, adaptive, context-sensitive interaction Trust source and reasoning provenance, automated trust inference [McGuinness & Pinheiro da Silva, ISWC 2003, J. Web Semantics 2004]

Paulo Pinheiro da Silva Explainable System Challenge Explanation Proof Markup Language Information Manipulation Data

Paulo Pinheiro da Silva Proof Markup Language: Node Sets and Inference Steps AND Intro (^I) Modus Ponens (MP) A Direct Assertion From Doc1 B Direct Assertion from Doc2 A->(A^B) Direct Assertion From KB1 A->(A^B) A A^B MP A B A^B ^I DA A^B Direct Assertion (DA) from KB1 A DAG of PML Node Sets (a collection of justifications) Extracted Proofs for the conclusion A^B

Paulo Pinheiro da Silva Encoding Hybrid and Distributed Proof Fragments conclusion: A ^ B Proof Markup Language has a web-based solution for distribution Specification written in W3Cs OWL Each node set has one URI Node sets can be used to combine proofs generated by multiple agents OMEGA [Siekmann et al.,CADE2002] has a nice solution for hybrid proofs hasLanguage: KIF (and A B) rule: Modus Ponens (MP) hasEngine: JTP

Paulo Pinheiro da Silva Information Manipulation Traces Proof Markup Language DifferencesFormal Proofs Information manipulation traces Use of rulesMandatory Optional use or use of unregistered rule Sentences Written in some formal language (e.g., KIF, CL, DIMACS, etc.) Written in a formal or informal language including natural language Use of multiple representation languages UncommonCommon Proof Markup Language covers the full spectrum of information manipulation traces! [Pinheiro da Silva, McGuinness & Fikes, IS 2005]

Paulo Pinheiro da Silva Explainable System Challenge Explanation Proof Markup Language Provenance Meta-data Information Manipulation Data

Paulo Pinheiro da Silva Infrastructure: IWBase Meta-data useful for disclosing knowledge provenance and reasoning information such as descriptions of inference engines along with their supported inference rules Information sources such as organizations, publications and ontologies Languages along with their axioms Core IWBase as well as domain IWBases OWL files for interoperability and database for scaling [McGuinness & Pinheiro da Silva, IIWeb 2003]

Paulo Pinheiro da Silva Statistics for relevant domain independent meta-data: 12Languages 6Derived Rules 10Method Rules 38Declarative Rules 56Axioms 29Inference Engines Infrastructure: Core IWBase select

Paulo Pinheiro da Silva Explainable System Challenge Explanation Presentation Provenance Meta-data Information Manipulation Data Proof Markup Language

Paulo Pinheiro da Silva Browsing Proofs (1/2) Enable the visualization of proofs (and abstracted proofs) Proofs can be extracted and browsed from both local and remote PML node sets and can be combined Links provide access to proof-related meta-information select

Paulo Pinheiro da Silva Browsing Proofs (2/2)

Paulo Pinheiro da Silva Explainable System Challenge Explanation Presentation Abstraction Provenance Meta-data Information Manipulation Data Proof Markup Language

Paulo Pinheiro da Silva Knowledge Provenance Elicitation A^B DA^IMP A DA B A->(A^B) DA A->(A^B) A A^B MP A B A^B ^I A^B Dir.Ass. (CNN,BBC)(BBC,NYT) (CNN) CNNBBCNYT Why should I believe this? Google-2.0 says A^B is the answer for my question. has opinion Provenance information may be essential for users to trust answers. Data provenance (aka data lineage) is defined and studied in the database literature. [Buneman et al., ICDT 2001] [Cui and Widom, VLDB 2001] Knowledge provenance extends data provenance by adding data derivation provenance information [Pinheiro da Silva, McGuinness & McCool, Data Eng. Bulletin, 2003]

Paulo Pinheiro da Silva Knowledge Provenance Example Answer Source

Paulo Pinheiro da Silva Abstracting Proofs Explanation tactics (a.k.a. rewriting rules) may be used to abstract proofs into more understandable and manageable explanations Enable the use of axioms as inference rules preventing the presentation of primitive (and potentially less interesting and useful) rules Eliminate intermediate results from proofs

Paulo Pinheiro da Silva Abstracting Proofs: An Example (1/2) Direct assertion (Holds (owner JoesephGradgrind GradgrindFoods) Apr1_03) (Holds* (hasOffice JoesephGradgrind GradgrindFoods) Apr1_03) Generalized Modus Ponens (Holds ((hasOffice JoesephGradgrind GradgrindFoods) Apr1_03) (implies (and (Holds (owner ?person ?object) ?when) (organization ?object)) (Holds* (hasOffice ?person ?object) ?when)) Direct assertion (organization GradgrindFoods) Assumption (not (Ab (hasOffice JosephGradgrind ?where) ?when)) Direct assertion (Holds (owner JoesephGradgrind GradgrindFoods) Apr1_03) Direct assertion (organization GradgrindFoods) (Holds (hasOffice JoesephGradgrind GradgrindFoods) Apr1_03) (implies (and (Holds* ?f ?t)) (not (Ab ?f ?t)) (Holds ?f ?t)) Organization Owner Typically Has Office at Organization Generalized Modus Ponens Direct assertion ABSTRACTED PROOF (Holds ((owner ?person ?object) ?when) (implies (and (Holds (owner ?person ?object) ?when)) (organization ?object)) (Holds* (hasOffice ?person ?object) ?when)) (implies (and (Holds* ?f ?t)) (not (Ab ?f ?t)) (Holds ?f ?t)) (not (Ab (hasOffice ?person ?object) ?when)) Direct assertion (Holds ((hasOffice ?person ?object) ?when) Generalized Modus Ponens (Holds* ((hasOffice ?person ?object) ?when) Generalized Modus Ponens (organization ?object) Direct assertion Abstractor algorithm 1)Match conclusion (key for selecting tactics) 2)Match leaf nodes 3)Unify 5)Apply the assertion-level rule 6)Propagate justified nodes Direct assertion Tactic Library Explanation tactic: Organization Owner Typically Has Office at Organization 4)Propagate conclusion

Paulo Pinheiro da Silva Abstracting Proofs: An Example (2/2) Direct assertion (Holds (owner JoesephGradgrind GradgrindFoods) Apr1_03) Direct assertion (organization GradgrindFoods) (Holds (hasOffice JoesephGradgrind GradgrindFoods) Apr1_03) Organization Owner Typically Has Office at Organization ABSTRACTED PROOF ABSTRACTED PROOF IN DISCURSIVE STYLE A rule says that the owner of an organization typically has an office in an organization Because JosephGrardgrind owned GradgrindFoods on April 1 st 2003 GradgrindFood is an organization therefore JosephGradgrind had an office at GradgrindFoods on April 1 st, Assertion-level rules are introduced in [Huang, PRICAI 1996]. Maybury describes strategies for rewriting abstracted proofs into English [AAAI 1991, AAAI 1993]. Explanation tactics supports multi-level abstraction of proofs

Paulo Pinheiro da Silva Explainable System Challenge Explanation Interaction Presentation Understanding Abstraction Provenance Meta-data Information Manipulation Data Proof Markup Language

Paulo Pinheiro da Silva Explaining Answers: GUI Explainer Users can exit the explainer providing feedback about their satisfiability with explanation(s) Users can ask for alternative explanations Select action

Paulo Pinheiro da Silva Explainable System Challenge Explanation Presentation Abstraction Proof Markup Language Inference Meta-Language Inference Rule Specs Provenance Meta-data Information Manipulation Data Interaction Understanding

Paulo Pinheiro da Silva Inference Meta Language (InferenceML) ndUI: '(forall (' N ')' q ')' |- ' (forall (' N - N.i ')' q[t/N.i] ')';; (Name N) (Sent q) (Term t) Example: An inference rule involves pattern of transformations on expressions to produce a conclusion InferenceML uses schemas to state such transformations InferenceML defines a schema to be a pattern, which is any expression of CL in which: some lexical items have been replaced by a schematic variable (or meta-variable)

Paulo Pinheiro da Silva (and A B) MP (A) DA (implies (A) (and A B)) DA Checking Proofs MP: x; '(implies ' x y ')' |- y ;; (Sent x y) (implies (A) (and A B)) ;|- From IWBase (A)(and A B) binding of expressions to schematic variables: x binds to (A) y binds to (and A B) the rule schema instantiates directly to: (A) ; (implies (A) (and A B)) |- (and A B) =

Paulo Pinheiro da Silva Trust Explainable System Challenge Explanation Presentation Abstraction Inference Meta-Language Inference Rule Specs Provenance Meta-data Information Manipulation Data Interaction Understanding Proof Markup Language

Paulo Pinheiro da Silva IWTrust: Trust in Action (CNN,XYZ)(XYZ,NYT) (CNN) A^B DA^IMP A DA B A->(A^B) DA A->(A^B) A A^B MP A B A^B ^I B DA CNNXYZNYT Why should I trust the answer? Google-2.0 says A^B is the answer for my question. ?? ? Trust can be inferred from a Web of Trust. IWTrust provides infrastructure for building webs of trust. The infrastructure includes a trust component responsible for computing trust values for answers. IWTrust is described in [Zaihrayeu, Pinheiro da Silva & McGuinness, iTrust 2005] A^B

Paulo Pinheiro da Silva Inference Web and Paulo Paulo is a co-technical leader of the Inference Web project Paulo was the main IW developer during 1 ½ years Paulo has been the manager of the IW development team including members with the following profile: 1 research programmer 3 masters students 1 Ph.D. student Paulo has organized the IW weekly meetings Paulo has been responsible for presenting and demonstrating IW solutions at several DARPA and ARDA PI meetings Paulo has participated of the writing of grant proposals

Paulo Pinheiro da Silva Overview 1. What are explainable systems and why should we care about them? 2. Inference Web: Enabling Explainable Systems 3. Explainable Systems in Action 4. Explainable Systems 10 years from now

Paulo Pinheiro da Silva Application Areas Information extraction – IBM (UIMA), Stanford (TAP) Information integration – USC ISI (Prometheus/Mediator); Rutgers University (Prolog/Datalog) Task processing – SRI International (SPARK) Theorem proving First-Order Theorem Provers – SRI International (SNARK); Stanford (JTP); University of Texas, Austin (KM) SATisfiability Solvers – University of Trento (J-SAT) Expert Systems – University of Fortaleza (JEOPS) Service composition – Stanford, University of Toronto, UCSF (SDS) Semantic matching – University of Trento (S-Match) Debugging ontologies – University of Maryland, College Park (SWOOP/Pellet) Problem solving – University of Fortaleza (ExpertCop) Trust Networks – U. of Trento (IWTrust) No single explanation approach has been used in so many diversified areas as Inference Web!

Paulo Pinheiro da Silva Extraction as Inference Goal: To provide browsable justifications of information extraction Strategy: Reuse, adapt, and integrate existing technology: justification technology - Inference Web extraction technology - IBMs UIMA Requires that systems to describe their processing as logical inferences Requires a new perspective: IE as Inference [Murdock, Pinheiro da Silva et al., AAAIs SSS 2005]

Paulo Pinheiro da Silva Extraction As Inference: An Example (1/2) Direct assertion from KB1 (Holds (owner JoesephGradgrind GradgrindFoods) Apr1_03) (Holds* (hasOffice JoesephGradgrind GradgrindFoods) Apr1_03) Generalized Modus Ponens (Holds ((hasOffice JoesephGradgrind GradgrindFoods) Apr1_03) Direct assertion from KB1 (implies (and (Holds (owner ?person ?object) ?when) (organization ?object)) (Holds* (hasOffice ?person ?object) ?when)) Assumption (not (Ab (hasOffice JosephGradgrind ?where) ?when)) (implies (and (Holds* ?f ?t)) (not (Ab ?f ?t)) (Holds ?f ?t)) Generalized Modus Ponens Direct assertion from KB1 Document Coreference (organization GradgrindFoods) Extracted Entity Classification IBM Cross-Annotator Coreference Joseph Gradgrind is the owner of Gradgrind Foods [organization] [refers to GradgrindFoods] Entity Identification IBM EAnnotator Joseph Gradgrind is the owner of Gradgrind Foods [organization] Entity Recognition Direct assertion fromgradgrind.txt Joseph Gradgrind is the owner of Gradgrind Foods (organization GradgrindFoods) Direct assertion from KB1 Solution: A taxonomy of extraction tasks expressed as inference rules Components that record IE justifications using rules in the taxonomy We have identified 9 types of extraction inferences: 6 for analysis, and 3 for integration

Paulo Pinheiro da Silva Extraction As Inference: An Example (2/2) Paulo is a PhD student at University of Manchester. Paulo At Manchester, UK Paulo At University of Manchester University of Manchester At Manchester, UK transitivity of At University of Manchester is located in Manchester, UK. Why should I believe that these documents say that? Why should I believe these? Why should I believe this? Theorem Proving Information Extraction [Betty]

Paulo Pinheiro da Silva Explaining Tool Responses Explain (v. tr.) 1 : To offer reasons for the actions, beliefs, or remarks of (oneself). Questions and Answers Requests and Responses Generalization Inferences for explaining answers (aka beliefs) Inferences for explaining answers (aka beliefs), and tasks (including actions) New perspective: Task processing as inference 1 Dictionary.com

Paulo Pinheiro da Silva NL Explainer: An Example : What are you doing now? : I am trying to get an approval to buy a laptop. : Why? [note: Why? is rephrased to Why are you trying to get an approval to buy a laptop?] : I have completed the previous requirement to get quotes so I am now working on get approval. : OK, I am happy with your explanation. Levering explanation dialogues as in [Fiedler, IJCAI 2001] Using natural language support as in [Allen et al., AAMAS 2002]

Paulo Pinheiro da Silva Overview 1. What are explainable systems and why should we care about them? 2. Inference Web: Enabling Explainable Systems 3. Explainable Systems in Action 4. Explainable Systems 10 years from now

Paulo Pinheiro da Silva Inference Web Contributions Trust Explanation Presentation Abstraction Inference Meta-Language Inference Rule Specs Provenance Meta-data Information Manipulation Data Interaction Understanding Proof Markup Language 1.Language for encoding hybrid, distributed proof fragments based on web technologies. Support for both formal and informal proofs (information manipulation traces). 2. Support (registry, language, services) for knowledge provenance. 4. Multiple strategies for proof abstraction, presentation and interaction. 5. End-to-end trust value computation for answers. 3.Declarative inference rule representation for checking hybrid, distributed proofs. 6. Comprehensive solution for explainable systems

Paulo Pinheiro da Silva Open Issues Automated generation of explanation tactics Performance for abstracting and checking proofs Use of machine learning and user modeling to support interaction Adaptive explanations Explanation contexts Modeling user knowledge Metrics and evaluations for explainable systems

Paulo Pinheiro da Silva Three Years From Now An initial research community working on explainable systems Adaptive explanations based on user modeling IWBase registration of a large set of software systems Registration of a comprehensive set of primitive rules Established library of explanation tactics First generation of metrics and evaluation methods for explainable systems Inference Web is a solution for the Semantic Web proof and trust layers

Paulo Pinheiro da Silva Ten Years From Now An established research community working on explainable systems A theory for explainable systems Established metrics for explainable systems First (or second) generation of industrial explainable systems A standard language for encoding information manipulation traces (probably derived from PML among other proposals). The language will include support for the following: probabilistic reasoning inductive reasoning

Paulo Pinheiro da Silva and Inference Web Immediate connections Explaining Task Processing TaskTracer CALO with Intelligent Information Systems team Explaining Tool Responses Explaining WYSIWYT – with End Users Shaping Effective Software team Potential connections Explanation generation Filtering Learning Explanation-based learning with Learning and Adaptive Systems team Explaining pattern and object recognition from videos and graphs with Computer Graphics and Vision