1. www.monash.edu.au Advanced Topics in Data Mining and Research Directions CSE5610 Intelligent Software Systems Semester 1, 2006

2. www.monash.edu.au 2 Outline What is Knowledge Representation? Ideas, Concepts, Theories: –Logics and Mechanized (Automated) Reasoning –Ontologies Systems: –Knowledge Systems –Open Knowledge Systems, and the Web

3. www.monash.edu.au 3 A Knowledge Representation Exercise Using Natural Language Imagine writing down on a piece of paper everything you know about driving, sentence by sentence And now pass it to a friend Your friend will use it to answer queries (about driving) from others Questions: –What language to represent your knowledge? –What language to represent queries? –How will your friend find the answer to queries?

4. www.monash.edu.au 4 Knowledge Representation - I Origins in ‘Artificial Intelligence’ (1955) - AI? –to make systems that do things which normally would require human intelligence, or to build systems that exhibit intelligent behaviour –Physical symbol system hypothesis: "A physical symbol system has the necessary and sufficient means for intelligent action.“ – A. Newell and H. Simon Plato’s student Aristotle (384-322 BC) was probably the first person to work on knowledge representation, compiling and putting down knowledge of his time Represent knowledge in a way that (1) the machine can “understand” (or “interpret and execute”) and (2) humans can understand Generally difficult – we know more than we think!

5. www.monash.edu.au 5 Knowledge Representation - II Applies theories and techniques from –Logic – provides rules of inference & formal language structures in which to represent knowledge –Ontology – what is there (in the application domain)? Provides understanding of what to represent –Computation – executable (knowledge should and can be manipulated by the computer – automate reasoning with the knowledge)

6. www.monash.edu.au 6 Logic Propositional logic First Order Logic –Conceptual Graphs –Common Logic Controlled English Sorted/Typed First Order Logic Higher Order Logics Modal Logic Temporal Logic Interval Logic Causal Logic Fuzzy Logic Linear Logic Non-monotonic Logic Description Logics Etc…

7. www.monash.edu.au 7 Propositional Logic - I A proposition is a statement that can either be true or false. Given statements, operators such as “  ” (and), “  ” (or), “  ”(not) and “  ” (material implication) can be applied to the statements to get new statements. A syntax for propositional logic statements -> operators and symbols denoting propositions (cf. operators and symbols for arithmetic expressions – e.g. 2+3) A semantics for propositional logic statements -> truth value of a statement (cf. value of arithmetic expressions – e.g., 2+3 = 5)

8. www.monash.edu.au 8 Propositional Logic - II E.g.s: Encoding knowledge expressed in English sentences in propositional logic “John is here” ‘John is here’ “Mary is driving and John is here” ‘Mary is driving’  ‘John is here’ “John is not here” or “It is not the case that John is here”  ‘John is here’ “John is here implies there is food is here” ‘John is here’  ‘There is food here’

9. www.monash.edu.au 9 Propositional Logic - III Equivalences: Let p and q be propositions. Then p  q = def  (  p   q) p  q = def  p  q p  q = def (p  q)  (q  p) [equivalence] Really need only  and  !

10. www.monash.edu.au 10 Propositional Logic - IV Reasoning in propositional logic? Truth tables E.g.: Suppose ‘Mary is driving’ is true and ‘John is here’ is false, then is ‘Mary is driving’  ‘John is here’ true or false?

11. www.monash.edu.au 11 Propositional Logic - V pq p  q True False TrueFalse By definition: p  p p TrueFalse True pq p  q True FalseTrue False True pq p  q True False True FalseTrue

12. www.monash.edu.au 12 Propositional Logic - VI ‘Mary is driving’‘John is here’‘Mary is driving’  ‘John is here’ True False TrueFalse

13. www.monash.edu.au 13 Propositional Logic - VIII What about ((  ((‘Mary is driving’  ‘John is here’)  ‘John is here’)  ‘John is here’  (‘John is here’  ((‘Mary is driving’  ‘John is here’)  ‘John is here’))   ‘John is here’ )  ((  ((‘Mary is driving’  ‘John is here’)  ‘John is here’)  ‘John is here’  (  ‘John is here’  ((  ‘Mary is driving’  ‘John is here’)  ‘John is here’))  ‘John is here’) ? Mechanized or automated reasoning! True tables can be manipulated by a computer program.

14. www.monash.edu.au 14 Propositional Logic - IX Arithmetic expressionsPropositional logic Syntax for expressions Operators and numbers e.g., 2+5 Operators and symbols for propositions e.g., p  q Semantics (Formal Meaning) for expressions Numerical value of the expression when evaluated e.g., semantics of “2+5” is 7. Truth value of the proposition when evaluated e.g., semantics of “p  q” is true or false depending on truth values of p and q.

15. www.monash.edu.au 15 Rules of Inference - I E.g.: If John is a living man then John is conscious. John is a living man. --------------------------------------- John is conscious.

16. www.monash.edu.au 16 Rules of Inference - IV Forms of inference (philosopher Pierce): Deduction (analytic reasoning): Given p  q and p, infer q Induction (synthetic reasoning – need justification): Given p and q, infer p  q Abduction (or Inference to Best Explanation (IBE)) (synthetic reasoning – need justification): Given p  q and q, infer p

17. www.monash.edu.au 17 First-Order Logic (FOL) - I There is a man in the room, or There exists x such that x is a man and x is in the room.  x. man(x)  in_room(x) All subjects are easy, or For all x,if x is a subject, then x is easy.  x. subject(x)  easy(x) Additions to propositional logic: - Operators called quantifiers: “  ” (there exists), “  ” (forall) -Predicates with variables - can express relationships e.g., father_of(jack,john) -function symbols – for more descriptive power e.g., father_of(person(jack),person(john)) FOL also calledPredicateCalculus! ‘First Order’ as it does NOT quantify over predicates!

18. www.monash.edu.au 18 First-Order Logic - II One can translate sentences in English to a statement in first-order logic, and automate reasoning over them. All birds fly. Poly is a bird. --------------- Poly flies. By (generalized) modus ponens:  x. bird(x)  flies(x) bird(polly) ------------------------ flies(polly) Given that bird(polly), can polly fly? I.e. flies(polly)? By modus ponens, yes.

19. www.monash.edu.au 19 Imagine writing down on a piece of paper everything you know about driving, sentence by sentence, in first-order logic. Then, asking questions. One can apply rules of inference to answer questions. A Knowledge Representation Exercise Using First Order Logic

20. www.monash.edu.au 20 First-Order Logic - III Prolog: a programming language based on logic where rules (called Horn clauses) are written in a certain way so that they are amenable to computation E.g.: If B1  B2  B3 then A is written as A if B1,B2,B3 (or A :- B1,B2,B3 in Prolog syntax) A if true is often simply written A and is called a fact as it is not dependent on anything.

21. www.monash.edu.au 21 First-Order Logic - IV A if B1,B2,B3 How do we read this rule? Declarative reading: intuitive Procedural reading: to establish A, establish B1, B2 and B3. Given: A if B1,B2,B3 B1 if C1, C2. B2 if C3. B3 if C4. How do you establish A?

22. www.monash.edu.au 22 A Simple Prolog Example Given a database of one rule and one fact: flies(x) :- bird(x). bird(polly). Query: flies(polly)? Prolog returns: Yes. Roughly, Prolog automatically reasons as follows: To establish flies(polly), according to the rule, establish bird(polly) But bird(polly) is already a fact. Done.

23. www.monash.edu.au 23 Imagine writing down on a piece of paper everything you know about driving, sentence by sentence, as Prolog Horn clauses. Then, ask questions by issuing queries to the Prolog system Prolog does the inferencing automatically in order to answer the question. A Knowledge Representation Exercise Using Prolog Free download of a Prolog system: http://www.swi-prolog.org/ & downloadable Mercury system: http://www.cs.mu.oz.au/research/mercury/download/release.html

24. www.monash.edu.au 24 Conceptual Graphs A cat is on a mat. Every cat is on a mat. John is going to Boston by bus. In typed FOL: (  x:Go)(  y:Person)(  z:City)(  w:Bus) (name(y,'John')  name(z,'Boston')  agnt(x,y)  dest(x,z)  inst(x,w)) [CG pictures from http://www.jfsowa.com/cg/cgexampw.htm]

25. www.monash.edu.au 25 Common Logic Controlled English (Sowa, 2004) http://www.jfsowa.com/clce/specs.htm CLCE supports full first-order logic “Anyone who can read ordinary English can read sentences in CLCE with little or no training. Writing CLCE, however, requires practice in learning to stay within its syntactic and semantic limitations. “ E.g.: CLCE: Some person is between a rock and a hard place. PC:  x  y  z. person(x)  rock(y)  place(z)  (between(x,y,z)  hard(z)) Makes knowledge representation and translations into first-order logic easier.

26. www.monash.edu.au 26 Description Logics (DLs) - I Emphasis is providing a logic to represent and reason with concepts in a given domain (also called terminological systems) There are operators to relate concepts. E.g.s: How do we represent the notion of the ‘female person’? Person  Female Concept definition for ‘woman’: Woman  Person  Female What about ‘a mother is a woman’? Mother  Woman

27. www.monash.edu.au 27 Description Logics (DLs) - II E.g.s: ‘Persons that are not female’: Person   Female ‘Female or male’: Female  Male ‘individuals having a female child’:  hasChild.Female ‘individuals all of whose children are female’:  hasChild.Female ‘individuals having at least three children and at most two Female relatives’: (  3 hasChild)  (  2 hasFemaleRelative)

28. www.monash.edu.au 28 Description Logics (DLs) – III KB = TBox + ABox TBox: (the concept definitions) Woman  Person  Female : ABox: (concept and role assertions) Woman(ANNA)  Rich hasChild(ANNA,JACOPO) : From “Basic Description Logics”, F. Baader & W. Nutt:

29. www.monash.edu.au 29 Description Logics (DLs) – IV The meaning of the expressions are sets (basically, a concept denotes a set of entities) e.g., Woman  Person  Female Roughly, “Woman is the set formed by the intersection of the set of all persons and the set of all females”

30. www.monash.edu.au 30 Description Logics (DLs) – V [“Basic Description Logics”, F. Baader & W. Nutt]

31. www.monash.edu.au 31 Description Logics (DLs) – VI Can ask questions like Is JUDY a Mother? Is the concept Man disjoint from Woman? IS concept C subsumed by D? : Does Iokaste have a child that is patricide and that itself has a child that is not a patricide? Example From “Basic Description Logics”, F. Baader & W. Nutt:

32. www.monash.edu.au 32 Description Logics (DLs) – VII Arithmetic expressions Propositional logic Description logic Syntax for expressions Operators and numbers e.g., 2+5 Operators and symbols for propositions e.g., p  q ??? , , , , ,… Semantics (Formal Meaning) for expressions Numerical value of the expression when evaluated e.g., semantics of “2+5” is 7. Truth value of the proposition when evaluated e.g., semantics of “p  q” is true or false depending on truth values of p and q. ??? Concepts as Sets…

33. www.monash.edu.au 33 Ontology Definitions Uses Examples DL and ontologies

34. www.monash.edu.au 34 Ontology - Definitions From www.formalontology.itwww.formalontology.it "1. A science or study of being: specifically, a branch of metaphysics relating to the nature and relations of being; a particular system according to which problems of the nature of being are investigated; first philosophy. 2. a theory concerning the kinds of entities and specifically the kinds of abstract entities that are to be admitted to a language system.“ T. Gruber: “an ontology is an explicit specification of a conceptualization” Philosophy: Ontology is a systematic account of Existence What is there? What are the things that exists? How to categorize them? Formal ontology: domain independent Material ontology: domain specific

35. www.monash.edu.au 35 Ontology – Uses Aid in knowledge representation e.g.s, a. what name for predicates in FOL? handheld_device(X), pda(X), or pocket_computer(X)? b. What to represent about this domain? Remove ambiguity in descriptions via the use of a common vocabulary Facilitate understanding of a shared knowledge repository Intelligent agents can understand each other Knowledge and databases can be queried effectively – mismatch between words used in querying and storage Ecommerce applications – http://www.ontology.org/main/papers/faq.htmlhttp://www.ontology.org/main/papers/faq.html …

36. www.monash.edu.au 36 Ontology – Uses Develop a set of written instructions, without diagrams, that would enable a 3rd party to reproduce a small Lego model, say 10 bricks Taken from H. Smith http://www.ontology.org/ main/presentations/sweden- may2000-ia-for-netmarkets.ppt

37. www.monash.edu.au 37 Ontology - Examples http://ontology.buffalo.edu/

38. www.monash.edu.au 38 Ontology - Examples http://www.jfsowa.com/ontology/index.htm

39. www.monash.edu.au 39 Ontology - Examples http://www.jfsowa.com/ontology/index.htm

40. www.monash.edu.au 40 Ontology - Examples See http://ontology.buffalo.edu/smith/http://ontology.buffalo.edu/smith/ For biomedical, geospatial, social, cognitive ontologies

41. www.monash.edu.au 41 Ontology - Examples Suggested Upper Merged Ontology (SUMO) by IEEE Standard Upper Ontology Working Group (http://ontology.teknowledge.com/) “The goal of this Working Group is to develop a standard upper ontology that will promote data interoperability, information search and retrieval, automated inferencing, and natural language processing. ”

42. www.monash.edu.au 42 Ontology - Examples Wordnet (http://www.cogsci.princeton.edu/~wn/) Yahoo (informal categories – http://www.yahoo.com) Ontolingua (http://www.ksl.stanford.edu/software/ontolingua/) OWL/DAML : ontologies for Weapons of Mass Destruction, Terrorists, and Terrorist Attack Types, Government, Geography, Economy, Military, Communications, Transportation, etc.Weapons of Mass DestructionTerroristsTerrorist Attack TypesGovernmentGeographyEconomyMilitary CommunicationsTransportationetc (http://reliant.teknowledge.com/DAML/)

43. www.monash.edu.au 43 DL and Ontologies Ontologies developed by domain experts via a process typically called ontological engineering using tools such as ontology editors, and ontologies are expressed using some language A logic-based language such as DLs are useful for representing ontologies in an unambiguous way which can be queried and manipulated by the computer See “Description Logics for Ontologies” http://www.cs.man.ac.uk/~horrocks/ESSLLI2003/crc.pdf But DLs founds to be inadequate for some domains. See “Ontology and Medical Terminology: Why Description Logics are not enough?” http://ontology.buffalo.edu/medo/TEPR2003.pdf

44. www.monash.edu.au 44 Dealing with Uncertainty Argumentation-based systems Fuzzy logic Belief revision Label propositions & rules with certainty factors etc

45. www.monash.edu.au 45 Knowledge Systems Knowledge engineering – might involve ontological engineering or more domain-specialized or application- specific Cyc http://www.cyc.com Expert systems –http://www.computer.privateweb.at/judith/http://www.computer.privateweb.at/judith/ –http://www.aiinc.ca/demos/whale.htmlhttp://www.aiinc.ca/demos/whale.html –http://www.claes.sci.eg/Home/wes.htmhttp://www.claes.sci.eg/Home/wes.htm –http://cs.anu.edu.au/software/shyster/http://cs.anu.edu.au/software/shyster/ –http://www.expertise2go.com/ (on the Web expert systems e.g., Wine advisor)http://www.expertise2go.com/Wine advisor –http://www.pcai.com/web/ai_info/expert_systems.htmlhttp://www.pcai.com/web/ai_info/expert_systems.html –http://www.amzi.com/http://www.amzi.com/ –http://sourceforge.net/projects/pess/ (Prolog Expert Systems Shell)http://sourceforge.net/projects/pess/ Knowledge Based Information Agents etc

46. www.monash.edu.au 46

47. www.monash.edu.au 47 Open Knowledge Systems Open systems – peculiar properties: Heterogeneity, no central control, dynamic, Internet-based, trust, multiple perspectives & multi-user, identity & naming, … –The Web –Peer to peer systems Reasoning with the Web: E.g.s, –LogicWeb: Logic Programming for the Web –Prolog and web toolkits –FLORID (http://www.informatik.uni-freiburg.de/~dbis/florid/) –Semantic Web: W3C initiative Reasoning with p2p systems: LogicPeer (e.g. app - dYahoo), Edutella (http://edutella.jxta.org/) Ontology reconciliation: single ontology vs many theories

48. www.monash.edu.au 48 The Semantic Web Key distinguishing feature of the Semantic Web compared to today’s Web is: “Content of the (Semantic) Web will be usable by machines (i.e. software agents)” – Mike Uschold "The Semantic Web is an extension of the current web in which information is given well-defined meaning, better enabling computers and people to work in cooperation." -- Tim Berners-Lee, James Hendler, Ora Lassila, The Semantic Web, Scientific American, May 2001The Semantic Web

49. www.monash.edu.au 49 The Semantic Web How does a software agent know if I am a child of Mary, then Mary is my mother? How does a software agent know that a Web page about cars is a Web page about vehicles? How does a software agent know that if A is the parent of B, and B is the parent of C, then A is the grandparent of C? How does a software agent know that information about pizza restaurants in the area is relevant to someone looking for Italian food in the area? How does a software agent know that if Jane is in the MCG, then she would be in Melbourne? How does a software agent know that site A is more trustworthy than site B? Represent such knowledge in a formal language so that the software agent can process and apply it

50. www.monash.edu.au 50 The Semantic Web From the Semantic Web Roadmap http://www.w3.org/DesignIssues/Semantic.html 1.XML provides a basic format for structured documents, with no particular semantics. 2.The basic assertion model provides the concepts of assertion (property) and quotation. (This is provided by the RDF Model and Syntax Specification). This allows an entity-relationship-like model to be made for the data, giving it the semantics of assertions propositional logic.basic assertion model RDF Model and Syntax Specification 3.The schema language provides data typing and allows document structure to be constrained to allow predictable computable processing. XML schema's datatypes are used.schema language 4.The Ontology layer (WebOnt working group) provides more powerful schema concepts, such as inverse, transitivity, and so on. Uniqueness and/or unambiguousness of properties, when known, allow a system to spot different identifiers which in fact are talking about the same thing.

51. www.monash.edu.au 51 The Semantic Web 5.A conversion language allows the expression of inference rules allowing information in one schema to be inferred from a document in another.conversion language 6.An evolution rules language allows inference rules to be given which allow a machine with a certain algorithm to do convert documents from one RDF schema into another. This is a fundamental key to evolution of the technology. There may be more than one rules standard, as different class of rule-based system have different capabilities. Query languages assume different forms of query engine, but are basically the same problem space as rule systems. One can imagine standardizing both certain query engines and a language for defining query engines.evolution rules languageevolutionQuery languages 7.The logical layer turns a limited declarative language into a Turing-complete logical language, with inference and functions.logical layer 8.A proof language is a form of RDF which allows one agent to send to another an assertion, together with the inference path to that assertion from assumptions acceptable to the receiver.

52. www.monash.edu.au 52 The Semantic Web – “Cake” Agents and the Semantic Web, J. Hendler, IEEE Intelligent Systems, March/April 2001.

53. www.monash.edu.au 53 The Semantic Web Structured data: XML Simple assertions and querying: RDF, TRIPLE (http://www.dfki.uni- kl.de/frodo/triple/) More complex assertions (concepts and ontologies): OWL (DAML) >Nothing can be both a Tree and a Person >Parent and Child are inverses Expressing Rules and Inferencing: ruleML (SWRL), TRIPLE, MetaLog, F- OWL, SHIQ Proofs: InferenceWeb (http://www.ksl.stanford.edu/software/IW/)http://www.ksl.stanford.edu/software/IW/ Applied to Web Services: OWL-S

54. www.monash.edu.au 54 RDF Example Ora Lassila (http://description.org/schema/Creator, http://www.w3.org/Home/Lassila, "Ora Lassila") [http://www.w3.org/Talks/1998/1126-mjd-newmedia/slide11-0.html]

55. www.monash.edu.au 55 OWL Example - Wine http://www.ksl.stanford.edu/projects/wine/explana tion.html

56. www.monash.edu.au 56 Semantic Web Proof Example Example

57. www.monash.edu.au 57 Harnessing the Web and the Semantic Web “Old” information agents – domain- specialized “screen-scrapers”, NL processors, text mining “New” information agents - Semantic Web Agents (Now if we have the machine-readable Web, how do we program agents that can read the Semantic Web?)

58. www.monash.edu.au 58 Loke, S.W. LogicWeb: Using Logic Programming to Extend the World Wide Web, Proceedings of the Second Joint AUUG and Asia Pacific World Wide Web Conference, Melbourne, Australia, pages 220 - 226, September 1996. Han, Y., Loke, S.W, and Sterling, L. Agents for Citation Finding on the World Wide Web, Proceedings of the 2nd Conference on the Practical Application of Intelligent Agents and Multi-Agent Technology, 1997. Loke, S.W. and Davison, A. A Logic Programming Approach to Generating Web- based Guided Tours, Proceedings of the 5th International Conference and Exhibition on The Practical Application of Prolog, London, UK, April 1997. Loke, S.W. and Davison, A. A Two-level World Wide Web Model with Logic Programming Links, Proceedings of the 2nd Workshop on "Logic Programming Tools for INTERNET Applications" held in conjunction with the 14th International Conference on Logic extracted CIFI:Citation Finder (1) search engine result page containing link to home page -> (2) home page -> (3) publication page

59. www.monash.edu.au 59 ARIS: Information Agent Shell Generalize from CIFI: use an expert systems shell approach SubjectFinder, SportsFinder, etc richness of knowledge representation, SportsARISEngine,... Search Engines ARIS Engine KnowledgeBase ARIS WWW

60. www.monash.edu.au 60 From: http://www.cs.umd.edu/users/hendler/AgentWeb.html

61. www.monash.edu.au 61 Additional References Knowledge Representation: Logical, Philosophical and Computational Foundations, J. Sowa, Brookes/Cole The Art of Prolog, L. Sterling and E. Shapiro, MIT Press, 1994. Description logic Handbook >http://www.hpl.hp.com/semweb/download/DescriptionLogicsIntro.pdf >http://www.inf.unibz.it/~franconi/dl/course/dlhb/dlhb-01.pdf >http://www.inf.unibz.it/~franconi/dl/course/dlhb/dlhb-02.pdf Semantic Web books: –The Semantic Web : A Guide to the Future of XML, Web Services, and Knowledge Management, M.C. Daconta, Wiley Computer –Spinning the Semantic Web: Bringing the World Wide Web to Its Full Potential, D. Fensel (Editor) et al., MIT Press Main Web sites on the Semantic Web: –http://www.w3.org/2001/sw/, http://www.semanticweb.orghttp://www.w3.org/2001/sw/http://www.semanticweb.org Introduction to Knowledge Systems, Mark Stefik, Morgan Kaufmann, 1995. Many books on expert systems