1. Versatile Information Systems, Inc. 2005 International Semantic Web Conference www.vistology.com An Application of Semantic Web Technologies to Situation Awareness Christopher J. Matheus Versatile Information Systems, Inc. Mieczyslaw M. Kokar, Kenneth Baclawski Northeastern University Jerzy A. Letkowski Middle New England College www.vistology.com

2. Versatile Information Systems, Inc. 2005 International Semantic Web Conference www.vistology.com Overview ● Situation Awareness ● Ontologies and Rules ● Situation Awareness Assistant: SAWA ● A Simple Scenario ● Lessons Learned

3. Versatile Information Systems, Inc. 2005 International Semantic Web Conference www.vistology.com Our Problem Domain ● Formal yet Practical Applications of Situation Awareness ● Situation Awareness (SAW): – an understanding of what’s going on in an evolving situation e.g. supply logistics, financial markets, battlefields – involves fusion of object-level data from multiple sources into meaningful higher-order relations – highly context dependent and goal directed (i.e., requires domain knowledge) ● Requirements for effective SAW apps: – domain knowledge about relevant objects and their properties – specification of conditions that define higher-order relations – a means for reasoning about time-dependent sensor information in the context of the given domain knowledge – much in common with SW goals of knowledge representation and processing but with real-time and uncertainty concerns

4. Versatile Information Systems, Inc. 2005 International Semantic Web Conference www.vistology.com AFRL Research Focus ● US AFRL supported effort to formalize and automate the identification and monitoring of relevant relations in evolving situations ● Phase I: Formalization of SAW – Formal definition of situation awareness using Speckware and DAML/OWL ● Phase II: SAW Assistant (SAWA) – Prototype system to support the detection and monitoring of relevant relations

5. Versatile Information Systems, Inc. 2005 International Semantic Web Conference www.vistology.com General Methodology Working with Subject Matter Experts we first: ● develop ontologies for describing domain-specific object classes and properties ● develop rules to define complex relations that are grounded in observable data annotated by the ontologies We then: ● populate an inference engine with ontologies and domain rules ● establish an input stream of events describing object observations annotated using the domain ontologies ● use the inference engine to process the event stream and detect evolution of higher-order relations

6. Versatile Information Systems, Inc. 2005 International Semantic Web Conference www.vistology.com Ontologies and Rules ● Need ways to represent domain knowledge concerning: – Situation Objects, their Attributes and their inter-Relations ● OWL provides a solid basis for these needs – Formal semantics facilitates reasoning with generic reasoners (e.g. Jess with OWL axioms) – Reuse of existing of tools (parsers, consistency checkers, etc) – Main drawback: limited representational power ● SWRL used to represent more complex relations – Permits representation of more complex relationships ● e.g. there are two Units U1 and U2 in-region R AND U1 is a member-of a Force F1 AND Unit U2 is a member-of Force F2 AND F1 is not equal to F2 – Has advantage of formal semantics defined as an extension to the semantics of OWL DL – Easily converted into Jess rules using XSLT

7. Versatile Information Systems, Inc. 2005 International Semantic Web Conference www.vistology.com SAW Core Ontology

8. Versatile Information Systems, Inc. 2005 International Semantic Web Conference www.vistology.com domain knowledge SAWA Architecture Knowledge ManagementRuntime System SMCRMA TDBEMC ConsVISor GUI RuleVISor Protégé Ontology Editor SWRL Editor Consistency Checker event annotations SAWA goals and queries OWL SWRL

9. Versatile Information Systems, Inc. 2005 International Semantic Web Conference www.vistology.com SAWA Knowledge Management Protégé with ezOWL ConsVISor RuleVISor

10. Versatile Information Systems, Inc. 2005 International Semantic Web Conference www.vistology.com SAWA Runtime System TDB SMC RMA EMC GUI events ontologies and rules SMC: Situation Management Comp. EMC: Event Management Comp. TDB: Triples Database RMA: Relation Monitoring Agent GUI: Graphical User Interface ● Java Components (RMI) ● RMA and TDB – Based on Jess inference engine – Store RDF triples in working memory – Includes OWL axioms for inferring implicit triples – Plus procedural attachments for SWRL built-ins ● TDB supports OQL and What-if reasoning

11. Versatile Information Systems, Inc. 2005 International Semantic Web Conference www.vistology.com SAWA Runtime GUI

12. Versatile Information Systems, Inc. 2005 International Semantic Web Conference www.vistology.com Supply Line Scenario Simple Scenario: hasSupplyLine – defines a unit to be in supply if a series of roads can be traced from the unit to a supply station through regions under friendly control

13. Versatile Information Systems, Inc. 2005 International Semantic Web Conference www.vistology.com Simple Supply Logistics Ontology

14. Versatile Information Systems, Inc. 2005 International Semantic Web Conference www.vistology.com hasSupplyLine Rule Set <swrlb:notEqual arg1="?region1" arg2="?region2"/> <swrlb:notEqual arg1="?regionA" arg2="?regionB"/>

15. Versatile Information Systems, Inc. 2005 International Semantic Web Conference www.vistology.com SAWA Runtime GUI

16. Versatile Information Systems, Inc. 2005 International Semantic Web Conference www.vistology.com Lessons Learned ● OWL is very useful for base ontology representation – Pros: tools, formal semantics, extensible, triples-based – Cons: triples-based (binary predicates), lack of complex implication ● Rules on top of OWL is an effective way to utilize the benefits of OWL while overcoming some of its limitations ● Limitations of SWRL: – restricted to binary predicates - not always natural and work around is cumbersome – built-ins defined as axioms but are needed as functions – need gensym() and assert() to generate and assert new entities – need for negation as failure – syntax not intended for human processing ● SWRL is too low level of a language for knowledge engineering – even with a graphical editor – need higher-level language(s) that can be automatically translated to low-level triples representation

17. Versatile Information Systems, Inc. 2005 International Semantic Web Conference www.vistology.com