1. A Core Ontology for Situation Awareness Christopher J. Matheus Versatile Information Systems, Inc. Mieczyslaw M. Kokar Kenneth Baclawski Northeastern University/VIS

2. July 9, 2003 Fusion’2003, Cairns Matheus, Kokar, Baclawski2 Acknowledgments Contract: F30602-02-C-0039 Michael L. Hinman, AFRL/IFEA John Salerno, AFRL/IFEA Contractor: Versatile Information Systems, Inc.

3. July 9, 2003 Fusion’2003, Cairns Matheus, Kokar, Baclawski3 Objectives Show a core ontology for situation awareness (SAW Ontology) Show alternative designs of SAW Ontology Show how SAW Ontology can be extended to satisfy specific requirements

4. July 9, 2003 Fusion’2003, Cairns Matheus, Kokar, Baclawski4 The Situation Awareness Problem Knowing states of objects doesn’t necessarily mean understanding what’s going on (football) Examples: close_to, under_attack, retreating, operational_readiness, …. Need information about multiple objects,history, background knowledge, context, evolution over time…. Need to derive relationships (no direct measurements) Which ones? –100 objects  2 10000 possible relations! Need a theory of how the world “works” in a given context (ontology, situation)

5. July 9, 2003 Fusion’2003, Cairns Matheus, Kokar, Baclawski5 SAW Definition Situation Awareness (SAW) is the perception of the elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future. - Endsley & Garland, 2000

6. July 9, 2003 Fusion’2003, Cairns Matheus, Kokar, Baclawski6 Situations – First Class Citizens “One of the starting points for situation semantics was the promotion of real situations from second class citizens to first class citizens.” “By a situation, then, we mean a part of reality that can be comprehended as a whole in its own right – one that interacts with other things. By interacting with other things we mean that they have properties or relate to other things.” - John Barwise, The Situation in Logic, 1989

7. July 9, 2003 Fusion’2003, Cairns Matheus, Kokar, Baclawski7 Informal SAW Definition Situation Awareness is knowledge of the following elements: Goal (for the level of decision making); Theories of the world (ontology); Knowledge of which theories in the ontology are relevant to the Goal at time t (current time) and at t+1 (in the near future); Objects that are relevant to the Goal at time t and t+1; and Relations among the objects that are relevant to the Goal at time t and t+1.

8. July 9, 2003 Fusion’2003, Cairns Matheus, Kokar, Baclawski8 Core SAW Ontology

9. July 9, 2003 Fusion’2003, Cairns Matheus, Kokar, Baclawski9 Attribute Values and Events

10. July 9, 2003 Fusion’2003, Cairns Matheus, Kokar, Baclawski10 “Snapshot” Design Everything is captured for each time instant Advantage: easy to retrieve (by time index) Disadvantages: Keep records even if nothing changes Information must arrive in lock-step fashion (fixed delta-t)

11. July 9, 2003 Fusion’2003, Cairns Matheus, Kokar, Baclawski11 Time-Interval Design Attributes and relations for arbitrary time intervals Problem: where to keep uncertainty info for relations? Relation – would be constant over time Time Interval – would be the same for all relations

12. July 9, 2003 Fusion’2003, Cairns Matheus, Kokar, Baclawski12 Property Value with Certainty Uncertainty part of Relations and Attributes through PropertyValue Problem: PropertyValues are associated with time events and not arrival of new information

13. July 9, 2003 Fusion’2003, Cairns Matheus, Kokar, Baclawski13 Event Notices PropertyValues are associated with EventNotices, i.e., with arrival of new information (e.g., Level 1 events)

14. July 9, 2003 Fusion’2003, Cairns Matheus, Kokar, Baclawski14 Battlefield Scenario

15. July 9, 2003 Fusion’2003, Cairns Matheus, Kokar, Baclawski15 SAW Ontology Extensions Q: Is Core SAW Ontology sufficient to represent this scenario? A: No. Q: Can it be extended, or would it need to be changed? A: It needs to be extended

16. July 9, 2003 Fusion’2003, Cairns Matheus, Kokar, Baclawski16 Battlefield Ontology

17. July 9, 2003 Fusion’2003, Cairns Matheus, Kokar, Baclawski17 Battlefield Obstacle Ontology

18. July 9, 2003 Fusion’2003, Cairns Matheus, Kokar, Baclawski18 Battlefield Relation Ontology

19. July 9, 2003 Fusion’2003, Cairns Matheus, Kokar, Baclawski19 SAW Process Flow

20. July 9, 2003 Fusion’2003, Cairns Matheus, Kokar, Baclawski20 Summary of the Process 1.Possess a Theory of the World, T O, consisting of a number of interrelated theories T 1, T 2, T 3 … and specify all of them in a formal language. 2.Post a Goal T g in terms of the formal language. 3.Demonstrate the process of selecting relevant theories, T 1, T 2, T 3 …from among the theories of the world. 4.Collect events W 1, W 2, W 3 … and specify them in a formal language. 5.Specify (in the formal language) and then select relevant models, M 1, M 2, M 3 … of the relevant theories. 6.Combine the relevant theories (theory fusion) within the formal methods tool (Specware) using the category theory operator of colimit. 7.Similarly, combine the relevant models (model fusion) so that the combined model satisfies the combined theory from step 6 above. 8.Prove/disprove the Goal theorem using the combined theory. This proof includes the fusion of: a. theories b. models c. uncertainty.

21. July 9, 2003 Fusion’2003, Cairns Matheus, Kokar, Baclawski21 Conclusions Showed a core SAW Ontology Analyzed alternative approaches Showed extensibility of the ontology to more complex scenarios Discussed the SAW process More research needed on: SAW case studies, ontology extensions (agree to use the same ontologies), ontology tools – especially efficient reasoners