1. The Sensor Fusion Lab & Center of Excellence in C4I
Predictive Situation Awareness Model for a Critical Infrastructure Protection System
The Sensor Fusion Lab & Center of Excellence in C4I
Cheol Young Park & Kathryn B. Laskey

2. We use a PSAW software system for efficient PSAW
1. PSAW Situation awareness to Predictive Situation Awareness (PSAW)
Situation awareness is the ability to perceive elements in the environment, comprehend the current situation, and project the future status [Endsley, 1988]
Predictive Situation Awareness (PSAW) places particular emphasis on the ability to make predictions about a temporally evolving situation
A strong PSAW capability supports more efficient and effective decision making
We use a PSAW software system for efficient PSAW
What do we want to get from PSAW?
Endsley, M. R. (1988). Design and evaluation for situation awareness enhancement. Paper presented at the Human Factors Society 32nd Annual Meeting, Santa Monica, CA.

3. How to get answers for these PSAW questions?
1. PSAW PSAW Questions
What is the type of the target?
What is the origin of contact?
What is the missile targeting?
What is the enemy doing and why?
What are the types of the situations?
How many military vehicles are going to encounter?
How high will be the level of danger to a region?
Where will be the target located?
Where will be the event occur?
What type activity will be the target performing?
What situation will be the targets involved in?
How to get answers for these PSAW questions?

4. What is Bayesian reasoning?
1. PSAW Bayesian Network (BN): What is the type of the target?
What is Bayesian reasoning?

5. How do we reason about more complex situations?
1. PSAW Inference of Bayesian Network (BN): Answer for the PSAW question
How do we reason about more complex situations?

6. BN is not expressive enough to handle:
1. PSAW Insufficient Expressive Power of Bayesian Network
BN is not expressive enough to handle:
Different number of entities for different situations
Uncertainty about relationship between entities
Situation evolving in time
Therefore, we need a more expressive representation
Case 1: 2 Sensors
Case 2: 4 Sensors & 2 Times

7. 2. MEBN in PSAW
Multi-Entity Bayesian Networks (MEBN) Model (MTheory) for an illustrative example
MFrag Context Input Resident Node Node Node

8. Power of MEBN What is MEBN? 2. MEBN in PSAW Case 1: 2 Sensors
& 5 Times
Case 2: 4 Sensors
& 2 Times
[Laskey et al, 2000], [Wright et al, 2002], [Costa et al, 2005], [Suzic, 2005], [Costa et al, 2009], [Carvalho et al, 2010] ... used MEBN for PSAW
What is MEBN?

9. Is it easy to develop an MEBN model?
2. MEBN in PSAW
Advantages of MEBN
Complex and Uncertain Situation
Flexible enough to represent a variety of complex situations
Human Understandable Model
Is it easy to develop an MEBN model?

10. 2. Difficulty of developing a MEBN model
Manual MEBN modeling by a domain expert is a labor-intensive and insufficiently agile process
Therefore, it is necessary to implement MEBN modeling automation using a machine learning method
What is MEBN learning?

11. 3. Process for Human-aided MEBN Learning in PSAW
Requirements
World Model/Rules for Initial Reasoning Model
Learned
Reasoning Model
Stakeholder Needs/Mission
Analyze
Requirements
Design
World Model and Rules
Construct Reasoning Model
Test
Reasoning Model
Reasoning Model
Reference MEBN model for PSAW
Uncertainty Modeling Process for Semantic Technology (UMP-ST) [Carvalho, 2011]
For, in the real world, search space to learn a MEBN model is too large and complex to investigate all possible structures, variables, and parameters
For this reason, the proposed method relies partially on expert's knowledge and insight to reduce search spaces
A process for Human-aided MEBN learning in PSAW is a framework to develop a MEBN model from the domain expert's knowledge as well as data
What is the Reference MEBN model for PSAW?

12. What is a concept of the reference model?
3. Process for Human-aided MEBN Learning in PSAW
Necessity of Reference MEBN model for PSAW
It is inefficient to design a new MEBN model from the ground up
To address this issue, it is necessary to know what kinds of random variables (RV), relationships between RVs, local probability distributions for RVs, and entities associated with RVs are needed to represent a situation in PSAW, as well as how to match these elements to elements in PSAW to determine a mapping rule between MEBN and PSAW
A reference MEBN model for PSAW specifies reference MFrags, RVs, and entities which support the design of a MEBN model for PSAW
What is a concept of the reference model?

13. Core Elements of PSAW So What? Sensor, Target, Reported Target, Time
4. A reference PSAW-MEBM model
Core Elements of PSAW
Sensor, Target, Reported Target, Time
So What?

14. Reference RVs and Layers
4. A reference PSAW-MEBM model
Reference RVs and Layers
Target
Time
Reported
Target
Sensor
We distinguished three layers.
Observation, Object, and Situation layers.
The four core elements in the previous slides are here. Sensor, interpreted target, target, and time.
Each of them can be entities in PSAW.
And these entities are associated with random variables.
For example, observing conditions and situation relations.
This is a part of the reference PSAW-MEBM model.
We can use it to develop an MEBN model.
Where is this used?

15. 5. A PSAW-MEBN SYSTEM HERALD MEBN Model (Mtheory) for Critical Infrastructure Protection System
We developed an MBEN model or MTheory called HERALD MTheory.
For this, we referred to the reference PSAW-MEBM model in the previous slide.
We introduced a Critical Infrastructure Protection System briefly.
The Critical Infrastructure Protection System uses this HERALD MTheory.
Let me show the Critical Infrastructure Protection System in detail.
Where is this used?

16. What are operational scenarios for the case study?
5. A PSAW-MEBN SYSTEM Critical Infrastructure Protection System (HERALD) using the Reference MEBN model for PSAW
As a case study, we introduce a PSAW-MEBN model used for a prototype software PSAW system, called HERALD, which wards off attacks against critical infrastructure by means of early detection of threatening targets, identification of the targets, estimation of the target’s activities, and prediction of virtual short-term future situations
PSAW questions for HERALD
What is the type of the target?
Where will be the target located?
What type activity will be the target performing?
What mission does the target have?
How high will be the level of danger to the Critical Infrastructure?
What are operational scenarios for the case study?

17. Terrorist UAV Target Location? Target Type? Sensors Danger Level?
RED TEAM SCENARIOS
(1) In the terrorist attack scenario, on May 25, 20XX, a UH-60 helicopter carrying three armed terrorists head toward a nuclear power plant. At 800 m from the region, the helicopter airdrops the terrorists. The terrorists break through a surveillance system and infiltrate the plant. The terrorists occupy a critical infrastructure element at the plant, set up explosives at a vulnerable point of the critical infrastructure, and evacuate from the area.
Target Location?
Target Type?
Sensors
Danger Level?
(2) In the UAV attack scenario, on March 12, 20XX, three UAVs flying in formation at speed 74MPH head toward a nuclear power plant. The UAVs move to 12 miles from the plant and scatter to disperse from the line of sight of the Blue team. At 6 miles from the plant, each UAV containing a bomb flies into the plant as a suicide bomb attack.
Target Mission?
Target Activity?
UAV

18. 5. A PSAW-MEBN SYSTEM HERALD (a prototype software PSAW system)

19. 5. A PSAW-MEBN SYSTEM A Bayesian Network Derived from HERALD MTheory
Situation Layer
Danger Level?
Mission Layer
Target Mission?
Activity Layer
Target Activity?
Object (Property) Layer
Target Type?
Detection Layer
(Observations) T1 T2 T3

20. A process for Human-aided MEBN Learning in PSAW
6. Summary
Multi-Entity Bayesian Networks (MEBN) in Predictive Situation Awareness (PSAW)
A process for Human-aided MEBN Learning in PSAW
A reference PSAW-MEBM model
A PSAW-MEBN System (HERALD), Critical Infrastructure Protection System

21. We will improve reasoning speed and accuracy for the HERALD system
7. Future Work
The presented model will be extended to address other questions (e.g., When will a relationship between targets change?)
We will improve reasoning speed and accuracy for the HERALD system
We hope to apply HERALD to a variety of actual systems

22. Cheol Young Park, PhD candidate, at GMU
Thank you
Cheol Young Park, PhD candidate, at GMU

23. REFERENCES
Park, C. Y., Laskey, K. B., Costa, P. C. G., & Matsumoto, S. A Predictive Situation Awareness Reference Model using Multi-Entity Bayesian Networks. (2014). Proceedings of the Seventeenth International Conference on Information Fusion (FUSION 2014), Salamanca, Spain, 2014.
Park, C. Y., Laskey, K. B., Costa, P. C. G., & Matsumoto, S. (2013). Multi-Entity Bayesian Networks learning for hybrid variables in situation awareness. In Information Fusion (FUSION), th International Conference on (pp ). IEEE.
Carvalho, R. N. (2011). Probabilistic Ontology: Representation and Modeling Methodology. PhD Dissertation. George Mason University.
Laskey, K. B. (2008). MEBN: A Language for First-Order Bayesian Knowledge Bases. Artificial Intelligence, 172(2-3).
Costa, P. C. G. (2005). Bayesian Semantics for the Semantic Web. PhD Dissertation. George Mason University.
Steinberg, A.N., Bowman, C.L., & White, Jr., F.E. (1998). Revisions to the JDL Data Fusion Model. Proc. 3rd NATO/IRIS Conf. Quebec City, Canada.
Pearl, J. (1988). Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. San Mateo, CA, USA: Morgan Kaufmann Publishers.
Endsley, M. R. (1988). Design and evaluation for situation awareness enhancement. Paper presented at the Human Factors Society 32nd Annual Meeting, Santa Monica, CA.
Laskey, K. B., D’Ambrosio, B., Levitt, T. S., & Mahoney, S. M. (2000). Limited Rationality in Action: Decision Support for Military Situation Assessment. Minds and Machines, 10(1),
Wright, E., Mahoney, S. M., Laskey, K. B., Takikawa, M. & Levitt, T. (2002). Multi-Entity Bayesian NetworksforSituation Assessment. Proceedings of the Fifth International Conference on Information Fusion.
Costa, P. C. G., Laskey, K. B., Takikawa, M., Pool, M., Fung, F., & Wright, E. J. (2005). MEBN Logic: A Key Enabler for Network Centric Warfare. Proceedings of the 10th ICCRTS.
Suzic, R. (2005). A generic model of tactical plan recognition for threat assessment. In Defense and Security (pp ). International Society for Optics and Photonics.
Costa, P. C. G., Laskey, K. B., & Chang, K. C. (2009). PROGNOS: Applying Probabilistic Ontologies To Distributed Predictive Situation Assessment In Naval Operations. Proceedings of the 14th ICCRTS.
Carvalho, R. N., Costa, P. C. G., Laskey, K. B., & Chang, K. C. (2010). PROGNOS: predictive situational awareness with probabilistic ontologies. In Proceedings of the 13th International Conference on Information Fusion.