Presentation is loading. Please wait.

Presentation is loading. Please wait.

Dynamic Decision Making Lab www.cmu.edu/ddmlab Social and Decision Sciences Department Carnegie Mellon University 1 MODELING AND MEASURING SITUATION AWARENESS.

Similar presentations


Presentation on theme: "Dynamic Decision Making Lab www.cmu.edu/ddmlab Social and Decision Sciences Department Carnegie Mellon University 1 MODELING AND MEASURING SITUATION AWARENESS."— Presentation transcript:

1 Dynamic Decision Making Lab Social and Decision Sciences Department Carnegie Mellon University 1 MODELING AND MEASURING SITUATION AWARENESS IN INDIVIDUALS AND TEAMS Cleotilde Gonzalez In Collaboration with: Lelyn Saner, Octavio Juarez, Mica Endsley, Cheryl Bolstad, Haydee Cuevas, and Laura Strater

2 Computational Models of SA –Individual aspects of SA –Design aspects of SA –Organizational aspects of SA Measures of SA –Individual SA –Shared SA Conclusions 2 Agenda

3 Situation Awareness the Perception of the Elements in the Environment within a Volume of Time and Space, the Comprehension of their Meaning, and the Projection of their Status in the Near Future. Formation of SA influenced by:  Individual abilities  Interactions with others  Environment

4 Integrated theory of mind: ACT-R (Anderson & Lebiere, 1998) –Shared attention (Juarez & Gonzalez, 2003, 2004) –Learning theory (Gonzalez, Lerch & Lebiere, 2003; Gonzalez & Lebiere, 2005) –Representation of Recognition (Gonzalez & Quesada, 2003) –Learning and decision making in dynamic systems (Gonzalez et al., 2003; Martin, Gonzalez & Lebiere, 2004) Micro and Macro Cognition: Convergence and Constraints Revealed in a Qualitative Model Comparison (Lebiere, Gonzalez & Warwick, 2009) 4 Computational Cognitive Models

5 5

6 6 A SA meta-architecture provided a full set of cognitive models interacting with OTB, and resulting in the “commander’s SA” (Gonzalez et al., 2004; Juarez & Gonzalez, 2003)

7 7 Computational Models of Design Aspects of SA (Juarez & Gonzalez, 2006)

8 Computational Models of SA –Individual aspects of SA –Design aspects of SA –Organizational aspects of SA Measures of SA –Individual SA –Shared SA Conclusions 8 Agenda

9 Individual Measures of SA: SAGAT Situation Awareness Global Assessment Technique (SAGAT) Human-in-the-loop simulation exercises Use of SAGAT queries (from GDTA) Stop at random times and query the user Compare response with reality of the situation –Examples: What is the aircraft altitude? –What is the aircraft activity in this sector (en route, inbound to airport, outbound to airport) –Which aircraft will need a new clearance to achieve landing requirements? SAGAT score: accuracy of the responses

10 Individual SA measures, learning and working memory Can we learn to be aware? Effects of task practice and working memory influence situation awareness (SA) - Gonzalez & Wimisberg, 2007 How do we measure individual SA –Queries may be answered while the simulation display is not visible or covered (Endsley, 1995) or while the display is visible, uncovered (Durso et al., 1995). 10

11 Methods The design was a 2 x 18 mixed design. Participants were randomly assigned to one of two conditions (covered or uncovered display) and they were asked to run the simulation 18 times (trials). Individuals were asked to answer SA queries while the simulation was paused Participants took the Visual Span Test (VSPAN) (Shah & Miyake, 1996). 11

12 12

13 13

14 Results

15

16

17

18 The correlation between SA scores and VSPAN decreased over time SA scores were higher in the uncovered condition than in the covered condition –This is due mostly to perception The effect of practice was significant only in the covered condition, but not in the uncovered condition 18 Summary of results relevant for individual measures of SA

19 Measures of Shared SA (Saner, Bolstad, Gonzalez & Cuevas, in press; Saner, Bolstad, Gonzalez & Cuevas, in preparation) Ground TRUTH X1 X2 X3 X4 X5 X6 X7 Person 1 X1 X3 X4 X6 X7 X8 X9 Person 2 X1 X2 X4 X6 X8 Shared SA-the degree to which team members possess the same SA on shared SA requirements (i.e. on the information that they both need to know) (Endsley,1995, 1995b; Endsley & Jones, 2001) A good measure of shared SA needs to account for the ACCURACY

20 Shared SA Person 1 Q1 Q2 Q3 Q4 Q5 Q6 Q7 Person 2 Q1 Q2 Q3 Q4 Q5 Q6 Q7 SimQ1 SimQ2 SimQ3 SimQ4 SimQ5 SimQ6 SimQ7 Situation Awareness Global Assessment Technique (SAGAT) - Seven queries while task is stopped - Objective knowledge of situation Score Similarity = 1-absolute value of [(p1-p2)/(p1+p2)] Range from 0 to 1 A good measure of shared SA needs to account for the SIMILARITY

21 Method Training at Joint Personnel Recovery Agency (JPRA) - JFCOM 16 servicemen, 3 DoD contractors; Age M=33.85 Randomly assigned to one of four Teams: – Navy, Army, Special Operations, or Joint Service Utilized Cross-Training – Five scenarios over 3 days – Each scenario had 3 to 12 incidents – Scenarios randomly stopped 3 times for SAGAT, Communication, and Workload measures – Received training prior to the exercise

22 Methods and Procedure Joint Service Cell (p1, p2, p3, p4) Special Operations Cell (p13, p14, p15, p16, p17) Army Cell (p5, p6, p7, p8) Navy Cell (p9, p10, p11, p12) Joint Personnel Recovery Agency (JPRA) training exercise Four team groups (i.e. cells) Five Predictors of Shared SA – Experience Similarity- years in real service – Shared JPRA Knowledge- prior experience with recovery operations – Shared Cognitive Workload- subjective ratings, five NASA- TLX scales – Communication Distance- inverse frequency of communication – Organizational Hub Distance- degree of dissociation from Joint Service Cell

23 Possible Models Classic Hierarchy Expected

24 Results

25 Conclusions – Measures of Shared SA Development of a Shared SA measure must account for both, accuracy and similarity of SA between members of an organization As shared knowledge increased, so did shared SA. Organizational Hub Distance (OHD) is key predictor –Physical Distance and Joint Cell Membership Unexpected Role of OHD –Participants processed new information directly

26 Possible Models Expected Observed We observed that being in branch cells was associated with higher SSA rather than being in the joint cell

27 The success of Computational Models of SA, depends on appropriate and robust measures of individual and shared SA –Although individual measures and procedures exist, there is a huge need for defining the methods and procedures for measuring SA at the team level We investigated measures of SA at both, the individual and team levels –We created a shared SA measure that builds on individual SA Computational models of both, SA and SSA can incorporate these measures. 27 Conclusions


Download ppt "Dynamic Decision Making Lab www.cmu.edu/ddmlab Social and Decision Sciences Department Carnegie Mellon University 1 MODELING AND MEASURING SITUATION AWARENESS."

Similar presentations


Ads by Google