1. Threat & Error Management
As you already know CRM was developed in the commercial sector as a response to aviation mishaps caused by human error and not by maintenance or aircraft related issues or problems. The Navy has adopted CRM and is currently on the 2nd generation of CRM with ACT being the 1st generation. TEM was developed as the 5th generation CRM in the commercial sector. CRM on the commercial side has not stopped evolving and they are currently up to the 7th generation of CRM. We have included this brief to help you stay informed of what is currently happening with CRM and where it is going. There are a couple of facets with in TEM that the Navy has adopted some new programs in conjunction with SMS.
Similar to “defensive driving”…”TEM does not teach pilots how to technically fly an airplane; instead, it promotes a proactive philosophy and provides techniques for maximizing safety margins despite the complexity of one's flying environment…TEM training can be framed as defensive flying for pilots” (Merritt & Klinect, 2006)
C A PPT

2. Enabling Objectives Explain importance of Threat & Error Management
Define threat
Describe categories of threats
Explain human limitations that lead to error
Define error
Describe categories of error
Explain T & E Management model and how to use it inside and outside of the aircraft

3. Mission Effectiveness
Threat and Error Model
Mission Effectiveness
Safe
Operations
Threats
Prepare
Prepare
Prepare
Errors
CRM/TEM
Repair
Repair
Undesired
Aircraft
State
Recover
Incident/Accident 3

4. Why Threat and Error Management?
History:
Commercial aviation thought that CRM was suit cased until the late 80s and early 90s when they had what they called the “big 5 in 5.” (5 major commercial mishaps that highlight human error and CRM as causal factors in 5 years)
CAL decided to do something, because if the commercial sector continues to crash aircraft then people won’t fly and they will ultimately lose business and money.
So, in 1996 Continental airlines and the University of Texas got together and threw some more money into CRM to figure out what were the mistakes being made and how to correct then and came up with TEM.
“It began with a simple question: “Do the concepts taught in training transfer to normal, everyday flight operations?” – Delta and UT

5. Why Threat and Error Management?
The idea behind all CRM Courses is to define the “best practices” in applying threat and error management counter-measures to reduce or eliminate the consequences of threats and errors, which are precursors of accidents and incidents. Safer operations can be had by imbedding the best practices of our pilots into our training and everyday operations. Pilots learn many of their positive traits from the sharing of ideas and experiences with their peers and then applying them to their own operational philosophy.
Read to yourself,
They realized like us, a lot of information is shared from aviator to aviator.
Best practices: A concept we use on assist visits to point out and acknowledge the various T/M/S programs that go above and beyond the minimums.

6. Why Threat and Error Management?
Tenets of the original University Texas Continental Human Factors Study (1996/2000):
To analyze adverse affects of errors within aviation
To define training needs to reduce crew related errors
To define organizational strategies to recognize and manage threat and error
None of the mishaps in the “big 5 in 5” were due to maintenance or aircraft related problems, they were due to human error and crew coordination break downs.
Bottom line if they continue they will lose money.
Continental and UT decided the best course of action was to collect data to see where the errors are being committed and in turn come up with a way or ways to manage those errors.\
If UAL was only 99.99% safe, they would have 1 accident every 4 days!

7. Why Threat & Error Management?
1st Generation – Cockpit Resource Management
Aircrew Coordination Training (ACT)
2nd Generation – Crew Resource Management
3rd Generation – Broadening the Scope
4th Generation – Integration and Proceduralization
Where we’re currently at
5th Generation – Error Management
6th Generation – Threat and Error Management

8. Simple weight and balance problem? Right?
Apparently not, at some point as the CG shifted aft and the donkey was getting light, they decided to keep loading.

11. C-2 Aft CG Crash

12. Why Threat & Error Management?
In the past: We focused on eliminating human error in aviation.
Contemporary acknowledgement: Errors are inevitable, so…
We must instead focus on ways to effectively MANAGE and REDUCE errors
Can we eliminate error?
Not if we consider to err is human, right?
However, we can reduce and mitigate and that is where TEM comes into the picture…

13. Why are we still doing CRM Training?
TEM is what we do…
…CRM is how we do it
MISSION EFFECTIVENESS
CRM 7 Critical Skills
Decision Making
Assertiveness
Mission Analysis
Communication
Leadership
Adaptability / Flexibility
Situational Awareness

14. Because Errors Exist

15. THREAT and ERROR MANAGEMENT MODEL

16. Adaptability / Flexibility Situational Awareness
MISSION EFFECTIVENESS
Mission Effectiveness
Safe
Operations
Threats
Prepare
Strategies
Errors
CRM/TEM
Repair
CRM 7 Critical Skills
Decision Making
Assertiveness
Mission Analysis
Communication
Leadership
Adaptability / Flexibility
Situational Awareness
Resist /
Resolve
Undesired
Aircraft
State
Recover
Incident/Accident

17. Mission Effectiveness
How do you define:
MISSION EFFECTIVENESS
The measurable results and impact of a mission based on how well it is executed. It is achieved through CRM/TEM using standard operating procedures, instructions and assessment of all available resources.
CRM - The effective use of all available resources by individuals, crews and teams to safely and efficiently accomplish the mission or task
Safe Operations
How do we define safe operations?
Invite class members to give their definition of safe operations. Typically you may hear participant responses such as: “no bent metal”, “no incidents or accidents”. Noble intentions indeed, but these typical responses are all outcomes resulting from something else; a process. Safe Operations.
If the term “safety” is mentioned, have the participants define what safety is. The participants may find safety somewhat nebulous to define; a moving target. Being tech savvy, have the participants “Google” it. Webster’s defines safety as being free from risk, harm, or danger. Safety and/or Safe operations are synonymous. Safety/ Safe Operations is a process not an outcome. (verb not a noun) Safe Operations is a process that creates an environment where Threats have been effectively managed using our CRM/TEM skills.
Challenge those that are not proactive. (Free from accidents/incidents… well are you lucky?)
Here is the civilian Flight Operations Manual definition:,
Successfully managing threats and errors
Effective implementation of CRM/TEM skills
Creating an environment where threats and errors are identified and managed effectively

18. MISSION EFFECTIVENESS
Threats
MISSION EFFECTIVENESS
Threats
 What are threats?
Threats occur outside the influence of the crew
Increase operational complexity
Require extra attention to maintain safety margins
LOSA data—3 threats per flight
What would be a typical threat?
A THREAT is anything that increases operational complexity that, if not managed properly, can decrease the safety margins.

19. Threats
ACTIVE
Factors that increase the likelihood of an error being committed, or of not accomplishing the mission.
LATENT
Aspects of the Detachment, Squadron or Strike Group organization, or individual that are not always easily identifiable, but that predispose the commission of errors, or the emergence of overt threats 19

20. We EFFECTIVELY PREPARE
Threats
Can we eliminate THREATS from aviation? No
How do we operate in spite of the fact that we cannot eliminate the THREATS from aviation?
We EFFECTIVELY PREPARE
and manage them.
20 of 20 20

21. Threats What is our strategy to manage threats? Identify and PREPARE
MISSION EFFECTIVENESS
Safe
Operations
Threats
Data indicates that crews who proactively deal with threats make fewer errors than crews who react to threats as they occur (DELTA)
Managing Threats
How do we manage threats?
Identifying and
Prepare for the threat
Prepare for threats—use our available resources
SOP, NATOPS,
Communicate plans and intentions to:
Other pilot
Appropriate resources/agencies
Develop contingency plans
Comprehensive brief
We cannot anticipate all threats.
Pick one or two of the specific examples and ask the pilots to discuss ways to prepare…ultimately most answers should come back to “following SOP is the best method to prepare for most predictable/foreseen Threats.
Identifying and preparing for Threats gets us back to safe operations. To manage Threats effectively, crews must identify threats which must naturally start with “situation awareness” (Identifying the Threat in the first place) . Crews Prepare for Threats by developing strategies to counter the Threats. How do we Prepare for Threats a crew may encounter?
Ask the class, what do we mean by prepare? Developing strategies for use as countermeasures to the Threats a crew identifies. How do we prepare? The crew prepares for Threats using SOP’s, checklists, NATOPS and STOP procedures. Our airplanes also provide a means to Prepare for Threats when crews use the automation, TCAS, GPWS, weather radar, and the other “Bells and Whistles” that are installed.
Threats left unmanaged increase operational complexity and provide an opportunity for crew error taking us away from Safe Operations.
Identify and PREPARE

22. Strategies to Prepare for Threats Things we can use in our tool bag to
counter threats:
ORM
Clearly defined roles of aircrew members
Complete briefing and effective communication
Limit being “heads down” at critical times
NATOPS / System Knowledge
CRM
What strategies do you use to counter
threats? 22

23. Verbalize, Verify, Monitor
VVM
Known as Cross Check Backup in DoD HFACS
Break the chain by recognizing errors and fixing them.
Don’t hope someone else will notice!
VVM is a key element of Threat and Error Management process.
The verbalization by both pilots “verifies” a shared understanding of what is expected and then both pilots “monitor” to ensure strict adherence to all instructions.
Verbalize, Verify, Monitor (VVM) is a key element of Threat and Error Management process
When should we employ Verbalize – Verify – Monitor? During every phase of the flight beginning when you arrive in the weather room until you leave the Flight Deck at the end of the flight.
 Pre-flight - In the weather room the crew discusses the flight paperwork – “Verbalize”. They “verify” fuel loads, aircraft weights, passenger and cargo loading, mission specifics, weather and alternates. During flight, the crew “Monitors” the flight plan and weather to ensure that the flight is progressing satisfactorily and Threats and Errors are being managed.
Taxi Clearances - Prior to taxi the crew “Verbalizes” the expected taxi plan When cleared to taxi, the CP reads back the taxi clearance, the pilot restates (“Verbalizes”) the clearance to ensure a shared and correct understanding. The verbalization by both pilots “verifies” a shared understanding of what the ground controller expects and then both pilots “monitor” the taxi operation to ensure strict adherence to all instructions.
Cleared to new altitude - When cleared to a new altitude the co-pilot reads back the clearance – both pilots verbalize the new setting is correct, and both visually verify the new setting and monitor the aircraft for the correct response 23

24. Adaptability/Flexibility Situational Awareness
CRM/TEM Skills
What skills do we have to help us IDENTIFY
and MANAGE (PREPARE for)?
Threats
Decision Making
Assertiveness
Mission Analysis
Communication
Leadership
Adaptability/Flexibility
Situational Awareness
24 of 20

25. Threats Threats Threats = Red Flags Do not equal errors
Increased potential for error
Threats = Red Flags
This is our introduction to the RED FLAG concept. It is important to build recognition here.
Also important to note here that, as the slide states, “a threat does not equal an error”. It only allows for the potential to commit an error. However, a threat should be perceived as a Red Flag to the crew, and even talked about amongst themselves as such.
“Threats must be identified and assessed as “Red Flag” warnings. When crews successfully recognize and acknowledge threats as red flags, they are in a better position to manage that threat so it becomes inconsequential….to effectively manage threats, they must be identified, then assessed, and then countered” (Gunther) 25

26. Errors Safe Threats Operations Errors
What can happen if we ignore or mismanage a THREAT?
MISSION EFFECTIVENESS
Safe
Operations
Threats
Errors
  No doubt—errors occur every flight.
What is an error?
Crewmember actions or inactions
which lead to deviations
reduce safety margins
LOSA data—2 errors per flight
Errors
26 of 20

27. Definition of Error Error: Error Management:
Crew action or inaction that leads to deviations from expectations, reduces safety margins, and may occur from either mismanaged threats or mistakes.
Error Management:
Process of correcting an error before it becomes consequential, i.e. Undesired Aircraft State and Incident/Accident
Threats come “at” the crew, while errors
come “from” the crew
Errors are defined as flight crew actions or inactions that: 1. Lead to a deviation from crew or organizational intentions or expections; 2. Reduce safety margines; and 3. Increase the probability of adverse operational events on the ground or during flight (M&K)
Simply put, threats come “at” the crew, while errors come “from” the crew (M&K) 27

28. Categories of Errors
Procedural - Following procedures but wrong execution (wrong altitude setting dialed into the FCP)
Communication - Missing info or misinterpretation (miscommunication with ATC)
Proficiency - Error due to a lack of knowledge/currency (lack of automation knowledge)
Decision - Crew decision unbounded by procedures that unnecessarily increased risk (unnecessary navigation through adverse wx)
Intentional Noncompliance - Violations (checklist from memory) 28

29. Operating with Errors How do we operate in an environment
where we cannot eliminate human errors?
Minimize and Manage Them!
The facilitation objective is to get the class to say that errors can only be minimized and managed to no consequence, but not eliminated
29 of 20 29

30. There are two categories of Error Management Tools
RESIST:
The hardware and software built into the system that trap ERRORS
RESOLVE:
The human skills that are used to trap and correct ERRORS
30 of 20

31. Errors Resist Hardware Human-ware What Are Your Systems Doing For You?
Resolve
GPWS
TCAS
FMC
Windshear
Detection
Map Display
WX Radar
GPS
Hardware
What Are Your Systems Doing For You?
Autopilot
Warning Systems
Monitoring
Professionalism
This model has become our framework for discussing our role as risk managers. clicks
We learned that using our best Crew Resource Management skills were necessary to recognize the threats in our operating environment and to develop strategies to manage such threats that naturally increased our workload and increased our potential to error and possibly lead us into an unsafe condition. Ultimately, our entire effort to reduce error or to trap and mitigate the consequence of error was captured in this conceptual model. It suggests we use all the tools available to us to operate effectively and to avoid the negative consequence of uncorrected error.
We have come to recognize if we do this effectively, then we are more likely to accomplish a safe outcome. To do less is not acceptable. Again, we are addressing our first objective to operate as Safely as possible.
As captains, how do you model use of this risk management process? How do you make sure the team is functioning as a team focused on Safety First?
Challenging
Systems
Knowledge
Attitude
Experience
Human-ware
What Tools Are You Bringing To The Fight?
Health 31

32. Strategies for Errors What is our strategy to manage errors?
Mission Effectiveness
Threats
Prepare
Managing Errors
  An error occurs—now what?
Identify the error
Respond by repairing the error
Monitor/crosscheck for adherence to SOP / NATOPS
Manage workload to prioritize complex tasks
Communicate plans—other pilot and appropriate resources/agencies
Choose appropriate levels of automation
Briefing—common errors expected in unusual conditions
Always, the process must start with having enough SA to identify. Identifying and Repairing Errors gets us back to safe operations.
Ask the class, what do we mean by Repair? Developing strategies for use as countermeasures to Errors a crew identifies. How do we repair crew error? The crew can repair Errors using SOP’s, checklists, flight planning paperwork.
Our airplanes also provide a means to repair Errors when crews correctly use automation, TCAS, GPWS, weather radar, and the other “Bells and Whistles” that are installed.
Try to draw from the previous examples of threats that an error could have made from (e.g. snow falling on taxi-out was the threat, after engine start you forget to turn on the engine anti-ice, of course once you realize (identify) you turn the engine anti-ice on and run- up the engine, thus “repair”).
Errors
Repair
Identify and REPAIR
32 of 20

33. What skills do we have to help us Manage Errors?
CRM/TEM Skills
What skills do we have to help us Manage Errors?
Errors
Decision Making
Assertiveness
Mission Analysis
Communication
Leadership
Adaptability/Flexibility
Situational Awareness
33 of 20 33

34. What can Happen if We Don’t “Repair” the Error ?
Undesired Aircraft State: A position, speed, attitude, condition, or configuration of an aircraft that reduces safety margins. A safety-compromising state that results from ineffective error management
Mission Effectiveness
“An undesired aircraft state (UAS) is defined as a position, speed, altitude, or configuration of an aircraft that: 1. Results from flight crew error, actions, or inaction, and 2. Clearly reduces safety margins (M&K)
“In other words, a UAS is a safety-compromising state that results from ineffective error management” (M&K)
Undesired Aircraft State
What can result when we mismanage errors?
We can find ourselves in an Undesired Aircraft State (UAS)
Definition of a UAS:
any position, speed, attitude, condition, or configuration of an aircraft
that reduces safety margins
often a result of mismanaged errors
A continuing trend away from safe operations
Your engine fails between V1 and V2:
Threat?
Error?
UAS?
It depends…
Threats
Prepare
Prepare
Errors
Repair
Undesired
Aircraft
State
34 of 20

35. How do we Manage an Undesired Aircraft State?
Mission Effectiveness
Threats
Prepare
Prepare
Prepare
Errors
Response to an Undesired Aircraft State
First step to manage Undesired Aircraft State?
Identify (announce)
Recover
Restore the airplane to safe operations in a timely manner
Situation awareness
Monitor/Cross-check
Communication
Leadership Effectiveness
Identifying and recovering from a Undesired Aircraft State gets us back to safe operations. Ask the class participants, what do we mean by Recover? Developing strategies for use as countermeasures for a UAS a crew identifies. How do we recover from a UAS? The crew can recover from a UAS using SOP’s, NATOPS, flight planning. Our airplanes also provide a means to recover from a UAS when a crew correctly use automation, TCAS, GPWS, weather radar, and the other “Bells and Whistles” that are installed.
Slide Description
Identify Recover
Objective
Continue building the tool.
Facilitation Strategy
Have the class describe how INSERT MISHAP HERE recovered/or didn’t/ from their UAS’s and how they could have recovered sooner to avoid the incident.
Slide Transition
Next slide incident/accident
Repair
Repair
Undesired
Aircraft
State
Recover
Identify and RECOVER 35

36. What if we do not Recover?
MISSION EFFECTIVENESS
Threats
Prepare
Prepare
Prepare
Errors
Repair
Repair
Undesired
Aircraft
State
Recover
Incident/Accident
36 of 20

37. Adaptability / Flexibility Situational Awareness
How are TEM and
CRM related?
MISSION EFFECTIVENESS
CRM 7 Critical Skills
Decision Making
Assertiveness
Mission Analysis
Communication
Leadership
Adaptability / Flexibility
Situational Awareness
TEM is what we do…
…CRM is how we do it
What
How

38. What is a Norm?

39. What is a Norm?
Unwritten practice accepted by most members of a given group
Can be positive or negative
Effective/Ineffective
What have you seen in your previous flight experience?
What level of failed components does the C-2 community routinely accept to accomplish the mission and where do you draw the line?
A “tech-cedure,” time saver, and convenience
Canvas and have everyone answer for their experience…
Why do norms start?
Has anyone heard “I know that this is how they do it in the FRS, but this is how we do it here?”

40. What are your expectations?
Compliant with SOP
When you walk to the aircraft what are your expectations? What is your tolerance?
Example: ready or alert launch and need to get off deck and only complete enough of the checklist to take-off and finish the rest once airborne…
Expected to conduct a full tactical pre-flight but actually don’t do all of the mission systems until airborne…
Knowing that, if you, your wife, your child, whomever, were going under the knife for a surgical procedure what would your expectations be of that surgeon?
First click brings in colors
Next click brings in ‘Compliant with SOP’ build on that first, it will be the easiest to get from the class.
Next click brings in ‘Reckless Behavior”. This too should be easy to illustrate with extreme examples.
Next click brings in ‘At risk behavior.’ Example: Procedure says that a doctor will scrub his hands for 60 seconds. What if he only does it for 45 seconds instead. Is he Compliant? No. Reckless? No. At Risk? Yes.
Last click animates the DRIFT LINE.
At Risk Behavior
Reckless Behavior

41. Norms
Drift is the gradual departure from an intended course due to external factors.
Compliant with SOP
Introduce and illustrate drift
They start out being compliant, but over time begin to drift away from being compliant and become at risk behavior with increased potential to become reckless behavior and cause a mishap
Highlights the DRIFT and reintroduces the definition. Why do we get away from Compliant behavior? Facilitate this for a minute.
What are some of the Internal/external factors that may cause this to happen?
At Risk Behavior
Reckless Behavior

42. Normalization of Deviance
Diane Vaughan developed her theory of the normalization of deviance in The Challenger Launch Decision. She details how, during the developmental phase of the Space Shuttle Program, the normalization of deviance resulted in a dangerous design flaw in the design of the spacecraft. The group that was assessing the joints on the solid rocket boosters conducted analysis to find the "limits and capabilities of joint performance. Each time, evidence initially interpreted as a deviation from expected performance was reinterpreted as within the bounds of acceptable risk"[7]. The acceptance of this risk led to the Challenger exploding on the morning of January 28, 1986.
Morton-Thiokol was contracted by NASA to manufacture the Solid Rocket Boosters (SRB) that were used in the Space Shuttle Program. In 1981, a problem with the putty that was used to seal the O-rings on the SRBs was discovered. When the putty was added to the boosters, bubbles formed. During take-off, the gases from inside of the SRB would go through the bubbles resulting in a "localized high temperature jet which was drilling a hole right into the O-ring"[8]. Morton-Thiokol changed the putty and the method of putty application and considered it fixed. The engineers knew that the putty erosion could still occur, but with a very low probability of a catastrophic disaster. NASA determined that the erosion of the putty was an acceptable risk of flight. NASA and Morton-Thiokol characterized the erosion as an anomaly that was to be expected since the SRBs were such a new technology. Subsequent test flights showed putty erosion that was deemed acceptable by NASA and Morton-Thiokol even though the joint actually "deviated from expected performance"[9].
NASA and Morton-Thiokol suffered from the normalization of deviance when assessing the safety of the SRBs. Diane Vaughan states, "As [NASA and Morton-Thiokol] recurrently observed the problem with no consequence they got to the point that flying with the flaw was normal and acceptable"[10]. On January 28, 1986, the normalization of deviance within the two organizations contributed to the loss of the Space Shuttle Challenger and the seven astronauts on board.
From:

43. Normalization of Deviance
Defined by Sociologist Dr. Diane Vaughan
“The gradual process through which unacceptable practice or standards become acceptable. As the deviant behavior is repeated without catastrophic results, it becomes the social norm for the organization.”
“To people outside of the organization, the activities seem deviant; however, people within the organization do not recognize the deviance because it is seen as a normal occurrence.”
Diane Vaughan is an American sociologist who is a professor at Columbia University's Department of Sociology.

44. Normalization of Deviance

45. How much deviance do they expect?
Compliant with SOP
At Risk Behavior
Reckless Behavior
Let’s put it in perspective and close the analogy loop here.
How much drift are the passengers expecting you to allow? NONE.
Your CO/Skipper expects zero drift from you right?
The only thing that these Marines are told is that we are to flight them from point A to point B so they can do their job, so that is what they expect from us, they expect us to accomplish our mission in a effective and safe manner…
That being said, my challenge to you is thus: if the norms in your community are good then get them put into NATOPS or an SOP, otherwise stop using them…

46. Why Discuss Norms?
Norms can lead to mismanagement of Threats, Errors, and UASs
Just because it “worked” one time doesn’t mean it will always “work”
They can lead to the ‘drift’ from SOP compliance to reckless bevhaior
If a Norm is good, put it in the SOP/NATOPS

47. Case Study E-2 / C-2 CRM Instructors:
Execute your CRM-I Case Study emphasizing the Threat and Error Management Model or use a recent HAZREP and the Threat and Error Management Model to discuss an applicable CRM Case Study.

48. Safe Threats Operations CRM/TEM Errors Incident/Accident
MISSION EFFECTIVENESS
Safe
Operations
Threats
Prepare
Strategies
CRM 7 Critical Skills
Decision Making
Assertiveness
Mission Analysis
Communication
Leadership
Adaptability / Flexibility
Situational Awareness
Errors
CRM/TEM
Repair
Resist /
Resolve
Undesired
Aircraft
State
Identify:
Threats
Strategies
Errors
Resist/Resolve
Recover
Incident/Accident

49. Summary Why Threat & Error Management (TEM) Threat Categories
Error Categories
TEM Model
TEM is what we do…
…CRM is how we do it

50. Questions?