1. Turning a Team of Experts Into an Expert Team: Contributions From the Science of Teamwork and Simulation
Eduardo Salas, Ph.D.
Department of Psychology and
Institute for Simulation & Training
University of Central Florida

2. “…medicine is a team sport, with two exceptions: people’s lives depend on it and there are no coaches.”
-Atul Gawande, MD
In sports, when the team loses, the whole team loses, but in medicine, only the patient loses.

3. How do you enhance teamwork? What is the efficacy of team training?
Outline
What is teamwork?
How do you enhance teamwork?
What is the efficacy of team training?
Highlight some quantitative findings…
Series of meta-analyses
Highlight some qualitative findings…
Offer some observations
Offer some insights, principles…

4. Defining Characteristics of Teams
Specialized member roles and responsibilities
Task-relevant knowledge
Intensive communication
Adaptive mechanisms
Hierarchically organized
Two or more individuals
Multiple information sources
Meaningful task interdependencies
Coordination among members
Common, valued goals

5. The Environment... Complex, multi-component decisions
Rapidly evolving, ambiguous situations
Information overload
Severe time pressure
Severe consequences of error
Adverse physical conditions
Performance/command pressure
Distributed, multi-operator problems

6. Can we turn teams of experts into expert teams?
The Science…
Can we turn teams of experts into expert teams?

7. But… What is teamwork? How do we enhance it?
Does team training work in healthcare?
Can we derive some evidence based principles of team training?
What do we know from 30 years of science?
Inform Healthcare

8. The Drivers of Team Performance
Cognitions
Behaviors
Attitudes
Knowledge
Skills
Affect
“Think”
“Do”
“Feel”

9. The Drivers of Team Performance
Knowledge Competencies:
Cue/strategy associations—SMMs
“Knowledge of how to get knowledge”
Teammate characteristics, familiarity
Team mission, objectives, norms, resources
Roles and responsibilities
Individual-task proficiency, taskwork
Team Performance
Cognitions
Behaviors
Attitudes
Knowledge
Skills
Affect
“Think”
“Do”
“Feel”

10. The Drivers of Team Performance
Skill Competencies:
Mutual performance monitoring & adaptability
Supporting/back-up behavior
Team leadership
Task-related assertiveness
Conflict resolution
Closed-loop communication
Team Performance
Cognitions
Behaviors
Attitudes
Knowledge
Skills
Affect
“Think”
“Do”
“Feel”

11. The Drivers of Team Performance
Attitude Competencies:
Collective efficacy
Shared vision
Team cohesion
Mutual trust
Collective/team orientation
Value of teamwork
Team Performance
Cognitions
Behaviors
Attitudes
Knowledge
Skills
Affect
“Think”
“Do”
“Feel”

12. What effective teams do…

13. What effective teams do...
Hold shared mental models
…have members who anticipate each others’ needs
…can coordinate without the need to communicate overtly
Optimize resources
…self-correct
…compensate for each other
…reallocate functions
…can adapt their strategies under stress

14. What effective teams do...
Have clear roles and responsibilities
…manage expectations
…ensure team members’ roles are clear but not overly rigid
… understand each others’ roles and how they fit together
Share a clear and valued vision
…have a common purpose

15. What effective teams do...
Have strong team leadership
…are led by someone with good leadership skills and not just technical competence
…have team members who believe the leaders cares about them
…provide situation updates
…foster both task related behaviors and teamwork, coordination and cooperation

16. What effective teams do...
Engage in a cycle (a discipline) of pre-brief performance debrief
…regularly provide feedback to one another, both individually and as a team (“de-brief”)
…establish and revise team goals and plans
…differentiate between higher and lower priorities
…periodically diagnose team "effectiveness,” including processes, results, and team vitality (morale, retention, energy)

17. What effective teams do...
Develop a strong sense of “collective”—trust, teamness, confidence
…manage conflict well—team members confront each other effectively
…have a strong sense of team orientation
…trust other team members’ “intentions”
…strongly believe in the team’s collective ability to succeed
…develop collective efficacy

18. What effective teams do...
Engage in “rhythms” of performance
…manage time
…change at the “midpoint”

19. How do you promote effective teamwork?
Before task…
During task…
After task…

20. Promoting Teamwork … BEFORE
Compose the team: membership/selection
(Klimoski & Jones, 1995)
Select team members for required competencies
Personality tests
Simulations
Situational judgment tests
Select team leaders
Not much research!

21. Promoting Teamwork… BEFORE
Enhance individual team member capabilities
Individual taskwork training
Goal setting/ performance management
Individual feedback
On-the-job training
Coaching/mentoring
Team leader training (Tannenbaum et al., 1998)

22. Promoting Teamwork… BEFORE
Improve Team interactions/processes
Role clarification
Who does what with whom?
Team building
Facilitated process analysis
Self correction
Team pre-brief/de-brief interventions
Set expectations
Some validity

23. Promoting Teamwork … BEFORE
Build team competencies (Salas & Cannon-Bowers, 2001)
Cross-training (e.g positional rotation, position clarification training)
Team coordination training/CRM (Wiener, Helmeich & Kanki, 1993)
Meta-cognition training (Cohen et al., 1998)

24. Promoting Teamwork … DURING
Maintaining Teamwork
Scenario-based training
Guided Team self-correction (Smith-Jentsch et al., 1998)
Designed to teach teams to correctly identify errors and adapt “in real time” to compensate for those errors
Focus on behaviors:
Mutual performance monitoring tasks
Stress awareness
Compensatory behavior
Taking initiative
Communication

25. Guided Team Self-Correction
Self-Correction Process
Event Review
Error Identification
Feedback Exchange
Planning for Future
Enhanced Cognitions &
Shared Mental Models
Improved:
Expectations
Explanations
Task Understanding
Attitudes
Collective Orientation
Higher Cohesion
Behaviors
Enhanced Team Processes:
-Coordination
-Communication

26. Team Training Defined Team Training IS… Team Training is NOT...
A family of learning strategies
The systematic acquisition of teamwork KSAs
Focus on cognitive, behavior, and/or affective team-based change
Planned activity
Based on pedagogical principles
Team Training is NOT...
A group of people in front of a mannequin
A place, program or workbook
Not just saying “do better” at M&M conferences

27. Team Training Objectives
Structure of Team Training
Tools
Team Task Analysis
Task Simulation & Exercises
Feedback
Principles
Methods
Information-Based
Demonstration-Based
Video
Practice-Based
Guided Practice
Role Play
Strategies
Cross-Training
Coordination Training
Team Leader Training
Others...
Team Training Objectives
Content
Competencies
Knowledge
Skills
Attitudes

28. Team Training Strategies
Cross-training
Event-based approach to training
Scenario-based training
Self-correction training
Guided self-correction training
Assertiveness training
Stress exposure training
Team adaptation training
Team leader training
CRM and derivatives

29. How we know team training works…

30. Quantitative Perspective
Answers the question: Does team training work?
Meta-Analysis…?
Technique which summarizes all previous individual studies
Eliminates problems of using different measures, samples, etc.
Uncovers ‘true’ relationship
Reported as:
“r”  Uncorrected
or “ρ” (pronounced “rho”) Corrected for measurement and sampling errors

31. The Quantitative Literature
2 Meta-Analyses
Does Team Training Improve Team Performance? A Meta-Analysis (Salas, DiazGranados, et al., 2008)
Testing Three Team Training Strategies in Intact Teams: A Meta-Analysis (Salas, Nichols, & Driskell, 2007)

32. The Quantitative Literature
Comprehensive Search (1955 – 2008)
Over 300 empirical articles
245 journal articles
33 conference presentations
20 theses/dissertations
13 book chapters
13 technical reports
3 unpublished works
In order for us to be able to conduct a quantitative analysis of the literature on team training and team building we conducted a comprehensive search that spanned the years The entire database that was used for the three meta-analyses that I will review for you today contained 300 articles that was represented by a variety of forms of publications, but mostly were academic journal articles.

33. The Quantitative Literature
Over 10,000 teams represented
Aviation teams - 6,962
Lab teams - 1,845
Organizational teams - 1,527
Military teams - 464
Medical teams - 181
Education teams - 26
Sports teams - 12
These 300+ articles represented over 10,000 teams. Each article in this database examined the effects of team training on performance. Depending on the type of team training and the outcomes measured we meticulously coded each article and assessed whether it was usable to test our hypotheses.

34. Salas, DiazGranados, Klein, Burke, Stagl, Goodwin, & Halpin (2008)
Quantitative Results
Salas, DiazGranados, Klein, Burke, Stagl, Goodwin, & Halpin (2008)
Assessed relative effectiveness of these interventions on team outcomes:
Cognitive
Teamwork knowledge
Transactive memory
Affective
Attitudes towards teamwork
Burnout
Process
Communication
Coordination
Performance
Simulation performance
Number of errors
45 studies; 93 effect sizes
4 healthcare studies

35. Salas, DiazGranados, Klein, Burke, Stagl, Goodwin, & Halpin (2008)
Quantitative Results
Salas, DiazGranados, Klein, Burke, Stagl, Goodwin, & Halpin (2008)
Results
Overall, team training was shown to have a moderate, positive effect on team functioning (ρ = .34)
Moderator analysis investigating effect of team training on distinct outcomes
Four outcomes investigated:
Cognitive outcomes (ρ = .42)
Affective outcomes (ρ = .35)
Process outcomes (ρ = .44)
Performance outcomes (ρ = .39)

36. What do these results mean?
Team training does work!
Training content, team membership stability and team size impact the effectiveness of team training interventions.
Team training can explain 12%-19% of the variance of a team’s performance which means:
Reducing medical errors
Saving lives
Saving aircrafts
Increasing the bottom line

37. The Quantitative Literature
TTMA (Salas, Nichols, & Driskell, 2007)
Investigated relative contributions of three different team training strategies
Cross-training (CT)
Team members rotate positions during training
Develops shared understanding of KSAs and responsibilities of team members
Team coordination and adaptation training (TCT)
Teaches how to alter coordination strategy and to reduce the amount of communication necessary for successful task performance
Guided team self-correction training (GSC)
Use facilitated debriefs to learn how to diagnose the team’s problems and to develop effective solutions

38. The Quantitative Literature
TTMA (Salas, Nichols, & Driskell, 2007)
7 studies; 28 effect sizes
Results
Overall, team training improved performance (r = .29)
Accounted for 8.4% of variance
Team training effectiveness by components
TCT (r = .61) – Accounted for 37.2% of variance
GSC (r = .45) – Accounted for 20.0% of variance
CT (r = - .09) – No effect
Independent contributions to team training effectiveness
TCT (r = .30) – Accounted for 9.0% of variance
Cross-training and guided team self-correction training did not make significant independent contributions

39. What do these results mean?
TACT/CRM contributes the most to improving team effectiveness
TACT/CRM can explain 37% of the variance of a team’s performance
Outcomes that were examined in the primary studies included:
Shared expectations
Implicit coordination
Overall teamwork
Communication

40. does team training work?:
Why, how, and when
does team training work?:
A focus on healthcare

41. A Qualitative Review
Conducted a content analysis of the team training literature, examining factors such as…
How training was designed?
What content was included?
What instructional strategies were used?
How was it implemented?
How was it evaluated?

42. Description of Qualitative Database
Comprehensive Search ( )
Resulting in over 450 articles
Both empirical and theoretical
Representing team training in:
Aviation- 75 articles
Military- 150 articles
Sports- 11 articles
Organizations- 110 articles
Medicine- 120 articles
Of these 120, 37 were empirical studies of medical team training
Coded to determine principles & best practices for team training in healthcare

43. The Qualitative Method
Coding framework was established
50 pieces of information from each study (N=37)
Themes:
Background information
Training design
Components and features of training
Training evaluation
Content analysis conducted by analyzing text data
Generate themes
Provide understanding of phenomenon
Articles coded by 2 raters
Graduate students skilled in qualitative data analysis

44. Medical Team Training Database
Specialty Targeted for Team Training
16% of the articles explicitly targeted Anesthesia
14% explicitly targeted Emergency Medicine
5% explicitly targeted Pediatrics
35% targeted some other specialty area
24% targeted mixed specialties
5% did not identify a target specialty
Type of Providers Targeted for Training
22% specified physicians and nurses
5% specified anesthesiologists
11% specified students/residents
49% identified multiple types of providers
14% specified other classifications or did not identify a targeted population

45. Medical Team Training Database
Type of Team Training
62% of the articles used CRM
5% of the articles used team building
3% of the articles used cross training
3% of the articles used goal setting
30% of the articles used alternative types of team training
Training Focus: Teamwork vs. Taskwork
68% focused explicitly on
teamwork
30% focused on a combination of teamwork and taskwork
3% did not specify the focus of the training

46. Medical Team Training Database
Training Participation
54% participated in teams
25% of these reported intact teams
10% of these reported ad-hoc teams
65% of these did not specify
20% participated as individuals
27% did not specify 46 46

47. Medical Team Training Database
Instructional Methodology
86% used a mix of both traditional informational methods of instruction (e.g. lecture/classroom) plus modeling/demo, and practice or simulation
8% used practice and simulation only
3% of studies relied on informational methods only
3% could not be determined
Opportunities for Practice
86.5% explicitly reported providing opportunities for practice
50% of these reported using high fidelity simulation
25% of these reported using role play
6% of these reported using low fidelity simulation
19% of these did not specify the type of practice opportunity

48. Medical Team Training Database
Feedback
54% explicitly reported that feedback was given after practice scenarios
100% of these reported that feedback was given face to face
Majority was behavior based feedback
Training Duration
51% reported keeping training to 1 day or less
Range: 30 min – 8 hrs
32% reported that training lasted more than 1 day
Range: 2 days – several months

49. Medical Team Training Database
Training Evaluation
19% evaluated only trainee reactions (e.g. useful, enjoyable, etc.)
78% evaluated beyond training reactions
10 studies evaluated knowledge/learning/affect
21 evaluated behavior/process
9 evaluated organizational outcomes
Results
(Level 1 & 2)
54% explicitly reported that participants responded positively to training
30% explicitly reported improvement in participant learning/knowledge/affect after training

50. Medical Team Training Database
Results
(Level 3 & 4)
27% explicitly reported improvements in participant behavior on the job (e.g. communication, brief/debrief, preparedness)
24% reported the impact of training impact organizational outcomes…
Taylor et al (2008) : After CRM training significant positive changes in microalbumin and lower extremity amputation prevention
Created an ‘all outcome index’ (range 0% - 100%) and saw marginally significant improvement after training from 42.96% to 45.34% (p = .079)
Dunn et al. (2007): Several campuses indicated that adverse events such as wrong site surgery and wrong procedure have decreased (though no explicit empirical evidence provided!)

51. The Science of Simulation-Based Training….
What works!

52. Simulation-Based Training
Used to accelerate expertise…
Explicit links are developed and maintained between training requirements, exercise design, observation, and feedback…

53. Needed Links in Simulation-Based Team Training…
Tasks & KSAs
Learning Outcome
Scenarios & Events
Measures
Performance History
Feedback

54. The Science…

55. Myth
Simulation = Training

56. Reality Simulation is just a tool! A delivery mechanism
Need to embed instructional features!

57. SBT Works Best With…
…Clear & Precise Learning Outcomes
…Carefully Crafted Scenarios
…Diagnostic Measures
…Robust Observations Protocols

58. SBT Works Best With…
…Providing Developmental Feedback
…An Organized Pedagogical System
…Thinking About What Happens Before, During, and After SBT

59. Some Principles to Optimize SBT for Healthcare:
Beyond the bells and whistles

60. Clear & Precise Learning Outcomes
Tips:
Develop learning outcomes; serve as guide
Training teamwork competencies, not tasks
Define teamwork requirements
Conduct a cognitive task analysis; uncover aspects of the job
Principle 1: Focus on uncovering the team based competencies

61. Carefully Crafted Scenarios
Tips:
Create opportunities to demonstrate competencies (“triggers”)
Embed metrics (and instructional features)
Link “triggers” to competencies and learning outcomes
Principle 2: The Scenario is the Curriculum

62. Principle 3: Seek to diagnose KSA deficiencies
Diagnostic Measures
Tips:
Ensure metrics are diagnostic of clinical performance for AAR
Reinforce desired clinical behaviors
Focus on process measures
Design event-based measurement protocols
Principle 3: Seek to diagnose KSA deficiencies

63. Robust Observation Protocols
Tips:
Train the observers
Focus on both the individual and team level
Focus on 4-5 key performance dimensions
Principle 4: Develop multi-level performance
assessment tools

64. Principle 5: Provide constructive &
Feedback
Tips:
Make it timely
Provide diagnostic and developmental feedback
Focus on both: Individual and team level feedback
Create discipline to debriefing
Principle 5: Provide constructive &
team-based feedback

65. An Organized Pedagogical System
Tips:
Provide Information…
Trainees build declarative knowledge of the KSAs involved in teamwork
Demonstrate Behaviors…
Trainees learn ‘rough approximation’ of teamwork behaviors through observational learning
Create Practice Opportunities…
Trainees learn by doing…good performance is reinforced, poor performance is corrected
Provide Feedback and remediation…
Trainees refine teamwork competencies (deficiencies in KSAs are identified and addressed with corrective feedback and additional targeted training)
Principle 6: Apply Sound Instructional
Design Principles

66. Think About What Happens Before, During, and After SBT
Tips:
Sound positive messages about SBT & safety
Make sure you involve top-level support
Make sure key players are on board
Prepare organizational climate
Create opportunities to practice (after SBT)
Principle 7: Adapt a system approach to SBT

67. Principle 7: Adapt a system approach to SBT
Continued…
Tips:
Show management support (after SBT)
Ensure trainee motivation
Promote self-efficiency
Reinforce desired behaviors (after SBT)
Provide incentives
Principle 7: Adapt a system approach to SBT

68. Team training can only be successful if…
The design and delivery is scientifically rooted
The focus is on team-based competition
Physicians take ownership
Teamwork is rewarded and reinforced by both providers and administration
Practice opportunities are embedded…simulation!
Mechanics to maintain shared mental models are in place…briefs/debriefs

69. Thank You For Your Time! Eduardo Salas, Ph.D.
Department of Psychology and
Institute for Simulation & Training
University of Central Florida