1. TeamSTEPPS TM National Implementation Measurement The following slides are not part of the TeamSTEPPS Instructor Guide. Due to federal 508 compliance requirements only hard copies are available.

2. T EAM STEPPS 05.2 Mod 1 05.2 Page 2Page 2 Measurement TeamSTEPPS Phases Requires Measurement

3. T EAM STEPPS 05.2 Mod 1 05.2 Page 3Page 3 Measurement Why Measure? To know if your TeamSTEPPS implementation worked Step 5 of Implementation Planning Assist in modifying implementation plan Planning future implementations Information for leadership Producing a scholarly product.

4. T EAM STEPPS 05.2 Mod 1 05.2 Page 4Page 4 Measurement How to Measure - Kirkpatrick Level I – Reactions Level II – Learning Level III – Behavior Level IV – Outcomes (Like it and Useful) (Think, Do, Feel) (Transfer to the job) (Organizational results) MULTILEVELAPPROACHMULTILEVELAPPROACH

5. T EAM STEPPS 05.2 Mod 1 05.2 Page 5Page 5 Measurement Available Measures Curriculum Tools Course Evaluation Form Team Performance Observation Tool Team Assessment Questionnaire Learning Benchmarks New Tools – Not in Instructor Guide AHRQ HSOPS Culture Survey - Site Assessment TeamSTEPPS Attitude Survey Teamwork Perceptions Questionnaire

6. T EAM STEPPS 05.2 Mod 1 05.2 Page 6Page 6 Measurement Course Evaluation Form Level I Reactions Found under Tab A, Appendix B Includes evaluation questions for all components of the TeamSTEPPS Fundamentals Course Customizable Used for CME/CEU Provides information about what trainees thought about the training

7. T EAM STEPPS 05.2 Mod 1 05.2 Page 7Page 7 Measurement Team Performance Observation Tool Level II Learning and Level III Behavior Found under Tab A, Appendix C Tool for observing team performance Site assessment Measure training effectiveness Observers should practice using the tool Can be adopted to a particular unit Note: There are other tools for measuring teamwork in the literature

8. T EAM STEPPS 05.2 Mod 1 05.2 Page 8Page 8 Measurement Team Assessment Questionnaire Level III Behavior Found under Tab A, Appendix D 55-item self report questionnaire regarding teamwork in the unit Customizable Uses Site assessment Measure training effectiveness Cautions Not aligned with TeamSTEPPS Social desirability bias

9. T EAM STEPPS 05.2 Mod 1 05.2 Page 9Page 9 Measurement Learning Bench Marks Level II Learning Found under Tab A, Appendix F A 23-tem multiple-choice test Uses Assess knowledge of teamwork Measure changes in knowledge Cautions Item tend to be too easy

10. T EAM STEPPS 05.2 Mod 1 05.2 Page 10Page 10 Measurement AHRQ Patient Safety Culture Surveys Hospital Survey on Patient Safety Culture Nursing Home Survey on Patient Safety Culture Medical Office Survey on Patient Safety All can be found at: http://www.ahrq.gov/qual/patientsafetyculture/

11. T EAM STEPPS 05.2 Mod 1 05.2 Page 11Page 11 Measurement AHRQ Patient Safety Culture Surveys Level III Behavior and Level IV Outcomes A free, easy to use, reliable and valid instruments to assess patient safety sponsored by AHRQ Comprehensive instruments that assesses staff perceptions of patient safety culture Institution-wide Specific work area or unit Measures various dimensions of patient safety culture Perceptions of safety Teamwork within units Teamwork between units

12. T EAM STEPPS 05.2 Mod 1 05.2 Page 12Page 12 Measurement How can institutions use the Patient Safety Culture Surveys? Raise awareness about patient safety issues Assess their patient safety culture Track changes in patient safety over time Conduct internal and external benchmarking Evaluate the impact of patient safety interventions (e.g., TeamSTEPPS)

13. T EAM STEPPS 05.2 Mod 1 05.2 Page 13Page 13 Measurement Can institutions customize the surveys? AHRQ recommends only making changes that are absolutely necessary Modifications can endanger benchmarking If modifications are necessary, follow guidance provided in the Survey User’s Guide. Do not “shorten” instruments by deleting single questions; instead, delete all items in dimensions that are of lower priority Add new questions to the end of the survey

14. T EAM STEPPS 05.2 Mod 1 05.2 Page 14Page 14 Measurement What are the key data collection procedures? Sampling Goal is for survey respondents to be representative of all staff or specific subset(s) Length The surveys take approximately 10 minutes to complete Data collection mode Paper-based data collection recommended to ensure strong response rate Instructions and guidance for web-based collection is provided Analysis and reporting Each survey comes with a Users Guide that provides instructions for analysis and templates for reporting data

15. T EAM STEPPS 05.2 Mod 1 05.2 Page 15Page 15 Measurement What support is available? Patient Safety Culture Surveys Survey forms Survey user’s guide Survey feedback template Summaries of technical assistance telephone briefings Benchmarking data base and reports E-mail questions to the user support contractor safetyculturesurvey@westat.com

16. T EAM STEPPS 05.2 Mod 1 05.2 Page 16Page 16 Measurement TeamSTEPPS Attitude Survey Level II Learning 30-item self-report tool that assesses attitudes toward TeamSTEPPS core competencies Respondents rate their agreement with items on a 5-point Likert scale Measures attitudes toward Team Structure Leadership Situation Monitoring Mutual Support Communication Located at teamstepps.ahrq.gov

17. T EAM STEPPS 05.2 Mod 1 05.2 Page 17Page 17 Measurement Teamwork Perceptions Questionnaire Level III Behavior Similar to Team Assessment Questionnaire but aligned with TeamSTEPPS Measures staff perceptions of: Team Structure Leadership Situation Monitoring Mutual Support Communication Available at teamstepps.ahrq.gov

18. T EAM STEPPS 05.2 Mod 1 05.2 Page 18Page 18 Measurement Summary New tools are being developed that can help with measuring the success of your program Site assessment Program effectiveness Other measures exist in the literature Observational measures Culture surveys National Implementation project will: Share available tools Provide assistance with measures and analysis