1 Examining the Impact of Narrative Case Studies in Toolbox Talks for Building Construction Terri Heidotting, Ed.D. Education and Information Division NIOSH Best Practices in Occupational Safety and Health, Education, Training, and Communication: Ideas That Sizzle Baltimore, Md - October 2002

2 Injury and Fatality Incidence Rates Sources: BLS Census of Fatal Occuptional Injuries and Employment by Industry, 2001 ( BLS Incidence Rates of Nonfatal Occuptional Injuries and Illnesses by Industry Division and Selected Case Types, , Table 7 ( )

3 What are Toolbox Talks Also called tailgate talks used in wide range of industries – construction, mining, agriculture Brief (10-15 minute) weekly safety training sessions (talks) Safety talks conducted on the work site Safety talk conducted by the site supervisor, foreman, safety supervisor, or senior employee

4 Background Previous focus groups on improving toolbox talks - Role of narratives / stories Need for quality training materials that are: –Inexpensive and easily accessible –Have a wide variety of safety topics –Evaluated for their effectiveness Safety training materials developed with input from the industry Instructional design of materials grounded in accepted learning theories

5 Training Intervention Effectiveness Research (TIER) Formative Evaluation: –Determination of training needs –Conceptualization of goals and objectives to meet the needs Process Evaluation: –Development of draft training materials –Field testing of data collection instruments Outcome Evaluation: –Controlled study to determine if intended outcomes are obtained and sustained –Identification if critical elements Impact Assessment: –Longitudinal study - Did training met the educational needs identified in Stage 1 –Examination of the impact of training on the learner and learners environment NIOSH Publication Number: Website:

6 Pilot Study Draft toolbox lessons –Treatment – with story (narrative) –Control – without story (companys own lessons) Data collection and analyses –Focus groups –Interviews –Observations of training sessions Participants –Electricians, carpenters, laborers with 4 to 38 years of experience Feedback from instructors and employees

7 Toolbox Talks – Meeting the Needs of Instructors and Employees Instructors –Uncomfortable presenting materials in front of others –Want to present quality materials that will keep the employees interest and encourage their participation Typical toolbox materials are pretty lame It just goes in one ear and out the other Employees arent paying attention…dont care to be there…they want to sign their name and leave. –Want materials that are easy to use Dont have pre-training preparation time Employees –Want quality materials that are interesting Sometimes it seems like it never happened Ineffective toolbox talks are rushed for time…youre going to lose your interest right off the bat –Want topics that reflect work being done –Get to the point!

8 Pilot Versions of NIOSH Toolbox Talks for Construction

9 NIOSH Narrative Toolbox Talks What Worked Real-life stories would get my attention Stories that reflect incidents in the employees geographical area more interesting Discussion questions pertaining to the story facilitated participation and interest Stories made the information easier to understand and remember Employees and instructors reported they could relate to the person in the story Safety topics reflected work done at the site Real Life Incident A carpenter and his co-worker were overcome by carbon monoxide (CO) when they entered a basement area where a gasoline-powered generator was lowered into the basement to serve as the source of power. The first worker used a ladder to climb down into the basement to check the generator and was overcome by carbon monoxide. The second worker, who probably entered the basement to assist the downed worker, was also overcome. The workers were dead when they were found and the basement was…….

10 NIOSH Narrative Toolbox Talks Changes Made Dont include unnecessary details Need variety --Everybody dies Bullet additional information for easier manipulation by instructor Present story discussion questions immediately afterwards Vocabulary level of text should be simple Text structured to be read verbatim Pictures added

11 Revised NIOSH Toolbox Talks for Construction

12 Next Steps... Formal study – treatment and control groups –Impact of real life cases Pre-training versus post-training changes in: –Knowledge gains –Safety attitudes –Site observations –Satisfaction with training –Potential differences in injury rates –Instructor feedback Ongoing improvements in training lessons Development of public domain materials accessible to all

13 For more information please contact: Terri Heidotting, Ed.D. National Institute for Occupational Safety and Health Education and Information Division 4676 Columbia Parkway, C-10 Cincinnati, Oh / NIOSH