1. Evolving from WAS to IS: Societal Impacts in NWS Service Assessments Kevin Barjenbruch, NWS, Salt Lake City, UT Daniel Nietfeld, NWS, Omaha, NE Julie Demuth, NCAR Societal Impacts Program Kevin Barjenbruch, NWS, Salt Lake City, UT Daniel Nietfeld, NWS, Omaha, NE Julie Demuth, NCAR Societal Impacts Program 2008 Summer WAS*IS Workshop August 9, 2008 2008 Summer WAS*IS Workshop August 9, 2008

2. February 5-6, 2008, tornado outbreak 82 tornadoes, 5 rated a 4 on Enhanced Fujita Tornado Scale 57 fatalities; most since May 31, 1985, and 13 th of all time 350+ injuries Over $400M in property damage Dubbed Super Tuesday Tornado Outbreak

3. Overarching Goal/Activation: Conducted to evaluate NWS performance during significant (high-impact) events –Major economic impacts –Multiple fatalities or numerous serious injuries –Unusually high public or media interest Usually convened just once or twice a year NWS Service Assessments

4. Team Composition: Experts from both inside and outside the National Weather Service Process: Study the event itself and as well as actions before, during, and immediately after Work with designated officials at National Weather Service Headquarters to author report –Recommend changes in NWS procedures, products, and services to improve future performance Briefing provided to NWS Corporate Board Service Assessment posted for public consumption NWS Service Assessments

5. Inward Focus on NWS Procedures, actions, and equipment through –Internal review of operations in: National Centers, Weather Forecast Offices, Center Weather Service Units, and River Forecast Centers) –External assessment with various customers of the information – Trending toward IS! Emphasis on quantitative assessment –Fatalities and injuries –Damage –Verification, etc. NWS Service Assessments Service assessments moving from WAS…

6. 2008 assessments have included a Societal Impacts Analysis Component –Super Tuesday Tornado Outbreak –Mother’s Day Weekend Tornadoes in Oklahoma and Missouri, May 10, 2008 –Midwest Floods of June 2008 Increased emphasis on qualitative assessment –Actions taken –Changes in information delivery, etc. NWS Service Assessments To IS…

7. Utilize lessons learned, best practices, and analysis of customers/partners of weather information to –Improve delivery of hazard information (format and content) to our customers and partners –Improve clarity of hazard information –Focus research and training –Allocate resources NWS Service Assessments Desired outcomes Provide better and more understandable weather information so that people will take action to protect life and property!

8. Societal Impacts in Super Tuesday

9. Motivation This was a well-warned event …with such good information. Why did so many people die? Why don’t people do what they’re “supposed” to do … to make the “right” decision? We get frustrated when we put “good” weather information out there and people don’t make the “right” decisions!

10. Societal impacts scope The task – To try to understand why so many people died and the details of those fatalities –E.g., age, gender, whether warning was heard, warning source, whether they heeded the warning, structure where they died, whether they sought shelter, whether safer shelter was available Can learn so much by having people walk you through all this … by letting them tell you their stories! An opportunity – To gather empirical info about people’s actual warning response behaviors –what info people had, how they interpreted it (knowledge) –how people perceived the situation (perceptions) –what decisions people made (decision-making)

11. Methodology Semi-structured interviews with the public Targeted, convenience, and snowball sampling 41 interviews in the 6 WFOs visited (assessment team broke into 3 sub-teams) –Kevin and I did 17 public interviews over 4 days in the field, another day on the phone

12. Some of the questions When did you first realize there was a threat of a tornado in this area? –How did you learn about the threat? (Sources, environmental cues) –What were you thinking after you received that information? (Trust? Confusion? Uncertainty? Barriers to action?) –What did you do next? (Confirmation?) –Were there any unique circumstances about your day that affected your experience during the tornado event? What? Have you ever been in a similar type of extreme-weather situation in the past? (Experience, false alarms) –Did anything from that experience influence what you did during this most recent event? –Have you ever been warned about an extreme weather event in the past that did not occur? Think back over the entire tornado event, from the time you learned there was a tornado threat through when the tornado actually occurred. –Is there any other information you would have liked to have had?

13. A Few Key Results Knowledge –People get information from multiple sources Most commonly via television Also commonly from other people –People get information multiple times –Misconceptions about sirens Perceptions –Integration of seasonality, weather salience, situational awareness about the event –Personalization of the threat Seeking confirmation of the threat (e.g., Atkins woman, couple) Personal risk perception and optimism bias (e.g., Hardin Co. family, Arkansas family)

14. A Few Key Results Decision-making and sheltering –Decision-making is not a singular event; it happens numerous times throughout the warning process –Vast majority of people who received warning information sought shelter in best location available to them –Most people heeded the warning and sought shelter in the best available location, but… –Most people did not have an underground shelter or safe room available to them Fatalities –Collected as much good data as we could –Nearly 2/3 of victims were in mobile homes

15. Reflections and Wisdom from a Veteran WAS*ISer

16. WAS*IS Folded into Super Tuesday The value of WAS*IS Community to fall back on for support/knowledge Recognition that the NWS is not “all that” –Through interaction at workshop –Through partnership projects –Through the assessment Some exposure to quantitative and qualitative research Visibility=Opportunity Coping with sleep deprivation

17. WAS*IS Folded into Super Tuesday The OMG now what! Surveys The art of interviewing –Opportunity to observe and learn from Julie Excel…data analysis

18. Thoughts and Inspirations from a New WAS*ISer

19. I was a ship without a sail… Hurricane Charley Service Assessment – 2004 –EM Interviews –Media Interviews –Hours and hours of transcribing –Had a good “teacher” (Sociologist Betty Morrow) –NO knowledge of good wine Super Tuesday Service Assessment – 2008 –Learned from a great “mentor” (Julie and I suppose Kevin) –Was inflicted by passion for doing this work –Truly connected my tornado warnings with the impacts of them –Became a layexpert in Italian Reds Adrift on the open sea of societal impacts…

20. Storm-Based Warnings Beneficial ? Confusing ? Media interpretation ? Implications to NWS ?

21. My Job… Where should I focus my energy in the next 20 years? How can I influence the forecasters in my office? The internal workings of the WFO should be driven by the external impacts of our actions How should my seminars be structured? When I collaborate with UNL and Creighton University, which research projects should I emphasize and pursue?

22. Little Sioux Boy Scout Camp June 11, 2008 EF3 Tornado 4 children killed 12 minute lead time Sirens sounded Sought shelter We now have a forecaster who wants to do a study on this event, from a societal impacts perspective I can at least now call Julie and say “HELP!”

23. My Other Job… Teach “Severe Storms” at UNL –Shared my Service Assessment experience with students –Wow – student interest! UNL Alumni Advisory Board Short Course? Curriculum?

24. June 3, 1980 28 years later Community rebuilding Economic Impact Psychological Impact Sheltering New perspective

25. Julie’s Reflections and Lessons Learned

26. The essentials Partnerships among social scientist, research meteorologists, operational meteorologist, policy makers, practitioners, etc. –Kevin  links operational meteorologists, users –Julie  links users, social science research –Dan  links research meteorologists, operational meteorologists –The three of us are so much more effective together Interest and willingness to work together, to listen, learn, exchange ideas! To co-produce knowledge. Passion!

27. 3 Huskers + ISTJ + ENFP + INFJ = LATE NIGHTS! But I function well (had some of my best thoughts even) at 6:30 a.m. … after going to bed at 5:00 a.m.! Even still, it’s important to know who else on the team needs Starbucks daily (or twice daily, or …) Speaking of Starbucks, there are a lot of cops in Little Rock … and they’ll pull you over for *almost* running a red light. Speaking of Kevin’s driving, have a happy place that you can go to in an instant… Opportunities like this can change your life in ways you never dreamed possible…

28. Great strides in meteorology How has meteorology advanced in 15 years? How have warning and forecast operations advanced since 1993? We’ve undergone a paradigm shift of warning services –Extremely high expectations –Relatively proud confidence in meeting them –Will continue to advance and improve But …there is room for improvement...

29. The next great strides… To create “good / effective” products and services for users Provide people information that they actually want and use rather than what we think they do (or should) want and use