1. NCAR’s Societal Impacts Program: Research and Outreach Jeffrey K. Lazo, Director Julie Demuth, Associate Scientist NCAR Societal Impacts Program NOAA/NWS Seminar November 1, 2007 Jeffrey K. Lazo, Director Julie Demuth, Associate Scientist NCAR Societal Impacts Program NOAA/NWS Seminar November 1, 2007

2. Outline Overview of NCAR’s Societal Impacts Program (SIP) Results from recent U.S. household survey –Assess people’s views and attitudes about weather forecast information, including uncertainty information

3. Overview of the SIP

4. Origins of SIP Very little is currently known about: –economic value of current and improved weather information –users’ sources, perceptions, uses, and value of weather information –users’ understanding of, use of, and preferences for weather forecast uncertainty information –best practices for integrating new weather information into users’ environment Relevance to NOAA –Crucial to fulfilling NOAA mission –NOAA has minimal capacity to address these issues –Greater demands for NOAA to justify research and activities for budget decisions

5. Origins of SIP Collaborative Program on Societal and Economic Benefits of Weather Information –NOAA–NCAR collaboration –Peer reviewed –Unanimously accepted USWRP Interagency Working Group, July 2003 –Program commenced April 1, 2004, with hiring of a Director SIP as part of NCAR –Independent non-federal –History of interdisciplinary research in the interactions between weather and society –Unique within university community, tap extensive expertise in social sciences

6. Scope of SIP Mission –Improve the societal gains from weather forecasting by infusing social science and economic research, methods, and capabilities into the planning, execution, and analysis of weather information, applications, and research directions. Implementation –Research –Outreach and education –WAS*IS program –Community support

7. Current SIP research Communicating uncertainty in weather forecasts –Will discuss methods and results in more detail Overall U.S. sector sensitivity assessment Individual sector sensitivity assessment– transportation Warning decisions in extreme weather events Hurricane household valuation study User needs assessment Hydrometeorological testbed

8. Overall U.S. Sector Sensitivity Assessment (OUSSSA) “...one-third of the private industry activities, representing annual revenues of some $3 trillion, have some degree of weather and climate risk.” (Dutton, 2002) Evaluate sensitivity of 11 U.S. economic “super” sectors to weather variability Writing results for BAMS manuscript

9. Individual Sector Sensitivity Assessment–Transportation (ISSA-T) Assess transportation sector’s use and value of weather forecast information –To understand how the transportation sector is affected by weather and what weather information it deems useful –To develop valid and reliable socio-economic methods for assessing use and value Methodology –5 subsectors: air, rail, water, truck, pipeline –Eliciting judgments from experts in academia, private sector, and government

10. Warning Decisions in Extreme Weather Events How extreme weather warnings are communicated, interpreted, and used by different participants –Explore interactions among 4 groups: forecasters, public officials, media, public –Focus on hurricanes and flash floods; challenges for decision making in the face of risk and uncertainty Collaborators (NSF funded) –Kathleen Tierney and Jeannette Sutton -- U. of Colorado, Natural Hazards Center –Ann Bostrom -- U. of Washington, School of Public Affairs

11. Other SIP research User needs assessment –Summarize best practices for introducing weather decision-support technologies into new environments Hurricane household valuation study –Assess use and value of improved hurricane forecasts to households Hydrometeorological testbed –Assess societal impacts and economic value for American River area in California

12. Weather and Society * Integrated Studies (WAS*IS) Vision: To change the weather enterprise by comprehensively and sustainably integrating social science into meteorological research and practice Mission –Build an interdisciplinary, grassroots community of practitioners, researchers, & stakeholders who are dedicated to integrating meteorology and social science –Provide opportunities to learn and examine ideas, methods, & examples related to integrated weather-society work

13. WAS*IS Workshops –Original Boulder WAS*IS (Nov 2005, Mar 2006) –WAS*IS Norman (Apr 2006) –2006 Summer WAS*IS (Jul 2006) –WAS*IS Australia (Jan-Feb 2007) –2007 Summer WAS*IS (Jul 2007) 145 WAS*ISers (so far)!

14. WAS*IS Current and future activities –Program evaluation –Edited compendium (NCAR assessment initiative funding) –Interaction with AMS Policy Program –2008 Summer WAS*IS workshop (tentative) Output –WAS*IS overview paper by Demuth et al. forthcoming in November issue of BAMS –Conferences and meetings –Networking and collaborations Outcome –NWS “Advanced” Workshop, October 24-25 –New research and activities

15. SIP outreach and education Website –Extreme Weather Sourcebook –Societal Aspects Page –Digital Library Weather and Society Watch Newsletter Economics Primer WxSoc newsgroup

16. SIP community support Hurricane Forecast Socio-Economic Working Group –Special issue Natural Hazards Review (August 2007) –Joint NSF-NOAA funding opportunity (FY08) THORPEX –Societal and Economic Research and Applications (SERA) WG –NAT SERA (BAMS article forthcoming) Economics primer –NOAA funded WMO Public Weather Service Taskforce WMO CAS Strategic Planning AMS Annual Partnership Topic on Hurricane Disasters

17. Nationwide Survey of U.S. Households To assess people’s views and attitudes about weather forecasts and weather forecast uncertainty

18. Motivation: Community recognition Per NRC (2006) “Completing the Forecast” report recommendations: –Overarching Rec 2: collaborating with users and partners in the Enterprise and engaging and using social and behavioral science expertise –Rec 4.4: study uncertainty terms, icons, and other communications methods AMS (2002) statement on enhancing weather information with probability forecasts Other NRC reports, conference sessions, etc. Recent formation of the AMS Ad Hoc Committee on Uncertainty in Forecasts (ACUF)

19. Objectives To support the meteorological community in effectively providing weather forecast information, including uncertainty, by: –Assessing people’s sources, perceptions, uses and value of weather forecast information  Also assess how these factors change over time –Exploring people’s perception of, interpretation of, and preferences for weather forecast uncertainty information

20. Survey design and implementation Survey questions developed: –based on questions asked in previous survey research –to investigate fundamental research questions and issues raised by previous related work Pre-tested survey during development and implementation Implemented survey in November 2006 –Controlled-access, web-based with sample population provided by survey sampling company Analysis based on N=1465 respondents

21. Weather forecast research questions From where and how often do people get weather forecast information? For what locations or regions do people get forecasts? What times of day do people get forecasts? For what reasons do people get forecasts? What weather forecast parameters are important to people? What is people’s willingness to pay for forecasts?

22. How often do you get forecasts from the sources listed below? 0510152025303540 Local TV Cable TV Commercial or public radio Other webpages Newspapers NWS webpages Friends, family, co-workers, etc. NOAA Weather Radio Telephone weather info source Cell phone, PDA, pager, etc. Average household accesses weather forecasts 115 times per month! With over 113 million U.S. households, this totals to over 150 Billion forecasts accessed a year! Mean Monthly Access

23. On average, how often do you use weather forecasts for the activities listed below? 0%20%40%60%80%100% Simply knowing what the weather will be like Planning how to dress self or children Planning weekend activities Planning travel Planning yard work or outdoor house work Planning social activities Planning getting to work or school Planning job activities Rarely or neverLess than half the time About half the time More than half the timeUsually or alwaysNot applicable

24. How important is it to you to have the information below as part of a weather forecast? 0%20%40%60%80%100% When precip will occur Chance of precip Where precip will occur Type of precip High temp Amount of precip Chance of amount of precip Low temp Wind speed Humidity Time of day high temp will occur Time of day low temp will occur Cloudiness Wind direction Not at all importantA little importantSomewhat importantVery importantExtremely important

25. Do you feel that the NWS services you receive are worth more than, exactly, or less than $N a year to your household? Given values for $N varied among $2, $5, $10, $30, $60, $90, $120, $150, $180, $210, $240 $0 $50 $100 $150 $200 $250 0%20%40%60%80%100% Fitted Percent of Respondents Willingness to Pay

26. Uncertainty research questions How much confidence do people have in different types of weather forecasts? Do people infer uncertainty into deterministic forecasts and, if so, how much? How do people interpret a type of uncertainty forecast that is already commonly available and familiar: probability of precipitation forecasts? To what extent do people prefer to receive deterministic vs. uncertainty-explicit forecasts? In what formats do people prefer to receive forecast uncertainty information?

27. Do people infer uncertainty into deterministic forecasts and, if so, how much? (Perception)

28. Suppose the forecast high temperature for tomorrow for your area is 75°F. What do you think the actual high temperature will be? 0% 10% 20% 30% 40% 50% 75°F74-76°F (± 1°F) 73-77°F (± 2°F) 70-80°F (± 5°F) 65-85°F (± 10°F) Other Percent of Respondents

29. To what extent do people prefer to receive deterministic vs. uncertainty-explicit forecasts? In what formats do people prefer to receive forecast uncertainty information? (Preferences)

30. 0%10%20%30%40%50% Prefer Channel A (deterministic) Prefer Channel B (uncertainty) Like both channels Like neither channel I don't know Percent of Respondents Suppose you are watching the local evening news … The Channel A weather forecaster says the high temperature will be 76°F tomorrow The Channel B weather forecaster says the high temperature will be between 74°F and 78°F tomorrow.

31. All the choices below are the same as a probability of precipitation of 20%. Do you like the information given this way? Chance of precipitation is 20% There is a 1 in 5 chance of precipitation The odds are 1 to 4 that it will rain There is a slight chance of rain tomorrow  Percent  Frequency  Odds  Text Asked this question 3 ways – using PoPs of 20%, 50%, and 80% with corresponding text descriptions from NWS Asked this question 3 ways – using PoPs of 20%, 50%, and 80% with corresponding text descriptions from NWS

32. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% PercentFrequencyOddsText N = 489,489,487 PoP of 20% Percent of respondents who said “yes” PoP of 50% PoP of 80%

33. To summarize Weather forecast views and attitudes, including on uncertainty –150 billion served! –Majority of people like uncertainty info and many prefer it –People have preferences for how uncertainty information is conveyed Dissemination of results –Submitted manuscript with uncertainty results to Weather and Forecasting this week –Will submit manuscript with sources, uses, perceptions info to BAMS in early 2008 Connecting physical and social science in these ways will help more effectively communicate weather forecast information, including uncertainty!

34. Future work Future research questions to pursue –People’s interpretations of and preferences for uncertainty information in other formats (including graphics), in different weather situations, across different media, etc. –People’s use of different types of uncertainty information Grant funded this survey Seeking additional funding for future research in this area

35. Importance of social science Empirical research employing both quantitative and qualitative social science methods are needed –E.g., surveys, focus groups, interviews Need range of social science expertise –E.g., economics, communication, psychology, decision science

36. Societal Impacts Program... aims to improve societal gains from weather forecasting by infusing social science and economic research, methods, and capabilities into the planning, execution, and analysis of weather information, applications, and research directions research WAS*IS outreach and education community support research WAS*IS outreach and education community support

37. Thank you! Contact –Jeff Lazo, SIP Director (lazo@ucar.edu) –Julie Demuth (jdemuth@ucar.edu) –Rebecca Morss (morss@ucar.edu) www.sip.ucar.edu www.sip.ucar.edu/wasis