1. New Methods to Bridge Long-Term Policy Analysis and Robust Decision Making Robert Lempert Director RAND Frederick S. Pardee Center for Longer Range Global Policy and the Future Human Condition BLOSSOM Workshop European Environmental Agency April 30, 2008

2. 2 4-30-08 Long-Term Decisions Present Difficult Challenges Long-term decisions occur –When reflecting on potential events decades or more in the future –Causes decision makers to choose near-term actions different than those they would otherwise pursue Long-term decisions occur –When reflecting on potential events decades or more in the future –Causes decision makers to choose near-term actions different than those they would otherwise pursue Long-term policy analysis often fails to persuade because –Conditions of deep uncertainty prevail –Short-term needs loom large –Views of the future often anchored in the present Long-term policy analysis often fails to persuade because –Conditions of deep uncertainty prevail –Short-term needs loom large –Views of the future often anchored in the present

3. 3 4-30-08 Long-Term Decisions Present Difficult Challenges Long-term decisions occur –When reflecting on potential events decades or more in the future –Causes decision makers to choose near-term actions different than those they would otherwise pursue Long-term decisions occur –When reflecting on potential events decades or more in the future –Causes decision makers to choose near-term actions different than those they would otherwise pursue “Missions to Mars” at Disneyland’s Tomorrowland ca 1955 Long-term policy analysis often fails to persuade because –Conditions of deep uncertainty prevail –Short-term needs loom large –Views of the future often anchored in the present Long-term policy analysis often fails to persuade because –Conditions of deep uncertainty prevail –Short-term needs loom large –Views of the future often anchored in the present

4. 4 4-30-08 Scenarios Attractive for Long-Term Analysis, but Have Weakness in Contentious Public Debates At their best, scenarios can help decision makers –Reduce overconfidence –Challenge their mental models –Overcome organizational and psychological barriers to considering threatening or inconvenient futures At their best, scenarios can help decision makers –Reduce overconfidence –Challenge their mental models –Overcome organizational and psychological barriers to considering threatening or inconvenient futures

5. 5 4-30-08 Scenarios Capture the Key Concept That a Multiplicity of Plausible Futures May Be as Close as We Get to the Truth Rabbi Eliezer Ashkenazi (1580) chose to interpret the Tower of Babel story not as a challenge to divine power to which the Lord's response was to divide the human race but rather the opposite. He saw the story as an attempt to establish a universal religious regime which God "was obliged to separate …since the proliferation of doctrines aids and stimulates the investigator to attain the desired truths.”

6. 6 4-30-08 Scenarios Attractive for Long-Term Analysis, but Have Weakness in Contentious Public Debates At their best, scenarios can help decision makers –Reduce overconfidence –Challenge their mental models –Overcome organizational and psychological barriers to considering threatening or inconvenient futures But in contentious public debates, scenario methods can have difficulty –Engaging mental models of diverse stakeholders –Systematically informing decisions under uncertainty –Addressing surprise and discontinuities At their best, scenarios can help decision makers –Reduce overconfidence –Challenge their mental models –Overcome organizational and psychological barriers to considering threatening or inconvenient futures But in contentious public debates, scenario methods can have difficulty –Engaging mental models of diverse stakeholders –Systematically informing decisions under uncertainty –Addressing surprise and discontinuities

7. 7 4-30-08 Outline Robust decision making (RDM) provides framework for effective long-term analysis RDM’s “Scenario Discovery” approach offers useful scenario concept for public debates Recent work measures the impacts of these approaches with decision makers Robust decision making (RDM) provides framework for effective long-term analysis RDM’s “Scenario Discovery” approach offers useful scenario concept for public debates Recent work measures the impacts of these approaches with decision makers

8. 8 4-30-08 RDM Views Scenarios As Part of Process Identifying and Building Consensus for Robust Strategies Key Robust Decision Making Concepts: Construct ensemble of long-term scenarios that highlight key tradeoffs among near-term policy choices Consider near-term choices as one step in a sequence of decisions that evolve over time Use robustness criteria to compare alternative strategies Key Robust Decision Making Concepts: Construct ensemble of long-term scenarios that highlight key tradeoffs among near-term policy choices Consider near-term choices as one step in a sequence of decisions that evolve over time Use robustness criteria to compare alternative strategies –A robust strategy performs well compared to the alternatives over a wide range of plausible futures

9. 9 4-30-08 New Technology Allows Computer to Serve As “Prosthesis for the Imagination” Robust Decision Making (RDM) is a quantitative decision analytic approach that –Characterizes uncertainty with multiple, rather than single, views of the future –Evaluates alternative decision options with a robustness, rather than optimality, criterion Robust Decision Making (RDM) is a quantitative decision analytic approach that –Characterizes uncertainty with multiple, rather than single, views of the future –Evaluates alternative decision options with a robustness, rather than optimality, criterion

10. 10 4-30-08 New Technology Allows Computer to Serve As “Prosthesis for the Imagination” Robust Decision Making (RDM) is a quantitative decision analytic approach that –Characterizes uncertainty with multiple, rather than single, views of the future –Evaluates alternative decision options with a robustness, rather than optimality, criterion –Iteratively identifies vulnerabilities of plans and evaluates potential responses Robust Decision Making (RDM) is a quantitative decision analytic approach that –Characterizes uncertainty with multiple, rather than single, views of the future –Evaluates alternative decision options with a robustness, rather than optimality, criterion –Iteratively identifies vulnerabilities of plans and evaluates potential responses Candidate strategy Identify vulnerabilities Assess alternatives for ameliorating vulnerabilities RDM combines key advantages of scenario planning and quantitative decision analysis in ways that –Decision makers find credible –Contribute usefully to contentious debates RDM combines key advantages of scenario planning and quantitative decision analysis in ways that –Decision makers find credible –Contribute usefully to contentious debates

11. 11 4-30-08 Stylized Sustainability Example Summarizes RDM Approach What near-term actions can help ensure economic growth and environmental quality over the 21 st century? Economic growth rate (%) 1.02.03.04.0 –1.0 0 1.0 5.0 3.0 4.0 2.0 0 India since 1960 U.S. 1890-1930 U.S. since 1950 U.S. in 20th century China since 1960 Brazil since 1980 Russia since 1993 Decoupling rate (%) (Rate at which technology, without regulation, reduces emissions per GDP) Analysis suggests testing alternative strategies over this range of futures

12. 12 4-30-08 Compare “Fixed” Near-Term Strategies Across Scenarios Near term Choose policies Assume near-term policy continues until changed by future generations Future decision- makers recognize and correct our mistakes Future RAND MR-1626-RPC

13. 13 4-30-08 1.02.03.04.0 –1.0 0 1.0 5.0 3.0 4.0 2.0 0 Conventional world scenario Strategy’s Performance Strategy’s Performance No regret Mild A lot Overwhelming 1.02.03.04.0 –1.0 0 1.0 5.0 3.0 4.0 2.0 0 Economic growth rate (%) Decoupling rate (%) Stay the Course U.S. in 20th century U.S. since 1950 Initial Scan Suggests No Fixed Emission Reduction Target Is Robust Strategy Vulner- abilities Alternatives 1.Run simulation thousands of times 2.Display “scenario maps” showing deviation of proposed strategy from optimality over many futures

14. 14 4-30-08 1.02.03.04.0 –1.0 0 1.0 5.0 3.0 4.0 2.0 0 Conventional world scenario Strategy’s Performance Strategy’s Performance No regret Mild A lot Overwhelming 1.02.03.04.0 –1.0 0 1.0 5.0 3.0 4.0 2.0 0 Economic growth rate (%) Decoupling rate (%) Stay the Course U.S. in 20th century U.S. since 1950 Initial Scan Suggests No Fixed Emission Reduction Target Is Robust 1.02.03.04.0 –1.0 0 1.0 5.0 3.0 4.0 2.0 0 Conventional world scenario 1.02.03.04.0 –1.0 0 1.0 5.0 3.0 4.0 2.0 0 Crash Effort U.S. in 20th century U.S. since 1950

15. 15 4-30-08 Craft Near-Term Adaptive Strategy That Aims to Balance Environmental and Economic Goals PresentFuture NO Does the carrying capacity change? Choose policies to maximize utility Determine best policy to meet milestone Select near-term milestone YES Is milestone achievable with current approach? Relax milestone YES NO Implement policy RAND MR-1626-RPC

16. 16 4-30-08 Robust Strategy Reduces Uncertainty By Performing Well No Matter What Future Comes to Pass Adaptive Strategy Economic growth rate (%) 1.02.03.04.0 –1.0 0 1.0 5.0 3.0 4.0 2.0 0 No regret Mild A lot Overwhelming U.S. in 19th century U.S. since 1950 U.S. in 20th century Decoupling rate (%) Strategy Vulner- abilities Alternatives

17. 17 4-30-08 Outline Robust decision making (RDM) provides framework for effective long-term analysis RDM’s “Scenario Discovery” approach offers useful scenario concept for public debates Recent work measures the impacts of these approaches with decision makers Robust decision making (RDM) provides framework for effective long-term analysis RDM’s “Scenario Discovery” approach offers useful scenario concept for public debates Recent work measures the impacts of these approaches with decision makers

18. 18 4-30-08 Long-Term RDM Scenarios Highlight Trade-offs Among Near-Term Decisions 1.Run simulation model for many different combinations of uncertain input parameters 2.Identify those clusters of cases that highlight tradeoffs among near-term candidate strategies 1.Run simulation model for many different combinations of uncertain input parameters 2.Identify those clusters of cases that highlight tradeoffs among near-term candidate strategies Candidate strategy Identify vulnerabilities Assess alternatives for ameliorating vulnerabilities Example future conditions highlighting near-term tradeoffs: –2007 Congressional Reauthorization of Terrorism Risk Insurance Act: In what situations would ending TRIA cost the taxpayer more than retaining the program? –California water planning: Under what conditions would future climate change impacts suggest modifying current long-range water management plans? Example future conditions highlighting near-term tradeoffs: –2007 Congressional Reauthorization of Terrorism Risk Insurance Act: In what situations would ending TRIA cost the taxpayer more than retaining the program? –California water planning: Under what conditions would future climate change impacts suggest modifying current long-range water management plans?

19. 19 4-30-08 Scenario Discovery Implements This Concept for Computer-Assisted Scenario Development........... 1.Indicate policy-relevant cases in database of simulation results

20. 20 4-30-08 Scenario Discovery Implements This Concept for Computer-Assisted Scenario Development........... Uncertain input variable 2 1.Indicate policy-relevant cases in database of simulation results 2.Statistical analysis finds low- dimensional clusters with high density of these cases Uncertain input variable 1

21. 21 4-30-08 Scenario Discovery Implements This Concept for Computer-Assisted Scenario Development........... Uncertain input variable 2 1.Indicate policy-relevant cases in database of simulation results 2.Statistical analysis finds low- dimensional clusters with high density of these cases 3.Clusters represent scenarios and driving forces of interest to decision makers Uncertain input variable 1

22. 22 4-30-08 Scenario Discovery Implements This Concept for Computer-Assisted Scenario Development........... Uncertain input variable 2 1.Indicate policy-relevant cases in database of simulation results 2.Statistical analysis finds low- dimensional clusters with high density of these cases 3.Clusters represent scenarios and driving forces of interest to decision makers Uncertain input variable 1 Density: How many cases inside the scenario are policy-relevant? (e.g. 75%) Coverage: How many of all the policy- relevant cases do the scenarios include? (e.g. 82%) Interpretability: Is the number of scenarios and driving forces sufficiently small to understand? (e.g. 1 scenario with two driving forces) Density: How many cases inside the scenario are policy-relevant? (e.g. 75%) Coverage: How many of all the policy- relevant cases do the scenarios include? (e.g. 82%) Interpretability: Is the number of scenarios and driving forces sufficiently small to understand? (e.g. 1 scenario with two driving forces) Approach provides measures of merit for scenario quality

23. 23 4-30-08 Scenario Discovery May Improve Impact of Scenarios in Contentious Public Debates For instance, recent scenario discovery work on U.S. Federal terrorism insurance program was –Based on a scenario not considered in the official budgetary analysis by government agencies –Quoted on the floor of the United States Senate by a program supporter –Criticized as “insidious” by program opponents But neither side in the debate could gain traction by quarrelling with our choice of scenario and its key driving forces For instance, recent scenario discovery work on U.S. Federal terrorism insurance program was –Based on a scenario not considered in the official budgetary analysis by government agencies –Quoted on the floor of the United States Senate by a program supporter –Criticized as “insidious” by program opponents But neither side in the debate could gain traction by quarrelling with our choice of scenario and its key driving forces

24. 24 4-30-08 Outline Robust decision making (RDM) provides framework for effective long-term analysis RDM’s “Scenario Discovery” approach offers useful scenario concept for public debates Recent work measures the impacts of these approaches with decision makers Robust decision making (RDM) provides framework for effective long-term analysis RDM’s “Scenario Discovery” approach offers useful scenario concept for public debates Recent work measures the impacts of these approaches with decision makers

25. 25 4-30-08 How Does Climate Change Affect California’s Inland Empire Utilities Agency (IEUA)? IEUA currently serves 800,000 people –May add 300,000 by 2025 Water presents a significant challenge IEUA currently serves 800,000 people –May add 300,000 by 2025 Water presents a significant challenge

26. 26 4-30-08 How Does Climate Change Affect California’s Inland Empire Utilities Agency (IEUA)? IEUA currently serves 800,000 people –May add 300,000 by 2025 Water presents a significant challenge IEUA currently serves 800,000 people –May add 300,000 by 2025 Water presents a significant challenge IEUA’s 2005 long-range Urban Water Management Plan (UWMP) –Aims to meet needs of growing population, but –Did not address climate change

27. 27 4-30-08 How Does Climate Change Affect California’s Inland Empire Utilities Agency (IEUA)? IEUA currently serves 800,000 people –May add 300,000 by 2025 Water presents a significant challenge IEUA currently serves 800,000 people –May add 300,000 by 2025 Water presents a significant challenge We conducted several analyses to help IEUA assess impact of climate change on their 2005 UWMP –Traditional scenarios –Probabilistic risk analysis –Scenario Discovery

28. 28 4-30-08 Conducted Workshops to Measure Impact of Alternative Analyses on IEUA –Four IEUA workshops presented modeling results to participants including: Agency professional managers and technical staff Local elected officials Community stakeholders –“ Real-time” surveys measured participants’ Understanding of concepts Willingness to adjust policy choices based on information presented Views on RDM, traditional scenarios, and probabilistic risk analysis –Four IEUA workshops presented modeling results to participants including: Agency professional managers and technical staff Local elected officials Community stakeholders –“ Real-time” surveys measured participants’ Understanding of concepts Willingness to adjust policy choices based on information presented Views on RDM, traditional scenarios, and probabilistic risk analysis

29. 29 4-30-08 Disagree strongly Agree somewhat Agree strongly Agree somewhat Participants Ranked Scenario Discovery More Useful, But More Difficult to Understand Is easy to explain to decisionmakers Provides information on how to improve plan Provides results that can be used in planning Scenario Discovery Traditional Scenarios Questionnaire item from first 3 workshops –Traditional scenarios Gave IEUA much of the information they needed Emphasized the importance of achieving goals in IEUA’s plan –Scenario discovery Provided more useful information for evaluating alternatives to plan Sparked discussion of adaptive strategies

30. 30 4-30-08 Surveys Suggest RDM Analysis Changed Participants’ Views Participants provided: –Information on most effective RDM visualizations After the workshop: –35% said consequences of bad climate change now appeared “more serious” than before –75% though the ability of IEUA planner to plan for and manage effects was “greater” than before Overall, analysis: –Increased support for near-term modifications to current IEUA plan –Suggests that participants’ willingness to acknowledge a serious climate change threat increased after they felt more confident they could address the threat Participants provided: –Information on most effective RDM visualizations After the workshop: –35% said consequences of bad climate change now appeared “more serious” than before –75% though the ability of IEUA planner to plan for and manage effects was “greater” than before Overall, analysis: –Increased support for near-term modifications to current IEUA plan –Suggests that participants’ willingness to acknowledge a serious climate change threat increased after they felt more confident they could address the threat

31. 31 4-30-08 Key Concepts Choose scenarios to highlight tradeoffs among near-term decisions Otherwise number of potentially interesting scenarios remains unlimited Use analytics to facilitate human creativity in designing policies robust across many futures Measure scenarios’ impacts on decision makers to help improve process and methods Designing measurements makes purpose clear Can use framework for general thinking about long-term policy under deep uncertainty Not just as basis for a modeling exercise Choose scenarios to highlight tradeoffs among near-term decisions Otherwise number of potentially interesting scenarios remains unlimited Use analytics to facilitate human creativity in designing policies robust across many futures Measure scenarios’ impacts on decision makers to help improve process and methods Designing measurements makes purpose clear Can use framework for general thinking about long-term policy under deep uncertainty Not just as basis for a modeling exercise

32. 32 4-30-08 For More Information http://www.rand.org/international_programs/pardee/ Thank you!