Presentation on theme: "Asking better questions to build better models Sharon Dawes."— Presentation transcript:
Asking better questions to build better models Sharon Dawes
Some policy questions Should we eradicate polio? How can schools help third graders become good readers? What kinds of public investments lead to urban vitality? What is the optimal approach to immigration? How does health insurance coverage affect health care costs? Can a ban on large sugary drinks reduce obesity and diabetes? Does public posting of sanitary inspections lead to cleaner restaurants?...
All models are wrong, some are useful. Box, 1979
Why models are wrong Complexity Boundaries Dynamics Data Time
Boundaries Organizations, jurisdictions, sectors Groups, individuals and their interests Existing policies and laws Contextual factors Processes Practices Geography Behaviors What goes in the model, what is left out, and why and how do the choices matter?
Dynamics Feedback structure of URBAN1, Ghaffarzadegan, et al.
The “best available data” (BAD) is inherently flawed Knowledge problems Meaning Explicitness Codifiability Embeddedness Dynamics Data problems Accuracy Completeness Bias Timeliness Availability...
Time Problem definition Goal setting Policy formulation Strategy formulation Design Development Deployment Implementation Operations Performance assessment Evaluation
Policy modeling over time Problem definition Goal setting Policy formulation Strategy formulation Design Development Deployment Implementation Operations Performance assessment Evaluation Understanding the problem and the options Exploring alternative approaches Understanding the policy system Evaluating and interpreting policy impacts
Stakeholders bring Different knowledge and points of view Different goals and preferences Different capabilities and authority to act Different cultures (national, organizational, professional) Different amounts and kinds of power and autonomy Different levels of confidence and trust Different amounts of tolerance for risk
An example: eradication of polio with thanks to Dr. Kimberly Thompson & colleagues
The collaborators gratefully acknowledge financial support in the form of unrestricted gifts to the Harvard Kids Risk Project and grants from the US Centers for Disease Control and Prevention (CDC): U50/CCU300860, U01 IP000029, NVPO N37 (FY2005), M-33379, M-33679, M-35172, U66 IP000169, the World Health Organization (WHO) APW , and the Bill & Melinda Gates Foundation: , , I thank Radboud Duintjer Tebbens for embarking and continuing on the entire journey with me. I also thank collaborators from the CDC, including James Alexander, Lorraine Alexander, Larry Anderson, Gregory Armstrong, Albert Barskey, Brenton Burkholder, Cara Burns, Victor Cáceres, Jason Cecil, Susan Chu, Steve Cochi, Kathleen Gallagher, Howard Gary, John Glasser, Steve Hadler, Karen Hennessey, Hamid Jafari, Julie Jenks, Denise Johnson, Bob Keegan, Olen Kew, Nino Khetsuriani, Robb Linkins, Naile Malakmadze, Rebecca Martin, Eric Mast, Steve McLaughlin, Steve Oberste, Mark Pallansch, Becky Prevots, Hardeep Sandhu, Nalinee Sangrujee, Jean Smith, Philip Smith, Peter Strebel, Linda Venczel, Gregory Wallace, Steve Wassilak, Margie Watkins, and Bruce Weniger, and from the WHO, including Bruce Aylward, Fred Caillette, Claire Chauvin, Philippe Duclos, Esther deGourville, Hans Everts, Marta Gacic-Dobo, Tracey Goodman, Ulla Griffiths, David Heymann, Scott Lambert, Asta Lim, Jennifer Linkins, Patrick Lydon, Chris Maher, Linda Muller, Roland Sutter, Rudi Tangermann, Chris Wolff, and David Wood. I also thank the Global Polio Laboratory Network, Harrie van der Avoort, Francois Bompart, Anthony Burton, Konstantin Chumakov, Laurent Coudeville, Walter Dowdle, Paul Fine, Michael Galway, Shanelle Hall, Neal Halsey, Tapani Hovi, Kun Hu, Dominika Kalkowska, Samuel Katz, Jong-Hoon Kim, Tracy Lieu, Marc Lipsitch, Anton van Loon, Apoorva Mallya, Phil Minor, John Modlin, Van Hung Nguyen, Walter Orenstein, Carol Pandak, Peter Patriarca, Christina Pedreira, Stanley Plotkin, Hazhir Rahmandad, Robert Scott, John Sever, Thomas Sorensen, John Sterman, Robert Weibel, Jay Wenger, and Peter Wright. Collaborators
Thompson, K. M. (2012). The role of risk analysis in polio eradication: Modeling possibilities, probabilities, and outcomes to inform choices. Expert Review of Vaccines, 11(1), 5-7. Thompson, K. M. (2006). Poliomyelitis and the role of risk analysis in global infectious disease policy and management. Risk Analysis, 26(6), Thompson, K. M., & Duintjer Tebbens, R. J. (2012). Current polio global eradication and control policy options: Perspectives from modeling and prerequisites for OPV cessation. Expert Review of Vaccines, 11(4), Thompson, K. M., & Duintjer Tebbens, R. J. (2011). Challenges related to the economic evaluation of the direct and indirect benefits and the costs of disease elimination and eradication efforts. In S. L. Cochi & W. R. Dowdle (Eds.), Disease Eradication in the 21st Century: Implications for Global Health. Cambridge, MA: MIT Press. Thompson, K. M., & Duintjer Tebbens, R. J. (2008a). The case for cooperation in managing and maintaining the end of poliomyelitis: Stockpile needs and coordinated OPV cessation. The Medscape Journal of Medicine, 10(8),190. Retrieved from Thompson, K. M., & Duintjer Tebbens, R. J. (2008b). Using system dynamics to develop policies that matter: Global management of poliomyelitis and beyond. System Dynamics Review, 24(4), Thompson, K. M., & Duintjer Tebbens, R. J. (2007). Eradication versus control for poliomyelitis: An economic analysis. The Lancet, 369(9570), Thompson, K. M., & Duintjer Tebbens, R. J. (2006). Retrospective cost-effectiveness analyses for polio vaccination in the United States. Risk Analysis, 26(6), Selected academic results
Plain language communication of results
Some suggestions Build relationships with policy makers and policy advisors Treat models as learning objects to be improved by interaction with the people who own the policy problem. Adopt techniques that engage stakeholders group model building, participatory design, value-sensitive design Learn and speak the language of practice Custom-make presentations for policy makers and managers Consider multiple methods to understand context and complexity case studies, test beds, evaluation studies Update, iterate, adjust, repeat
Many questions, many answers, better models Policy makers Other analysts Policy stakeholders Policy modelers The big policy question ÷ Elements of complexity Boundary questions Data challenges Dynamic interactions Time considerations X Multiple iterations Plain language = Insights, better understanding of policy options and effects
To continue the discussion eGovPoliNet project, Policy Making 2.0 community on Linked In PIN-l listserv PIN website: pin.asu.edu Handbook of Policy Informatics (forthcoming, MIT Press) International Digital Government Research Conference (dgo) Policy Informatics Panel, June 17-20, Quebec City International Research Society for Public Mgmt (IRSPM) Policy Informatics Track, April 10-12, Prague Twitter hashtag: #policyinformatics