1. Research and Public Policy Evans School of Public Affairs April 30, 2013 Annie Pennucci Associate Director Washington State Institute for Public Policy www.wsipp.wa.gov

2. Upcoming Examples The Research Background Washington State Institute for Public Policy Created by the 1983 Legislature Mission: carry out non–partisan research on projects assigned by the Legislature or the Institute’s Board of Directors Senator Mike Carrell Senator Karen Fraser Senator Jeanne Kohl-Welles Representative Cary Condotta House Republican Vacancy House Democratic Vacancy Ken Conte, House Staff Richard Rodger, Senate Staff David Schumacher, OFM Director Gubernatorial Appointee Vacancy Sandra Archibald, Univ. of WA Les Purce, The Evergreen State Col. Robert Rosenman, WA State Univ. Rodolfo Arévalo, Eastern WA Univ. Board of Directors Senator Mark Schoesler, Co-Chair House Democratic Vacancy, Co-Chair Slide 2 of 22

3. Upcoming Examples The Research Background WSIPP Capitol Washington State Institute for Public Policy Slide 3 of 22

4. Upcoming Examples The Research Background 1983 legislative intent Apply expertise of academia to needs of state policymakers Connect The Evergreen State College (based in Olympia) with state government Goal: better informed policy making Washington State Institute for Public Policy Slide 4 of 22

5. Upcoming Examples The Research Background Current role Applied research Legislature assigns and funds most studies Reports are easy to understand and accessible to policymakers Non-partisan staff Washington State Institute for Public Policy Slide 5 of 22

6. Upcoming Examples Background The Research Three Types of Legislative Assignments to WSIPP 1. What Works? Benefit-Cost Analysis (research reviews) 2. Outcome evaluations of specific Washington programs (primary research) 3. Miscellaneous Are there evidence-based policy options that improve public outcomes, at less cost? Slide 6 of 22

7. Upcoming Examples Background The Research We systematically analyze all, credible evaluations of real world ways to improve key public outcomes. 1a. What Works? What Doesn’t? Crime Education, Early Ed. Child Abuse & Neglect Substance Abuse Mental Health Developmental Disabilities Teen Births Employment Public Assistance Public Health Housing What does the weight of the evidence tell us? Slide 7 of 22

8. Upcoming Examples Background The Research 1b. What’s Cost-Beneficial? For programs and policies determined to “work” How much does it cost? What are the long-term benefits in monetary/fiscal terms? Which are the best investments from the state’s perspective? What is the risk of success/failure? Slide 8 of 22

9. Upcoming Examples Background The Research 2. Outcome Evaluations Do Washington programs & policies achieve their intended effects? Comparison group research Administrative and survey data Outcomes-focused Slide 9 of 22

10. Upcoming Examples Background The Research 3. Miscellaneous Often qualitative (e.g., describe a program or process or survey stakeholders regarding their views) Review other states’ policies in comparison with Washington’s Key Institute role = independence, objectivity “Other duties as assigned” Slide 10 of 22

11. Upcoming The Research Background Examples Slide 11 of 22 Example #1: How Does Class Size Impact Student Outcomes? Report to the 2013 Legislature Test scores and high school graduation Literature review (meta-analysis) Supplemented by WSIPP analysis of state-level data Cost estimation Risk analysis

12. Upcoming The Research Background Examples Slide 12 of 22 How to find studies: Keywords in combination (class size, evaluation, K-12, outcomes, public schools, effective, and so on) Databases (ProQuest and other academic journal sources, Google Scholar) Previously published reviews/meta-analyses. Comb reference lists. Organization databases (e.g., the Campbell Collaboration, What Works Clearinghouse)Campbell Collaboration What Works Clearinghouse Literature Search

13. Upcoming The Research Background Examples Slide 13 of 22 Literature Review Screening studies: Is the study an evaluation or descriptive? Does the evaluation address the research question? Is there a valid comparison group? What outcomes are measured?

14. Upcoming The Research Background Examples Slide 14 of 22 An effect size (or in the case of class size, an “elasticity”) Represents the difference in outcomes between the experimental and comparison groups Tells us how much change we might expect in an outcome if a program or policy is implemented Must be interpreted within the context of the policy environment Meta-analysis

15. Upcoming The Research Background Examples Slide 15 of 22 *The measured outcomes include student test score gains, high school graduation, and dropout rates. The chart plots the 77 credible effects we found in the research literature. Our summary line is a weighted average. How a 10% Decrease in Class Size Affects Student Outcomes*

16. Upcoming The Research Background Examples Slide 16 of 22 Per-Student Benefits and Costs of Reducing Class Size by One Student Grade CostsAvg. Benefits Avg. Net Value K$198$2,302$2,104 1$198$1,218$1,021 2$198$725$528 3$198$578$381 4$179$422$243 5$179$366$187 6$179$347$168 7$162$358$196 8$162$336$175 9$160$306$146 10$160$301$141 11$160$378$218 12$160$353$193

17. Upcoming The Research Background Examples Slide 17 of 22 Chance that benefits outweigh costs 0% 50% 100% K123456789101112 Grade Where Class Size Is Reduced 1 st grade: 94% Close to 50% Economics & Risk: Reducing Class Size by One Student

18. Upcoming The Research Background Examples Slide 18 of 22 Benefits Per Individual Crime reduction $5,485 Lower CJ / victim costs Main Source of Benefits Test scores & graduation rates $13,266 Increased earnings Special education reduction $1,081 Lower K-12 costs Grade repetition reduction $307 Lower CW / victim costs Fewer out-of-home placements $557 Lower health care system costs $756 Lower Medicaid enroll. Total Benefits Per Individual $22,693 Cost Per Individual Benefits Per Dollar of Cost $7,489 $3.06 Nearly 100% chance of benefits exceeding costs Less child abuse & neglect $1,240 Lower CW / victim costs Example #2: Early Childhood Education (low income) 2011 dollars

19. Upcoming The Research Background Examples Slide 19 of 22 WSIPP “Consumer Reports” Lists Evidence-based policy options ranked by return on investment

20. Example The Research Background Upcoming Slide 20 of 22 Current Legislative Proposals In the 2013 session: Reduce class size in K-3 (or K-1) Expand early childhood education for low-income children by 10% Assign WSIPP additional reviews to identify evidence- based options for K-12 education funding increases

21. Example The Research Background Upcoming Slide 21 of 22 Current WSIPP Projects Innovative K-12 Public Schools in Washington State State Need Grant (Student Outcomes) Outcomes of Juveniles Tried in Adult Courts I-502 Evaluation: Costs and Benefits of the Legalization of Cannabis in Washington State Impacts on Crime from Different Approaches to Policing

22. Example The Research Background Upcoming Slide 22 of 22 Potential* Upcoming WSIPP Projects How can the state control Medicaid costs? Which components of early childhood education programs are evidence-based? Develop a risk assessment tool for individuals involuntarily committed for mental illness Compile an inventory of evidence-based approaches to substance abuse intervention *Depends on 2013 legislative session