Less Pain, More Gain: An Evidence-Based Approach to Long-term Deficit Reduction Jon Baron Coalition for Evidence-Based Policy March 2013

Coalition for Evidence-Based Policy A nonprofit, nonpartisan organization. Mission: To increase government effectiveness through rigorous evidence about “what works.” Coalition has no affiliation with any programs or program models – thus serves as an objective, independent resource on evidence-based programs. Funded independently, by national philanthropic foundations (e.g., MacArthur, William T. Grant).

Program reforms shown to produce major savings without loss of quality or benefit: Successful welfare-to-work programs (Riverside, L.A., Portland) Evaluated in large, convincing randomized trials Increased employment and earnings by 20-50% Produced net taxpayer savings of $1700-$6000/person Had important policy impact (1996 welfare reform act)

Cost-saving program reforms, continued: Transitional Care Model (a nurse-led hospital discharge and home follow-up program) Evaluated in two well-conducted randomized trials Reduced rehospitalizations 30-50% Produced net savings of $4000/patient Successful national replication could save $10 billion/year

Cost-saving program reforms, continued: Subsidized Guardianship as alternative to foster care Evaluated in large randomized trial in Illinois Increased permanent placements by 8% Reduced days in foster care by 16% Saved Illinois $2300/child ($90 million over 10 years) Expanded nationally (2008), CBO scored $800M savings

Too few examples currently exist, because: Rigorous evaluations are still relatively uncommon in most program areas. Promising ideas, when rigorously tested, too often are found not to work.

Promising ideas, when rigorously tested, too often are found not to work Medicare Coordinated Care Demonstration (MCCD) Preliminary studies had suggested large cost savings (e.g., 25%) However, the more definitive, randomized MCCD found a net cost increase of 11%.

Other examples: In medicine: 50-80% of interventions found promising in phase II (nonrandomized studies or small efficacy trials) are found ineffective in phase III (sizable RCTs). In business: Google and Microsoft have conducted ~13,000 RCTs of new products/strategies, 80-90% of which found no significant effect (Manzi). In K-12 education: Of 120 interventions evaluated in large, convincing RCTs funded by Institute of Education Sciences , ~90% had weak or no effects.

Proposed strategy for long-term deficit reduction: “If you want to increase your success, double your failure rate.” - Thomas Watson, IBM Accelerate development of promising, innovative program reforms with potential for cost savings. Greatly increase the number that are rigorously evaluated.

How to do this? Low-cost rigorous evaluations make it possible to test hundreds of program models/strategies each year, rather than a select few (as is currently done). Federal “waivers” from law/regulation can be used to stimulate state and local innovation and evidence- building aimed at entitlement savings. New high-visibility “prize competition” seeks to jump- start low-cost trials and testing of cost-saving reforms.

Coalition for Evidence-Based Policy Jon Baron

Examples: Medicare Health Support Pilot (disease management) Preliminary studies suggested a high ROI (3:1 to 7:1) However, large randomized MHS found a net cost increase of 4-11%.

OMB & CBO Budget Scoring Creates New Incentive for Development/Expansion of Evidence-Based Programs OMB will credit entitlement savings a program generates, that have been demonstrated in “rigorous experimental research.” (2010 guidance) CBO stated that it “gives greater weight to demonstrations and experiments that use random assignment …” when scoring the cost of legislation. (2011)

Comprehensive Child Development Program: Impact on Mothers’ Employment Rate

Impact on Family Welfare Receipt

Impact on Percent of Children “At Risk” in Cognitive Development & Behavior Treatment

Impact on Percent of Children “At Risk” in Cognitive Development & Behavior TreatmentControl

Career Academy Students Non-randomized Comparison Group* *The comparison group consists of similar students in similar schools nationwide. Their estimated rates of postsecondary completion are statistically adjusted to control for observed differences between their background characteristics and those of the Career Academy group. Randomized Trial Results Comparison Group Study Results* Impact of Career Academies on Completion of a Postsecondary Credential Source: Data provided by James Kemple, MDRC Inc. Career Academy Students Randomized Control Group

Websites that identify evidence-based programs Top Tier Evidence Social Programs that Work Blueprints for Violence Prevention What Works Clearinghouse (K-12 Education) Best Evidence Encyclopedia (K-12 Education)