1. The Oregon Health Insurance Experiment
Sarah Taubman, MIT and UW
February 11, 2015

2. Background on Medicaid
Medicaid is the public health insurance program for the low-income individuals
Federal program run by the states
Some flexibility in setting eligibility criteria
Historically coverage requires low-income and low assets AND additional categorical requirements (pregnant, disabled, etc.)
2014 ACA Medicaid expansions extend coverage to all low-income individuals (<138% of FPL) in participating states

3. Previous literature and approaches
Observational
Many, many studies comparing the those with insurance (public or private) to those without
Fundamental problem of confounding or selection
Quasi-Experimental
Diff-in-diff approaches using policy changes
Regression discontinuity approaches using cut-offs
Experimental
Considered the “gold standard” for causal inference
Very rarely used in social and health policy

4. The Experiment Oregon’s Medicaid expansion program
Low income adults, not categorically eligible
Budget to enroll some by not all of eligible population
Slots allocated by lottery
Opened waitlist in early 2008 (~80,000 signed-up)
Randomly selected ~30,000 names from the waitlist
~10,000 successfully enrolled in Medicaid
Evaluating the effects of public health insurance on health care use, health, financial risk exposure using the lottery as a randomized control trial

5. The Experiment Oregon’s Medicaid expansion program
Low income adults, not categorically eligible
Budget to enroll some by not all of eligible population
Slots allocated by lottery
Opened waitlist in early 2008 (~80,000 signed-up)
Randomly selected ~30,000 names from the waitlist
~10,000 successfully enrolled in Medicaid
Evaluating the effects of public health insurance on health care use, health, financial risk exposure using the lottery as a randomized control trial

6. health care use, financial
The Experiment
Lottery Selection
Outcomes:
health care use, financial
strain, health, others
Medicaid Coverage
Confounders:
demographics, income
and employment, health
status, access, others

7. Imperfect Take-up and Cross-over
Intent-to-treat analysis
Comparing all randomized in with all randomized out
Assume no differential loss-to-follow-up
Random treatment assignment
 Causal effect of randomization (but not of treatment per se)
As-treated or Per-protocol analysis
Comparing all treated with all not treated
Observational estimate (not based on randomization)
Are those the only options?

8. Instrumental Variable Analysis
Using randomization as an instrument for treatment
Assuming no direct effect of randomization; no defiers
Random treatment assignment
 Causal effect of treatment in compliers
Lottery Selection
Outcomes:
health care use, financial
strain, health, others
Insurance Coverage
Confounders:
demographics, income
and employment, health
status, access, others

9. Analytic Approach Intent to treat effect of lottery selection
Comparing all selected with all not selected
Random treatment assignment
No differential selection for outcome measurement
Local average treatment effect of Medicaid
Using lottery selection as an instrument for coverage
~25 percentage point increase in Medicaid enrollment
No effect of lottery except via insurance coverage

10. Archived Analysis Plans
Pre-specified analysis plans
Used control distributions to understand data
Coded analysis using scrambled data
Archived plan with J-PAL and NBER
Did plans for evidence from the first year, in-person interviews, labor force activity, emergency department data (all available at

11. Data State administrative records on Medicaid enrollment
Administrative data on outcomes (through Sept 2009)
Hospital discharge data, emergency department data, credit report data, mortality data
Mail survey data on outcomes
Main survey fielding summer and fall of 2009
Effective response rate: 50%
In-person interview data on outcomes

12. In-person data collection
Questionnaire and health examination including
Survey questions
Anthropometric and blood pressure measurement
Dried blood spot collection
Catalog of all medications
Fielded between September 2009 and December 2010
Average response ~25 months after lottery began
Limited to Portland area: 20,745-person sample
12,229 interviews for effective response rate of 73%

13. Sample and subsamples
Sample is all individuals who signed-up for the lottery
74,922 individuals (66,385 households)
Different subsamples for some specific analyses:
74,922 for hospital and mortality data analysis
61,790 matched to SSA data
49,980 matched to credit report data
24,464 in Portland-area for ED analysis
23,922 mail survey respondents
12,229 in-person interview respondents

14. Sample characteristics
Ages 19-64; average age: 41
56% women
82% white; 4% black; 12% Hispanic
Average household income (2008): $13,053
Report poor health
All samples balanced on pre-randomization measures (demographics and administrative data)

15. Results Health care use Financial strain Health
Hospital discharge data
Emergency department data
Surveys
Financial strain
Credit reports
Health
Mortality from vital stats (low power – none observed)
Surveys and specific conditions

22. Health Care Use Results
Hospitalizations (discharge data, ~1yr)
30%↑probability of hospital admission – concentrated in non-ED-origin
Substantial (but imprecise) increase in total resource use
Emergency department use (administrative data, ~1yr)
20%↑probability of ED visit; 40%↑number of ED visits
Concentrated in outpatient visits; no types with significant decline
Other use (self-reports, ~1yr and ~2yrs)
Increases in probability & # of outpatient visits and Rx drugs
Increases preventive care, including mammograms, cholesterol screening
Increase in probability of having usual source of care; quality
Implied 25-35% increase in spending for insured

23. Results Health care use Financial strain Health
Hospital discharge data
Emergency department data
Surveys
Financial strain
Credit reports
Health
Mortality from vital stats (low power – none observed)
Surveys and specific conditions

27. Financial Hardship Results
Reduction in collections (credit reports)
25% ↓ probability of unpaid medical bills sent to collection
No observed effects on other measures
Reduction in strain, OOP, money owed (self-reports)
Substantial reduction across measures, including elimination of catastrophic OOP health spending
Implications for distribution of burden/benefits
Some borne by patients, some by providers (or those to whom passed through) – only 2% of bills sent to collection ever paid

28. Results Health care use Financial strain Health
Hospital discharge data
Emergency department data
Surveys
Financial strain
Credit reports
Health
Mortality from vital stats (low power – none observed)
Surveys and specific conditions

30. Focusing on specific conditions
Measured:
blood pressure
cholesterol levels
glycated hemoglobin
depression
Reasons for selecting these:
Reasonably prevalent conditions
Can be measured by trained interviewers and lab tests
Clinically effective medications exist
Markers of longer term risk of cardiovascular disease
A limited window into health status

35. Results on specific conditions
Large reductions in depression
Increases in diagnosis and medication
In-person estimate of -9 percentage points in being depressed
Glycated hemoglobin
No significant effects on HbA1c; wide confidence intervals
Blood pressure and cholesterol
No significant effects on diagnosis or medication
No significant effects on outcomes
Framingham risk score
No significant effect (in general or sub-populations)

36. Additional Results Employment and earnings Program participation
SSA data
Surveys
Program participation
SSA data (SSI/SSDI)
State administrative data (TANF and SNAP)

39. Summary One to two years after expanded access to Medicaid:
Increases in health care use (and associated costs)
Increases in compliance with recommended preventive care
Improvements in quality and access
Reductions in financial strain
Improvements in self-reported health
Improvements in depression
No significant change in specific physical measures
Sense of the relative magnitude of the effects
Use and access, financial benefits, general health, depression
Physical measures of specific chronic conditions

40. Extrapolation to ACA Expansion
Context quite relevant for health care reform:
Similar population to 2014 Medicaid expansions
But important caveats to bear in mind:
Oregon and Portland vs. US generally
Voluntary enrollment vs. mandate
Partial vs. general equilibrium effects
Short run (1-2 years) vs. medium or long run

41. Pickert, Kate. “5 Things the Oregon Medicaid Study Tells Us About American Health Care.” Time Magazine. May 2, 2013.
Conover, Chris. “Does The Oregon Health Study Show That People Are Better Off With Only Catastrophic Coverage?” Forbes. May 7, 2013.
Sargent, Greg. “A War Over Medicaid.” Washington Post. May 2, 2013.
Bennett, Dashiell. “How to Use the Oregon Medicaid Study to Your Ideological Advantage.” The Atlantic. May 2, 2013.
Lowrey, Annie. “Medicaid Access Increases Use of Care, Study Finds.” New York Times. May 1, 2013.
Rubin, Jennifer. “Spending on Medicaid Doesn’t Actually Help the Poor.” Washington Post. May 2, 2013.
Neyfakh, Leon. “Is Health Insurance an Antidepressant?” Boston Globe. June 23, 2013.
Barro, Josh. “Yes, the Oregon Health Study Matters.” Bloomberg. May 3, 2013.
Others:
- "Four Reasons Why the Oregon Medicaid Results Are Even Worse Than They Look" (Forbes)
- "Medicaid has mixed record on improving health for poor, study says" (LA Times)
- "Second Thoughts On Medicaid From Oregon's Unique Experience" (NPR)
- "The Oregon Medicaid Experiment Changes Nothing. Here's The Experiment We Really Need." (Slate)
“Oregon’s lesson to the nation: Medicaid works” – Blue Oregon
“Here’s what the Oregon Medicaid study really said” – Klein, Wash Post

42. Updating based on our findings
“Medicaid is worthless or worse than no insurance”
We see increases in utilization and perceived access and quality, reductions in financial strain, improvement in self-reported health, improvement in depression, and can reject large declines in several physical measures
“Health insurance expansion saves money”
In short run we see increases in utilization and cost, including hospital use, emergency department use, primary and preventive care use and prescription drugs

43. Always an Adventure