1. Using survey design to reduce undercounts of public health insurance coverage
Brian Robertson, PhD, VP Research
Mark Noyes, MPH, Research Analyst

2. Acknowledgement HealthSource RI & Freedman Healthcare
South Dakota Department of Social Services
Oregon Health Authority
The Vermont Department of Health & Vermont Department of Financial Regulation Insurance Division

3. Subtitle Survey Background The Undercount Problem Hypothesis Approach
Results
Conclusions
Subtitle

4. Survey Background

5. State Health Insurance Surveys
Data presented from Oregon, Rhode Island, South Dakota and Vermont
Full studies other states: Georgia, Pennsylvania, Maine
Smaller studies in: Colorado, Louisiana, New York, Wisconsin
Topics include health insurance, plan characteristics, barriers to care and health care usage
Data ranges from 2014 – 2017
MDR has been conducting health insurance surveys since 2001
113,000 surveys gathering data on 284,000 people
All surveys done via telephone with a live interviewer

6. The Four States State Year Surveys+ Individuals Oregon 2017 9,000
21,000
Rhode Island
2016
5,000
10,600
South Dakota
2015
2,500
5,500
Vermont
2014
4,000
8,900

7. The Undercount Problem

8. Undercounts Defined
Undercounts of Medicaid populations are well-known and persistent when measuring health insurance.
22.9% - Scope of undercount in a recent study comparing American Community Survey results to administrative Medicaid records (Turner, 2014)
Many surveys rely on the traditional question design to obtain information about insurance coverage.
Surveys that use such a design include ACS, BRFSS, NHIS, and SIPP.

9. American Community Survey - Background
The American Community Survey
A national survey of more than 2.2 million households
Topics include housing characteristics, employment, health insurance, and detailed demographic information
Gathering information about health insurance coverage is not the primary focus of the American Community Survey
Still used as one of the most important national metrics of health insurance coverage

10. The Traditional Insurance Question Design
The American Community Survey Questions:
Is this person CURRENTLY covered by any of the following types of health insurance or health coverage plans? Mark "Yes" or "No" for EACH type of coverage :
Insurance through a current or former employer or union (of this person or another family member)
Insurance purchased directly from an insurance company (by this person or another family member)
Medicare, for people 65 and older, or people with certain disabilities
Medicaid, Medical Assistance, or any kind of government-assistance plan for those with low incomes or a disability
TRICARE or other military health care
VA (including those who have ever used or enrolled for VA health care)
Indian Health Service
Any other type of health insurance or health coverage plan – Specify

11. The Problems with the Traditional Design
The traditional design may not include health coverage options available in a specific state
For example, state specific Medicaid programs which are more recognizable to individuals taking the survey
Given the complexity of the health insurance landscape, the enrollment process, and other factors, a person may be unsure of the actual type of coverage
For example, residents who enroll through a Health Exchange may have private health insurance but they may also have coverage through Medicaid

12. Factors Survey Design Should Consider
Medicaid Program Name:
Residents identify coverage by the name of the state Medicaid program
Residents still have coverage through legacy Medicaid programs
Specific Medicaid Plan Names:
Residents identify their coverage by names given to state Medicaid programs for specific groups of people
Residents identify coverage by the name of the managed care or coordinated care organization through which they receive care

13. Factors Survey Design Should Consider, cont.
Medicaid Program Administrator:
In cases where Medicaid is administered through private carriers, some residents report their coverage as private
In cases where Medicaid offer premium assistance option to purchase ESI residents consider their coverage as private
Enrollment Method:
If enrolled through the federal or state health exchange residents may identify their coverage by the name of the state, simply as the Exchange or as “Obamacare.”
Government Insurance:
Some residents identify their insurance simply as government insurance or with a non-insurance government service like SSI or IHS.

14. Hypothesis

15. Those in green can be addressed by modifying the traditional design
Our Hypothesis
The majority of the Medicaid undercount in surveys arises from limitations of the traditional design.
Combating social stigma is less important than providing respondents clear choices that they understand.
Modifying the traditional design will more accurately capture Medicaid enrollment. This includes:
Modifying or adding response categories to the traditional design
Adding in follow-up questions in cases where a person may be confused about the type of health insurance coverage
Those in green can be addressed by modifying the traditional design
Those is red require follow-up questions to verify the type of coverage

16. Challenges Identifying Medicaid Coverage – Examples
Factor
Oregon
Rhode Island
South Dakota
Vermont
Medicaid program name
Oregon Health Plan
RIte Care
Medicaid
Green Mountain Care
Medicaid plan name, managed care plan
Healthy Kids (Medicaid program for children)
 Rhody Health Partners, Connect Care Choice
Dr. Dynasaur (Medicaid program for children)
MCO or CCO
Numerous
PC Plus
Medicaid program administration
State
Private insurance carriers
Enrollment method
State portal or the Federal Exchange
State Exchange: HealthSource RI
State Exchange: Vermont Health Connect
Type of government insurance 
"Government" insurance, through the government, through SSI/SSDI, welfare or disability
Confusing Medicaid with Medicare
Other factors
Premium assistance programs using Medicaid funds to purchase ESI
Medicaid often combined with Indian Health Services
Pre-ACA Medicaid program phased out but still identified

17. Approach

18. The MDR Approach to Minimizing Undercounts
Step 1: Modify the traditional question design
Step 2: Add follow-up questions where there may be confusion as to the type of coverage
Step 3: Add follow-up questions to check key characteristics that distinguish Medicaid from other insurance types.

19. Modify the traditional design.
Our Approach: Step 1
Modify the traditional design.
Expand the traditional design answer categories to include commonly used pseudonyms for Medicaid:
The state specific name(s) of Medicaid (Read)
The name of the Medicaid program administrator [Unread option],
The name of the Medicaid managed care organization [Unread option],
A category for the state specific Exchange, “The Exchange,” “Obamacare” [Unread option], and
A category for “through the government,” “government insurance,” “Disability,” “SSI” [Unread option].

20. Modified tradition design: an example from Rhode Island
Is PERSON covered by any type of health insurance?
IF YES ASK: Which of the following types of insurance is this person covered by?
Private health insurance (Employer based or through a company like Blue Cross)
Medicare
RIte Care (name of the state Medicaid Program)
Rhody Health Partners (Medicaid managed care program)
Medicaid or Rhode Island Medical Assistance
Military, Veterans, or TRICARE
Some other type of insurance? (SPECIFY)
UNREAD OPTIONS
CONNECTCARE CHOICE (Medicaid managed care program)
RITE SHARE (Medicaid premium assistance program)
HEALTH SOURCE RI (The name of the state health exchange)
THOUGH THE EXCHANGE, THE EXCHANGE, OBAMACARE
THROUGH THE STATE, GOVERNMENT, GOVERNMENT INSURANCE
SSI, SSDI, WELFARE, DISABILITY
INDIAN HEALTH SERVICES
NO INSURANCE COVERAGE

21. Our Approach: Step 2
Add follow-up questions where there may be confusion
to determine the type of coverage.
Residents indicating the have health insurance though the Exchange, provide the name of the state health Exchange as their source of coverage, or indicate they have “Obamacare,”
For those under age 65 indicating coverage through Medicare, verify that coverage is through Medicare and not Medicaid.
If resident mentions coverage through the government or government insurance.
If residents mentioned coverage through a non-insurance government service such as SSI, SSDI, disability, or welfare
If residents report only having care through Indian Health Services
For all follow-ups: Provide clarifying information to assist resident in identifying the type of coverage.

22. Follow-ups for those indicating coverage through the Exchange (Rhode Island):
HealthSource RI is a unique resource that connects Rhode Islanders to affordable healthcare coverage. It provides the tools, resources and information you need to stay informed and healthy. It also provides a way for residents to know whether they qualify for health insurance coverage through RIte Care, RIte Share, or through a private health insurance plan for which a monthly premium is paid.
Do you know if PERSON is enrolled in RIte Care, RIte Share or if PERSON is enrolled in a private health plan?
CLARIFYING INFORMATION:
The Medicaid program, also known as RIte Care gives the individual access to no-cost health coverage.
Those with RIte Care do not pay a monthly premium.
You would have received TWO health insurance cards - one from your health plan AND a medical assistance ID card. You would need to show both of these cards when receiving services.
RIte Share is Rhode Island's Premium Assistance Program that helps Rhode Island families afford health insurance through their employer by paying for some or all of the employee's cost.
Your insurance would be provided through your employer
You would have received a state Medicaid card along with your member ID card.
HealthSource RI currently offers 20 private health insurance plans for individuals and their families from three health insurance companies: Blue Cross & Blue Shield of Rhode Island, Neighborhood Health Plan of Rhode Island, and UnitedHealthcare.
UNLIKE RIte Care individuals or families enrolled in these health plans will pay a monthly premium.
The private health plans available through HealthSource RI are organized into four “metal” categories: Bronze, Silver, and Gold, and Platinum.

23. Our Approach: Step 3
Add follow-up questions to check key characteristics that
distinguish Medicaid from other insurance types.
The insurance provider or carrier in states where Medicaid is administered by private carriers (follow-up to determine if the insurance is provided through Medicaid),
The source of insurance coverage (identified as through the government, by the Medicaid plan name or plan administrator rather than through an employer or purchased directly)
If ESI is provided through a premium assistance program (in states offering these programs).

24. Follow-up questions asked of those with private insurance
Follow-ups: In Rhode Island, Medicaid is administered through private carriers.
Follow-up questions asked of those with private insurance
Is PERSON’s private health insurance provided through Blue Cross, United Healthcare, Neighborhood Health Plan, or some other company?
ASK OF THOSE WITH COVERAGE THROUGH UNITED HEALTH CARE OR NEIGHBORHOOD HEALTH PLAN:
Does PERSON have insurance through Rhode Island's RIte Care Program?
CLARIFYING INFORMATION
Those with RIte Care do not pay a monthly premium.
If you have coverage through RIte Care, you would have received TWO health insurance cards - one from your health plan AND a medical assistance ID card. You would need to show both of these cards when receiving services.

25. Follow-up questions asked of those with private insurance
Follow-ups: Rhode Island offers a premium assistance program through Medicaid
Follow-up questions asked of those with private insurance
Is PERSON's health insurance through person’s work, or a union, association or trust, someone else’s work, or a union, association or trust, or some other source?
ASK IF THEY ARE INSURED THROUGH AN EMPLOYER:
Is person receiving premium assistance from the state of Rhode Island’s RIte Share program to help pay the cost of PERSON's monthly premium?
CLARIFYING INFORMATION:
If you have coverage through RIte Share, you receive a check from the state of Rhode Island once a month to help pay your premium.
RIte Share is Rhode Island's Premium Assistance Program that helps Rhode Island families afford health insurance through their employer by paying for some or all of the employee's cost.

26. Results

27. Effectiveness: Modifications to Traditional Design

28. Our Results
Of all residents eventually classified as having coverage through Medicaid:
Overall, only 27.4%, on average were identified through “traditional design” questions
This ranges from only 2% in Oregon to 91.9% in South Dakota
By modifying the traditional design to include additional categories, we identified 89.7% of those classified with Medicaid coverage
The values ranged from 76.3% in Rhode Island to 95.2% in Oregon
However, these simple modifications to the traditional design did not identify all of those with Medicaid coverage.

29. Effectiveness: Follow-up identification

30. Conclusions

31. Conclusions
The traditional question design featured in many large scale, federal studies persistently underestimates the rate of individuals using means-tested health insurance programs.
This distorts both the rate at which people are reported to have private insurance and the rate at which people are reported to be uninsured.
Much of this undercount is due to confusion on the part of survey respondents.
We can use survey design to counteract this confusion and arrive at more accurate results.

32. Modifying the Traditional Design
Modifying the traditional design to reduce undercounts requires:
Modifying or adding response categories to items most likely to be used by respondents
Asking follow-up questions of individuals providing unclear responses
Verifying insurance type by asking follow-ups on key plan characteristics

33. Successes
Using the above methods, Market Decisions Research has consistently provided results with smaller undercounts than federal surveys
Oregon – 1.4%
South Dakota – 12%
Rhode Island – 8%

34. Reducing Undercounts
While survey design can reduce undercounts and provide more valid data, it cannot completely eliminate them.
Some level of social stigma does prevent some individuals from reporting use of public insurance programs
Some individuals are confused on a level that cannot be reduced with survey design
Proper weighting techniques can eliminate the effect of undercounts on final measurements
Reducing the size of the adjustments provides the most precise measurements

35. We would be happy to answer any questions you have.
Thank you!
We would be happy to answer any questions you have.
Brian Robertson, PhD
VP Research
Market Decisions Research
(207) ext. 102
Mark Noyes, MPH
Research Analyst
Market Decisions Research
(207) ext. 114