1. VerdierView Graph # 1 OVERVIEW Problems With State-Level Estimates in National Surveys of the Uninsured Statistically Enhancing the Current Population Survey (CPS) to Obtain Better State-Level Estimates State-Specific Surveys - State surveys - Urban institute survey - Community tracking study survey

2. VerdierView Graph # 2 MEASURING THE UNINSURED Surveys Can Measure the Number of People Without Insurance at a Particular Point, and Their Characteristics (Age, Sex, Income, Employment Status) Later Surveys Using the Same Methodology Can Measure Changes Over Time But Surveys Cannot Definitively Measure the Impact of CHIP on the Number of Uninsured Children - Imprecision in surveys - CHIP may substitute for private insurance

3. VerdierView Graph # 3 CURRENT POPULATION SURVEY Nationwide Monthly Census Bureau Survey 48,000 Households Interviewed Each Month - 75% phone, 25% in-person - 90 % response rate - Rotating sample - half interviewed again the following year Main Purpose Is to Determine Employment Status - March survey has supplemental questions on health insurance coverage

4. VerdierView Graph # 4 ADVANTAGES OF THE CPS Done Every Year Methodology Is Consistent From State to State and (Generally) From Year to Year Large National Sample Makes National-Level Estimates Reasonably Reliable Large Percentage of In-Person Interviews Improves Coverage and Reliability Rotating Sample Helps in Measuring Year-to-Year Changes

5. VerdierView Graph # 5 LIMITATIONS OF THE CPS State-Level Estimates of the Uninsured in the Annual March CPS Are Unreliable - State sample sizes are small - over 1000 households in only 11 States - Results in large margins of error, especially for sub- populations such as low-income uninsured children - Three-year averages smooth out year-to-year variations, but large margins of error remain - total number of children uninsured, using 3-year 1994-1996 average (10/97 CPS analysis) –California: 1,466,000 to 1,886,000 –Delaware: 12,000 to 32,000

6. VerdierView Graph # 6 LIMITATIONS OF THE CPS (CONT.) Other Problems With the CPS - May over-estimate number of full-year uninsured - full-year vs. point-in-time responses - Does not directly ask people whether they are uninsured - people may actually have insurance when they say they don’t - May under-estimate the number of people on Medicaid - federal administrative data show 20-30% more Medicaid enrollees than CPS does

7. VerdierView Graph # 7 STATISTICALLY ENHANCING THE CPS Purpose Is to Increase Reliability of State-Level CPS Estimates for Subpopulations - Reduces margin of error by increasing sample sizes - Can get more reliable detail on characteristics of lower-income families with children - family income, age of children, parental employment, insurance coverage

8. VerdierView Graph # 8 STATISTICALLY ENHANCING THE CPS (Cont.) Increases Sample Sizes in One State by “Borrowing” Similar Households From Other States - Households must be similar with respect to age, race, income, family type, employment status - Relationships between household characteristics and insurance coverage must be similar in borrowing and lending states - can be tested by looking at unemployment rates, economic growth, composition of employment by firm size, and other data

9. VerdierView Graph # 9 STATISTICALLY ENHANCING THE CPS (CONT.) Potential Uses and Limitations - Provides better information for program planning and design - permits more informed decisions about cost sharing, outreach, crowd-out, potential fiscal impact - Can provide better estimates of year-to-year changes in number of low-income uninsured children than unenhanced CPS - but large margins of error still remain - methodological limits of CPS also remain

10. VerdierView Graph # 10 STATE-SPECIFIC SURVEYS State-Specific Surveys May Be More Reliable Than Enhanced CPS - But will not be consistent from State to State unless part of a national survey - To measure impact of CHIP on uninsured, baseline survey must be conducted before implementation of CHIP, and another afterward - Good State-specific surveys can be costly

11. VerdierView Graph # 11 STATE-SPECIFIC SURVEYS (CONT.) Elements of a Good State Survey - Cost of roughly $100-200,000 per state, depending on sample size, method of data collection, desired response rate, and length of interview - sample size of 2,000-3,000 households - combination of phone and some in-person interviews –in-person is much more expensive - response rate of 70% or higher - reasonably focused questions

12. VerdierView Graph # 12 STATE-SPECIFIC SURVEYS (CONT.) Example: 1998 Maine Health Insurance Coverage Survey - Conducted by Mathematica for University of Southern Maine and Maine Medicaid Agency - cost about $150,000 (mostly public, with some private funds) - Surveyed 2,500 households with children - entirely by phone (75% response rate) - Comparison of survey results to state Medicaid enrollment files - one-third of 450 households reporting Medicaid coverage could not be linked to state files - nearly 20 percent of those enrolled in Medicaid told surveyors they were not

13. VerdierView Graph # 13 STATE-SPECIFIC SURVEYS (CONT.) Urban Institute Survey of 13 “Assessing the New Federalism” States (AL, CA, CO, FL, MA, MI, MN, MS, NJ, NY, TX, WA, WI) - Average of 3,300 households per state - One round in 1997 (pre-CHIP) and additional rounds in 1999 and 2001 - results of 1999 round won’t be available for BBA- required March 2000 state CHIP evaluations - Survey over-samples lower-income families with children - 30% of 50,000 households surveyed - 1999 round will ask several CHIP-specific questions

14. VerdierView Graph # 14 Community Tracking Study Survey - Conducted by Center for Studying Health System Change and Mathematica - Focus is on communities rather than states - 30,000 households in 60 communities nationwide - 12 high-intensity sites are oversampled –5 sites are in states not in Urban Institute survey - Pre- and post-CHIP surveys of insurance coverage - one round in 1996-97, another in 1998-99 - second round data not available until mid-2000 STATE-SPECIFIC SURVEYS (CONT.)

15. VerdierView Graph # 15 Annual March CPS Is Not Reliable at the State Level Statistically Enhanced CPS Can Produce More Reliable State Estimates, but Probably Not Reliable Enough to Track Impact of CHIP on Number of Uninsured Children Some State-Specific Surveys May Be More Reliable Than Enhanced CPS, but Will Not Be Comparable Across All States CONCLUSIONS

16. VerdierView Graph # 16 Precise State-By-State Impact of CHIP on Number of Uninsured Children Will Likely Remain Unknown Implications for States - Be prepared to respond to and use CPS, urban institute, CTS, and other surveys - Federal government will have difficulty holding states accountable for impact of CHIP on number of uninsured children - CHIP officials should think hard about what accountability requires within their own states CONCLUSIONS (CONT.)