CASE STUDY: NATIONAL SURVEY OF FAMILY GROWTH Karen E. Davis National Center for Health Statistics Coordinating Center for Health Information and Service NCHS Data Users Conference, Washington, D.C. July 12, 2006

NCHS SURVEYS Records based –Data from vital and medical records – e.g. NDI, VSCP Population based –Data from personal interviews – e.g. NHIS, NSFG, NIS, SLAITS,NHANES Establishment based –Data from hospital records/facility interviews – e.g. NAMCS, NHAMCS, NHDS, NNHS 3

PRESENTATION OUTLINE The NSFG Cycle 6 Sample SRS versus complex? Estimates from NSFG Data Cycle 7 Sample 4

NSFG Background NSFG data are a source of information on childbearing, pregnancy, and related events Collects statistics on family formation, childbearing, marriage, divorce, and cohabitation Conducted periodically: 1973, 1976, 1982, 1988, 1995,

NSFG Cycle 6 Previous cycles were nationally representative area samples of the civilian noninstitutionalized population of women Cycle 6 surveys both men and women years of age Main data collection Mar. 2002–Feb

Key Analytical Goals of Cycle 6 Compare key statistics within Cycle 6 by race/ethnicity and age Compare key statistics between survey cycles 7

Key Statistics of Cycle 6 Proportion of teenagers who have ever had sexual intercourse; Proportion of teenagers who used a condom at their most recent intercourse; Proportion of each 5-year age group who are currently using the oral contraceptive pill; and Proportion of childless women by age who have impaired fecundity. 8

Cycle 6 Sample Design 121 PSU stratified multistage cluster sample of households Primary selection: blocks –Select 1,414 segments –Select 40 households per segment 55,000 selected housing units 9

Cycle 6 Distribution by Region 10 RegionNumber of SegmentsNumber of HUs Northeast34113,420 Southeast33613,145 West39915,675 Midwest32412,760 Total140055,000 Source: ISR

Cycle 6 Sample Design Features 12,571 completed interviews –Females and Males –Higher rates for African-Americans, Hispanics and year-olds 11

Cycle 6 Analytical Subdomains 12 Age FemalesMales BlackHispanicOtherBlackHispanicOther

Cycle 6 Sample Design Features Within Household Selection –Select one eligible person at random –Implement selection in Blaise software –Age-gender-race/ethnicity selection cells –Vary sampling rates across cells 13

Cycle 6 Sample Design Why not use Simple Random Sampling? 14

Simple Random Sampling All frame elements have an equal chance of selection All combinations of frame elements (of the given sample size) have an equal chance of selection 15

Simple Random Sampling Costly data collection – sampling individual members of the population Does not assure representation of special sub-groups of interest No complete, up-to-date listing of individual members of the population 16

Use of complex designs Cluster sampling is a low-cost device for fixing the probability of including each member of the population in the sample Complete listings of clusters (e.g. counties) are readily available Special subgroups are sampled at higher rates to assure sample size 17

Use of complex designs Can reduce sampling error over simple random sampling depending on allocation of strata Can reduce travel or other data collection costs Unequal selection probabilities can increase the sample size of rare units 18

Cycle 6 Sample Design Complex design is essential for NSFG targets: –Females and Males –Higher rates for African-Americans, Hispanics and year-olds 19

Cycle 6 Responsive Design More timely data on field costs and response rates ISR developed SurveyTrak system for daily info on hours of effort required to obtain interviews Field administrative data used for statistical modeling and statistical process control analysis Prevent cost over-runs and manage fieldwork given limited budget Implement double sample in final month 20

NSFG Weighting The product of four factors is used to create a single weight for each case: Factor 1: Base sampling weights –Inverse of selection probabilities Factor 2: First-stage ratio adjustment –Counteracts sampling variation across primary sampling units within a stratum 21

NSFG Weighting Factor 3: Nonresponse adjustment –Includes eligibility, noncontact, and refusal adjustments Factor 4: Poststratification adjustment –Uses external population totals for ratio adjustments by age, gender, and race/ethnicity 22

NSFG Imputation Logical Imputation –Deduce missing answer from answers to other questions Multivariate sequential regression –Model based on nonmissing predictors 23

NSFG Variance Estimation Should reflect complex design including unequal selection probabilities and clustering of respondents A variance estimate based on a simple random sample will likely underestimate the actual sampling variance 24

NSFG Variance Estimation Taylor Series or pseudo-replication Complex variance estimation software (e.g. SUDAAN, WesVarPC, SAS) 25

NSFG Design Effects Provide a summary measure of the combined effects of stratification, clustering, and unequal weighting on the variance of a survey estimate. Design Effects are generally larger for subgroups that are oversampled because of the sample design. 26

NSFG Cycle 7 Continuous interviewing 4400 male and female respondents per year Content similar to Cycle 6 Collect data more frequently Responsive design Reduce cost per case 27

NSFG Cycle 7 Sample Design Use the same 8 large Metro areas each year, plus 25 new PSUs each year –Year 1: 33 PSUs (8 +25) ( ) –Year 2: 58 PSUs (8 +50) (2008) –Year 3: 83 PSUs (8 + 75) (2009) –Year 4: 108 PSUs ( ) (2010) 28

Sample Size Yields for Cycle 7 29 Cycle 6Cycle 7 (2002) Jun Dec 2007 Jun Dec 2010 Total12,5716,60019, ,2711,200 3,100 Male 4,9282,970 8,900 Female 7,6433,63010,900 Hispanic 2,7121,313 3,940 Black 2,4601,340 4,020 Source: ISR

References for the NSFG Design: NCHS Publications National Survey of Family Growth, Cycle 6: Sample Design, Weighting, Imputation and Variance Estimation. Series 2, No. 142 (June, 2006) Plan and Operation of Cycle 6 of the National Survey of Family Growth. Series 1, No. 42 (August, 2005) Both are available at NCHS website. 30

NCHS website: Contact: Karen E. Davis 31