1. Paul Biemer, UNC and RTI Bac Tran, US Census Bureau Jane Zavisca, University of Arizona SAMSI Conference, 11/10/2005 Latent Class Analysis of Rotation Group Bias: The Case of Unemployment

2. Overview Motivation: To understand measurement error in the official unemployment rate Method: Latent Class Analysis: measurement error as classification error Distinction from previous research: Focus on measurement error mechanisms, as opposed to correcting marginal estimates. Ultimate goal: To improve survey design.

3. The Official Unemployment Rate In Labor Force { Source: The Current Population Survey, 2004

4. The Official Unemployment Rate Categories Employed: worked at least one hour in previous week, or temporarily absent from job. Unemployed: not employed and actively looking for work (unprompted categories), or temporarily laid off. Not in Labor Force (NILF): All others.

5. Evidence for Measurement Error in Labor Force Status (LFS) in the CPS 1.Re-interview inconsistency 2.Rotation group bias

6. Re-interview Inconsistency 1% random sample of original sample of 50,000 households is re-interviewed monthly (without replacement). Re-interview occurs in same week as the original interview. Inconsistent responses suggest measurement error.

7. Re-interview Inconsistency (2001-2003) 8.9% of cases are inconsistently classified.

8. Unemployment Inconsistency (2001-2003)

9. Rotation Group Design

10. Rotation Group Bias (2002 Full CPS)

11. What Could Cause Rotation Group Bias? Non-response bias: rotation groups may represent different populations. Differences in interview setting telephone vs. face-to-face proxy vs. self Time in sample effect Improved understanding of questionnaire Embarrassment at admitting prolonged unemployment Interview changes behavior

12. Latent Class Analysis to Test Hypotheses Sources of Rotation Group Bias Non-response bias (different populations): Does latent employment status vary by rotation group? Measurement error: Does rotation group influence error rates? Differences in setting: Does interview mode (telephone vs. face-to-face) initial interview influence error rates? Does interview mode account for apparent rotation group effects on error rates? Social pressure: Gender influences latent employment status Does gender also influence error rates? Does the effect of rotation group vary by gender?

13. Correlation between Month-in-Sample and Interview Mode

14. Re-interview Data Set N = 24,297 (un-weighted data) X = True Labor Force Status (Latent Variable) A = Observed Labor Force Status at Inititial Interview B = Observed Labor Force Status as Time 2 (Reinterview)

15. Basic Latent Class Model X A B X, A|X, B|X Shorthand: (with usual constraints for identifiability)

16. Grouping Variable X A B S, X|S, A|X, B|X S

17. External Variable influencing Classification Error X A B SM, X|S, A|XM, B|XM SM

18. Grouping versus External Variables X A B SM, X|S, A|XMS {AXM AXS}, B|XMS {BXM BXS} SM

19. Covariates S = Gender Men: 47% Women: 52% M = Month in Sample 1 or 5: 28% 2-4, 6-8: 72% T = Interview Mode (Initial Interview) Telephone: 72% In Person: 18%

20. Statistical Power & Identifiability Issues Large total N, but relatively small N for unemployed. More variables means more identifiable models, but also diminishing cell counts and boundary solutions.

21. Principles of Model Construction Always include X|S A|X B|X Assume 3 latent classes & S as grouping variable Fit classification table of A*B*M*T*S. Vary following effects M as grouping variable M &/or T affecting classification error for A & B T affecting A but not B S affecting A & B when identifiable based on other restrictions (including interaction of M & S)

22. Principles of Model Construction Try equality constraints Equal influence of M & or S on error rate for A & B. Error rate for T at time A = error rate at time B (when T does not affect B).

23. Principles of Model Selection Limit search to theoretically plausible models. Limit search to identifiable models. Overall model fit P-value of likelihood ratio test vs. saturated model >.01 Dissimilarity index <.05 Model selection among those meeting above criteria: Bayesian information criterion (BIC) Likelihood ratio test for nested models Check substantive interpretation within set of possible best models.

24. Best-Fitting Models

25. Estimated Unemployment Rate Model 1 (similar to other top models) UE = 4.9% Observed M.I.S. 1 & 5 UE = 6.0% Observed M.I.S. 2-4, 6-8 UE = 4.7%

26. Conditional Probabilities for Employment Status

27. Conditional Probabilities for A|TX & B|TX

28. Conditional Probabilities for A|MX & B|MX

29. Summary Findings Change in structural model (treating month-in- sample as grouping variable) does not change the preferred measurement model. Models fit nearly as well without M as grouping variable; casts doubt on non-response bias hypothesis. M-I-S bias is not just a function of interview mode. Covariate effects (esp. S) on response error should be examined further in model with more df; need another grouping variable.

30. Unresolved Issues Ambiguous results for model selection Most interested in fit of unemployment classification, but this is overwhelmed in measures of overall fit Software limitations: clustering, local & boundary solutions, standard errors not consistently output

31. Future Research Agenda Try finer coding of month-in-sample Develop models for other variables: age, race, proxy vs. self Pool more years of data Develop hypotheses & interpretation based on review of: experimental work analyses of non-response related models including Markov latent class models of employment status transitions

32. Rotation Group Bias (2001-2003, reinterview data)