1. NLSCY - Variance

2. Objectives of the Presentation - Demonstration zWhy is it necessary to compute the variance? zHow can the variance be computed with NLSCY data?

3. Why Compute the Variance? zNLSCY data come from a probabilistic survey: yVariability associated with estimates produced with data from any probabilistic survey yTo make valid inferences about the population of interest, that variability must be measured

4. Main Difficulty: Complexity of the NLSCY’s Sampling Plan zTwo different sample frames used to select the sample: yLabour Force Survey (LFS), itself a survey with a complex sample design yBirth Register zUse of two frames for certain groups (five-year-olds, Cycle 3)

5. Complexity of the NLSCY’s Sampling Plan (continued) zChildren’s selection probabilities very uneven zNon-response adjustments that cross strata boundaries zEmpty clusters from the LFS

6. Effects of the Complexity of the NLSCY’s Sampling Plan zNo exact analytical formula for computing the variance because of the complex sample design. zNo commercial application can fully take the NLSCY’s complexity into account in computing the variance.

7. How to Compute the Variance for the NLSCY z3 solutions: yApproximate sampling variability tables provided in the user’s guide (in the form of coefficients of variation (CVs)). yApproximate CV tables for a number of specific subject areas (Excel spreadsheet). yUse bootstrap weights and SAS program supplied by the NLSCY.

8. How to Compute the Variance for the NLSCY zOf these 3 solutions: yThe first two can be used for exploratory analysis. These 2 methods provide an approximation of the variance yOnly the third solution computes the variance “more exactly”

9. Sampling Variability Tables zVery very limited… zUsers’ guide explains how to use them

10. Approximate CV Tables (Excel) zBrief excerpt:

11. Approximate CV Tables (Excel) zLet’s go directly to the CV table and take a closer look...

12. Approximate CV Tables (Excel) zOriginally created to answer the question: yIs the Cycle 5 sample size large enough? zApproximation of the exact variance: yTakes the sample design into account by using bootstrap weights. yOn the other hand, uses a random variable instead of real variables.

13. Approximate CV Tables (Excel) zCVs available for many subject areas, for a number of proportions. zLots of additional information available: ysample size, projected size, confidence interval

14. Approximate CV Tables (Excel) zFunctions: yCan choose areas of interest and obtain an approximate CV. yPossibility of making queries xExample: What subject areas have CVs of less than 25%?

15. Approximate CV Tables (Excel) zIn a nutshell: yMuch more detailed than the tables provided in the user’s guide, but … ythey can’t replace exact variance calculation ylimited number of subject areas

16. Bootstrap Weights and NLSCY_VES zComputing the variance using the replicate (bootstrap) method: yfor longitudinal estimates yfor cross-sectional estimates yfor all cycles yfor all desired subject areas

17. Bootstrap Weights and NLSCY_VES zSAS program called NLSCY_VES is provided for computing the variance: yset of macros yeasy to use, well documented yexamples provided ycomputes variance for totals, ratios, differences between ratios, linear regressions, logistic regressions

18. Bootstrap Basics zA. Select a subsample from the original sample with replacement zB. For this subsample, calculate the weights as if it were the full sample zRepeat A and B many times (1,000) to produce a set of bootstrap weights

19. Bootstrap Basics (continued) zFor a given estimate: yCalculate the estimate with each set of weights yCalculate the variance of the estimates obtained

20. Structure of NLSCY_VES zAll the macros are in SAS file: NLSCY_VES.sas yNo changes needed zAnother SAS program calls the macros and allows the user to set the various parameters.

21. What You Need zThe SAS file NLSCY_VES.sas zThe SAS program to call the macros zSet of data for which the variance is required zThe file of 1,000 bootstrap weights for the appropriate cycle and type of analysis (longitudinal or cross-sectional)

22. Conclusion zThe variance must be computed if we are to make valid inferences zThe sample design must be taken into account if we want the variance calculation to be valid. Otherwise, we may draw incorrect conclusions

23. Conclusion (continued) zThe NLSCY provides 3 tools for computing the variance: ytables in the user’s guide (too limited…) yExcel file for many domains (to get an idea) ybootstrap weights (the best approach)

24. Questions