NLSCY – Suggestions for papers

Objectives of the Presentation zEmphasize proper ways to use the NLSCY data zIdentify the key factors we are looking at when reviewing a paper

NLSCY zNLSCY is a longitudinal survey with a complex sample design zAs a result, the data must be used properly: y Data limitations must be known yWeights, variance, valid statistical tests must be used

The factors – What are we looking for when reviewing a paper? z1 - Data sources, domain of study z 2 - Type of study z 3 - Use of the design weights z 4 - Proper variance calculation z 5 - Statistical tests and analytical methods

The factors – 1 - Data sources zAre the data sources clearly identified? y Cycle(s) used? y Which subgroups or domains? yWhich variables are used? y NLSCY target population well defined? y Are the NLSCY data limitations clearly identified?

The factors – 1 - Data sources (cont’d) y Responses rates to the key variables stated? y How is the partial non-response handled? y Sample sizes provided? Are the sample sizes sufficient? Confidentiality issues?

The factors – 2 - Type of study z What type of study is done? y Longitudinal? y Cross-sectional y NLSCY used as a repeated survey? y Mix of the above?

The factors – 3 - Design weights z Are the design weights used? y Appropriate weights (cross-sectional, longitudinal) used? y Revised weights or the original weights? y Re-weighting done to account for the partial non-response? zIf the weights are not used, why?

The factors – 4 - Variance z How is the variance calculated for all the estimates? y CV look-up tables? y Excel spreadsheet with CVs for proportions? y Using the Bootstrap weights? y Using a SAS or SPSS procedure, without taking into account the design? : a big DON’T. y No variance? Another big DON’T

The factors – 5 - Statistical tests z Were statistical tests used? y If so, are they clearly identified? y Are they appropriate for what you are doing?

The factors – In brief z Would a reviewer be able to reproduce your results and reach the same conclusions, given you have used all the appropriate methods (weights, variance, statistical tests)?

Questions