Longitudinal Designs

Research Objective Show direct cause & effect Study relationships among variables for existing groups Explain outcomes after the fact

Type of Design True Experiment Quasi-Experiment Cross-Sectional Longitudinal Explanatory Case Study Exploratory Case Study

Purpose The purpose of longitudinal designs is to understand the relationships between broad social phenomena (a recession) and the life history of individuals (whether someone completed a college degree or not) on an outcome condition on at least two points in time, often several points in time (financial security)

Longitudinal vs. Multiple Point in Time Cross-Sectional Designs Multiple point in time cross-sectional designs can show the impacts of broad social events on comparison groups: e.g., effects of the recession on different age groups Longitudinal designs allow us to understand how the effects of these broad events that affect everyone differently because of individual life history circumstances; e.g., effects of the recession on different age groups (group or aggregate effects) and how those effects vary due to individual differences among people due to life history conditions like college degree or not, single mother or not, age at first employment (individual effects).

Distinctive Features Select a representative sample from a population or populations, often cohorts based on age or some other factor Groups or populations are defined based on the independent or predictor variables or cohorts Repeat measurement on at least two points in time, often more Do not take a new sample – you must get the same data from the same participants (actual people) for all measurements

Did you select one or more characteristics to define the population(s), often cohorts, of interest? Yes No Not a longitudinal Did you select a (preferably statistically) representative sample of each population or cohort of interest based on the characteristics used to define them? Yes No Not a longitudinal Did you collect information from every individual in each sample at multiple points in time? Yes No Not a longitudinal Longitudinal Design

When to Use Longitudinal designs take a long time to complete, unless you use the retrospective design which is much weaker than the prospective design Only use a longitudinal design when you want to understand the interactions between broad social events or processes and the individual life histories of study participants

Mortality You must get information from the same people each time. Start with a larger sample than you need because mortality will be a problem in any extended study. Mortality is a threat to the internal validity of most longitudinal designs. If an individual fails to respond at any point in time you basically cannot use the data from that person Assessment of how “drop outs” and those who you can find at each point in time is therefore critical to good design because you need to make sure that the drop-outs and stay-ins do not differ in ways that could affect the results of your study.

Measuring Change We saw in our discussion of true and quasi-experiments that measurement at multiple points in time (multiple post tests) makes data analysis complicated. Not surprising then, data analysis, especially statistical analysis, in longitudinal designs is complex. Raw scores are particularly problematic. The following slides provide a very brief discussion of four commonly used ways to calculate change over time, in ascending order of preference.

Ways of Measuring Change over Time Raw score Residual Percentage Standardized score

Raw Score The value of the dependent variable Problems Change = T2-T1 Must be a reliable measure because you get measurement errors at T1 and T2 and they compound Extreme scores are more likely to change due to regression to the mean Must have at least interval level data

Residual Deals with the problem of extreme scores and regression to the mean, and also, to some extent, with the problem of testing sensitization. This is a simplified, conceptual version. Calculate the average change for all subjects. Use this in a regression equation to predict the T2 score for each individual subject, based on his/her T1 score.

Must have at least interval level data Subtract the predicted T2 score for each subject from the actual, measured T2 score What’s left is the residual score, the amount of change that occurred independently of the original T1 score Hence the effect of the T1 score is removed Must have at least interval level data

Percentage (T2-T1/T1)*100 You must have ratio data for the dependent or outcome variable Note RATIO, interval data is not good enough!

Standardized z-Scores The standardized z-score is how many standard deviations from the mean the score lies. Standardized z-scores permit the researcher to compare relative (not just absolute) change between individuals over time.

Raw Salary

Relative Scores The next slide shows the same data as the raw score, but this time the data are presented as the Z score -- how many standard deviations each individual’s salary is above or below the mean.

Z-Scores

Comparing Change with Z scores The next slide shows the same data for the change in z score between each four year period is calculated. Compare this slide to the slide showing raw salary. In that slide, it appears that “pretty much everyone’s salary is going up.” The next slide shows that there are major differences between individuals relative to each other. Some people have big increases relative to others, while others have big decreases relative to others. Put simple, there are “winners and losers” in the salary race, something you cannot really see in the slide of raw salary.

Change in z-Scores

Moral of the Story Use longitudinal designs only when tracking individual change is important to understanding or explaining the phenomenon you are studying. If you have no need to incorporate and understand individual level change to understand the phenomenon as a whole, use a simpler design.

Consult a statistician well before you ever start the study Consult a statistician well before you ever start the study. These data analyses are complex. You need a statistician. Do this early – there’s nothing worse than visiting your local friendly statistician with your data in hand and hear him/her say: “The way you did that just does not work. These data cannot be analyzed in a meaningful way.”

Longitudinal designs are very powerful Longitudinal designs are very powerful. They are the only designs we have that really permit us to incorporate the effects of time and what happens to the individual to explain change in the status of whole groups in society over time. So do not be afraid of them. Use them. But do so intelligently.