Event History Analysis 5 Sociology 8811 Lecture 19 Copyright © 2007 by Evan Schofer Do not copy or distribute without permission

Announcements Class topic: Time-varying data: example More details on Cox models & other fully parametric Proportional Hazard models Paper Assignment #2 handed out today Due April 26

EHA Models: In Greater Depth Issues: –Properties of Cox models (semi-parametric) and fully parametric models Plus relevant assumptions, diagnostics Strategies for Outliers Model Fit –Choosing a model. What should you do? –Other issues Accelerated failure time models Frailty Etc..

Event History Example What factors affect how soon a country passes an environmental protection law? Event: Passing an environmental law in a given year Risk set: All countries that have not yet passed an environmental protection law –We decided that risk begins at 1970 (when such laws were invented) Countries independent after 1970 are treated as entering the analysis “late” Option #2: Duration since independence (age) –But, that was less appropriate for the research question.

Example: Environmental Laws Cross-national time series dataset of nearly 100 countries Event: when a country writes its first comprehensive environmental law (e.g., EPA) Data taken from various sources Independent variables: GDP, population, democracy, degradation, education, domestic and international NGOs Time duration: analyses are from In other words, countries enter the “risk set” in 1970, or when they become independent Total sample of 97 countries 73 countries have an event between 1970 and 1998.

Time-Varying Data Structure In the previous example, each row of data was a separate survey respondent Because survey respondents were not tracked over multiple years, this data was not “time-varying” In the current example, we have the advantage of time-varying data Each row of data is a country-year Our independent variables may change over time.

States, Spells, and Events Example (India): … … 1998 Year State Spell #2 Spell #1 Law written

States, Spells, and Events Example (Iran): … … 1998 Year State Spell #1 No law written as of 1998

Time-Varying Data Structure newname2newid3yearlaweventnumstartendssespop INDIA INDIA INDIA INDIA INDIA INDIA INDIA INDIA INDIA INDIA INDIA INDIA INDIA INDIA Example: Law written SpellState Population

Time-Varying Data Structure newname2newid3yearlaweventnumstartendssespop INDIA INDIA INDIA INDIA INDIA INDIA INDIA INDIA INDIA INDIA INDIA INDIA INDIA INDIA Stset command: stset end, failure(es==1) origin(1970) Note: It is common to drop cases that are not at risk (ex: if start state = 1) BUT, it is not necessary… Stata drops cases after the event by default…unless you specify exit(time.)

Time-Varying Data Structure What if countries pass multiple laws? Called “repeated events 1. start state could be reset to zero 2. We can override the stata default of removing cases after the first event occurs: exit(time.) newname2newid3yearlaweventnumstartendssespop INDIA INDIA INDIA INDIA INDIA INDIA INDIA INDIA INDIA INDIA INDIA INDIA INDIA INDIA

Cumulative Survivor Function

Cumulative Survivor Function by Region

Cumulative Survivor Function West vs. non-West

Smoothed Hazard Function West vs. non-West

Constant Rate Model: Example Simple one-variable model comparing west vs. non-west streg west, dist(exponential) nohr Exponential regression -- log relative-hazard form No. of subjects = 97 Number of obs = 2047 No. of failures = 81 Time at risk = 2047 Wald chi2(1) = Log pseudolikelihood = Prob > chi2 = (Std. Err. adjusted for 97 clusters in newid3) | Robust _t | Coef. Std. Err. z P>|z| [95% Conf. Interval] west | _cons |

Constant Rate Model: Example Model with time-varying covariates No. of subjects = 92 Number of obs = 1938 No. of failures = 77 Time at risk = 1938 Wald chi2(6) = Log pseudolikelihood = Prob > chi2 = (Std. Err. adjusted for 92 clusters in newid3) | Robust _t | Coef. Std. Err. z P>|z| [95% Conf. Interval] gdp | degradation | education | democracy | ngo | ingo | _cons | Democratic countries enact laws at a higher rate than less-democratic countries

Constant Rate Model: Example Same model – with Hazard Ratios No. of subjects = 92 Number of obs = 1938 No. of failures = 77 Time at risk = 1938 Wald chi2(6) = Log pseudolikelihood = Prob > chi2 = (Std. Err. adjusted for 92 clusters in newid3) | Robust _t | Haz. Ratio Std. Err. z P>|z| [95% Conf. Interval] gdp | degradation | education | democracy | ngo | ingo | A 1-point increase in democracy increases the hazard rate by 25.8%!

Constant Rate Model : Example What if we expect global civil society to have a particularly strong effect in the non-West? Option #1: Create an interaction term No. of subjects = 92 Number of obs = 1938 No. of failures = 77 Time at risk = 1938 Wald chi2(8) = Log pseudolikelihood = Prob > chi2 = (Std. Err. adjusted for 92 clusters in newid3) | Robust _t | Coef. Std. Err. z P>|z| [95% Conf. Interval] gdp | degradation | education | democracy | ngo | ingo | nonwest | ingoXnonwest | _cons |

Constant Rate Model : Example What if we expect global civil society to have a particularly strong effect in the non-West? Option #2: Include only non-Western countries in the analysis No. of subjects = 76 Number of obs = 1720 No. of failures = 61 Time at risk = 1720 Wald chi2(6) = Log pseudolikelihood = Prob > chi2 = (Std. Err. adjusted for 76 clusters in newid3) | Robust _t | Coef. Std. Err. z P>|z| [95% Conf. Interval] gdp | degradation | education | democracy | ngo | ingo | _cons |

Cox Models The basic Cox model: Where h(t) is the hazard rate h 0 (t) is some baseline hazard function (to be inferred from the data) This obviates the need for building a specific functional form into the model Also written as:

Cox Model: Example Mostly similar to exponential model… Cox regression -- Breslow method for ties No. of subjects = 92 Number of obs = 1938 No. of failures = 77 Time at risk = 1938 Wald chi2(6) = Log pseudolikelihood = Prob > chi2 = (Std. Err. adjusted for 92 clusters in newid3) | Robust _t | Coef. Std. Err. z P>|z| [95% Conf. Interval] gdp | degradation | education | democracy | ngo | ingo | Most effects = similar… though education effect loses significance…

Cox Model Issues: Ties 1. How to handle ties in data It is mathematically complex to estimate models when there are tied failures –That is: two cases that have events at the exact same time Several mathematical approaches: –Breslow approximation – simplest approach Stata default, but not the best choice! –Efron approximation – generally better More computationally intensive, but given the power of modern computers it is not an issue Efron is generally preferred

Cox Model Issues: Ties –Exact marginal – “continuous time approximation” –Box-Steffensmeier & Jones: “Averaged Likelihood” Assumes ties didn’t happen EXACTLY at the same time… and considers all possible orderings –Exact partial – “discrete” –Box-Steffensmeier & Jones: “exact discrete method” Assumes ties happened EXACTLY at the same time –Advice: Use Efron at a minimum Exact methods are often more accurate –Exact marginal often makes most sense… events rarely occur at the EXACT same time –But, exact methods can take a LONG time. –For big datasets with many ties, Efron is OK.

Cox Model: Baseline Hazard Cox models involve a “baseline hazard” Note: baseline = when all covariates are zero Question: What does the baseline hazard look like? –Or baseline survivor & integrated hazard? –Stata can estimate the baseline survivor, hazard, integrated hazard. Two steps: 1. You must ask stata to save the info when you run the Cox model –Ex: stcox gdp degradation education democracy ngo ingo, robust nohr basehc(h0) 2. Use “stcurve” command to plot the baseline curves –Ex: stcurve, hazard OR stcurve, survival

Cox Model: Baseline Hazard Baseline rate: Adoption of environmental law

Cox Model: Baseline Hazard Note: It may not always make sense to plot the baseline hazard Baseline shows hazard when X variables are zero Sometimes zero values aren’t very useful/interesting –Example: Does it make sense to plot hazard of countries adopting laws, if GDP is zero? Hazard rate is quite low In some cases, you’ll just get a flat zero curve –Or extremely high values –Solutions: 1. Rescale indep vars before running cox model 2. Use stcurve to choose relevant values of vars.

Cox Model: Estimated Hazards You can also use stcurve to plot estimated hazard rates based on values of indep vars Ex: What is hazard curve if democracy = 1, 5, 10? Strategy: use “at” subcommand: stcurve, hazard at(democ=1) at2(democ=10) NOTE: All other variables are pegged at the mean…

Cox: Estimated Hazard Rate Hazard rate for adoption of environmental law

Proportional Hazard Assumption Key assumption: Proportional hazards Estimated Hazard ratios are proportional over time i.e., Estimates of a hazard ratio do NOT vary over time –Example: Effect of “abstinence” program on sexual behavior Issue: Do abstinence programs lower the rate in a consistent manner across time? –Or, perhaps the rate is lower initially… but then the rate jumps back up (maybe even exceeds the control group). –Groups are assumed to have “parallel” hazards Rather than rates that diverge, converge (or cross).

Proportional Hazard Assumption Strategies: 1. Visually examine raw hazard plots for sub- groups in your data Watch for non-parallel trends A simple, crude method… but often identifies big violations

Proportional Hazard Assumption Visual examination of raw hazard rate Parallel trends in hazard rate look good!

Proportional Hazard Assumption 2. Plot –ln(-ln(survival plot)) versus ln(time) across values of X variables What stata calls “stphplot” Parallel lines indicate proportional hazards Again, convergence and divergence (or crossing) indicates violation –A less-common approach: compare observed survivor plot to predicted values (for different values of X) What stata calls “stcoxkm” If observed are similar to predicted, assumption is not likely to be violated.

Proportional Hazard Assumption -ln(-ln(survivor)) vs. ln(time) – “stphplot” Convergence suggests violation of proportional hazard assumption (But, I’ve seen worse!)

Proportional Hazard Assumption Cox estimate vs. observed KM – “stcoxkm” Predicted differs from observed for countries in West

Proportional Hazard Assumption 3. Piecewise Models Piecewise = break model up into pieces (by time) –Ex: Split analysis in to “early” vs “late” time If coefficients vary in different time periods, hazards are not proportional –Example: stcox var1 var2 var3 if _t < 10 stcox var1 var2 var3 if _t >= 10 Look for large changes in coefficients!

Proportional Hazard Assumption In a piecewise model, coefficients would differ in non-proportional models Proportional Non-Proportional Here, the effect is the same in both time periods Early Late Early Late Here, the effect is negative in the early period and positive in the late period

Piecewise Models Look at coefficients at 2 (or more) spans of time EARLY. stcox gdp degradation education democracy ngo ingo if year < 1985, robust nohr _t | Coef. Std. Err. z P>|z| [95% Conf. Interval] gdp | degradation | education | democracy | ngo | ingo | LATE. stcox gdp degradation education democracy ngo ingo if year >= 1985, robust nohr _t | Coef. Std. Err. z P>|z| [95% Conf. Interval] gdp | degradation | education | democracy | ngo | ingo | Note: Effect of ngo is larger in early period

Proportional Hazard Assumption 4. Tests based on re-estimating model Try including time interactions in your model Recall: Interactions – effect of A on C varies with B If effect of variable X on hazard rate (or ratio) varies with time, then hazards aren’t proportional –Recall example: Abstinence programs Perhaps abstinence programs have a big effect initially, but the effect diminishes (or reverses) later on

Proportional Hazard Assumption Red = Abstinence group; green = control Proportional Non-Proportional In non-proportional case, the effect of abstinence programs varies across time

Proportional Hazard Assumption Strategy: Create variables that reflect the interaction of X variables with time Significant effects of time interactions indicate non- proportional hazard Fortunately, inclusion of the interaction term in the model corrects the problem. Issue: X variables can interact with time in multiple ways… –Linearly –With “log time” or time squared –With time dummies –You may have to try a range of things…

Proportional Hazard Assumption Red = Abstinence group; green = control Linear time interaction Effect grows consistently over time Try “Abstinence*time” Interaction with time-period… Effect differs early vs. late Try “Abstinence*DLate”

Proportional Hazard Assumption 5. Grambsch & Therneau test –Ex: Stata “ estat phtest” Test for non-zero slope of Schoenfeld residuals vs time –Implies log hazard ratio function = proportional Can be applied to general model, or for each variable stcox gdp degradation education democracy ngo ingo, robust nohr scaledsch(sca*) schoenfeld(sch*). estat phtest Test of proportional hazards assumption Time: Time | chi2 df Prob>chi global test | Significant chi-square indicates violation of proportional hazard assumption

Proportional Hazard Assumption Variable-by-variable test “estat phtest”:. estat phtest, detail Test of proportional hazards assumption Time: Time | rho chi2 df Prob>chi gdp | degradation | education | democracy | ngo | ingo | global test | Note: Certain variables are especially problematic…

Proportional Hazard Assumption Notes on estat phtest : –1. Requires that you calculate “schoenfeld residuals” when you run the original cox model –And, if you want a test for each variable, you must also request scaled schoenfeld residuals –2. Test is based on identifying non-zero time trend… but how should we characterize time? Options: normal/linear time, log time, time dummies, etc –Results may differ depending on your choice –Ex: estat phtest, log – specifies “log time” Plot of smoothed Schoenfeld residuals can indicate best way to characterize time –Linear trend (not a curve) indicates that time is characterized OK –Ex: estat phtest, plot(ngo) OR estat phtest, log plot(ngo)

Proportional Hazard Assumption What if the assumption is violated? 1. Improve model specification Add time interactions to address nonproportionality Ex: If high democracies are not proportional to low democracies, try adding “highdemoc*time” Variables can be interacted with linear time, log time, time dummies, etc., to address the issue 2. Model groups separately Split sample along variables that are non-proportional.

Proportional Hazard Assumption What if the assumption is violated? 3. Use a stratified Cox model Allows a different baseline hazard for each group –But, you can’t estimate effect of stratifying variable! Ex: stcox var1 var2 var3, strata(Dhighdemoc) 4. Use a piecewise model Split time into chunks… in which PH assumption is met –Requires sufficient sample size in all time periods! 5. Live with it (but temper your conclusions) Allison points out: Cox model is reasonably robust –Other issues (e.g., model misspecification) are bigger issues.