1. Lecture 5: Non-Parametric Estimation
Survival Function
Cumulative Hazard Function
Confidence Intervals

2. Switching Gears Now, abandon parametric assumptions
Very common in survival
Why
Not one “catch all” distribution
No central limit theorem for large samples

3. Censoring Assumption:
Potential censoring time is unrelated to potential event time
Reasonable?
Estimation approaches are biased when this is violated
Violation example:
Sick patients tend to miss clinical visits more often
High school drop-out: Kids who move may be more likely to drop-out

4. Terminology D distinct event times t1 < t2 < t3 < … < tD
Ties allowed
At time ti, there are di events
Yi is the number of individuals at risk at ti
Yi is all the people who have event times > ti
di/Yi is an estimate of the conditional probability of an event at ti, given survival to ti

5. Conditional Probabilities
Recall:
Which means:
And if we have > 2:

6. How does this relate to S(t)?

7. Kaplan-Meier Estimation
AKA ‘product-limit’ estimator
Step-function
Size of the steps depends of
Number of events at time t
Pattern of censoring before t

8. Kaplan-Meier Estimation
Greenwood’s formula
Most common variance estimator
Point-wise

9. Proof of Greenwood’s formula

10. Proof cont’d

11. Proof cont’d

12. Proof cont’d

13. Example Kim paper Event = time to relapse Data:
10, 20+, 35, 40+, 50+, 55, 70+, 80, 90+

15. Plot it…

16. Cumulative Hazard
Use H(t) = -ln(S(t))

17. Nelson-Aalen Estimator
Better small sample properties than KM
Variance of NA estimator

18. Uses of NA Model Identification Estimates of h(t) Survival Function
Recall H(t) vs. t
More later (chapter 12)
Estimates of h(t)
Slopes of H(t)
Survival Function
S(t) = exp(-H(t))
S(t) using NA for H(t) is called the Fleming-Harrington/Breslow method

19. Kim Example Using NA approach10 1
0.9 20 9 35 8
0.788 40 7 50 6 55 5
0.63 70 4 71 3 80 2
0.315 90

21. Fleming-Harrington Estimate
Almost equivalent to NA estimate of
Handles ties slightly differently
If there were 3 deaths out of 10 at risk:
Nelson estimate increments the hazard by 3/10
Fleming and Harrington increments the hazard by 1/10 + 1/9 + 1/8

22. Kim Example KM: black, FH: red, NA: green

23. Ties
di = (1, 0, 1, 1, 1, 1, 0, 0, 0, 1, 0)
Yi = (10, 20, 40, 40, 40, 50, 50, 70, 71, 80, 90)

25. R for KM, NA, and FH
library(survival) t<-c(10,20,40,40,40,50,50,70,71,80,90) d<-c(1,0,1,1,1,1,0,0,0,1,0) st<-Surv(t,d) help(survfit) fit.km<-survfit(st~1) fit.na<-survfit(st~1, type="fleming-harrington") fit.fh<-survfit(st~1, type="fh") fit.km fit.fh summary(fit.km) summary(fit.na) summary(fit.fh) plot(fit.km, conf.int=F, xlab="Time to Relapse (months)", ylab="Survival Function", lwd=2) lines(fit.na, type="s", lwd=2, col=2) lines(fit.fh, type="s", lwd=2, col=3)

26. Cumulative Hazard
fun: an arbitrary function defining a transformation of the survival curve.
fun = log is alternative way to draw a log-survival curve (but with the axis labeled with log(S) values)
fun = sqrt would generate a curve on the square root scale
Four often used transformations can be specified with a character argument instead
“log” is the same as using the log=T option
“event” plots the cumulative events (f(Y) = 1-y)
“cumhaz” plots the cumulative hazard function f(y) = -log(y)
“cloglog” creates a complimentary log-log survival plot (f(y) = log(-log(y)) along with log scale for the x-axis).

27. Help for Generic “plot” Function in R
R Documentation plot {graphics} Generic X-Y Plotting Description Generic function for plotting of R objects. For more details about the graphical parameter arguments, see par. For simple scatter plots, plot.default will be used. However, there are plot methods for many R objects, including functions, data.frames, density objects, etc. Use methods(plot) and the documentation for these. Usage plot(x, y, ...)

28. Help File for “plot.survfit”
R Documentation plot.survfit {survival} Plot method for survfit objects Description A plot of survival curves is produced, one curve for each strata. The log=T option does extra work to avoid log(0), and to try to create a pleasing result. If there are zeros, they are plotted by default at 0.8 times the smallest non-zero value on the curve(s). Usage ## S3 method for class 'survfit' plot(x, conf.int=, mark.time=TRUE, mark=3, col=1, lty=1, lwd=1, cex=1, log=FALSE, xscale=1, yscale=1, firstx=0, firsty=1, xmax, ymin=0, fun, xlab="", ylab="", xaxs="S", ...)

29. Cumulative Hazard
>class(fit.km) [1] “survfit” >plot(fit.km, conf.int=F, fun="cumhaz", lwd=2, xlab="Time to Relapse (months)", ylab="H(t)") >lines(fit.na, type="s", fun="cumhaz", lwd=2, col=2, conf.int=F) >lines(fit.fh, type="s", fun="cumhaz", lwd=2, col=3 , conf.int=F) >legend(2, 1.4, c("Kaplan-Meier","Nelson-Aalen","Fleming-Harrington"), col=1:3, lwd=2)
-So we can see this looks a little different from the “plot” we would use to plot a variable x vs. y in that we can call on this “fun” argument because this is actually invoking the function plot.survfit

30. Cumulative Hazard

31. Interpreting S(t) and H(t)
General philosophy
Bad to extrapolate
In survival
Bad to put a lot of stock in estimates at late time points
Have less data at later times

32. Observations? Convergence to H(t) = lt with increasing N
Could apply to parametric smoothing to get estimate of h(t), just the slope of the line H(t) versus t
More divergence for the upper end, where denominator data (risk set) is smaller

33. Textbook discusses bias in S(t) at tmax
Can estimate S(t) by 0 beyond tmax (negatively biased)
Can estimate S(t) = S(tmax) for t > tmax (positively biased)
When there is no censoring, the product limit estimator reduces to the empirical survival function
-The KM estimator for S(t) is well defined for all times less than the largest observed study time.
-If the largest observed time is a death ,the survival curve is estimated to be 0 beyond this point. However, it the largest time is censored, then the value of S(t) beyond t_max is undermined since we do not know this person would have died if they had not been censored
-Efron suggested estimating S(t) beyond t_max by ay setting S(t) = 0 for all times greater than t_max- which negatively biases the estimate at later times
-Altenatively, Gill suggested estimating S(s) for t_max as S(t_max), which biases the estimate in the opposite direction
-NOTE: both estimators of S(t) beyind tmax have similar large sample properties and converge to the true survival function for large N. However, t has been showin in simulations that the Gill estimate has better small sample properties and therefore is preferred.

34. Point-wise Confidence Intervals
Constructed to ensure that the true value of S(t) at a particular t, falls in the interval with (1 - a)% confidence
Notation:
Recall that this is the sum in the Greenwood’s formula:

35. “Linear” CIs Most commonly estimated is stats package
It is a point-wise CI for t
For simplicity of notation, assume 95% confidence

36. There are Other Better Options
Transformations have better properties
Two main approaches:
Log transformations: based on cumulative hazard approach
Arcsine square root

37. Log Transformation
Define q:
Then, the 95% CI is

38. Derivation of the Log Transformation

39. Log-log transformation
Since the survival function estimates a probability, it is bounded by 0 and 1
Taking the log results in bounds:
Taking the opposite results in bounds
Taking the double log results in bounds
-This one nice because are unrestricted unlike the linear and log versions

40. Complimentary log-log transformation
-Draw picture

41. Log-Log Transformation
Can estimate a confidence interval for the double log transformation
Estimate variance (delta method)
Use estimate to define CI according to: -

42. Log-Log Transformation
To get the CI for the survival function at time t
Must back transform from the double log

43. Arcsin Squareroot
Very ugly:

44. Cumulative Hazard CIs Linear Log Arcsin square root
See KI & Mo page 107

45. Which to Use When? For N > 25 and < 50% censoring
Log and log-log are good
Arcsin square root good
Both given ~ nominal coverage for 95% CI
Exception: extreme right tail where there is little data
Linear approach requires much larger N for good coverage

46. Which to Use When? Arcsin square root Log Linear
Slightly conservative
A little wider than necessary
Log
Slightly anti-conservative
A little too narrow
Linear
Overly anti-conservative
Too narrow
Large Samples: all about the same

47. Remember… Valid for point-wise intervals
Common incorrect interpretation:
Plot a set of point-wise 95% CIs
Interpret as confidence “band”
These “bands” are too narrow!

48. Example: Tongue Cancer data

49. R Code
library(survival) tongue<-read.csv("H:\\BMTRY_722_Summer2015\\Tongue.csv") dat<-Surv(tongue$Time, tongue$Cens) type<-tongue$Type plot(survfit(dat~type), conf.int=T, col=c(1,2),lty=c(2,1), lwd=c(2,2), xlab="Time to Death (months)", ylab="Survival", cex.axis=0.9) legend(300, .9, c("Aneuploid","Diploid"), lty=c(2,1), col=c(1,2), lwd=c(2,2), cex=0.8)

50. Add CIs: “plain”

51. Just Diploid Tumors

52. R Code
fit.lin<-survfit(dat[type==2]~1, conf.type="plain") fit.log<-survfit(dat[type==2]~1, conf.type="log") fit.loglog<-survfit(dat[type==2]~1, conf.type="log-log") plot(fit.log, conf.int=T, col=2, lwd=2, lty=4) lines(fit.loglog, conf.int=T, col=3, lwd=2, lty=2) lines(fit.lin, conf.int=T, col=1, lwd=2, lty=1) legend(165, 1, c("Linear","Log","Log-Log"), col=c(1,2,3), lty=c(1,4,2), lwd=2) summary(fit.lin) summary(fit.log) summary(fit.loglog)

53. R Results: Linear
>summary(fit.lin) Call: survfit(formula = dat[type == 2] ~ 1, conf.type = "plain") time n.risk n.event survival std.err lower 95% CI upper 95% CI …

54. R Results: Log
> summary(fit.log) Call: survfit(formula = dat[type == 2] ~ 1, conf.type = "log") time n.risk n.event survival std.err lower 95% CI upper 95% CI …

55. R Results: Log-Log
> summary(fit.loglog) Call: survfit(formula = dat[type == 2] ~ 1, conf.type = "log-log") time n.risk n.event survival std.err lower 95% CI upper 95% CI …