Term 4, 2006BIO656--Multilevel Models 1 PROJECTS ARE DUE By midnight, Friday, May 19 th Electronic submission only to Please name the file: [myname]-project.[filetype] or [name1_name2]-project.[filetype]

Term 4, 2006BIO656--Multilevel Models 2 Efficiency-Robustness Trade-offs First, we consider alternatives to the Gaussian distribution for random effects Then, we move to issues of weighting, starting with some formalism Then, move to an example of informative sample size And, finally give a basic example that has broad implications of choosing among weighting schemes

Term 4, 2006BIO656--Multilevel Models 3 Alternatives to the Gaussian Distribution for Random Effects

Term 4, 2006BIO656--Multilevel Models 4 The t-distribution Broader tails than the Gaussian So, shrinks less for deviant Y-values The t-prior allows “outlying” parameters and so a deviant Y is not so indicative of a large, level 1 residual

Term 4, 2006BIO656--Multilevel Models 5 Creating a t-distribution Assume a Gaussian sampling distribution, Using the sample standard deviation produces the t-distribution Z is t with a large df t 3 is the most different from Z for t-distributions with a finite variance

Term 4, 2006BIO656--Multilevel Models 6

Term 4, 2006BIO656--Multilevel Models 7 With a t-prior, B is B(Y), increasing with |Y -  |

Term 4, 2006BIO656--Multilevel Models 8 Z is distance from the center (1-B) = ½ = 0.50

Term 4, 2006BIO656--Multilevel Models 9 Z is distance from the center (1- B) = 2/3 = 0.666

Term 4, 2006BIO656--Multilevel Models 10 Estimated Gaussian & Fully Non-parametric priors Estimated Gaussian & Fully Non-parametric priors for the USRDS data

Term 4, 2006BIO656--Multilevel Models 11 USRDS estimated Priors

Term 4, 2006BIO656--Multilevel Models 12

Term 4, 2006BIO656--Multilevel Models 13

Term 4, 2006BIO656--Multilevel Models 14

Term 4, 2006BIO656--Multilevel Models 15

Term 4, 2006BIO656--Multilevel Models 16

Term 4, 2006BIO656--Multilevel Models 17 Informative Sample Size (Similar to informative Censoring) Informative Sample Size (Similar to informative Censoring) See Louis et al. SMMR 2006

Term 4, 2006BIO656--Multilevel Models 18

Term 4, 2006BIO656--Multilevel Models 19

Term 4, 2006BIO656--Multilevel Models 20

Term 4, 2006BIO656--Multilevel Models 21

Term 4, 2006BIO656--Multilevel Models 22

Term 4, 2006BIO656--Multilevel Models 23

Term 4, 2006BIO656--Multilevel Models 24

Term 4, 2006BIO656--Multilevel Models 25 Choosing among weighting schemes Choosing among weighting schemes “Optimality” versus goal achievement

Term 4, 2006BIO656--Multilevel Models 26 Inferential Context Question What is the average length of in-hospital stay? A more specific question What is the average length of stay for: –Several hospitals of interest? –Maryland hospitals? –All hospitals? –

Term 4, 2006BIO656--Multilevel Models 27 “Data” Collection & Goal Data gathered from 5 hospitals Hospitals are selected by some method n hosp patient records are sampled at random Length of stay (LOS) is recorded Goal is to: Estimate the “population” mean

Term 4, 2006BIO656--Multilevel Models 28 Procedure Compute hospital-specific means “Average” them –For simplicity assume that the population variance is known and the same for all hospitals How should we compute the average? Need a goal and then a good/best way to combine information

Term 4, 2006BIO656--Multilevel Models 29 “DATA” Hospital # sampled n hosp Hospital size % of Total size: 100  hosp Mean LOS Within- hospital variance  2 /  2 /  2 /  2 /  2 /15 Total ??

Term 4, 2006BIO656--Multilevel Models 30 Weighted averages & Variances Weighted averages & Variances (Variances are based on FE not RE) Weighting approach Weights x100 MeanVariance Ratio 100*(Var/min) Equal Proportional to Reciprocal variance Population  hosp Each weighted average is mean = Reciprocal variance weights minimize variance Is that our goal?

Term 4, 2006BIO656--Multilevel Models 31 There are many weighting choices and weighting goals Minimize variance by using reciprocal variance weights Minimize bias for the population mean by using population weights (“survey weights”) Use policy weights (e.g., equal weighting) Use “my weights,”...

Term 4, 2006BIO656--Multilevel Models 32 General Setting When the model is correct All weighting schemes estimate the same quantities – same value for slopes in a multiple regression So, it is clearly best to minimize variance by using reciprocal variance weights When the model is incorrect Must consider analysis goals and use appropriate weights Of course, it is generally true that our model is not correct!

Term 4, 2006BIO656--Multilevel Models 33 Weights and their properties But if  1 =  2 =  3 =  4 =  5 =  then all weighted averages estimate the population mean:    k  k So, it’s best to minimize the variance But, if the hospital-specific  k are not all equal, then Each set of weights estimates a different target Minimizing variance might not be “best” For an unbiased estimate of set  w k =  k

Term 4, 2006BIO656--Multilevel Models 34 The variance-bias tradeoff General idea Trade-off variance & bias to produce low Mean Squared Error (MSE) (Estimate - True) 2 MSE = Expected(Estimate - True) 2 Variance + (Bias) 2 = Variance + (Bias) 2 Bias is unknown unless we know the  k (the true hospital-specific mean LOS) But, we can study MSE ( , w,  ) In practice, make some “guesses” and do sensitivity analyses

Term 4, 2006BIO656--Multilevel Models 35 Variance, Bias and MSE as a function of (the  s, w,  ) Consider a true value for the variation of the between hospital means (  * is the “overall mean”) T =  (  k -  * ) 2 Study BIAS, Variance, MSE for weights that optimize MSE for an assumed value (A) of the between- hospital variance So, when A = T, MSE is minimized by this optimizer In the following plot, A is converted to a fraction of the total variance A/(A + within-hospital) –Fraction = 0  minimize variance –Fraction = 1  minimize bias

Term 4, 2006BIO656--Multilevel Models 36 The bias-variance trade-off The bias-variance trade-off X-axis is assumed variance fraction Y is performance computed under the true fraction Assumed kk

Term 4, 2006BIO656--Multilevel Models 37 Summary Much of statistics depends on weighted averages Weights should depend on assumptions and goals trustIf you trust your (regression) model, –Then, minimize the variance, using “optimal” weights –This generalizes the equal  case worryIf you worry about model validity (bias for    –You can buy full insurance by using population weights –But, you pay in variance (efficiency) –So, consider purchasing only the insurance you need by using compromise weights