Raymond J. Carroll Texas A&M University Postdoctoral Training Program: Non/Semiparametric Regression and Clustered/Longitudinal Data

2 Where am I From? College Station, home of Texas A&M I-35 I-45 Big Bend National Park Wichita Falls, my hometown

3 Raymond CarrollAlan Welsh Naisyin WangEnno Mammen Xihong Lin Oliver Linton Acknowledgments Series of papers are on my web site Lin, Wang and Welsh: Longitudinal data Linton and Mammen: time series data

4 Outline Longitudinal models: panel data Background: splines = kernels for independent data Nonparametric case: do splines = kernels? Semiparametric case: partially linear model: does it matter what nonparametric method is used?

5 Panel Data (for simplicity) i = 1,…,n clusters/individuals j = 1,…,m observations per cluster SubjectWave 1Wave 2…Wave m 1XXX 2XXX …X nXXX

6 Panel Data (for simplicity) i = 1,…,n clusters/individuals j = 1,…,m observations per cluster Important point: The cluster size m is meant to be fixed This is not a time series problem where the cluster size increases to infinity We have equivalent time series results

7 The Nonparametric Model Y = Response X = time-varying covariate Question : can we improve efficiency by accounting for correlation?

8 Independent Data Two major methods Splines (smoothing, P-splines, etc.) with penalty parameter = Kernels (local averages, local linear, etc.), with kernel function K and bandwidth h

9 Independent Data Two major methods Splines (smoothing, P-spline, etc.) Kernels (local averages, etc.) Both are linear in the responses Both give similar answers in data Silverman showed that the weight functions are asymptotically equivalent In this sense, splines = kernels

10 The weight functions G n (t=.25,x) in a specific case for independent data Kernel Smoothing Spline Note the similarity of shape and the locality: only X’s near t=0.25 get any weight

11 Working Independence Working independence: Ignore all correlations Posit some reasonable marginal variances Splines and kernels have obvious weighted versions Weighting important for efficiency Splines and kernels are linear in the responses The Silverman result still holds In this sense, splines = kernels

12 Accounting for Correlation Splines have an obvious analogue for non- independent data Let be a working covariance matrix Penalized Generalized least squares (GLS) Because splines are based on likelihood ideas, they generalize quickly to new problems Kernels have no such obvious analogue

13 Accounting for Correlation Kernels are not so obvious Local likelihood kernel ideas are standard in independent data problems Most attempts at kernels for correlated data have tried to use local likelihood kernel methods

14 Kernels and Correlation Problem: how to define locality for kernels? Goal: estimate the function at t Let be a diagonal matrix of standard kernel weights Standard Kernel method: GLS pretending inverse covariance matrix is The estimate is inherently local

15 Kernels and Correlation Specific case: m=3, n=35 Exchangeable correlation structure Red:  = 0.0 Green:  = 0.4 Blue:  = 0.8 Note the locality of the kernel method The weight functions G n (t=.25,x) in a specific case for independent data

16 Splines and Correlation Specific case: m=3, n=35 Exchangeable correlation structure Red:  = 0.0 Green:  = 0.4 Blue:  = 0.8 Note the lack of locality of the spline method The weight functions G n (t=.25,x) in a specific case for independent data

17 Splines and Correlation Specific case: m=3, n=35 Complex correlation structure Red: Nearly singular Green:  = 0.0 Blue:  = AR(0.8) Note the lack of locality of the spline method The weight functions G n (t=.25,x) in a specific case for independent data

18 Splines and Standard Kernels Accounting for correlation: Standard kernels remain local Splines are not local Numerical results can be confirmed theoretically Don’t we want our nonparametric regression estimates to be local?

19 Results on Kernels and Correlation GLS with weights Optimal working covariance matrix is working independence! Using the correct covariance matrix Increases variance Increases MSE Splines Kernels (or at least these kernels)

20 Better Kernel Methods: SUR Iterative, due to Naisyin Wang Consider current state in iteration For every j, assume function is fixed and known for Use the seemingly unrelated regression (SUR) idea For j, form estimating equation for local averages/linear for j th component only using GLS with weights Sum the estimating equations together, and solve

21 SUR Kernel Methods It is well known that the GLS spline has an exact, analytic expression We have shown that the SUR kernel method has an exact, analytic expression Both methods are linear in the responses Relatively nontrivial calculations show that Silverman’s result still holds Splines = SUR Kernels

22 Nonlocality The lack of locality of GLS splines and SUR kernels is surprising Suppose we want to estimate the function at t All observations in a cluster contribute to the fit, not just those with covariates near t Somewhat similar to GLIM’s, there is a residual- adjusted pseudo-response that has expectation = response Has local behavior in the pseudo-response

23 Nonlocality Wang’s SUR kernels = pseudo kernels with a clever linear transformation. Let SUR kernels are working independence kernels

24 The Semiparametric Model Y = Response X,Z = time-varying covariates Question: can we improve efficiency for  by accounting for correlation?

25 The Semiparametric Model General Method: Profile likelihood Given , solve for  say Your favorite nonparametric method applied to Working independence Standard kernel SUR kernel

26 Profile Methods Given , solve for  say Then fit WI/GLS to the model with mean Standard kernel methods have awkward, nasty properties SUR kernel methods have nice properties Semiparametric asymptotically efficient

27 ARE for  of Working Independence Cluster Size: Black: m = 3 Red: m = 4 Green: m = 5 Blue: m = 6 Scenario: X’s: common correlation 0.3 Z’s: common correlation 0.6 X & Z: common correlation 0.6  : common correlation  Note: Efficiency depends on cluster size

28 Profile Methods Given , solve for  say Then fit GLS to the model with mean If you fit working independence for your estimate of , there is not that great a loss of efficiency

29 ARE for  of Working Independence/Profile Method Cluster Size: Black: m = 3 Red: m = 4 Green: m = 5 Blue: m = 6 Scenario: X’s: common correlation 0.3 Z’s: common correlation 0.6 X & Z: common correlation 0.6  : common correlation  Note: Efficiency depends on cluster size

30 Conclusions In nonparametric regression Kernels = splines for working independence Working independence is inefficient Standard kernels splines for correlated data SUR kernels = splines for correlated data In semiparametric regression Profiling SUR kernels is efficient Profiling: GLS for  and working independence for  is nearly efficient