Generalized Additive Model Shadi Ghasemi 7/27/2019

What is GAM? 7/27/2019

General Linear Model We have some response/variable we wish to study, and believe it to be some function of other variables. y is the variable of interest, assumed to y be normally distributed with mean  and variance σ 2 , and the Xs are the predictor variables/covariates in this scenario. The predictors are multiplied by the coefficients (beta) and summed, giving us the linear predictor, which in this case also directly provides us the estimated fitted values. 7/27/2019

in its basic form it can be very limiting in its assumptions about the data generating process for the variable we wish to study. 7/27/2019

Generalized Linear Model Generalized linear models incorporate other types of distributions (Of the exponential family), and include a link function g(.) relating the mean µ, or stated differently, the estimated fitted values E(y), to the linear predictor Xb, The general form is thus 7/27/2019

In Linear Model we assume a Gaussian (i. e In Linear Model we assume a Gaussian (i.e. Normal) distribution for the response, we assume equal variance for all observations, and that there is a direct link of the linear predictor and the expected value , i.e. Xb = . In fact the typical linear regression model is a generalized linear model with a Gaussian distribution and identity link function. 7/27/2019

Unlike classical linear models, which presuppose a Gaussian (i. e Unlike classical linear models, which presuppose a Gaussian (i.e., normal) distribution and an identity link, the distribution of Y in a GLMs may be any of the exponential family distributions (e.g., Gaussian, Poisson or binomial) and the link function may be any monotonic differentiable function (like logarithm or logit). 7/27/2019

The main improvements of GLMs over LS regression are: (1) the ability to handle a larger class of distributions for the response variable Y. (2) the relationship of the response variable Y to the linear predictor (LP) through the link function g(E(Y)). (3) it incorporates potential solutions (like quasi-likelihood) to deal with overdispersion 7/27/2019

Generalized Additive Model another generalization to incorporate nonlinear forms of the predictors. gives the new setup relating our new, now nonlinear predictor to the expected value, with whatever link function may be appropriate. 7/27/2019

GAMs are parameterized just like GLMs, except that some predictors can be modeled nonparametrically in addition to linear and polynomial terms for other predictors. The probability distribution of the response variable must still be specified, and in this respect, a GAM is parametric. In this sense they are more aptly named semi-parametric models. 7/27/2019

7/27/2019

One can envision the different regression models as being nested within each other, with simple and multiple LS linear regression (SLR and MLR) being the two most limiting cases, and GAMs the most general: 7/27/2019

A crucial step in applying GAMs is to select the appropriate level of the ‘‘smoother’’ for a predictor. This is best achieved by specifying the level of smoothing using the concept of effective degrees of freedom. A reasonable balance must be maintained between the total number of observations and the total number of degrees of freedom used when fitting the model (sum of levels of smoothness used for each predictor). 7/27/2019

What is a Smoother? 7/27/2019

7/27/2019

7/27/2019

For comparison the regular regression fit is also provided. the result from such a fitting process, specifically lowess, or locally weighted scatterplot smoothing. For comparison the regular regression fit is also provided. 7/27/2019

This is the biggest difference from Generalized Linear Model (GLM) This is the biggest difference from Generalized Linear Model (GLM). It allowes an ‘approximation’ with sum of functions, (these functions have separated input variables), not just with one unknown function only. 7/27/2019

Linear functions have , while nonlinear functions produce non-zero values. 7/27/2019

Fitting the Generalized Additive Model There are a number of approaches for the formulation and estimation of additive models. The back-fitting algorithm is a general algorithm that can fit an additive model using any regression-type fitting mechanism. 7/27/2019

Variable selection methods and diagnostics Variable selection is basically the same for all the described regression models, although evaluation criteria like the Akaike Information Criterion can be used with GLMs and GAMs. In all models, one can use predefined rules such as deviance reduction as measured with the chi-square statistic, or approaches that minimize AIC. 7/27/2019

Comparison between GLM and GAM 7/27/2019

7/27/2019

7/27/2019

7/27/2019

7/27/2019

7/27/2019

7/27/2019

7/27/2019