2. Generalized Additive Model
Shadi Ghasemi
7/27/2019

3. What is GAM?
7/27/2019

4. General Linear Model
We have some response/variable we wish to study, and believe it to be some function of other variables.
y is the variable of interest, assumed to y be normally distributed with mean  and variance σ 2 , and the Xs are the predictor variables/covariates in this scenario.
The predictors are multiplied by the coefficients (beta) and summed, giving us the linear predictor, which in this case also directly provides us the estimated fitted values.
7/27/2019

5. in its basic form it can be very limiting in its assumptions about the data generating process for the variable we wish to study.
7/27/2019

6. Generalized Linear Model
Generalized linear models incorporate other types of distributions (Of the exponential family), and include a link function g(.) relating the mean µ, or stated differently, the estimated fitted values E(y), to the linear predictor Xb,
The general form is thus
7/27/2019

7. In Linear Model we assume a Gaussian (i. e
In Linear Model we assume a Gaussian (i.e. Normal) distribution for the response, we assume equal variance for all observations, and that there is a direct link of the linear predictor and the expected value , i.e. Xb = .
In fact the typical linear regression model is a generalized linear model with a Gaussian distribution and identity link function.
7/27/2019

8. Unlike classical linear models, which presuppose a Gaussian (i. e
Unlike classical linear models, which presuppose a Gaussian (i.e., normal) distribution and an identity link, the distribution of Y in a GLMs may be any of the exponential family distributions (e.g., Gaussian, Poisson or binomial) and the link function may be any monotonic differentiable function (like logarithm or logit).
7/27/2019

9. The main improvements of GLMs over LS regression are:
(1) the ability to handle a larger class of distributions for the response variable Y.
(2) the relationship of the response variable Y to the linear predictor (LP) through the link function g(E(Y)).
(3) it incorporates potential solutions (like quasi-likelihood) to deal with overdispersion
7/27/2019

10. Generalized Additive Model
another generalization to incorporate nonlinear forms of the predictors.
gives the new setup relating our new, now nonlinear predictor to the expected value, with whatever link function may be appropriate.
7/27/2019

11. GAMs are parameterized just like GLMs, except that some predictors can be modeled nonparametrically in addition to linear and polynomial terms for other predictors.
The probability distribution of the response variable must still be specified, and in this respect, a GAM is parametric. In this sense they are more aptly named semi-parametric models.
7/27/2019

12. 7/27/2019

13. One can envision the different regression models as being nested within each other, with simple and multiple LS linear regression (SLR and MLR) being the two most limiting cases, and GAMs the most general:
7/27/2019

14. A crucial step in applying GAMs is to select the appropriate level of the ‘‘smoother’’ for a predictor. This is best achieved by specifying the level of smoothing using the concept of effective degrees of freedom.
A reasonable balance must be maintained between the total number of observations and the total number of degrees of freedom used when fitting the model (sum of levels of smoothness used for each predictor).
7/27/2019

15. What is a Smoother?
7/27/2019

16. 7/27/2019

17. 7/27/2019

18. For comparison the regular regression fit is also provided.
the result from such a fitting process, specifically lowess, or locally weighted scatterplot smoothing.
For comparison the regular regression fit is also provided.
7/27/2019

19. This is the biggest difference from Generalized Linear Model (GLM)
This is the biggest difference from Generalized Linear Model (GLM). It allowes an ‘approximation’ with sum of functions, (these functions have separated input variables), not just with one unknown function only.
7/27/2019

20. Linear functions have , while nonlinear functions produce non-zero values.
7/27/2019

21. Fitting the Generalized Additive Model
There are a number of approaches for the formulation and estimation of additive models.
The back-fitting algorithm is a general algorithm that can fit an additive model using any regression-type fitting mechanism.
7/27/2019

22. Variable selection methods and diagnostics
Variable selection is basically the same for all the described regression models, although evaluation criteria like the Akaike Information Criterion can be used with GLMs and GAMs.
In all models, one can use predefined rules such as deviance reduction as measured with the chi-square statistic, or approaches that minimize AIC.
7/27/2019

23. Comparison between GLM and GAM
7/27/2019

24. 7/27/2019

25. 7/27/2019

26. 7/27/2019

27. 7/27/2019

28. 7/27/2019

29. 7/27/2019

30. 7/27/2019