Presentation is loading. Please wait.

Presentation is loading. Please wait.

SADC Course in Statistics Choosing the best model (Session 08)

Similar presentations


Presentation on theme: "SADC Course in Statistics Choosing the best model (Session 08)"— Presentation transcript:

1 SADC Course in Statistics Choosing the best model (Session 08)

2 To put your footer here go to View > Header and Footer 2 Learning Objectives At the end of this session, you will be able to use a simple descriptive approach to select of the most appropriate subset of explanatory variables apply methods of variable selection (based on statistical tests) in a meaningful way to get the best model appreciate the effect on t-probabilities when xs are added or dropped from a model understand dangers of using automatic selection procedures

3 To put your footer here go to View > Header and Footer 3 Example of choosing best set of xs Consider data (fictitious) from a retrospective study of patients surviving less than 4 months after being diagnosed as having acute leukaemia. Objective: To identify factors affecting survival time. Variables were: y = survival time (days) after diagnosis x1 = no: of chemotherapy sessions x2 = total volume of blood transfused x3 = no: of days of hospital care x4 = age of patient (years).

4 To put your footer here go to View > Header and Footer 4 Start with a matrix plot

5 To put your footer here go to View > Header and Footer 5 Summary statistics for all regressions How many possible regression models exist? Example with x1 and x3 to show summaries: Source | SS df MS F Prob>F Model | Residual | Total | No. of parameters fitted (p) = 3 R 2 p = / = Adjusted R 2 p = 1 – / =

6 To put your footer here go to View > Header and Footer 6 Descriptive approach (all regressions) No. of xs p = No. of parameters Terms in model R2R2 Adj. R 2 Res. M.S. None x x x x x1, x x1, x x1, x x2, x x2, x x3, x x1, x2, x x1, x2, x x1, x3, x x2, x3, x x1, x2, x3, x

7 To put your footer here go to View > Header and Footer 7 A descriptive approach… continued Plot R 2 versus no. of parameters (p) in model Which model would you select on the basis of these results?

8 To put your footer here go to View > Header and Footer 8 A descriptive approach… continued Which model would you select on the basis of the residual mean square? Alternatively, plot residual mean square. Small residual mean square is good!

9 To put your footer here go to View > Header and Footer 9 An inferential approach… Use a sequential procedure to select variables that contribute most, and significantly, to the regression model. Three popular methods exist: Forward selection Backward elimination Stepwise regression

10 To put your footer here go to View > Header and Footer 10 Forward selection … Select the best single variable - see slide 6 Ask, Is it contributing significantly? Answer: Yes (see below) y | Coef. Std. Err. t P>|t| x4 | const. | Now consider 2-variable models with x4.

11 To put your footer here go to View > Header and Footer 11 Two-variable models with x y | Coef. Std.Err. t P>|t| x4 | x1 | const.| x4 | x2 | const.| x4 | x3 | const.|

12 To put your footer here go to View > Header and Footer 12 Three-variable models with x4, x y | Coef. Std.Err. t P>|t| x4 | x1 | x2 | const. | x4 | x1 | x3 | const. | Model with x1, x2 and x4 would be selected! - despite x4 now being non-significant!

13 To put your footer here go to View > Header and Footer 13 Backward elimination gives x1,x y | Coef. Std.Err. t P>|t| x1 | x2 | x3 | x4 | x1 | x2 | x4 | x1 | x2 |

14 To put your footer here go to View > Header and Footer 14 Stepwise selection procedure… This is similar to forward selection, but at each stage of the process, all xs in the model are re-assessed to check if those that entered the model at an earlier stage still remain important. Note: Software packages allow automatic use of one of these with pre-specified p- values for selection and deletion of variables. Usually available only with quantitative xs.

15 To put your footer here go to View > Header and Footer 15 Discussion… in small groups Look back at results. What do you observe with the forward and backward procedures. Do they give the same results? Did the selection using forward seem sensible, given that for x4, the p-value =0.205? Can you work out what model would results with a stepwise selection procedures? Is it a good idea to use such automatic selection procedures available in software packages? If not, why not?

16 To put your footer here go to View > Header and Footer 16 Discussion continued… Suppose a medical researcher told you that a model without x2 was not meaningful, how would you proceed with your model selection? What other latent (lurking) variables, measurable or non-measurable, might affect y? What further steps would you undertaken before accepting the final model?

17 To put your footer here go to View > Header and Footer 17 Practical work follows to ensure learning objectives are achieved…


Download ppt "SADC Course in Statistics Choosing the best model (Session 08)"

Similar presentations


Ads by Google