Presentation is loading. Please wait.

Presentation is loading. Please wait.

Introduction to Multilevel Models: Getting started with your own data University of Bristol Monday 31 ST March– Friday 4th April 2008 Course handouts.

Similar presentations

Presentation on theme: "Introduction to Multilevel Models: Getting started with your own data University of Bristol Monday 31 ST March– Friday 4th April 2008 Course handouts."— Presentation transcript:

1 Introduction to Multilevel Models: Getting started with your own data University of Bristol Monday 31 ST March– Friday 4th April 2008 Course handouts

2 Resources Centre for Multilevel Modelling Provides access to general information about multilevel modelling and MlwiN. Includes Multilevel newsletter (free electronic publication) discussion Lemma will include training repository

3 1.0 Introductions Participants introduce themselves : Who you are? Whare are you from?

4 2.00 Multilevel Data Structures Multilevel modelling is designed to explore and analyse data that come from populations which have a complex structure. In any complex structure we can identify atomic units. These are the units at the lowest level of the system. The response or y variable is measured on the atomic units. Often, but not always, these atomic units are individuals. Individuals are then grouped into higher level units, for example, schools. By convention we then say that students are at level 1 and schools are at level 2 in our structure.

5 2.01 Levels, classifications and units A level(eg pupils, schools, households, areas) is made up of a number of individuals units(eg particular pupils, schools etc). The term classification and level can be used somewhat interchangeably but the term level implies a nested hierarchical relationship of units (in which lower units nest in one, and one only, higher-level unit) whereas classification does not.

6 2.02 Two-level hierarchical structures Students St1 St2 St3 St1 St2 St1 St2 St3 St1 St2 St3 St4 School Sc1 Sc2 Sc3 Sc4 Students within schools Unit diagram one node per unit Schoo l Student Classification diagram one node per classification Students within a school are more alike than a random sample of students. This is the clustering effect of schools.

7 2.03 Data frame for student within school example Classifications or levels Respons e Explanatory variables Studen t i School j Student Exam score ij Student previous Examination score ij Student gender ij School type j MState MState FState FPrivate MPrivate MPrivate FState 6* Are schools more variable in their progress for students with low prior attainment? 7 Do students make more progress in private than public schools? 8* Are students in public schools less variable in their progress? 1 Do Males make greater progress than Females? 2 *Does the gender gap vary across schools? 3* Are Males more or less variable in their progress than Females? 4 *What is the between-school variation in students progress? 5 *Is School X (that is a specific school) different from other schools in the sample in its effect? * Requires multilevel model to answer

8 2.04 Variables, levels, fixed and random classifications Schoo l Student Given that school type(state or private) classifies schools, we could redraw our classification diagram as Schoo l Student Schoo l type Do we now have a 3-level multilevel model? We can divide classifications into two types : fixed classifications and random classifications. The distinction has important implications for how we handle the classifying variable in a statistical analysis. For a classification to be a level in a multilevel model it must be a random classification. It turns out that school type is not a random classification.

9 2.05 Random and Fixed Classifications A classification is a random classification if its units can be regarded as a random sample from a wider population of units. For example the students and schools in our example are a random sample from a wider population of students and schools. However, school type or indeed, student gender has a small fixed number of categories. There is no wider population of school types or genders to sample from. Traditional or single level statistical models have only one random classification which classifies the units on which measurements are made, typically people. Multilevel models have more than one random classification.

10 2.06 Other examples of two-level hierarchical structures Repeated measures, panel data Mutivariate response models

11 2.07 Repeated Measures data Classifications or levels ResponseExplanatory variables Stude nt i School j Student Exam score ij Student previous Examination score ij Student gender ij School type j MState MState FState FPrivate MPrivate MPrivate FState In the previous example we have measures on an individual at two occasions a current and a prior test score. We can analyse change (that is progress) by specifying current attainment as the response and prior attainment as a predictor variable. However, when there are measurements on more than two occasions there are advantages as treating occasion as a level nested within individuals. Such a two level strict hierarchical structure is known as a repeated measurement or panel design

12 2.08 Classification, unit diagrams and data framesfor repeated measures structures. P1 P2 P O1 O2 O3 O4 O1 O2 O1 O2 O3 Person Measurement Occasion Classifications or levels ResponseExplanatory variables Occasio n I Person J Height ij Age ij Gender j 11755F 21856F 31957F 12827M 22918M 13885F 23936F 33967F Perso n H- Occ1 H- Occ2 H- Occ3 Age- Occ1 Age- Occ2 Age- Occ3 Gende r F 28291*78*M F Wide form 1 row per individual Long form 1 row per occasion(required by MLwiN)

13 2.09 Repeated Measures Cntd Atomic units are occasions not individuals. Modelling between individual variation in growth, growth curves. In a multilevel repeated measures model data need not be balanced or equally spaced. Explanatory variables can be time invariant (gender) or time varying (age)

14 2.10 Multivariate responses within individuals Sometimes we may wish to model not a single response (y-variable) we may have many. For example, we may wish to consider jointly English and Mathematics exam scores for students as two possibly related responses. We can regard this as a multilevel model with subjects (English and Maths) nested within students St1 St2 St3 St4 … E M E E M M Subject Student A multilevel multivariate response model can estimate the covariance (or correlation) matrix between responses and efficiently handle missing data.

15 2.11 Data frames for multivariate response models Studen t Englis h Score Maths Score Gende r 19575M 255*F 36540F 4*75M Classifications or levels ResponseExplanatory variables Exam Subject I Stude nt J Exam Score ij Eng- Indic ij Math- Indic ij Gender- Eng j Gender- Math j Eng M0 Math M Eng F0 Eng F0 Math M Math M Wide form 1 row per individual Long form 1 row per measurement(required by MLwiN)

16 2.12 Three level structures Student Schoo l Class Student St1 St2 St3 St1 St2 St1 St2 St3 St1 St2 St3 St4 School Sc1 Sc2 Sc3 Class C1 C2 C1 C2 Students:classes:schools MLM allow a different number of students in each class and a different number of classes in each school. Bennett(1976) used a single level model to asses whether teaching styles affected test scores for reading and mathematics at age 11. The results prompted a call for return to traditional or formal teaching methods. This analysis did not take account of the dependency structures in the data: students in a class more similar than a random sample of students, likewise classes in a school. Subsequent ML analysis found the effects of traditional methods non-significant.

17 2.13 Data Frame for 3 level model, students: classes: schools Classifications or levelsResponseExplanatory variables Stude nt I Class j School k Current Exam score ijk Student previous Examination score ijk Student gender ijk Class teaching style jk School type k MFormalState MFormalState FFormalState FInformalState MInformalState MFormalPrivate FFormalPrivate FFormalPrivate MInformalState

18 2.14 Other three level structures Multivariate responses on four health behaviours (drinking, smoking exercise & diet) on individuals within communities, such a design will allow the assessment of the how correlated are the behaviors at the individual level and the community level and to do so taking account of other characteristics at both the individual and community level. We can also can assess the extent to which there are unhealthy communities as well as unhealthy individuals Repeated measures within students within schools. This allows us to look how learning trajectories vary across students and schools. A repeated cross-sectional design with students:cohorts:schools

19 2.15 Repeated cross-sectional design Sc1 Sc2 Sc St1 St2.... St1 St St1 St2... St1 St2... St1 St St1 St2... School Student Cohort Above are unit and classification diagrams where we have Exam scores for groups of students who entered school in 1990 and a further group who entered in The model can be extended to handle an arbitrary number of cohorts. In a multilevel sense we do not have 2 cohort units but 2S cohort units where S is the number of schools.

20 2.16 Four level hierarchical structures student within class within school within LEA multivariate responses within repeated measures within students within schools repeated measures within patients within doctor within hospital people within households within postcode sectors within regions By now you should be getting a feel about how basic random classifications such as people, time, multivariate responses, institutions, families and areas can be combined within a multilevel framework to model a wide variety of nested population structures. Here areas some examples of 4- level nested structures. As a final example of a strict hierarchy we will consider a doubly nested repeated measures structure.

21 2.17 repeated measures within students within cohorts within schools St1 St2... St1 St2.. St1 St2.. St1 St Sc1 Sc2... Cohort Msmnt occ student School O1 O2 O1 O2 O1 O2 O1 O2 Cohorts are now repeated measures on schools and tell us about stability of school effects over time Measurement occasions are repeated measures on students and can tell us about students learning trajectories.

22 2.18 Non-hierarchical structures So far all our examples have been exact nesting with lower level units nested in one and only one higher-level unit. That is we have been dealing with strict hierarchies. But social reality can be more complicated than that. In fact we have found that we need two non-hierarchical structures which in combination with strict hierarchies have been able to deal with all the different types of designs, realities and research questions that we have met Cross-classified structures Multiple membership structures

23 School S1 S2 S3 S4 Pupils P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 Area A1 A2 A3 School S1 S2 S3 S4 Pupils P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 Area A1 A2 A3 School S1 S2 S3 S4 Pupils P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 Area A1 A2 A3 In this structure schools are not nested within areas. For example Pupils 2 and 3 attend school 1 but come from different areas Pupils 6 and 10 come from the same area but attend different schools 2.19 Cross-classified Model Schools are not nested within areas and areas are not nested within schools. School and area are are cross-classified areaschool pupil

24 2.20 Tabulation of students by school and area to reveal across-classified structure area 1area 2area 3 School 1P1,P3P2 School 2P5P4 School 3 P6,P7P8 School 4 P10P9,P11,P12 Pupils P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 Area A1 A 2 A3 School S1 S2 S3 S4 area 1area 2area 3 School 1P1,P2,P3 School 2P4,P5 School 3 P6,P7,P8 School 4 P9,P10,P11, P12 All elements in a row lie in a single column Elements in a row span multiple columns, Elements in a column span multiple rows A1 A2 A3 S1 S2 S3 S4 P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12

25 2.21 Data frame for pupils in a cross-classification of schools and areas Classifications or levels Respons e Explanatory variables Stu den t i Scho ol j Are a k Exam score i(jk) Student gender i(jk) Area IMD k School type j 11175M24State 21271F46State 31191F24State 42268M46Private 52137M24Private 63267F46Private 73282F46State 83385M11State 94354M11Private M46Private F11Private M11Private

26 2.22 Other examples of cross-classified structures Students within a cross-classification of primary school by secondary school. We may have students exam scores at age 16 and wish to assess the relative effects of primary and secondary schools on attainment at age 16 Exam marks within a cross classification of student and examiner, where a students paper is marked by more than one examiner to get an indication of examiner reliability. examiner 1examiner 2examiner 3 student 1m1m2 student 2 m3 m4 Student 3 m5m6 Student 4 m7m8 Patients within a cross-classification of GP practice and hospital. Note in this case we have at most 1 level one unit(mark) per cell in the cross-classification.

27 2.24 Multiple membership models Health outcomes where patients are treated by a number of nurses, patients are multiple members of nurses Students move schools, so some pupils are multiple members of schools. Where atomic units are seen as nested within more than one unit from a higher level classification :. School S1 S2 S3 S4 Pupils P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12P1 P8 Teacher Pupil

28 School S1 S2 S3 S4 Pupils P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 Area A1 A2 A Combining structures: crossed-classifications and multiple membership relationships P1 Lets take the cross-classified model of the previous slide but suppose Pupil 1 moves in the course of the study from residential area 1 to 2 and from school 1 to 2 Now in addition to schools being crossed with residential areas pupils are multiple members of both areas and schools. Pupil 8 has moved schools but sill lives in the same area P8 Student 7 has moved areas but still attends the same school P7

29 2.24 Classification diagram for multiple membership model Student School Area area 1area 2area 3 School 1P1,P3 P1, P2 School 2 P1, P5 P1, P4 School 3P6,P7 P7, P8 School 4P10 P8, P9,P11,P1 2 School S1 S2 S3 S4 Pupils P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 Area A1 A2 A3 P1P8 P7 Students nested within a cross-classification of school by area Students multiple members of schools Students multiple members of areas

30 N1 N2 N3 N4 P1 P2 P3 P4 P5 P6 GP1 GP2 GP3 H1 H Combining structures : crossed, nested and multiple membership relationships Patients can be treated by more than one nurse during their stays in hospital, patients are multiple members of nurses Nurses work in only one hospital therefore nurses are nested within hospitals Patients nested within referring GPs. GPs crossed with nurses. GPs crossed with Hospitals. Patient Nurse Hospital GP practice

31 Pupils P1 P2 P3 P6 P7 P8 P4 P5 P9 P10 P11 P12 School S1 S3 S2 S4 School type state private 2.26 Distinguishing Variables and Levels Classifications or levelsResponseExplanatory Variables Pupil I School j School Type k Pupil Exam Score ijk Previous Exam score ijk Pupil gender ijk 11State7556M 21State7145M 31State9172F 12Private6849F 22Private3736M Etc School type is not a random classification it is a fixed classification, and therefore a variable not as a level. Random classification if units can be regarded as a random sample from a wider population of units. Eg pupils and schools Fixed classsification is a small fixed number of categories. Eg State and Private are not two types sampled from a large number of types, on the basis of these two we cannot generalise to a wider population of types of schools, Similarly gender….. NO!

32 3.0 Work with partner discussing what type of Multilevel data Structure corresponds to participants data(20 mins) Draw free-hand a classification diagram giving labels for units at each level and linking the nodes by appropriate arrows to reflect nested, crossed or MM relationships Complete a schematic data frame for your data set. Either use overheads provided or whatever software you find convenient.

33 4.0 Discussion of Exercise 3.0 Each participant takes 2 minutes to present the multilevel structure for their research problem

34 5: Modelling varying relations: from graphs to equations There are NO general laws in social science that are constant over time and independent of the context in which they are embedded Rein (quoted in King, 1976)

35 5. 1 Varying relations plot Simple set up -Two level model -houses at level 1 nested within districts at level 2 Single continuous response: price of a house Single continuous predictor: size = number of rooms and this variable has been centred around average size of Rooms



38 5. 3 General Structure for Statistical models Response = general trend + fluctuations Response = systematic component + stochastic element Response = fixed + random Specific case: the single level simple regression model ResponseSystematic PartRandom Part Price of Costhouse House=average- + of + residual Pricesized extravariation houseroom InterceptSlopeResidual

39 5 4 Simple regression model is the outcome, price of a house is the predictor, number of rooms, which we shall deviate around its mean, Rooms

40 5.5 Simple regression model (cont) is the price of house i is the individual predictor variable between house variance; conditional on size is the fixed slope term: is the residual/random term, one for every house Summarizing the random term: ASSUME IID Mean of the random term is zero Constant variability (Homoscedasticy) No patterning of the residuals (i.e, they are independent) is the intercept;

41 5.6 Random intercepts model Micro-model Macro-model: index parameter as a response Price of average = citywide + differential for district j price district j Premium Discount Citywide line Differential shift for each district j : index the intercept Substitute macro into micro…….

42 5.7 Random intercepts COMBINED model Substituting the macro model into the micro model yields Grouping the random parameters in brackets Fixed part Random part (Level 2) Random part (Level 1) District and house differentials are independent

43 5.8 The meaning of the random terms Level 1 : within districts between houses Between district variance conditional on size Level 2 : between districts Within district, between-house variation variance conditional on size

44 5.9 Variants on the same model Combined model in full Combined model Is the constant ; a set of 1s In MLwiN Differentials at each level

45 5.10 Random intercepts and random slopes

46 5. 11 Random intercepts and slopes model Micro-mode l Macro-model (Random Intercepts) Note: Index the intercept and the slope associated with a constant, and number of rooms, respectively Macro-model (Random Slopes) Slope for district j = citywide slope + differential slope for district j Substitute macro models into micro mode l…………

47 5.12 Random slopes model Substituting the macro model into the micro model yields Multiplying the parameters with the associated variable and grouping them into fixed and random parameters yields the combined model:

48 5.13 Characteristics of random intercepts & slopes model Fixed part Random part (Level 2) Random part (Level 1)

49 5. 14 Interpreting varying relationship plot through mean and variance- covariances 0++++E -++++D +++++C undefined0+++B 0+0+A CovarianceVarianceMeanVarianceMeanGraph Intercept/Slope terms associated with Slopes terms associated with Predictor Intercepts: terms associated with Constant

50 pre-test attain pre-test attain

51 5.16 Random intercepts and slopes model in MLwiN

52 6.1 Fitting models in MLwiN Work through (at your own pace) Chapter 4 of the manual; Random slopes and intercepts models Dont be afraid to ask!

53 Summary of Sessions 5+6

54 S1: Type of questions tackled by multilevel modelling I 2-level model: current attainment given prior attainment of pupils(1) in schools(2) NB assuming a random sample of pupils from a random samples of schools Do Boys make greater progress than Girls (F) Are boys more or less variable in their progress than girls?(R) What is the between-school variation in progress? (R) Is School X different from other schools in the sample in its effect? (F) continued…….

55 S2: Type of questions tackled by multilevel modelling II Are schools more variable in their progress for pupils with low prior attainment? (R) Does the gender gap vary across schools? (R) Do pupils make more progress in denominational schools?(F) Are pupils in denominational schools less variable in their progress? (R) Do girls make greater progress in denominational schools? (F) (cross-level interaction)

56 S3 Problems with not doing a multilevel analysis Substantive: the between school variability and what factors reduce it are generally of fundamental interest to us. A single level model gives us no estimate of between school variability. Technical: If the higher level clustering is not properly accounted for in the model then inferences we make about other predictors will be incorrect. We will tend to infer a relationship where none exists.

57 S4 : Fixed and Random classifications Random classification Generalization of a level (e.g., schools) Random effects come from a distribution All schools contribute to between-school variance Fixed classification Discrete categories of a variable (eg Gender) Not sample from a population Specific categories only contribute to their respective means

58 S5 When levels become variables... Schools can be treated as a variable and placed in the fixed part; achieved by a set of dummy variables one for each school; target of inference is each specific school; each one treated as an island unto itself No shrinkage but no help; from rest of the data; hence unreliable estimates when no of pupils in school is small Schools in the random part, treated as a level, with generalization possible to ALL schools (or population of schools), in addition can predict specific school effects given that they come from an overall distribution Shrinkage towards zero for unreliably estimated schools

59 S6 Recap on: Random intercept models(parallel lines) x 1ij school 2 school 1 u 0,1 u 0,

60 S7 Recap on: Random intercepts and slopes model x 1ij school 2 school 1 u 1,1 u 0,2 u 1,2

61 S8 Model in Manual : p54

62 S9 Estimates in Manual : p54

63 S10 Plot of predictions for schools: p56

64 7: Multilevel residuals

65 8.0 Contextual effects In the previous sections we found that schools vary in both their intercepts and slopes resulting in crossing lines. The next question is are there any school level variables that can explain this variation? Interest lies in how the outcome of individuals in a cluster is affected by their social contexts (measures at the cluster level). Typical questions are Does school type effect students' exam results? Does area deprivation effect the health status of individuals in the area? In our data set we have a contextual school ability measure, schav. The mean intake score is formed for each school, these means are ranked and the ranks are categorised into 3 groups : low mid 75%

66 8.1 Exploring contextual effects and the tutorial data Does school gender effect exam score by gender? Do boys in boys schools do better or worse or the same compared with boys in mixed schools? Do girls in girls schools do better or worse or the same compared with girls in mixed schools? Does peer group ability effect individual pupil performance? That is given two pupils of equal intake ability do they progress differently depending on whether they are educated in a low, mid or high ability peer group?

67 8.2 School gender effects girl boysch girlsch boy/mixed school = girl/mixed school = boy/boy school = girl/girl school =

68 8.3 Peer group ability effects The effect of peer group ability is modelled as being constant across gender, school gender and standlrt. For example, comparing peer group ability effects for boys in mixed schools and boys in boys schools: *standlrt ij : boy,mixed school,low(reference group) : boy,mixed school,mid : boy mixed school high boy,boy school,low boy,boy school,mid boy,boy school,high } Boys school =0.187

69 8.4 Cross level interactions There may be interactions between school gender, peer group ability, gender and standlrt. An interesting interaction is between peer group ability and standlrt. This tests whether the effect of peer group differs across the standlrt intake spectrum. For example, being in a high ability group may have a different effect for pupils of different ability. This is a cross level interaction because it is the interaction between a pupil level variable(standlrt) and a school level variable(schav).

70 8.5 Cross level interactions contd Note that high ability pupils (standlrt=2.6) score nearly 1sd higher if they are educated in high rather than low ability peer groups. Which leads to three lines for the low,mid and high groupings standlrt ij :low ( )+( ) standlrt ij :mid ( )+( ) standlrt ij :high

71 9.1 Repeated measures. We may have repeated measurements on individuals, for example: a series of heights or test scores. Often we want to model peoples growth. We can fit this structure as a multilevel model with repeated measurements nested within people. That is: Occasion O1 O2 O3 O1 O2 O1 O2 O3 O4 Person P1 P2 P3…

72 9.2 Advantages of fitting repeated measures models in a multilevel framework Fitting these structures using a multilevel model has the advantages that data can be Unbalanced (individuals can have different numbers of measurement occasions) Unequally spaced (different individuals can be measured at different ages) As opposed to traditional multivariate techniques which require data to be balanced and equally spaced. Again the multilevel model requires response measurements are MCAR or MAR.

73 9.3 An example from the MLwiN user guide Repeated measures model for childrens reading scores This (random intercepts model) models growth as a linear process with individuals varying only in their intercepts. That is for the 405 individuals in the data set The global mean is predicted by { The jth childs growth curve is predicted by

74 9.4 Further possibilities for repeated measures model We can go on and fit a random slope model. Which in this case allows the model to deal with children growing at different rates. We can fit polynomials in age to allow for curvilinear growth. We can also try and explain between individual variation in growth by introducing child level variables. If appropriate we can include further levels of nesting. For example, if children are nested within schools we could fit a 3 level model [occasions:children:schools]. We could then look to see if childrens patterns of growth varied across schools.

75 10.0 Variance functions or modelling heteroscedasticity Tabulating normexam by gender we see that the means and variances for boys and girls are (–0.140 and 1.051) and (0.093 and 0.940). We may want to fit a model that estimates separate variances for boys and girls. The notation we have been using so far assumes a common intercept( 0 ) and a single set of student residuals, e i, with a common variance e 2. We need to use a more flexible notation to build this model.

76 10.1 Working with general notation in MLwiN A model with no variables specified in general notation looks like this. A new first line is added stating that the response variable follows a Normal distribution. We now have the flexibility to specify alternative distributions for our response. We will explore these models later. The 0 coefficient now has an explanatory x 0 associated with it. The values x 0 takes determines the meaning of the 0 coefficient. If x 0 is a vector of 1s then 0 will estimate an intercept common to all individuals, in the absence of other predictors this would be the overall mean. If x 0 variable, say 1 for boys and 0 for girls, then 0 will estimate the mean for boys.

77 10.2 A simple variance function The new notation allows us to set up this simple model where x 0i is a dummy variable for boy and x 1i is a dummy variable for girl. This model estimates separate means and variances for the two groups. This is an example of a variance function because the variance changes as a function of explanatory variables. The function is :

78 10.3 Deriving the variance function We arrive at the expression (1)

79 10.4 Variance functions at level 2 The notion of variance functions is powerful and not restricted to level 1 variances. The random slopes model fitted earlier produces the following school level predictions which show school level variability increasing with intake score. The model Can be rewritten as So the between school variance is

80 10.5 Two views of the level 2 variance Given x 0 = [1], we have Which shows that the level 2 variance is polynomial function of x 1ij View 1: In terms of school lines predicted intercepts and slopes varying across schools. View 2 : In terms of a variance function which shows how the level 2 variance changes as a function of 1 or more explanatory variables.

81 10.6 Elaborating the level 1 variance Maybe the student level departures around their schools summary lines are not constant. Note at level 2 we have 2 interpretations of level 2 random variation, random coefficients (varying slopes and intercepts across level 2 units) and variance functions. In each level 1 unit, by definition, we only have one point, therefore the first interpretation does not exist because you cannot have a slope given a single data point. 2 schools 2 students

82 10.7 Variance functions at level 1 If we allow standlrt(x 1ij ) to have a random term at level 1, we get So the student level variance is now: The resulting graph shows decreasing level 1 variance wrt standlrt extenuates the importance of school level factors driving variation in the outcome score, particularly for high ability pupils

83 10.8 Modelling the mean and variance simultaneously In our model The global mean is predicted by The j th school mean is predicted by The student level variance is The school level variance is Where as ordinary regression: estimates the global relationship and has a single catch all bucket for the variance.

84 11.00 Applied Paper – Example of Variance functions Understanding the sources of differential parenting: the role of child and family level effects. Jenny Jenkins, Jon Rasbash and Tom OConnor Developmental Psychology 2003(1)

85 11.01 Mapping multilevel terminology to psychological terminology Level 2 : Family, shared environment Variables : family ses, marital problems Level 1 : Child, non-shared environment, child specific Variables : age, sex, temperament

86 11.02 Background Recent studies in developmental psychology and behavioural genetics emphasise non-shared environment is much more important in explaining childrens adjustment than shared environment has led to a focus on non-shared environment.(Plomin et al, 1994; Turkheimer&Waldron, 2000) Has this meant that we have ignored the role of the shared family context both empirically and conceptually?

87 11.03 questions One key aspect of the non-shared environment that has been investigated is differential parental treatment of siblings. Differential treatment predicts differences in sibling adjustment What are the sources of differential treatment? Child specific/non-shared: age, temperament, biological relatedness Can family level shared environmental factors influence differential treatment?

88 Parents have a finite amount of resources in terms of time, attention, patience and support to give their children. In families in which most of these resources are devoted to coping with economic stress, depression and/or marital conflict, parents may become less consciously or intentionally equitable and more driven by preferences or child characteristics in their childrearing efforts. Henderson et al This is the hypothesis we wish to test. We operationalised the stress/resources hypothesis using four contextual variables: socioeconomic status, single parenthood, large family size, and marital conflict The Stress/Resources Hypothesis Do family contexts(shared environment) increase or decrease the extent to which children within the same family are treated differently?

89 Previous analyses, in the literature exploring the sources of differential parental treatment ask mother to rate two siblings in terms of the treatment(positive or negative) they give to each child. The difference between these two treatment scores is then analysed. This approach has several major limitations… How differential parental treatment has been analysed

90 11.06 The sibling pair difference difference model, for exploring determinants of differential parenting Where y 1i and y 2i are parental ratings for siblings 1 and 2 in family I x 1i is a family level variable for example family ses Problems One measurement per family makes it impossible to separate shared and non-shared random effects. All information about magnitude of response is lost (2,4) are the same as (22,24) It is not possible to introduce level 1(non-shared) variables since the data has been aggregated to level 2. Family sizes larger than two can not be handled.

91 11.07 With a multilevel model… Where y ij is the jth mothers rating of her treatment of her ith child x 1ij are child level(non-shared variables), x 2j are child level(shared variables) u j and e ij are family and child(shared and non-shared environment) random effects. Note that the level 1 variance is now a measure of differential parenting

92 11.08 Advantages of the multilevel approach Can handle more than two kids per family Unconfounds family and child allowing estimation of family and child level fixed and random effects Can model parenting level and differential parenting in the same model.

93 11.09 Overall Survey Design National Longitudinal Survey of Children and Youth (NLSCY) Statistics Canada Survey, representative sample of children across the provinces Nested design includes up to 4 children per family PMK respondent 4-11 year old children Criteria: another sibling in the age range, be living with at least one biological parent, 4 years of age or older 8, 474 children 3, 860 families 4 child =60, 3 child=630, 2 child=3157

94 11.10 Measures of parental treatment of child Derived form factor analyses.. PMK report of positive parenting: frequency of praise of child, talk or play focusing on child, activities enjoyed together =.81 PMK report of negative parenting: frequency of disapproval, annoyance, anger, mood related punishment =.71 Will talk today about positive parenting PMK is parent most known to the child.

95 Child specific factors Age Gender Child position in family Negative emotionality Biological relatedness to father and mother Family context factors Socioeconomic status Family size Single parent status Marital dissatisfaction

96 11.12 Model 1: Null Model The base line estimate of differential parenting is 3.8. We can now add further shared and non-shared explanatory variables and judge their effect on differential parenting by the reduction in the level 1 variance.

97 11.13 Model 2 : expanded model

98 11.14 positive parenting Child level predictors Strongest predictor of positive parenting is age. Younger siblings get more attention. This relationship is moderated by family membership. Non-bio mother and Non_bio father reduce positive parenting Oldest sibling > youngest sibling > middle siblings Family level predictors Household SES increases positive parenting Marital dissatisfaction, increasing family size, mixed or all girl sib- ships all decrease positive parenting Lone parenthood has no effect.

99 11.15 Differential parenting Modelling age reduced the level 1 variance (our measure of differential parenting) from 3.8 to 2.3, a reduction of 40%. Other explanatory variables both child specific and family(shared environment) provide no significant reduction in the level 1 variation. Does this mean that there is no evidence to support the stress/resources hypothesis.

100 11.16 Testing the stress/resource hypothesis The mean and the variance are modelled simultaneously. So far we have modelled the mean in terms of shared environment but not the variance. We can elaborate model 2 by allowing the level 1 variance to be a function of the family level variables household socioeconomic status, large family size, and marital conflict. That is Reduction in the deviance with 7df is 78.

101 11.17 Graphically …

102 11.18 Modelling the mean and variance simultaneously We show a possible pattern of how the mean, within family variance and between family variance might behave as functions of HSES in the schematic diagram below. HSES positive parenting Here are 5 families of increasing HSES(in the actual data set there are 3900 families. We can fit a linear function of SES to the mean. The family means now vary around the dashed trend line. This is now the between family variation; which is pretty constant wrt HSES However, the within family variation(measure of differential parenting) decreases with HSES – this supports the SR hypothesis.

103 12 Multivariate response models We may have data on two or more responses we wish to analyse jointly. For example, we may have english and maths scores on pupils within schools. We can consider the response type as a level below pupil. S1 S2… P1 P2 P3 P4…. E M E M

104 12.01 Rearranging data schoolpupilEnglishMaths Often data comes like this with one row per person For MLwiN to analyse the data we require the data matrix to have one row per level 1 unit. That is one row per response measurement x 0 is 1 if response for this record is English, 0 otherwise x 1 is 1 if response for this record is Maths, 0 otherwise schoolpupilsubjectx0x0 x1x

105 12.02 Writing down the model Where y 1j is the english score for student j and y 2j is the maths score for student j. The means and variances for english and maths( 0, 1, u0 2, u1 2 ) are estimated. Also the covariance between maths and english, u01 is estimated. Note there is no level 1(e ij ) variance. This can be seen if we consider the picture for one pupil.

106 12.03 Advantages of framing a multivariate response model as a multilevel model The model has the following advantages over traditional multivariate techniques: It can deal with missing responses-provided response data is missing completely at random(MCAR) or missing at random(MAR) that is missingness is related to explanatory variables in the model. Covariates can be added giving us the conditional covariance matrix of the responses. Further levels can be added to the model

107 12.04 Example from MLwiN user guide mean for written = 46.8 Variance(written) = mean for coursework = Variance(coursework) = covariance(written, coursework) = pupils have two responses : written and coursework That is we have two means and a covariance matrix, which we could get from any stats package. However, the data are unbalanced. Of the 1905 pupils 202 are missing a written response and 180 are missing a coursework response.

108 12.05 Further extensions We can add further explanatory variables. For example, female. We see that females do better for coursework than males and worse than males on written exams males do better on written exams. We can add further levels. Here we partition the covariance structure into student and school components.

109 13.0 MCMC estimation in MlwiN MCMC estimation is a big topic and is given a pragmatic and cursory treatment here. Interested students are referred to the manual MCMC estimation in MLwiN available from In the workshop so far you have been using IGLS (Iterative Generalised Least Squares) algorithm to estimate the models.

110 13.1 IGLS versus MCMC IGLS MCMC Fast to compute Slower to compute Deterministic convergence-easy to judge Stochastic convergence-harder to judge Uses mql/pql approximations to fit discrete response models which can produce biased estimates in some cases Does not use approximations when estimating discrete response models, estimates are less biased In samples with small numbers of level 2 units confidence intervals for level 2 variance parameters assume Normality, which is inaccurate. In samples with small numbers of level 2 units Normality is not assumed when making inferences for level 2 variance parameters Can not incorporate prior information Can incorporate prior information Difficult to extend to new models Easy to extend to new models

111 13.2 Bayesian framework MCMC estimation operates in a Bayesian framework. A bayesian framework requires one to think about prior information we have on the parameters we are estimating and to formally include that information in the model. We may make the decision that we are in a state of complete ignorance about the parameters we are estimating in which case we must specify a so called uninformative prior. The posterior distribution for a paremeter given that we have observed y is subject to the following rule: p( |y) p(y| )p( ) Where p( |y) is the posterior distribution for given we have observed y p(y| ) is the likelihood of observing y given p( ) is the probability distribution arising from some statement of prior belief such as we believe ~N(1,0.01). Note that we believe ~N(1,1) is a much weaker and therefore less influential statement of prior belief.

112 13.3 Applying MCMC to multilevel models In a two level variance components model we have the following unknowns There joint posterior is Likelihood – what the data says-estimated from data Prior belief- supplied by the researcher Posterior – final answers- a combination of likelihood and priors

113 13.4 Gibbs sampling Evaluating the expression for the joint posterior with all the parameters unknown is for most models, virtually impossible. However, if we take each unknown parameter in turn and temporarily assume we know the values of the other parameters, then we can simulate from the so called conditional posterior distribution. The Gibbs sampling algorithm cycles through the following simulation steps. First we assume some starting values for our unknown parameters :

114 13.5 Gibbs sampling cntd We now have updated all the unknowns in the model. This process is repeated many times until eventually we converge on the distribution of each of the unknown parameters.

115 13.6 IGLS vs MCMC convergence IGLS algorithm converges, deterministically to a distribution. MCMC algorithm converges on a distribution. Parameter estimates and intervals are then calculated from the simulation chains.

116 13.7 Other MCMC issues By default MLwiN uses flat, uniformative priors see page 5 of MCMC estimation in MLwiN (MEM) For specifying informative priors see chapter 6 of MEM. For model comparison in MCMC using the DIC statistic see chapters 3 and 4 MEM. For description of MCMC algorithms used in MLwiN see chapter 2 of MEM.

117 13.8 When to consider using MCMC in MLwiN Some of the more advanced models in MLwiN are only available in MCMC. For example, factor analysis (chapter 19), measurement error in predictor variables (chapter 14) and CAR spatial models (chapter 16) Other models, can be fitted in IGLS but are handled more easily in MCMC such as multiple imputation (chapter 17), cross-classified(chapter 14) and multiple membership models (chapter 15). If you have discrete response data – binary, binomial, multinomial or Poisson (chapters 11, 12, 20 and 21). Often PQL gives quick and accurate estimates for these models. However, it is a good idea to check against MCMC to test for bias in the PQL estimates. If you have few level 2 units and you want to make accurate inferences about the distribution of higher level variances. All chapter references to MCMC estimation in MLwiN.

118 14.0 Generalised Multilevel Models 1 : Binary Responses and Proportions

119 14.1 Generalised multilevel models So Far Response at level 1 has been a continuous variable and associated level 1 random term has been assumed to have a Normal distribution Now a range of other data types for the response All can be handled routinely by MLwiN Achieved by 2 aspects a non-linear link between response and predictors a non-Gaussian level 1 distribution

120 14.2 Typology of discrete responses

121 14.3 Focus on modelling proportions Proportions eg death rate; employment rate; can be conceived as the underlying probability of dying; probability of being employed Four important attributes of a proportion that MUST be taken into account in modelling (1)Closed range: bounded between 0 and 1 (2)Anticipated non-linearity between response and predictors; as predicted response approaches bounds, greater and greater change in x is required to achieve the same change in outcome; examination analogy (3)Two numbers: numerator subset of denominator (4)Heterogeneity: variance is not homoscedastic; two aspects (a) the variance depends on the mean; as approach bound of 0 and 1, less room to vary ie Variance is a function of the predicted probability (b) the variance depends on the denominator; small denominators result in highly variableproportions

122 14.4 Modelling Proportions Linear probability model: that is use standard regression model with linear relationship and Gaussian random term But 3 problems (1) Nonsensical predictions: predicted proportions are unbounded, outside range of 0 and 1 (2) Anticipated non-linearity as approach bounds (3) Heteogeneity: inherent unequal variance dependent on mean and on denominator Logit model with Binomial random term resolves all three problems (could use probit, clog-clog)

123 14.5 The logistic model: resolves problems 1 & 2 Models not the proportion but a non-linear transformation of it (solves problems 1+2) The relationship between the probability and predictor(s) can be represented by a logistic function, that resembles a S- shaped curve

124 L = LOGe(p/ (1-p)) L = Logit = the log of the odds p = proportion having an attribute 1-p =proportion not having the attribute p/(1-p) = the odds of having an attribute compared to not having an attribute As p goes from 0 to 1, L goes from minus to plus infinity, so if model L, cannot get predicted proportions that lie outside 0 and 1; (ie solves problem 1) Easy to move between proportions, odds and logits 14.6 The Logit transformation

125 14.7 Proportions, Odds and Logits

126 14.8 The logistic model where e is the base of the natural logarithm linearized by the logit transformation(log = natural logarithm) The underlying probability or proportion is non- linearly related to the predictor

127 14.9 The logistic model: key characteristics The logit transformation produces a linear function of the parameters. Bounded between 0 and 1 Thereby solving problems 1 and 2

128 14.10 Solving problem 3:assume Binomial variation Variance of the response in logistic models is presumed to be binomial: Ie depends on underlying proportion and the denominator In practice this is achieved by replacing the constant variable at level 1 by a binomial weight, z, and constraining the level-1 variance to 1 for exact binomial variation The random (level-1) component can be written as

129 14.11 Multilevel Logistic Model Underlying proportions/probabilities, in turn, are related to a set of individual and neighborhood predictors by the logit link function Linear predictor of the fixed part and the higher-level random part Assume observed response comes from a Binomial distribution with a denominator for each cell, and an underlying probability/proportion

130 14.12 Estimation 1 Quasi-likelihood (MQL/PQL – 1 st and 2 nd order) –model linearised and IGLS applied. –1st or 2nd order Taylor series expansion (to linearise the non-linear model) – MQL versus PQL are higher-level effects included in the linearisation –MQL1 crudest approximation. Estimates may be biased downwards (esp. if within cluster sample size is small and between cluster variance is large eg households). But stable. –PQL2 best approximation, but may not converge. –Tip: Start with MQL1 to get starting values for PQL.

131 14.13 Estimation 2 MCMC methods: get deviance of model (DIC) for sequential model testing, and good quality estimates even where cluster size is small; start with MQL1 and then switch to MCMC

132 14.14 Variance Partition Coefficient y ij ~Binomial( ij,1) logit( ij | x ij, u j, ) = + x 1ij + u j Var(u j ) = u 2 var(y ij - ij ) = ij (1- ij ) Level 1 variance is function of predicted probability The level 2 variance u 2 is on the logit scale and the level 1 variance var(y ij - ij ) is on the probability scale so they can not be directly compared. Also level 1 variance depends on ij and therefore x 1ij. Possible solutions include i) set the level 1 variance = variance of a standard logistic distribution ; ii) simulation method

133 14.15 VPC 1: Threshold Model But this ignores the fact that the level –1 variance is not constant, but is function of the mean probability which depends on the predictors in the fixed part of the model

134 14.16 VPC 2: Simulation Method

135 14.17 Multilevel modelling of binary data Exactly the same as proportions except The response is either 1 or 0 The denominator is a set of 1s So that a Yes is 1 out of 1, while a No is 0 out of 1

136 14.18 Chapter 9 of Manual: Contraceptive Use in Bangladesh 2867 women nested in 60 districts y=1 if using contraception at time of survey, y=0 if not using contraception Covariates: age (mean centred), urban residence (vs. rural)

137 14.19 Random Intercept Model: PQL2

138 14.20 Variance Partition Coefficient Threshold model approach0.21/( )=0.060 Simulation approach (M=5000, mean age) Urban0.050 Rural0.045

139 14.21 MLwiN Gives UNIT or (subject) SPECIFIC Estimates the fixed effects conditional on higher level unit random effects, NOT the POPULATION-AVERAGE estimates iethe marginal expectation of the dependent variables across the population "averaged " across the random effects In non-linear models these are different and the PA will generally be smaller than US, especially as size of random effects grows Can derive PA fom US but not vice-versa (next version give both)

140 14.22 Unit specific / Population average Probability of adverse reaction against dose Left: subject-specific; big differences between subjects for middle dose (the between –patient variance is large), Right is the population average dose response curve, Subject-specific curves have a steeper slope in the middle range of the dose variable

141 15.0 Multilevel Multinomial Models Logistic models handle the situation where we have a binary response(two response categories eg alive/dead or pass/fail.) Where we have a response variable with more than two categories we use multinomial models. Two types of multinomial response: First we deal with unordered multinomial responses Ordered – attitude scales(strongly disagree...strongly agree) or exam grades. Unordered – eg voting prerference(lab, tory, libdem, other) or cause of death.

142 15.1 Extending a binary to a multinomial model log[ i / (1- i )]= x 1i..... (1) Take a binary variable (y i ) which is 1 if an individual votes tory 0 otherwise. The underlying probability of individual i voting tory is i. We model the log odds of voting tory as a function of explanatory variables Lets call i = 1i = prob of individual i voting tory and 2i =(1- 1i )= prob of individual i not voting tory We can now write (1) as log[ 1i / 2i ]= x 1i.....

143 15.2 Moving to more than two response categories Suppose now that y i can take three values {1,2,3} vote tory, vote labour, vote lib dem. Now log[ 1i / 3i ]= x 1i..... log[ 2i / 3i ]= x 1i i is probability of individual i votes tory 2i is probability of individual i votes labour 3i is probability of individual i votes lib dem Now we must choose a reference category, say vote lib dem, and model the log odds of all remaining categories against the reference category. Therefore with t categories we need t-1 equations to model this set of log odds ratios. In our case

144 15.3 Notation For s = 1 log[ i (1) / i (3) ]= (1) x 1i..... log[ 1i / 3i ]= x 1i..... log[ 2i / 3i ]= x 1i log[ i (s) / i (t) ]= s) (s) x 1i s=1,..,t-1 The MLwiN software uses the notation Often in papers you will see the more succinct notational form For s = 2 log[ i (2) / i (3) ]= (2) x 1i..... Which becomes

145 15.4 Interpretation(odds ratios) log[ 1i / 3i ]= x 1i..... log[ 2i / 3i ]= x 1i..... We can interpret as with logistic regression. In the political example, {1,2,3} vote tory, vote labour, vote lib dem. is the change in the log odds of voting tory as opposed to lib dem for 1 unit increase in x 1i. is the change in the log odds of voting labour as opposed to lib dem for 1 unit increase in x 1i. and expo( gives odds ratios

146 15.5 Interpretation(probabilities) log[ 1i / 3i ]= x 1i..... log[ 2i / 3i ]= x 1i..... Probability of voting tory for individual i Probability of voting labour for individual i Or in general notation

147 15.6 Multilevel Multinomial models Suppose the individuals in the voting example are clustered into constituencies and we wish to include constituency effects in our model. We include intercept level residuals for each log odds equiation in our model log[ 1ij / 3ij ]= x 1ij +u 0j log[ 2ij / 3ij ]= + x 1ij u 2j u 0j is the effect of the constituency j on the log odds of voting tory as opposed to lib dem. So if u 0j is 1 the log odds of voting tory as opposed to lib dem increase by 1 compared to u 0j where u 0j = 0 (the is average constituency) Likewise u 2j is the effect of the constituency j on the log odds of voting labour as opposed to lib dem.

148 15.7 Variance of level 2 random effects log[ 1ij / 3ij ]= +u 0j x 1ij log[ 2ij / 3ij ]= u 2j + x 1ij u0ju2ju0ju2j ~N( 0, u ) u = u0 u02 u2 u0 is the betwen constituency variance of the vote tory:lib dem log odds ratio u2 is the between constituency variance of the vote labour:lib dem log odds ratio u02 is the constituency level covariance between tory and labour constituency level effects. A negative covariance means there is a tendency for constituencies where labour do well as opposed to libdems; for tories to do badly as opposed to the libdems and vice versa.

149 16.0 Ordered categorical data Where there is an underlying ordering to the categories a convenient parameterisation is to work with cumulative probabilities that an individual crosses a threshold. For example, with exam grades GradeprobabilityThresholdCumulative probability D 1i D C 2i C: (C,D) 1i + 2i B 3i B:(B,C,D) 1i + 2i + 3i A 4i A:(A,B,C,D) 1i + 2i + 3i + 4i=1 With an ordered multinomialwe work with the set of cumulative probabalities As before with t categories in the the model has t-1 categories.

150 16.1 Writing the ordered multinomial model log( log odds of D log( log odds of C log( log odds of B The threshold probability k are given by antilogit( k ) We must have to ensure

151 16.2 Adding covariates to the model log( h i log odds of D log( h i log odds of C log( h i log odds of B h i = x 1i..... Note that the covariates h i are the same for each of the response threshold categories. x 1i log odds of D x 1i log odds of C x 1i log odds of B Log odds xixi This means that the log odds ratios and odds ratios for threshold category membership are independent of the predictor variables.

152 16.3 Proportional odds models Sio far we have assumed that the odds ratios of response category membership remains constant wrt predictor variables. This is known as the proportional odds assumption. We can test the assumption that odds ratios of response category membership being independent of predictor variables by fitting: log( x 1i log odds of D log( x 1i log odds of C log( x 1i log odds of B Now if our assumptions are correct will be very similar. We can formally test using the intervals and tests window

153 16.4 Multilevel ordered multinomial models log( h i log odds of D log( h i log odds of C log( h i log odds of B h i = x 1i +u 0j u 0j is a random effect for school j, which shifts all the threshold probabilities equally for all kids in school j. Again odds ratios for category membership are independent of u 0j k + x 1i + u 0j for +ve u 0j k + x 1i + u 0j for -ve u 0j Log odds xixi k + x 1i

154 16.5 Higher level variances u 0j ~N(0, u0 ) The greater u0 The greater the variability in the school level shifts in the response threshold probabilities.

155 17 Non-hierarchical multilevel models Two types : Cross-classified models Multiple membership models

156 17.01 Cross-classification For example, hospitals by neighbourhoods. Hospitals will draw patients from many different neighbourhoods and the inhabitants of a neighbourhood will go to many hospitals. No pure hierarchy can be found and patients are said to be contained within a cross-classification of hospitals by neighbourhoods : nbhd 1nbhd 2Nbhd 3 hospital 1xxx hospital 2xx hospital 3xxx hospital 4xxxx Hospital H1 H2 H3 H4 Patient P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 Nbhd N1 N2 N3

157 17.02 Other examples of cross-classifications pupils within primary schools by secondary schools patients within GPs by hospitals interviewees within interviewers by surveys repeated measures within raters by individual(e.g. patients by nurses)

158 17.03 Notation With hirearchical models we have subscript notation that has one subscript per level and nesting is implied reading from left. For example, subscript pattern ijk denotes the ith level unit within the jth level 2 unit within the kth level 3 unit. If models become cross-classified we use the term classification instead of level. With notation that has one subscript per classification, that captures the relationship between classifications, notation can become very cumbersome. We propose an alternative notation that only has a single subscript no matter how many classifications are in the model.

159 17.04 Single subscript notation Hospital H1 H2 H3 H4 Patient P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 Nbhd N1 N2 N3 inbhd(i)hosp(i) We write the model as Where classification 2 is nbhd and classification 3 is hospital. Classification 1 always corresponds to the classification at which the response measurements are made, in this case patients. For patients 1 and 11 equation (1) becomes:

160 17.05 Classification diagrams Hospital Patient Neighbourhood Hospital Patient Neighbourhood Nested structure where hospitals are contained within neighbourhoods Cross-classified structure where patients from a hospital come from many neighbourhoods and people from a neighbourhood attend several hospitals. In the single subscript notation we loose informatin about the relationship(crossed or nested) between classifications. A useful way of conveying this informatin is with the classification diagram. Which has one node per classification and nodes linked by arrows have a nested relationship and unlinked nodes have a crossed relationship.

161 17.06 Data example : Artificial insemination by donor 1901 women 279 donors 1328 donations ovulatory cycles response is whether conception occurs in a given cycle In terms of a unit diagram: Donor Woman Cycle Donation Or a classification diagram:

162 17.07 Model for artificial insemination data artificial insemination We can write the model as ParameterDescriptionEstimate(se) intercept-4.04(2.30) azoospermia *0.22(0.11) semen quality0.19(0.03) womens age> (0.14) sperm count0.20(0.07) sperm motility0.02(0.06) insemination to early-0.72(0.19) insemination to late-0.27(0.10) women variance1.02(0.21) donation variance0.644(0.21) donor variance0.338(0.07) Results:

163 17.08 Multiple membership models Where level 1 units are members of more than one higher level unit. For example, Pupils change schools/classes and each school/class has an effect on pupil outcomes Patients are seen by more than one nurse during the course of their treatment

164 17.09 Notation Note that nurse(i) now indexes the set of nurses that treat patient i and w (2) i,j is a weighting factor relating patient i to nurse j. For example, with four patients and three nurses, we may have the following weights n1(j=1)n2(j=2)n3(j=3) p1(i=1)0.50 p2(i=2)100 p3(i=3)00.5 p4(i=4)0.5 0 Here patient 1 was seen by nurse 1 and 3 but not nurse 2 and so on. If we substitute the values of w (2) i,j, i and j. from the table into (1) we get the series of equations :

165 17.10 Classification diagrams for multiple membership relationships Double arrows indicate a multiple membership relationship between classifications patient nurse We can mix multiple membership, crossed and hierarchical structures in a single model patient nurse hospital GP practice Here patients are multiple members of nurses, nurses are nested within hospitals and GP practice is crossed with both nurse and hospital.

166 17.11 Example involving, nesting, crossing and multiple membership – Danish chickens Production hierarchy 10,127 child flocks 725 houses 304 farms Breeding hierarchy 10,127 child flocks 200 parent flocks Child flock House Farm Parent flock As a unit diagram: As a classification diagram:

167 17.12 Model and results ParameterDescriptionEstimate(se) intercept-2.322(0.213) (0.162) (0.187) hatchery (0.255) hatchery (0.252) hatchery (0.388) parent flock variance0.895(0.179) house variance0.208(0.108) farm variance0.927(0.197) Results:

168 All the children born in the Avon area in 1990 followed up longitudinally Many measurements made including educational attainment measures Children span 3 school year cohorts(say 1994,1995,1996) Suppose we wish to model development of numeracy over the schooling period. We may have the following attainment measures on a child : m1 m2 m3 m4 m5 m6 m7 m8 primary school secondary school Alspac data

169 Measurement occasions within pupils M. Occasion PupilP. Teacher At each occasion there may be a different teacher P School Cohort Pupils are nested within primary school cohorts Primary schoolArea All this structure is nested within primary school Pupils are nested within residential areas Structure for primary schools

170 M. occasions PupilP. Teacher P School Cohort Primary school Area Nodes directly connected by a single arrow are nested, otherwise nodes are cross- classified. For example, measurement occasions are nested within pupils. However, cohort are cross-classified with primary teachers, that is teachers teach more than one cohort and a cohort is taught by more than one teacher. T1T2T3 Cohort Cohort Cohort A mixture of nested and crossed relationships

171 It is reasonable to suppose the attainment of a child in a particualr year is influenced not only by the current teacher, but also by teachers in previous years. That is measurements occasions are multiple members of teachers. m1 m2 m3 m4 t1 t2 t3 t4 M. occasions PupilP. Teacher P School Cohort Primary school Area We represent this in the classification diagram by using a double arrow Multiple membership

172 If pupils move area, then pupils are no longer nested within areas. Pupils and areas are cross-classified. Also it is reasonable to suppose that pupils measured attainments are effected by the areas they have previously lived in. So measurement occasions are multiple members of areas M. occasions Pupil P. Teacher P School Cohort Primary school Area M. occasions Pupil P. Teacher P School Cohort Primary school Area Classification diagram without pupils moving residential areas Classification diagram where pupils move between residential areas BUT… What happens if pupils move area?

173 Classification diagram where pupils move between areas but not schools If pupils move schools they are no longer nested within primary school or primary school cohort. Also we can expect, for the mobile pupils, both their previous and current cohort and school to effect measured attainments M. occasions Pupil P. Teacher P School Cohort Primary school Area M. occasions Pupil P. TeacherP School Cohort Primary school Area Classification diagram where pupils move between schools and areas If pupils move area they will also move schools

174 And secondary schools… M. occasions Pupil P. TeacherP School Cohort Primary school Area We could also extend the above model to take account of Secondary school, secondary school cohort and secondary school teachers If pupils move area they will also move schools cntd

175 Remember we are partitioning the variability in attainment over time between primary school, residential area, pupil, p. school cohort, teacher and occasion. We also have predictor variables for these classifications, eg pupil social class, teacher training, school budget and so on. We can introduce these predictor variables to see to what extent they explain the partitioned variability Other predictor variables

176 18 Significance testing and model comparison Individual fixed part and random coefficients at each level Simultaneous and complex comparisons Comparing nested models: likelihood ratio test Use of Deviance Information Criteria

177 18.1 Individual coefficients Akin to t tests in regression models Either specific fixed effect or specific variance-covariance component –H0: is 0;H1: is not 0 Procedure: Divide estimated coefficient by their standard error –Judge against a z distribution –If ratio exceeds 1.96 then significant at 0.05 level Approximate procedure; asymptotic test, small sample properties not well- known. OK for fixed part coefficients but not for random (typically small numbers; variance distribution is likely to have + skew)

178 18.2 Simultaneous/complex comparisons & recommended for random part testing Example: Testing H 0: 2 – 3 = 0 AND 3 = 5 H0: [C][ k [C] is the contrast matrix (p by q) specifying the nature of hypothesis (q is number of parameters in model; p is the number of simultaneous tests) FILL Contrast matrix with 1 if parameter involved -1 if involved as a difference 0 not involved otherwise [ is a vector of parameters (fixed or random); q [k is a vector of values that the parameters are contrasted against (usually the null); these have to be set

179 Example: Testing H 0: 2 – 3 = 0 AND 3 = 5 –q = 4 (intercept and 3 slope terms) –p = 2 (2 sets of tests) [C] [ ] [k] * = Overall test against chi square with p degrees of freedom Output –Result of the contrast –Chi-square statistic for each test separately –Chi-square statistic for overall test; all contrasts simultaneously

180 Testing in fixed part 1 slope for Standlrt; 2 BoySch from mixed 3 GirlSch from mixed 4 Boysch from Girlsch Model > Intervals& tests >Fixed coefficients; 4 tests Basic Statistics > Tail Areas Chi square; CPRObability

181 Testing in random part 1 school variance 2 difference between school and student variance Model > Intervals& tests >Random coefficients; 2 tests Basic Statistics > Tail Areas Chi square; CPRObability e-007

182 18.6 Do we need a quadratic variance function at level 2? -> CPRObability e-007 CPRO 4 1 Benchmarks CPRO CPRO

183 18.7 Comparing nested models: likelihood ratio test Akin to F tests in regression models, i.e., is a more complex model a significantly model better fit to the data; or is simpler model a significantly worse fit Procedure: –Calculate the difference in the deviance of the two models –Calculate the change in complexity as the difference in the number of parameters between models –Compare the difference in deviance with a chi-square distribution with df = difference in number of parameters Example: tutorial data do we get a significant improvement in the fit if we move from a constant variance function for schools to a quadratic involving Standlrt?

184 -2*log(lh) is : quadratic -2*log(lh) is : constant ->calc b3 = b2-b >cpro e-010 NB significantly worse fit; ie need quadratic

185 18.9 Deviance Information Criterion Diagnostic for model comparison Goodness of fit criterion that is penalized for model complexity Generalization of the Akaike Information Criterion (AIC; where df is known) Used for comparing non-nested models (eg same number but different variables) Valuable in Mlwin for testing improved goodness of fit of non-linear model (eg Logit) because Likelihood (and hence Deviance is incorrect) Estimated by MCMC sampling; on output get Bayesian Deviance Information Criterion (DIC) Dbar D(thetabar) pD DIC Dbar: the average deviance from the complete set of iterations D(thetaBar): the deviance at the expected value of the unknown parameters pD:the Estimated degrees of freedom consumed in the fit, ie Dbar- D(thetaBar) DIC: Fit + Complexity; Dbar + pD NB lower values = better parsimonious model Somewhat contoversial! Spiegelhalter, D.J., Best, N.G., Carlin, B.P. and van der Linde, A. (2002). Bayesian measures of model complexity and fit. Journal of the Royal Statistical Society, Series B 64:

186 18.10 Some guidance any decrease in DIC suggests a better model But stochastic nature of MCMC; so, with small difference in DIC you should confirm if this is a real difference by checking the results with different seeds and/or starting values. More experience with AIC, and common rules of thumb………

187 18.11 Example: Tutorial dataset example Model 1: NULL model: a constant and level 1 variance Model 2: additionally include slope for Standlrt Model 3: 65 fixed school effects (64 dummies and constant) Model 4: school as random effects Model 5: 65 fixed school intercepts and slopes Model 6: random slopes model; quadratic variance function Best = Model 6 Note: random models (4 & 6) have more nominal parameters than their fixed equivalents but less effective parameters and a lower DIC value (due to distributional assumptions)

Download ppt "Introduction to Multilevel Models: Getting started with your own data University of Bristol Monday 31 ST March– Friday 4th April 2008 Course handouts."

Similar presentations

Ads by Google