Measurement data quality in social networks analysis. Assess reliability and validity in egocentered social networks. Complete networks Egocentered networks Background
Index Reliability and Validity MTMM Model Data Multilevel analysis Results and interpretation
Reliability and Validity MTMM Model Confirmatory Factor Analysis (CFA) specification of the MTMM model. Y ij = m ij M j + t ij T i + e ij (1) () where: Y ij : response or measured variable “i” measured by method “j”. T i : unobserved variable of interest (trait). Related to validity. M j : variation in scores due to the method. Related to invalidity. m ij and t ij : factor loadings on the method and trait factors. e ij : random error, which is related to lack of reliability. Reliability and Validity
Figure 1 : Path diagram for the MTMM model for trait (T i ) and method (M j ). MTMM model
Traits T1T1 Frequency of contact T2T2 Feeling of closeness T3T3 Feeling of importance T4T4 Frequency of the alter upsetting to ego Methods M1M1 Face-to-face interviewing M2M2 Telephone interviewing MTMM model
Y 11 Y 21 Y 31 Y 41 Y 22 Y 12 Y 32 Y 42 M1M1 M2M2 T1T1 T2T2 T3T3 T4T4 e 11 e 21 e 31 e 41 e 12 e 22 e 32 e 42 Figure 2: Path diagram of a CFA MTMM model for two methods and four traits. MTMM model
Var (Y ij ) = m ij 2 Var (M j ) + t ij 2 Var (T i ) + Var (e ij ) (2)() Validity and Reliability for CFA MTMM model: Reliability coefficient = (3) Validity coefficient = (4)
Kogovšek, et al., 2002: Estimating the reliability and validity of personal support measures: full information ML estimation with planned incomplete data. Social Networks, 24, T 1 Frequency of contact T 3 Feeling of importance T 2 Feeling of closeness T 4 Frequency of the alter upsetting to ego GNFirst interviewSecond interview M 1 Face-to-faceM 2 Telephone Table 1: The design of the study Data Representative sample of inhabitants of Ljubljana
Multilevel analysis Two-level MTMM model. The highest level: group level = egos = g The lowest level: individual level = alters = k Multilevel MTMM model
The mean centred individual scores for group “g” and individual “k” can be decomposed into: Within group component (5) Between group component (6) where: is the total average over all alters and egos. is the average of all alters of the g th ego. Y gk is the score on the name interpreter (questions) of the k th alter chosen by the g th ego. G is the total number of egos. n is the number of alters within each ego. N=nG is the total number of alters. Multilevel MTMM model
Sample covariance matrices: Multilevel MTMM model SW=SW=SB=SB= S T = S B + S W = (7)(8) (9) Population covariance matrices: T = B + W (10) Y ij = m Bij M Bj + t Bij T Bi + e Bij + m wij M wj + t wij T wi + e wij (11)() Y Bij Y Wij
Härnqvist Method Separate analysis for S B and S W Group measure S w is the ML estimator of Σ W S B is the ML estimator of Σ W +cΣ B (12) Multilevel MTMM model Model estimated by Maximum Likelihood (ML).
Figure 3: Multilevel CFA MTMM Model. Multilevel MTMM model
Interpretation: We can obtain 2 reliabilities and 2 validities for each trait-method combination. To analyse each component separately: Y ij = m Bij M Bj + t Bij T Bi + e Bij + m wij M wj + t wij T wi + e wij (11)() Y Bij Y Wij Decompose the variance: Var (Y ij ) =m ij 2 w Var (M jW ) + m ij 2 B Var (M jB ) + t ij 2 w Var (T iW ) + t ij 2 B Var (T iB ) + (13) () Var (e ijw ) + Var (e ijB ) Multilevel MTMM model
Analysis: Multilevel MTMM model Analysis 1: traditional analysis on S T. ML estimation. Analysis 2: traditional analysis on S W. ML estimation. Analysis 3: traditional analysis on S B, which is a biased estimate of Σ B. ML estimation. Analyses 2 and 3 together constitute the recommendation of Härnqvist (1978). Analysis 4: multilevel analysis, to fit Σ W and Σ B simultaneously. ML estimation.
Table 1: Goodness of fit statistics. Results and interpretation
Table 2: Decomposition into 6 variance components. Analysis 4. Table 2 trait variance withinT1T1 T2T2 T3T3 T4T4 M1M M2M method variance within* M1M M2M error variance within M1M M2M trait variance between M1M M2M method variance between* M1M M2M error variance between* M1M M2M * Boldfaced for small non- significant variances constrained to zero. Results and interpretation
Table 3Table 3: Decomposition into 6 variance components* trait variance withinT1T1 T2T2 T3T3 T4T4 M1M1 67,2%66,0%66,3%55,2% M2M2 67,5%70,2%70,6%55,0% method variance within* M1M1 2,6%3,6%3,1%3,7% M2M2 0,0% error variance within M1M1 13,3%19,2%17,4%26,0% M2M2 12,6%17,4%14,7%20,6% trait variance between M1M1 15,3% 8,1%10,5%15,1% M2M2 16,0% 9,0%11,6%15,6% method variance between* M1M1 0,0% M2M2 0,8%1,2%1,0%1,1% error variance between* M1M1 1,5%3,1%2,7%0,0% M2M2 3,2%2,2%2,1%7,7% * Boldfaced for small non-significant variances constrained to zero. Results and interpretation
Within levelBetween levelOverall level T1T1 T2T2 T3T3 T4T4 T1T1 T2T2 T3T3 T4T4 T1T1 T2T2 T3T3 T4T4 Reliability coef M1M1 0,920,890,900,840,950,850,891,000,920,880,890,86 M2M2 0,920,900,910,850,920,910,930,830,920,900,910,85 Validity coef M1M1 0,980,970,980,971,00 0,98 M2M2 1,00 0,980,940,960,971,000,99 Table 4Table 4: Multilevel reliabilities and validities* Results and interpretation * Boldfaced for small non-significant variances constrained to zero.
Table 5Table 5: Within part. Comparison of analyses 2 (S W ) and 4 (multilevel). * Boldfaced for small non-significant variances constrained to zero. Results and interpretation
Table 6Table 6: Between part. Comparison of analyses 3 (S B ) and 4 (multilevel). * Boldfaced for small non-significant variances constrained to zero. Results and interpretation
Table 7Table 7: Overall analysis. Comparison of analyses 1 (S T ) and 4 (multilevel). * Boldfaced for small non-significant variances constrained to zero. Results and interpretation
T1T1 T2T2 T3T3 T4T4 t ij 2 w Var(T iw )/ [t ij 2 w Var(T iw ) + t ij 2 B Var(T iB )] Table 8Table 8: Percentages of variance at within level form M1 and M2 Results and interpretation T1T1 T2T2 T3T3 T4T4 Var(eijw)/ Var(Yij)
Contribution: To consider egocentered networks as hierarchical data. To specify a multilevel MTMM. Interpretation from measurement theory of different % of variance. Results and interpretation