1. Creating Composite Measures Using Factor Analysis: The Total Illness Burden Index Sherrie H. Kaplan, PhD, MPH Professor of Medicine UC Irvine School of Medicine Academy Health ARM June 8-10, 2008

2. Some Background…

3. Role of Purpose of Measurement Changes content of aggregate measure Changes tolerance of error Changes psychometric requirements of aggregate Changes level of confidence, dissemination strategy

4. How to create composites: Lessons from psychometrics… Choose measures that broadly represent underlying (latent) construct (sampling from domain of observables); each item adds unique informationChoose measures that broadly represent underlying (latent) construct (sampling from domain of observables); each item adds unique information Hypothesize structure of items in composites before analysis (what measures what?)Hypothesize structure of items in composites before analysis (what measures what?)

5. How to create composites: Lessons from psychometrics… Conduct confirmatory cluster, latent variable analyses (construct validity)Conduct confirmatory cluster, latent variable analyses (construct validity) Decide on scoring methods (simple algebraic sum, weighting, conjunctive or compensatory); test scoring methodsDecide on scoring methods (simple algebraic sum, weighting, conjunctive or compensatory); test scoring methods Test reliability, predictive validity of derived compositeTest reliability, predictive validity of derived composite

6. Models for Composite Scoring Conjunctive scoring (ands): highest, lowest levels achieved define score –Rheumatoid arthritis trials: patient responded if : at least a 20% improvement in tender joint count and 20% improvement in swollen joint count and at least 20% improvement in 3 out of 5 of the following: pain assessment, global assessment, physician assessment, etc. Compensatory scoring (ors): high scores on one component make up for low scores on another

7. Models for weighting Expert defined –Conditioned by expert representation Regression-based –Conditioned by database (provider, patient sample, sample size) Factor analysis-based –Conditioned by variables included in factor analysis Reliability-based –Conditioned by database (sample size)

8. Classic Measurement Theory: Using Factor Analysis to Create Composites Each factor represents latent construct Correlations of items with factors (factor loadings) represent statistical structure of set of variables Factor analysis does not require items have difficulty structure

9. , Cronbachs alpha Measure of internal consistency reliability Given by formula: –Where: N = number of tests σ Yi 2 = variance of item i σ x 2 = total test variance

10. , Cronbachs alpha Alpha is unbiased reliability estimator if items have equal covariances (means and item variances may differ); i.e. have common factor in factor analysis

11. Total Illness Burden: The Latent Construct Patient-reported composite measure of severity of multiple diseases Taken together represent increasing risk for substantial declines in health and increased risk for mortality (1-5 years post initial observation)

12. Purposes of Measurement Post hoc case-mix adjustment A priori risk stratification of clinical trials Improve clinical decision making for tailoring treatment

13. Subdimensions … Pulmonary disease Heart disease Stroke and neurologic disease Gastrointestinal conditions Other cancers (excluding prostate) Arthritis Foot and leg conditions

14. Subdimensions (cont ) Eye and vision conditions Hearing problems Hypertension Diabetes

15. Sample Questions: COPD 1.During the past 6 months, how often did you have wheezing? a. Never b. Once or twice  c. About once a week  d. Several times a week  e. Several times a day 

16. Sample Questions: COPD 4. During the past 6 months, did you use extra pillows in order to sleep at night because of problems with your breathing? a.No  b.Yes, 1 pillow  c.Yes, 2 pillows  d.Yes, 3 or more pillows 

17. Steps in Constructing Subdimensions Transformed variables to uniform metric by clinical definition of severity Tested reliability of clinically defined scale (Cronbach s alpha >.70) Created composite of each subdimension using simple algebraic sum, mean Items in each subdimension varied Validated each subdimension as scale using SF-36, etc.

18. Steps in Constructing Composite Conducted principal components analysis, higher order factor analysis using scales as entries First factor explained 67% of variance Other factors had Eigen values, scree indicating single factor solution Factor loadings ranged from.40 -.70 Used factor loadings to create composite Validated derived composite

19. Understanding and Reducing Disparities in Diabetes Care: Coached Care for Diabetes Sherrie H. Kaplan, PhD, MPH Sheldon Greenfield, MD NovoNordisk Lund, Sweden May 28-20, 2008

20. Characteristics of Patient Sample CharacteristicsRegistry (n=3,894) Survey Sample (n=1001) Mean age58.960.1 % Male43.239.3 % White33.125.8 % Hispanic50.548.9 % Asian16.425.3 % Medicare21.420.9 % Medicaid50.754.1 % Commercial19.216.2 % Uninsured8.7

21. Principal Components Analysis: First Factor TIBI ScaleSample 1Sample 2 GI disease.604.628 Atherosclerotic heart dis.650.621 Neurologic problems.433.360 Hearing problems.452.388 Hypertension.328.448 Cardiopulmonary.712.704 Feet problems.613.587 Arthritis.449.618 Vision problems.475.367

22. Cronbach s alpha (.799) TIBI Scale Scale Mean α if item deleted GI disease.58.767 Atherosclerotic heart dis.79.783 Neurologic problems.19.799 Hearing problems.36.798 Hypertension.74.799 Cardiopulmonary.81.755 Feet problems.67.787 Arthritis.57.790 Vision problems.51.798

23. Correlation of TIBI with Patient- Reported Health Status Measures by Ethnic Group Health Status Measures Whites Mexican- American Vietnamese SF-36 PFI10 -.63-.38-.55 CESD.50.55 Diabetes Burden.37.33.37

24. Other TIBI Validation Studies …

29. Preventing Cardiovascular Disease: Identification of Co-Morbidity Subgroups who may not Benefit from Aggressive Diabetes Management Greenfield S, Nicolucci A, Pellegrini F, Kaplan SH

30. QuED Study Prospective cohort study of consecutively enrolled patients with diabetes who completed TIBI at enrollment in Italian Quality of Care and Outcomes in Type 2 Diabetes Study Group (n=2,613)

31. QuED Study Patient Characteristics Mean age 62.7 [10.3] % Female 45 % < 5 yrs education 52 % BMI > 30 28 HbA1c value 7.2

32. QuED: Total 5-yr CV Events by TIBI TIBI GroupHR95% HR CIP-value 0-31 3-60.950.61-1.47.81 6-91.110.74-1.67.61 9-121.461.02-2.1.04 >121.571.16-2.12.003

33. QuED: % 5-yr Survival by TIBI TIBI Group%HR (95% CI)P-value 0-385.9 1 3-685.0 1.11(.75-1.65).59 6-980.2 1.42(1.00-2.02).05 9-1280.0 1.41(0.96-2.08).08 >1275.6 1.63(1.25-2.13) <.000

34. Complex diabetes patients, those with the greatest burden from competing co-morbidities (highest TIBI scores) may benefit less from aggressive glycemic control due to their increased risk of mortality from other causes before those benefits could be realized.

35. Conclusions Using factor analysis, possible to derive latent construct that reflects patients total illness burden Potentially useful in case-mix adjustment, clinical trials design, clinical decision making Future research aimed at improving sensitivity, specificity, particularly at intermediate ranges