1. 1 11/17/2015 Psychometric Modeling and Calibration Ron D. Hays, Ph.D. September 11, 2006 PROMIS development process session 10:45am-12:30 pm

2. 2 11/17/2015 PROMIS Domains Physical functioning (Hays/Bjorner) Pain (Revicki) Fatigue (Lai) Emotional distress (Choi/Reise) Social/role participation (Bode/Hahn)

3. 3 11/17/2015 Datasets Cancer Item Banks (Northwestern) Digitalis Investigation Group Study--randomized double- blind placebo-controlled trial evaluating effect of digoxin on mortality in 581 patients with heart failure and sinus rhythm. IMMPACT--internet-based survey of individuals with chronic pain from the American Chronic Pain Association website Medical Outcomes Study--observational study of persons with hypertension, diabetes, heart disease, and/or depression in Boston, Chicago, and Los Angeles WHOQOL-100 data (n = 442 from U.S. field center)

4. 4 11/17/2015 Types of Analyses Classical Test Theory StatisticsClassical Test Theory Statistics IRT Model AssumptionsIRT Model Assumptions Model FitModel Fit Differential Item FunctioningDifferential Item Functioning Item CalibrationItem Calibration

5. 5 11/17/2015 Classical Test Theory Statistics Out of rangeOut of range Item frequencies and distributionsItem frequencies and distributions Inter-item correlationsInter-item correlations Item-scale correlationsItem-scale correlations Internal consistency reliabilityInternal consistency reliability

6. 6 11/17/2015 IRT Model Assumptions (Uni)dimensionality(Uni)dimensionality Local independenceLocal independence MonotonicityMonotonicity

7. 7 11/17/2015 Sufficient Unidimensionality Confirmatory factor modelsConfirmatory factor models  One factor  Bifactor (general and group factors)

8. 8 11/17/2015 Local Independence After controlling for dominant factor(s), item pairs should not be associated.After controlling for dominant factor(s), item pairs should not be associated.  Look at residual correlations (> 0.20)

9. 9 11/17/2015 Monotonicity Probability of selecting a response category indicative of better health should increase as underlying health increases.Probability of selecting a response category indicative of better health should increase as underlying health increases. Item response function graphs withItem response function graphs with  y-axis: proportion positive for item step  x-axis: raw scale score minus item score

10. 10 11/17/2015

11. 11 11/17/2015 Category Response Curves for Samejima’s Graded Response Model

12. 12 11/17/2015 Model Fit Compare observed and expected response frequencies by item and response categoryCompare observed and expected response frequencies by item and response category Items that do not fit and less discriminating items identified and reviewed by content expertsItems that do not fit and less discriminating items identified and reviewed by content experts

13. 13 11/17/2015 Differential Item Functioning Uniform DIFUniform DIF  Threshold parameter Non-uniform DIFNon-uniform DIF  Discrimination parameter Gender, race/ethnicity, age, diseaseGender, race/ethnicity, age, disease

14. 14 11/17/2015 Item Calibration Item parameters (threshold, discrimination)Item parameters (threshold, discrimination) Mean differences for studied disease groupsMean differences for studied disease groups

15. 15 11/17/2015 Example of Lessons Learned in Secondary Analyses Emotional distress Cannot be adequately modeled as a unidimensional construct. Limited representation of positive end of construct Several items having some response options that provide little information.

16. 16 11/17/2015 Documentation eRoom Public website: http://www.nihpromis.org/http://www.nihpromis.org/ Peer-reviewed manuscripts, e.g.: Hays, R. D. et al. (submitted). Item response theory analyses of physical functioning items in the Medical Outcomes Study. Reeve, B. B. (submitted). Psychometric evaluation and calibration of health-related quality of life items banks: Plans for the Patient-Reported Outcome Measurement Information System (PROMIS) Presentations: Tomorrow 4:15-6:15

17. 17 11/17/2015 Questions?

18. 18 11/17/2015 Datasets Subjected to Psychometric Analysis Cancer Fatigue: Cancer Item Banking Project at NWU Cancer Pain: Cancer Item Banking Project at NWU Cancer Social: Cancer Item Banking Project at NWU CSSCD: Cooperative Study of Sickle Cell Disease (pediatric) CHC: Chronic Hepatitis C Study CHS: Cardiovascular Health Study DIG: Digitalis Investigation Group Quality of Life Sub-study IMMPACT: Multiple Pain Projects MOS: Medical Outcomes Study NGHS: National Growth and Health Study (peds.) Q-Score: Cancer Quality of Life Project at NWU WHOQOL: World Health Organization Quality of Life Project

19. 19 11/17/2015 Datasets Subjected to Psychometric Analysis PROMIS Domains Emotional DistressFatiguePainPhysical Function Social Role Participation CHS (NWU/Cook) DIG (UCLA/Hays) Q-Score (NWU/Bode) CHC (Medtap/Revicki & Chen) Cancer Pain (NWU/Lai) IMMPACT (Medtap/Revicki & Chen) CHS (NWU/Cook) DIG (UCLA/Hays) CHS (NWU/Cook) Cancer Social (NWU/Bode) CHS (NWU/Cook) Cancer Fatigue (NWU/Lai) WHOQOL (UCLA/Hays) WHOQOL (UCLA/Hays) MOS (UCLA/Hays & Spritzer) Q-Score (NWU/Lai) WHOQOL (UCLA/Hays) WHOQOL (UCLA/Hays) WHOQOL (UCLA/Hays) CSSCD (NWU) NGHS (NWU) NGHS (NWU)