Ron D. Hays September 12, 2006 Network Testing ~ 5 or 6pm

Slides:



Advertisements
Similar presentations
Item Response Theory in a Multi-level Framework Saralyn Miller Meg Oliphint EDU 7309.
Advertisements

How Should We Assess the Fit of Rasch-Type Models? Approximating the Power of Goodness-of-fit Statistics in Categorical Data Analysis Alberto Maydeu-Olivares.
Confirmatory factor analysis GHQ 12. From Shevlin/Adamson 2005:
Item Response Theory in Health Measurement
1 Evaluating Multi-Item Scales Using Item-Scale Correlations and Confirmatory Factor Analysis Ron D. Hays, Ph.D. RCMAR.
Item Response Theory in Patient-Reported Outcomes Research Ron D. Hays, Ph.D., UCLA Society of General Internal Medicine California-Hawaii Regional Meeting.
Overview of field trial analysis procedures National Research Coordinators Meeting Windsor, June 2008.
A Method for Estimating the Correlations Between Observed and IRT Latent Variables or Between Pairs of IRT Latent Variables Alan Nicewander Pacific Metrics.
Item Response Theory. Shortcomings of Classical True Score Model Sample dependence Limitation to the specific test situation. Dependence on the parallel.
A Different Way to Think About Measurement Development: An Introduction to Item Response Theory (IRT) Joseph Olsen, Dean Busby, & Lena Chiu Jan 23, 2015.
Classical Test Theory By ____________________. What is CCT?
Multivariate Methods EPSY 5245 Michael C. Rodriguez.
Performance of PROMIS® CATs Versus Short forms in Detecting Change Ron D. Hays October 3, 2011 International Association for Computer.
Primer on Evaluating Reliability and Validity of Multi-Item Scales Questionnaire Design and Testing Workshop October 25, 2013, 3:30-5:00pm Wilshire.
Psychometric Properties of the Consumer Assessment of Healthcare Providers and Systems (CAHPS) Clinician and Group Adult Visit Survey September 11, 2012.
Measurement Invariance of the Pediatric Version of PROMIS Symptom Scales between American and Chinese Children with Cancer Changrong Yuan 1,* , Pamela.
SAS PROC IRT July 20, 2015 RCMAR/EXPORT Methods Seminar 3-4pm Acknowledgements: - Karen L. Spritzer - NCI (1U2-CCA )
Dealing with Omitted and Not- Reached Items in Competence Tests: Evaluating Approaches Accounting for Missing Responses in Item Response Theory Models.
1 Assessing the Minimally Important Difference in Health-Related Quality of Life Scores Ron D. Hays, Ph.D. UCLA Department of Medicine October 25, 2006,
Development of Physical and Mental Health Summary Scores from PROMIS Global Items Ron D. Hays ( ) UCLA Department of Medicine
1 11/17/2015 Psychometric Modeling and Calibration Ron D. Hays, Ph.D. September 11, 2006 PROMIS development process session 10:45am-12:30 pm.
Validity and Item Analysis Chapter 4. Validity Concerns what the instrument measures and how well it does that task Not something an instrument has or.
Validity and Item Analysis Chapter 4.  Concerns what instrument measures and how well it does so  Not something instrument “has” or “does not have”
Item Response Theory (IRT) Models for Questionnaire Evaluation: Response to Reeve Ron D. Hays October 22, 2009, ~3:45-4:05pm
Multitrait Scaling and IRT: Part I Ron D. Hays, Ph.D. Questionnaire Design and Testing.
Item Factor Analysis Item Response Theory Beaujean Chapter 6.
Psychometric Evaluation of Questionnaire Design and Testing Workshop December , 10:00-11:30 am Wilshire Suite 710 DATA.
Item Response Theory in Health Measurement
Ron D. Hays, Ph.D. Joshua S. Malett, Sarah Gaillot, and Marc N. Elliott Physical Functioning Among Medicare Beneficiaries International Society for Quality.
Demonstration of SEM-based IRT in Mplus
This is a mess... How the hell can I validate the consumer behaviors’ scales of my SEM model? Maria Pujol-Jover 1, Irene Esteban-Millat 1 1 Marketing Research.
Item Response Theory Dan Mungas, Ph.D. Department of Neurology University of California, Davis.
Multitrait Scaling and IRT: Part I Ron D. Hays, Ph.D. Questionnaire.
Two Approaches to Estimation of Classification Accuracy Rate Under Item Response Theory Quinn N. Lathrop and Ying Cheng Assistant Professor Ph.D., University.
Evaluating Multi-item Scales Ron D. Hays, Ph.D. UCLA Division of General Internal Medicine/Health Services Research HS237A 11/10/09, 2-4pm,
Health-Related Quality of Life in Outcome Studies Ron D. Hays, Ph.D UCLA Division of General Internal Medicine & Health Services Research GCRC Summer Session.
Chapter 2 Norms and Reliability. The essential objective of test standardization is to determine the distribution of raw scores in the norm group so that.
Evaluating Patient-Reports about Health
1University of Oklahoma 2Shaker Consulting
Psychometric Evaluation of Items Ron D. Hays
Further Validation of the Personal Growth Initiative Scale – II: Gender Measurement Invariance Harmon, K. A., Shigemoto, Y., Borowa, D., Robitschek, C.,
Which is the Best Instrument for Assessing Burnout?
QDET2, Miami, FL, Hibiscus A
Comparability of the health assessment questionnaire between countries: psychometric examination of cross-national measurement equivalence Sam Norton,
Reliability and validity of the BREQ-2 for measuring high school students’ motivation for physical education Stuart Forsyth¹, David Rowe¹, and Nanette.
Chapter 15 Confirmatory Factor Analysis
A Different Way to Think About Measurement Development:
UCLA Department of Medicine
Evaluating Patient-Reports about Health
UCLA Department of Medicine
Evaluating Multi-Item Scales
Ron D. Hays GIM-HSR Friday Noon Seminar Series November 4, 2016
assessing scale reliability
Evaluating IRT Assumptions
الاختبارات محكية المرجع بناء وتحليل (دراسة مقارنة )
Dr. Chin and Dr. Nettelhorst Winter 2018
Validation of the Portuguese DSM-IV-MR-J
By ____________________
EPSY 5245 EPSY 5245 Michael C. Rodriguez
Confirmatory factor analysis
Sean C. Wright Illinois Institute of Technology, APTMetrics
Psychometrics Working Group Friday Harbor Laboratories
Introduction to IRT for non-psychometricians
Evaluating the Psychometric Properties of Multi-Item Scales
Evaluating Multi-item Scales
Multitrait Scaling and IRT: Part I
Evaluating Multi-item Scales
SEM: Step by Step In AMOS and Mplus.
Psychometric testing and validation (Multi-trait scaling and IRT)
UCLA Department of Medicine
Presentation transcript:

Ron D. Hays September 12, 2006 Network Testing ~ 5 or 6pm Physical Functioning Ron D. Hays September 12, 2006 Network Testing ~ 5 or 6pm

Physical Functioning Items Form C 112 items (16 HAQ like) Form G 56 items (4 HAQ like) Sub-domains IADL Upper Central Lower

Analyses Conducted Descriptive statistics Classical test theory/reliability Monotonicity/scalability Dimensionality IRT

Classical Test Theory Form C (112 items) Form G (49 items) Alpha =.99 Range of item/total correlations (corrected for item overlap): 0.40 (able to brush teeth) to 0.85 (able to do yard work) Form G (49 items) Alpha =.98 Range of item/total correlations (corrected for item overlap): 0.30 (open previously opened jar) to 0.87 (help limit climbing flight of stairs)

Monotonicity/Scalability Monotonicity assessed using Mokken Scale Procedure No items in Form C or G had monotonicity problems (minimum violation index < .03) Scalability assessed using Loevinger’s H Form C H ranged from 0.50 (health limit you eating a meal in a normal time) to 0.80 (able to run or jog for two miles) Form G H ranged from 0.36 (open previously opened jar) to 0.84 (help getting dressed) Scalability “strong” when H > 0.5

Unidimensionality Confirmatory factor analysis conducted using Mplus with polychoric correlations and WLSMV estimator Form C CFI = 0.973, RMSEA = 0.091 Factor loadings ranged from 0.74-0.95 Form G CFI = 0.926, RMSEA = 0.136 Factor loadings ranged from 0.54-1.04*

IRT Analysis Graded Response Model fit using MULTILOG Form C Form G Too few observations in some categories for 28 items to estimate SEs Form G Too few observations in some categories for 3 items to estimate SEs So collapsed categories and reran model

Form C GRM Results χ 2 = 17,087.3 (1-PL); χ 2 = 16,829.9 (2-PL) Marginal reliability = 0.9924 Slopes ranged from 1.68 (PFB6) to 4.07 (PFB15) Thresholds ranged from -2.78 (PFA54) to 3.06 (PFA39) PFB6 (health limit you in eating a meal within a normal time) PFB15 (able to change the bulb in a table lamp) PFA54 (able to button your shirt) PFA39 (able to run at a fast pace for two miles)

Form G GRM Results χ 2 = 21,019.4 (1-PL); χ 2 = 20,407.7 (2-PL) Marginal reliability = 0.9803 Slopes ranged from 1.06 (PFC27) to 5.08 (PFC16) Thresholds ranged from -4.69 (PFC43) to 3.19 (PFC33) PFC27 (use jar opener or help to open previously opened jars) PFC16 (need buttonhook, zipper pull, other gadget or help to get dressed) PFC43 (able to use hands to turn faucets, etc.) PFC33 (able to run ten miles)

Questions?