International Perthes Study Group October 6-7, 2017 Patient Reported Outcomes Measurement Information System (PROMIS) for Patients with Perthes Disease Hiroko Matsumoto, PhDc Joshua E. Hyman, MD Columbia University Medical Center

What is PROMIS?

Patient-Reported Outcomes Measurement Information System What is PROMIS? Patient-Reported Outcomes Measurement Information System PROMIS NIH-funded psychometric evaluation that has grown significantly over last several years: (http://www.hhs.gov/asl/testify/t051208a.html) $90-million since 2004. PROMIS measures the following metrics: (NIHPromis.org) Patient-reported outcomes: Subjective, e.g. how a patient perceives about their symptoms and function (http://www.nihpromis.org/measures/domainframework1)

What is PROMIS? PROMIS Mission Statement A comprehensive instrument to assess pain and pain behaviors “PROMIS creates an opportunity for clinicians and patients to develop a common language around self-reported health status and in the process allow better assessment of the unique information from patients thereby enabling improved quality of care and quality of clinical research.” (NIHPromis.org)

Patient-Reported Outcomes Measurement Information System What is PROMIS? Patient-Reported Outcomes Measurement Information System Universal domains Domain = Specific areas of importance regarding health and well-being Cuts across different diseases E.g., Sleep disturbance, depressive symptoms, ability to participate in usual role, global health Generic measures versus disease specific measures Source: HealthMeasures

PROMIS Pediatric Self and Proxy: 21 Domains * *Profile domains: measuring the most important PROMIS concepts Source: HealthMeasures

Patient-Reported Outcomes Measurement Information System What is PROMIS? Patient-Reported Outcomes Measurement Information System Item banks Collection of items all measuring the same thing Can be administered in different ways (CAT, short forms) always producing a score on the same metric Item Bank Short Form CAT Computer Adaptive Test Source: HealthMeasures

Static versus Dynamic Short From Static versus Dynamic Static: Questions to follow do not change based on patient responses. Questions presented in same sequence, regardless of prior answers.

PROMIS Example of Short Form: Pain Interference

Computer Adaptive Test Static versus Dynamic Static versus Dynamic Dynamic option: Questions are presented in changing sequence, with following sequence based on prior answers. Purpose: Minimizes # questions; Greater measurement precision; Presented as Computer Adaptive Test (CAT). CAT Computer Adaptive Test

PROMIS Example of CAT Form: Pain Interference It was hard to stay standing when I had pain

Longitudinal tracking of HRQoL is a strength of PROMIS PROMIS T-score allows adult and pediatric scores to be compared directly  Longitudinal studies Mean: 50 40 60 T-score of: e.g. Raw score of 80 (Peds pain interference) e.g. Raw score of 65 (Adult pain interference) 68% of population T-score of 60 = 15.8% of population at your age and gender perceive to have higher HRQoL than you

PROMIS Psychometrically sound Brief Applicable in a variety of settings Applicable across groups Cover the full range of the domain Individual domain stands alone Available for use across ages Score on one metric Source: HealthMeasures

Data Collection by REDCap

PROMIS Validity Study in Perthes Disease Funded by POSNA

Methods Study Design and setting: Prospective Validation Study 13 institutions from International Perthes Study Group Inclusion Criteria: Diagnosis of Perthes Ages 8-17 Exclusion Criteria: Surgical treatment within 6 months prior to enrollment

What is Validity? The degree to which any measurement approach or instrument succeeds in describing or quantifying what it is designed to measure

Assessing Validities Types Methods Translational Validity Face Validity Content Validity Criterion Validity Predictive Validity Concurrent Validity Postdictive Validity Construct Validity Internal Construct Validity Convergent Validity (Convergent Evidence) Discriminant Validity (Discriminant Evidence) Factorial Evidence (observed variables vs. latent construct) External Construct Validity Nomological Validity Nomological validity: the degree to which a construct behaves as it should within a system of related constructs. Ex. As age increases, physical function decreases

Construct Validity Is the measure consistent with the theoretical concept being measured? All tests of validity ultimately designed to support/refute the instrument’s construct validity

Convergence and Discriminant Validities Convergent Evidence Demonstrates that your measure correlates highly with measures of the same construct Groups known to differ along construct have different scores on measure Discriminant Evidence No to low correlation with measures of different constructs

PROMIS Pediatric Self and Proxy: 21 Domains * *Profile domains: measuring the most important PROMIS concepts Source: HealthMeasures

Construct Validity: Known Group Method Waldenstrom Disease stages Early: (IA – IIA) Late: (IIB – IIIB) Healed: (IV)

Demographics Characteristics N = 186 Age 10.23 ± 3.13 Gender (Male) 151 (82.1%) Waldenström Stages Early (IB; IIA) 24/186 (12.9%) Late (IIB; IIIA; IIIB) 63/186 (33.9%) Healed (IV) 99/186 (53.2%) Ethnicity White 101/186 (54.3%) Black 3/186 (1.6%) Asian Hispanic 4/186 (2.2%) Other/Unknown 75/186 (40.3%)

Within Physical Domain: Increased mobility was associated with lower pain and fatigue Convergent Validity

Within Mental Health Domain: Increased Depression was associated with more Anxiety, and Anger Convergent Validity

Across Domains: Increased Mobility was associated with less Anxiety, Depression and Anger Convergent Validity

Across Domains: Increased Pain Interference was associated with more Anxiety, Depression and Anger Convergent Validity

Across Domains: Increased Fatigue was associated with more Anxiety, Depression and Anger Convergent Validity

Discriminant Validity Across Domains: Peer Relationship have no to low associations with Physical Health Discriminant Validity

Discriminant Validity Across Domains: Peer Relationship have no to low associations with Mental Health Discriminant Validity

All domains had the worst scores in the Early Stage group Categories/Means Early Stage N=24 Late Stage N=63 Healed N=99 Fatigue 42.9 39.9 41.4 Mobility 40.2 45.5 48.6 Pain Interference 49.1 45.7 46.0 Anxiety 49.6 43.7 43.9 Depression 49.8 45.4 Anger 51.3 43.2 44.3 Peer Relations 48.0 51.7 51.2 ) NB: Minimally important difference (MID): Smallest change of a PRO that is perceived to be important by the patient  2-3 point (Yost et al. 2011)

Conclusions PROMIS has construct validity in measuring QoL of patients with Perthes disease Demonstrate that PROMIS domains correlates highly with measures of the same construct Low correlation with domains measuring different construct Groups known to differ along construct have different scores on measure However…………

Discussions When compared with U.S. average, some domain scores in patients with Perthes disease were counterintuitive: Lower scores: Fatigue, pain interference, anxiety, depression, anger Intuitive domain scores: Lower score in Mobility Lower score in peer relations in Early Stage group

Discussions and Next Steps PROMIS not working? Disease Specific Instrument Skewed group? More patients to recruit Counterintuitive but reality? Responsive testing (measure changes) Further Known Group validity testing within Healed Group – Stratify by Stulberg classification

Thank You! Hiroko Matsumoto hm2174@columbia.edu