Presentation is loading. Please wait.

Presentation is loading. Please wait.

Nan Rothrock, Ph.D. Northwestern University May 22, 2012.

Similar presentations

Presentation on theme: "Nan Rothrock, Ph.D. Northwestern University May 22, 2012."— Presentation transcript:

1 Nan Rothrock, Ph.D. Northwestern University May 22, 2012

2 s Problems in patient-reported outcome measures s PROMIS approach to PRO instrument development s Available PROMIS instruments s Reliability, validity s PROMIS and the FDA

3 Many measures of same health concept Widely varying quality Difficult to compare and combine data... across studies... across conditions Complex Long

4 Questionnaire with a wide range - but low precision 1 2 Questionnaire with a high precision - but small range

5 National Institutes of Health, 2003

6 Attack the Patient- Reported Outcome (PRO) Tower of Babel Harness modern psychometric methods Improve quality and interpretability of PROs Bruegel, 1563

7 Nine-year commitment of NIH $80+ million investment 15 funded research sites

8 Methodology Measures (Instruments) Software

9 Item = question or statement a patient answers Instrument = collection of items Legacy = existing instrument that is gold standard or a commonly used and widely accepted instrument

10 Domain focused, not disease focused Domain = feeling, function or perception you want to measure (e.g., anxiety, physical function, general health perceptions) Item Banks A large collection of items measuring one domain Any and all items can be used to provide a score Can be administered as Computerized Adaptive Tests (CATs) or fixed-length short forms

11 Literature review Focus groups Archival data analysis Expert review/ consensus Binning and winnowing Literacy level analysis Expert item revision Cognitive interviews Translation review Large-scale testing Validation studies Calibration decisions Intellectual property Short form CAT Statistical analysis Domain Framework


13 Self-Reported Health SocialHealth Mental Health Physical Health Symptoms Function Affect Behavior Cognition Relationships Function

14 Pain Behavior Pain Interference Sleep-related Impairment Sleep Disturbance Fatigue Physical Function Pain Interference Fatigue Upper Extremity Mobility Asthma Impact Adult Pediatric/Parent Proxy Sexual Function Physical Health Pain Intensity

15 Mental Health Anxiety Depression Psychosocial Illness Impact Anger Applied Cognition Concerns Anxiety Depression Anger Adult Applied Cognition Abilities Alcohol Use Alcohol Consequences Alcohol Expectancies Pediatric/Parent Proxy

16 Social Health Ability to Participate in Roles & Activities Emotional Support Companionship Informational Support Peer Relationships Adult Instrumental Support Satisfaction with Roles & Activities Social Isolation Pediatric/Parent Proxy

17 Reprinted with permission of the PROMIS Health Organization and the PROMIS Cooperative Group © 2007.

18 Available Universal Spanish In Process German Portuguese Mandarin Chinese French Italian Norwegian Others – see

19 T Score Mean = 50 Standard Deviation = 10 Referenced to the US General Population

20 Adult GI Symptoms Self-efficacy for management of chronic disease Pediatric Pain Behavior, Quality, Intensity Physical Activity Experience of Stress Subjective Well-being Impact of Child Illness on Family Family Belongingness



23 SE = 3.3 rel = 0.90 SE = 2.3 rel = SF items HAQ 20 items CAT 10 items PROMIS Short Form 20 items PROMIS Short Form 10 items rheumatoid arthritis patients US general population Error



26 PROMIS Depression




30 Importance of PRO development to include patient voices Importance of sound measurement Confusion in selecting an instrument because of huge array of choices Ongoing discussions via Interagency Clinical Outcomes Assessment Working Group to qualify PROMIS Fatigue measures, attendance and presentations at PRO Consortium

31 FDA Approach evaluate content validity in each clinical population in which the measure may be used PROMIS Approach there is commonality in patients experiences of symptoms/outcomes and their impact on QOL Need to re-validate a well-developed & valid instrument in a target population is questionable Magasi, S. et al (2011) Content validity of patient-reported outcome measures: Perspectives from a PROMIS meeting. Quality of Life Research


33 N PROMIS Fatigue SF v1.0 PROMIS Fatigue MS MeanSDMeanSD Expanded Disability Status Scale (EDSS) Mild (0-4) Moderate ( ) Severe ( ) Fatigue Severity (0-10 NRS) None/Mild (0-1) Moderate (2-4) Severe (5-10) Vitality (item from the MOS) None/A little Some Quite a lot Very Much PROMIS Fatigue SFv1.0 and PROMIS Fatigue MS Scores by Disability Status, Fatigue Severity, and Vitality Scores Cook et al, QOLR, 2011

34 Supplement with targeted measures Item banking allows flexible item choice without loss of a standard scoring base Alternative is a messy array of contenders that fail to communicate across themselves regarding severity or result interpretation

35 Comparability Provide the ability to compare or combine results from multiple studies. Reliability and Validity Reduce response burden. Improve measurement precision. Simplify administration via computer-based administration, scoring, and reporting




39 IRT-based MIDs on a T-Score scale Multiple cross-sectional and longitudinal anchors (18) Summarized with nonparametric statistics (median, interquartile range)

40 InstrumentT-Score MID Points T-Score MID Effect Sizes* Raw Score MID Points Raw Score MID Effect Sizes Fatigue Pain Interference Physical Function Anxiety Depression *Calculated as the T-Score MID divided by the Assessment 1 T-Score standard deviation Calculated as the Raw Score MID divided by the Assessment 1 Raw Score standard deviation Recommended IRT-based T-Score MIDs and Raw Score MIDs for PROMIS- Cancer Short Forms in Advanced Cancer Patients

41 CAT T-Score MID Points T-Score MID Effect Sizes Fatigue Pain Interference Physical Function Anxiety Depression Recommended IRT-based T-Score MIDs for PROMIS-Cancer CATs

Download ppt "Nan Rothrock, Ph.D. Northwestern University May 22, 2012."

Similar presentations

Ads by Google