1. Patient-Reported Questionnaires in Clinical Practice: Just Another Laboratory Test David Cella, Ph.D. Professor and Chair Department of Medical Social Sciences Feinberg School of Medicine

2. Health Reform 2010 Major Commitment of the Obama Administration Sweet Spot: Right treatment to the right people Avoid unnecessary / inappropriate treatments Universally available and affordable Chronic diseases Major expense Rarely “cured” Most important outcome is often quality of life

3. Health care providers do not routinely use health-related quality of life (HRQL) data to guide diagnosis, treatment, or performance improvement. …if they did, would it make a difference? …how should HRQL data be presented? The Problem

4. Measurement Science: Not the Problem In chronic illness care, the problem is not the lack of HRQL measures a large selection of generic and disease- specific measures exist The problem is the unhelpful way the data are presented to clinicians

5. 5 Continuum of Disease-specific and Generic Health Measures 5 Clinical Markers Specific Symptoms Impact of Disease- specific Problems Generic Functioning, Well-being and Evaluation Adapted from: Wilson and Cleary, JAMA, 1995 Ware, Annual Rev. Pub. Health, 1995 (1)(2)(3)(4)

6. 6 6 Clinical Markers Specific Symptoms Impact of Disease- specific Problems Generic Functioning, Well-being and Evaluation Adapted from: Wilson and Cleary, JAMA, 1995 Ware, Annual Rev. Pub. Health, 1995 (1)(2)(3)(4) dd Spirometry Over the last 4 weeks I have had shortness of breath Almost every day Several days a week A few days a month Not at all Shortness of Breath Continuum of Disease-specific and Generic Health Measures

7. 7 7 Clinical Markers Specific Symptoms Impact of Disease- specific Problems Generic Functioning, Well-being and Evaluation Adapted from: Wilson and Cleary, JAMA, 1995 Ware, Annual Rev. Pub. Health, 1995 (1)(2)(3)(4) dd Over the last 4 weeks I have had shortness of breath Almost every day Several days a week A few days a month Not at all How much did your lung/respiratory problems limit your usual activities or enjoyment of everyday life? Not at all A little Moderately Extremely Spirometry Shortness of Breath Respiratory -specific Continuum of Disease-specific and Generic Health Measures

8. 8 8 Clinical Markers Specific Symptoms Impact of Disease- specific Problems Generic Functioning, Well-being and Evaluation Adapted from: Wilson and Cleary, JAMA, 1995 Ware, Annual Rev. Pub. Health, 1995 (1)(2)(3)(4) dd Over the last 4 weeks I have had shortness of breath Almost every day Several days a week A few days a month Not at all How much did your lung/respiratory problems limit your usual activities or enjoyment of everyday life? Not at all A little Moderately Extremely In general, would you say your health is… Excellent Very good Good Fair Poor Spirometry Shortness of Breath Respiratory -specific Generic Continuum of Disease-specific and Generic Health Measures

9. Can Patient Reported Outcomes (PROs) be Treated as Just Another Lab Result? 9

10. 10 What Do We Need to Launch a Useful Clinical PRO Lab Outcomes that matter to patients and providers Practical (brief) and inexpensive Covers a range, including normal Greater precision Comparability of scores Ease of interpretation Reference values / normal ranges

11. SyMon-L Symptoms Tracked Pain Fatigue (lack of energy) Shortness of breath Coughing Chest tightness Difficulty breathing Nausea Poor appetite Weight loss Bothered by treatment side effects Emotional distress Cognitive dysfunction Dissatisfaction with HRQL

12. Symptom Summary Report

13. Results: Clinical intervention responses to symptom alerts Types of intervention N% Medical/clinical (e.g., medications, MD appt) 23320% Education42036% Support, coping 45539% Coordination of care (e.g., referrals) 504% Acuity of Interventions Critical (e.g., PE, ER visit, admission) 223% Important (e.g., change in meds, clinic visit, IV therapy) 10316% Standard of care (e.g., reassessment, education) 37459% Non-essential (e.g., follow-up) 9815% System-related (e.g., study questions) 376%

14. Symptom Monitoring Report

15. QoL predicts survival in NSCLC StudyWhat predicts? Stanley (1980)Initial PS Kaasa, Mastekaasa & Lund (1989)General sx (pain, fatigue) Psychosocial well being Ganz et al. (1991)Overall QoL Loprinzi et al. (1994)Physician-rated PS Patient-rated PS Ruckdeschel (1994)Patient-reported QOL Herndon et al. (1999)Pain Montazeri et al. (2001)Pre-diagnosis overall QoL Eton et al (2003)Patient-reported QoL

16. Symptoms (LCS) Emotional Physical Social Functional Quality of Life FACT-L

17. Baseline patient characteristics predicting survival Order of entry FACT TOI Metastatic symptoms ECOG PS Stage (IIIB vs IV) Paclitaxel (Arm A vs B/C) Risk ratio* 0.98 1.44 1.57 1.39 0.81 p value <0.001 0.016 0.039 *Risk of death Other possible explanatory variables included age, sex, g-csf, weight loss, disease symptoms, systemic symptoms, comorbidity and other FACT subscales

18. <58, no <58, yes >58, no >58, yes Initial, improved? Proportion surviving 1.0 0.8 0.6 0.4 0.2 0.0 0 1 2 3456 7 89 10 1112 Initial TOI and improvement at 6 weeks* (n=352) *Patients with missing QoL excluded Eton et al, JCO, 2003

19. LCS Mean Change By Week By Objective Response, Trial 39

20. Survival by Disease-related Symptom Improvement Subset surviving > 8 weeks

21. Survival by Response & Symptom Improvement Subset surviving > 8 weeks

22. Survival by Symptom Improvement in SD Subset Subset surviving > 8 weeks * * All patients with an objective response of SD survived > 8 weeks.

23. 23 Content of Widely-Used Patient- Reported Outcome Measures Source: Adapted from Ware, 1995 Reported health transition SIP = Sickness Impact Profile (1976) HIE = Health Insurance Experiment surveys (1979) NHP = Nottingham Health Profile (1980) QLI = Quality of Life Index (1981) COOP = Dartmouth Function Charts (1987) DUKE = Duke Health Profile (1990) MOS FWBP = MOS Functioning and Well-Being Profile (1992) MOS SF-36 = MOS 36-Item Short-Form Health Survey (1992) QWB = Quality of Well-Being Scale (1973) EUROQOL = European Quality of Life Index (1990) HUI = Health Utility Index (1996) SF-6D = SF-36 Utility Index (Brazier, 2002) PsychometricUtility Related SIPHIENHPCOOPDUKEMOS FWBP MOS SF-36 QWBEURO -QOL HUISF-6D CONCEPTS Physical functioning Social functioning Role functioning Psychological distress Health perceptions (general) Pain (bodily) Energy/fatigue Psychological well-being Sleep Cognitive functioning Quality of life PROMIS = Patient Reported Outcomes Measurement Information System = Quality of Well-Being Scale (1973) PROMIS

24. PROMIS integrates the fields of… Information Technology PROMIS Qualitative Research Survey Research Psychometrics

25. Psycho- metric Testing Item Bank (IRT-calibrated items reviewed for reliability, validity, and sensitivity) Short Form Instruments CAT Items from Instrument A Item Pool Items from Instrument B Items from Instrument C New Items  Questionnaire administered to large representative sample             Secondary Data Analysis Cognitive Testing Focus Groups Content Expert Review

26. CaPS bank in development PROMIS v1.0 bank PROMIS area tested but no bank developed for v1.0 Area addressed (in part) by bank within lineage Area not addressed yet * = Additional cancer-specific PROMIS bank PROMIS Domain Framework

27. 27 What Do We Need to Launch a Useful Clinical PRO Lab? Outcomes that matter to patients and providers Practical (brief) and inexpensive Covers a range, including normal Greater precision Comparability of scores Ease of interpretation Reference values / normal ranges

28. PROMIS Profile Short Forms 28 * reduced set (the full bank has 112 items) was used for real data simulation purposes Anxiety 29 Anxiety 29 Depression 28 Depression 28 Fatigue 95 Fatigue 95 Pain Impact 41 Pain Impact 41 Sleep Disturbance 27 Sleep Disturbance 27 Physical Function 86* Physical Function 86* Social Role 14 Social Role 14 4 6 8 Mental Physical Social

29. PROMIS Profile Short Forms 29

30. PROMIS Profile Subscales: Test Information Functions 30

31. PROMIS Profile Subscales: Correlations with Full Banks

32. 32 “Ceiling Effect” Outcomes that matter to patients and providers Practical (brief) and inexpensive Covers a range, including normal Greater precision Comparability of scores Ease of interpretation Reference values / normal ranges What Do We Need to Launch a Useful Clinical PRO Lab?

33. PROMIS-57 Profile by General Health Rating: In general, would you say your health is: 33

34. PROMIScore - PSF 34

35. 35 Outcomes that matter to patients and providers Practical (brief) and inexpensive Covers a range, including normal Greater precision Comparability of scores Ease of interpretation Reference values / normal ranges What Do We Need to Launch a Useful Clinical PRO Lab? g

36. Computerized Adaptive Testing (CAT) Estimates location (severity; capability) of a person on a domain (concept) by selecting questions based on that person’s prior answers Iteratively estimates a person’s standing on the domain (e.g., depressive symptoms) and administers only the most informative items, achieving precision with a minimum possible number of questions.

37. Beginning of CAT T-Score = 50SE = 10 Best Item- I felt depressed

38. I felt depressed 1.Never 2.Rarely 3.Sometimes 4.Often 5.Always T-Score = 52SE = 4 Next Best Item- I felt like a failure

39. I felt like a failure 1.Never 2.Rarely 3.Sometimes 4.Often 5.Always T-Score = 53SE = 3 Next Best Item- I felt worthless

40. I felt worthless 1.Never 2.Rarely 3.Sometimes 4.Often 5.Always T-Score = 55SE = 2 Next Best Item- I felt helpless

41. I felt that nothing could cheer me up 1.Never 2.Rarely 3.Sometimes 4.Often 5.Always T-Score = 55SE = 2

42. IRT expands the range of what we can measure Rose et al, J Clin Epidemiol 2007 (accepted) SE = 0.32 rel = 0.90 SE = 0.22 rel = 0.95 SF-36 items CAT 10 items Full Item Bank measurement precision (standard error) normed theta values HAQ items SF-12 items representative sample rheumatoid arthritis patients US-Representative Sample 0.4 0.5 0.3 0.2 0.1 -4-3-2-10+1+2

43. PROMIS Fatigue Short Form Garcia SF et al, J Clin Onc, 2007. © 2007 Reprinted with permission of the PROMIS Health Organization and the PROMIS Cooperative Group

44. 7-item Short-form 7-item CAT 98-item Bank No Fatigue Severe Fatigue SE=0.32 (r=0.90) SE=0.22 (r=0.95) Comparison of Measurement Precision Full-length Item Bank vs. Legacy vs. CAT vs. Short-form Standard Error

45. 45 CAT Assess Health Dynamically When Necessary Patient scores here CAT = Computerized Adaptive Testing

46. 46 Outcomes that matter to patients and providers Practical (brief) and inexpensive Covers a range, including normal Greater precision Comparability of scores Ease of interpretation Reference values / normal ranges What Do We Need to Launch a Useful Clinical PRO Lab?

47. 47 We Need the Health Equivalent of a Two-Sided Tape Measure 52 centimeters = 20.5 inches …and easy-to-use conversion tables with reference data and action thresholds

48. PROMIS Depression Bank and CES-D

49. Fatigue Experience and Impact 95-item bank Legacy Instruments SF-36 Vitality Scale (4 items) FACIT-Fatigue Scale (13 items)

51. 4-item SF36/Vitality 4-item CAT 13-item FACIT-Fatigue 13-item CAT 98-item Bank No Fatigue Severe Fatigue SE=0.32 (r=0.90) SE=0.22 (r=0.95) Comparison of Measurement Precision Full-length Item Bank vs. Legacy vs. CAT vs. Short-form Standard Error

52. 52 What do the results mean? Outcomes that matter to patients and providers Practical (brief) and inexpensive Covers a range, including normal Greater precision Comparability of scores Ease of interpretation Reference values / normal ranges What Do We Need to Launch a Useful Clinical PRO Lab?

53. 53 What Does a Change in Score Mean? 304050 Chronic Lung Disease Physical Component Summary (PCS) Diabetes Type II Congestive Heart Failure Average Adult Asthma After Rx Asthma Before Rx Treatment Average Well Adult 50% reduction in disease burden 33% reduction in hospitalization Substantial increase in work productivity Subsequent cost savings

54. LowLow HighHigh                                                       PRO Bank Person Score Interpretation Aids                                                     Q Q Q Q Q Q Q Q Q Likely Likely UnlikelyUnlikely Item Location Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q    

55. People and Items Distributed on the Same Metric: Fatigue People with more fatigue Items less likely to be endorsed Items more likely to be endorsed People with less fatigue

56. LowLow HighHigh                                                       PRO Bank Person Score Interpretation Aids                                                     Q Q Q Q Q Q Q Q Q Likely Likely UnlikelyUnlikely Item Location Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q    

57. LowLow HighHigh                                                       PRO Bank Person Score Interpretation Aids                                                     3040506070 M = 50, SD = 10    

58. LowLow HighHigh                                                       Interpretation Aids: Cancer example           Fatigue Score=60                                           3040506070     This patient’s fatigue score is 60, significantly worse than average (50). Cancer patients who score 60 on fatigue tend to answer questions as follows: …”I have been too tired to climb one flight of stairs: VERY MUCH …”I have had enough energy to go out with my family: A LITTLE BIT Click here if you would like to see this patient’s individual answers

59. LowLow HighHigh                                                       Interpretation Aids: Cancer example           Fatigue Score=40                                           3040506070     This patient’s fatigue score is 40, significantly better than average (50). People who score 40 on fatigue tend to answer questions as follows: …”I have been too tired to climb one flight of stairs: SOMEWHAT …”I have had enough energy to go out with my family: VERY MUCH Click here if you would like to see this patient’s individual answers

60. 60 How Long Per Concept? It depends Population monitoring 1-2 questions Group-level outcomes monitoring 5-7 questions Patient-level measurement/management 10-12 questions or CAT

61. 61 Matching Methods to Applications 1 2 3 4 5 6 7 1 3 5 7 Single-Item 1 2 3 4 5 Multi-Item Scale Population Monitoring Group-Level Outcomes Monitoring Patient-Level Management “Item Pool” (CAT Dynamic) 7 6 Most Functionally Impaired Noisy Individual Classification Very Accurate Individual Classification

62. www.nihpromis.org

63. PROMIS Family of Delivery Platforms (2009  ) Interactive Voice Recognition Internet Administered Personal Interview Self Administered Telephone Interview Hand- held Device Multiple Assessment Options TV

64. Conclusions Patient-reported outcomes are: Well characterized Easily measured Increasingly interpretable Rarely used in Clinical Practice Going forward, clinician engagement is the key to setting and applying outcome targets and practice standards