1. PROMIS: The Right Place at the Right Time? David Cella, Ph.D. Department of Medical Social Sciences Northwestern University Chair, PROMIS Steering Committee Principal Investigator, Statistical Coordinating Center

2. What is PROMIS? A nine-year commitment of NIH to improve and standardize measurement of patient reported outcomes (PROs) 50 million dollar investment (excl. ARRA) An effort to harness new psychometric methods to improve the quality and interpretability of PROs An attack on the PRO “Tower of Babel”

3. PROMIS Roadmap Goal: Create highly valid and reliable item banks and associated computerized adaptive testing Encourage wide adoption to improve assessment of self-reported symptoms other health-related quality of life domains – … across a wide range of chronic diseases

4. UNC –Chapel Hill ● ● Duke University ● Stanford University Stony Brook University ● Stony Brook University ● ● University of Pittsburgh ● University of Washington Northwestern ♥ NIH ● NIH PROMIS Network: 2004- 2009

5. “Item Bank” A large collection of items measuring one thing Items in the same bank are linked on a common metric Basis for –Computer Adaptive Testing (CAT) Items are selected to maximize precision and retain clinical relevance –Customized short forms tailored to the population and research purpose

6. Building a Bank Testing and Analysis Depression Defined and Calibrated Bank CAT and Multiple Short Forms Items Deposited After Review item

7. “PROMIS-1” Highlights Consensus-driven PRO framework Qualitative study of > 1,000 people Quantitative study of > 20,000 people Eleven Item Banks available to collaborators Website and functioning Assessment Center Ongoing clinical validation in chronic disease populations Clear path/agenda for “PROMIS-2” and expanded collaborations

8. CaPS bank in development PROMIS v1.0 bank PROMIS area tested but no bank developed for v1.0 Area addressed (in part) by bank within lineage Area not addressed yet * = Additional cancer-specific PROMIS bank PROMIS Domain Framework

9. Item Banking Item Bank Local Independence Measurement Range Target Population Static Forms Multi-Stage CAT Item banks can support a variety of scale assembly strategies

10. Advantages of Short-Forms Developed from Item Banks Flexibility in length and content –Select items matched to clinical features and severity in the target population –Select items known to provide the most information Any form selected or created produces scores on a common metric

11. PROMIS Profile Short Forms 11 * reduced set (the full bank has 112 items) was used for real data simulation purposes Anxiety 29 Anxiety 29 Depression 28 Depression 28 Fatigue 95 Fatigue 95 Pain Impact 41 Pain Impact 41 Sleep Disturbance 27 Sleep Disturbance 27 Physical Function 86* Physical Function 86* Social Role 14 Social Role 14 4 6 8 Mental Physical Social

12. 12 PROMIS Profile Subscales: Test Information Functions

13. PROMIS Profile Subscales: Correlations with Full Banks

14. 14 Estimated Power to detect a small effect (d=0.2) when the study population is centered on the general population (T-Score = 50)

15. 15 Estimated Power to detect a small effect (d=0.2) when the study population is centered on 1 SD worse than the general population

16. Computerized Adaptive Testing (CAT) Select questions based on a person’s response to previously administered questions. Iteratively estimate a person’s standing on a domain (e.g., depressive symptoms) Administer most informative items Desired level of precision can be obtained using the minimal possible number of questions.

17. Beginning of CAT T-Score = 50SE = 10 Best Item- I felt depressed

18. I felt depressed 1.Never 2.Rarely 3.Sometimes 4.Often 5.Always T-Score = 52SE = 4 Next Best Item- I felt like a failure

19. I felt like a failure 1.Never 2.Rarely 3.Sometimes 4.Often 5.Always T-Score = 53SE = 3 Next Best Item- I felt worthless

20. I felt worthless 1.Never 2.Rarely 3.Sometimes 4.Often 5.Always T-Score = 55SE = 2 Next Best Item- I felt helpless

21. I felt that nothing could cheer me up 1.Never 2.Rarely 3.Sometimes 4.Often 5.Always T-Score = 55SE = 2

22. Fatigue Experience and Impact 95-item bank Legacy Instruments SF-36 Vitality Scale (4 items) FACIT-Fatigue Scale (13 items)

23. 4-item SF36/Vitality 4-item CAT 13-item FACIT-Fatigue 13-item CAT 98-item Bank No Fatigue Severe Fatigue SE=0.32 (r=0.90) SE=0.22 (r=0.95) Standard Error Comparison of Measurement Precision Full-length Item Bank vs. Legacy vs. CAT vs. Short-form

24. PROMIS Fatigue Short Form Garcia SF et al, J Clin Onc, 2007. © 2007 Reprinted with permission of the PROMIS Health Organization and the PROMIS Cooperative Group

25. 7-item Short-form 7-item CAT 98-item Bank No Fatigue Severe Fatigue SE=0.32 (r=0.90) SE=0.22 (r=0.95) Standard Error Comparison of Measurement Precision Full-length Item Bank vs. Legacy vs. CAT vs. Short-form

26. 26 PROMIS Profile Subscales vs CATs: Correlations with Full Banks

27. PROMIS Assessment Center and Opportunities for Interaction A publicly available, adaptable and sustainable Internet-based system that: 1.Hosts 11 item banks for precise, valid, and efficient health status assessment via short forms or CAT 2.Collects and analyze patients’ responses 3.Provides instant health status reports to users to: Enable and enhance research Improve clinical decision-making Facilitate policy-making by health plan and systems and public programs 4. Supports health services / comparative effectiveness research applications

28. http://www.nihpromis.org

29. 29

30. Assessment Center Aims Assessment Center supports… Research and clinical applications CAT Custom or off-the-shelf short forms Non-PROMIS measures welcome Study set up, maintenance and storage Scoring, reporting and interpretation

31. Domains Items in Bank Items in First Short Form Emotional Distress – Anger298 Emotional Distress – Anxiety297 Emotional Distress – Depression288 Fatigue957 Pain – Behavior397 Pain – Impact416 Physical Function12410 Satisfaction with Discretionary Social Activities127 Satisfaction with Social Roles147 Sleep Disturbance278 Wake Disturbance168 Global Health10 2009 PROMIS Ver 1.0 Instruments

32. http://www.nihpromis.org http://www.nihpromis.org Thank you http://www.nihpromis.org http://www.nihpromis.org