Development of Patient-Centered Questionnaires FDA/Industry Workshop – Washington DC 2005 Cindy Rodenberg, Ph.D Procter & Gamble Pharmaceuticals

2 Why Develop A New Instrument? Development of clinical treatments requires a suitable instrument to measure aspects particular to the disease and population. Previously published instruments may: Not adequately measure intended treatment effect – e.g. new research area Not adequately reflect the patient population Use language that is dated Not translated or linguistically harmonized

3 Population-specific: Is that really important?

4 Three Steps of Instrument Development Preliminary Instrument Development Face and content validity Qualitative Content Validation Translation and linguistic validity Quantitative Analysis And Validation Reliable, valid, and sensitive to treatment effect

5 Step 1: Preliminary Instrument Development Item Generation - Identify characteristics of the disease and ensure adequate sampling of content to provide valid measurement Face validity – Appears relevant to the outcome intended to capture. Content validity – Sampling of items, presentation, and measurement of all aspects of the disease or states relevant to the patient.

6 Importance of content validity Study specific instruments may be biased …if since the sponsor decides what it wants to ask, and thus can emphasize areas where the product should excel and deemphasize potentially troublesome areas… Smith, N. Quality of Life Studies from the Perspective of an FDA Reviewing Statistician, Drug Information Journal, 1993; 27:

7 Methods – Item Generation Experts Menopausal Sexual Interest Questionnaire (MSIQ) – 10 item questionnaire judged to capture key components of sexual desire and response Literature review/Existing questionnaires Mini Mental from the WAIS Intelligence Test Patient-centered approach - Focus groups and individual patient interviews Profile of Female Sexual Function (PFSF)

8 Patient Centered Approach Focus groups and individual interviews Structured, Semi-structured, or Unstructured Subjects selected to ensure adequate representation across relevant population subgroups Sample until data saturation – in other words, no new information. Approximately 5-10 subjects per subgroup

9 Response Levels Direct Estimation Method – Directly quantify magnitude of a trait Visual Analog Scale: Arthritic Pain? Adjectival Scale: Worst pain ever No Pain PoorFairGoodExcellent How would you rate your overall response to treatment? 1234

10 Step 2: Qualitative Content Validation Elimination and refinement to ensure items: Represent patients symptoms/experiences Have clear and unitary meaning Have similar meaning across translations Cognitive Interview Technique 1-on-1 Interview Interviewer probes on thought process used by patient in determining response Iterative process

11 PFSF Content Validation I feel like a sexual personFrench: I feel like a prostitute I felt relaxed about sexRelaxed about the subject of sex – not actually having sex I felt apathetic about sexVocabulary too high It took forever to get aroused Patients who didnt have sex over the past month responded never

12 Step 3: Quantitative Statistical Approaches Methods for item reduction and domain identification Methods for assessing reliability and validity

13 Item Reduction and Domain Identification Data quality assessment Missing data frequencies Item Frequency Distributions – Floor/Ceiling effects Eliminated item - We had sex any time and any place Ability to detect known group differences T-test - parametric Area Under an ROC Curve - nonparametric

14 Item Reduction and Domain Identification Principle components analysis and Factor analysis: Unidimensional domains – measuring some facet of same underlying construct Items loading across multiple factors or small loadings (<0.4) across all factors are eliminated Multi-trait analyses – Item-total correlations to assess: Convergent/Divergent Validity – Items correlate more with their own domain than with other domains Software available – Multi-trait Analysis Program

15 Item Reduction - Factor Analysis Factor Item DesireResponsive- ness Disinterest I wanted to avoid sex I avoided having sex I looked forward to sex Sex didnt matter to me I didnt care about sex at all

16 Evaluating an Instrument - Validity Validity - Property of measuring what is intended to be measured Content validity – adequate sampling and representation of relevant disease characteristics Concurrent validity – Good correlation with a Gold Standard Construct validity – Extent to which an instrument measures the underlying constructs purported to represent

17 Construct Validity Convergent validity - Instrument correlates with other measures off related aspects Divergent validity - Instrument is not correlated with measures on unrelated aspects Known-groups validity Ability to distinguish groups known to differ Treatment Sensitivity

18 Treatment Sensitivity Decreases in Distress significantly greater on Testosterone than on Placebo

19 Desire score differentiates normal libido women from low libido women

20 Evaluating an Instrument - Reliability Reliability – agreement between two or more measures of the same thing Test-retest reliability – reproducibility of score over separate measurement occasions Pearson correlation coefficient Intraclass correlation coefficient Internal consistency reliability – homogeneity of items within a domain Cronbach's alpha

21 ReferencesReferences Fayers P.; Hays R. Assessing QoL in Clinical Trials. Second edition: Streiner, D. L.; Norman, G. R. Health measurement scales: A practical guide to their development and use 2nd Ed. Oxford University Press: Shrout, P.E.; Fleiss, J. L. Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 1979, 86, (2), 420 – 428. Nunnally, J. C.; Bernstein, I. H. Psychometric Theory, 3rd ed. McGraw- Hill: New York, Juniper, E. F.; Guyatt, G. H.; Jaeshke, R. How to develop and validate a new health-related quality of life instrument. In Quality of life and pharmacoenconmics in clinical trials, 2nd Ed., Soilker, B., Ed.; Lippincott- Rqven: Philadelphia, 1996; Rodenberg, C.A.; Kuznicki J.; Yiu G. Instrument Development and Validation. In Encyclopedia of Biopharmaceutical Statistics. 2002