1. Validity of Fatigue Recall Ratings Mark P. Jensen, Ph.D. ASCPRO Steering Committee Meeting, October 2007

2. Issue of symptom recall is critical FDA Guidance “…instruments that require patients to rely on memory, especially if they must recall over a period of time, or to average their response over a period of time may threaten the accuracy of PRO date” (p. 11)

3. Validity of pain recall Tendency to over-estimate previous pain (on average; e.g., Jamison et al., 1989; Porzelius, 1995; Stone et al., 2004; Tasmuth et al., 1996; but not always -- see Babul and Darke, 1994; Kwilosz et al., 1984; finding may be related to possible tendency to consider periods of pain only; Kikuchi et al., 2006; Stone et al., 2004) There is evidence for “end” effects (although the effect sizes of end effects are rather weak 12% & 13% in 7-day recall, Stone et al., 2000; Jensen et al., 1996; 2%-4% in 24-hr recall, Jensen et al., in press)

4. Validity of pain recall Assessor status impacts accuracy (Phone call from physician elicited accurate recall ratings; call from RA elicited over- estimates; Williams et al., 2007). Variability of pain impacts accuracy (Higher variability during the week = larger over-estimate, Stone et al., 2005; and = weaker association, Kikuchi et al., 2006). Without clear instructions, people use varying strategies to determine recalled average pain ( Broderick et al., 2006; Williams, et al., 2000).

5. Validity of pain recall Recall of worst pain tends to be more strongly associated with measures of functioning then recall of average pain (e.g., Shi et al., under review; Tittle et al., 2003; Yun et al., 2004; Zelman et al., 2005) Recall of worst pain (past 7 days) is less accurate than recall of average pain, which is less accurate than least pain (Jensen et al., 1996) But…

6. Validity of pain recall Recall of average pain over the last week shows strong associations to actual averages from diaries (e.g., ICC =.82, Bolton, 1999; r >.90, Jamison et al., 2006; r =.85, Jamison et al., 1989; r =.78, Jensen et al., 1996; r =.72, Stone et al., 2000; rs =.75 &.79, Stone et al., 2004; r =.68 for h/a, Kikuchi et al., 2006). Recall ratings are responsive to changes in pain with treatment (100s [1,000s?] of clinical trials; too many to list)

7. What to do? Option 1 - Assess current pain multiple times, - Compute worst, least, and average pain from multiple ratings. - Resulting measures are likely more valid (then recall) of actual worst least and average pain. - Problem solved…

8. Except… - Assessment burden and cost. - Between 5% and 10% missing data. - Increasing accuracy of recall ratings may or may not increase ability of measure to detect change. - Judgments of validity are tied to use; fly swatter vs. bazooka.

9. Validity of pain recall Pain recall ratings - At times over-estimate past pain, - Are influenced by: most recent pain, variability, pain domain assessed, and status of assessor, and - Are adequately valid for assessing characteristic or average pain in clinical trials.

10. Validity of fatigue recall ratings - One study (Meek, Lareau, and Anderson, 2001) - 30 patients with chronic obstructive pulmonary disease rated today’s “tiredness” (and also shortness of breath) daily on 0 – 10 scales for 14 days - Asked to rate 2-wk average, greatest, and least tiredness and dyspnea on final visit

11. Validity of fatigue recall ratings - Recall ratings of average, greatest and least tiredness (and also dyspnea) were similar

12. Validity of fatigue recall ratings - In regression analyses, today’s tiredness was the strongest predictor of recall rating (consistent with an end effect) - Association between actual and recalled average was not computed

13. Validity of fatigue recall ratings - Recall (mostly 7 days, sometimes past 2-3 days or past 24 hours) ratings are commonly used in fatigue research - Recall fatigue ratings are responsive to expected changes in treatment - Findings from the one study on fatigue recall and clinical trial results are consistent with pain recall research, suggesting that…

14. Validity of fatigue recall ratings - - …fatigue recall ratings are (probably) adequately valid for assessing characteristic or average fatigue in clinical trials; but - - Research is needed to - compare worst, average and least recall, - determine if accuracy can be improved (e.g., using of electronic diaries); but - remember to examine validity with respect to the planned purpose