1. The (ab)use of symptom scores in asthma clinical trials: a systematic review Geoff Frampton & Jonathan Shepherd Southampton Health Technology Assessments Centre

2. Background Southampton Health Technology Assessments Centre Symptom scores in asthma clinical research Symptoms are important to patients Goal of therapy is asthma control The multidimensional character of asthma is best captured using multiple outcomes

3. - binary (yes/no) scales - symptoms combined with other variables - summary proportions (e.g. % symptom-free days) Excluded: Any numeric symptom scales Included: 0 3 1 2 Scope of this presentation

4. Background Southampton Health Technology Assessments Centre Ideal features of patient-reported outcomes Validity Responsiveness Reliability (internal consistency, reproducibility) Interpretability

5. Southampton Health Technology Assessments Centre To determine by systematic review: Do they fulfil the requirements of patient-reported outcomes ? How are the scores interpreted clinically ? What types of symptom score scales are used in asthma clinical trials ? Objective Do they reflect circadian variation in asthma ? How are they analysed ?

6. Methods National Institute for Health and Clinical Excellence (NICE) review and update of guidelines on inhaled corticosteroids and long-acting beta agonists for management of asthma in adults and children Systematic reviews of asthma drug effectiveness and safety (SHTAC and PenTAG) Source of evidence: www.hta.ac.uk Systematic extraction of symptoms data Southampton Health Technology Assessments Centre

7. Results 87 randomized controlled trials (RCTs) published in 32 English language journals, 1985 to 2006 0 2 4 6 8 10 12 14 198119861991199620012006 Total number of RCTs RCTs that used tested scales RCTs that measured symptom scores J:\SHTAC\Projects\Research\Healthcare Associated Infections\Catheter education in critical care\Correspondence with expertsJ:\SHTAC\Projects\Research\Healthcare Associated Infections\Catheter education in critical care\Correspondence with experts 78 RCTs (90%)

8. Principal symptom domains in 78 asthma RCTs Overall asthma severity Not reported Dyspnoea Wheeze Cough Asthma duration or frequency Multiple domains Other 32% 15% 13% 9% 8% 7% Southampton Health Technology Assessments Centre 21 different symptom domains Results – symptom assessments

9. Numerical symptom scales used in 78 asthma RCTs 4-item 6-item 7-item Scale not reported > 7 items 5-item Other Type of scale 46% 19% 12% 10% 6% 5% 3% Southampton Health Technology Assessments Centre 10 unique numeric scales Results – symptom assessments

10. Southampton Health Technology Assessments Centre Timing of symptom assessments in 78 asthma RCTs Day Night Unclear Not reported Sum of (day + night) 32% 28% 5% 8% 6% am pm other 8% 7% 8 different timings of symptom assessments Results – symptom assessments

11. 1234567891011 11 randomized controlled trials -6 2 1 0 134 Symptom score Southampton Health Technology Assessments Centre Results – circadian patterns Asthma severity scores DayNight

12. 134 Southampton Health Technology Assessments Centre Method of analysis of symptom scores 19862006200119961991 Number of RCTs Not reported Results – analysis methods Statistically significant effects of inhaler treatment on scores 76% 37%

13. Discussion Southampton Health Technology Assessments Centre Asthma symptom scales appear to be almost all unique How should we interpret the numeric scores clinically ? (e.g. minimal important difference; NNT) There is hardly any information on their reliability, validity and responsiveness Which are the correct statistical analyses? What do the robust diurnal patterns in symptom scores mean ?

14. Southampton Health Technology Assessments Centre In conclusion, asthma clinical trials should… 2. Provide guidance on clinical interpretation 1. Share the use of fewer, tested, symptom score instruments 4. Explain the statistical testing rationale and any assumptions The views and opinions expressed in this presentation are those of the authors and do not necessarily reflect those of the UK Department of Health Thank you for your attention! 3. Precisely report the timing of symptom assessments