Supporting syntheses of the literature SHINE Quality – The Whole Picture 26 th November 2004

Content Definition and purpose Types of syntheses The role of the information specialist –Identifying previous reviews, scoping searches, retrieval of primary studies Challenges of the role

Definition and purpose Definition –reduce large quantities of primary information into ‘palatable pieces for digestion’ by decision makers (Mulrow, 1995) –produce reliable results on which to base decisions by using robust processes to locate, appraise and synthesise research evidence (Song et al., 2000) Purpose –Quality improvement

Systematic reviews ‘…a review [that] has been prepared using some kind of systematic approach to minimising biases and random errors, and that the components of the approach will be documented in the materials and methods section.’ (Chalmers & Altman, 1995) Organisations undertaking systematic reviews –Cochrane Collaboration –Campbell Collaboration –Centre for Reviews and Dissemination

Health Technology Assessments ‘a multidisciplinary field of policy analysis that studies the medical, social, ethical, and economic implications of the development, diffusion and use of a health technology’ (INAHTA, 2000) Organisations undertaking HTAs –NHS QIS –NICE –INAHTA

Guidelines ‘Clinical guidelines are systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances.’ ( Organisations preparing evidence-based guidelines –SIGN –NICE –GIN

One topic: three syntheses Systematic review –In alcohol dependence, is acamprosate or naltrexone more effective for preventing relapse? HTA –Prevention of relapse in alcohol dependence Guideline –The management of harmful drinking and alcohol dependence in primary care

The role of the information specialist To comprehensively identify studies to answer the review question(s) Stages –Identifying previous reviews –Undertaking scoping searches –Retrieving primary studies

Identifying previous reviews Preliminary evaluation of the evidence base –Has this question already been answered? –If not, can an answerable question be formulated? –Is there a reasonable evidence base? CDSR, DARE, HTA database, Guidelines Finder, National Guideline Clearing House Ongoing reviews Bibliographic databases, journals and websites

Scoping searches ‘Scoping searches establish the feasibility of a review question and investigate how searching and appraisal can be operationalised.’ (Booth & Fry-Smith, 2003) –establish the volume and quality of the primary literature –aid development of the review protocol –indicate how much the review might cost

Retrieval of primary studies Formulating clear search questions Recognising the ‘best’ evidence to answer the question(s) Identifying the most likely sources to search Searching information sources effectively (Snowball, 2002)

Formulating clear search questions The product –The ‘well-built’ clinical question (Richardson et al., 1995) –The ‘well-formulated’ question (Counsell, 1997) –The ‘anatomy’ of a question (Sackett et al., 1997) The process –Focusing the question (Oxman et al., 1993) –Formulating the question (Counsell, 1997) –Formulating the problem (Cochrane Handbook) Slide reproduced with kind permission from Andrew Booth

Formulating clear search questions ‘Ask a poor question and you will get a poor review. A clear question also helps the reader rapidly assess whether the review is relevant to his or her own clinical practice’ (Counsell, 1997). Clearly framed questions ‘guide much of the review process including strategies for locating and selecting studies or data, for critically appraising their relevance and validity, and for analysing variation among their results’ (Cochrane Handbook). Slide reproduced with kind permission from Andrew Booth

Formulating clear search questions POPULATION/PATIENT – who? INTERVENTION/EXPOSURE – what? COMPARISON – compared with? OUTCOME – with what result? PICO, PIOC or PECO Slide reproduced with kind permission from Andrew Booth

Worked example 1 Population/Patients: are persons with alcohol dependence Intervention: who are treated with acamprosate Comparison: compared with those treated with naltrexone Outcome(s): less likely to relapse (into heavy drinking and dependence)?

Worked example 2 Population/Patients: should pregnant women be offered Intervention: first trimester ultrasound scanning (nuchal translucency measurement) Comparison: or second trimester serum screening Outcome(s): accuracy of risk assessment for trisomy 21 (Down’s Syndrome)?

Translating PICO into a search strategy POPULATION Alcoholics OR Alcoholism And INTERVENTION Acamprosate OR Campral And COMPARISON Naltrexone OR Trexan And OUTCOME Temperance OR Sobriety Slide reproduced with kind permission from Andrew Booth

Is PICO effective (1)? Statistically significant correlation between use of PICO-structured form and complexity of search strategy (P = 0.002) and between clinical requests handled by PICO-structured form and fewer items retrieved (P = 0.028). However, librarians preferred minimally structured forms to PICO-structured forms in every dimension except informativeness. (Booth et al., 2000) Slide reproduced with kind permission from Andrew Booth

Recognising the ‘best’ evidence to answer the question Therapy Diagnosis, screening Aetiology Prognosis, natural history Cost-effectiveness Qualitative (McKibbon, 1999)

Recognising the ‘best’ evidence to answer the question T: In alcohol dependence, is acamprosate or naltrexone more effective for preventing relapse? D: What is the accuracy of screening for alcohol problems in primary care? A: What are the environmental risk factors associated with binge drinking in college students? P: What is the long-term prognosis of patients with alcoholic cardiomyopathy following total abstinence? CE: Are brief interventions cost-effective for problem drinkers? Q: What are adolescents' perceptions of parental alcoholism?

Identifying the sources to search (1) Subject –General, eg: MEDLINE, EMBASE –Subject-specific, eg: AMED, CINAHL, BNI Type of question/type of evidence –Therapy, eg: Cochrane Database of Systematic Reviews (CDSR); Cochrane Central Register of Controlled Trials (CENTRAL) –Cost-effectiveness, eg: NHS Economic Evaluation Database (NHS EED); Health Economic Evaluations Database (HEED)

Identifying the sources to search (2) Databases –MEDLINE, EMBASE, CINAHL, PsycINFO, AMED, HMIC Searchable e-journal packages –Science Direct, Ingenta, Registers of Research –mRCT, National Research Register, cancer.gov Conference Proceedings –Index to Conference Proceedings, ISI Proceedings Dissertations –Digital Dissertations, Index to Theses Interested parties –Professional associations, voluntary groups Internet ….

Searching information sources effectively Translating the parts of the question (PICO) into search terms Combining search terms using Boolean logic Sensitivity vs precision Sensitivity maximising –Synonyms –Subject headings and free-text –Truncation and wildcards ($, ?, *) –Proximity operators (adj, near, same) Search filters, quality filters

1.Alcoholism/ 2.Alcohol drinking/ 3.alcoholic? 4.alcoholism.tw 5.dipsomania$.tw 6.or/1-5 7.Taurine/ 8.acamprosate.tw 9.campral$.tw. 10.or/7-9 A search strategy 11.Naltrexone/ 12.naltrexone.tw 13.trexan.tw 14.or/ Temperance/ 16.temperance.tw 17.sobriety.tw 18.((control$ or reduc$ or restrict$) adj2 drink$).tw 19.or/ and 10 and 14 and 19 P

Challenges of the role Publication bias Time vs quality trade-off Complexity of review questions Widening perspectives

References Alderson P, Green S, Higgins JPT eds. Cochrane Reviewers’ handbook [updated March 2004]. Available from [Accessed 11 October 2004]. Booth A and Fry-Smith A Developing the research question. In: Etext on Health Technology Assessment (HTA) information resources, Available from Internet, [Accessed 14 March 2003]. Booth A, O'Rourke AJ, Ford NJ Structuring the pre-search reference interview: a useful technique for handling clinical questions. Bull Med Libr Assoc 88(3): Chalmers I and Altman DG eds Systematic reviews. London, BMJ Publishing Group. Counsell C Formulating questions and locating primary studies for inclusion in systematic reviews. Annals of Internal Medicine, 127(5),

References INAHTA INAHTA: International Network of Agencies for Health Technology Assessment: global networking for effective healthcare, Stockholm, INAHTA. McKibbon A PDQ evidence-based principles and practice. Hamilton, Ontario: BC Decker Inc. Mulrow C Rationale for systematic reviews. In: Chalmers I and Altman DG eds. Systematic reviews. London, BMJ Publishing Group. Oxman AD, Sackett DL and Guyatt GH Users' guides to the medical literature. I. How to get started. The Evidence-Based Medicine Working Group. JAMA, 270(17), Richardson WS, Wilson MC, Nishikawa J and Hayward RS The well- built clinical question: a key to evidence-based decisions. ACP Journal Club, 123(3), A12-A13.

References Sacket DL, Richardson WS, Rosenbery W, Haynes RB Evidence- based medicine. How to practice and teach EBM. 1st ed. London: Churchill Livingston, Snowball R 'Find the evidence'--reflections on an information skills course for community-based clinical health-care staff at the Cairns Library, Oxford. Health Information & Libraries Journal, 19(2), Song F, Eastwood AJ, Gilbody S, Duley L and Sutton AJ Publication and related bias. Health Technol Assess, 4 (10). Villanueva EV, Burrows EA, Fennessy PA, Rajendran M and Anderson JN Improving question formulation for use in evidence appraisal in a tertiary care setting: a randomised controlled trial. BMC Medical Informatics and Decision Making, 1(1), 4.

Janette Boynton Senior Health Information Scientist