Systematic review an overview and posing the question Dr. Will Whiteley CSO Clinical Academic Training Fellow Division of Clinical Neurosciences, University of Edinburgh

Why review the literature systematically? We often need to make sense of multiple studies addressing the same question We can either pick those studies that most agree with our preconceptions or make an unbiased assessment of as many of the studies as possible The unbiased approach is known as systematic review, and when done well can lead to the most reliable synthesis of the data.

Cumulative meta-analysis Expert opinion RCTs - clear benefit Experts follow The use of thrombolysis in myocardial infarction

Proteus effect Early extreme contradictory estimates may appear in published research: The Proteus phenomenon in molecular genetics research and randomized trials John P.A. Ioannidisa, Thomas A. Trikalinosa

Systematic review aims to: answer a specific question, reduce bias in the selection and inclusion of studies, appraise the biases of the included studies to summarise them objectively commonly require more time and people than narrative reviews

Systematic review – a medical novelty? astronomy Imperial army medicine quantum physics psychology, education, criminology, agriculture, ….

Sir R.A. Fisher Karl Pearson ‘To call in the statistician after the experiment is done may be no more than asking him to perform a post-mortem examination: he may be able to say what the experiment died of’

Advantages and disadvantages of systematic review to your research projects A systematic review of your research question will take longer than a simple narrative review The results of your systematic review (even if in includes a meta-analysis) may be wrong 65% 70% Discrepancies between meta analysis and subsequent randomised controlled trials N Engl J Med 1997;337:536-42.

However, a systematic review is more likely to be published and cited than a narrative one!

Systematic review – overview of process Clear question to answer, and pre-specified selection criteria Clearly defined methods for data extraction – to remove as much bias as possible from the data selection process Comprehensive search of the published (and if time available unpublished) literature – to ensure ‘negative’ studies are found which might not be published in the top tier journals Duplication of data extraction and study selection Summary estimation – if sufficient high quality studies are found

Posing the research question In neurologically asymptomatic people, what is the prevalence of imaging findings on high resolution brain MR images? e.g. Patients Intervention Control Outcome Decide what type and quality of studies you will accept; this will be different for an established treatment to a new biomarker of disease A good question will help to define your search strategy and study selection In patients within one week after ischaemic stroke, what is the prognostic value of IL6 for disability or death? In patients within 48 hours of subarachnoid haemorrhage, what is the value of oral simvastatin compared to control for prevention of radiological vasospasm?

Data extraction Decide prior to looking at the literature what data you will need to answer your question What is likely to contribute to study related heterogeneity Design a form – either paper or in the database program of your choice Make sure it is attractive and easy to fill in Go through a few iterations of design Data extract in duplicate (choose a helpful colleague)

Measuring bias Most study designs have potential for bias that can be minimised by taking appropriate precautions ‘Quality’ is a measure of this bias Set a ‘quality threshold’ for inclusion e.g. prospective cohort studies only or Investigate effect of quality on outcome Quality measures are available for a number of study designs Prospective observational studies MOOSE Diagnostic studies STARD Randomised controlled trials QUORUM Genetic associations HuGE reviews etc. etc.

In summary Systematic review aims to reduce bias in assessing literature Select you question carefully – make sure it is answerable Pre-specify your methods of study selection and data-extraction Perform your search strategy ….