Presentation on theme: "Critical Appraisal. Evidence –Based Medicine Formulate problem Track down best evidence Appraise the evidence critically Implement results in clinical."— Presentation transcript:
Evidence –Based Medicine Formulate problem Track down best evidence Appraise the evidence critically Implement results in clinical practice Evaluate performance
All published studies in reputable journals: Have the perfect design ? Use the most appropriate statistical analysis? Present the results in the best way?
Methodological issues I How were the subjects recruited? Who was included/excluded ? What justification is given for the sample size? Was bias avoided/minimised?
“A sample of patients were interviewed within the first week of their admission to hospital. 46 people were assessed, representing 15% of admissions over the study period”
Early trials of anti-histamine agents for seasickness during World War 1 All soldiers crossing the Atlantic on Ship A had active drug All soldiers crossing the Atlantic on Ship B had placebo
“The main contribution of statisticians in medical research is not to carry out statistical analyses but to inject a bit of logic into the situation”
Methodological issues II Was random allocation to treatment used? Was potential degree of blindness used ? What was the duration and completeness of follow-up?
Example 1 “Prevention of wound infection”
Randomisation techniques In a randomised controlled trial: “Patients were allocated to either a control or intervention group using date of birth”
Randomisation Date of birth ‘randomisation’ = Systematic allocation (not randomisation) Open to bias
RCT ‘programming error’ in allocation of treatment
Presentation of Results
Descriptive Information Summary statistics (proportions, means, medians, ranges, standard deviations) Characteristics of non-respondents Characteristics of withdrawals from clinical trials
“…mean (sd) of social class = 3.2 (1.2)”
Baseline characteristics Treatment A Treatment B Male48%50%P=0.84 Age58.643.2P=0.19 Smokers19%13%P=0.35
“We have chosen to use parametric statistics to stay in line with the data analysis reported in the literature”
5-year survival rate after pancreatic resection Treatment A (n= 8) 25% Treatment B (n=19) 58% (p=0.35)
Example 2 “Patients with peripheral circulatory disorders”
Report from Danish Newspaper Impressive improvement in crime statistics 1983-1984 Number of reported crimes reduced by almost 50%
Report from Danish Newspaper In 1983, a man had made nuisance phone calls to the same woman (199 times) Each had been reported and counted separately The man was apprehended early in 1984
“Opportunistic” [Fishing expeditions] Pore over data until a ‘significant’ association is found Devise a biologically plausible hypothesis to fit the association
Study of parents’ occupational exposures as a risk factor for birth defects in their offspring Seven major categories of occupation identified No significant relationship (for mothers or fathers)
Study of parents’ occupational exposures as a risk factor for birth defects in their offspring Categories split into: 64 separate occupations for mother 80 for fathers 5 significant relationships found
“Procrustean” Deciding on the hypothesis to be proved and matching the data to fit the hypothesis ie. selective reporting [Procrustes, a robber in Greek mythology, made all his victims fit the length of his bed by stretching or cutting off their legs]
Exposure redefined to strengthen the association In a study of adverse effects of oral contraceptives on the outcome of pregnancy Exposure defined as OC used within 600 days before a delivery or miscarriage
Exposure redefined to strengthen the association Why 600 days? Why not 365 days (ie. 1 year)? Or 2 years? Or 18 months? Or 500 days?........
Example 3 “Exposure to X-rays”
Study by Western Electric Company to assess effects of illumination on production at the Hawthorne plant in Chicago Control group – worked under constant illumination Experimental group – worked under varying illumination
Hawthorne Effect Production increased in experimental and control groups at same rate
Hawthorne Effect Production increased in experimental and control groups at same rate Increases in production were caused by the increased attention that workers received from management
Example 4 “Bran study”
Regression to the Mean Sir Francis Galton’s publication: “Regression towards mediocrity in hereditary stature” Very tall parents tend to have children who are not quite as tall, while short parents tend to have taller children