1. Interpreting Basic Statistics
Beginners’ statistics for assessing the effectiveness of an intervention
Tom Osborne, Librarian

2. Basic Medical Statistics
Statistics which compare risks
Statistics which test confidence
Forest Plots
Statistics which analyse clinical investigations and screening
Statistics which test differences

3. Statistics Which Compare Risk

4. CER & EER
Control event rate (CER) = Risk of outcome event in control group
= no in the control group with event = ?
total no in the control group
Risk of mortality in the control group is ?%
Experimental event rate (EER) = Risk of outcome event in experimental group
= no in the experimental group with event = ?
total no in the experimental group
Risk of mortality in the statin group is ?%

5. Relative Risk (RR)
Compares the risk of having an event between two groups
RR=1 the event is equally likely in both groups
RR<1 event is less likely to happen than not (i.e. the intervention reduces the chance of having the event)
RR>1 event is more likely to happen than not (increases the chances of having the event)
The smallest value an RR can take is 0

6. Relative Risk RR = EER /CER =
Compares the risk of having an event between two groups
RR = EER /CER =
RR = EER /CER = 0.11/0.14 = 0.78
The relative risk (RR) of mortality in the pravastatin group is 0.78 (approximately three-quarters) of that in the control group.

7. Relative Risk/Odds Ratio

8. Relative Risk Reduction (RRR)
The reduction in rate of the event in the treatment group relative to the control group
RRR = 1 – RR = ?
The relative risk was ?% lower for statin than the control group or there is a ?% reduction in risk for patients in the statin group relative to those patients in the control group.
RRR = 1 – RR = = 0.22 = 22%
The relative risk was 22% lower for statin than the control group or there is a 22% reduction in risk for patients in the statin group relative to those patients in the control group.

9. Absolute Risk Reduction (ARR)
The difference in absolute risk of a particular event between 2 groups. Also know as the risk difference.
ARR = 0 no difference between the 2 groups
ARR = CER – EER = ?
The absolute risk of mortality was ?% lower in the statin group than in the control group or statins reduces the risk of mortality by ?%
ARR = CER – EER = 14% – 11% = 3%
The absolute risk of mortality was 3% lower in the statin group than in the control group or statins reduces the risk of mortality by 3%

10. Numbers Needed to Treat (NTT)
The number of people who need to be treated in order to prevent one additional outcome of interest.
NNT = 1/ARR = ?
? patients have to be treated with statin in order to avoid one additional death
NNT = 1/ARR = 33
33 patients have to be treated with statin in order to avoid one additional death
A very small NNT means that a favourable outcome occurs in nearly every patient who receives the treatment and in few patients in a comparison group
NNTs close to 1 are theoretically possible, they are almost never found in practice
Useful for comparing the effectiveness of a number of different interventions

11. RR vs. ARR
Consider 2 RCTs of a new drug done on 2 populations at risk of a heart attack over 10 years
RCT1 (n=200)
High risk group: 90/100 of those not receiving the drug (control) will have a heart attack. 60/100 of those receiving the drug will have a heart attack.
RCT 2 (n=200)
Low risk group: 3/100 of those not receiving the drug (control) will have a heart attack. 2/100 of those receiving the drug will have a heart attack.

12. RR =
RRR =
The relative risk was ?% lower for new drug than the control group for high risk patients
RR = (60/100) / (90/100) = 0.666˙
RRR = 1-RR = ˙ = 0.333˙
The relative risk was 33% lower for new drug than the control group for high risk patients
RR = (2/100) / (3/100) = 0.666˙
The relative risk was 33% lower for new drug than the control group for low risk patients
RR =
RRR =
The relative risk was ?% lower for new drug than the control group for low risk patients

13. ARR =
NNT =
? patients have to be treated with statin to avoid one additional death
ARR = CER – EER = (90/100) – (60/100) = 0.3
NNT = 1/ARR = 1/0.3 = 3.3
4 patients have to be treated with statin to avoid one additional death
ARR = CER – EER = (3/100) – (2/100) = 0.01
NNT = 1/ARR = 1/0.01 = 100
100 patients have to be treated with statin to avoid one additional death
The absolute risk of mortality was 30% lower in the new treatment group than in the placebo group.
The absolute risk of mortality was 1% lower in the new treatment group than in the placebo group.
Look at the baseline risk – These phenomena may be factors in the design of drug trials. For example, a drug may be tested in severely affected people in who the ARR is likely to be impressive, but is subsequently markets for use in less severely affected patients in whom the ARR will be substantially less.
RRR usually remains unchanged across disease severity
ARR will vary according to events rates
RRR does not take into account initial baseline risk of the outcome event
To be meaningful the RRR should always be viewed in context with the underlying baseline risk (incidence) of the event
ARR =
NNT =
? patients have to be treated with statin to avoid one additional death

14. Odds Ratio
Expresses the odds of having an event compared with not having an event:
OR=1 the event is equally likely in both groups (i.e. no difference)
OR<1 event is less likely to happen than not (i.e. the treatment reduces the chance of having the event)
OR>1 event is more likely to happen than not (increases the chances of having the event)
The smallest value an OR can take is 0
Calculate the Odds Ratio:
OR = (498/4014) ÷ (633/3869) ≈ ?
The odds ratio for mortality for people taking statins compared to the control is ?
Online Calculator:
0.76

15. Statistics Which Test Confidence

16. P-values
The probability (ranging from zero to one) that the results observed in a study could have occurred by chance. (Bandolier)
Convention states we accept p-values of p<0.05 to be statistically significant. (Bandolier)
The P value is computed from the F ratio which is computed from the ANOVA table.
P value
Interpretation
P<0.05
The result is unlikely to be due to chance, a statistically significant result.
P>0.05
The result is likely to be due to chance, not a statistically significant result.
P= 0.05
the result is quite likely to be due to chance, not a statistically significant result.

17. Significant at Cut Off? P value P<0.05 P<0.01 P<0.001 P=0.049

18. Confidence Intervals What is a confidence interval?
If the same trial were to be repeated many times over, the 95% CI would define the range of values within which the true population estimate would be found in 95% of occasions
What can a confidence interval indicate?
Whether a result is statistically significant
Indication of precision
Strength of the evidence
Online

19. Interpreting CIs Measure of effect Interpretation of CI
Binary outcome,
Ratio
If a CI for an RR or OR, includes 1 then we are unable to demonstrate statistically significant difference between the two groups
Continuous outcome, Mean difference
If a CI for a RRR, ARR, includes 0 we are unable to demonstrate a statistically significant difference between the two groups compared

20. Confidence Intervals
“Trials examined the effect of education programmes on improvement in lung function in asthma sufferers”
Study
Mean difference (95% CI)
Christiansen
0.35 ( )
Weingarten
1.24 ( )
Toelle
0.47 ( )
Are educational programmes effective at increasing lung function?
Which study/studies show a significant result?
Which study demonstrated the strongest evidence?
Yes (all answers
2 & 3 because they DON’T cross one
Toelle – less breadth, doesn’t cross 0
Measure of effect
Interpretation of CI
Binary outcome,
Ratio
If a CI for an RR or OR, includes 1 then we are unable to demonstrate statistically significant difference between the two groups
Continuous outcome, Mean difference
If a CI for a RRR, ARR, includes 0 we are unable to demonstrate a statistically significant difference between the two groups compared

21. Forest Plots

22. Forest Plots
“Effect of probiotics on the risk of antibiotic associated diarrhoea”

23. The label tells you what the comparison and outcome of interest are
Effect of probiotics on the risk of antibiotic associated diarrhoea
Look at the title on the forest plot – i.e. the outcome under investigation
The outcome is diarrhoea, ask: ‘Do we want the probiotics to increase or reduce the event. We want it to reduce the event, therefore to show that the treatment works we look for odds ratios <1
On the LHS – list of authors of the individual studies included in the systematic review/meta-analysis for the outcome under investigation

24. Scale measuring treatment effect. Take care when reading labels!
Effect of probiotics on the risk of antibiotic associated diarrhoea

25. Treatment effect sizes for each study (plus 95% CI)
Effect of probiotics on the risk of antibiotic associated diarrhoea
The Squares represent the Odds ratios for each of the individual studies (Odds ratios also given in the column on the RHS)
The size of the squares represents the weight each study is given in the Meta-analysis (also seen in last column). The larger the square the more weight the study carries in the MA (usually due to a larger sample size). Which is the largest study?

26. The % weight given to each study in the pooled analysis
Effect of probiotics on the risk of antibiotic associated diarrhoea
The size of the squares represents the weight each study is given in the Meta-analysis (also seen in last column). The larger the square the more weight the study carries in the MA (usually due to a larger sample size). Which is the largest study?
Why do we weight studies?
One approach to combining trials would be just to add up all the treatment groups together and control groups together and compare?
Breaks the power of randomisation
Imbalances within trials will introduce bias
Point estimate

27. Horizontal lines are confidence intervals Diamond shape is pooled effect Horizontal width of diamond is confidence interval
Effect of probiotics on the risk of antibiotic associated diarrhoea
The horizontal lines crossing each square are the 95% Confidence intervals for the individual studies. The longer the line, the bigger the confidence interval, the less precise the study results in comparison.
NB You can have lots of trials that cross the line of no effect yet when cumulative show an effect, this is because as individual trials there is no statistically significant difference but when added together the result is more powerful and stats. sig

28. The vertical line in middle is the line of no effect For ratios this is 1, for means this is 0
Effect of probiotics on the risk of antibiotic associated diarrhoea
The line of no effect (ie where there is no difference between the treatment and comparison) is at 1 (because we are looking at CIs around Odds ratios). Therefore any study whose confidence intervals cross this line are not statistically significant and we don’t know whether the treatment works or not.
Ask: Which study yielded the most precise results (Adam) and why? (Shortest CI)
Ask: Which study yielded the least precise results (Lewis) and why? (longest CI)

29. Exercise
Do the studies favour group behaviour for smoking cessation?
Which study/studies show a significant result?
Which study demonstrated the strongest evidence? 25

30. Statistics Which Analyse Clinical Investigations and Screening

31. Sensitivity Specificity
Sensitivity:  If a person has a disease, how often will the test be positive (true positive rate)?  Put another way, if the test is highly sensitive and the test result is negative you can be nearly certain that they don’t have disease. 
Specificity:  If a person does not have the disease how often will the test be negative (true negative rate)? In other terms, if the test result for a highly specific test is positive you can be nearly certain that they actually have the disease.
Used to analyse the value of screening or tests

32. Sensitivity Specificity
A Sensitive test helps rule out disease (when the result is negative). Sensitivity rule out or "Snout"
Sensitivity= true positives/(true positive + false negative)
A very specific test rules in disease with a high degree of confidence Specificity rule in or "Spin".
Specificity=true negatives/(true negative + false positives)

33. SnNOut & SpPIN!!!!!
A very specific test, when positive, helps rule-in disease (SpPIn). For example, if a test was 95% specific but only 70% sensitive, and 10% of patients had the disease, you get the following 2 x 2 table: 14 out of 25 patients with a positive test have the disease
Disease
No Disease
Positive test 14 9
Negative test 6
171

34. SnNOut & SpPIN!!!!!
A test that is very sensitive is generally very good at ruling out disease when negative. The acronym is "SnNOut". For example, consider a test which is 95% sensitive, 60% specific, with a pre-test probability of disease of 10%: Only 1 of 109 patients with a negative test has the disease in question.
Disease
No Disease
Positive test 19 72
Negative test 1
108

35. Quick Quiz
A very sensitive test, when negative, helps you: a: Rule-in disease b: Rule-out disease c: Confuse medical students d: Save money A test which is highly specific, when positive, helps you:
B: A test that is very sensitive is generally very good at ruling out disease when negative.  The acronym is "SnNOut". 
A: A very specific test, when positive, helps rule-in disease (SpPIn). 

36. Two-way table & Calculations
Disease
No Disease
Positive A
B (false positive)
Negative
C (false negative) D
Sensitivity: If the patient has the disease, we need to know how often the test will be positive: This is calculated from A
A + C
Specificity: If the patient is in fact healthy, we want to know how often the test will be negative: This is given by: D
D + B

37. Two-way table & Calculations
Disease
No Disease
Positive A
B (false positive)
Negative
C (false negative) D
Positive Predictive Value: If the test result is positive, what is the likelihood that the patient will have the condition: A
A + B
Negative Predictive Value: If the test result is negative, what is the likelihood that the patient will be healthy: This is given by: D
D + C

38. Exercise
100 patients were tested for haematemesis. The presence or absence of gastric cancers was diagnosed from endoscopic findings and biopsy:
Present
Absent
Positive 20 30
Negative 5 45
Calculate the Sensitivity = If gastric cancer is present, there is ? chance of the test picking it up
Calculate the Specificity = If there is no gastric cancer there is ? chance of the test being negative – but ? will have a false positive result.
Calculate the PPV = There is a ? chance, if the test is positive, that the patient actually has gastric cancer.
Calculate the NPV = There is a ? chance, if the test is negative, that the patient does not have gastric cancer. However, there is still a ? chance of a false negative, i.e. that the patient does have gastric cancer.
Calculate the Sensitivity = If gastric cancer is present, there is 80% (0.8) chance of the test picking it up
Calculate the Specificity = If there is no gastric cancer there is 60% (0.6) chance of the test being negative – but 40% (0.4) will have a false positive result.
Calculate the PPV = There is a 40% (0.4) chance, if the test is positive, that the patient actually has gastric cancer.
Calculate the NPV = There is a 90% (0.9) chance, if the test is negative, that the patient does not have gastric cancer. However, there is still a 10% chance of a false negative, i.e. that the patient does have gastric cancer.

39. Statistics Which Test Differences
Much more difficult statistics!

40. Parametric Tests Analysis of Variance (ANOVA) t test
Compares the means of two or more samples to see if whether they come from the same population. A table is created and then used to calculate f values and P-Values.
t test
Testing the probability that samples come from a population with the same value. Proves the study has had an effect. It's pretty much impossible to interpret the t-value without knowing the sample sizes in the study. For the overwhelming vast majority of situations, a t- value of 6.67 will be "statistically significant.” The further away from 0 the better. Use P-Values.
Parametric tests are only used when data follow a ‘normal’ distribution.

41. Mann-Whitney U test
A non-parametric statistic used when data are not normally distributed (and thus unsuitable for parametric tests).
Doesn’t state the size of a difference, only the likeliness of difference. For example a study looking at the ages of two groups of triaged patients might use a Mann-Whitney U test to test the hypothesis that there’s no difference in the ages between the two groups.
Very difficult to understand
Statisticians ‘rank’ data and compare the ranks
Go straight to the P-Value for results

42. Chi-Squared Usually written as X
A measure of the difference between actual and expected frequencies. Difficult to interpret by itself, dependent on a number of other factors studied. Gives approximate P-Value and is inappropriate for small samples.
Use the P-Value to see likelihood there is no difference between the groups.
Some papers will give the “Fisher’s exact test” results instead. This is usually stronger as it gives an exact P-Value. Other non-parametric tests include: the Wilcoxon Signed-Rank Test, the Kruskal-Wallis Test, and the Friedman Test 2

43. Just use the P- Value
And breathe…

44. Library outreach service
The library
Level 5, Education Centre
Upper Maudlin St
Tel. ext .