2 Accuracy and Precision

Accuracy How close a measurement is to the actual or “true value” high accuracy true value low accuracy true value 3

Precision How well several measurements agree with each other high precision low precision 4

Precision or the reproducibility of a set of measurements. A precise sample estimate will have a very small random error of estimation. Precision

Accuracy

Precision

Accuracy and Precision What can you say about the accuracy and precision in each of the following: High precision, low accuracy High precision, high accuracy 8

Question: How will each of the following affect accuracy and precision? 1. A meter stick that is missing the first centimeter. 2. A scale that has a zero point that is really five pounds above zero. 9

Solution How will each of the following affect accuracy and precision? 1. The shortened meter stick will produce measurements that have poor accuracy, but good precision is possible. 2. The poorly calibrated scale will give a weight that is not accurate, but good precision is possible. 10

A blood sample was taken from a patient and four different assays were used to measure blood glucose. The true value of the blood glucose was known to be 4.5 mmol/L. State whether the accuracy and the precision of each assay is high or low: Assay A: 4,6; 4,6; 4,8; 4,5; 4,5; 4,4; Assay B: 4,3; 3,5; 5,3; 4,6; 5,5; 3,7 Assay C: 3,5; 3,6; 3,3; 3,5; 3,4; 3,5 Assay D: 8,5; 6,4; 5,3; 7,6; 4,8; 9,3 11

Literature incosistency? High accuracy and low precision or both – low accuracy and low precision?

How confident are we in the estimation of mean/proportion we have calculated?

14

15

Measures of precision: 1. Standard error of mean, SEM Standard error of proportion, SE(p) 2. Confidence interval for mean Confidence interval for proportion

Standard error of mean, SEM Number of patients Standard deviation, SD SEM is smaller (estimate is more precise): the larger is N (number of patients) the smaller is SD (dispersion of data)

19

95% confidence interval for mean, 95% CI  Together with SEM, 95% CI is also the measure of precision  Unlike SEM, 95% CI also estimates accuracy of the result ie. 95% is accurate that interval includes true (population) mean)

95% confidence interval for mean If we draw a 100 samples from our population we would find the true population value within 95% confidence interval in 95 samples samples

Critical values for 90%, 95% and 99% level of confidence 90% CI => mean ± 1.65 SEM 95% CI => mean ± 1.96 SEM 99% CI => mean ± 2.58 SEM Level of Confidence - Critical Value 0.75, or 75% , or 80% , or 85% , or 90% , or 95% , or 98% , or 99% 2.58

Example 1 The average systolic BP before treatment in study A, of a group of 100 hypertensive patients, was 170 mmHg. After treatment with the new drug the mean BP dropped by 20 mmHg. If the 95% CI is 15–25, this means: 23 we can be 95% confident that the true effect of treatment is to lower the BP by 15–25 mmHg.

Example 2 In study B 50 patients were treated with the same drug, also reducing their mean BP by 20 mmHg, but with a wider 95% CI of -5 to +45. This CI includes zero (no change). This means: 24 there is more than a 5% chance that there was no true change in BP, and that the drug was actually ineffective..

Watch out for... The size of a CI is related to the sample size of the study. Larger studies usually have a narrower CI. 25

Example 3 – Meta analysis Fig. Plot of 5 studies of a new antihypertensive drug. 1.Which study showed the greatest change? 2.Did all the studies show change in favour of the intervention? 3.Were the changes statistically significant?

Proportion 1. Standard error of proportion, SE(p) SE( p) = √(p(1 – p)/n) 2. Confidence interval for proportion

The standard deviation describes the variability of a sample; does not describe the sample The standard error of the mean (SEM) does not describe the sample but uncertainty describes the uncertainty of how the sample mean represents the population mean.

Krebs NF, Westcott JE, Culbertson DL et. al. Comparison of complementary feeding strategies to meet zinc requirements of older breastfed infants. Am J Clin Nutr. 2012; 96:30-35 “Mean (±SEM) total absorbed zinc amounts were 0.80 ± 0.08, 0.71 ± 0.09, and 0.52 ± 0.05 mg/d for the: meat, iron-and-zinc-fortified infant cereal, and whole-grain, iron-only-fortified infant cereal groups of infants.” SEMCI Meat Fe&Zn Fe

Common mistake in the literature Misuse of standard error of the mean (SEM) when reporting variability of a sample. A critical evaluation of four anaesthesia journals. P. Nagele* British Journal of Anaesthesia 90 (4): (2001)

SD CI Standard deviation tells us about the variability (spread) in a sample. The CI tells us the range in which the true value (the mean if the sample were infinitely large) is likely to be.

What does a small standard error tell us about the sample estimate of the mean? That it is highly variable That the population standard deviation may be small That the sample size is probably small That it is imprecise

What will tend to make the standard error larger? A small variance A large standard deviation Imprecise data Inaccurate data 33