1. Interpreting numbers ScotPHO training course March 2011 Dr Gerry McCartney Head of Public Health Observatory Division NHS Health Scotland gmccartney@nhs.net

2. Approaching numbers: some questions to ask What is being counted? –Definitions –Type of numbers - counts, means, etc. Who/where (population): what is the denominator? When (time): what time period do they cover? How (source): where did they come from? Why were they produced: is there an agenda? 68% of Doctors don’t listen to their patients SMOKERS ON PILL DOUBLE STROKE RISK

3. Incidence and Prevalence Dealt with in more detail later in course Incidence describes the number of new cases in the population over a period of time Prevalence describes the number of cases present in a population at any one point in time

4. Framework for interpreting numbers Could your interpretation be affected by either: Error Chance Confounding (the mixing of two effects) Bias (systematic departure from truth – either deliberate or unintentional)

5. Example: COPD (lung disease) variation Is COPD more common in Board A or Board B? Errors (e.g. different definitions used in each Board)? Chance (e.g. no confidence intervals used)? Bias (e.g. are there systematic differences in how disease is recorded)? Confounding (e.g. are there mixed effects – such as age structure)? NHS Board ACases per 1,000 population NHS Board BCases per 1,000 population Anytown230Smalltown190 Othertown280Seatown210 Bigtown159Hilltown149 Overall223Overall183

6. Sources of error: Mistakes in data collection, data recording, data storage, data transmission Coding errors, transcription errors Can be random or systematic Do the numbers add up? Are the number plausible?

8. Chance and interpreting numbers Most figures report data for a sample from a larger population A different sample would give a different result Year to year fluctuation can be due to chance The size of the sample dictates the degree to which a difference is likely to be due to chance Confidence intervals and p-values give estimates of the precision of a value –E.g. Relative risk of heart disease amongst diabetics is 7.4 (95% CI 6.5- 8.6) means that there is a less than 1 in 20 chance of the true value lying outwith the range 6.5 to 8.6

9. Bias – identification and interpretation Bias is a systematic alteration of figures away from the true value Examples Selection bias – critically appraise sampling strategy, loss to follow-up, response rate Information bias – completeness of data, calibration, participant self-report, recall time Publication bias – think about a funnel plot

10. Confounding: when separate effects are mixed together In this example, the effect of location is mixed with (confounded by) the effect of age The population of Western Isles is older so has higher rates of CHD admission Is CHD more common in Western Isles after taking age into account? NHS Western IslesNHS Lothian CHD admissions per 100,000 per year 350200

11. Methods for dealing with confounding Design Randomisation (only for experimental studies) Restriction (e.g. narrow the comparison groups by age, sex, ethnicity, socioeconomic status) Matching Analysis Stratification (i.e. compare sub-groups, but has dangers) Standardisation* Multivariate analysis* *dealt with in more detail elsewhere

12. Standardisation: brief interpretation A method of “removing” the effect of other factors to allow a “fair” comparison The other factors are most commonly age and sex, but standardisation can be used for other factors Standardisation shows the rates you would get if the population had a “standard” age and sex structure NHS Western IslesNHS Lothian Crude CHD admissions per 100,000 per year 350200 Directly age standardised CHD admissions per 100,000 per year 250220

13. Standardised Mortality Ratios (SMRs): This is a comparison of mortality in a population with a ‘standard’ population taking account of age structure The standard population is allocated a value of 100 for whatever the mortality rate is The age and sex standardised mortality of the population of interest is then divided by that in the standard population to give a figure for comparison with the 100 An SMR of 150 indicates that mortality is 50% higher after accounting for age and sex differences

14. Interpreting associations: does A cause B? AB AB A B AB C ? Causal relationship Confounding Chance

15. Some quick notes on interpreting graphs: Beware of: ambiguity, distortion and distraction

16. Data Ambiguity

17. Data distortion (1) 10.110.210.310.4 0123456701234567

18. Data Distortion (2)

19. Data Distortion (3)

20. Data distortion (4) General acute inpatient discharges with an alcohol-related diagnosis in any position, by gender, Scotland, 1982/3 - 2009/10

21. Data distortion (5) General acute inpatient discharges with an alcohol-related diagnosis in any position, by gender, Scotland, 1982/3 - 2009/10

22. Summary Always ask the questions: what, who, where, when, how and why Think about possible problems with data: errors, chance, bias and confounding Even when things are associated they may not be cause and effect Beware of the possibility of graphs creating distortions, distractions or ambiguity

23. Questions gmccartney@nhs.net