# 1 Clinical Epidemiology Using Evidence to Guide Practice A bit of thinking and a few simple sums.

## Presentation on theme: "1 Clinical Epidemiology Using Evidence to Guide Practice A bit of thinking and a few simple sums."— Presentation transcript:

1 Clinical Epidemiology Using Evidence to Guide Practice A bit of thinking and a few simple sums

2 What you need to know for the MRCGP Sums ARR, RRR, NNT (calculate) plus OR Impact of baseline risks P values, Confidence Intervals (principles, not how to calculate) Power (principles) Diagnosis and screening –Sensitivity, specificity, PPV, NPV (LRs unlikely?) Concepts Types of studies, including SR & MA, qualitative research (principles), health economics (principles) Where to find evidence (in the real world) How to use EBM in the consultation

3 Who likes shopping? (Oh no, I really hate that) Apples – were £30 a bag, now only £20 a bag. Would you go out and buy apples if the saving was ONLY described as ONE THIRD OFF? Apples – 3p a bag, now 2p a bag 1.Saving is 1p a bag 2.Saving is STILL one third 1.Saving is £10 per bag (Original rate – new rate). 2.Saving is one third or 33%. (original rate – new rate / original rate; i.e. 3-2 = 1, 1/3 = one third, 1/3 x 100 = 33%

4 Describing differences between treatments In a RCT, 50% of people died using medicine A. Only 45% of people died when they were given medicine B. –How much better is B than A? –Does it matter how we describe those differences? –What is the best way, or the fairest way, of describing differences?

5 In a RCT, 50% of people died using medicine A. Only 45% of people died when they were given medicine B. 1.The difference is 5%. (Control rate – experimental rate = 50% - 45% = 5%.) Absolute risk reduction (ARR) or risk difference 2.The difference is 10%. Control rate – experimental rate / control rate; 50% - 45% = 5% / 50% = 1/10 = 10%. Relative risk reduction (RRR) or risk ratio Which of these is best or fairer?

6 Same medicines, different people In a RCT, 5% of people died using medicine A. Only 4.5% of people died when they were given medicine B. 1.The difference is 0.5%. (Control rate – experimental rate = 5% - 4.5% = 0.5%.) ARR = 0.5%. 2.The difference is 10%. Control rate – experimental rate / control rate; 5% - 4.5% = 0.5% / 5% = 1/10 = 10%. RRR = 10%.

7 Imagine two trials…

8 So lets get this straight The RRR stays constant in different populations. The ARR alters in different populations – it will be much more impressive if the population has a lot of events – i.e. has a high baseline risk. But if there are not many events then a 10%, 20% or even a 30% reduction in a rare event doesnt amount to much benefit. And EVERYBODY has to take the intervention (and so is at risk of side effects).

9 Numbers needed to treat (NNT) Medicine A cures 50% of people Medicine B cures 60% of people ARR = 10% RRR = 20% Another way of looking at the absolute rate is to divide it into 100: In this case 100/10 = 10. i.e. treat ten people with B rather than A and 1 will benefit.

10 Same medicines, different people again In RCT 1: 50% of people died using medicine A Only 45% of people died using medicine B. ARR 5%, RRR 10%, NNT 20. In RCT 2: 5% of people died using medicine A. Only 4.5% of people died using medicine B. ARR = 0.5%. RRR = 10%. NNT = 200. In the higher risk population, we would only need to treat 20 people with B rather than A to save one. But in the lower risk population we would need to treat 200 people with B rather than A to save one.

11 Effect of baseline risk on ARR 0% 2% 1% Event = a coronary death or a non-fatal MI 0%6% 3%1%2%4%5% Baseline annual risk of an event ARR 4S CARE LIPID WOSCOPS ACTC

12 Mini-test (1) Calculate the ARRs, RRRs and NNTs for these trials: 1.Medicine A 15% have an MI, Medicine B 12% have an MI. 2.Medicine A 7% have an epileptic fit, Medicine B 5% have an epileptic fit. 3.Medicine A 12% develop diabetic retinopathy, Medicine B 6% develop diabetic retinopathy. 4.Medicine A 27% are readmitted with heart failure, Medicine B, 24% are readmitted with heart failure.

13 Mini-test (1) answers Calculate the ARRs, RRRs and NNTs for these trials: ARRRRRNNT 1. 3%20%33 2. 2%29%50 3. 6%50%17 4. 3%11%33

14 Clopidogrel in ACS Its really beneficial; Id want all of my patients to be taking it Most people do fine just on aspirin. Adding clopidogrel prevents only a few people having an event and theres the increased bleeding risk The specialist starts it and I dont question that

15 N Engl J Med 2001; 345: 494-502. 20% relative risk reduction 2.1% absolute risk reduction NNT 48 38% relative risk increase 1% absolute risk increase NNT 100

16 Translation: Clopidogrel significantly reduces the absolute risk of: a)CV Death, MI, Stroke taken together by 2.1% (p < 0.001) NNT 48 b)CV Death, MI, Stroke, and Refractory Ischaemia taken together by 2.3% (p < 0.001) NNT 43 Most benefit is achieved by 30 days with MI There is no effect on all cause mortality There is a large (relative 38%) significant excess of major bleeds

17 This is appalling Nuovo J, et al. JAMA 2002; 287: 2813 –4. Ann Intern Med, BMJ, JAMA, Lancet, NEJM 1989, 1992, 1995, 1998. Treatment RCTs 359 eligible articles. NNT reported in 8 (6 of these in 1998) ARR reported in 18 (10 of these in 1998). Put another way, 93% of all RCTs only report relative risk.

18 Odds ratios or relative risks? Macfarlane J et al. BMJ 2002; 13: 105-9 Patients who took antibiotics Patients who did not take antibiotics TOTAL Patients who were given a leaflet 4955104 Patients not given a leaflet 6338101 TOTAL11293205

19 Patients who took antibiotics Patients who did not take antibiotics TOTAL Patients who were given a leaflet 4955104 Patients not given a leaflet 6338101 TOTAL11293205 Relative risk: (49/104) / (63/101) = 0.76. i.e the relative risk of patients taking an antibiotic if they were given a leaflet is reduced by 24%. (Also called risk ratio)

20 Odds ratio: (49/55) / (63/38) = 0.54. There was a 46% reduction in the ratio of those taking antibiotics who had a leaflet compared with the ratio of those taking antibiotics who did not have a leaflet. Patients who took antibiotics Patients who did not take antibiotics TOTAL Patients who were given a leaflet 4955104 Patients not given a leaflet 6338101 TOTAL11293205

21 Absolute risk reduction: (63/101) – (49/104) = 0.15. Also known as the risk difference. i.e. the difference in the risk of taking antibiotics depending on whether a leaflet was used or not. Patients who took antibiotics Patients who did not take antibiotics TOTAL Patients who were given a leaflet 4955104 Patients not given a leaflet 6338101 TOTAL11293205

22 NNT: 1 / 0.15 = 7. i.e. 7 people need to be given a leaflet In order for 1 additional person not to take antibiotics Patients who took antibiotics Patients who did not take antibiotics TOTAL Patients who were given a leaflet 4955104 Patients not given a leaflet 6338101 TOTAL11293205

23 What you need to know for the MRCGP ARR, RRR, NNT (calculate) – plus OR Impact of baseline risks P values, Confidence Intervals (principles, not how to calculate) Power (principles) Diagnosis and screening –Sensitivity, specificity, PPV, NPV (LRs unlikely) Types of studies, including SR & MA, qualitative research (principles), health economics (principles) Where to find evidence (in the real world) How to use EBM in the consultation

24 matters

25 How does the size of the study affect things? Counsell CE, et al. BMJ 1994 309: 1677-1681. [Bandolier Nov 2002] Investigators used a dice to simulate outcomes in a trial Treatment arm vs. control arm Roll of a dice = outcome in the trial: 1-5 survival 6 = death Did for treatment group then repeated for control group Number of times the dice was rolled varied from 5 to 100.

26 But its the size that matters Results according to number of times the dice was rolled: Variation in outcome was largest in the smallest studies i.e the chance of a spurious result decreased with increasing numbers included in the trial More consistency in results Wide variation in results

27 How good is the evidence for the management of schizophrenia Thornley B, et al. BMJ 1998; 317: 1181-84 Size of trials (n=1941; 59 studies did not report study size)

28 Sub-group analyses – caveat emptor ISIS 2 trial: 17,187 patients, 417 hospitals up to 24 hours after MI. Randomised to either streptokinase, aspirin or placebo in 2x2 factorial design Streptokinase alone and aspirin alone each produced a highly significant reduction in 5-week vascular mortality: ARR 2,8%, together ARR vs double placebo 5.2%. To try and allay concerns re benefit:safety ratio in subgroups, the Lancet pushed for subgroup analyses. The authors agreed – but with the proviso that they should analyse by astrological star signs and that this should appear first in the table of subgroup results. The result? Gemini and Libra: aspirin of no benefit. All other star signs: aspirin strongly beneficial

29 What you need to know for the MRCGP ARR, RRR, NNT (calculate) – plus OR Impact of baseline risks P values, Confidence Intervals (principles, not how to calculate) Power (principles) Diagnosis and screening –Sensitivity, specificity, PPV, NPV (LRs unlikely) Types of studies, including SR & MA, and qualitative research (principles) Where to find evidence (in the real world) How to use EBM in the consultation

30 P< 0.05 The Shrine of Statistics The Sacred P-Value

31 Are you happy with 1 in 20?

32 Did he just say P = 0.05 ???? P = 0.027 means…… this result occurs BY CHANCE 1 time in 36; If P = 0.0001, 1 time in 10,000 by chance

33 Confidence intervals are the range of values between which we could be 95% certain that this result would lie if this intervention was applied to the general population

34 Confidence Intervals Estrogen Replacement Therapy in Women with a History of Proliferative Breast Disease

35 0.910.68 1.2 Risk could be this low Risk could be this high 95% C.I. Since the 95% CI crosses 1.0, the difference is not significant 1.0 Confidence intervals are the range of values between which we could be 95% certain that this result would lie if this intervention was applied to the general population

36 6 month data: Incidence rate per year Celecoxib NSAIDsNNT All patients: upper GI ulcer complications alone0.76%1.45% (P=0.09)- combined with symptomatic ulcers 2.08%3.54% (P=0.02)68 For patients not taking aspirin: upper GI ulcer complications alone 0.44%1.27% (P = 0.04) 121 combined with symptomatic ulcers 1.40% 2.91% (P= 0.02)66 For patients taking aspirin: upper GI ulcer complications alone 2.01%2.12% (P = 0.92) - combined with symptomatic ulcers 4.70%6.00% (P = 0.49). - Silverstein FE, et al. JAMA 2000; 284: 1247-1255

37 Kaplan-Meier estimates for ulcer complications according to traditional definition Jüni P, et al. BMJ 2002; 324: 1287-1288

38 What you need to know for the MRCGP ARR, RRR, NNT (calculate) – plus OR Impact of baseline risks P values, Confidence Intervals (principles, not how to calculate) Power (principles) Diagnosis and screening –Sensitivity, specificity, PPV, NPV (LRs unlikely) Types of studies, including SR & MA, qualitative research (principles), health economics (principles) Where to find evidence (in the real world) How to use EBM in the consultation

39 0 1010 2020 3030 No Disease A B Percent of population VALUE Arbitrary Units Set cut off at AA lot of people who do not have the disease are labeled as having it (false positives) Set cut off at B A lot of people who do have the disease are labeled as not having it (false negatives) Disease C

40 How many people in the study? Positive Negative Test Disease PresentAbsent 100

41 How many had the disease? Positive Negative Test Disease PresentAbsent 5010050

42 How many with the disease had a positive test? How many without the disease had a negative test? Positive Negative Test Disease 5 45 5 PresentAbsent 50 100 50

43 What was the prevalence of disease in those tested? Positive Negative Test Disease 5 45 5 PresentAbsent Prevalence = 50/100 = 50% 50 100 50

44 So …………Sensitivity and specificity Positive Negative Test Disease 5 45 5 PresentAbsent 5/50 false negatives; i.e. sensitivity = 45/50 =90% 5/50 false positives; i.e. specificity = 45/50 = 90% 50 100 50

45 Positive Predictive Value and Negative Predictive Value Positive Negative Test Disease 5 45 5 PresentAbsent PPV 45/50 = 90% NPV 45/50 = 90% 50 100 50

46 Watch what happens when the prevalence drops to 10% NB. PLEASE remember this bit!!!!!!! Positive Negative Test Disease 1 9 81 9 PresentAbsent Prevalence = 10/100 = 10% 1/10 false negatives; i.e. sensitivity = 9/10 =90% 9/90 false positives; i.e. specificity = 81/90 = 90% PPV 9/18 = 50% NPV 81/82 = 99% 1090100 18 82

47 H Pylori infection in a population with a 25% prevalence MeReC Bulletin 2001; 12 (1): 1-4 646945475.586 Near- patient serologica l tests 32597759091 Laborator y serologica l tests 112998895.597.5 Breath test ( 14 C) 11199899696.5 Breath test ( 13 C) False Negative results (%) False positive results (%) Negative predictive value (%) Positive predictive value (%) Specificity (%) Sensitivity (%)

48 So what does all this mean? In primary care many people have a low chance of having the disease they are being tested for. If they get a positive test then they may have the disease – or it could be a false positive. They may need more tests to sort out whether they truly, truly have the disease. (But what will the patient think when you tell them their initial test indicates they may have something and they need further tests?) If they get a negative test, and they are unlikely to have the disease, then its really very unlikely that they have it when they have tested negative. And MOST IMPORTANTLY, try only to test people for anything if they are in a high risk group for having the disease. Testing lots of people will do more harm than good.

49 What you need to know for the MRCGP Sums ARR, RRR, NNT (calculate) – plus OR Impact of baseline risks P values, Confidence Intervals (principles, not how to calculate) Power (principles) Diagnosis and screening –Sensitivity, specificity, PPV, NPV (LRs unlikely) Concepts Types of studies, including SR & MA, qualitative research (principles), health economics (principles) Where to find evidence (in the real world) How to use EBM in the consultation

50 My brain hurts Mr T F Gumby

51 Take a break

52 What you need to know for the MRCGP ARR, RRR, NNT (calculate) – plus OR Impact of baseline risks P values, Confidence Intervals (principles, not how to calculate) Power (principles) Diagnosis and screening –Sensitivity, specificity, PPV, NPV (LRs unlikely) Types of studies, including SR & MA, qualitative research (principles), health economics (principles) Where to find evidence (in the real world) How to use EBM in the consultation

53 Three Steps to (primary care consultation) Heaven Now there are Three Steps To Heaven Just listen and you will plainly see And as life travels on And things do go wrong Just follow steps one, two and three

54 Acknowledgements Prof Allen Shaughnessay, Tufts University, Boston Prof Dave Slawson, University of Virginia, Charlottesville Prof Trisha Greenhalgh, University College, London

55 Step 1 Make the best attempt at an initial (often provisional) diagnosis – open questions – clarify ideas, concerns and expectations (Can work in biomedical, anticipatory and hermeneutic relationships with patients) – examine appropriately – exclude red flags – Beware of the limits of pattern recognition; use Bayesian approaches where possible Step one - you find a girl to love

56 Development of Medical Expertise Psychology of Medical Decision Making: Vanderbilt University. Stage 1. Elaborated causal networks –Learned during the basic science years –Facts and relationships –Nodes and links –Causal models of disease processes

57 Target condition Anatomy, physiology, biochemistry Epidemiology, risk factors Pathophysiology Clinical features -Forceful features -relevant cues Clinical management options -Acute - chronic Consultation skills

58 Stage 2. Compilation of abridged networks –Starts when exposed to real patients –Knowledge gets compiled (rewritten, automated) simplified causal models explain signs and symptoms associated with diagnostic labels

59 Stage 3. Illness Scripts. –Based on repeated experience with patients –Illness scripts are sufficient to diagnose and treat diseases. Lists of features that characterize the disease Specification of what to do Information about context Information about temporal features of disease

60 Stage 4. Cases. Patient encounters stored as instance scripts. –Based on long experience –Physician remembers many individual patients –Each has a different variant of the disease –New (or newly sick) patients are recognized as similar to Patient X treated as Patient X was treated

61 Making a diagnosis Elstein AS and Schwarz A. BMJ 2002; 324: 729–732 Problem solving –Testing hypotheses (novices), (complex) Failure to generate correct hypothesis data collected thoroughly but ignore / misunderstand / misinterpret too economical, but interpret accurately what is available –Pattern recognition (experts), (familiar) Mental models –Neither are proof against error (see Klein, BMJ 2005; 330: 781– 784) Decision making –Pre test probability, LR, post test probability (see Gill CJ, et al. BMJ 2005; 330: 1080–1083.) –A minority sport (consciously)

62

63 Learn the basic patterns

64 Then can see them in new situations.

65

66 Dwivedi G, et al. BMJ 2006; 332: 406 Mrs Patel (28 years, 7 month old baby) presents as an emergency with a severe episode of R chest pain. Pain started 2 months ago, well localised over the R chest wall with no radiation. Precipitated by exertion, particularly by pushing daughter in pushchair, limits recreational activities with child Pain not exacerbated by deep inspiration, coughing or twisting; no dyspnoea, palpitations or dizziness. Previous consultation = musculoskeletal pain. PMH: Type 2 diabetes 3 years, well controlled on oral medication (HbA1c 7.5%); hypothryoidism, l-thyroxine 25micrograms daily Never smoked, no history of hypertension, lipid status unknown

67 On examination –Obese (BMI 34.6) –P 70/min, regular, BP 130/70 –All peripheral pulses palpable, no bruits –Heart sounds are normal, no murmurs –Chest clear –No breast lumps or tenderness –Abdo examination normal –Resting ECG = sinus rhythm, poor R wave progression in anterior chest leads, inverted and biphasic T waves in V2 and V3.

68 Whats the diagnosis? –Musculoskeletal? –Dyspepsia? –Pulmonary embolism? –Cholecystitis? –Ischaemic heart disease? –Something else?

69 Mr Patel, 55 years, presents as an emergency with a severe episode of R chest pain. Pain started 2 months ago, well localised over the R chest wall with no radiation. Precipitated by exertion, particularly by pushing granddaughter in pushchair, limits recreational activities with child Pain not exacerbated by deep inspiration, coughing or twisting; no dyspnoea, palpitations or dizziness. Previous GP consultation = musculoskeletal pain. PMH: Diabetes 3 years, well controlled on oral medication (HbA1c 7.5%); hypothryoidism, l-thyroxine 25micrograms daily Never smoked, no history of hypertension, lipid status unknown

70 Mrs Patel Seen and admitted by cardiologist HB 13.1, ESR 20 (slightly raised), Biochemistry normal D-Dimer normal (<250ng/ml) TC 6, LDL 3.13, TG 3.47 TFTs: free T4 <5pmol/l, TSH 151mIU/l Treadmill: 3min51sec Bruce protocol – chest discomfort, maximum HR 155, no ECG changes indicative of ischaemia Echo after exercise test = large area of reduced systolic wall thickening affecting anterior septum and apex with obvious LV dilatation Angiogram = critical lesion proximal LAD artery. Angioplasty + DE stent Aspirin, clopidogrel and atorvastatin started, thyroxine dose doubled 6 months plus later is doing well

71 Has my neighbor won the lottery? Pre test probability – extremely low Test 1 – my neighbour says hes won – still low likelihood (hes a joker) LR+ Test 2 – (he says hes won and) he disappears on holiday for three weeks and comes back with QE2 stickers on his luggage– slightly more likely LR+ Test 3 – (he says hes won, hes had an expensive holiday and) he pulls up in a gleaming new Bentley Continental – more and more likely LR+ Test 4 – (he says hes won, has had an expensive holiday and bought an expensive car and) he is no longer my neighbor, having bought a very big house in the country LR+ Test 5 – (the police arrest him and charge him with a £50million gold bullion robbery) LR-

72 How many times more (or less) likely are you to have an MI if you develop chest pain and have these signs / symptoms? Chest pain sharp or stabbing Chest pain radiates to left arm Chest pain most important symptom Nausea or vomiting Perspiring BP < 80mmHg Chest pain reproduced by palpation e.g. 1 = no change in likelihood; 2 = twice as likely; 10 = ten times more likely; 0.5 = half as likely; 0.1 = ten time less likely etc.

73 Youve just estimated likelihood ratios The likelihood ratio is the number of times more or less likely it is that someone with that symptom has the disease The greater the LR, the more likely the person is to have the disease –A +LR more than 10 tends to rule in disease (but beware if the disease is very unlikely) The smaller the LR, the less likely person is to have the disease –A –LR less than 0.1 tends to rule out disease (but beware if the disease is very likely)

74 Positive Negative Test Disease 5 45 5 PresentAbsent 50 100 50 Positive Likelihood ratio: 45/50 (= 0.90) divided by 5/50 (= 0.10). LR+ = 9 Likelihood ratios express how many more times (or less times) a test result is to be found in diseased people compared with non-diseased people.

75 Positive Negative Test Disease 5 45 5 PresentAbsent 50 100 50 Negative likelihood ratio: 5/50 (= 0.10) divided by 45/50 (= 0.90). LR- = 0.11

76 You use LRs without knowing it… Gill CJ, et al. BMJ 2005; 330: 1080-1083 Most symptoms have +LRs around 2 and –LRs around 0.5 On their own they dont add much The clever bit is that you can link them together…

77 Is This Patient Having a Myocardial Infarction? Panju AA, et al. JAMA 1998; 280: 1256-1263

78

79

80 Step 1 Make the best attempt at an initial (often provisional) diagnosis – open questions – clarify ideas, concerns and expectations – examine appropriately – exclude red flags – Beware of the limits of pattern recognition; use Bayesian approaches where possible Step one - you find a girl to love

81 You have just moved to a new town and are looking for a GP practice for your own care and that of your new family. Two doctors are accepting patients:- Jane completed her VTS six months ago. She gained the highest marks out of the 2000 candidates when she sat her MRCGP exam and won the Fraser Rose medal. Susan completed her postgraduate training 10 years ago. She passed her MRCGP exam at the end of her VT with distinction. Who do you choose as your doctor?

82 Systematic review: the relationship between clinical experience and quality of health care Choudhry NK, et al. Annals of Internal Medicine 2005; 142: 260–273 32 of the 62 evaluations (52%) reported decreased performance with increasing years in practice for all outcomes assessed. 13 (21%) reported decreasing performance with increasing experience for some outcomes but no association with other assessed. 1 (2%) reported increasing performance with increasing experience for all outcomes assessed.

83 Step 2 Be aware of the evidence base in relation to the possible management options – balance efficacy, safety, cost, and patient factors – never always, and never never – get used to not knowing everything and have a plan for finding the best answer – check regularly on the evidence for the management of the conditions seen often – be prepared to look up the evidence for the management of conditions not seen often Step two – she falls in love with you

84 EFFECTIVESAFE COST PATIENT FACTORS BenificenceNon-malfeasance Justice Patient autonomy Barber N. BMJ 1995; 310: 923-925

85 QUALITY OF CARE SYSTEMATIC APPLICATION OF POLICY Low High 0% 100%

86 Chief Medical Officer Annual Report 2005 The way in which clinical decisions are made, the extent to which they depart from research evidence, and the extent to which they depart from research evidence, and the factors that determine compliance with best practice have….been extensively studied…. Despite this, the solution to the problem of clinical practice variation has not been found…..

87

88 Financial incentives Guidelines Data feedback Academic detailing Educational Decision support (IT) Audit Opinion leaders Media Patient – mediated

89 Benefits of implementation strategies Grimshaw J, et al. Journal of Continuing Education in the health professions 2004; 24: S31-S37 Changes in practitioner behaviour; in the desired direction, were reported in 86% of the comparisons made. The median effect size overall was approximately 10% improvement in absolute terms.

90 "We surveyed one acute medical take in our hospital. In a relatively quiet take, we saw 18 patients with a total of 44 diagnoses. The guidelines that the on call physician should have read remembered and applied correctly for those conditions came to 3679 pages. This number included only NICE, the Royal Colleges and major societies from the last 3 years. If it takes 2 min to read each page, the physician on call will have to spend 122h reading to keep abreast of the guidelines" (for one 24h on-call period). Allen D, Harkins KJ. Lancet 2005; 365: 1768

91 There are 1500 pages indexed in Medline each day. Prof Sir JA Muir Gray Best Current Evidence Strategy. March 2006 (So which ones will you choose to read?)

92 Abstracts lie (lots) Pitkin RM, et al JAMA 1999; 281: 1110-1111 Random samples from 44 articles and their abstracts from Annals, BMJ, Lancet, JAMA, NEJM (12 months from July 1996), and 44 articles CMAJ (15months from July 1996) were compared with the original articles 19% of abstracts contained statements that were inconsistent with the full article 11% of abstracts contained statements that were not present in the full article

93 20% of RCTs dont report all outcomes Chan A-W, Altman DG. BMJ 2005; 330: 753-756 519 RCTs in 553 publications were examined for incompletely reported outcomes per trial. Original authors were surveyed (response rate 69%). 32% denied the existence of unreported outcomes when there was evidence to the contrary in their publications. On average, 20% of outcomes measured in RCTs were incompletely reported. These outcomes were more likely to be non significant (OR 2.0, 95% CI 1.6 to 2.7) for efficacy outcomes; OR 1.9, 95% CI 1.1 to 3.5 for harm outcomes.

94

95 So you cant keep up to date by reading primary research in journals. And if you do read some primary research in journals, theres a decent chance what you read is poorly designed, poorly conducted and poorly reported. And youve only read that research – not all of the research which may be relevant. What if youve just read the one aberrant study and there are lots of others showing results that are the exact opposite – and you haven't seen those?

96 Clinician reading journals Evidence- based treatment for my patient

97 Evidence based medicine 1. Formulate question 2. Efficiently track down best available evidence 3. Critically review the validity and usefulness of the evidence 4. Implement changes in clinical practice 5. Evaluate performance Recognise lack of certainty

98 If you ask doctors, they say they need information about once a week. But if you debrief them, they raise about 2 questions for every three patients Covell DG et al. Ann Intern Med 1985 103: 596–599 Many potential questions are not recognised by general practitioners (over confidence?, failure to recognise uncertainty?) Barrie AR et al. BMJ 1997; 315: 1512–1515 Answers to most questions are not immediately pursued. Ely JW et al. BMJ 1999; 31: 358-361 Doctors spent an average of less than 2 minutes pursuing an answer, and they used readily available print and human resources. Only two questions (out of over 1100) led to a formal literature search.

99 Travel agents consultation

100

101 Pattern recognition, mental maps and short cuts are often used FOR DIAGNOSTIC DECISION MAKING Management decision making –Is the same problem (lots of complex information to compute) –But creating mental maps to make sense of this is NOT the preferred approach when there are better ways to keep up to date with the evidence for interventions

102 Mindlines Gabbay and le May. BMJ 2004; 329: 1013–1016 Clinicians rarely accessed, appraised, and used explicit evidence directly from research or other formal sources; rare exceptions were where they might consult such sources after dealing with a case that had particularly challenged them.

103 Mindlines Gabbay and le May. BMJ 2004; 329: 1013–1016 Not once was a guideline read. Expert computer systems were rarely used (never in real time). Instead, they relied on what we have called "mindlines," collectively reinforced, internalised tacit guidelines, which were informed by brief reading, but mainly by their interactions with each other and with opinion leaders, patients, and pharmaceutical representatives and by other sources of largely tacit knowledge that built on their early training and their own and their colleagues' experience. The clinicians, in general, would refine their mindlines by acquiring tacit knowledge from trusted sources, mainly their colleagues, in ways that were mediated by the organisational features of the practice, such as the nature and frequency of meetings, the practice ethos, and its financial and structural features, including the computer system.

104 Reading and critical appraisal can (largely) be replaced by using brief summaries of evidence from trusted sources

105 Finding the best answer, first time Cochrane Library NICE, (NSFs) EBM DTB MeReC Bandolier Ivy League journals Clinical Evidence InfoPOEMs, Prodigy BestTreatments NPC ref sheets Textbooks Usefulness Medline, Google scholar InfoRetriever, DrCompanion, self-assembly

106 Post-modern EBM skills 1. Recognise the need to check mindlines 2. Find the summarised evidence 3. Understand the language of the evidence 4. Translate that into terms your patient can understand 5. Enable your patient to play a full part in shared decision making (if they want to)

107 Individuals need…… Basic clin epi (how to read a summary) –Using Evidence to Guide Practice Information Mastery induction –Where to find summaries from trusted sources Point of Care Information Tools –DrCompanion, InfoRetriever Consultation translation skills Local trusted colleague(s) – networked CPD programme for COMMON conditions, unbiased, evidence-based, summarised, reinforced Programme to spread the skills required to hunt for answers efficiently using the above tools

108 Step 3 Reach agreement on a management plan present options where data is available, use x in 100 will benefit, y in 100 will be harmed (not percentages) consider supporting with visual representation e.g. smiley faces, CV risk charts if an immediate decision is not required, consider materials for patient self-study safety netting housekeeping Step three – you kiss and hold her tightly

109 Clopidogrel in ACS Its really beneficial; Id want all of my patients to be taking it Most people do fine just on aspirin. Adding clopidogrel prevents only a few people having an event and theres the increased bleeding risk The specialist starts it and I dont question that

110 N Engl J Med 2001; 345: 494-502

111 Absolute risk Relative risk Numbers needed to treat P values Confidence Interval

112

113

114 Clopidogrel in ACS Its really beneficial; Id want all of my patients to be taking it. Most people do fine just on aspirin. Adding clopidogrel prevents only a few people having an event and theres the increased bleeding risk. The specialist starts it and I dont question that.

115 What would happen to 100 people like you who take sleeping tablets for more than a week. These SEVEN people sleep better, which means they get an extra 25 minutes sleep a night They also wake up once less every 2 nights For 76 people the tablets do NOTHING, good or bad These SEVENTEEN people have side effects One of them may be serious, like a fall or car crash

116 Antibiotics for bronchitis Little P, et al. JAMA 2005; 293: 3029–3035

117 Level 3-4 diagnostic skills Can work in biomedical, anticipatory and hermeneutic relationships with patients (ICE) Finds and understands the strength and the language of summaries of evidence from trusted sources Aware of diagnostic cognitive biases; uses baseline probability and decision rules routinely Translates that evidence for patients, uses DAs appropriately, discusses OICJ routinely The New Generalist Pinwheel

118 Back to hard core exam prep

119 Web based resources An Introduction to Information Masterywww.poems.msu.edu/InfoMastery National Prescribing Centre www.npc.nhs.uk Drug And Therapeutics Bulletin www.which.net/health/dtb/main.html Bandolierwww.ebandolier.com Clinical Evidencewww.clinicalevidence.org Effective Health Care Bulletinwww.york.ac.uk/inst/crd Cochrane Collaborationwww.cochrane.co.uk National Institute for Clinical Excellencewww.nice.org.uk United Kingdom Medicines Informationwww.ukmi.nhs.uk Centre for Evidence Based Medicinewww.cebm.net InfoPOEMSwww.infopoems.com CASPwww.phru.nhs.uk/casp/casp.htm National electronic library for healthwww.nelh.nhs.uk

120 How to read a paper Trisha Greenhalgh BMJ 1997; 315 : 180 – 3 : 243 – 6 : 305 – 8 : 364 – 6 : 422 – 5 : 480 – 3 : 540 – 3 : 596 – 9 : 672 – 5 : 740 – 3 www.bmj.com Getting research findings into practice Various authors BMJ 1998; 317 : 72 – 5 : 139 – 42 : 200 – 3 : 273 – 6 : 339 – 42 : 405 – 9 : 465 – 8 www.bmj.com NB Health economics

121 Books Greenhalgh T How to read a paper 2nd edn London: BMJ Books 2001 Sackett DL, Richardson WS, Rosenberg WMC, Haynes RB. Evidence-based medicine: how to practice and teach EBM. London: Churchill- Livingstone, 1996. Muir Gray JA. Evidence-based Healthcare. Churchill Livingstone. 1997.

122

123 Are you suffering from information overload? There are over 30,000 biomedical journals in print. How do you cope? Using Evidence to Guide Practice 5 interactive modules available via the Internet covering the principles of EBM and the influences on prescribing E-Learning Learn at a pace and time to suit you ALL FIVE MODULES FOR ONLY £35 Go to www.npc.co.uk for more details

Similar presentations