Centre for Evidence-Based Medicine Jeremy Howick Centre for Evidence-Based Medicine Thank you to the organizers Jon, Federica, and Phyllis for doing such a wonderful job organizing the conference.

1. There are no hierarchies 2. EBM has always recognized mechanisms 3. Philosophers of science have ignored problems with the evidential role of mechanisms 4. The fact that ‘mechanistically sound’ hypotheses are more believable is irrelevant. 5. The best way to improve evidence would be to have a minimum effect size requirement Specifically I will make 5 mildly related points.

1. There are no hierarchies: GRADE Step 1: Initial ranking Step 2: upgrade/downgrade Step 3: assign final grade Downgrade for: - Inconsistency - Indirectness - Imprecision - Publication bias HIGH MODERATE LOW VERY LOW RT: High OS: low Perhaps the most highly regarded system, for ranking evidence (GRADE) is not a hierarchy, but a system that carefully studies and eventually ranks evidence according to thoughtful series of criteria. Upgrade for: - Large consistent effect - Dose response - Confounders only reducing size of effect

1. There are no hierarchies: OCEBM PICK A BOX! Cohort Study randomized trial Case Series systematic review Mechanistic reasoning The evidence ranking system I helped design is intended to be used by busy clinicians, and we explicitly ask users to think of it as a heuristic. If you had 7 minutes in the clinic, and you were faced with a question about the therapeutic benefits of an intervention, where would you look? (Where is likely to provide you with the best evidence?) It is true, of course, that people teach the EBM view as a hierarchy, but that is an interpretation of EBM, and if the oversimplified interpretation of EBM is the target, then you are making a sociological point and this needs to be made clear.

2. EBM has always recognized a role for mechanisms The second point is that EBM has always recognized a role for knowledge from mechanisms. Here is a quote from the paper that introduced the movement to the wider academic community.

3. Proponents of mechanisms have ignore problems with mechanisms This is a point related to the one Mike made earlier. Medicine probably did harm for thousands of years because it relied on mechanistic evidence. This has not been addressed by proponents of mechanisms as evidence. Bloodletting is the most obvious example, and you need trepanning like you need a hole in the head ;). My third point is that proponents of evidence from mechanisms have failed to recognize the historical and current problems with using apparent knowledge of mechanisms as evidence. Given that the claims in support of mechanistic evidence are supposed to improve practice this is an example where philosophers’ empirical claims lack empirical support. There are several other problems with reasoning from mechanisms I list elsewhere including complexity. Sandra pointed out yet another one I had not thought of yesterday. By contrast, EBM proponents are vociferous critics of badly conducted randomized trials and systematic reviews.

4. Getting the contrast class right An argument even I used to believe counted as support of mechanistic evidence is that hypotheses tested in randomized trials that are based on sound mechanisms are more likely to be true than randomly generated hypotheses. The problem here is that randomly generated hypotheses is not the relevant contrast class. It is a straw man to compare these two methods for generating hypotheses. The relevant contrast class is hypotheses that are generated by empirical observation. A classical example is chicken soup. Chicken soup has been used for generations to cure the flu. There is a wealth of observational evidence supporting the benefits of chicken soup yet there is no randomized trial. You might say empirically generated hypotheses can be tested alongside ‘mechanistically generated’ hypotheses. But there are three good reasons to begin with the former: Cost Save animals They lead to improved practice whether we get a positive or negative answer. Even cavemen knew apples fell from trees before they actually did fall from trees.

4. Effect size This is an area of research I’m interested in exploring that has the potential to (a) reduce what is often referred to as ‘industry bias’ (because it is more difficult to obtain a large effect than a small one), and may have other benefits in terms of promoting more innovation as well. It is also an area where philosophers of science are in a position to make a real difference but have ignored. It is interesting from a statistical point of view, from an epistemological point of view, and it has the potential to improve practice immediately by reducing industry bias.

1. There are no hierarchies 2. EBM has always recognized mechanisms 3. Philosophers of science have ignored problems with the evidential role of mechanisms 4. The fact that ‘mechanistically sound’ hypotheses are more believable is irrelevant. 5. The best way to improve evidence would be to have a minimum effect size requirement Specifically I will make 5 mildly related points.

Thanks again! jeremy.howick@phc.ox.ac.uk