Presentation on theme: "Presentation at Radstats British Library, London 24 February 2012"— Presentation transcript:
1 Presentation at Radstats British Library, London 24 February 2012 How to smell a rat: Examples of scientific and regulatory failure in medicineDr Aubrey BlumsohnPresentation at Radstats British Library, London 24 February 2012
2 Nature of science – and how we define scientific fraud
7 HistoryEinsteinInsanity: doing the same thing over and over again and expecting different results.
8 What is Science? UK Science Council (2009) Spent a year working out a new definition of science.Claim as the first "official definition of science" ever published.“Science is the pursuit of knowledge and understanding of the natural and social world following a systematic methodology based on evidence”
9 What is Science? – Mertonian Norms CommunalismEveryone can see what’s going onUniversalismDoing of it not restricted to special peopleDisinterestednessAn arms length attitude.Skepticism (Organized Skepticism) -Claims exposed to critical scrutiny. Awkward questions. Merton, Robert K. (1973), "The Normative Structure of Science",
10 Science demands awkward people It takes all sorts of people
11 Example 1: “Mr Nobody” and Journal of the American Medical Association (JAMA)
21 The ClassicsWilliam Summerlin (Sloan Kettering, 1984) - faked transplantation experiments in white mice by blackening patches of their skin with a penVijay Soman (Yale, 1980) a diabetologist - wrote 12 papers based on invented data. His co-author Felig had to resign
22 The ClassicsRobert Slutsky (Radiologist, University of California). Published 137 papers between 1978 and one every 20 days. Many faked.John Darsee (Cardiologist, Harvard, 1981). Observed falsifying data. His boss, Eugene Braunwald ignored the problem. Later investigations discovered more than a 100 fraudulent papers
23 In these odd cases, the stories, as told, usually minimise the involvement of senior colleagues, University administrators, Hospitals and regulatory bodies who hid the misconduct before it was eventually exposed
24 The impressionThe choice of cases implicitly defines what we mean by scientific fraud/misconductFraud is rareIt involves a particular type of activity (painting a mouse)Usually involves an aberrant (male) acting alone (or duping otherwise honourable collaborators) “Problems of scientific misconduct are rare - the product of psychopathic behaviour’ originating in temporarily deranged minds” President of the National Academy of Sciences (at Congressional Hearings)
25 And the solution Is to create yet more procedures and guidance And for Universities to create robust, secret, legalistic procedures to investigate these rare rogues
26 Only one problem. As Arnold Schwarzenegger once said: “it's bullshit—all of it”
27 The realityNot rareScientific misconduct often not an individual activity – but a group activity (or involves group collusion)
28 Group-think Hannah Arendt (1965) "rule by nobody".No single individual feels responsible for what is ultimately done.Indeed many people have gone to great lengths to make sure that this exact state of affairs achievedLegislatorsEthics CommitteesTrade bodiesRegulatorsSystem designers responsible for fraud?
29 The reality The offender is often not a lone white male psychopath Corporate fraud
30 The realityCorporation acting at the University interface
33 The realityor a regulator ….or government itself
34 The mechanisms also differ The mechanism of misconduct is also usually not direct fabrication of findings but something far more nuanced
35 Defining pornography"hard-core pornography" was hard to define, but "I know it when I see it“ Potter Stewart (Associate Justice of the US Supreme Court)
36 MisconductMisconduct is nothing to do with the correctness of conclusionTo err is human, and scientific!!!It has to do with behavioursBehaviours which could disturb the scientific recordNot binaryWe need to get away from ludicrous discussion about restrictive definitions and start discussing behavioursOpenly
37 Example 2: The fearful junior Approached by a Junior DoctorIn order to gain her pathology registration she had to submit her dissertationShe could not write up her dissertation or prepare the main area of work for publication because her supervisor had made this impossible – for years…..She could only present the secondary aspectsThe work involved looking at physiological changes in healthy volunteers randomised to atorvastatin or placeboThe study showed that atorvastatin induced a diabetes like stateHer supervisor was funded by the manufacturer of atorvastatin
38 Example 3: Does Seroquel cause diabetes and weight gain
39 But there was a cloud on the horizon. The F. D. A But there was a cloud on the horizon. The F.D.A. was beginning to look into the issue of Diabetes in patients on Atypical Antipsychotics and wrote to Astrazeneca
40 Then somebody noticed the obvious that they had more than just 18 months of data in these subjects
46 What are data? Data is not the same as representation of that data
47 Getting a hamburger when what you need is the original cow
48 Example 5: Vioxx APPROVE Trial New York Times May 31, 2006
49 Look at the languageMerck & Co. acknowledged that it misidentified a statistical methodMerck has contended that the study showsThe company said in a statement yesterdayMerck has insisted that its 18-month argument holds upwhile outside scientists have cast doubt on the company's interpretation.Merck research chief Peter Kim said the company stood by its analysis
51 Example 6: Vioxx AVANTAGE Trial On Oct. 21, 1999 a 73-year-old female participant dies in a clinical trialThe trial was Merck’s Advantage trial of VioxxShe died a few minutes after calling her son to tell him she felt short of breath
52 The Advantage trial was a very short (12-week) clinical trial eight people taking Vioxx suffered heart attacks or sudden cardiac death in the trial (compared with only one not on the drug)The difference was statistically significant, but Merck never showed the data that way
53 In the published study, the “author” (Dr In the published study, the “author” (Dr. Lisse, University of Arizona), the 8 deaths became` 5, a difference that did not reach statistical significance.The paper never mentioned the other cardiac deaths of patients taking Vioxx, including the 73-year-old woman.
55 Dr. Lisse said that while he was listed as the paper's first author, Merck actually wrote the report. "Merck designed the trial, paid for the trial, ran the trial," Dr. Lisse said. "Merck came to me after the study was completed and said, 'We want your help to work on the paper.' The initial paper was written at Merck, and then it was sent to me for editing."
56 Results from the trials were also supposed to be blinded when coded During coding, Dr. Eliav Barr, a Merck employee judged that the woman “had probably died of a heart attack."
57 Dr. Reicin a Merck executive immediately shot back : "I think this should be called an unknown cause of death." A few hours later, she wrote, "I would prefer unknown cause of death so we don't raise concerns."Read
58 In November 2001, an F.D.A. reviewer who had examined all the company's data concluded in a report to other agency officials that Merck had cheated. But the F.D.A. never told anyone either.Read
59 Dr. Lisse, the study "author" said he had never heard of the case of the woman who died, until told of it by the New York Times.Read
60 Example 7: Vioxx - Fake Journals Merck paid Elsevier to create a number of fake scientific journalsThis came to light 2009 in the context of a civil suit filed by Graeme Peterson, who suffered a heart attack in 2003 while on VioxxThese were designed in such a way that the average medical reader would “mistake the publication for a genuine peer reviewed medical journal”The articles supported Merck Drugs
63 One of the members of Australasian Journal of Bone and Joint Medicine's "Honorary Editorial Board," Peter Brooks, a rheumatologist said: he didn't recall who asked him to serve on the board, …. "You get involved in a whole bunch of things at this level".
64 Elsevier issued at Press release and an apology We “published a series of sponsored article compilation publications, on behalf of pharmaceutical clients, that were made to look like journals and lacked the proper disclosures. This was an unacceptable practice, and we regret that it took place.”But what of the academics involved, and their institutions?
65 REGULATORS AND SYSTEMS Medical Writers (Ghost writers)Medical JournalsMHRA (UK Drug regulator)Medical Ethics systemUK Research Integrity OfficeGeneral Medical Council
66 The illusion of due diligence "We're looking for the illusion of due diligence, Mr. Pope. Two criminal acts successfully prosecuted -- it gives us that illusion."
69 “When the time comes for publication, the medical writer writes the publication in consultation with the named author(s). This usually begins with discussions between the medical writer and authors(s) about the main features of the paper, after which the medical writer writes the first draft. The draft is then revised (sometimes many times) after review by the author(s). Thus although the authors do not actually write the paper, they do have control over its content, which is far more important. Naturally, they have access to the report of the study and its data tables, so they can verify that the paper is accurate. They often don't have access to the raw data, but it is worth repeating that raw data are of very little use when writing a paper. Results of a study need summarising and analysing before they mean anything, and it is those summaries and analyses that are important for proper interpretation of a study, not the raw data themselves. “Adam Jacobs – European Medical Writers Association BMJ 21 Nov 2004
71 Motto – Which? MagazineNo advertising, no bias, no hidden agenda. Just expert advice from an independent source
72 Motto – BMJ“We protect our reputation by careful balance of editorial and advertising, which means your messages will always stand out.” (http://group.bmj.com/group/advertising)
73 The Medicines and Healthcare Regulatory Agency (MHRA)
74 MHRA Has absolutely nothing to say on the problem of ghostwriting Has nothing to say on the problem of denial of access to data (to authors or the scientific community in general)Have no apparent concerns nor any apparent remit to deal with events at Sheffield UniversityEven suggested that it might be “illegal” for authors to obtain the randomisation codes in a study they are authoring because commercial companies “own” those codes.
75 RisedronateFOI:Did the MHRA receive and scrutinise in any way the raw data relating to the three key clinical trials which led to approval of RisedronateHas the MHRA got the randomisation codes
76 Risedronate FOI response: We cannot tell you because part of the response is commercially confidentialReply: The questions required answers of “yes” or “no”. Which parts of the Yes or No are commercially confidentialEventual FOI response: The MHRA did not scrutinise or retain any actual raw data before licensing Risedronate. They simply accepted what the company told them.
78 The MHRA (HOC SC Report 2005) "practices have developed which act against the public interest."The MHRA relies on company data presented as reports, in its decision whether or not to licence a drug. Raw data is very rarely analysed.
79 The MHRA (HOC SC Report 2005) “The MHRA Chairman suggested that trust underpinned the stance of the MHRA towards the companies they regulate. We inferred that this extended to the routine acceptance of companies’ summaries of the results of tests on their drugs as true reflections of the raw data on which they were based.”“Evidence that regulatory agencies are acquiescing in biased later publication of the information they hold.”
80 The MHRA (HOC SC Report 2005) “Lack of transparency has played a major part in allowing failings to continue.”“100% of the funding for the MHRA comes from the industry it regulates”"In view of the failings of the MHRA, we recommend a fundamental review of the organisation."
81 The General Medical Council Body that “regulates” doctorsA science-based professionMight have thought they would take scientific integrity seriouslyLong history of a very lax approach to scientific fraudParticularly when it involves very senior academics
82 Dr David Hurlstone Sheffield Teaching Hospitals NHS Trust Bowel cancer research2006 BUPA Foundation Research Award2007 Medical Futures Innovation Oncology & Cancer Care AwardGMC - dishonesty in three research studies published in 2007 & 2008 (not retracted)Suspended from Medical Register for 3 monthsOn return to work retained Clinical Excellence Award
83 What did he do? In three publications And so on….. Submitted authors’ consent forms to journal with false signatures of collaboratorsStudied patients without ethical approvalCreated consent forms for patient records in retrospectStated falsely that patients had been randomisedMade false statements about microscopy having been carried outStated falsely that two blinded specialists had exampled samples – they did not existStated falsely that an independent nurse participated in the study and that the study endoscopist was blindedAnd so on…..
84 GMC and Richard Eastell 3 publications intended by Procter and Gamble ( )Authors actively denied randomisation codes by sponsorEastell prepared to go along regardlessPrepared to induce a colleague to do the sameFalse conclusions in all manuscriptsResearch Dean of a Medical School R&D Director of Sheffield NHS Trust
86 How to get out of this?Didn’t write the false statement The “medical writer” wrote that as wellI don’t need raw data as a scientist because I would not know how to analyse it (and presumably neither would my coauthors). It is very hard to analyse data.The statement was in 2002.In 2002 I had a watershed moment when I realised this kind of activity was inappropriate.I would never have done it again after 2002, and I will never do this sort of thing again.Legal manoeuvres to avoid having the GMC panel examine any information subsequent to this 2002 paper and repeated events
87 The GMC response 2002 statement Epiphany in 2002 2003/2004/2005??? Data are raw dataData are having the dataIt was falseIt was misleadingIt was negligentEpiphany in 20022003/2004/2005???No punishment - “it would interfere with his ability to get research funding”!This type of behaviour has no impact on patient care (according to the GMC)
88 Dr Guirong Jiang Radiologist Working as a postdoctoral fellow at the University of SheffieldOne of the most experienced radiologists in the world in terms of diagnosis and definition of vertebral fractureSeveral complaints about Richard EastellIncluding his wish to author in the absence of dataApparent secret contracts involving Eastell, Sheffield University and Commercial Companies preventing publicationJiang not permitted to see contracts apparently preventing her right to publish findings she believed to be true based on studies in which she was involved