Presentation on theme: "How to smell a rat: Examples of scientific and regulatory failure in medicine Dr Aubrey Blumsohn Presentation at Radstats British Library, London 24 February."— Presentation transcript:
How to smell a rat: Examples of scientific and regulatory failure in medicine Dr Aubrey Blumsohn Presentation at Radstats British Library, London 24 February 2012
Nature of science – and how we define scientific fraud
Data – and access to data
Individual responsibility in science
Science Data Regulators Individual responsibility
History Einstein –Insanity: doing the same thing over and over again and expecting different results.
What is Science? UK Science Council (2009) –Spent a year working out a new definition of science. –Claim as the first "official definition of science" ever published. Science is the pursuit of knowledge and understanding of the natural and social world following a systematic methodology based on evidence
What is Science? – Mertonian Norms Communalism –Everyone can see whats going on Universalism –Doing of it not restricted to special people Disinterestedness –An arms length attitude. Skepticism (Organized Skepticism) - –Claims exposed to critical scrutiny. Awkward questions. Merton, Robert K. (1973), "The Normative Structure of Science",Merton, Robert K.
Science demands awkward people It takes all sorts of people
Example 1: Mr Nobody and Journal of the American Medical Association (JAMA)
Letter #1 to JAMA
Letter #2 To BMJ
Scientific deception - and its mis-definition
The Classics William Summerlin (Sloan Kettering, 1984) - faked transplantation experiments in white mice by blackening patches of their skin with a pen Vijay Soman (Yale, 1980) a diabetologist - wrote 12 papers based on invented data. His co- author Felig had to resign
The Classics Robert Slutsky (Radiologist, University of California). Published 137 papers between 1978 and one every 20 days. Many faked. John Darsee (Cardiologist, Harvard, 1981). Observed falsifying data. His boss, Eugene Braunwald ignored the problem. Later investigations discovered more than a 100 fraudulent papers
In these odd cases, the stories, as told, usually minimise the involvement of senior colleagues, University administrators, Hospitals and regulatory bodies who hid the misconduct before it was eventually exposed
The impression The choice of cases implicitly defines what we mean by scientific fraud/misconduct –Fraud is rare –It involves a particular type of activity (painting a mouse) –Usually involves an aberrant (male) acting alone (or duping otherwise honourable collaborators) Problems of scientific misconduct are rare - the product of psychopathic behaviour originating in temporarily deranged minds President of the National Academy of Sciences (at Congressional Hearings)
And the solution Is to create yet more procedures and guidance And for Universities to create robust, secret, legalistic procedures to investigate these rare rogues
Only one problem. As Arnold Schwarzenegger once said: it's bullshitall of it
The reality Not rare Scientific misconduct often not an individual activity – but a group activity (or involves group collusion)
Group-think Hannah Arendt (1965) –"rule by nobody". No single individual feels responsible for what is ultimately done. Indeed many people have gone to great lengths to make sure that this exact state of affairs achieved –Legislators –Ethics Committees –Trade bodies –Regulators System designers responsible for fraud?
The reality The offender is often not a lone white male psychopath Corporate fraud
The reality Corporation acting at the University interface
The reality or a journal editor
The reality or a regulator …. or government itself
The mechanisms also differ The mechanism of misconduct is also usually not direct fabrication of findings but something far more nuanced
Defining pornography "hard-core pornography" was hard to define, but "I know it when I see it Potter Stewart (Associate Justice of the US Supreme Court)
Misconduct Misconduct is nothing to do with the correctness of conclusion –To err is human, and scientific!!! It has to do with behaviours Behaviours which could disturb the scientific record Not binary We need to get away from ludicrous discussion about restrictive definitions and start discussing behaviours –Openly
Example 2: The fearful junior Approached by a Junior Doctor In order to gain her pathology registration she had to submit her dissertation She could not write up her dissertation or prepare the main area of work for publication because her supervisor had made this impossible – for years….. She could only present the secondary aspects The work involved looking at physiological changes in healthy volunteers randomised to atorvastatin or placebo The study showed that atorvastatin induced a diabetes like state Her supervisor was funded by the manufacturer of atorvastatin
Example 3: Does Seroquel cause diabetes and weight gain
But there was a cloud on the horizon. The F.D.A. was beginning to look into the issue of Diabetes in patients on Atypical Antipsychotics and wrote to Astrazeneca
Then somebody noticed the obvious that they had more than just 18 months of data in these subjects
Example 4 – Famovir and Herpes Famvir is used for treatment of herpes zosterFamvir There was a nine year delay before partial publication of findings which had adverse implications for the sponsor
What are data? Data is not the same as representation of that data
Getting a hamburger when what you need is the original cow
New York Times May 31, 2006 Example 5: Vioxx APPROVE Trial
Look at the language Merck & Co. acknowledged that it misidentified a statistical method Merck has contended that the study shows The company said in a statement yesterday Merck research chief Peter Kim said the company stood by its analysis Merck has insisted that its 18-month argument holds up while outside scientists have cast doubt on the company's interpretation.
Example 6: Vioxx AVANTAGE Trial On Oct. 21, 1999 a 73-year-old female participant dies in a clinical trial The trial was Mercks Advantage trial of Vioxx She died a few minutes after calling her son to tell him she felt short of breath
The Advantage trial was a very short (12-week) clinical trial eight people taking Vioxx suffered heart attacks or sudden cardiac death in the trial (compared with only one not on the drug) The difference was statistically significant, but Merck never showed the data that way
In the published study, the author (Dr. Lisse, University of Arizona), the 8 deaths became` 5, a difference that did not reach statistical significance. The paper never mentioned the other cardiac deaths of patients taking Vioxx, including the 73-year-old woman.
Dr. Lisse said that while he was listed as the paper's first author, Merck actually wrote the report. "Merck designed the trial, paid for the trial, ran the trial," Dr. Lisse said. "Merck came to me after the study was completed and said, 'We want your help to work on the paper.' The initial paper was written at Merck, and then it was sent to me for editing."
Results from the trials were also supposed to be blinded when coded During coding, Dr. Eliav Barr, a Merck employee judged that the woman had probably died of a heart attack."
Dr. Reicin a Merck executive immediately shot back : "I think this should be called an unknown cause of death." A few hours later, she wrote, "I would prefer unknown cause of death so we don't raise concerns."
In November 2001, an F.D.A. reviewer who had examined all the company's data concluded in a report to other agency officials that Merck had cheated. But the F.D.A. never told anyone either.
Dr. Lisse, the study "author" said he had never heard of the case of the woman who died, until told of it by the New York Times.
Example 7: Vioxx - Fake Journals Merck paid Elsevier to create a number of fake scientific journals This came to light 2009 in the context of a civil suit filed by Graeme Peterson, who suffered a heart attack in 2003 while on Vioxx These were designed in such a way that the average medical reader would mistake the publication for a genuine peer reviewed medical journal The articles supported Merck Drugs
One of the members of Australasian Journal of Bone and Joint Medicine's "Honorary Editorial Board," Peter Brooks, a rheumatologist said: he didn't recall who asked him to serve on the board, …. "You get involved in a whole bunch of things at this level".
Elsevier issued at Press release and an apology –We published a series of sponsored article compilation publications, on behalf of pharmaceutical clients, that were made to look like journals and lacked the proper disclosures. This was an unacceptable practice, and we regret that it took place. But what of the academics involved, and their institutions?
REGULATORS AND SYSTEMS Medical Writers (Ghost writers) Medical Journals MHRA (UK Drug regulator) Medical Ethics system UK Research Integrity Office General Medical Council
The illusion of due diligence "We're looking for the illusion of due diligence, Mr. Pope. Two criminal acts successfully prosecuted -- it gives us that illusion."
Medical Writers (and Ghostwriters)
When the time comes for publication, the medical writer writes the publication in consultation with the named author(s). This usually begins with discussions between the medical writer and authors(s) about the main features of the paper, after which the medical writer writes the first draft. The draft is then revised (sometimes many times) after review by the author(s). Thus although the authors do not actually write the paper, they do have control over its content, which is far more important. Naturally, they have access to the report of the study and its data tables, so they can verify that the paper is accurate. They often don't have access to the raw data, but it is worth repeating that raw data are of very little use when writing a paper. Results of a study need summarising and analysing before they mean anything, and it is those summaries and analyses that are important for proper interpretation of a study, not the raw data themselves. Adam Jacobs – European Medical Writers Association BMJ 21 Nov 2004
Motto – Which? Magazine No advertising, no bias, no hidden agenda. Just expert advice from an independent source
Motto – BMJ We protect our reputation by careful balance of editorial and advertising, which means your messages will always stand out. (http://group.bmj.com/group/advertising)
The Medicines and Healthcare Regulatory Agency (MHRA)
MHRA Has absolutely nothing to say on the problem of ghostwriting Has nothing to say on the problem of denial of access to data (to authors or the scientific community in general) Have no apparent concerns nor any apparent remit to deal with events at Sheffield University Even suggested that it might be illegal for authors to obtain the randomisation codes in a study they are authoring because commercial companies own those codes.
Risedronate FOI: –Did the MHRA receive and scrutinise in any way the raw data relating to the three key clinical trials which led to approval of Risedronate –Has the MHRA got the randomisation codes
Risedronate FOI response: –We cannot tell you because part of the response is commercially confidential Reply: The questions required answers of yes or no. Which parts of the Yes or No are commercially confidential Eventual FOI response: The MHRA did not scrutinise or retain any actual raw data before licensing Risedronate. They simply accepted what the company told them.
The MHRA (HOC SC Report 2005) "practices have developed which act against the public interest." The MHRA relies on company data presented as reports, in its decision whether or not to licence a drug. Raw data is very rarely analysed.
The MHRA (HOC SC Report 2005) The MHRA Chairman suggested that trust underpinned the stance of the MHRA towards the companies they regulate. We inferred that this extended to the routine acceptance of companies summaries of the results of tests on their drugs as true reflections of the raw data on which they were based. Evidence that regulatory agencies are acquiescing in biased later publication of the information they hold.
The MHRA (HOC SC Report 2005) Lack of transparency has played a major part in allowing failings to continue. 100% of the funding for the MHRA comes from the industry it regulates "In view of the failings of the MHRA, we recommend a fundamental review of the organisation."
The General Medical Council Body that regulates doctors A science-based profession Might have thought they would take scientific integrity seriously Long history of a very lax approach to scientific fraud –Particularly when it involves very senior academics
Dr David Hurlstone Sheffield Teaching Hospitals NHS Trust Bowel cancer research 2006 BUPA Foundation Research Award 2007 Medical Futures Innovation Oncology & Cancer Care Award GMC - dishonesty in three research studies published in 2007 & 2008 (not retracted) Suspended from Medical Register for 3 months On return to work retained Clinical Excellence Award
What did he do? In three publications –Submitted authors consent forms to journal with false signatures of collaborators –Studied patients without ethical approval –Created consent forms for patient records in retrospect –Stated falsely that patients had been randomised –Made false statements about microscopy having been carried out –Stated falsely that two blinded specialists had exampled samples – they did not exist –Stated falsely that an independent nurse participated in the study and that the study endoscopist was blinded And so on…..
GMC and Richard Eastell 3 publications intended by Procter and Gamble ( ) Authors actively denied randomisation codes by sponsor Eastell prepared to go along regardless Prepared to induce a colleague to do the same False conclusions in all manuscripts Research Dean of a Medical School R&D Director of Sheffield NHS Trust
How to get out of this? Didnt write the false statement The medical writer wrote that as well I dont need raw data as a scientist because I would not know how to analyse it (and presumably neither would my coauthors). It is very hard to analyse data. The statement was in –In 2002 I had a watershed moment when I realised this kind of activity was inappropriate. –I would never have done it again after 2002, and I will never do this sort of thing again. Legal manoeuvres to avoid having the GMC panel examine any information subsequent to this 2002 paper and repeated events
The GMC response 2002 statement –Data are raw data –Data are having the data –It was false –It was misleading –It was negligent Epiphany in /2004/2005??? No punishment - it would interfere with his ability to get research funding! This type of behaviour has no impact on patient care (according to the GMC)
Dr Guirong Jiang Radiologist Working as a postdoctoral fellow at the University of Sheffield One of the most experienced radiologists in the world in terms of diagnosis and definition of vertebral fracture Several complaints about Richard Eastell –Including his wish to author in the absence of data –Apparent secret contracts involving Eastell, Sheffield University and Commercial Companies preventing publication –Jiang not permitted to see contracts apparently preventing her right to publish findings she believed to be true based on studies in which she was involved