Presentation on theme: "Bazian Better health at lower cost 30 June 2011. Overview 2 Introduction to Bazian Evidology and cost vs quality Examples of Bazian work Search."— Presentation transcript:
Overview 2 Introduction to Bazian Evidology and cost vs quality Examples of Bazian work Search and appraisal example – aetiology of ADHD Behind the Headlines Brief introduction to NHS Evidence Hubs
Overview 3 Introduction to Bazian Evidology and cost vs quality Examples of Bazian work Search and appraisal example – aetiology of ADHD Behind the Headlines Brief introduction to NHS Evidence Hubs
Bazian Started by two clinicians from the BMJ Clinical Evidence - Anna Donald and Vivek Muthu Employ doctors, epidemiologists, librarians and scientists, and train intensively in evidence-based methods, communication, and implementation Interpret evidence to make it useful for policy, not just a dry summary of the literature 4 Our Purpose: To clarify the cost-quality relationship in healthcare....
Working with clients we use the following skills to answer questions and apply answers Demand and capacity modelling Health Technology Assessments Data Analysis / Literature reviews Clinical process flows Economic analysis
6 What is the clinical effect of doing X? Who in this population benefits/risk from X? What is the cost of achieving X? What is the national and local average for X? Is there any national guidance? What is a realistic target for this population? How is X achieved? What we should (not) do? How much do we need? What it will cost? What it should deliver? How do we monitor performance? Typical questions we answer ?
With a dedicated team of experienced clinicians, information specialists and health analysts 7 Clinical medicine Public health HTA & Systematic review Information research Health Services Research Statistics Health economics Clinical sciences Project managementMethods advisersSpecialist clinical advisers
Ev·i·do·l·o·gy n. A medical specialty that enables medical research to be incorporated into clinical practice. [Latin videre to discern, comprehend; evideri to appear plainly]
Overview 9 Introduction to Bazian Evidology and cost vs quality Examples of Bazian work Search and appraisal example – aetiology of ADHD Behind the Headlines Brief introduction to NHS Evidence Hubs
What is happening in healthcare? 10 Variable outputs
Expenditure does not relate to quality 11 Source: The Commonwealth Fund: Mirror, Mirror on the Wall
Examining the cost quality relationship UK – Dept of Health is prioritising its workload NICE QIPP Dartmouth Atlas Cost/Quality US budget proposal Savings to fund healthcare will come from among other things: “create ‘quality incentive payments’ for hospitals” ($12.1 billion), “drive down hospital readmission rates for Medicare patients” ($8.43 billion). “Comparative effectiveness research $1.1billion) 14 Myth Quality in healthcare is proportional to cost
What works, what doesn’t & what we don’t know Based on 2400+ systematic reviews of treatments Source: Clinical Evidence, Volume 14, December 2005 15
Questions need to be answered Cost How can we reduce spend and tackle budget deficits without compromising the clinical effectiveness of treatments? Evidence for treatment How should evidence on healthcare effectiveness guide commissioning decisions? Budget Allocations How might commissioners focus resources on treatments which work best and divert them away from those that do not? Evidology uses all medical research to inform clinical and management decisions 16
What evidence can do… Evidence enables insurers, purchasers and providers of healthcare to: identify, adopt, monitor and develop better and more efficient health services and care processes for the populations they serve address the real causes of medical cost escalation disinvest from poorer and less efficient health services and care processes To permit us, the consumers of healthcare, to: enjoy longer, healthier, happier lives afford the healthcare that we desire 17
18 Savings example #1: knee arthroscopy Each year over 18,000 knee arthroscopies are performed on the NHS Many of these procedures are done on patients with osteoarthritis for whom evidence suggests joint injections, or no intervention at all, would have been just as effective and a lot cheaper: ProcedureCost per patient Knee arthroscopy£1000-£2,500 Joint injections< £100 No intervention£0 Source Evidence-Based Healthcare & Public Health (2005) 9, pp 192-196, Author: Bazian Ltd
Savings example #2: Smoking cessation Before EB: Lots of GP’s opinions whether it was worth advising people to quit smoking as routine practice, or whether it made no difference, other than to alienate them.. The evidence: Brief advice to encourage people to quit smoking delivered by a physician, compared with no advice: AR 5.9% with brief advice v 4.1% with no advice OR 1.69, 95% CI 1.45 to 1.98 Combining nicotine replacement therapy with advice: AR 16.8% with advice + NRT v 10.2% with advice + placebo OR 1.74, 95% CI 1.64 to 1.86 After EB: GP practices in the UK now adopt smoking cessation advice as a routine policy, rather than as an optional, if-you-feel-like-it event. 19
The long and painful process of putting evidence into practice is worth it because… Essential to escape the healthcare cost escalation There really are ‘better’ and ‘worse’ treatments and care configurations and we need to be able to distinguish between them It is the only way to reveal previously opaque cost-quality relationships Essential for maintaining competitiveness as healthcare becomes increasingly commoditised 22
Why don't commissioners do this already? Asking commissioners to ‘find their own evidence’ is a bit like asking GPs to do their own MRI scans. They simply do not have the years of training, experience, equipment, or time to do so. They need a responsive and dedicated evidence service – a bit like a specialist radiology service – which provides evidence answers in real time. 23
Overview 24 Introduction to Bazian Evidology and cost vs quality Examples of Bazian work Search and appraisal example – aetiology of ADHD Behind the Headlines Brief introduction to NHS Evidence Hubs
26 Deliverables Provision of documents including training, guidance for PCTs, evidence synthesis reports and draft scopes Specialist support for Appeals Example #1 CSAS: ensuring commissioners engage with NICE To help commissioners engage in NICE TAs so that population needs and opportunity costs are represented. We are the clinical and analytical engine behind the CSAS reports for NICE.
What will this new NICE TA mean for my PCT? How can I influence the NICE process? 27 “Analysis revealed that: For an average sized local health economy, for £1.4million, 20 women can be treated with Drug X as an alternative to Drug Y... At the end of 1 year, about 3 extra people will be alive who otherwise would not have been After 2 years there will be no overall survival benefit for the women treated with combination therapy....”
28 Example #2 EMSCG: improving commissioning of RRT To help commissioners purchase renal replacement therapy services across the East Midlands, including long- term planning for the dialysis service. Objectives 1.to map the geographical variation in need across the East Midlands (age/sex and ethnicity) 2.to report on capacity and provision of dialysis services by provider and compare this to need 3.to qualitatively describe any pressures within the system.
Interactive pathway simulation identified future service efficiencies, and gained buy-in from specialist clinicians 31
Map of demand density vs local capacity to plan unit location
33 Some more single-slide examples of work we’ve done…
34 Deliverable An “Evidence Brief” which includes; Policy background and description of local demographics and issues from existing reports Evidence summary and policy suggestions Description of the technology Evidence tables Methodology What are the psychological outcomes following gender reassignment surgery for gender dysphoria? We carried out independent evidence-based analysis of important positive and negative outcomes, including psychological effects, suicide rates and quality of life.
35 Deliverables Bazian summarised the findings of the review, describing implications with regard to the issues raised. Including how to improve and update the eligibility criteria used by the NHS in England. Presentation of findings to NSC or expert group We carried out a guideline comparison and survey of international current practice to ensure fair access to the drugs used to treat these disorders are in line with international practice. How do clinicians use Ultra Orphan Drugs for Lysosomal Storage Disorders?
36 Deliverable A literature review An economic report An interactive economic model in spreadsheet (.xls) format suitable for downloading by users from a website. What is the effect of Functional Electrical Stimulation (FES) for drop foot of the central neurological origin? To form the basis of CEP’s agreement with supplier for delivery of those products within appropriate constraints
37 Deliverable A literature review An interactive economic model in spreadsheet (.xls) format. Can we have a guideline for…? Revised and updated the original guideline for the recognition and assessment of acute pain in children
Overview 38 Introduction to Bazian Evidology and cost vs quality Examples of Bazian work Search and appraisal example – aetiology of ADHD Behind the Headlines Brief introduction to NHS Evidence Hubs
Search strategy Must balance sensitivity and specificity To refine list of putative risk factors we carried out a brief scoping in PubMed An information specialist reviewed the results and extracted a list of potential risk factors. This list was augmented by adding risk factors suggested by the reference group. Key search terms covering these risk factors were included in the search strategy to increase likelihood of identifying relevant studies Sources: MEDLINE, EMBASE, CINAHL and PsycINFO Hand-searched the reference lists of relevant papers The reference group and “key informants” identified by the reference group were contacted to supply personal reference lists and information on ongoing, unpublished or in- press research Grey literature sources such as Google Scholar, TRIP and OAIster.Google ScholarTRIPOAIster 40
ADHD aetiology search (Medline) 41 1 Attention Deficit Disorder with Hyperactivity/ (13393) 2 (attenti$ adj3 (deficit$ or disorder$ or hyperactiv$ or hyper?activ$ or adhd or addh or ad??hd or "add")).mp. [mp=title, original title, abstract, name of substance word, subject heading word] (18981) 3 ((hyperkin$ or hyper?kin$) adj3 (deficit$ or disorder$ or hkd)).mp. [mp=title, original title, abstract, name of substance word, subject heading word] (496) 4 or/1-3 (19291) 5 (cohort$ or longitudinal or follow?up or randomi$ or randoml$ or RCT or prospective or retrospective or case?control or cross?section$ or observational or epidemiolog$ or case?series or linkage$ or epigenetic$).mp. [mp=title, original title, abstract, name of substance word, subject heading word] (1573617) 6 ((genome$ or gene$ or chromosome$) adj5 (wide or analys$ or map$ or associa$ or environ$)).mp. [mp=title, original title, abstract, name of substance word, subject heading word] (332066) 7 ((candidate or suscept$) adj5 (gene$ or loci$)).mp. [mp=title, original title, abstract, name of substance word, subject heading word] (46254) 8 between group$.tw. (43128) 9 ((twin$ or sibling or adopti$) adj3 (stud$ or analys$ or pair$)).mp. [mp=title, original title, abstract, name of substance word, subject heading word] (13716) 10 or/5-9 (1891632) 11 (risk$ or aetiol$ or etiol$ or caus$ or associat$ or linkage$ or heritab$).mp. (3734422) 12 (dopamine$ or serotonin$ or hla-dr$ or adrenergic or catechol O-Methyltransferase or cannabinoid$ or polymorphi$ or genom$ or thyroid).mp. [mp=title, original title, abstract, name of substance word, subject heading word] (877609) 13 ((maternal or paternal or parent$ or famil$ or mother$ or father$ or prenatal or pregnan$) adj3 (smok$ or tobacco or depression or stress or restrictiveness or substance or alcohol$ or coffee or social or class or education or psychiatri$ or disorder$)).mp. [mp=title, original title, abstract, name of substance word, subject heading word] (50975) 14 exp family/ (182571) 15 (lead or mercury or manganese or polychlorinated biphenyl or hexachlorobenzene or pollutant$ or PBB or PCB or DDE or DDT or pesticide$ or organophosphate$ or pyrethroid$ or zinc or iron or magnesium or omega-3 or sugar$ or "e numbers" or e-numbers or colouring$ or toxin$ or additive$ or diet$ or malnutrition$ or malnour$ or undernutrition$ or undernour$).mp. [mp=title, original title, abstract, name of substance word, subject heading word] (1253538) 16 exp pesticides/ or exp coloring agents/ or exp flavoring agents/ or exp sweetening agents/ or Environmental Exposure/ (422004) 17 ((pregnan$ or delivery or birth) adj3 (weight or preterm or complication$ or BMI or season)).mp. [mp=title, original title, abstract, name of substance word, subject heading word] (164841) 18 exp Pregnancy Complications/ or exp Obstetric Labor Complications/ (276386) 19 (breast-feeding or breast-fed).mp. [mp=title, original title, abstract, name of substance word, subject heading word] (24937) 20 exp Infant Nutritional Physiological Phenomena/ (35245) 21 (head adj3 (trauma$ or injur$)).mp. [mp=title, original title, abstract, name of substance word, subject heading word] (22597) 22 exp Craniocerebral Trauma/ (95881) 23 (television or video or computer).mp. [mp=title, original title, abstract, name of substance word, subject heading word] (385594) 24 exp "Play and Playthings"/ (6132) 25 (childhood adj3 depriv$).mp. [mp=title, original title, abstract, name of substance word, subject heading word] (102) 26 sleep depriv$.mp. [mp=title, original title, abstract, name of substance word, subject heading word] (6647) 27 or/11-26 (6092705) 28 4 and 10 and 27 (3536) 29 (animal$ not human$).sh. (3286408) 30 (letter or editorial).pt. (910772) 31 28 not (29 or 30) (3446) Condition Aetiology & causes Study type
Note that searches vary according to client or project need 44
Appraisal and inclusion/exclusion Three stages: 1.First pass appraisal at the level of title and abstract, appraised simply on whether the study was relevant to the question. No decision was made on study quality. The appraisals were carried out by an information specialist and a health research analyst. Tagged keywords in Reference Manager were used to provide an audit trail of accepted and rejected articles. The full text versions of articles selected for further appraisal were then acquired 2.Second stage appraisal by a health research analyst at the level of abstract only and applying the full inclusion and exclusion criteria detailed below. Those that did not provide enough information at the level of the abstract were obtained at full text, for third stage appraisal 3.Third stage appraisal by a health research analyst at the level of full text and applying the full inclusion and exclusion criteria detailed below 45
Overview 48 Introduction to Bazian Evidology and cost vs quality Examples of Bazian work Search and appraisal example – aetiology of ADHD Behind the Headlines Brief introduction to NHS Evidence Hubs
Summary of the news stories, the research and the take home message Where did the story come from? What kind of scientific study was it? What were the results of the study? What interpretations did the researchers draw from these results? What does the NHS Knowledge Service make of this study? Links to headlines, science, and further reading 4 May, 201550 Content
How do we critically appraise the research? Ask a number of key questions: 1.Has the research come from a reliable source? 2.What question was the research trying to answer? 3.How well designed was the study for answering the question? 4.How well was the study conducted? 5.What are the results? 6.What do the results mean for the average person? 7.How is the study reported in the news? 4 May, 201551
1. Has the research come from a reliable source? Has the research been fully published? 12% of a random sample of 200 press releases from 20 US academic medical centres in 2005 promoted unpublished research from scientific meetings * 5% of these included a caution about the unpublished nature of the research * Has the publication been peer reviewed? How was the research funded? 68% reported sources of funding * Are there any potential conflicts of interest? 4% noted potential conflicts of interest * *Woloshin et al. Ann Int Med 2009;150(9):613-618 4 May, 201552
2. What question was the research trying to answer? Does a treatment work? Acupuncture for back pain Blood pressure drugs for all over 55s Does a treatment cause harm? Aspirin and brain bleed Can an intervention prevent a disease? Aspirin and cancer Does a diagnostic test work? Test for malignant prostate cancer What causes a disease/condition? Nicotine gum and cancer Autism genetics How does a treatment work/an exposure cause a disease? Cancer magnet protein How common is a disease/condition/behaviour? CAM use among cancer patients 4 May, 201553
3. How well designed was the study for answering the question? Some study designs are more appropriate than others for answering certain types of questions Was the study in humans? among potential therapeutic drugs reported in highly cited animal trials in top scientific journals 1980 to 2000 † : 45% were not followed up by a published human RCT (by 2006) 18% were found not to be effective in a published human RCT 37% were found to be effective in a published human RCT 11% (8 interventions) were approved for use in patients 43.5% of 2005 US academic medical press releases were of animal or laboratory studies; 10% of these included a caution about extrapolating results to humans* about 20% of BtH studies in 2008 were animal or laboratory studies † Hackam & Redelmeier JAMA 2006; 296(14): 1731-1732 *Woloshin et al. 2009 4 May, 201554
Hierarchy of study designs RCTs Cohort studies Case-control studies Cross sectional studies Case series Case reports Expert opinion Anecdotal evidence 4 May, 201555 Systematic reviews US academic medical centres press releases 2005* Behind the Headlines 2008 10% 6% 16% 22% 8% 13% 4% 1% 27% controlled observational studies 3% uncontrolled intervention studies *Woloshin et al. 2009 3%
4. How well was the study conducted? How many participants were there? What type of participants were included? How were they selected? Was an appropriate control used? Was the study prospective or retrospective? Was the outcome a clinical outcome (e.g. death, heart attack) or a proxy outcome (e.g. blood pressure)? How was the outcome measured? Validated scale? Researchers and/or participants blinded to treatment? Particularly important for subjective outcomes (e.g. pain) How many people dropped out? Was the method of analysis appropriate? 4 May, 201556
5. What are the results? How are the results presented? Benefits and harms Absolute and relative figures Contrast the impact of: “Wonder drug reduces deaths from horrible disease from 0.1% to 0.05%” “Wonder drug prevents 5 deaths for every 10,000 people treated” “Wonder drug halves deaths from horrible disease” Are the results statistically significant? Which results are statistically significant? Primary outcome(s) Secondary outcome(s) Only one comparison out of many comparisons Between group comparison or within group comparison versus baseline Are the results clinically significant? Are authors’ conclusions justified? 4 May, 201557
6. What do the results mean for the average person? Who do the results apply to? What outcome do they apply to? Should people change their behaviour because of this study? 4 May, 201558
7. How is the study reported in the news? Does the news give a reasonable summary or is it over-interpreted? Does the headline answer a different question to that asked by the study? Do the photos/illustrations accurately represent the study? Does the body text clearly and accurately report: study source potential conflicts of interest question asked whether study was in humans what outcomes were assessed study design size of study results (absolute figures as well as relative figures) limitations to the study reasonable implications of study findings? 4 May, 201559
Summary There is a hierarchy of study designs: some designs are more appropriate for certain questions than others All studies have limitations, these may be related to: how appropriate the study design is for answering the question asked how the study was conducted how the results are interpreted how far the results can be extrapolated It’s important to know what these limitations are when determining what conclusions can be drawn from the study 4 May, 201560
4 May 201561 Type of question and its appropriate study design Type of questionAppropriate study design Exploration of hypothesesQualitative research TreatmentRCT or crossover RCT HarmsRCT ideally, or observational PrognosisCohort AetiologyCohort DiagnosisCross-sectional PrevalenceCross-sectional IncidenceCohort
4 May 201562 Size isn’t everything Error Study size Source: Rothman, 2002 Systematic error (bias) Random error (chance)
Overview 63 Introduction to Bazian Evidology and cost vs quality Examples of Bazian work Search and appraisal example – aetiology of ADHD Behind the Headlines Brief introduction to NHS Evidence Hubs
4 May, 201564 NHS Evidence Created following the Darzi report, ‘High Quality Care for All’ (2008). The report recognised that NHS staff wanted to work with the best quality information available, but could find it difficult to access NHS Evidence search portal and service was launched on 30 April 2009, operating within NICE “NHS Evidence aims to be the service that health and social care staff rely on for up-to-date, relevant, high quality evidence-based local and national information and knowledge to inform decision-making resulting in better quality patient care.” Doesn’t happen overnight!
4 May, 201565 Specialist collections, all change Previously, over 30 “specialist collections”, outsourced to a range of organisations within the NHS and wider health community. The specialist collections also produced Annual Evidence Updates on key conditions, and supported engagement with and use of evidence within their specialist community In April 2011, NHS Evidence was launch with a redesigned evidence service Core to the delivery of this improved service was the redesign of the existing Specialist Collection service into a new specialist information management service which provided by two Evidence Hubs: one within NICE (Manchester), and one at Bazian The Hubs are invisible to the user. It’s not a replacement of the Specialist Collections but rather a completely new model of content provision Other, smaller Hubs provide content on screening, public health, medicines etc
4 May, 201567 Contract HostLocation Number of collections managed Total FTE excl Clinical Leads Infections Health Protection Agency London 11.6 Kidney and male urogenital diseases Health Care Libraries, Oxford University Library Services Oxford 11 Later life & Mental health MinervationOxford 22.2 Learning disabilities MinervationOxford 11.4 Musculoskeletal & Orthopaedics Arthritis Research Campaign Chesterfield 22 Oral health Cardiff UniversityCardiff 11.1 Skin disorders University of NottinghamNottingham 11 Supportive and palliative care University of SheffieldSheffield 11.2 Surgery, theatres, perioperative and critical care Royal College of Surgeons of England London 12 Women's health Nuffield Department of Obstetrics and Gynaecology, John Radcliffe Hospital Oxford 12 Contract HostLocation Number of collections managed Total FTE excl Clinical Leads Cancer Oxford Radcliffe Hospitals NHS Trust Oxford 12.4 Cardiovascular including Stroke and Vascular University of SurreyGuilford 35.2 Respiratory Sheffield Children's Hospital/University of Sheffield Sheffield 11 Commissioning & Health management & I&I King's FundLondon 33.8 Complementary and alternative medicine Royal London Homoeopathic Hospital London 11.2 Emergency care & Nutrition and metabolic (inc Diabetes) Warwick Medical School, University of Warwick Warwick 22.7 ENT and audiology Oxford Radcliffe Hospitals NHS Trust Oxford 11 Ethnicity and health De Montfort University Leicester 10.9 Eyes and vision Moorfields Eye Hospital Foundation NHS Trust London 11.3 Gastroenterology and liver diseases & Neurological conditions Royal Free Hospital Medical Library, Royal Free Hampstead NHS Trust London 21.4 Genetic conditions Birmingham Women's Healthcare NHS Trust (National Genetics Education and Development Centre) Birmingham 11
4 May, 201568 Bazian Evidence HubNICE Evidence Hub Subject Area: Organ system or clinical speciality based Anaesthetics Cancers Cardiovascular system disorders Child Health Complementary and alternative therapies Commissioning Critical care Diabetes Diagnostics Ear, nose and throat Fertility and childbirth Eyes and vision Gastrointestinal disorders Genetics Gynaecological disorders Health management Hepatic disorders Later life Learning disabilities Mental health and illness Neonates and neonatal care Nervous system disorders Oral and dental health Perioperative care Renal and urogenital disorders Sexual health Surgery Allergies Endocrine disorders Emergency and urgent care Equality and Diversity Palliative and End of Life Care Haematological and immunological disorders Infections Medically unexplained symptoms Metabolic disorders Musculo-skeletal disorders Orthopaedics Respiratory disorders Skin conditions Supportive care Trauma Total27 15 * * *
4 May, 201569 “Key product and activities” Identifying and adding value to evidence resources FAST ingested resources (inc. core and non-core sources) Orphan sources (bibliographic) Orphan sources (other) Providing an evidence notification service Eyes on Evidence and “Important new evidence” Contributing to producing Evidence Updates About n=30 a year Identifying Research Recommendations for UK DUETs Through Evidence update process Supporting engagement with Specialist Communities Health Management and Commissioning Expertise Working with The Kings Fund
Thank you for listening! 70 Introduction to Bazian Evidology and cost vs quality Examples of Bazian work Search and appraisal example – aetiology of ADHD Behind the Headlines Brief introduction to NHS Evidence Hubs