AHRQ IQI Clinical Validation Panels Sheryl Davies, MA Stanford University.

Slides:

Advertisements

Similar presentations

Chapter 3: Clinical Decision-Making for Massage

Advertisements

Evidence-based Dental Practice Developing guidelines or clinical recommendations Slide #1 This lecture follows the previous online lecture on evidence.

Measure Criteria MD Measurement, P4P Phyllis Torda NCQA February 2008.

WORKFORCE PLANNING June 2011 Amr Fouad Training & Research Sector Ministry of Health & Population.

The Application of Propensity Score Analysis to Non-randomized Medical Device Clinical Studies: A Regulatory Perspective Lilly Yue, Ph.D.* CDRH, FDA,

Solving the Faculty Shortage in Allied Health 9 th Congress of Health Professions Educators 4 June 2002 Ronald H. Winters, Ph.D. Dean College of Health.

Adverse Patient Safety Events: Costs of Readmissions and Patient Outcomes Following Discharge Didem M. Bernard, Ph.D. William E. Encinosa, Ph.D.

VALIDATING AHRQ PATIENT SAFETY INDICATORS with the VA NATIONAL SURGICAL QUALITY IMPROVEMENT PROGRAM POSTOPERATIVE ADVERSE EVENTS Dennis Tsilimingras, M.D.,

Surgery volume and operative mortality: A re-examination using fixed-effects regression Amresh Hanchate, PhD Section of General Internal Medicine Boston.

Pay-for-Performance Programs: the U.S. Experience Eric Schneider, M.D., M.Sc. Harvard School of Public Health Brigham and Womens Hospital Boston, MA.

Multinational Comparisons of Health Systems Data, 2008 Support for this research was provided by The Commonwealth Fund. The views presented here are those.

1 Using Data to Drive Health System Performance Commissioned from Ovations by the National Primary and Care Trust Development Programme.

Introduction Discussion Results Mobile Health Clinics are transportable health care units that deliver high-value community based health care to underserved.

Primary Care – Changing Future 1 PRIMIS 23 rd April 2002 Metropole Birmingham.

2008 Johns Hopkins Bloomberg School of Public Health Setting Up a Smoking Cessation Clinic Sophia Chan PhD, MPH, RN, RSCN Department of Nursing Studies.

2014 National Patient Safety Goals

Promoting Regulatory Excellence Self Assessment & Physiotherapy: the Ontario Model Jan Robinson, Registrar & CEO, College of Physiotherapists of Ontario.

Lecture 3 Validity of screening and diagnostic tests

THE COMMONWEALTH FUND Figure 1. More Than Two-Thirds of Opinion Leaders Say Current Payment System Is Not Effective at Encouraging High Quality of Care.

7/16/08 1 New Mexico’s Indicator-based Information System for Public Health Data (NM-IBIS) Community Health Assessment Training July 16, 2008.

MEDICAL HOME 1/2009 Mary Goldman, D.O., President of MAOFP.

Ron D. Hays, Ph.D. Alex Y. Chen, M.D. UCLA Children’s Hospital LA

Health System Funding Reform Funding Release – HNHB “Dramatically improving the patient experience through Quality, Integration and Value” Audit.

© Copyright, The Joint Commission 2015 National Patient Safety Goals.

Participation Requirements for a Patient Representative.

Sub-heading ADMINISTRATOR EVALUATION AND SUPPORT SYSTEM Athletic Program Leader Proposed Adaptations.

Improving the quality of medical and surgical care NCEPOD Dr Marisa Mason.

The Impact of Intensive Care Unit Structure on Post-operative Outcomes Following Congenital Heart Surgery: Analysis of a Multi-institutional Database Danielle.

1 Overview of Organization & Comparative Quality Data Ron Spingarn, Deputy Director Healthcare Information Division Phone:

Quality Improvement Research Carolyn Clancy, MD Director Agency for Healthcare Research and Quality Secretary’s Advisory Committee on Human Research Protections.

ACHD Patients Should Receive Treatment in Adult Institutions Society of Thoracic Surgeons Adult Cardiac Surgery Database reports 39,872 adults undergoing.

Impact of the MS-DRGs on the AHRQ QIs

Critical Appraisal of Clinical Practice Guidelines

Agency For Healthcare Quality and Research Quality Indicators NH Health Care QA Commission AHRQ Subcommittee Report July 31, 2009.

Overview Public Reporting Cardiovascular Data Recommendations.

CAHPS® Survey to Evaluate the Patient’s Experience with the Medical Home Ron D. Hays, Ph.D. UCLA Department of Medicine RAND Health Program November 18,

Implementing universal Lynch Syndrome screening in a large healthcare system.

AATS/STS Congenital Heart Disease Symposium: Unsettled and Unanswered Questions in Congenital Heart Surgery Lessons Learned from the SVR Trial: Update.

Method Two month data collection period (Feb-Mar 2004) NHS and independent hospitals in England, Wales, N Ireland, Guernsey, Isle of Man and Defence Secondary.

Leapfrog’s “Survival Predictor”: Composite Measures for Predicting Hospital Surgical Mortality May 7, 2008.

Elizabeth A. Martinez, MD, MHS Johns Hopkins Medical Institutions September 10, 2008 Organization of Care and Outcomes in Cardiac Surgery AHRQ grant 1K08HS A1.

What do we know about overall trends in patient safety in the USA? Patrick S. Romano, MD MPH Professor of Medicine and Pediatrics University of California,

Expanding the Uses of AHRQ’s Prevention Quality Indicators: Validity from the Clinician Perspective Presented by: Sheryl Davies, MA Stanford University.

Outpatient Services and Primary Health Care Heidi Kinsell Master of Health Administration (MHA) Health Services Research, Management and Policy 1.

AHRQ Quality Indicators 2005 AHRQ QI User Meeting September 26, 2005 Marybeth Farquhar, AHRQ.

Development & Evaluation of the Forthcoming AHRQ Neonatal Quality Measures AHRQ 2007 Annual Conference, Session 3C Overview of the AHRQ Quality Indicators.

The Technology Exchange for Cancer Health Network (TECH-Net) AHRQ Annual Conference: Improving Healthcare, Improving Lives September 27, 2007 Teresa M.

Going Public Ben Yandell, PhD, CQE Clinical Information Analysis Norton Healthcare, Louisville, KY

END Obesity Dr Gul Bano © S Nussey. What is obesity?

Healthcare Workforce and Regionalization of Services: Lung Cancer Resections Stephen C. Yang, M.D. Chief of Thoracic Surgery The Arthur B. and Patricia.

AHRQ PSIs and IQIs in National Pay for Reporting September 14, 2009 AHRQ QI Conference Shaheen Halim, Ph.D. Centers for Medicare & Medicaid Services.

AHRQ Quality Indicators NQF Update Marybeth Farquhar, PhD, MSN, RN QI Users Meeting AHRQ 2 nd Annual Conference Rockville, MD September 10, 2008.

Panel Reviews of the AHRQ QIs: AHRQ and NQF panels Sheryl Davies, MA Stanford University.

BlueCross BlueShield of Illinois a Division of Health Care Service Corporation (HCSC), a Mutual Legal Reserve Company Blue Cross Blue Shield of Illinois.

Inpatient Quality Reporting In Colorado Sept HCUP User Group Meeting.

Final Audited Registration Report. Final Audited Total Registered Report Total Registered 11,034 Physician 4,483 AHPAHP 850 Industry Professional 1,844.

Volume of care and outcomes: synthesis of the available evidence. Eliana Ferroni 16 th October 2012.

Developing and Implementing Intervention Studies Using Geriatric Assessment Supriya Gupta Mohile, M.D., M.S. Assistant Professor of Medicine James Wilmot.

Management of Morning Hyperglycemia Following Cardiac Surgery LUMC 2ICU CV-Surgical Team CV AnesthesiaNursing Staff Pharmacy StaffAnesthesia Residents.

AHRQ QI Guide to Comparative Reporting AHRQ Annual Conference September 10, 2008 Bethesda, MD Presented by Sheryl Davies.

Making Surgery Safer: Preventing Post Operative Myocardial Infarction Departments: Anesthesia, Cardiology, General Surgery, Orthopaedics, Primary Care,

Building an Evidence-Based Nursing Practice

Insights from the NCDR® STS/ACC TVT Registry.

Performance Measurement and Rural Primary Care: A scoping review

Compensation Committee 2017 Goals – Updated

Value Based Contracting in Action

Method Two month data collection period (Feb-Mar 2004)

GMHC Board of Directors November 14, 2016

Results from the first National Lung Cancer Organisational Audit.

Rick A. Nishimura et al. JACC 2019;73:

Presentation transcript:

AHRQ IQI Clinical Validation Panels Sheryl Davies, MA Stanford University

Outline Purpose Purpose Composition and Recruitment Composition and Recruitment Rating Process Rating Process Overall themes Overall themes Indicator results Indicator results Take home lessons Take home lessons

Purpose of the Clinical Panels IQIs and PQIs IQIs and PQIs – Developed – Based on established indicators – Did not undergo panel review Panel review establishes face validity of the indicators Panel review establishes face validity of the indicators Standardize available evidence for all AHRQ QIs Standardize available evidence for all AHRQ QIs – Establish face validity for one stakeholder group – Update evidence Panel review for IQIs considered for NQF endorsement Panel review for IQIs considered for NQF endorsement

Methods Modified RAND/UCLA Appropriateness Method (Nominal Group Technique) Modified RAND/UCLA Appropriateness Method (Nominal Group Technique) Physicians of various specialties/subspecialties and other health professionals were recruited with the assistance of relevant organizations Physicians of various specialties/subspecialties and other health professionals were recruited with the assistance of relevant organizations 22 contacted organizations nominated 22 contacted organizations nominated – 103 clinicians nominated 60 accepted 45 eligible Panelists selected in order to form diverse panels Panelists selected in order to form diverse panels

Panelists Female: 18% Female: 18% Academic: 71% Academic: 71% Geographic Geographic – East: 44% – West: 29% – Other: 27% Practice setting Practice setting – Urban: 67% – Suburban: 42% – Rural: 18% Funding of primary hospital: Funding of primary hospital: – Private: 44% – Public: 27% Underserved patient population: 56% Underserved patient population: 56%

Panel methods: Ratings Initial ratings Initial ratings – Packet of information summarizing evidence – Approx. 10 questions Tailored to the indicator type Tailored to the indicator type 9 point scale 9 point scale Overall usefulness for quality improvement, comparative reporting Overall usefulness for quality improvement, comparative reporting – Compiled ratings provided to panelists Conference call Conference call – Discuss differences – Consensus on definition changes Final ratings Final ratings – Empirical analyses provided – Using same questionnaire as initial ratings

Results: Overarching themes Case mix variability Case mix variability Reliability Reliability Volume measures as indirect measures of quality Volume measures as indirect measures of quality – Composite measures

Cardiac Panel Reviewed 5 indicators: Reviewed 5 indicators: – AAA Volume/Mortality – Pediatric Heart Surgery Volume/Mortality – Bilateral Catheterization 11 clinicians: vascular surgeons, pediatric cardiologists, pediatric cardiovascular surgeons, interventional cardiologists, pediatric ICU nurse, surgical nurse 11 clinicians: vascular surgeons, pediatric cardiologists, pediatric cardiovascular surgeons, interventional cardiologists, pediatric ICU nurse, surgical nurse

Cardiac Panel IndicatorOverall – QIOverall - Comparative AAA MortalityAcceptable (7)Unclear (6) AAA VolumeAcceptable (7) Bilateral catheterization Unclear (5)Unclear with disagreement (5) Pediatric heart surgery volume Acceptable (8) Pediatric heart surgery mortality Acceptable (8)

Cardiac Panel IndicatorOverall – QIOverall - Comparative AAA MortalityAcceptable (7)Unclear (6) AAA VolumeAcceptable (7) Case mix variability Ruptured vs. unruptured; endovascular vs. open Bias: Slight overadjustment for endovascular (12%) and underadjustment for ruptured (12%) Total volume (ruptured and unruptured) best predictor of outcomes Stratify by surgical approach (endovascular vs. open)

Cardiac Panel IndicatorOverall – QIOverall - Comparative AAA MortalityAcceptable (7)Unclear (6) AAA VolumeAcceptable (7) Case mix variability Ruptured vs. unruptured; endovascular vs. open Bias: Slight overadjustment for endovascular (12%) and underadjustment for ruptured (12%) Total volume best predictor of outcomes Stratify by surgical approach (endovascular vs. open)

Cardiac Panel IndicatorOverall – QIOverall - Comparative Pediatric heart surgery volume Acceptable (8) Pediatric heart surgery mortality Acceptable (8) Case mix variability Supported use of RACHS Correlations of hospital volume for each RACHS complexity are robust (r = 0.74 – 0.95) Best predictor of outcome is total volume, rather than by complexity

Cardiac Panel IndicatorOverall – QIOverall - Comparative Bilateral catheterization Unclear (5)Unclear w/ disagreement (5) Modification: Expand list of appropriate indications for bilateral catheterization Primarily a resource indicator Charting of indications may be poor May result in decrease of appropriate uses

Surgical Resection Panel Reviewed 4 indicators: Reviewed 4 indicators: – Esophageal Resection Volume/Mortality – Pancreatic Resection Volume/Mortality 13 clinicians: thoracic surgeons, general surgeons (including GI and oncology), oncologists, internist, gastroenterologists, surgical nurse 13 clinicians: thoracic surgeons, general surgeons (including GI and oncology), oncologists, internist, gastroenterologists, surgical nurse

Surgical Resection Panel IndicatorOverall – QIOverall - Comparative Esophageal resection mortalityAcceptable (7) Esophageal resection volumeAcceptable (7) Pancreatic resection mortalityAcceptable (7) Pancreatic resection volumeAcceptable (7)

Surgical Resection Panel Esophageal resection Case mix variability Risk adjustment performs well. Underestimates risk for patient with middle esophageal and unspecified site cancer (22%, 37%). Pancreatic resection Case mix variability Over-estimates risk for total pancreatectomy (68%), lesser extent underestimates risk for Whipple (15%). Issue raised with 3M. Low rates

Geriatric Panel Reviewed 4 indicators: Reviewed 4 indicators: – Acute Stroke Mortality – Hip Fracture Mortality – Hip Replacement Mortality – Incidental Appendectomy 14 clinicians: internists (including geriatrics and hospital medicine), neurologists, general surgeon, interventional radiologist, orthopedic surgeons, neurosurgeon, diagnostic radiologist, nurse, physical therapist 14 clinicians: internists (including geriatrics and hospital medicine), neurologists, general surgeon, interventional radiologist, orthopedic surgeons, neurosurgeon, diagnostic radiologist, nurse, physical therapist

Geriatric Panel IndicatorOverall – QIOverall - Comparative Acute stroke mortality Unclear (6.5)Unclear with disagreement (5) Incidental appendectomy Acceptable (7)Unclear (6) Hip fracture mortalityAcceptable (7) Hip replacement mortality Unclear due to disagreement (7) Unclear (6)

Geriatric Panel IndicatorOverall – QIOverall - Comparative Acute stroke mortality Unclear (6.5)Unclear with disagreement (5) Incidental appendectomy Acceptable (7)Unclear (6) Hip fracture mortalityAcceptable (7) Hip replacement mortality Unclear due to disagreement (7) Unclear (6) Case mix variability: Stroke type (hemorrhagic, ischemic, subarachnoid) Risk adjustment accounts for almost all difference in risk Patient factors such as delay in presenting for care

Geriatric Panel IndicatorOverall – QIOverall - Comparative Acute stroke mortality Unclear (6.5)Unclear with disagreement (5) Incidental appendectomy Acceptable (7)Unclear (6) Hip fracture mortalityAcceptable (7) Hip replacement mortality Unclear due to disagreement (7) Unclear (6) Exclude patients with hip fracture Case mix variability Risk adjustment somewhat overestimates risk for revision Rates very low, reliability concerns

Geriatric Panel IndicatorOverall – QIOverall - Comparative Acute stroke mortality Unclear (6.5)Unclear with disagreement (5) Incidental appendectomy Acceptable (7)Unclear (6) Hip fracture mortalityAcceptable (7) Hip replacement mortality Unclear due to disagreement (7) Unclear (6) Is it still being done? If it is still being done, it shouldnt be done. Then it is a good indicator I am having a hard time getting excited about this indicator

Geriatric Panel IndicatorOverall – QIOverall - Comparative Acute stroke mortality Unclear (6.5)Unclear with disagreement (5) Incidental appendectomy Acceptable (7)Unclear (6) Hip fracture mortalityAcceptable (7) Hip replacement mortality Unclear due to disagreement (7) Unclear (6) Limit to the elderly Case mix variability Risk adjustment accounts for both repair type and fracture location

Conclusion Overall good reception of indicators Overall good reception of indicators Recommendations considered in context of other validation efforts Recommendations considered in context of other validation efforts Indicator revisions implemented in February 2008 Indicator revisions implemented in February 2008 Further efforts to improve indicators or develop additional evidence Further efforts to improve indicators or develop additional evidence

Questions?