Institute of Medicine Committee onthe Committee on the National Quality Report on Health Care Delivery Funding: Agency for Healthcare Research and Quality Study Period: October 1999 to March 2001 Margarita Hurtado, Ph.D. American Institutes for Research

Institute of Medicine 2 Committee Members WILLIAM L. ROPER (Chair) Dean, School of Public Health, University of North Carolina at Chapel Hill ARNOLD M. EPSTEIN (Vice Chair) John H. Foster Professor and Chair, Department of Health Policy and Management, Harvard School of Public Health BECKY CHERNEY President and CEO, Central Florida Health Care Coalition DAVID C. CLASSEN Associate Professor of Medicine, University of Utah and Vice President, First Consulting Group JOHN M. COLMERS Program Officer, Milbank Memorial Fund ALAIN ENTHOVEN Marriner S. Eccles Professor of Public and Private Management, Graduate School of Business, Stanford University JOSÉ J. ESCARCE Senior Natural Scientist, The RAND Corporation SHELDON GREENFIELD Director, Primary Care Outcomes Research Institute, New England Medical Center

Institute of Medicine 3 Committee Members (continued) JUDITH HIBBARD Professor, Department of Planning, Public Policy and Management, University of Oregon HAROLD S. LUFT Caldwell B. Esselstyn Professor of Health Policy and Health Economics and Director, Institute for Health Policy Studies, University of California, San Francisco ELIZABETH McGLYNN Senior Researcher, The Rand Corporation (served until July 2000) SCOTT C. RATZAN Senior Technical Advisor and Population Leadership Fellow, Center for Population, Health, and Nutrition, U.S. Agency for International Development MARK D. SM ITH President and CEO, California HealthCare Foundation WILLIAM W. STEAD Associate Vice Chancellor for Health Affairs and Director, Informatics Center, Vanderbilt University Medical Center ALAN M. ZASLAVSKY Associate Professor of Statistics, Department of Health Care Policy, Harvard Medical School

Institute of Medicine 4 Background President’s Advisory Commission (1998)President’s Advisory Commission (1998) Recommends improvement in measurement of quality of care. Healthcare Research and Quality Act of 1999 (PL )Healthcare Research and Quality Act of 1999 (PL ) ANNUAL REPORT ANNUAL REPORT. “Beginning in FY2003,…, shall submit to Congress an annual report on national trends in the quality of health care... ” Agency for Healthcare Research and Quality (AHRQ)Agency for Healthcare Research and Quality (AHRQ) Commissioned IOM study to define a vision for the Quality Report. IOM- Crossing the Quality Chasm Report (2001)IOM- Crossing the Quality Chasm Report (2001) Recommends continued support for quality monitoring and tracking. Need Quality Report to evaluate achievement of aims.

Institute of Medicine 5 Why a National Health Care Quality Report? To inform Congress, the Administration and other policy makers. To identify actionable areas to improve health care quality and monitor the effects of policies. To serve as a barometer of quality & systematically assess progress in meeting specific aims or national goals. To raise public awareness about the state and progress of quality of health care delivery in the country.

Institute of Medicine 6 The Committee’s Task Most important questions need to answer to be able to evaluate the level & change in quality of care Major aspects of quality or categories that should be included Examples of specific measures in each category Recommendations on the production of the report The committee was asked to define:

National National Health Care Health Care Quality Quality Reports Reports National Health Care Quality Framework (Chapter 2) Measure Selection Criteria and Guidelines (Chapter 3) Review of Data Sources (Chapter 4) Categories of Measures Measure Set National Health Care Quality Data Set Audience- Centered Reporting Criteria (Chapter 5) ProcessProduct FROM THE FRAMEWORK TO THE NATIONAL HEALTH CARE QUALITY REPORT(S)

Institute of Medicine 8 Committee and Study Process Committee and Study Process  Committee met four times February to November 2000  Commissioned 4 papers  Workshop, May 22 to 23, 2000  Call for Measures Private Sector, June to July 2000  Report reviewed by external experts

Recommendations

Institute of Medicine 10 Areas Addressed by Recommendations Framework and Categories:Framework and Categories: Recommendation 1 Measure Criteria & Guidelines:Measure Criteria & Guidelines: Recommendations 2 to 6 Data Sources, Data Collection & Analyses:Data Sources, Data Collection & Analyses: Recommendations 7 to 9 Audience-Centered Reporting CriteriaAudience-Centered Reporting Criteria: Recommendation 10

Institute of Medicine 11 Framework and Categories : Recommendation 1 two dimensions The conceptual framework should address two dimensions Components of Health Care QualityComponents of Health Care Quality –Safety –Effectiveness –Patient Centeredness –Timeliness Consumer Perspectives on Health Care NeedsConsumer Perspectives on Health Care Needs –Staying Healthy –Getting Better –Living with Illness or Disability –Coping with the End of Life Equity The conceptual framework should also analyze Equity as an issue that cuts across both dimensions.

Institute of Medicine 12 The Framework as a Matrix Rec. 1

Institute of Medicine 13 Components of Health Care Quality  Safety refers to “avoiding injuries to patients from care that is intended to help them” (Institute of Medicine, 2001).  Effectiveness refers to “providing services based on scientific knowledge to all who could benefit, and refraining from providing services to those not likely to benefit (avoiding overuse and underuse)” (Institute of Medicine, 2001).  Patient centeredness refers to health care that establishes a partnership among practitioners, patients, and their families (when appropriate) to ensure that decisions respect patients’ wants, needs, and preferences and that patients have the education and support they require to make decisions and participate in their own care.  Timeliness refers to obtaining needed care and minimizing unnecessary delays in getting that care. Rec. 1

Institute of Medicine 14 Consumer Perspectives on Health Care Needs Staying healthy refers to getting help to avoid illness and remain well. Getting better refers to getting help to recover from an illness or injury. Living with illness or disability refers to getting help with managing an ongoing, chronic condition or dealing with a disability that affects function. Coping with the end of life refers to getting help to deal with a terminal illness. Rec. 1

Institute of Medicine 15 The Framework as a Matrix Rec. 1

Institute of Medicine 16 Examples of Questions Quality Report Should Address Overall  What do we know about the level of quality of care in the United States? Is quality improving, staying the same, or worsening? Components of Health Care Quality  Is the system providing care safely and decreasing the rate of patient injuries and harm?  Is care patient centered and tailored to the needs, values, and preferences of consumers? (continued)

Institute of Medicine 17 Rec. 1 Questions Quality Report Should Address (continued) Consumer Perspectives on Health Care Needs  How well does the health care system help people to cope with the end of life?  What is the quality of care for people with breast cancer, diabetes, or other specific conditions?Equity  What types of patients or consumers are receiving better quality of care?  Which states or regions of the country provide better care?

Institute of Medicine 18 Measure Criteria & Guidelines: Recommendation 2 General Criteria for Individual MeasuresGeneral Criteria for Individual Measures –Importance –Scientific Soundness –Feasibility Criteria for Measure SetsCriteria for Measure Sets –Balance –Comprehensiveness –Robustness AHRQ should apply criteria to assess the desirable attributes of potential quality measures and measure sets for inclusion in the Quality Report.

Institute of Medicine 19 Criteria for Individual Measures Importance 1. Impact on health 2. Meaningfulness 3. Susceptibility to influence by health care system Scientific Soundness 4. Validity 5. Reliability 6. Explicitness of evidence base Feasibility 7. Existence of prototypes 8. Availability of required data across the system 9. Cost or burden of measurement 10. Capacity of data and measure to support subgroup analyses Rec. 2

Institute of Medicine 20 Criteria for Measure Sets Balance Comprehensiveness Robustness Rec. 2

Institute of Medicine 21 Measure Criteria & Guidelines: Recommendation 3 The Agency for Healthcare Research and Quality should have an ongoing independent committee or advisory body to help assess and guide improvements over time in the National Health Care Quality Report.

Institute of Medicine 22 Measure Criteria & Guidelines: Recommendation 4 The Agency for Healthcare Research and Quality should set the long-term goal of using a comprehensive approach to the assessment and measurement of quality of care as a basis for the National Health Care Quality Data Set.

Institute of Medicine 23 M easure Criteria & Guidelines: Recommendation 5 When possible and appropriate, and to enhance robustness, facilitate detection of trends, and simplify presentation of the measures in the National Health Care Quality Report, AHRQ should consider combining related individual measures into summary measures of specific aspects of quality. AHRQ should also make available to the public information on the individual measures included in any summary measure, as well as the procedures used to construct them.

Institute of Medicine 24 Measure Criteria and Guidelines: Recommendation 6 Data Set should reflect a balance of outcome-validated process measures and condition- or procedure-specific outcome measures. Given the weak links between most structures and outcomes of care and interests of consumers and providers in practice-related aspects as well as outcome measures, structural measures should be avoided.

Institute of Medicine 25 Data Sources, Collection & Analysis: Recommendation 7 AHRQ should assess potential data sources for the National Health Care Quality Data Set according to the following Criteria for Individual Data Sources Credibility and Validity of the Data National Scope and Potential to Provide State-level Detail Availability and Consistency of the Data Over Time and Across Sources Timeliness of the Data Ability to Support Population Subgroup and Condition-specific Analyses Public Accessibility of the Data Criterion for Ensemble of Data Sources Comprehensive

Institute of Medicine 26 Data Sources, Collection & Analysis: Recommendation 8 To obtain the data for the Quality Report, AHRQ needs to In the short term: Draw on a MOSAIC of public and private data sources for the National Health Care Quality Data Set Complement existent data sources by new ones to address all aspects proposed In the medium and long term: Encourage development of a comprehensive health information infrastructure (including standardized, electronic clinical data systems) to support the vision of the data set for the Quality Report

Institute of Medicine 27 Data Sources, Collection & Analysis: Recommendation 9 The data for the National Health Care Quality Report should be nationally representative and, in the long term, reportable at the state level.

Institute of Medicine 28 Designing the Report: Recommendation 10 The National Health Care Quality Report should be Produced in several versions tailored to key audiences –policy makers –consumers –purchasers –providers –researchers Feature limited number of key findings and minimum number of measures needed to support these findings

Institute of Medicine 29 Some Reporting Guidelines  Select 3 to 5 key findings for attention in the report  Make report available in print and on the Web  Use benchmarks or standards for comparisons  Choose findings that have strong statistical evidence Rec. 10

Institute of Medicine 30 Challenges Populating the framework matrix with sufficient measures by selecting from existing ones and defining new ones where needed Establishing a comprehensive quality data set accessible to the public and to researchers Keeping the report focus narrow (3-5 key findings) despite complexity, visibility and importance of the subject Defining useful summary measures where appropriate

Institute of Medicine 31 Conclusions The National Health Care Quality Report fills an important gap. It should serve to guide policy for quality improvement, record progress made, and increase awareness. It can become the most comprehensive means of assessing and tracking the quality of health care delivery in the United States. Despite the challenges, the vision of the National Health Care Quality Report provided by the IOM Committee is achievable. The recommendations set forth in the report should provide appropriate guidance for making this vision a reality.

Report Release AHRQ Briefing –March 22, 2001 Congressional Briefing –March 28, 2001 Public Release –March 30, 2001 Full Report will be available to the public from NAP and on the Web

National National Health Care Health Care Quality Quality Reports Reports National Health Care Quality Framework (Chapter 2) Measure Selection Criteria and Guidelines (Chapter 3) Review of Data Sources (Chapter 4) Categories of Measures Measure Set National Health Care Quality Data Set Audience- Centered Reporting Criteria (Chapter 5) ProcessProduct FROM THE FRAMEWORK TO THE NATIONAL HEALTH CARE QUALITY REPORT(S)

Institute of Medicine 34 Questions Quality Report Should Address Overall  What do we know about the level of quality of care in the United States? Is quality improving, staying the same, or worsening? Components of Health Care Quality  Is the system providing care safely and decreasing the rate of patient injuries and harm?  Is the care provided effective and contributing to desired outcomes?  Is care patient centered and tailored to the needs, values, and preferences of consumers?  Is care provided in a timely manner? (continued)

Institute of Medicine 35 Questions Quality Report Should Address (continued) Consumer Perspectives on Health Care Needs  How well does the health care system help people maintain good health and avoid illness?  How well does the system care for people when they become sick?  How well does the system care for people with chronic conditions or people with disabilities?  How well does the health care system help people to cope with the end of life?  What is the quality of care for people with breast cancer, diabetes, or other specific conditions? (continued) Rec. 1

Institute of Medicine 36 Questions Quality Report Should Address (continued) Equity  What types of patients or consumers are receiving better quality of care? Who is better off? Who is worse off?  Which states or regions of the country provide better care? Are differences in quality over time and between geographic regions getting smaller or larger?  Are there unwarranted differences in the quality of care received by people of different ages, gender, race, ethnicity or other population characteristics? Are the differences increasing or decreasing over time? In which areas of quality of care are the differences the greatest?  What is the quality of care for those without health insurance compared to those with insurance? If there is a difference, is it increasing or decreasing over time? Rec. 1