1. HIT Policy Committee METHODOLOGIC ISSUES Tiger Team Summary Helen Burstin National Quality Forum Jon White Agency for Healthcare Research and Quality October 28, 2010

2. Methodologic Issues Tiger Team Members Helen Burstin (Co-chair) Jonathan White (Co-chair) Mark Weiner Ross Lazarus Bob Dolin Phil Renner Danny Rosenthal Mitra Rocca Abel Kho John Moquin Joanne Cuny Dan Malone Dave Baker 2

3. Methodologic Issues The Methodologic Issues tiger team was tasked to identify the critical methodologic issues that should be considered in the following areas: –Longitudinal measurement - the use of measures that assess patient experience and quality across different settings and providers across time. –Measures that demonstrate change over time (“delta” measures) - the use of measures that assess change in outcomes across time, rather than only achievement of a threshold. –Adverse event reporting systems, with emphasis on medication safety – relates to the use of IT systems for automatic collection and reporting of patient-level adverse events within clinical workflow. –Other issues that emerge from the prioritized measure concepts from the other Tiger Teams. Methodologic issues associated with patient reported outcomes have been addressed by the Patient and Family Engagement Tiger Team 3

4. Longitudinal Measurement: Methodologic Issues Standardized data collection and harmonization of measures across data platforms Need for a shared patient identifier Difficult to attribute results of care to specific providers Reliance on the problem list for assessment of conditions/diagnoses over time Lack of interoperability limits tracking across care settings All measure concepts are not equally amenable to longitudinal data capture- avoidance of “check box” measures Consider other approaches to measurement of key processes: –Structural approaches may provide better information (e.g., exchange of medication lists across providers) –Patient experience may be a more practical and valid approach –Claims data may be a better indicator for some indicators for longitudinal follow-up (e.g., first follow-up after hospitalization) 4

5. Delta Measurement: Methodologic Issues Consider measures most amenable to delta measurement Determine appropriate points in time for baseline and follow-up Consider standards for the degree of change (e.g., % gap to threshold value achieved) Ensure completeness of data across different data systems Account for outcomes that do not have a linear trajectory and may include a lower limit associated with harm Unclear if delta measures present a valid alternative to threshold measures that assess performance Additional testing on delta measures necessary. Do they add value? Difficult to track untoward clinical effects of improving clinical endpoints (e.g., dizziness resultant from blood pressure medications) Accounting for tests not performed for patients regardless of interaction with healthcare system (population health related) Strong interest to capture health status before and after preference- sensitive procedures (e.g., cataracts, back surgery) 5

6. Adverse Event Reporting: Methodologic Issues Numerous barriers to routine medication adverse event reporting include: –Limitations in coding and capture of medication-related adverse events –May be difficult to link event reporting back to specific drugs. Integrate adverse event reporting into clinician workflow, considering time Protect patient privacy when developing an adverse event reporting program Adverse event reporting programs requires data from disparate systems Computer algorithms that analyze data and identify adverse events are not very high-tuned to detect events Level of harm from adverse events is not well documented Measures can target high impact reactions to certain medications (e.g., warfarin) Alert-fatigue is an issue and a medication-related event may be missed if alerts are turned off 6

7. General Methodologic Issues Incremental quality measurement to ensure that the complexity of quality measures is in step with the capacity of available EHRs Integration of standardized EHR context into measurement Utilizing HIEs to capture needed data across providers (efficiency) Interoperable systems that can assure that original results are available (efficiency) Patient data accurately tracked across different providers and data platforms (population health) Standardizing the strata for race, ethnicity, language to allow for measure stratification (equity) Emerging technology that addresses health literacy (e.g., talking touch screens) (patient-reported outcomes) Inconsistencies in coding between types of providers (e.g., primary care and psychiatrists) 7

8. Methodologic Issues/Recommendations Problem List Accuracy of coding is highly variable depending on type of diagnosis Need for standards for coding problem list – e.g., active/inactive designation of a problem; date of onset for a given disease process Need for clinicians to consider the problem list as a quality measurement reporting tool Need conventions for how to deal with problems that have resolved and been deleted from problem list Need to establish rules for consistent use of problem list and past medical history Need guidance for proper use of problem list for reporting of new conditions 8