1 The HEAL NY Evaluation Process: HITEC Rainu Kaushal, MD MPH Chief, Division of Quality and Clinical Informatics Weill Cornell Medical College Executive Director, HITEC Director of Pediatric Quality and Safety Komansky Center for Children’s Health at NYPH

2 NYS and Healthcare Health IT is potentially transformative –Value still largely unproven especially for community based settings utilizing vendor based products NYS is a national leader in investment in health IT –Critical opportunity to study and learn from this investment –Need for common process of evaluation across projects to optimize learning

3 Why Evaluate? Demonstrate economic and quality value of HEAL investments Demonstrate effects on providers and consumers Iteratively refine health IT –Understand what is working well and why –Understand what can be improved and how Disseminate lessons learned and successes broadly

4 What is HITEC? The Health Information Technology Evaluation Collaborative consists of 5 universities –Weill Cornell –Columbia –SUNY Albany –SUNY Buffalo –University of Rochester Established with leadership and support from the New York State Department of Health (NYS DOH)

5 What is HITEC? Steering committee members –Rainu Kaushal, Executive Director, Weill Cornell –Lisa Kern, Deputy Director, Weill Cornell –Russell Bessette, SUNY Buffalo –Rachel Block, NYeC –George Hripcsak, Columbia DBMI –Jean Moore, SUNY Albany –Sandy Schneider, University of Rochester –Bill Schroth, NYS DOH (non voting member)

6 What is HITEC studying? How health IT affects: –Quality –Safety –Economics –Providers –Consumers –Public Health –Health IT & HIE adoption –Policy

7 HITEC HEAL 5 Evaluation Overview

8 What will the evaluation process be? Grantee with HITEC and NYS DOH will identify the research question for their organization HITEC will design evaluation and data collection tools Grantee will perform data collection and provide data in standardized format to HITEC HITEC will perform data analysis and disseminate results

9 Evaluation Activities HITEC, individual grantees, and the NYS DOH OHITT will identify the single most informative evaluation project for the grantee The research question must fall under evaluation categories (economic, quality, safety, provider, consumer, public health) appropriate for the HEAL 5 application category and use case chosen by the grantee

10 Use Cases & Potential Evaluations CategoryUse CasePotential Evaluations I. SHIN-NYConnecting New Yorkers and Clinicians Economic, consumer, provider HIE for Public HealthEconomic, public health Interoperable EHRs for Medicaid Economic, quality, safety Quality Reporting for Outcomes Economic, quality, safety

11 Use Cases & Potential Evaluations CategoryUse CasePotential Evaluations II. CISQuality Reporting for Outcomes Economic, quality, safety Clinical Decision Support in an HIE Environment Economic, quality, safety III. EHRImmunization Reporting with EHRs Economic, public health, provider Quality Reporting for Prevention Economic, quality, safety, provider Interoperable EHRs for Medicaid Economic, quality, safety, provider

12 Evaluation Timeline Time 0Grantee and HITEC sign contracts with NYS DOH Q1-2Grantee, NYS DOH, and HITEC meet to decide on a research question Q2-3HITEC develops an evaluation plan Q3-4HITEC and grantee obtain IRB approval Q5HITEC provides detailed data collection instruments and guidance for data collection

13 Evaluation Timeline (cont) Time Q6-7Grantee performs baseline data collection Q8-9HITEC analyses data and disseminates baseline results Q9-10Grantee performs post data collection Q11-12HITEC analyses data and disseminates post- intervention results

14 HITEC HEAL 5 Evaluation HITEC’s Structure, Designed to Facilitate Evaluations

15 Organizational Structure Health Information Technology Transformation Office at NYS DOH HITEC Steering Committee HITEC Project Management HITEC Evaluation Workgroups Strategic Partner Initiatives NYeC Incentives & Sustainability Consumer Advocacy Council HEAL 5 Grantees HITEC Dissemination Committee HITEC Stakeholder Committee

16 Evaluation Workgroups Six HITEC Evaluation Workgroups – one for each evaluation activity Timelines and deliverables will be proposed by the workgroups and by project management with subsequent approval by the HITEC steering committee.

17 Workgroups 1.The HIT Adoption Measurement Workgroup –Measures rates of adoption of health IT across NYS through a survey and places these findings into a policy context. 2.The Economic Value Workgroup –Quantifies the direct and indirect economic effects of health IT, including the effects of health IT on efficiency. 3.The Quality and Safety Workgroup –Aims to understand the effects of health IT adoption on clinical quality and safety

18 Workgroups (cont.) 4. The Provider Workgroup –Aims to understand providers’ interactions with health IT including effects on workflow, efficiency, satisfaction, and unintended consequences. 5.The Consumer Workgroup –Aims to design and implement survey-based studies to assess the effects of health IT on consumers 6.The Public Health Workgroup –Aims to describe the effects of health IT on public health measures

19 Project Management The HITEC Project Manager with Research Coordinators will manage relationships between HITEC workgroups and HEAL 5 grantees Will have the responsibility of communicating individualized bi-directional timeline and deliverable schedules for each grantee’s evaluations. Will coordinate meetings with HEAL 5 Grantees to explain and receive feedback on methodologies and tools.

20 Stakeholder Committee HITEC and NYeC will collaborate to establish this committee to maximize key stakeholder support for the evaluation process It will include the following representatives: –HEAL 5 Grantee representatives –NYS Office of Health Information Technology Transformation –NYeC –HSP representatives –Representatives from key state program offices including Medicaid and public health –Health plan and vendor representatives

21 Dissemination Committee Three primary goals: –To provide timely, interim feedback directly to HEAL 5 Grantees. –To provide timely, interim feedback to the NYS DOH. –To publish in academic journals and present in national meetings.

22 HITEC HEAL 5 Evaluation Key Grantee Responsibilities and Next Steps

23 Key grantee responsibilities Help choose a research question Designate appropriate staff and time to support the evaluation process –Senior leadership –Programmers –Data collectors Perform data collection as outlined in the evaluation plan Provide timely data in standardized format to HITEC

24 Grantees Should Prepare to Discuss with HITEC Who will be affected by their HEAL 5 project –Which providers, patients –How many providers, patients What exactly their project will entail When they expect implementation to “go live”

25 Grantees Should Prepare to Discuss with HITEC Which use case they would like their evaluation to focus on Which outcome they would like their evaluation to focus on (economic, quality, safety, provider, consumer, public health)

26 The Research Question HITEC will work with the grantee and NYS DOH to design a research question that: –Links the HEAL 5 project with a particular outcome for a particular population of patients or providers –Meets the FINER criteria…

27 The FINER Criteria for Research Questions F Feasible I Interesting N Novel E Ethical R Relevant Hulley SB, Cummings SR, Browner WS, Grady DG, Newman TB. Designing Clinical Research, 3 rd ed. Lippincott Williams & Wilkins: Philadelphia, 2007.

28 Next Steps HITEC will contact each of you to start this process this summer Grantees should prepare for this by collecting answers to the questions posed (who, what, when and which) and by starting to think about appropriate research questions

29 Contact Information Rainu Kaushal, MD, MPH Lisa Kern, MD, MPH Lauren Mercincavage