1. New and Improved: Registries for Evaluating Patient Outcomes and HIT
Elise Berliner, Ph.D.
Director, Technology Assessment Program
Center for Outcomes and Evidence

2. Agenda Introduction to the Second Edition
Elise Berliner, Ph.D., Task Order Officer
Review of major changes and new sections
Richard Gliklich, M.D., Senior Editor
Nancy Dreyer, Ph.D., M.P.H., Senior Editor
Planning for the Third Edition
Audience discussion, led by Dr. Berliner

3. Background
First edition, published in 2007, has been widely used as a reference for designing, operating, and evaluating patient registries.
Numerous citations (>60) in literature and in significant government publications (e.g., Federal Register, FCC Report to the President, etc.)
As registries continue to evolve, many new methodological and practical issues have arisen.

4. Purpose of the Second Edition
Update and expand the first edition of the User’s Guide with new information gathered from recent publications and reported experiences encountered by researchers and other professionals who utilize registries for research.
Expound on selected topics in the original guide and add new topics that deserve further in-depth discussion.

5. Process for Creating the Guide
Topics identified based on public comments received for the first edition.
New sections:
Author and reviewer teams assembled with similar balance to first edition.
Posted for public comment and revision.
Original chapters:
Original authors and reviewers were invited to participate. Additions made for new topic areas when necessary.
Full, revised handbook posted for public comment.
Open call for case examples.

6. Second Edition Collaborators
55 contributors from industry, academia, health plans, physician societies, and government.
49 invited peer reviewers, plus public comment.
38 case studies illustrate challenges and solutions (including 20 new examples).
Senior Editors: R. Gliklich; N. Dreyer. Managing Editor, M. Leavy. Outcome DEcIDE Center.

7. Second Edition: Table of Contents
1. Patient Registries (Overview)
2. Planning a Registry
3. Registry Design (includes Planning for the End of a Patient Registry)
4. Use of Registries in Product Safety Assessment
5. Data Elements for Registries
6. Data Sources for Registries
7. Linking Registry Data: Technical and Legal Considerations
8. Principles of Registry Ethics, Data Ownership, and Privacy
9. Recruiting and Retaining Participants in the Registry
10. Data Collection and Quality Assurance
11. Interfacing Registries With Electronic Health Records
12. Adverse Event Detection, Processing, and Reporting
13. Analysis and Interpretation of Registry Data To Evaluate Outcomes
14. Assessing Quality

8. Registries for Evaluating Patient Outcomes: A User’s Guide Second Edition
Richard Gliklich, M.D., Senior Editor
Nancy Dreyer, Ph.D., M.P.H., Senior Editor

9. Review of Major Changes
Changes to Sections on Creating and Operating Registries:
All chapters were updated. Some chapters had more significant additions.
Multiple new case studies.
Review of “Linking Registry Data.”
Review of “Interfacing Registries with EHRs.”
Highlighted Case Examples.

10. Updates to 1st Edition Chapters
“Planning a Registry” (Chapter 2) now discusses public-private partnerships.
“Registry Design” (Chapter 3) and “Analysis and Interpretation” (Chapter 13) were reorganized to align more closely.
“Data Elements” (Chapter 5) was expanded to discuss new data standards.

11. Updates to 1st Edition Chapters
“Principles of Registry Ethics, Data Ownership, and Privacy” (Chapter 8) addresses new legislation:
Patient Safety and Quality Improvement Act of (PSQIA)
Genetic Information Nondiscrimination Act of 2008 (GINA)
Health Information Technology for Economic and Clinical Health Act (HITECH Act)
“Adverse Event Detection, Processing, and Reporting” (Chapter 12) discusses risk evaluation and mitigation strategies (REMs).

12. Case Examples (new examples highlighted)
Identifying and Responding to Adverse Events Found in a Registry Database
Selecting Data Elements for a Registry

13. Case Examples (new examples highlighted)
Using Registries To Understand Rare Diseases
Creating a Registry To Fulfill Multiple Purposes and Using a Publications Committee To Review Data Requests
Using a Registry To Track Emerging Infectious Diseases
Using a Collaborative Approach To Plan and Implement a Registry
Using a Scientific Advisory Board To Support Investigator Research Projects
Determining When To Stop an Open-Ended Registry
Designing a Registry for a Health Technology Assessment
Assessing the Safety of Products Used During Pregnancy
Designing a Registry To Study Outcomes
Analyzing Clinical Effectiveness and Comparative Effectiveness in an Observational Study
Using a Registry To Assess Long-Term Product Safety
Using a Registry To Monitor Long-Term Product Safety
Using Performance Measures To Develop a Dataset
Developing and Validating a Patient-Administered Questionnaire
Understanding the Needs and Goals of Registry Participants
Using Validated Measures To Collect Patient-Reported Outcomes
Integrating Data From Multiple Sources With Patient ID Matching
Linking Registries at the International Level
Linking a Procedure-Based Registry With Claims Data To Study Long-Term Outcomes
Linking Registry Data To Examine Long-Term Survival
Considering the Institutional Review Board Process During Registry Design
Issues With Obtaining Informed Consent
Building Value as a Means To Recruit Hospitals

14. Case Examples (new examples highlighted)
Using Registry Tools To Recruit Sites
Using Proactive Awareness Activities To Recruit Patients for a Pregnancy Exposure Registry
Using Reimbursement as an Incentive for Participation
Data Collection Challenges in Rare Disease Registries
Managing Care and Quality Improvement for Chronic Diseases
Developing a Performance-Linked Access System
Using Audits To Monitor Data Quality
Challenges in Creating Electronic Interfaces Between Registries and Electronic Health Records
Creating a Registry Interface To Incorporate Data From Multiple Electronic Health Records
Technical and Security Issues in Creating a Health Information Exchange
Developing a New Model for Gathering and Reporting Adverse Events
Using Registry Data To Evaluate Outcomes by Practice
Using Registry Data To Study Patterns of Use and Outcomes

15. New Chapter: Linking Registry Data
Rationale: Increasingly, statistical methods are used to link data from multiple de-identified sources, including registries. For these projects:
What is the risk of identifying patients by combining data from multiple registries?
What are the legal and ethical requirements for researchers to ensure patient privacy and confidentiality?
Linking Registry Data: Overview
Chapter is divided into 2 sections: Technical Aspects and Legal Considerations.
Technical Aspects: What is a feasible technical approach to linking the data?
Legal Considerations: Is the linkage legally feasible under the permissions, terms, and conditions that applied to the original compilations of each data set?

16. 16

19. Linking Registry Data: Case Example
Goal: Study long-term patient outcomes for diagnostic cardiac catheterizations and percutaneous coronary interventions (PCI).
Data Sources: CathPCI registry lacks long-term follow- up. Medicare data lacks detailed procedure information.
Legal Approach: After review of HIPAA, Common Rule, project team determined that linkage required use of limited datasets only and IRB approval for linkage project.
Technical Approach: CathPCI data were linked with Medicare data using probabilistic matching techniques.

20. New Chapter: Interfacing Registries with Electronic Health Records (EHRs) 
Rationale: With national efforts to invest in EHRs, and to advance the evidence base in areas such as effectiveness, safety, and quality through registries and other studies, it is clear that interfacing registries with EHRs will become increasingly important to reduce data collection burden. This chapter explores issues of interoperability and a pragmatic “building-block approach” towards a functional, open-standards based solution.

21. Interfacing Registries & EHRs: Overview
4/21/2017
Interfacing Registries & EHRs: Overview
Chapter covers 4 main topics:
Role of EHRs and patient registries in health care.
Vision of EHR-registry interoperability.
Interoperability challenges.
Partial and potential solutions.

22. Interfacing Registries & EHRs: Definitions
4/21/2017
Interfacing Registries & EHRs: Definitions
EHR: an electronic record of health-related information on an individual that conforms to nationally recognized interoperability standards and that can be created, managed, and consulted by authorized clinicians and staff, across more than one health care organization.
Individual focused
Patient registry: an organized system that uses observational study methods to collect uniform data (clinical and other) to evaluate specified outcomes for a population defined by a particular disease, condition, or exposure, and that serves one or more predetermined scientific, clinical or policy purposes.
Population focused

23. Interoperability Challenges
Syntactic interoperability (communication): the ability of heterogeneous health information systems to exchange data.
Wiring, application protocol, standard messaging protocol
Most easily solved
Semantic interoperability (content): implies that the systems understand the data that has been exchanged at the level of defined domain concepts.
Depends on standard vocabulary and shared data elements
More difficult to solve; efforts include CDASH, ASTM Continuity of Care Record (CCR), HL7 Continuity of Care Document
Other issues: managing patient identifiers and authenticating users across multiple applications.

24. Partial and Potential Solutions
We lack a single, complete interoperability model, but, several open standard components provide a reasonably good start which can be improved with further development, testing, and adoption of additional open standard building blocks.
Signing, Privacy, Encryption
Data standards (e.g. HL7 and CDISC)
Content profile (e.g. CRD)
Integration profile (e.g. RFD)
Web services: http(s), web browsers
Physical network connection

25. Building-Block Approach: Retrieve Form for Data Capture/HITSP-TP50

26. Summary
EHR–registry interoperability will be increasingly important as adoption of EHRs and patient registries increases significantly.
Interoperability requires accurate and consistent data exchange and use of the information that has been exchanged.
While a complete interoperability solution does not yet exist, enough open standard building blocks do exist today to enable ‘functional interoperability’ which while imperfect, significantly improves workflow and reduces duplication of effort.

27. Interfacing Registries & EHRs: Case Examples
A registry focused on effectiveness in pain management was made interoperable with a commercial EHR using RFD communication.
ASTER project: interoperability was achieved for the purpose of reporting adverse event information to the FDA.
A commercial EHR was made interoperable with a quality reporting initiative for the American College of Rheumatology and with a Physician Quality Reporting Initiative Registry for reporting data to the Centers for Medicare and Medicaid Services.

28. Review of Major Changes
Review of “Planning for the End of a Registry.”
Review of “Use of Registries for Product Safety Assessment.”
Highlighted Case Examples.
Changes to Section III, Evaluating Registries.

29. New Section: When to End a Registry
Rationale: Registries may be developed without specific endpoints, making it difficult to determine when to end data collection. A more clear explanation of recommendations for stopping a particular registry is useful from several perspectives, including costs. This section addresses how to determine when a registry has met its objectives and should be closed.

30. Stopping Decisions and Registry Goals
Registry protocol should include specific and measurable endpoints against which to judge whether the project should continue or stop.
Measurable goals for endpoints will make it possible to determine whether the registry has achieved a core purpose and may lead to a reasonable stopping point.
Conversely, a registry that fails to meet measurable goals and appears to be unable to meet them in a reasonable time is also a candidate to be stopped.
Other reasons for stopping:
Incomplete or poor quality data.
Registry has outlived the question it was created to answer.
Funding and/or staffing is no longer available.

31. What Happens When a Registry Stops?
Stopping a registry could mean:
ceasing all data collection and issuing a final report.
ceasing to accrue new patients while continuing to collect data on existing patients.
Regardless, archiving rules should be checked and followed.
No clear ethical obligation to participants to continue a registry that has outlived its scientific usefulness.

32. What Happens When a Registry Stops?
Registries with one sponsor who has decided to stop the registry should be encouraged to engage other stakeholders in discussions of potential transitioning of the registry to other owners.
Issues of data ownership, property, confidentiality, and patient privacy would need to be satisfactorily addressed to make such transitions possible.
Reasons to consider preserving registry data:
If data may be capable of producing a recognized public health benefit that will continue if the registry does.
If registry has historical importance, such as a registry that tracks the outbreak of a novel infectious disease that may provide insight into the transmission of the disease if not now, then sometime in the future.
If longitudinal data may be useful for hypothesis generation.

33. Ending a Registry: Case Example
Bupropion Pregnancy Registry:
Open-ended, observational exposure-registration and followup study to monitor prenatal exposure to bupropion and detect any major teratogenic effect.
Registry collected data on over 1,500 exposed pregnant women over 10 years.
In 2007, advisory committee recommended discontinuation of the registry, based on its conclusion that sufficient information had accumulated to meet the scientific objective of the registry.

34. New Chapter: Use of Registries for Product Safety Assessments
Rationale: Registry populations often include patients that are significantly different from patients studied in clinical trials (age, comorbidities, etc.) and therefore are of particular interest in post- approval safety monitoring.
How should data be monitored for adverse events?
How is the “expected” rate calculated?
How often should the data be analyzed?
What is the legal and ethical responsibility of the registry for reporting, informing physicians and institutions of potential problems?

35. Use of Registries for Product Safety Assessments: Overview
Chapter divided into 4 sections:
Registries specifically designed for safety assessment.
Registries designed for purposes other than safety.
Signal detection in registries.
Potential obligations for registry developers in reporting safety issues.

36. Registries Designed for Safety Assessments
Registries are well suited to identify effects that can only be observed in a large and diverse population over an extended period of time (drug-drug interactions, genetic differences in drug metabolism, etc.).
When designing a registry for the purposes of safety, the size of the registry, the enrolled population, and duration of follow-up are all critical to ensure validity of the inferences made based on the data collected.
Some other challenges:
Recruiting a meaningful patient population.
Evaluating the utility of a registry when the entire population-at-risk has not been included (a common scenario.)
Understanding timing of treatments, switching of treatments, multiple therapies, dose effects, delayed effects, and patient compliance.

37. Registries Designed for Purposes Other than Safety
Registries examining comparative effectiveness, the natural history of a disease, evidence in support of national coverage decisions, or quality improvement efforts may gather and report AE data, but may not be able to reliably detect all events.
Facilitate safety reporting rather than evaluate it.

38. Use of Registries for Product Safety Assessment: Case Example
British Society for Rheumatology Biologics Register (BSRBR):
Prospective observational study to monitor the routine clinical use and long-term safety of biologics in patients with severe rheumatoid arthritis and other rheumatic conditions.
Registry enrolls patients on anti-TNF agents and a control cohort of patients on DMARDs.
Data from registry have been used to determine whether an increased risk of tuberculosis existed in patients treated with anti-TNF therapy. Analysis found a differential risk among the three anti-TNF agents (Dixon et al., Annal Rheum Dis 2010).

39. Evaluating Registries
Tables in this chapter were re-organized to group the good practices by concept.
Research Quality: Basic Elements and Potential Enhancements of Good Practice for Establishing and Operating Registries
Evidence Quality: Indicators of Good Evidence Quality and Indicators of Enhanced Good Evidence Quality for Registries
Chapter was updated to include good practices related to the new handbook sections.

44. Plan for the Third Edition
Development begins in Fall 2010.
Plan is to include 11 new chapters, plus case examples.
7 topics identified through public comments on second edition.

45. Topics for the Third Edition
Patient identity management
Protection of data from litigation
Data protection concerns
Public-private partnerships
Statistical techniques for analyzing combined data
Pregnancy registries
Registry transitions

46. Additional Topics
For discussion: What other topics should be addressed in the third edition?

47. Contact
Elise Berliner