Risk-Based Monitoring

Risk-Based Monitoring
Overview November 2016

TransCelerate Legal Disclaimer
These materials are intended to facilitate and reduce the burden on clinical trial sponsors and others in training personnel with regarding to risk-based monitoring methodologies. Each clinical trial sponsor or other company engaging in such training activities bears full responsibility for its own training and accompanying materials to ensure both the accuracy of the training and materials and compliance with all applicable local, state, and national laws and regulations. This training is not intended to replace any in-depth training that clinical trial sponsors or others may wish or need to provide to their personnel or investigator sites to educate them on required or desirable clinical trial monitoring methodologies. By using these training materials, you signify your assent to the below terms of use. If you do not agree to them, you are not authorized to copy, distribute, reproduce, or use these materials and should not do so. Disclaimer of Liability TransCelerate, its staff, or its Member Companies shall not be held liable for any improper or incorrect use of these training materials and assumes no responsibility for any user's use of them. In no event shall TransCelerate, its staff, or its Member Companies be liable for any direct, indirect, incidental, special, exemplary, or consequential damages (including, but not limited to: procurement or substitute goods for services; loss of use, data, or profits; or business interruption) however caused and on any theory of liability, whether in contract, strict liability, or tort (including negligence or otherwise) arising in any way out of the use of these training materials, even if advised of the possibility of such damage. This disclaimer of liability applies to any damages or injury, whether for tortious behavior, negligence, or under any other cause of action. Disclaimer of Warranties/Accuracy and Use of Information Material in the training materials may include technical inaccuracies or typographical errors. Changes may be periodically incorporated into this material. TransCelerate may make improvements and/or changes in the products, services and/or job aids described in these materials at any time without notice. These training materials are provided 'AS IS' WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT. Some jurisdictions do not allow the exclusion of implied warranties, so the above exclusion may not apply to you. Neither TransCelerate, its staff, or its Member Companies warrants or makes any representations regarding the use or the results of the use of the materials or information in this site, or the accuracy, adequacy, completeness, legality, reliability, or usefulness of the training materials.

What Is Risk Based Monitoring (RBM)?
10/8/2017 What Is Risk Based Monitoring (RBM)? An adaptive approach to clinical trial monitoring that directs monitoring focus and activities to the evolving areas of greatest need which have the most potential to impact patient safety and data quality.

Content Introduction to Risk-Based Monitoring (RBM) Methodology
Risk Assessment Risk Management Implementing an RBM Program 1 2 3 4

Section 1 Objectives At the conclusion of this section, attendees will be able to… Explain the rationale for Risk-Based Monitoring (RBM) Describe TransCelerate RBM and the RBM key assumptions Describe the RBM Methodology as compared to traditional monitoring methods

Section 1: Objective 1 Rationale for RBM

Industry Drivers to Change Traditional Monitoring Approach
Various Reasons for Change Adapt to Needs Cost- Benefit Ratio Complex Protocols Risk Mitigation New Technology Regulatory Shift Smarter Resource Allocation

Regulatory Agencies - Leading the Movement
2011 EMA Draft Reflection Paper & FDA Draft Guidance on RBM (issued August 2011) 1988 FDA Guidance – Monitoring of Clinical Investigations (retired 2010) 1998 FDA Guidance provides standards for minimal on-site monitoring 2009 CTTI focuses on clinical trial monitoring efficiency and effectiveness 2013 Final FDA Guidance and EMA Paper on RBM (issued August 2013) Formation of TransCelerate 1988 1996 1998 2007 2009 2010 2011 2013 2014 1996 ICH E6 provides flexibility in how trials are monitored 2007 Janet Woodcock, FDA, introduces risk-based approach concepts in clinical research FDA sponsor warning letters citing “inadequate monitoring” 2013 FDA supports TransCelerate with review of pilot RBM plans

RBM Industry Movement CTTI FDA Guidance EMA Reflections Paper
TransCelerate Paper Quality by Design Tailor monitoring approach Protocol quality impacts monitoring quality Quality Clinical Trial Data Assess Risk Combination of monitoring activities Tailor Monitoring Plan Risk Based Quality Management Plan Adapt Build on experience and advances RBM Methodology Holistic, proactive approach; risk assessment, mixture of remote & on-site monitoring

Transcelerate OVERVIEW and Key RBM assumptions
Section 1: Objective 2 Transcelerate OVERVIEW and Key RBM assumptions

TransCelerate and Flexible Implementation
As a reminder: TransCelerate members may choose whether to implement TransCelerate generated solutions If they choose to implement, they are free to do so however they see fit Best practices & change management discussions are intended merely to facilitate assessment of tools in order to improve them flex·i·ble ˈfleksəbəl/ adjective adjective: flexible 1. capable of bending easily without breaking. "flexible rubber seals" synonyms:pliable, supple, bendable, pliant, plastic; More

TransCelerate Today – 18+ Organizations and 14 Active Initiatives
2014 2013 18 Member Companies 15 Active Initiatives 2 Realized Initiatives 17 Member Companies 12 Active Initiatives 1 Exploratory Initiative 2012 17 Member Companies 5 Active Initiatives 1 exploratory Initiative 10 Founding Members 5 Active Initiatives

Engage with Key Stakeholders
We also recognized the need to collaborate not only across participating Member Companies, but also regulatory authorities and industry groups. Research and CRO Community Industry Initiatives Investigator Sites Regulatory Authorities

TransCelerate RBM Output
Delete this slide?

TransCelerate RBM Publications
Delete this slide?

TransCelerate Position Paper: Risk-Based Monitoring Methodology
Promoting risk mitigation and early issue detection Leveraging risk-based approaches & technology advances Taking a holistic, proactive approach through Off-site & Central Monitoring Implementing targeted On-site Monitoring TransCelerate BioPharma Inc. developed a methodology that shifts monitoring processes from an excessive concentration on Source Data Verification to comprehensive risk-driven monitoring. The TransCelerate RBM team started from an understanding that by building quality and risk management approaches into the scientific design and operational conduct of clinical trials, risks can be mitigated and issues can be detected early or prevented entirely. Additionally, the TransCelerate partners recognize that although current on-site monitoring practices do provide a level of control, advances in risk-based approaches and technology provide an opportunity for a more holistic and proactive approach. This philosophical shift in monitoring processes employs Centralized and Off-site mechanisms to holistically monitor important study parameters and uses adaptive On-site Monitoring to further support site processes, subject safety, and data quality. Through modernization, including use of technology enablers, efficiencies can be gained without impacting subject safety by implementing quality risk management approaches to clinical trial oversight.

TransCelerate RBM Methodology: Assumptions
Central and off-site monitoring are the foundation Monitoring activities are adapted based on issues/risks Tailor methodology to available technology Timely data entry and query resolution are critical Functional oversight and documents should respond to changes/risks RBM expectations can be formalized in SOPs Methodology applies to all types and phases of trials Communication plans should be tailored for efficiency Risk assessments should take place prior to protocol/CRF finalization

Quality by Design (QbD) Concepts
Program and Protocol Development Study Execution QbD RBM Build Quality into the scientific and operational design and conduct of clinical trials to: Effectively & efficiently answer intended questions about benefits/risks Identify fit for purpose data Focus on key risks to Patient safety & data integrity Design efficient MP to rapidly detect/correct issues Identify risks at the program, study, & site level to apply the appropriate level of monitoring through: Mapping the risks to appropriate monitoring plans Employing mechanisms to monitor important parameters (Central & Off-site monitoring activities) Smarter use of Technologies that enable effective oversight Targeted on-site interventions

TransCelerate RBM Methodology
Monitoring Plans Building QbD into design & planning of trial Conducting early and ongoing risk assessments Focusing on Critical Processes and Data Using Risk Indicators, Thresholds & Action Plans Adjusting monitoring activities based on risks Critical Variables RACT Focus on what matters Identify the risks; mitigate as possible Employ best mechanisms to monitor the remaining risks Target interventions based on identified quality issues

RBM Methodology as compared to traditional monitoring methods
Section 1: Objective 3 RBM Methodology as compared to traditional monitoring methods

Types of Monitoring Term “Monitoring” is used in different ways in the clinical trial context Site Monitoring Safety Monitoring Medical Monitoring Quality Control monitoring by Sponsor and CRO internal processes and systems Quality Control mechanisms at site

Value Proposition / Benefits When Delivered
Model Approach for High-Quality, Risk-Based Monitoring Unmet Need Sites have varying levels of experience and quality, but monitoring approaches are not designed to manage differences No model approach or best practices existed for risk-based monitoring Expectations when outsourcing are 100% SDV Research indicates that 100% SDV is not effective at identifying material risk; however, monitoring approaches remained unchanged Value Proposition Improved ability to ensure subject safety, data integrity & GCP compliance Allocates resources more efficiently Continuous improvement opportunities enabled by an approach that can be measured & refined over time Increase site personnel time to focus on their core job functions Increase inspection-readiness, audit and Inspection findings minimized with focus on what matters

Section 1 Summary Rationale for RBM is driven by industry, regulatory, risk & technology changes TransCelerate’s founding principles & key assumptions include a proactive quality by design approach to assess, mitigate, and manage risks RBM is intended to improve upon the “traditional” monitoring model

Section 2 Objectives At the conclusion of this section, attendees will be able to… Identify Critical Data and Processes for Risk-Based Monitoring (RBM) application Identify protocol risks

Identifying critical data/Processes for RBM application
Section 2: Objective 1 Identifying critical data/Processes for RBM application

Critical Data/Processes – Possible Questions
What are the data and/or processes which are critical to program and/or protocol success? What Critical Data must be collected in order to satisfy the objectives? What are the Critical Processes that must be done correctly to ensure subject safety, data quality, and/or GCP and regulatory compliance? Are there any Critical Processes in the program and/or protocol which are vulnerable to error?

Critical Data and Processes
Critical Processes Data that support primary & key secondary objectives Rationale: why is it critical? Endpoint - primary or secondary Safety - SAEs, events leading to discontinuation of treatment Other (specify) Processes that underpin safety or quality Rationale: why is it critical? Safety/ethical treatment - seeking appropriate medical consultation, investigating clinically significant findings Data quality – blinding, event adjudication, controlling inter- rater variability Compliance – GCP, local regulations, protocol Example: 1) AEs/SAEs 2) HbA1C Example: 1) Collection and reporting of AEs/SAEs 2) Collection, storage, shipment of labs

Examples of Critical Data/Critical Processes*
Therapeutic Area Endpoint Critical Data Critical Processes Metabolic Change in baseline HbA1C at Week X HbA1C Lab collection, storage, shipment Cardiovascular Number of CV events between treatment groups per adjudication committee AEs of stroke, MI, death AE collection & reporting Collection and submission of source info. to adjudication committee Oncology Progression Free based on RECIST 1.1 criteria per Independent radiology review committee (IRRC) Tumor measurements Performance of diagnostic tests Following RECIST 1.1 criteria Submission of scans and information to review committees All Safety AEs/SAEs AE collection and reporting *note these are only examples and may not be applicable to all studies in a therapeutic area

Conducting risk Assessments
Section 2: Objective 2 Conducting risk Assessments

Background Risk is defined as the combination of the probability of occurrence of harm, the severity of that harm and how easily that harm can be detected 4 major questions: What might go wrong? What is the likelihood it will go wrong? What are the consequences? How easy it to detect?

Risk Assessment and Categorization Tool (RACT)
Documentation of Critical Data & Processes Team alignment on “what matters” Focused Risk Assessment Consistent Approach to Assessment Each category** has a series of questions Examples of high, medium, & low risk Impact & Probability Determines the impact to the study if the risk becomes an issue Determines the probability of the risk occurring Output Provides direction on where the most attention is needed for monitoring and mitigation Determines the Overall Risk Level (high, medium, or low) for monitoring activities **Categories** Protocol Level RACT Program Level Safety Endpoints IP Technology Operational Experience Study Phase Subject Population Data Collection & CRF Source Study Medication /IP Blinding Clinical Supply Chain Protocol Complexity Operational Complexity Geography

RACT Completion Logistics
When? During study planning, before the protocol is finalized Who? A cross-functional group involving various roles and team members (e.g. Data Managers, Monitors, Clinical Scientists)

Identifying Risks Program Level Protocol/Trial Level
new/unique tools or procedures associated with the program? specific safety requirements or adverse events of special interest? any risks unique for the product such as storage requirements? risks inherent to the indication and/or therapeutic area? risks from a regulatory perspective? Do the new/unique tools and procedures differ from standard of care? Are there safety considerations as a result of comparator drugs or the indication? Any competing studies for the study population? Are any inclusion/exclusion criteria open to interpretation or unclear for study staff? Does the study complexity increase risks?

Risks vs Issues Risk Issue
What is the difference between a risk and an issue? Risk Issue Something that might happen in the future. Something that is currently happening (or already happened). Example: The Study Coordinator may resign during the study. The Study Coordinator resigned, effective immediately. No replacement was assigned yet.

Evaluating Risks High Risk Medium Risk Low Risk
Risk Level is determined by: Impact Probability Detectability The combined assessment of all of the components determines the risk level. High Risk Medium Risk Low Risk

TransCelerate RACT snap-shot

Section 2 Summary Risk assessment focuses on risks, Critical Data and Processes identified at the program, protocol and other risks (e.g., region/country/site) Early and proactive risk identification and assessment is a core activity of the RBM methodology

Section 3 Objectives At the conclusion of this section, attendees will be able to… Discuss development of risk mitigation plans including Risk Indicators and Thresholds in decision-making Document risk mitigations in function plans Describe how to conduct monitoring activities in the RBM model – including central, off-site & on-site monitoring

developing Risk Management Plans
Section 3: Objective 1 developing Risk Management Plans

Risk Management Planning
How do we determine our risks? RACT What helps us detect risk? Risk Indicators How will we know to take action? Threshold Breaches How will we respond? Planned Actions

Risk Indicators & Thresholds
Risk indicators are metrics used to monitor identified risk exposures over time Threshold A pre-determined level, point, or value (e.g., number, %, range) associated with a Risk Indicator that indicates the need for a follow-up action

Risk Indicator Categories and Examples
Outliers/trends in number of adverse events per subject visit/site Safety Incidence of temperature excursions Investigational Product Number of screen failures compared to average across sites Recruitment/Discontinuation Number of deviations per subject visit/site compared to average across sites Issue Management Abnormal trend or lack of variability in data Data Quality Amount of data outstanding for verification or review On-Site Workload Number of overdue or missing documents Essential Documents Staff turn-over Staffing, Facilities, Supplies

Different Threshold levels may be set
Drives different response levels A visual, color-coded system may be used Threshold Levels High Medium Low Warning range Awareness range Acceptable range

Actions: Examples of Responses to Thresholds
Possibly no action needed beyond ongoing monitoring Continue central and/or off-site monitoring Assess other data remotely Contact site to get additional information Contact site to get additional information Collect site documentation Visit site to review documentation not available remotely

TransCelerate RBM Toolkit: Companion Guide to Risk Indicators

Example Risk Indicator: Site has incorrectly administered IP more than expected. Threshold: Rate of incorrectly administered IP at a site is > 2 standard deviations more than the average rate across sites. Risk Detection: Central monitoring analytic to flag outlying sites. Risk Mitigation: Monitor to retrain the PI and/or delegate off-site on the importance of IP compliance to subject safety and data integrity. SDR the process for administering IP at next site visit. Other pre-defined mitigation efforts: Edit check to query where data is missing. SDV a sample of the subjects based on defined risk level.

Planning risk mitigations
Section 3: Objective 2 Planning risk mitigations

What is a Risk Mitigation Plan?
Documented plan, typically in functional plans Assigns responsibility Actions taken to prevent or decrease the probability of a risk becoming an issue

Integrated Quality Risk Management Plan (IQRMP)
Data Plans Training Plan Monitoring Plan Statistical Analysis Plan Safety Plan Medical Monitoring Plan Quality Plan Other Functional Plans IQRMP RACT Critical Data & Processes Risk Indicators / Thresholds

Section 3: Objective 3 Monitoring activities in the RBM model including: Central, Off-Site, & on-site monitoring

Understanding the Change: Operating Differently
From A Better Way Sampling or 100% SDV RBM Integrated, Formal Risk Assessment Targeted and Focused SDV Emphasis on Critical Data & Processes at Site Enable Central Monitoring Capability Routine monitoring activities: essential doc review, site qualification/training, adequate facilities, supplies

Understanding the Change: Current Site Monitoring  One Size Fits All
On-site Monitoring visits scheduled at regular intervals: Study Complexity What about… Study Duration / Phases Site Risk Level (site experience, site issues) SIV MV #1 MV #2 MV #3 MV #4 MV #5 MV #6 MV #7 COV 1st subject enrolled

Understanding the Change RBM – A Better Way
Remote Monitoring Between Visits – Site Interactions SIV MV #1 MV #2 MV #3 Triggered MV #4 COV 1st subject enrolled MV #1 MV #2 MV #3 MV #4 MV #5 MV #6 MV #7 COV 1st subject enrolled

RBM Monitoring Execution
Central Monitoring Remote assessment: study-level Risk Indicators Off-Site Monitoring Remote assessment: site-level Issues, data, site performance On-Site Monitoring On-site assessment: site quality via SDR/SDV, and investigational site GCP assessment Medical Review Safety Review Data Management Review Statistical Review Comprehensive approach to monitoring Fit for purpose

RBM: Central Monitoring
Remote evaluation carried out by sponsor/CRO personnel at a location other than the investigative site to: Use analytics to target on-site/off-site monitoring activities on areas of most need Identify higher risk sites to target additional monitoring Ensure routine review of data in near real time Identify unusual distribution of data earlier: trends & outliers

RBM: Off-Site Monitoring
Confirm timeliness and quality of data entry Review query resolution Review CRF to check protocol compliance Confirm site’s completion of previously identified actions Assess site’s recruitment and enrollment Monitor for changes in site staff Monitor delegation of responsibilities Conduct training

Traditional Approach: On-Site Monitoring
In person evaluation carried out by sponsor/CRO personnel at the investigative site location to: Identify missing data in source records and data entry errors in CRFs Assess compliance with protocol and investigational product accountability Evaluate Investigator supervision

Source Data Review (SDR)
Reviewing source documents for important areas where there is no associated CRF data field Monitoring the site’s Critical Processes Not a two-way review of Source to CRF Amount of review varies by risk

Source Data Verification (SDV)
Transcription Check Two-way check (Source to CRF and CRF to Source) Done on Critical Data only Amount varies by risk

Evaluating SDV as a Quality Control Measure in Clinical Trials
Purpose: Conduct a retrospective analysis of a large data set to assess the value of SDV as a quality measure Understand the optimal use of SDV Test the hypothesis that SDV does not have a significant effect on data quality

Challenging Value of SDV
Analyses Findings Literature Review 40 SDV-related articles reported in tabular format Retrospective Analysis % of Total eCRF Dataset Corrected by SDV – 1.09% (Median) % of Corrected eCRF Data Corrected by SDV Overall data corrections, 3.7% Of all data corrected (3.7%), 32.0% (median) attributable to SDV Any AEs or SAEs recorded within 1 or 7 days following on-site monitoring % of AEs Entered after SDV Median values are 7.5% (<=1 day) and 11.8% (<=7 days) % of SAEs Entered after SDV Median values of 1.7% (<=1 day) and 3.6% (<=7 days)

Retrospective SDV Analysis*: Impact of SDV on Site-Entered Data
1.09% of All Site-Entered Data is Corrected By SDV 96.3% of data is never corrected after entry to database (Insights Data warehouse) DM, Science Queries etc. System Queries * Evaluating Source Data Verification as a Quality Control Measure in Clinical Trials: Therapeutic Innovation & Regulatory Science 2014, Vol. 48(6)

Why Distinguish SDV and SDR?
Address different risks Answer different questions Use according to needs SDR and/or SDV can be temporarily increased or decreased depending on the type of issues and risks noted at the site, country/region, or study level

Section 3 Summary Risk Indicators and Thresholds Risk mitigations documented in functional plans Monitoring activities in the RBM model including: central, off-site and on-site monitoring

Section 4 Objective At the conclusion of this section, attendees will be able to… Discuss change management considerations with implementation within an organization and at sites.

RBM implementation: Change Management considerations
Section 4: Objective 1 RBM implementation: Change Management considerations

Change Management Considerations
What is the change? How big is the change and what does it look like? Where is the support for the change? Where are the roadblocks? Resistors? What will the change cost? What is the current state? What does the future look like? Is the organization ready to change? What are the risks involved in implementation of the change? What is causing the change? What are the concept shifts?

Challenges with Change
Cultural – We’ve always done it this way Systems – The systems we have require that we do it this way Fear of the unknown Doesn’t fit with SOPs Likes things the way they are Fear of failure Too busy Confused about the change Perceives change as more work )

Preparing for the Change
Consider a range of options, solutions and actions to implement the change

How to Get Started Collaborate with Change Management Experts, and Key Opinion Leaders Within Your Organization—Bring in the resistors too. Review Literature on Effective Change Management Set goals for value to be accomplished based on the future vision Identify Project Sponsor and Key Stakeholders. Executive support for the change is essential for success Perform assessment of technology needed to support RBM methodology, identify gaps.

Next Steps Toward Implementation of RBM: Case in Point
Plan the implementation - scope, future vision, risks Develop team structure to support implementation Perform stakeholder mapping, analysis & management Develop RBM process map Determine role competency requirement Develop communication plan Develop training plan Identify pilots to apply methodology Collect feedback & evolve the process as needed

How Does RBM Affect My Role?
10/8/2017 How Does RBM Affect My Role? Defines essential data and study priorities Evaluates and mitigates risks Provides in-stream data and reports Identifies trends Assess site for quality of performance - use sound judgment to adjust type & frequency of engagements Merges RBM principles with site interactions Training Takes accountability for site quality Embraces RBM to enhance compliance and accuracy of data Enters data into eCRF in near real time Site Staff Study Team: Project Manager, Stats, Data Managers, Medical Monitors, Pharmacovigilance/Safety Site Facing Roles: Monitor/CRA, Site Managers

Key Features of RBM Methodology
Monitoring is customized to sites/trials needs Schedule is flexible to comply with sites needs Identifies risks proactively Leverages technology for centralized surveillance Shares monitoring responsibilities across many functional areas Relies more heavily on central and off-site monitoring

Wrap-Up Plan the appropriate monitoring activities inclusive of Central, Off-site, and/or On-site monitoring Identify and resolve issues more quickly Focus on errors that matter Related to subject safety, data integrity, and/or regulatory compliance Adaptive monitoring Adjust type, activities, frequency Adjust for risks and issues Promoting risk mitigation and early issue detection Leveraging risk-based approaches and advances in technology Taking a holistic, proactive approach through Off-site and Central Monitoring Implementing targeted On-site Monitoring

Additional Information

Links TransCelerate Home Page http://www.transceleratebiopharmainc.org
FDA Guidance for Industry Oversight of Clinical Investigations - A Risk-Based Approach to Monitoring [Final]. EMA Reflection Paper on Risk Based Quality Management in Clinical Trials (EMA/INS/GCP/394194/2011). Clinical Trials Transformation Initiative. Effective and efficient monitoring as a component of quality.

Terminology Central Monitoring
A “remote evaluation carried out by sponsor personnel or representatives (e.g., Data Manager, Statistician, or Monitor)” (FDA Draft Guidance). Off-site Monitoring Monitoring activities as defined either within process documents or in the monitoring plan (MP) that occur away from the study site location (e.g., at a Monitor’s home or in a sponsor representative’s office). This is also commonly known as remote monitoring. On-site Monitoring “An in-person evaluation carried out by sponsor personnel or representative(s) at the site(s) at which the clinical investigation is being conducted” (FDA Draft Guidance). SDV (Accuracy) Commonly known as ‘transcription checking’, the process by which data within the CRF or other data collection systems are compared to the original source of information (and vice versa) to confirm that the data were transcribed accurately (i.e. data from source matches data in the CRF or other system and vice versa). SDR (Quality) Review of source documentation to check quality of source, review protocol compliance, ensure the Critical Processes and source documentation (e.g. accurate, legible, contemporaneous, original, attributable) are adequate, to ascertain Investigator involvement and appropriate delegation, and assess compliance to other areas (e.g. SOPs, ICH GCPs). SDR is not a comparison of source data against CRF data.

Risk-Based Monitoring

Similar presentations

Presentation on theme: "Risk-Based Monitoring "— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Risk-Based Monitoring

Similar presentations

Presentation on theme: "Risk-Based Monitoring "— Presentation transcript:

Similar presentations

About project

Feedback