1. Clinical Study Risk Management: What Does it Really Mean & How do You Do It? 1

2. Learning Objectives Define the key elements of risk management principles Examine case studies and lessons learned Evaluate impact on study risk plan

3. Risk Management provides proactive continuous quality improvement to the organization (vs. reactive quality improvement ) 3

4. Risk Definition Risk Context: A Structured Approach ISO 31000:2009 Risk Management Principles Overview Risk Identification Risk Analysis Risk Evaluation Risk Treatment (Risk Management) Risk Monitoring, Review, Validation Case Studies 4

5. Risk: Clinical Trials Risk is the ‘chance’ that a potential event will not go as planned. –Enrollment rate (*too fast, too slow) –Investigational Product (IP) supply –Only eligible subjects enrolled –Subject IP compliance –Subject retention –Subject dosing: correctly performed –Investigational Site business viability This potential event may have: –An impact on the quality of the clinical trial. –An impact in the ability to meet your trial goals, timelines, objectives. Risk is the ‘chance’ that a potential event will not go as planned. –Enrollment rate (*too fast, too slow) –Investigational Product (IP) supply –Only eligible subjects enrolled –Subject IP compliance –Subject retention –Subject dosing: correctly performed –Investigational Site business viability This potential event may have: –An impact on the quality of the clinical trial. –An impact in the ability to meet your trial goals, timelines, objectives.

6. Risk Management: Clinical Trials Risk Management: Systematic process for the assessment, control, communication and review of risks associated with the planning and conduct of clinical trials and clinical development programs. –Quality in clinical trials==Quality Risk Management –Risk: System, Project-Trial EMA reflection paper on risk based quality management in clinical trials November, 2013

7. Principles Clause 3 Mandate & Commitment Design of the framework: manage risks Framework Clause 4 RM Process Clause 5 Implementing risk management Monitoring and review of the framework Continual improvement of the framework Establish the context Communicate and consult Monitor and review Risk identification Risk analysis Risk treatment Risk evaluation Risk assessment Creates value Integral part of organizational processes Part of decision making Explicitly addresses uncertainty Systematic, structured & timely Based on best available information Tailored Takes human and cultural factors into account Transparent & inclusive Dynamic, iterative and responsive to change Facilitates continual improvement & enhancement of the organization P D C A ISO 31000:2009 Risk Management Principles & Guidelines

8. Risk Focus What impacts subject safety? What impacts study conduct? What impacts evaluation of your data? What impacts data: –Safety –Efficacy What impacts “answering your protocols null hypothesis’’ What impacts subject safety? What impacts study conduct? What impacts evaluation of your data? What impacts data: –Safety –Efficacy What impacts “answering your protocols null hypothesis’’

11. Risk Identification What can happen, when, where, how & why Systematic approach: Categorize risks (IP, Site, Vendor, etc.) Information gathering, knowledge: enterprise level, team, department, project Systems-processes, facilities, equipment, personnel, vendors Evidence based: compliance, performance, audits, inspections Regulations, laws, etc. Utilize industry references, tools: implement RM process –Industry Reference Tools Available: Categories-Questions- Training Materials: Clinical Trials Transformation Initiative Transcelerate BioPharma, Inc.

12. Key Points Similarities across studies and programs Cross – functional team needed to develop the risk list Should not expect to identify all risks- as new risks may be identified as the trial progresses Similarities across studies and programs Cross – functional team needed to develop the risk list Should not expect to identify all risks- as new risks may be identified as the trial progresses Trial Risks Too many inclusion criteria Too many CRFs Protocol is too complicated Patient population is too narrow EMA Risk Areas IP related risks Trial design risks Operational risks EMA Risk Reflection Paper on risk based quality management in clinical trials, p.10 November, 2013 Risk Identification: Clinical Trials

14. Risk Analysis What needs to go right? What could go wrong? What is the impact- consequence? What is the likelihood it will occur? How will I know? (detectability) What needs to go right? What could go wrong? What is the impact- consequence? What is the likelihood it will occur? How will I know? (detectability) 14

15. RISK ANALYSIS: Process to comprehend the nature (causes, sources) of the risk (positive, negative) and to determine the risk level IMPACT TO BUSINESS OBJECTIVES X LIKELIHOOD X DETECTABILITY OF OCCURRENCE of a FAILURE MODE (“RISK”) 15

16. 16

17. Risk Evaluation Occurs after the risk analysis Incorporate Quality by Design (QbD) Review to risk criteria: level of risk is acceptable/tolerable –Your organization determines Make decisions – based on the risk analysis results –Risks requiring being addressed –Risk priorities –Tolerance level Occurs after the risk analysis Incorporate Quality by Design (QbD) Review to risk criteria: level of risk is acceptable/tolerable –Your organization determines Make decisions – based on the risk analysis results –Risks requiring being addressed –Risk priorities –Tolerance level

18. 18

19. Risk Treatment A number of options may be identified for use Questions should include (ICH Q9 4.4) –Is the risk above an acceptable level? –What can be done to reduce or eliminate risks? –What is the appropriate balance among benefits, risks and resources? –Are new risks introduced as a result of the identified risks being mitigated or accepted? A number of options may be identified for use Questions should include (ICH Q9 4.4) –Is the risk above an acceptable level? –What can be done to reduce or eliminate risks? –What is the appropriate balance among benefits, risks and resources? –Are new risks introduced as a result of the identified risks being mitigated or accepted?

20. Risk Treatment MITIGATE - Corrective action to eliminate or reduce impact or likelihood Inexperienced site: Your company provides extensive training and more frequent monitoring visits, along with weekly phone calls regarding protocol and GCP adherence and questions AVOID - Cease activity to eliminate risk Inexperienced site: Not choose for the study, TRANSFER - Shift impact to another entity Inexperienced site: Hire a consultant to work with the site as a coach (may or may not be the CRA/Site Monitor) ACCEPT - No corrective action. Document acceptance decision and monitor Inexperienced site: Choose because there is an excellent sub-investigator and Coordinator assigned to the study. MITIGATE - Corrective action to eliminate or reduce impact or likelihood Inexperienced site: Your company provides extensive training and more frequent monitoring visits, along with weekly phone calls regarding protocol and GCP adherence and questions AVOID - Cease activity to eliminate risk Inexperienced site: Not choose for the study, TRANSFER - Shift impact to another entity Inexperienced site: Hire a consultant to work with the site as a coach (may or may not be the CRA/Site Monitor) ACCEPT - No corrective action. Document acceptance decision and monitor Inexperienced site: Choose because there is an excellent sub-investigator and Coordinator assigned to the study.

21. Develop Risk Management Plan Approach to implement the Risk Treatment Plan. Details the approach, the management components and resources to be applied to the management of risk. –Management components typically include procedures, practices, assignment of responsibilities, sequence and timing of activities. Approach to implement the Risk Treatment Plan. Details the approach, the management components and resources to be applied to the management of risk. –Management components typically include procedures, practices, assignment of responsibilities, sequence and timing of activities.

22. 22

23. Monitoring, Review and “Validation” Implement plan with the tolerance limits – critical element continuous monitoring of the risk Monitor the plan continuously: the *right risks* Ensure your controls are effective, efficient, and working- ongoing evaluation and confirmation Identify emerging risks, unknown risks – any source! Continuously monitor internal/external contexts for changes that may impact the plan, modify as needed Ongoing analysis, lessons learned (knowledge management) – feedback in the CQI loop and the risk management process

24. What are the Benefits and Value of Risk Management – as a Process and Discipline?

25. 25

26. In The First Part You Learned About  Risk in a clinical study –Effect = deviation from the expected (+ or -) Fewer serious AEs / AEs reported than expected Longer time taken to respond to queries Subjects being enrolled twice into a study –Objectives = safety of subjects, study budget (finances) –Characterized by potential events and consequences In our examples = lack of detection of a quality issue which may lead to non-acceptance of a study

27. In The First Part You Learned About  Risk in a clinical study –Likelihood of occurrence = chance something will happen In our examples = our historic knowledge about a site or a country under-reporting adverse events, or The chance a subject is ‘lost to follow up’ –Uncertainty = deficiency of information related to understanding an event, the consequence or the likelihood In our example = what do we know and what do we need to know about issues in the clinical study team, the data, the processes, the applications we use in a clinical study

28. In The First Part You Learned About  Risk Management in a clinical study - coordinated activities to direct and control a clinical study with regard to risk –Coordinated In our example = the organization / functions work together cooperatively to manage risk using a structured approach –Direct and control In our example = actions to take to ensure visibility of potential risks and known risk are minimized –We know which sites are poor performers –We know which sites have high incidence of “lost to follow-up” subjects

29. In The First Part You Learned About  Risk Management Framework in a clinical study = foundation for the design & implementation of processes to conduct effective risk management –Includes policies, procedures and training In our example = SOPs, goals for study teams, objectives for managers –Organizational arrangement = accountabilities, resources, processes, activities In our example = independent risk management group for study oversight

30. In The First Part You Learned About  Risk Management Plan = the activities and resources applied to manage risk –In our example = Data Management Plan, the functions / people / expertise involved, and the frequency of metrics review  Risk Owner = a person or entity with the accountability and authority to manage a risk In our example = each function or someone specified per risk

31. The Risk Matrix- Clinical Study SEVERITY CONSEQUENCESLIKELIHOOD Subject SafetyStudy TimelinesStudy Budget 12345 Very Unlikely UnlikelyPossibleLikely Very Likely 1No injuryNo effectNo costLow 2 Minor injury or inconvenience Minimal or inconsequential effect Minimal costLow Medium 3 Moderate injury requiring intervention Moderate effect on some study milestones Moderate increase in cost requiring management notification Low Medium High 4 Severe injury resulting in permanent damage Significant effect on all study milestones Significant increase in study cost requiring management approval LowMedium High 5Death Study must be halted or delayed Significant increase in cost such that study is not financially feasible Medium High

32. Risk Treatment RISK TREATMENT DESCRIPTION OF RISK TREATMENT APPLY TO OUR EXAMPLE: Under-reporting of AEs/SAEs MITIGATE Corrective action to eliminate or reduce severity of impact or likelihood Provide detailed, focused training to all sites regarding expectations for reporting of AEs/SAEs AVOID Cease activity to eliminate risk Select only sites with historical demonstration of correctly reporting AEs/SAEs TRANSFER Shift impact to another entity “Share” impact between site monitoring and data management ACCEPT No corrective action. Document acceptance decision and monitor If site is established as a good performing site, then risk is low – no action required

33. The Worksheets RISK Risk CategoriesRisk OwnerControlsMitigation ActionsPriorityComments Short Description of Risk LikelihoodSeverity of Impact Individual, Not Function Existing Controls Additional controls to lower the overall risk None, Must, Nice-to- have Comments

34. Exercise 1 Our proposal

36. Site performance Adverse events reporting, under-reporting

37. Site performance SAEs reported as Adverse Events

38. Detection of Liver Damage via Hy’s law Guidance for Industry Drug-Induced Liver Injury: Premarketing Clinical Evaluation ALT or AST >8xULN ALT or AST >5xULN for more than 2 weeks ALT or AST >3xULN and (TBL >2xULN or INR >1.5) ALT or AST >3xULN with the appearance of fatigue, nausea, vomiting, right upper quadrant pain or tenderness, fever, rash, and/or eosinophilia (>5%)

39. Other aspects to take into account No. of data corrections per field Number of corrections to a particular field, e.g. Adverse Event data, by a particular site

40. Other aspects to take into account No. of data corrections per field Number of corrections to a particular field by field, by patient

41. RISK: SAE under-reporting (AEs rather than SAEs reported) 41 RISK TREATMENT RISK: SAE under-reporting MITIGATE  Provide additional training for sites demonstrating under-reporting of SAEs AVOID  Build-in EDC MedDRA critical term list alerting sites of potential SAEs while completing the AE forms TRANSFER  Implement a Medical Monitor role with appropriate background for ongoing AE review ACCEPT  Not acceptable per se

42. RISK: AE entries performed too late 42 RISK TREATMENT RISK: AE entries performed too late MITIGATE  Stress importance during investigator meetings  Provide alerts in EDC system for site staff and sponsor monitor AVOID  Get local medical directors in countries involved in site awareness training  Pay bonus for site staff for AE entries within 24 hours after becoming aware of the AE / start of the AE TRANSFER  Implement a data review function with specific focus on AE reporting habits by sites ACCEPT  Accept for AEs but demand for SAEs in particular in late phase 3 or phase 4 studies

43. Clinical Study Risk Management – SUMMARY  Summarized Risk Management terminology and definitions as applied to a clinical study  Applied Risk Management principals to one potential clinical study risk area ―Risk identification ―Categorizing risk: Likelihood and Severity ―Risk treatment  Provided examples of useful tools (reports, listings, graphs) to support Risk Management activities  Identified potential “watch-outs” with respect to Risk Management, such as cultural differences mimicking risks 43

44. At minimum, you can –Complete the Risk Management table for your functional activities – including risk mitigation activities –Share the Risk Management table with your colleagues and educate them on it’s use Remember – any risk you are able to identify and mitigate is a benefit to your clinical study! 44 Clinical Study Risk Management Now what? Get involved! Determine if Risk Management is happening in your organization –Tools? –Systems? –Processes?

45. Thank you Kristin Neff VP Clinical Operations InvivoTherapeutics Johann Proeve VP Global Strategy & Development Advisor Bayer Healthcare 45

46. References Clinical Trials Transformation Initiative (CTTI), Quality by Design Workshops Project: Critical to Quality (CTQ) Factors, 07Jan 2012 EMA reflection paper on risk based quality management in clinical trials, November, 2013 FDA Guidance for Industry, Oversight of Clinical Investigations-A Risk Based Approach to Monitoring, August 2013 ISO 73:2009 Vocabulary for Risk Management ISO 31000:2009 Risk Management Principles and Guidelines ISO/IEC 31010:2009 Risk Management - Risk Assessment Techniques ICH Q9 Quality Risk Management Transcelerate BioPharma, Inc., Risk Based Resources Case study logo: http://www.paulfreiberger.com/the-case-study- a-perfect-tool-for-tech-marketers/