1. Auditing & Quality Assurance Igor Espinoza-Delgado, M.D. Gary Smith M.Sc. Cancer Therapy Evaluation Program

2. Health Care Delivery Research Good Clinical Practice Clinical Trials

3. Good Clinical Practice (GCP) An internationally recognized standard for the design, conduct, performance, monitoring, auditing, recording, analyses, and reporting of clinical trials. GCP provides assurance that the data and reported results are credible and accurate, and that the rights, integrity, and confidentiality of trial subjects are protected.

4. Regulatory Basis for Monitoring and Auditing  1962 Harris-Kefauver amendments to the food, drug, and cosmetic act required that FDA regulate the testing of new drugs as an investigational new drug (IND) prior to drug approval  Demonstration of efficacy and safety in well- controlled clinical investigations

5. Regulatory Basis for Monitoring and Auditing (Cont.)  FDA IND regulations (21CFR 312.50) requires sponsors to ensure proper monitoring of investigation(s), ensure that investigations are conducted in accordance with approved protocols as contained in the IND, and ensure that the FDA and all participating investigators are promptly informed of significant new adverse effects or risks with respect to the drug.

6. All those planned and systematic actions that are established to ensure that the trial is performed and the data are generated, documented (recorded), and reported in compliance with GCP and applicable regulatory requirements. Quality Assurance

7.  Patient Safety/Ethical Considerations  Respect for Persons  Beneficence: maximize possible benefits and minimize possible harms  Regulatory Considerations  Delay in product approval  Approval withdrawals  Sanctions Why Be Concerned With Quality?

8.  Monetary Considerations  $802 million: total average preclinical and clinical costs up to the time of receiving FDA marketing approval  $897 million fully capitalized cost to develop a new drug, including cost of conducting post marketing surveillance studies after receiving regulatory approval (Tufts Center for the Study of Drug Development) Why Be Concerned With Quality?

9.  Public Trust Why Be Concerned With Quality?

12.  Staff training/mentoring  Quality control  Data safety and monitoring  Study monitoring  Auditing Components of Quality Assurance

13. The act of overseeing the progress of a clinical trial, and of ensuring that it is conducted, recorded, and reported in accordance with the protocol, standard operating procedures (SOPs), GCP, and applicable regulatory requirements. Monitoring

14. A systematic and independent examination of trial related activities and documents to determine whether the evaluated trial related activities were conducted, and the data recorded, analyzed and accurately reported according to the protocol, sponsor’s standard operating procedures, GCP, and the applicable regulatory requirements. Audit

15. Auditing:  Snapshot in time  On-site process  Subset of patients on a trial Monitoring:  Continuous process  On-site and/or off-site  Includes all patients on a trial Quality Assurance Auditing vs. Monitoring

16. Objectives:  Assure accuracy and quality of data  Assure compliance with Federal regulations  Assure compliance with sponsor’s policies and procedures  Assure compliance with GCP  Serves as an educational tool On-Site Audit Program

17. Phase of Study Study MonitoringReporting Mechanism Auditing Phase 1 & Select Phase 2 CTMS + CTEP  Bi-weekly CRFs to CTMS  Monthly CTMS reports, downloads to CDUS  CTMS Pharm.D./M.D. once per year  CTMS/CRA Twice per year Phase 2 & 3 Cooperative Group Cooperative Group + CTEP CDUS quarterly Group physicians, CRAs, nurses once every 3 years Phase 2 & 3 Cancer Center/ Single Inst. CTEPCDUS quarterly CTEP + CTMS + peer physicians once every 3 years CTEP Monitoring and Auditing

18.  Special audits:  For cause audits  Response audits Other Types of Audits

19. NCI model:  Phase 1 CTMS monitored trials  Phase 2 & 3 trials:  Monitoring via electronic submission of data quarterly via the clinical data update system (CDUS) and adverse event expedited reporting system (AdEERS). Industry Model:  Similar to NCI model  Phase 2 & 3 trials:  Sites undergo on-site monitoring visits every 6 weeks with 100% of Case Report Forms included in source data verification during monitoring visit. Key Differences Between NCI and Industry Model

20. NCI model:  Audit unit is institution based  >10% of patient cases are audited  Scientific incentives Industry Model:  Audit unit is protocol based  >10% of sites are audited  Financial incentives Key Differences Between NCI and Industry Model (Cont.)

21. On the day of the audit:  Provide overview of research procedures (scientific review committee, IRB)  Provide overview of organization of the medical records and research charts  Have data management staff available throughout the process if questions arise Audit Preparation

22. On the day of the audit (cont.):  Meeting with principal investigator to review status of study  Pharmacy inspection  Exit interview with P.I. and staff

23.  Regulatory:  Documentation of initial IRB approval  Documentation of continuing reviews  Documentation of IRB approval of all amendments  Documentation of IRB review of reportable adverse events and IND safety reports  Documentation of other IRB correspondence pertaining to protocol Audit Preparation What Is Needed?

24.  Regulatory (cont.):  1572s for Principal Investigator and Co-Investigator*  Investigator CV/Medical License*  IRB Membership List*  Office of Human Research Protections (OHRP) Assurance Number  Laboratory Certification* and Lab Normal Ranges *primarily industry requirement Audit Preparation What Is Needed?

25.  Pharmacy:  Drug Accountability Records as provided by the sponsor along with shipping receipts, return receipts, and transfer forms when applicable  Appointment with pharmacy to conduct inspection Audit Preparation What Is Needed?

26.  Patient medical records:  ORIGINAL signed and dated informed consent (at time of enrollment as well as at time of re- consent, if applicable)  Baseline history and physical exam  Prior therapies including start and stop dates  Pathology report  Protocol required parameters (labs, x-rays, scans, EKG, etc.)  Physician orders Audit Preparation What Is Needed?

27.  Patient medical records (cont.):  Medication administration records  Nursing and physician progress notes  Off-study note  Research Records:  Eligibility checklist  Confirmation of registration including arm and/or dose assignment  Documentation of collection/submission of research tests (pharmacokinetics, marker studies, etc.)  Documentation of tumor measurements Audit Preparation What Is Needed?

28.  Research Records (cont.):  Outside labs and other study parameters  Outside physician records and correspondence  Appointment books  Subject diaries/calendars  Study flow sheets and other research records that are signed and dated on a real time basis by the health practitioner evaluating the patient  Protocol or study road maps Audit Preparation What Is Needed?

29. Classification of deficiencies:  Major: any variance from protocol-specified procedures that makes the resulting data questionable  Lesser: any variance that is judged to not have a significant impact on the outcome or interpretation of the study data Cooperative Group On-site Audit Program

30. Possible Actions:  Re-audit  Suspension of patient registration  Suspension of Investigator’s 1572  Problematic audit findings may be referred to other agencies, such as the FDA, the Office for Human Research Protections (OHRP), or the Office of Research Integrity (ORI) when warranted Unacceptable Audits

31.  Reapproval delayed less than 30 days  Reapproval delayed greater than 30 days but less than one year  Lack of documentation of full IRB approval of a protocol amendment that affect more than minimal risk  Delayed reapprovals for protocols closed to accrual for which all patients have completed therapy  Missing reapproval Most Frequently Occurring Deficiencies: IRB

32. Most Frequently Occurring Deficiencies: Consent Content  Lack of disclosure of all risks or side effects contained in model informed consent approved by NCI  Lack of disclosure of approximate number of participants  Lack of disclosure of extent of confidentiality of records  Lack of disclosure of the contact person for research questions, information regarding subject’s rights, and/or contact for research-related injury  Failure to disclose circumstances in which subject’s participation may be terminated by investigator without subject’s consent

33.  NCI drug accountability record forms (DARFs) incomplete or inaccurate  NCI DARFs not protocol and drug specific  Satellite NCI DARFs not accounted for  NCI DARFs not kept as primary transaction record Most Frequently Occurring Deficiencies: Pharmacy

34.  Consent form does not contain all required signatures or dates  Consent form does not include updates or information required by IRB  Consent form used was not current IRB- approved version at time of patient registration  Consent form not signed and dated by patient  Consent form signed after patient started on treatment Most Frequently Occurring Deficiencies: Informed Consent

35.  Documentation missing; Unable to confirm eligibility  Review of documentation confirms patient did not meet all eligibility criteria as specified by the protocol Most Frequently Occurring Deficiencies: Eligibility

36.  Treatment doses incorrectly administered, calculated or documented  Dose deviations incorrect (greater than +/- 10%)  Dose modifications unjustified  Unjustified delays in treatment  Additional agent(s)/ treatment given prohibited by protocol Most Frequently Occurring Deficiencies: Treatment

37.  Tumor measurements/evaluation of status or disease not performed according to protocol  Claimed response (partial response, complete response, etc.) cannot be verified  Protocol-directed response criteria not followed  Inaccurate documentation of initial sites of involvement  Failure to detect cancer (as in a prevention study) or failure to identify cancer progression Most Frequently Occurring Deficiencies: Response

38.  Follow-up studies necessary to assess toxicities not performed  Grades, types or dates/duration of serious toxicities inaccurately recorded  Recurrent under or over reporting of toxicities  Failure to report a toxicity that would require filing an adverse event report  Reported toxicities cannot be substantiated in source documents Most Frequently Occurring Deficiencies: Toxicity

39. Most Frequently Occurring Deficiencies: General Data Quality  Errors in submitted data  Delinquent data submission  Recurrent missing documentation, e.g., charts  Protocol-specified laboratory tests not documented

40. Quality Improvement General Record Keeping:  Remember, if it is not documented, it cannot be verified as being done  Follow standard procedures for documentation and error correction, including dating and initialing entries

43. Quality Improvement “The myth of perfect performance of the health care worker is unrealistic. We must change to an engineering mode of thinking that things will always go wrong. We need to protect the patient from results of errors, or failure of safe design. We not only need a ‘fail-safe’ design, but redundancy in the system, because a backup system protects the process. The system must possess two independent redundant steps, not interdependent steps.” (Quote from Richard J. Croteau, M.D., JCAHO executive director for strategic initiatives) 9