1. Data Management for Pharmaceutical Trials Michael A. Kohn, MD, MPP (Acknowledgment: Susanne Prokscha)

2. Final Project: Part A Send in or Demonstrate Your Study Database Due 9/18/2013 Send in a copy of your research study database*. We prefer a database that you are currently using or will use for a research study. However, a demonstration or pilot database is acceptable. *If you are unable to package your database in a file to email, you can send us a link or work out another way to review your database.

3. General description of database Data collection and entry (Forms) Error checking and data validation Reporting (e.g. NIH Report) Analysis (e.g., export to Stata) Security/confidentiality Back up Final Project: Part B Submit Your Data Management Plan Due 9/18/2013

4. Final Project (12 Questions) 1) What is your study? ("The [CUTE ACRONYM] study is a [DESIGN] study of the association between [PREDICTOR] and [OUTCOME] in [STUDY POPULATION]"). 2) What data points are you collecting? (Your data collection forms.) 3) Who collects the data? You? RAs? MDs? Chart abstractors? Maybe the study subjects enter the data themselves?

5. Final Project (cont’d) 4) How are the data collected? Written onto a paper form and then transcribed into a computer database? Entered directly into the computer? (If data are transcribed, who does it? Have you hired/will you hire somebody? Or have you enlisted/will you enlist some med students?) 5) Will the above-mentioned computer database be in Access, REDCap, QuesGen, Oncore, OpenClinica, SurveyMonkey, Medidata RAVE, or something else? 6) Try to provide a detailed data dictionary with the name, data type, description, and validation rules for each field (column) in the data table(s).

6. Final Project 7) If it's a multi-table database, even a hand-drawn relationships diagram would help but is not required. 8) How do you validate the data for correctness and monitor the data collection effort? (Usually you have some range checks on individual variables and you periodically query for outliers that are nonetheless within the allowed range.) 9) How will you create reports like the NIH Report? 10) You should periodically analyze the data, not only to look for problems, but also to see where the study is headed. How do/will you do this? Query in Access and export to Stata? 11) How will you protect your subjects' identifying data? 12) How will you ensure that you don't lose your data file in a computer crash or if a water pipe leaks?

7. Answering these 12 questions is an essential part of doing a clinical research study.

8. Acknowledgment Susanne Prokscha Principal CDM PTM Process Analyst September 2012

9. Using EDC-Rave to Conduct Clinical Trials at Genentech Susanne Prokscha Principal CDM PTM Process Analyst September 2012

10. Prokscha, 3 rd Edition

11. Jargon from the Clinical Trials World EDC (electronic data capture) eCRFs (electronic case report forms) CDM (Clinical Data Management) Edit checks (validation rules) Queries (can mean SQL queries of data tables or questions to study sites to clarify data) AE (Adverse Event), SAE (Serious Adverse Event) CFR 21 Part 11 (Code of Federal Regulations, Chapter 21, Part 11)

12. CFR 21 Part 11 Required for submission of electronic data to the FDA when applying for drug or device approval Audit trail of all data entries, updates, and deletions.

13. CDM Activities for EDC CDM ActivitiesComments Specify and build eCRFsCDM is the lead in designing the data collection forms. Protocol team contributes and approves. When approved, data managers or clinical programmers (CPs) build the forms. Specify and build edit checksCDM specifies the rules that check data for validity (“edit checks”). When an edit check triggers, it automatically creates a query* to the site for data clarification. CDM or CP programs them. Oversee study validation testing CDM oversees and conducts testing of the study (per 21 CFR Part 11) prior to production use. Monitor data entryEven though the site enters, often CDM is responsible for reporting to the protocol team on status of eCRF data 13

14. CDM Activities (continued) CDM ActivitiesComments Collect AE and SAE dataAdverse Event and Serious Adverse Event data typically have special collection and management activities. Code reported termsAEs, medications, and other broader terms are typically coded or matched against a dictionary/thesaurus to allow for appropriate counting of those events or terms. Manage non-CRF dataNearly all studies have data that does not come in on eCRFs but rather arrives electronically from vendors or suppliers. This includes: labs, EKG readings, microbiology, special assays, pharmacokinetic results, and so forth. Generate manual queriesAutomatic edit checks have limitations so protocol teams perform manual reviews and analyses of the data. This typically will result in queries that have to be manually entered into the system. 14

15. CDM Activities (continued) CDM ActivitiesComments Monitor query resolutionFor both automatic and manual queries, the sites are expected to resolve them all and within a reasonable period of time from issuance. CDM and clinical staff monitor query resolution. Perform lab data admin tasksLaboratory data from the eCRF or directly from the lab as non-CRF data has administrative tasks associated with it including linking normal ranges to reported values that fall to CDM. Oversee study lock activitiesAs the study nears close, final cleaning and data review activities take place. CDM is responsible for many of the final tasks. After lock, the data can be analyzed. 15

16. EDC Regulatory Environment Any system used to collect data that may be included as part of an NDA to the FDA must be compliant with 21 CFR Part 11 – the rule on electronic records and signatures. The rule includes system requirements such as:  Time-stamped, automatic audit trail  Secure account management and access controls The rule also includes procedural elements:  System and study level validation  Standard operating procedures 16

17. Case Study Large Phase III Trials at Genentech

18. Study Startup – Prep for “First Patient In” Study startup activities begin when the protocol summary is firm and rely on the final protocol to move forward until it is ready for subject data. Specify and build the eCRF  There will be 50-75 unique eCRFs for a Phase III study Specify and build the edit checks  There can be from 600 to 2000 individual edit checks  Estimate 1000 edit checks for a Phase III study Test the study application and release for production (“go live”) 18

19. Study Conduct – Collect and Clean Data Study conduct runs from first patient in to lock. Monitor site entry of data and responses to queries Perform Data Reviews  Some checks are more easily done by a humans via listings or reports  Medical review is performed by clinical science Create Manual queries  Example: MS Phase III trial with 105 sites and 450 subjects had 5000 manual monitor and data review queries  Example: Phase II trial with 32 sites and 120 subjects had about 800 manual queries 19

20. Study Conduct – Other Activities Collect non-CRF data (labs, etc.) Code adverse events and medications Collect and reconcile serious adverse events against the safety database An important event during study conduct that occurs in most Phase II and III trials is a database change or “amendment”. Most studies have multiple amendments. Genentech estimates a cost of $60- 70,000 per amendment. 20

21. Study Closeout – Prepare for Lock Preparation for lock of a study often begins prior to last patient “out” and must be complete before lock. Ensure data is complete (both eCRF and non-eCRF) Perform final data reviews  Final Data listing review  Review all coding assignments  Check that all SAEs are accounted for Obtain resolution for, or close, all open queries Obtain Principal Investigator Signature Prepare archival versions of eCRF for the sites and the sponsor files 21

22. Data Management Impact Even though data management is only 12% of the cost of a large trial, it is an essential component. If the data is not complete or accurate enough to be analyzed and sustain a conclusion, then the entire trial cost is wasted. 22

23. References For electronic records in clinical trials in general:  (FDA) 21 CRF Part 11; Electronic Records; Electronic signatures  (FDA) Guidance for Industry: Computerized Systems used in Clinical Investigations  (EMEA) Reflection Paper on Expectations for Electronic Source Documents used in Clinical Trials Using EDC vendors or hosts:  FDA presentation: “Guidance on the Use of Electronic Records and Electronic Signatures;” P. M. Beers Block, 12/2009  (FDA) DRAFT Guidance for Industry: Electronic Source Documentation in Clinical Trials 23