Using an Independent Statistician to Support a Data Monitoring Committee Patrick D. OMeara, Ph.D. Pat OMeara Associates, Inc. FDA/Industry Workshop September 2005

Outline Introduction Introduction Checklist Checklist 2 examples 2 examples Recommendations Recommendations

... the integrity of the trial is best protected when the statistician preparing unblinded data for the DMC is external to the sponsor, especially for the critical studies intended to provide definitive evidence of effectiveness. - Draft Guidance On the Establishment and Operation of Clinical Trial Data Monitoring Committees Introduction

Theres more to protecting the integrity of the study than just engaging an independent statistician to perform a survival analysis for the DSMB. Introduction

Checklist Charter Charter Contract Contract Documentation from Sponsor Documentation from Sponsor Database Database Support from the Sponsor Support from the Sponsor

DSMB Charter Independent statisticians role Independent statisticians role Who directs the IS work? Who directs the IS work? Sufficient freedom to create tables/analyses requested by the DSMB Sufficient freedom to create tables/analyses requested by the DSMB In effect the IS acts as an employee of the DSMB In effect the IS acts as an employee of the DSMB Roles of other organizations who provide data Roles of other organizations who provide data CRO CRO IVRS IVRS Central clinical lab Central clinical lab Biomarker lab Biomarker lab

Contract Tasks assigned to the IS Tasks assigned to the IS Programming of the tables, graphs, & listings Programming of the tables, graphs, & listings Who will write them? Who will write them? How will they be verified ON THE ISs SYSTEM? How will they be verified ON THE ISs SYSTEM? If written by the Sponsor, what changes are allowed? If written by the Sponsor, what changes are allowed? Who is in responsible for the data? Who is in responsible for the data? Especially important if a CRO has been retained to monitor the study and to prepare the data for interim analysis: Especially important if a CRO has been retained to monitor the study and to prepare the data for interim analysis: Adding variables to an analysis database Adding variables to an analysis database Correcting errors, inconsistencies Correcting errors, inconsistencies Interpretation Interpretation

Contract Time to assemble the report Time to assemble the report From time IS receives the data From time IS receives the data Once agreed Sponsor must not let it slide. Once agreed Sponsor must not let it slide. Time to prepare a presentation Time to prepare a presentation Especially important at formal interim analysis when auxiliary analyses may be needed to support result. Especially important at formal interim analysis when auxiliary analyses may be needed to support result.

Documentation from Sponsor Protocol & blank CRF Protocol & blank CRF Informed consent forms Informed consent forms Formal Statistical Analysis Plan Formal Statistical Analysis Plan Includes a clear statement of decision rules for any interim analysis: Includes a clear statement of decision rules for any interim analysis: Null and Alternative Hypotheses Null and Alternative Hypotheses Significance level Significance level Test statistics and methods Test statistics and methods Futility, efficacy Futility, efficacy Adjudication process & rules Adjudication process & rules Investigators Brochure Investigators Brochure

Documentation from Sponsor Proposed list of Tables, Graphs, Listings Proposed list of Tables, Graphs, Listings Any special consideration in the studies Any special consideration in the studies The definition and processing of SAEs in a clinical endpoint study The definition and processing of SAEs in a clinical endpoint study For example, stroke For example, stroke Biomarker lab data Biomarker lab data

Database Sources of the data Sources of the data CRF database (SAS) CRF database (SAS) Analysis database (SAS) Analysis database (SAS) Clinical laboratory (SAS, text) Clinical laboratory (SAS, text) IVRS (SAS, text, or Excel) IVRS (SAS, text, or Excel) Biomarker or specialty laboratory (text, Excel) Biomarker or specialty laboratory (text, Excel) SAE database (expedited review – pharmacovig.) SAE database (expedited review – pharmacovig.) Current death list (Excel) Current death list (Excel) 24-hour reports of SAE/Clinical endpoints 24-hour reports of SAE/Clinical endpoints

Database Detailed specifications – derivations of derived variable Detailed specifications – derivations of derived variable Annotated CRF Annotated CRF Description of special processes Description of special processes Topics that Sponsor and IS should discuss Topics that Sponsor and IS should discuss Frequency of updates Frequency of updates Timing before reports Timing before reports Robustness of interim cuts of the database Robustness of interim cuts of the database Do AEs disappear? Do AEs disappear? Are some data sources more reliable or current than others? Are some data sources more reliable or current than others?

Support from the Sponsor Availability by telephone or of Availability by telephone or of A Statistician who can speak with authority about the study and proposed analysis. A Statistician who can speak with authority about the study and proposed analysis. A Statistician/Data analyst/Programmer who can answer detailed questions about the data. A Statistician/Data analyst/Programmer who can answer detailed questions about the data. Face-to-face meeting with key project team members data management and statistics Face-to-face meeting with key project team members data management and statistics Learn the system that produces the data for DSMB Learn the system that produces the data for DSMB Especially useful when resolving inconsistencies. Especially useful when resolving inconsistencies.

Example 1 Two treatments; Planned subjects: ~2200; Two treatments; Planned subjects: ~2200; 28-day all-cause mortality 28-day all-cause mortality Two interim analyses planned Two interim analyses planned Protocol, SAP Protocol, SAP Tables, listings from Sponsors standard library Tables, listings from Sponsors standard library The Job: IS to reproduce TLG and present The Job: IS to reproduce TLG and present Randomization schedule from IVRS group Randomization schedule from IVRS group 4 data sets for analysis with detailed specification 4 data sets for analysis with detailed specification 8 days to prepare and ship report 8 days to prepare and ship report

Example 1 Statistician assigned to database quality Statistician assigned to database quality Sponsors project statistician provided SAS code to implement formal interim analysis. Sponsors project statistician provided SAS code to implement formal interim analysis. A dummy r.s. using A: odd, B: even A dummy r.s. using A: odd, B: even 4-5 test shipments of data before 1 st interim 4-5 test shipments of data before 1 st interim Using last test shipment, 100% check of all tables against set produced by Sponsor. Using last test shipment, 100% check of all tables against set produced by Sponsor.

Example 1 1 st Interim Report and Analysis 1 st Interim Report and Analysis Timeline for report was squeezed by 1-2 days Timeline for report was squeezed by 1-2 days DSMB decision to continue study without change DSMB decision to continue study without change 2 nd Interim Report and Analysis 2 nd Interim Report and Analysis Efficacy was demonstrated. Efficacy was demonstrated. Auxiliary analyses demonstrated consistent trends across many subgroups Auxiliary analyses demonstrated consistent trends across many subgroups DSMB recommended stopping for efficacy DSMB recommended stopping for efficacy IS presented results to Sponsor executive comm. IS presented results to Sponsor executive comm. Study stopped. Study stopped.

Example 1 Lessons learned Lessons learned Well-defined roles and responsibilities contributed to team environment. Well-defined roles and responsibilities contributed to team environment. Extra data transfers allowed practice so that final transfers went smoothly and tight timelines could be met. Extra data transfers allowed practice so that final transfers went smoothly and tight timelines could be met. Time spent reviewing database was a big contributor to success of the project. Time spent reviewing database was a big contributor to success of the project. 100% check of tables using dummy r.s. was essential. 100% check of tables using dummy r.s. was essential.

Example 2 Two treatments; ~1800 patients Two treatments; ~1800 patients All-cause mortality All-cause mortality DSMB meetings every 3 months at beginning DSMB meetings every 3 months at beginning Protocol, no SAP Protocol, no SAP Review imbalance every 8 deaths for 1 st 100 Review imbalance every 8 deaths for 1 st 100 IVRS, Sponsors DM, Biomarker, Clinical lab IVRS, Sponsors DM, Biomarker, Clinical lab Monthly updates of clinical database via FTP Monthly updates of clinical database via FTP Sponsor provided SAS programs for data from Clinical database (primary motivation $$) Sponsor provided SAS programs for data from Clinical database (primary motivation $$)

Example 2 Programs designed for VAX; local system PC. Programs designed for VAX; local system PC. File references in every program had to be changed. File references in every program had to be changed. Each program contained an extensive block of code to merge in randomization schedule. Each program contained an extensive block of code to merge in randomization schedule. Logic errors in several programs. Logic errors in several programs. After a month, client resent programs with changes but the changes were not documented and all the file references had to be changed again. After a month, client resent programs with changes but the changes were not documented and all the file references had to be changed again.

Example 2 Meanwhile, Biomarker data had been coded by the lab to prevent inadvertent blinding. Meanwhile, Biomarker data had been coded by the lab to prevent inadvertent blinding. Coded results were manually transcribed into database – no source record of original value. Coded results were manually transcribed into database – no source record of original value. Coded with C++ algorithm using radix-32. Coded with C++ algorithm using radix-32. When decoded found that there had been many transcription errors --- invalid values. When decoded found that there had been many transcription errors --- invalid values.

Example 2 DSMB expressed concern about decision rule for stopping for safety. DSMB expressed concern about decision rule for stopping for safety. Requested Monte Carlo study. Requested Monte Carlo study. Decided to meet again in 10 days to discuss. Decided to meet again in 10 days to discuss. Monte Carlo showed that under reasonable assumptions, chances of stopping were less than.05. Monte Carlo showed that under reasonable assumptions, chances of stopping were less than.05. Recommended changing rule so Pr(Stopping) ~.15. Recommended changing rule so Pr(Stopping) ~.15. Next DSMB, no improvement in safety – study stopped. Next DSMB, no improvement in safety – study stopped.

Example 2 Lessons learned: Lessons learned: Transferring SAS programs between systems requires careful planning and lots of work. Transferring SAS programs between systems requires careful planning and lots of work. Need to specify who is responsible for correct implementation on local system. Need to specify who is responsible for correct implementation on local system. Nothing in contract that said IS could do additional analyses suggested by DSMB. Nothing in contract that said IS could do additional analyses suggested by DSMB. Serious decisions by DSMB could result in liability. Serious decisions by DSMB could result in liability. Unforeseen problems can heavily influence the amount of time spent on a project. Unforeseen problems can heavily influence the amount of time spent on a project.

Recommendations Provide the independent statistician with all the available information about the study: Provide the independent statistician with all the available information about the study: Protocol Protocol Statistical Analysis Plan Statistical Analysis Plan Assign project team members to with the IS. Assign project team members to with the IS. Charter should state IS works for the DSMB. Charter should state IS works for the DSMB. IF Sponsor decides to provide programs, IF Sponsor decides to provide programs, Work with IS when designing Work with IS when designing State in contract & charter who is responsible State in contract & charter who is responsible Contract should indemnify IS. Contract should indemnify IS.

The End