1. Architecture Design of Generic Outcome Adjudication in CTMS Wenle Zhao, PhD Medical University of South Carolina, Charleston, SC, 29425, USA Society for Clinical Trials 36th Annual Meeting Arlington, VA, USA - May 17-20, 2015

2. Contents 1.The Motivation 2.The Challenges 3.The Generic Database Model 4.The Automated Coordination 5.Summary

3. 1.The Motivation 2.The Challenges 3.The Generic Database Model 4.The Automated Coordination 5.Summary

4. Patients Randomization Treatment ATreatment B Outcome AOutcome B Analysis Accurate Outcome ( Endpoint) Assessment is Important The validity of the trial results relies on the accuracy of outcome assessment.

5. Outcome Safety OutcomeEfficacy Outcome AE is Serious?Quality of Life Modified Ranking Scale NIH Stroke Scale Hamilton Rating Scale for Depression Hamilton Rating Scale for Anxiety Most Outcome Assessments are Subjective AE is Expected? AE is Related to Study Treatment? Glasgow Outcome Scale Extended The Problem: Uneven expertise and potential bias The Solution: Independent central adjudication

6. 1.The Motivation 2.The Challenges 3.The Generic Database Model 4.The Automated Coordination 5.Summary

7. Example 1: A Simple SAE Adjudication SAE CRF Submitted Is serious?Is expected?Is related? Done CRF Completed? May be not. How to notify the adjudicator? Auto Email. When it is expected? No schedule. Need more information? May be yes. CRF Updated after submission? Could happen.

8. Four challenges for coordinating outcome adjudication  Adjudication procedures vary among projects No fixed model for every procedure.  Timing of outcome event unpredictable No schedule.  Narrow time window for completion of adjudication “Time is of the essence.”  Data changing after adjudication procedure started The target is moving.

9. 1.The Motivation 2.The Challenges 3.The Generic Database Model 4.The Automated Coordination 5.Summary

10. The Generic DB Structure for Adjudication Procedures 1 ∞ ∞ 1 ∞ 1 1 1 ∞ 1 ∞ 1 Adjudication Procedure ID Form ID Procedure Name Adjudication Step ID Procedure ID Adjudication Step Name Enable-logic condition Disable-logic condition Question 1 Answer 1 Code Group ID …… Question 5 Answer 5 Code Group ID Edit User Group ID View User Group ID Adjudication Result ID Adjudication Step ID Record ID Freeze Record ID Answer 1 …… Answer 5 General Comments Locked Data (CRF) Form ID Form Name Table Name Form Table ID Subject ID Visit ID Q01 Q02 …… Updated By Updated On Form Freeze Table Freeze Record ID Subject ID Visit ID Q01 Q02 …… Updated By Updated On ID The Model Allow multiple steps. Allow multiple updates. Limit ≤ 5 close-ended Qs.

11. Two Tables are included in the Database Template

12. One table specifies definitions of each adjudication step. The other table stores adjudication results.

13. A freeze table is created for each data form table with adjudication

14. Adjudication Step Enable logicDisable logic Question 1 (Q1) Question 2 (Q2) Question 3 (Q3) Step 1 PM Review (S1) {New SAE CRF submission} {S2 submitted after S1} OR {S3 submitted after S1} CRF properly completed? 0 = No 1 = Yes Requires MSM review? 0 = No 1 = Yes Step 2 MSM Adjudication (S2) {S1Q1=1} AND {S1Q2=1} AND {no S2 submitted after S1} AND {no S3 submitted after S1} {S2 submitted after S1} OR {S3 submitted after S2} Serious? 0 = No 1 = Yes Unexpected? 0 = No 1 = Yes Related to study drug? 1 = Unrelated 2 = Unlikely 3 = Possible 4 = Probably 5 = Definitely Step 3 PM Review (S3) {S2 submitted} AND {no S3 submitted after S2} {S3 submitted after S1} MedWatch Form needed? 0 = No 1 = Yes Example 1: Definition of Three-step SAE Adjudication

15. Adjudication StepEnable logicDisable logic Question 1 (Q1) Question 2 (Q2) Step 1 Project Manager (PM) Review {New SAE CRF submission}{S2 submitted after S1} CRF properly completed? 0 = No 1 = Yes Requires CEC review? 0 = No 1 = Yes Step 2 Clinical Event Coordinator (CEC) Review {S1Q1=1} AND {S1Q2=1} AND {no S2 submitted after S1} {S1 submitted after S2} OR {S3 submitted after S2} OR {S4 submitted after S2} OR Even packet complete? 0 = No 1 = Yes Event type? 1 = Neurological 2 = Cardiac 3 = Systemic Step 3 Adjudicator 1 Review (S2Q1=1) AND {no S5 submitted after S3} {S1 submitted after S3} OR {S5 submitted after S3} Outcome category=? 1 = ischemic stroke 2 = symptomatic hemorrhagic transformation of an ischemic stroke 13 = transient ischemic attack Death type =? 1 = ischemic 2 = hemorrhagic vascular 3 = nonvascular Step 4 Adjudicator 2 (ADJ2) Review (S2Q1=1) AND {no S5 submitted after S4} {S1 submitted after S4} OR {S5 submitted after S4} Step 5 Adjudicator 3 (ADJ3) Review {S3 submitted after S1} AND {S4 submitted after S1} AND {S3Q1≠S4Q1 or S3Q2≠S4Q2} AND {no S6 submitted after S5} AND {S1 submitted after S5} OR {S6 submitted after S5} Step 6 Adjudication Committee Chair (ACC) Review {S5 submitted after S1} AND {S5Q1≠S3Q1 or S5Q2≠S3Q2} AND {S5Q1≠S4Q1 or S5Q2≠S4Q2} AND {no S1 submitted after S6} AND {S6 submitted after S1} Example 2: Definition of Six-step SAE Adjudication

16. The Generic Table for Adjudication Results

17. 1.The Motivation 2.The Challenges 3.The Generic Database Model 4.The Automated Coordination 5.Summary

18. Site submits a Case Report Form Form has adjudication procedure defined? System evaluates enable logics for each adjudication steps Enable logic = true & disable logic = false? System sends email to [Edit] user group members User views a Case Report Form User has adjudication step [Edit] permission & Enable logic = true & disable logic = false? System displays [Adjudication Edit] button for the user User adds/edits adjudication result and submits adjudication record The last freeze record is the same as the current CRF record? System creates new freeze record, links it to the adjudication record. Y Y Y N Y N N N Automated Coordination of Adjudication Activities

19. Site submits SAE CRF S1Q1 = Yes? Step S3 = PM Review S3Q1: MedWatch Form required? System composes MedWatch Form draft N Y Y N Step S1 = PM Reviews S1Q1: CRF properly completed? S1Q2: Requires MSM adjudication? Auto email #1 to PM Auto email #3 to MSM Auto email #2 to Site S1Q2 = Yes? Step S2 = MSM Adjudication S2Q1: Serious? S2Q2: Unexpected? S2Q3: Related to study treatment? Site adds/edits SAE CRF Auto email #4 to PM S2Q1 = Yes & S2Q2 = Yes & S2Q3 = Possible/ Probably / Definitely? S3Q1 = Yes? Auto email #6 to Site Site edits MedWatch PM Closes the Adjudication Procedure Auto email #7 to PM PM finalizes MedWatch and submits to FDA Auto email #8 to All Sites N Y Auto email #5 to PM Y N Example 1: Three-step SAE Adjudication Procedure

20. Site submits SAE CRF S1Q1 = Yes? Step S3 = ADJ1 Review S3Q1: Outcome category? S3Q2: Death type? Y N Step S1 = PM Reviews S1Q1: CRF properly completed? S1Q2: Requires CEC review? Auto email #1 to PM Auto email #3 to CEC Auto email #2 to Site S1Q2 = Yes? Step S2 = CEC Adjudication S2Q1: Event packet complete? S2Q2: Event type? Site adds/edits SAE CRF and upload the event packet Auto email #7 to ADJ3 Auto email #9 to PM PM Closes this Adjudication Procedure N Y Auto email #5 to ADJ1 Y Step S5 = ADJ3 Review S5Q1: Outcome category? S5Q2: Death type? Step S4 = ADJ2 Review S4Q1: Outcome category? S4Q2: Death type? S3 = S4? Auto email #6 to ADJ2 N System determines adjudicators 1, 2, and 3 based on S2Q2 and the Random number R. Auto email #4 to Site Y S2Q1 = Yes? N Auto email #8 to ACC Y Step S6 = Chair Review S6Q1: Outcome category? S6Q2: Death type? S3= S5 or S4 = S5? N Example 2: Six-step Outcome Adjudication Procedure

21. In this case, the Clinical Event Coordinator (CEC) completed the adjudication within few hours after the Study Manager (SM) ‘s completeness review passed.

22. In this case, the procedure has not been completed yet after two months, due to CRF data change.

23. 1.The Motivation 2.The Challenges 3.The Generic Database Model 4.The Automated Coordination 5.Summary

24. Adjudication Procedure From stepTo step Event count 1st quartile (25%) 2nd quartile (50%) 3rd quartile (75 %) hrs Three-step SAE adjudication CRF submitPM review125321 51 126 PM reviewMSM review12265 13 35 MSM reviewPM close122627 56 112 Six-step outcome event adjudication CRF submitPM review60926 120 477 PM reviewCEC review59996 197 364 CEC reviewADJ1/2 review46847 204 892 ADJ1/2 reviewADJ3 review7656 209 383 ADJ3 reviewACC review1023 25 PM: project manager MSM: medical safety monitor CEC: clinical event coordinator ADJ1/2: adjudicators 1 and 2 ADJ3: adjudicator 3 ACC: adjudication committee chair Our Experiences

25. The Strategy to Success  Allows multiple adjudication procedures per CRF.  Allows multiple steps for each adjudication procedure.  Allows different questions be asked for each adjudication step.  Only close-ended questions are allowed.  No more than 5 questions are allowed.  Using automated emails to help adjudication coordination.  Allows multiple CRF submissions after adjudication started.

26. Acknowledgement This research is partly supported by following NIH/NINDS grants: U01NS059041 (NETT Palesch, Y. & Durkalski, V.) U01NS062778(ProTECT, Palesch Y.) U01NS062835(POINT, Palesch, Y.) U01NS087748(StrokeNet, Palesch, Y. & Zhao, W.)  The followings are contributed to the design, development, validation and user feedback collection to this work: Keith Pauls(DCU senior programmer) Jaemyung Kim(DCU senior programmer) Catherine Dillon(DCU senior data manager) The Clinical Coordination Center of the NINDS NETT at University of Michigan  Thousands investigators in hundreds institutions and thousands patients participating our clinical trials.

27. Thank You! Contact me at: zhaow@musc.edu