1. Measuring the Feasibility and Cost of PRO-CTCAE Implementation in Trials Presentation to NCI Stakeholders Based on work of the PRO-CTCAE Task 8 (Feasibility) Committee October 5, 2011

2. Outline 1.Background 2.Rationale 3.Data sources: Correlative studies in cooperative group clinical trials – Approach to measuring feasibility – Approach to measuring cost

3. PRO-CTCAE Background Contract I (2008-2010) + ARRA (2010-2012) – Items developed Cognitive interviewing complete Validation study 90% complete – Web software developed Usability testing conducted to revise software

4. PRO-CTCAE Background Contract II (2010-2015) – Expand accessibility to patients Develop IVRS Spanish translation and linguistic validation – Determine optimal approach to clinician and patient AE reporting in cancer clinical trials – Assess feasibility and cost of implementation in cooperative group clinical trials

5. Rationale for Assessing Feasibility/Cost Towards future, need to understand feasibility and cost of implementing PRO-CTCAE in clinical trials at: – NCI level, Group level, Site/staff level (CRA and investigator), patient level – Including level of acceptance/enthusiasm at each level; barriers/effective strategies and infrastructure, effort, cost Can inform modifications of approach, strategic decisions, resource allocation

6. Data Source: Correlative Studies in 2 Clinical Trials RTOG 1012 (Manuka honey) – Randomized phase II symptom intervention trial – Primary outcome: Improvement of esophagitis-related pain during RT for lung cancer – PRO-CTCAE completed weekly by iPad in clinic, with backup data collection methods – Status: Activation 10/19/11 Alliance/NCCTG 1048 (rectal cancer) – Phase II/III trial – Hypothesis: neoadjuvant FOLFOX and restaging can risk stratify patients to selectively avoid combined modality therapy (CMT) before low anterior resection (LAR) with similar outcomes to CMT + LAR – PRO-CTCAE completed weekly by IVRS from home, with backup data collection methods – Status: CTEP review

7. Additional Studies Outside of Contract PRO-CTCAE is being integrated into several Alliance and RTOG trials as an exploratory endpoint by enthusiastic investigators Material Transfer Agreement (MTA) mechanism developed – Permitting investigator-initiated research – Supporting R01 and R21 applications

8. What Do We Mean By Feasibility and Cost? Feasibility – Willingness and ability of stakeholders/entities at each level to integrate PRO-CTCAE into trials Cost – Resources required at each level to implement PRO-CTCAE, including personnel effort, equipment, services

9. Measuring Feasibility: NCI Level System hosting – Logistics, technical difficulties, troubleshooting – Descriptive approach

10. Measuring Feasibility: Group Level Descriptive approach – Willingness to integrate into trials Leadership Clinical committees/Study Chairs Concept review committees – Feasibility of integrating into protocols Investigators Protocol editors Statisticians Preliminary – Highly feasible, increasing interest to use PRO-CTCAE – Reflected in survey

11. Feasibility: Site/Staff Level Staff feedback survey/semi-structured interviews Training and central coordination – Ability to centrally train and coordinate staff – Ability to train patients Patient enrollment – Proportion of eligible patients enrolled – Reasons for non-enrollment Technical issues – Also assessed in usability and validation studies Logistical barriers and recommendations SEE ATTACHED SURVEYS

12. Feasibility: Patient Level Proportion of patients willing to participate – Reasons for non-participation – Associated patient characteristics Proportion of enrolled patients using the system at each time point – Reasons for non-adherence – Associated patient characteristics

13. Cost Diagram COST Site Level Cooperative Group Level NCI Level Hosting Web IVRS Operations and maintenance Technical support System super admin System improvements Hosting Web IVRS Operations and maintenance Technical support System super admin System improvements Pre-study Develop protocol sections Develop forms During study Centralized teaching Monitoring QA Post-study (statistical) Data cleaning Statistical analysis Pre-study Develop protocol sections Develop forms During study Centralized teaching Monitoring QA Post-study (statistical) Data cleaning Statistical analysis CRAs Teach patients Monitor compliance Reminders/backup data collection Investigators/clinicians Address alerts CRAs Teach patients Monitor compliance Reminders/backup data collection Investigators/clinicians Address alerts

14. NCI-Level Costs Software system hosting, operations and maintenance – Obtain from CBIIT – Note: these are ongoing costs (i.e., not per-study)

15. Group Level (Per Trial) Administrative – Will estimate from feasibility trials, other trials, discussion with group leadership – In feasibility studies, providing % of site per-patient reimbursement – likely not required for standard use Alliance: 54% IDC RTOG: 27% IDC Central coordination – 20% effort of study coordinator a central coordinating site Statistical – Estimate from Alliance/Mayo statistical Center: $1500-2500/trial (includes analyst and statistician)

16. Site Level Plan for estimating cost in feasibility trials Based on measuring self-reported effort of CRAs and investigators/clinicians Developed by Committee including NCI and external methodological experts 1.CRA questionnaire (administered at 6-months) Learn system, teach patients, monitor, remind 2.CRA semi-structured interviews Follow-up on questionnaire, what could expedite 3.Investigator/clinician questionnaire Review patient reports SEE ATTACHED SURVEYS

17. Excerpts from CRA Survey How many hours of training did it take for you to learn how to use the PRO-CTCAE software system? (ask # and range) On average, how many minutes does it take you to train a patient to use the PRO-CTCAE software system? On average, how many minutes does it take you to train a patient? At each follow up visit, how much time do you spend with patients to assist them…? For each patient… on average how much time do you spend between visits addressing issues related to the PRO-CTCAE (helping patient with passwords, addressing automated alerts, etc)?

18. Discussion Points Adequacy of planned approaches – Feasibility at group level – Cost at site level Other potential approaches Future – Role of PRO-CTCAE data in study monitoring (DSMB) – Approach to AE data analysis and reporting – Regulatory role