1. Overview of the Standard Analyses & Code Sharing Working Group
PhUSE FDA Participants Support Group
19 September 2018

2. Agenda
Analysis and Display White Papers (ADW) project team – Mary Nilsson
Communication, Promotion, and Education (CPE) project team – Mary Nilsson
Code Sharing (Repository) project team – Hanming Tu
Note: There is a Test Dataset Factory project team within the Standard Analyses and Code Sharing Working Group that is not covered in today’s presentation

3. Analysis and Displays White Papers (ADW)
project team
Project Lead – Mary Nilsson
FDA Liaison – Nhi Beasley

4. ADW Project Description
This project includes the development of white papers that provide recommended Tables, Figures, and Listings for clinical trial study reports and submission documents. The intent is to begin the process of developing industry standards with respect to analysis and reporting for measurements that are common across clinical trials and across therapeutic areas.
Project Description as provided on the wiki page for the project

5. ADW Project Goals
Define a set of recommended tables, figures, listings and associated analyses
Ensure reviewers receive clinically relevant and meaningful analyses for benefit-risk assessment
Harmonization leads to efficiencies in both creation and review of analyses
Improve expertise in safety analytics across multiple disciplines involved with planning, interpreting, and report safety analyses
Promote good analytical practices, avoid poor practices
Facilitate cross-functional engagement in analytical planning
Excerpt from the Adverse Events White Paper:
The development of standard TFLs and associated analyses will lead to improved standardization of AE data from collection through data display and analysis. The development of standard TFLs will also lead to improved and harmonized product lifecycle management across therapeutic areas by ensuring that reviewers receive clinically relevant and meaningful analyses of patient safety for benefit-risk assessment. This white paper reflects recommendations that would lead to more consistent TFLs, but the recommendations should not be interpreted as “required” by any regulatory agency.
Another purpose of this white paper (along with the other white papers from the project team) is to improve expertise in safety analytics across the multiple disciplines involved with planning, interpreting, and reporting safety analyses. Statisticians can and should assist cross-disciplinary teams to understand and reduce bias in analysis planning and reporting. This assistance is important even when inferential statistics are not used. The potential for biased comparisons is especially a concern when multiple studies are combined (eg, via poor pooling practices for integrated summaries). Safety physicians have relied on qualitative analyses of case reports, looking at individual or small clusters of events. Recently, there has been an increased emphasis on aggregate reviews of safety data. As noted in Section VI of the CIOMS Working Group VI report [1], while medical judgment remains critical in the interpretation of safety data, descriptive and inferential statistical methods can help medical personnel decide whether chance variation is a possible explanation for what is observed or whether it is more likely that some genuine drug effect has occurred. This requires statisticians to increase their engagement and help cross-disciplinary safety management teams to think more quantitatively, This also requires nonstatistical disciplines to obtain a higher level of analytical expertise. For example, all members of a safety management team should understand that improper pooling can lead to biased and even paradoxical results. See Table 10.1 for an illustrative example of paradoxical results and see Section 10.1 for other areas where increased expertise is needed.

6. Vision: Fill the Gap on Analysis and Display Standards
Data Collection Systems
Observed Datasets
Analysis Datasets
Tables, Figures and Listings
Clinical Data
Flow
Trial Design
PRM
SDTM
ADaM
No Standards Exist
Industry
Standards
Alignment
CDASH
A lot of progress has been made with respect to standardization – mostly in the collection and data space. There’s a gap with respect to analyses and displays. CFAST = Coalition for Accelerating Standards and Therapies
(Therapeutic Areas)

8. ADW Project Deliverables
All final white papers from the project team are in the Working Group Deliverables Catalog
White Papers section

9. ADW Project – Wish List for FDA
Identify key contributors/co-authors for each white paper
Medical and statistical reviewers
Approximately 8 hours a month for 6 months
Broad participation in reviews
Medical and statistical reviewers across divisions
Approximately 2 hours
Help find reviewers from other regulatory agencies
Help with action items (follow-up tasks) where relevant
Network with MedDRA MSSO on defining gender-specific terms
Consider crowd-sourcing management of consolidated terms
MSSO=Maintenance and Support Services Organization

10. ADW Project – Wish List for FDA
Communicate the availability of the white papers
The adverse event white paper is included on the PhUSE page in FDA’s website – THANK YOU!
Add link to PhUSE Deliverables Catalog for full list of deliverables?
Are there additional places on the FDA website or in FDA documents to mention the white papers?
Promote following the recommendations in the white papers
White papers can be updated to reflect evolution of thought
Help us avoid conflicting advice
Note: Updated Clinical Review Template could have been a good opportunity
Worst case scenario for our project team: A sponsor follows the recommendations, but asked for different approaches by a medical reviewer (when the request is more of a matter of opinion as opposed to a compound/indication reason to be different)

11. Communication, Promotion, and Education (CPE)
project team
Project Leads – Wendy Dobson, Jared Slain
FDA Liaisons – Mat Soukup, Susan Duke

12. CPE Project Description
The success of working group “Standard Analyses and Code Sharing” relies on the acceptance, input, feedback and further development from/by the user community. A main priority will be to develop a Communication Plan that conceptualizes efficient ways to communicate working group progress and results, e.g. white papers, and the call for scripts. It will define target groups, timing, communication channels, and the presentation.
Project Description as provided on the wiki page for the project

13. CPE Project Goals/Ideas
Facilitate communication of deliverables
Improve navigation to information
Facilitate presentations/posters at multiple conferences
Keep wiki up-to-date
Facilitate creation of educational video clips
Study-size adjusted percentages completed
Help define places to mention deliverables
Write journal article(s)
Create a Safety Analytics site
Host a Safety Analytics workshop
Crowd-source educational library
Communications, Promotion, Education
High priorities
Help other projects with communication around their final deliverables
Improve navigation from PhUSE home page
More presentations/posters at multiple conferences
Keep wiki up-to-date
Key educational video clips (define high vs medium)
Links from FDA sites/documents/training
Write a journal article (need to define focus)
Medium priorities
Develop thorough/targeted communication plan
Improve index ranking
Create a Safety Analytics site (likely with ASA/DIA)
Safety Analytics Workshop - record and post (likely with ASA/DIA)
Crowd-source educational library (FDA/sponsors share what they have?) (likely with ASA/DIA)
Additional education videos (define high vs medium)
Work with ADW project on any additional ideas to improve visibility, especially with Safety Analytics
Low priorities
Work with ADW project on improving review process
Write additional journal articles

14. CPE Project – Wish List for FDA
Help with communication/navigation
Consider more sessions at professional meetings where PhUSE work can fit
Help with crowd-sourcing educational information
Primarily interested in safety analysis educational information
Share existing educational material
Help create missing material
Note: Appears we have the journal article covered for FDA co-authors (Mat, Susan)

15. Code Sharing (Repository)
project team
Project Leads – Hanming Tu, TBD
FDA Liaison – Mat Soukup

16. Code Sharing Project Description
Establish and maintain a collaboration platform for leveraging crowd-sourcing to improve the content and implementation of analyses for medical research and leading to better data interpretations and increased efficiency in the clinical drug development and review processes.

17. Background: Open Tool Box
Vision: From Everyone Building Their Own Tools to Shared Tools
Shared Value
Text
Shared tools

18. Overview: Script Repository
Vision: Collaboration platform and Shared Reusable Code Library
FDA
Industry
Academia
Script Repository in Github
Script Metadata
Test Data
White Paper Scripts
SAS, R, Spotfire, Etc.
Contributed Scripts
Code Test Environment

19. Code Sharing Project Goals
Establish and maintain a publicly available repository for storing program codes to be used as analytical tools for medical research
Develop guidance on managing repositories and script metadata for sharing scripts
Improve frontend for accessing and searching the repository
Review contributed scripts such as FDA scripts
Develop scripts based on white papers

20. Code Sharing – Wish List for FDA
Continue to share code
Some code shared, but we understand there is more that may make sense to share
Use code from the repository
FDA representatives stay aware of what has been contributed or developed and leverage where possible
Help identifying areas where crowd-sourcing the creation of a shared tool makes sense
Interactive safety review?
Reduce some common errors in e-submission?