1. FDA and Pharmaceutical Manufacturing Research Projects Jeffrey T. Macher Jackson A. Nickerson Co-Principal Investigators September 30, 2004

2. Presentation Overview  FDA Research Project status report  History  Project Goals  Approach and Expected Outcomes  Current Status  Pharmaceutical Manufacturing Research Project (PMRP) status report  History  Project Goals  Approach and Expected Outcomes  Current Status

3. FDA Research Project

4. FDA Research Project History  Research project idea emerged in Fall, 2001.  Observed significant increases in both number and severity of cGMP violations.  Approached FDA in Spring, 2002.  Dozens of meetings with CBER, CDER, ORA and district offices.  Formalized relationship with FDA in Fall, 2003.  Signed Material Transfer Agreement (MTA) in October, 2003.  Received data September, 2004.

5. FDA Research Project Goals  Risk-based assessment of FDA cGMP outcomes.  Identify attributes (currently recorded by the FDA) that impact inspection outcomes.  Transfer “learning” to FDA.

6. FDA Project Approach  Compile and link FDA databases.  Estimate the likelihood of various outcomes:  NAI, VAI, OAI; Warning Letters; Field Alerts; Product Recalls.  based on…  compound/product, facility, firm, FDA district, investigator and training derived factors.  in order to …  evaluate the allocation of investigational resources.  inform effectiveness of investigator training and management.

7. FDA Databases  COMIS (Supplement filings)  DQRS (Field alerts)  EES (Outsourcing)  FACTS (Inspections)  Product Listing  Product Recalls  Product Shortages  Registration  Warning letters  Training

8. Some basic “facts” about the FDA data  Years covered: 1990-2003.  Total number of facilities inspected: 3753.  Total number of “Inspections”: 38,341.  Total number of “Facility visits”: 14,162.

11. FDA-related Performance Outcomes  Inspection outcomes:  NAI  VAI  OAI »Warning letters  Field reports  Product recalls  Product availability

12. FDA-related Key Factors  Product-level and Process-level variables  NDA/ANDA.  Prescription/non-prescription.  Product class.  Product subclass.  Process indicator code.  Supplement history.  Extent of vertical integration.  History of regulatory outcomes for the product.  History of regulatory actions for related products.

13. FDA-related Key Factors  Facility-level variables  Age and Size.  Number of products produced.  Diversity of products produced: »Prescription/non-prescription; Product class; Product subclass; Process indicator code.  Increases/decreases in number of products and diversity of production n years prior.  Time since last ownership change.  Regulatory history (e.g., inspectional outcomes, warning letters and type).

14. FDA-related Key Factors  Firm-level variables  Age and Size.  Number of manufacturing locations.  Number of products produced.  Diversity of products produced.  Number of products introduced in past n years.  Number of NAI, VAI, and OAI at non-focal facilities in the past n years.  Number and type of warning letters at non-focal facilities in the past n years.

15. FDA-related Key Factors  FDA related variables  FDA District Identifier and annual allocated inspection hours.  Inspections »Length of cGMP inspection; Number of investigators; Reason for inspection; Time since last inspection.  Investigators »Annual experience of Pharma versus non-Pharma inspections. »Training: number and frequency of formal training classes.  Policy shifts (e.g., SUPACs)

17. Expected Outcomes of Analysis  Statistical analysis will estimate the probability of various outcomes based on counterfactuals.  Counterfactual analysis allows a risk-based assessment of changes in at least some FDA-related oversight policies.  Results will improve understanding of certain inspection outcomes based on attributes examined.  Risk assessments (in terms of increasing probability of occurrence) will be used to:  Inform oversight choices of FDA.  Identify underlying quality of manufacturers’ production and regulatory processes.

18. Status of FDA Research Project  Phase 1: Exploratory pilot study.  Completed Summer 2003.  Phase 2: Data collection  Complete for CDER and awaiting transfer.  Incomplete for CBER.  Phase 3: Data analysis  Data received.  Analysis underway (~2-3 months for initial findings).

19. Pharmaceutical Manufacturing Research Project (PMRP)

20. PMRP Project History  Research project idea emerged in Fall, 2001.  Explore whether cGMP violations related to managerial, organizational and technical practices.  Attempt to translate lessons learned from similar research study of manufacturing in semiconductor industry.  Began interviewing manufacturers in Spring, 2002.  Dozens of interviews with pharmaceutical, biotechnology, API, and contract manufacturers.  Launched internet-based questionnaire in Fall, 2003.  Went live in November, 2003.  Ongoing marketing and participation solicitation since.  First round closed but for a few firms finishing off the survey.

21. PMRP Goals  Develop standard set of benchmarks for measuring manufacturing and regulatory performance.  Identify managerial, organizational, and technical practices underlying manufacturing and regulatory performance.  Provide confidential “scorecard” to manufacturing facilities on how they perform against anonymous others.

22. PMRP Approach  Develop focused questionnaire of potential factors that influence manufacturing and regulatory performance.  Administer questionnaire over internet on secure-site.  Analyze data collected using a variety of econometric techniques.  Provide summary of findings and facility scorecard to participating manufacturers.

23. PMRP Database  Secured participation from cross section of U.S. and EU manufacturers.  21 firms.  58 manufacturing facilities.  Online survey with each manufacturing facility providing detailed data on between 1 and 5 compounds.

24. PMRP Performance Outcomes  Manufacturing Performance  Theoretical / Actual yield.  Batches started / failed.  Cycle time.  Regulatory Performance.  Field alerts/Biologic deviation reports.  Warning letters.  Consent decrees.  Deviations.  Supplement approval.

25. PMRP-related Key Factors  Company / SBU  Financial information.  Demographic information.  Manufacturing Facility  Financial information.  Demographic information (size, location, age, employees, etc.).  Product information (number, type, etc.).  Regulatory inspection information.  Extent of Outsourcing (development, manufacturing, APIs, etc.).

26. PMRP-related Key Factors  Product and Process Development  Location.  Organization.  Timing.  Human Resource Management  Appraisal, Promotion, Mobility, Demographics, Training, etc.  Extent and use of teams.  Deviation and Supplement Management  Extent and use of information technology.  Extent and use of Process Analytic Technology (PAT).  Organization.

27. Status of PMRP Project  Phase 1: Exploratory pilot study  Completed Summer, 2003.  Phase 2: Data collection  First round closed (but for a few stragglers).  Phase 3: Data analysis  Analysis will require 3-6 months.  Statistical and econometric analysis of data.  Final reports.