An Assessment of IPAC-RS’ Proposal Walter W. Hauck, Ph.D. Biostatistics Section Division of Clinical Pharmacology Thomas Jefferson University Philadelphia, PA, USA

2 How Assessed Does the IPAC-RS proposal address the issues raised regarding the FDA draft guidance?Does the IPAC-RS proposal address the issues raised regarding the FDA draft guidance? Do the details of the IPAC-RS proposal support their claims?Do the details of the IPAC-RS proposal support their claims?

3 FDA Draft Content Uniformity Standard (in part) FDA Draft Content Uniformity Standard (in part) Tier 1, N=10 containers, one dose per container Tier 1, N=10 containers, one dose per container Accept if : Accept if : 0 or 1 outside 80%-120% of labeled claim (LC) 0 or 1 outside 80%-120% of labeled claim (LC) none outside 75%-125% of LC none outside 75%-125% of LC If not accepted after Tier 1: If not accepted after Tier 1:

4 FDA Draft Content Uniformity Standard (in part), cont. Tier 2, N=20 additional containers (30 total), one dose per containerTier 2, N=20 additional containers (30 total), one dose per container Accept if: 0-3 outside 80%-120% of LC none outside 75%-125% of LCAccept if: 0-3 outside 80%-120% of LC none outside 75%-125% of LC Also requires sample mean within 85%- 115% of LC at each tier.Also requires sample mean within 85%- 115% of LC at each tier.

5 Structure of Criterion An inner interval with a count on maximum allowable number of units outside the interval (test by attributes)An inner interval with a count on maximum allowable number of units outside the interval (test by attributes) An outer interval with a “zero tolerance” criterion (“safety net”)An outer interval with a “zero tolerance” criterion (“safety net”) Additional criteria; i.e. limits on the sample meanAdditional criteria; i.e. limits on the sample mean

6 Issue #1 Acceptance criteria are of the form of a statistical hypothesis test, but there are no hypothesesAcceptance criteria are of the form of a statistical hypothesis test, but there are no hypotheses i.e., there is no specification of what constitutes an acceptable batch, only of what constitutes an acceptable samplei.e., there is no specification of what constitutes an acceptable batch, only of what constitutes an acceptable sample

7 What Does IPAC-RS Do? Proposal specifies an acceptable batch as -- at least 85% of the batch falls within 75%/125% of labeled claimProposal specifies an acceptable batch as -- at least 85% of the batch falls within 75%/125% of labeled claim

8

9 Some Clarification Consumer risk (false positive decision) here refers to passing a batch that lies outside the specifications of an acceptable batch.Consumer risk (false positive decision) here refers to passing a batch that lies outside the specifications of an acceptable batch. Producer risk here refers to failing to pass a batch that is as good or better than the specifications of an acceptable batchProducer risk here refers to failing to pass a batch that is as good or better than the specifications of an acceptable batch Both depend strongly on the specification of  85% within 75%/125% of labeled claimBoth depend strongly on the specification of  85% within 75%/125% of labeled claim

10

11 Comments Setting an acceptance criterion such as 85% within 75%/125% of labeled claim should mean that any batch that actually falls in the region is acceptableSetting an acceptance criterion such as 85% within 75%/125% of labeled claim should mean that any batch that actually falls in the region is acceptable Batch failure rate for acceptable batches is in control of sponsorBatch failure rate for acceptable batches is in control of sponsor Need to distinguish properties of batch from properties of sampleNeed to distinguish properties of batch from properties of sample

12

13 Issue #2 By specifying the sample size, the draft guidance denies the sponsor the opportunity to control their own producer risk (the probability that a batch that is acceptable will not pass)By specifying the sample size, the draft guidance denies the sponsor the opportunity to control their own producer risk (the probability that a batch that is acceptable will not pass)

14 What Does IPAC-RS Do? Proposal provides a choice of two-tier designs of varying sample sizes, all intended to control false positive rate at 5%Proposal provides a choice of two-tier designs of varying sample sizes, all intended to control false positive rate at 5% Restriction to two tiers is not necessary; their approach for tolerance intervals could also apply with alternate choices of number of tiers (e.g., 1 or 3)Restriction to two tiers is not necessary; their approach for tolerance intervals could also apply with alternate choices of number of tiers (e.g., 1 or 3)

15 Comments As long as any batch that meets set criteria is acceptable, then any sample size acceptable to the sponsor should be acceptable to the AgencyAs long as any batch that meets set criteria is acceptable, then any sample size acceptable to the sponsor should be acceptable to the Agency Sample size and number of tiers would need to be prespecifiedSample size and number of tiers would need to be prespecified

16 Issue #3 Test by value (parametric tolerance intervals) makes better use of the data than test by attributes (current approach)Test by value (parametric tolerance intervals) makes better use of the data than test by attributes (current approach)

17 What Does IPAC-RS Do? Based on parametric tolerance intervalsBased on parametric tolerance intervals Modified to reduce a statistical conservatism present with tolerance intervals [by adding the upper limit, 25f/k, on the sample standard deviation, S, and decreasing k.]Modified to reduce a statistical conservatism present with tolerance intervals [by adding the upper limit, 25f/k, on the sample standard deviation, S, and decreasing k.]

18 Issue #4 The outer acceptance interval, a “zero tolerance criterion,” becomes a substantial producer risk as the sample size increasesThe outer acceptance interval, a “zero tolerance criterion,” becomes a substantial producer risk as the sample size increases There is a conflict between presence of a zero tolerance criterion and flexibility in choice of sample sizesThere is a conflict between presence of a zero tolerance criterion and flexibility in choice of sample sizes

19 What Does IPAC-RS Do? The zero tolerance criterion is droppedThe zero tolerance criterion is dropped There is a need to be comfortable with this, but the zero tolerance criterion does appear to have offered little extra protectionThere is a need to be comfortable with this, but the zero tolerance criterion does appear to have offered little extra protection

20 Issues, Summary Yes, the IPAC-RS proposal does address issues raised regarding the criterion in the FDA draft guidance and of other proposed criteriaYes, the IPAC-RS proposal does address issues raised regarding the criterion in the FDA draft guidance and of other proposed criteria

21 IPAC-RS Claim Consumer risk is retained or improved compared to the FDA draft criterion while at the same time reducing producer riskConsumer risk is retained or improved compared to the FDA draft criterion while at the same time reducing producer risk

22

23 IPAC-RS Delivers? YES. How is this possible? 1Parametric tolerance intervals will reduce producer risk relative to nonparametric approaches for a given sample size and allowed level of consumer risk 2Elimination of zero tolerance criterion 3Larger sample sizes

24 How Possible, cont. FDA’s draft proposal is more liberal than it appearsFDA’s draft proposal is more liberal than it appears IPAC-RS shows FDA draft corresponds implicitly to at least 78% coverage (single-dose) or 85% (multi-dose) within 75%/125% of LCIPAC-RS shows FDA draft corresponds implicitly to at least 78% coverage (single-dose) or 85% (multi-dose) within 75%/125% of LC

25 Summary IPAC-RS’ report delivers as claimedIPAC-RS’ report delivers as claimed Statistical approach is an improvement over any other criterion being consideredStatistical approach is an improvement over any other criterion being considered There is a need to be careful in choice of constants (k’s and f) to be sure that consumer risk is properly controlledThere is a need to be careful in choice of constants (k’s and f) to be sure that consumer risk is properly controlled

26

27

28 Costs to Sponsors Increased by larger sample sizesIncreased by larger sample sizes Reduced for multi-dose products by combining through-container-life criterion into dose uniformity criterionReduced for multi-dose products by combining through-container-life criterion into dose uniformity criterion Reduced, potentially, for all by giving control of study design (and hence of producer risk) to sponsorReduced, potentially, for all by giving control of study design (and hence of producer risk) to sponsor

29 Bottom Line 85% within 75%/125% can now be the focus of discussion (as it should be)85% within 75%/125% can now be the focus of discussion (as it should be)