Presentation is loading. Please wait.

Presentation is loading. Please wait.

Statistically Based Validation Acceptance Criteria

Similar presentations

Presentation on theme: "Statistically Based Validation Acceptance Criteria"— Presentation transcript:

1 Statistically Based Validation Acceptance Criteria
Mark Varney Statistics Program Manager Abbott Quality and Regulatory Abbott Park, IL

2 FDA Process Validation Guidance, Jan 2011
Statistics mention 15 times “statistical” “statistics” “statistically” “statistician” – as a suggested team member Clear that FDA expects more statistical thinking in validation Some statisticians asked to be a team member may not be familiar with Quality Assurance applications and jargon Acceptance Sampling Statistical Process Control (SPC) Process Capability 2

3 FDA Process Validation Guidance Overview
Process Validation: The collection and evaluation of data, from the process design stage through commercial production, which establishes scientific evidence that a process is capable of consistently delivering quality product. 3

4 FDA Process Validation Guidance Overview
The new Guidance specifies a lifecycle approach: Stage 1 – Process Design Statistically designed experiments (DOE) Stage 2 – Process Qualification Design of facility and equipment/utilities qualification Process Performance Qualification (PPQ) SPC; Variance components; Acceptance Sampling; CUDAL, etc. Number of lots required is no longer specified as three Must complete Stage 2 before commercial distribution Stage 3 – Continued Process Verification (CPV) Continual assurance the process is operating in a state of control Data trending, SPC, Acceptance Sampling, etc. Guidance recommends scrutiny of intra- and inter-batch variation 4

5 Statistical Acceptance Criteria for Validation
Provide X% confidence that the requirement has been met Requirements: Process performance to consistently meet attributes related to identity, strength, quality, purity, and potency Statistical confidence required may be based on… Risk Scientific knowledge Criticality of attribute (AQL, etc.) Prior / historical knowledge Stage 1 knowledge Revalidation Is test an abuse test? 5

6 Three Common Situations
Provide statistical confidence that… A high percent of the population is within specification A population parameter is within specification - Mean; Standard Deviation; RSD; Cpk/Ppk A standard test (UDU, Dissolution, etc.) will pass 6

7 Assure a High % of Population in Spec
“90% confidence at least 99% of population meets spec” “90% confidence nonconformance rate <1%” “90% confidence for 99% reliability” Common statistical methods Continuous data: Normal Tolerance Interval Discrete data: High Confidence Binomial Sampling Plan 7

8 A Word about Confidence…
Which sampling plan provides more confidence? n=90, accept=0, reject=1 n=300, accept=0, reject=1 n=2300, accept=0, reject=1 99% confidence ≥95% conforming 95% confidence ≥99% conforming 90% confidence ≥99.9% conforming What you want to be confident of is usually more important than how confident you want to be 8

9 Validation: High Degree of Assurance
Phrase “high degree of assurance” mentioned four times “…the PPQ study needs to be completed successfully and a high degree of assurance in the process achieved before commercial distribution of a product.” ICH Q7A GMP for APIs: “A documented program that provides a high degree of assurance that a specific process, method, or system will consistently produce a result meeting pre-determined acceptance criteria.” Suggest 90% or 95% confidence is acceptable This confidence is more related to Type II error and Power Although α=0.05 / 95% confidence is common for Type I error, it is not as common for power, where 80% and 90% also common. 9

10 Assure a High % is Within Spec
Variables data: Normal Tolerance Interval* Example: Show with 90% confidence that at least 99.6% of powdered drug fill weights meet spec of mg. Test n=50 bottles; 1 every 5 minutes for 4 hrs Acceptance criterion: 90% confidence ≥99.6% meet spec Variables data with average, s.d.: use tolerance interval method must be within specification limits Why 99.6%? Production AQL is 0.4% for fill weight. *other methods may be used, such as variables sampling; may give lower Type I error 10

11 Fill Weight Example 11

12 Fill Weight Example Process is in statistical control, normality not rejected 90% confidence / 99.6% coverage tolerance interval: Pass: Tolerance interval lies within spec of We can be 90% confident ≥99.6% of containers meet spec Will be able to pass in-process 0.4% AQL sampling If process is stable, 90% confidence ≥95% of lots will pass Engineer friendly: tables or software can be used 12

13 2-Sided Normal Tolerance Interval Factors
Mean ± 3.35s must be within spec limits 13

14 1-Sided Normal Tolerance Interval Factors

15 A problem with most normality tests
Need to check for normality to use normal tolerance interval Process quality data is often rounded Or data is “granular” Most normality tests will interpret rounding as non-normality Example: n=100 from N(100,1.52) Unrounded Rounded 100 98.238 98 99.122 99 99.190 101 … , n=100 15

16 A problem with most normality tests
Unrounded data: normality not rejected by Anderson-Darling test 16

17 A problem with most normality tests

18 A problem with most normality tests
Rounding data causes most normality tests to fail SAS 9.2 Proc Univariate Tests: Unrounded Data Test Statistic p Value------ Shapiro-Wilk W Pr < W Kolmogorov-Smirnov D Pr > D >0.1500 Cramer-von Mises W-Sq Pr > W-Sq >0.2500 Anderson-Darling A-Sq Pr > A-Sq >0.2500 Rounded Data (to whole numbers) Shapiro-Wilk W Pr < W Kolmogorov-Smirnov D Pr > D <0.0100 Cramer-von Mises W-Sq Pr > W-Sq <0.0050 Anderson-Darling A-Sq Pr > A-Sq <0.0050 OK Reject normality 18

19 A problem with most normality tests
Two normality tests not substantially affected by granularity Ryan-Joiner test (Minitab 16) Omnibus skewness/kurtosis test For more information, see Seier, E. “Comparison of Tests for Univariate Normality.” 19

20 Confidence for Conformance Proportion
Usual 2-sided normal tolerance interval controls both tails This can present a problem for an uncentered process 20

21 Estimation for Conformance Proportion
Example: Removal Torque, Spec = 5.0 – 10.0 in-lbs 95% conf / 99% coverage tolerance interval: (4.85, 8.62) FAILS 21

22 Estimation for Conformance Proportion
Usual 2-sided normal tolerance interval controls both tails This can present a problem for an uncentered process Can use estimation for proportion conforming Also called bilateral conformance proportion Reduce probability of failing for uncentered processes Similar method used by ANSI Z1.9 for routine production sampling 22

23 Estimation for Conformance Proportion
Estimation for Conformance Proportion for Removal Torque: 95% confidence ≥ 99.07% conforming: PASS Lee, H., and Liao, C. “Estimation for Conformance Proportions in a Normal Variance Components Mode.” Journal of Quality Technology, Jan., 2012. Taylor, W. Distribution Analyzer, version 1.2. 23

24 Random Effects Process Tolerance Limits
Overall process tolerance limits may be constructed to take between-lot variation into account Example: Impurities n Mean StDev Min Max Lot Lot Lot Approx 90% confidence / 95% coverage tolerance interval1: Usual 90/95% tolerance interval for all data combined: 0.07 Appears conservative 1Krishnamoorthy, K. and Mathew, T. “One-Sided Tolerance Limits in Balanced and Unbalanced One-Way Random Models Based on Generalized Confidence Intervals.” Technometrics, Vol. 46, No. 1, Feb 24

25 What is an AQL? AQL = "Acceptance Quality Limit“
The quality level that would usually (95% of the time) be accepted by the sampling plan RQL = "Rejection Quality Limit“ The quality level that will usually (90% of the time) be rejected by the sampling plan Also called LTPD (Lot Tolerance Percent Defective) Also called LQ (Limiting Quality) 25

26 AQL / RQL AQL = 0.72% RQL = 7.56% AQL: Pr(accept)=0.95
RQL: Pr(accept)=0.10 AQL = 0.72% RQL = 7.56% 26

27 What is an AQL? Can be cast as a hypothesis test or confidence interval For routine acceptance sampling… Ho: p ≤ Assigned AQL H1: p > Assigned AQL α=0.05, “accept” lot if Ho not rejected But for validation… Ho: p > Assigned AQL H1: p ≤ Assigned AQL or desired performance level α =1-confidence; i.e., = .10 for 90% confidence Pass validation if Ho rejected 27

28 Typical AQLs in Pharma / Medical Devices
Product Attribute Typical Assigned AQLs Critical 0.04%, 0.065%, 0.1%, 0.15%, 0.25% Major Functional 0.25%, 0.4%, 0.65%, 1.0% Minor Functional 0.65%, 1.0%, 1.5% Cosmetic Visual 1.5%, 2.5%, 4%, 6.5% For validation, suggest 90% confidence that process ≤ assigned AQL Why 90%? Traditional probability used for RQL/LTPD/LQ This means for validation the assigned AQL is treated as an RQL If nonconforming rate is at AQL, will fail validation 90% of the time Selection of the AQL more important than confidence selected Much tighter than ANZI Z1.4/Z1.9 tightened (10-20% confidence) 28

29 Assure high % within spec: attributes data
Attributes data is binomial pass/fail data Example: n=230, a=0 provides 90% confidence ≥ 99% conforming; 90% confidence ≤1% nonconforming 29

30 Attributes Example for AQL: Fill Volume PPQ
Production assigned AQL is 1.0% AQL = “Acceptance Quality Limit” Assigned based on risk assessment If process is better than AQL, almost all mfg lots will be accepted Validation: Show with 90% confidence that the process produces ≤1.0% nonconforming units Multi-head filler; we know data are non-normal 90% confidence ≥99% are in spec Medical devices: 90% confidence for 99% “reliability” Assures that future AQL production sampling can be passed If process is at the AQL, ~95% of lots will pass AQL sampling 30 30

31 Example: Fill Volume Attributes plans: 90% confidence ≤1.0% nonconforming If the validation sampling plan passes… We have 90% confidence the nonconforming rate is ≤1.0% ANSI Z1.4 plans provide far less than 90% confidence Normal sampling: typically about 5% confidence Tightened: typically about 15% confidence Note: RQL=“Rejection Quality Limit” Also called LTPD (Lot Tolerance Pct Defective) or LQ (Limiting Quality) Sampling Plan RQL0.10 n=230, acc=0, rej=1 1.0% n=390, acc=1, rej=2 n1=250, a1=0, r1=2 n2=250, a2=1, r2=2 Z1.4 normal: n=80, acc=2 Z1.4 tightened: n=80, acc=1 31 31

32 PV Acceptance Criteria for Attribute Types
Comment AQL attributes Fill volume Tablet defects Extraneous matter, etc. ≥90% confidence that Nonconformance rate ≤ assigned AQL Non-AQL attributes Dissolution / UDU / Batch Assay Other tests ≥90% confidence that… USP test will be met ≥95% of the time ≥99% of results in spec (critical) ≥95% of results in spec (non-critical) Statistical Parameters Mean / sigma / RSD(CV) Cpk, Ppk Mean / sigma / RSD in spec Ppk ≥1.0, 1.33 or related to % coverage No within batch variation expected pH of a solution Label copy text Statistics not usually necessary May consider 3X-10X testing Assess between lot variation 32 32

33 Show Population Parameter Meets Spec
Show confidence interval for parameter in spec Example: API mean potency; spec of n=30 test results (3 from each of 10 drums) 95% C.I. for mean is traditional 95% C.I. for mean is – ; pass. Also need to analyze data across drums! 33

34 Process Capability/Performance Statistic Ppk
Measures process capability of meeting the specifications σLT is long-term sd, usual formula, includes variation over time; Cpk uses short-term estimate of sd 34

35 Ppk vs Percent Nonconforming
0.6 7.2% 0.7 3.6% 0.8 1.6% 0.9 0.69% 1.0 0.27% 1.1 0.10% 1.2 0.03% 1.3 0.01% 1.5 0.0007% 2.0 % In statistical control Normally distributed Centered in spec If1-sided: half % shown 35

36 Example: Show Ppk Meets Requirement
Provide 90% confidence process Ppk≥1.3 n=15 assay tests were obtained across each of 3 PPQ lots No significant difference in mean/variance in the 3 lots; pool data? Pass 36 36

37 Pooling OK if Process in Statistical Control

38 Intra-batch and Inter-batch Variation
Intra=Within batch: σw Inter=Between batch: σb 38

39 Special Cause Variation

40 Ppk if Process is Not in Statistical Control
Use of Ppk is controversial if process not in statistical control1 “Ppk has no meaningful interpretation” “statistical properties are not determinable” “a waste of engineering and management effort” Note: between-batch variation means not in classic statistical control If variance components model holds, estimate Ppk with σw and σb Usual confidence intervals from standards/software not applicable Confidence interval must take degrees of freedom for σb into account Difficulty in proving variance components model assumptions with small number of lots 1 Montgomery, Introduction to Statistical Quality Control 6th edition, p 363 40

41 Potential problems with Ppk over multiple lots
Usual Ppk confidence interval assumes normal distribution and process stable / in statistical control Any changes/trends within or between lots invalidates assumption Often differences in mean between batches Usual Ppk C.I. does not consider variance components Example: 30 samples from each of 5 lots (30-1)x5 = 145 degrees of freedom for within lot variation (5-1) = 4 degrees of freedom for between lot variation ASTM reference E2281 does not address this Alternative: Show Ppk for each lot meets requirement for 3 lots: ~87% overall confidence median process Ppk meets spec 4 lots: ~94% confidence 5 lots: ~97% confidence Or calculate a modified Ppk based on variance components C.I. 41 41

42 Example: Ppk if process not in statistical control
Plot your data! Ppk=1.77; lower 95% C.I. for Ppk using Minitab is 1.60. But should PPQ pass? Scientific understanding of trend? 42

43 Assure a Standard Test will Pass
Example: Uniformity of Dosage Units (Content Uniformity) Requirement: Pass USP‹905› Uniformity of Dosage Units ≥90% confidence USP test would be passed ≥95% of the time (coverage) See Bergum1 for specifics to determine acceptance criteria Why 90% confidence? Comparable to RQL probability. Why 95% coverage? Comparable to AQL probability for single test. Bayesian approach also available2 1Bergum, J. and Li, H. “Acceptance Limits for the New ICH USP 29 Content-Uniformity Test”, Pharmaceutical Technology , Oct 2, 2007 2Leblond, D., and Mockus, L. “Posterior Probability of Passing a Compendial Test.” Presented at Bayes-Pharma 2012, Aachen, Germany. 43

Download ppt "Statistically Based Validation Acceptance Criteria"

Similar presentations

Ads by Google