Presentation is loading. Please wait.

Presentation is loading. Please wait.

Analysis of Stability Data with Equivalence Testing for Comparing New and Historical Processes Under Various Treatment Conditions Ben Ahlstrom, Rick Burdick,

Similar presentations


Presentation on theme: "Analysis of Stability Data with Equivalence Testing for Comparing New and Historical Processes Under Various Treatment Conditions Ben Ahlstrom, Rick Burdick,"— Presentation transcript:

1 Analysis of Stability Data with Equivalence Testing for Comparing New and Historical Processes Under Various Treatment Conditions Ben Ahlstrom, Rick Burdick, Laura Pack, Leslie Sidor Amgen Colorado, Quality Engineering May 19, 2009

2 2 Agenda 1.Purpose of comparability for stability data 2.Problems with the p-value approach 3.Equivalence approach and acceptance criteria methods 4.Example

3 3 Example Data Packaging Data (Chow, Statistical Design and Analysis of Stability Studies, p. 116, Table 5.6)  2 package types (Bottle, Blister)  10 lots (5 for each package type)  6 time points (0 to 18 months)

4 4 Comparability Analysis for Stability Data  Purpose –Compare the rates of degradation  P-value Analysis Steps –Fit the regression lines (process*time interaction) –Calculate p-value for process*time –Compare p-value to  =0.05 –Draw conclusion about comparability pass (comparable) if p-value > 0.05 fail (not-comparable if p-value < 0.05) I.E.: Evaluate the slopes of the treatment conditions

5 5 P-value Analysis to Evaluate Comparability for Stability Data Bottle vs. Blister: Are the processes comparable?

6 6 P-value Approach  Hypotheses –H 0 : slopes are comparable –H A : slopes are not comparable  If p-value < 0.05, reject H 0  If p-value >0.05, fail to reject H 0 –Does not imply they are comparable, but rather that there isn’t enough evidence to say the slopes are different

7 7 P-value Analysis to Evaluate Comparability for Stability Data Packaging: Bottle vs. Blister Do we pass or fail the p-value test? We compare the slopes using p-values (Pass if p-value > 0.05 and Fail if p-value < 0.05) Pass: p=0.8453

8 8 Problems with P-value Approach  Reporting a P-value only tells us something about statistical significance. –A statistically significant difference in slopes does not necessarily have any practical importance relative to patient safety or efficacy. –P-values are non-informative because they do not quantify the difference in slopes in a manner that allows scientific interpretation of practical importance. –A p-value approach provides a disincentive to collect more data and learn more about a process.

9 9 Equivalence Testing Method 1.Fit the model with all historical and new process data (includes different storage conditions, orientations, SKU’s, container types) 2.Compute the difference in slopes for the desired comparison  Bottle vs. Blister 3.Compute the 95% one-sided confidence limits around the difference observed over the time frame of interest 4.If the confidence limits are enclosed by the equivalence acceptance criteria, conclude that the historical and new processes are comparable

10 10 Statistical Model  Parameters  i and β i are the overall regression parameters for the i th process  Random variables a j allow the intercepts to vary for each lot  is the time value for process i, lot j, and time k.  Model can be extended to more levels

11 11 Statistical Equivalence Acceptance Criteria (EAC) Goal Post is the space of expected historical performance Football = 95% one-sided CLs around difference between slopes over time frame of interest

12 12 Methods to Calculate Equivalence Acceptance Criteria (EAC)  Equivalence Acceptance Criteria (EAC) provide a definition of practical importance  The scientific client has the responsibility to determine a definition of practical importance (based on science, safety, specification, reg. commit., etc.)  Statistical methods can help establish a starting point for these decisions  Three statistical methods include: –Method 1: Common cause variability –Method 2: Excursion from Product Specification –Method 3: Historic Variability of Slope Estimates

13 13 3 Statistical Approaches for Defining EAC Method 1Method 2Method 3 EAC based on common cause variability of the historic process EAC based on product specification EAC based on historic variability of slope estimates -EAC is expressed as average change in response per month -Requires a specification -EAC is expressed as average change in response per month -Requires at least 3 different lots in historic data set -EAC is expressed as change response per month

14 14 Comparability in Profile Data Reference condition Time (months) Quality attribute 0 T Difference between intercepts t = 0 Total difference between conditions at time T (intercept and slope) A B Difference in response averages attributed to the difference in slopes B – A = δ New condition B-A

15 15 EAC Method 1: Common Cause Variability  Criteria is based on historical performance at various conditions Lot to Lot variability Measurement variability Multiplier aligned with other statistical limits used to separate random noise from a true signal Goal Post is the space of expected historical performance

16 16 EAC Method 1: Common Cause Variability T = Expiry = 18 months

17 17 Percent Label Claim, P-value approach vs. Equivalence Test P-valueEquivalence Slope Bottle Slope Blister P-value NA Slope difference over 18 months NA Goal Post NA +/ ResultPASS Key Point Slope estimates are the same for both approaches Difference in Slopes Equivalence graph

18 18  Maximum allowable difference in slopes where new and historic have < p% excursion rate at expiry  Typically p=0.01, 0.025, 0.05  Use historic data  Relates comparability to specification EAC Method 2: Product Specification

19 19 EAC Method 2: Product Specification Spec (LSL) K  Hist  New E (Expiry) Mean of historical at expiry Response 0 Time (months) Pth lower percentile centered at historic mean where P is probability of excursion Pth lower percentile centered at new mean Acceptable difference in slopes is  = K/E.

20 20 EAC Method 2: Product Specification  K is unknown, so replace term in brackets with lower one-sided (1-P)*100% individual confidence bound based on historical (prediction bound)  Assume Lower Spec Limit (LSL) = 95  Expiry = 18 months

21 21 EAC Method 3: Historic Slope Variability  Use historical data for calculation  Historical dataset provides n H independent estimates of the common slope β  EAC based on 99.5 th percentile of distribution of difference in slopes from same lot.  If observed slope difference is consistent with this variability, equivalence is demonstrated.

22 22 EAC Method 3: Historic Slope Variability ^ ^ ^

23 23 EAC Method 3: Historic Slope Variability  θ 3 is the 99.5 th percentile of the distribution of  is the 99.5 th percentile of the standard normal distribution  U is a 95% upper bound on the standard error for an estimate of β based on a single lot

24 24 Comparison of Equivalence Acceptance Criteria  Hard for a client to know what a difference in slopes of, say, 0.1 % looks like in a table  Once client sees graph, they can get a feel for what a difference in slope means  Can visualize what the possible range of regression lines could be to still claim equivalence

25 25 Comparison of Equivalence Acceptance Criteria  Based only on historical data  Graph is created before data for the new process is collected

26 26 Results by Method  H A : Show δ is less than some amount deemed practically important  Equivalence is demonstrated by computing two one-sided tests (TOST)  If the 95% lower one-sided confidence bound on δ is greater than -θ and the 95% upper one- sided confidence bound is less than θ, then equivalence is demonstrated

27 27 P-value Approach vs. Equivalence Approach P-value Approach H o : slopes are comparable H A : slopes are not comparable  P-value Statistical convention is to have research objective in H A Equivalence Approach H o : slopes are not comparable H A : slopes are comparable  Equivalence acceptance criteria set a priori  Based on interval estimates of slope difference using mixed regression model with random lots

28 28 Summary  P-value approach to comparability has numerous issues –High p-values do NOT prove equivalence –High p-values only indicate that there is NOT enough evidence to conclude slopes are different –At times, leads to ad hoc analysis requests when p-value is small –P-values sensitive to sample size  Goal posts allow you to state equivalence –Industry is moving in the direction of equivalence tests  Can be extended to accelerated studies Move to Equivalence Testing for Comparability

29 29 References  Limenati, G. B., Ringo, M. C., Ye, F., Bergquist, M. L., and McSorley, E. O. (2005). Beyond the t-test: Statistical equivalence testing. Analytical Chemistry, June 2005, pages 1A-6A.  Chambers, D., Kelly, G., Limentani, G., Lister, A., Lung, K. R., and Warner, E. (2005) Analytical method equivalency: An acceptable analytical practice. Pharmaceutical Technology, Sept 2005, pages  Richter, S., and Richter, C. (2002). A method for determining equivalence in industrial applications. Quality Engineering, 14(3), pages  Park, D. J. and Burdick, R. K. (2004). Confidence Intervals on Total Variance in a Regression Model with an Unbalanced Onefold Nested Error Structure, Communications in Statistics, Theory and Methods, 33, No. 11, pages

30 30 Back up slides

31 31 Back up slides  EAC Method 2  Equal Difference Assumption:  This assumption may not always hold –The p-value for the interaction between time, process, and temperature tests this assumption

32 32 Comparison of Equivalence Acceptance Criteria  Plot regression line for historical process  At time=0 the value is  Calculate  Plot 2 additional lines  Value at time=0 is  Values at time=T are


Download ppt "Analysis of Stability Data with Equivalence Testing for Comparing New and Historical Processes Under Various Treatment Conditions Ben Ahlstrom, Rick Burdick,"

Similar presentations


Ads by Google