1. Regression to the Mean as a Tool to Understand Out-of-Specification (OOS) Results
Jyh-Ming Shoung and Stan Altan
Midwest Biopharmaceutical Statistics Workshop
Session on Statistical Mitigation of OOS
and OOT in a QbD World
May 24, 2011

2. Outline
Introduction
OOS Guidance for Industry (October 2006) – the current regulatory view
Out-of-Specification (OOS) results viewed through the prism of Regression to the Mean(RtM)
OOS Case Study – applying RtM principles
Recommendations

3. Introduction the current regulatory view
OOS Guidance for Industry (October 2006)
Prescriptive approach to investigations addressing the question of root cause
Is there a flaw in the product or process?
Statistical references ;
Outlier testing - write it into the SOP
It should never be assumed that the reason for an outlier is error in the testing procedure, rather than inherent variability in the sample being tested.
Retesting (of the original sample)
The number of retests to be performed on a sample should be specified in advance by the firm in the SOP…based on scientifically sound, supportable principles. The number should not be adjusted depending on the results obtained.
Re-sampling (new sample) appears to be discouraged, the original sample is supposed to be large enough to permit re-testing
Phase I: Laboratory Investigation
A. Responsibility of the Analyst
B. Responsibility of Laboratory Supervisor
Phase II: Full-Scale OOS Investigation
A. Review of Production: The records and documentation of the manufacturing process should be fully reviewed
to determine the possible cause of the OOS results. A full-scale OOS investigation should consist of a timely,
thorough, and well-documented review. If this part of the OOS investigation confirms the OOS result and is
successful in identifying its root cause, the OOS investigation may be terminated and the product rejected.
OOS results may indicate a flaw in product or process design.
B. Additional Laboratory Testing
1. Retesting – retesting of a portion of the original sample. Decisions to retest should be based on the objectives of the testing and sound scientific judgment. The maximum number of retests to be
performed on a sample should be specified in advance in a written SOP.
2. Resampling – involves analyzing a specimen from any additional units collected as part of the
original sampling procedure or from a new sample collected from the batch, should that be necessary.
C. Reporting Testing Results
1. Averaging – appropriate uses and inappropriate uses
2. Outlier tests – should be written into SOPs for data interpretation including the specific outlier test
to be applied with relevant parameters specified in advance and should specify the minimum
number of results required to obtain a statistically significant assessment from the specified outlier
test.

4. OOS Guidance – What is the Issue?
Ignores statistical considerations
“In cases where … some of the individual results are OOS, some are within specification, ... the passing results are no more likely to represent the true value for the sample than the OOS results…and treat the reportable average of these values as an OOS result, even if that average is within specification. … every individual application of the official test should be expected to produce a result that meets specifications.”
This language is in conflict with the basic principle that the essential quality statement is captured in the ‘average’ and variability between dose units is captured in a separate protocol
An operational root cause will not be found for some or many OOSs, and this does not necessarily indicate a product or process flaw
An outlier test is a statistical analysis of the data obtained from testing and retesting. It will not identify the cause of an extreme observation and, therefore, should not be used to invalidate the suspect result.

5. Quality by Design –an opportunity to revisit OOS management?
Improved Process Knowledge
Systematic Development approach
Formulation understanding
Process understanding
Packaging understanding
Identify Critical to Quality Attributes (CQAs)
Process Understanding as input to risk management - Quality risk management
Control what is critical - Advanced control strategy
Model can be described with scientific understanding of the process and with statistical analysis/simulation for risk assessment

6. A process is well understood when…
All critical sources of variability are identified and explained
OOSs WILL happen occasionally
Variability is managed by the process
How do we integrate occurrence of OOS’s in a QbD framework
Product quality attributes can be accurately and reliably predicted over the design space
Basic Principle : Occurrence of isolated OOSs does not contradict quality in product manufacture
Guidance for Industry: PAT (Process Analytical Technology)— A Framework for Innovative Pharmaceutical Development, Manufacturing, and Quality Assurance (September 2004)

7. Tim Schofield (Specs Workshop, 2005)
Multiple repeated stability determinations of the same lot on stability necessarily lead to an increasing chance of an OOS
Such OOSs require context and mitigation
Statistical mitigation is completely consistent with QbD concepts of advanced risk management and control strategies

8. OOS’s viewed through the prism of Regression to the Mean in a QbD Context
An isolated OOS can be thought of as a random outcome related to incipient sources of variability and hence should be viewed as a normal part of data collection and testing
Compliance considerations have clouded the true understanding of the occurrence of OOSs
Under QbD, control strategies related to statistical considerations can be brought to bear to understand and assess the ‘truth’ of an OOS
OOSs can be mitigated through the application of a Regression to the Mean approach

9. Regression to the Mean (RtM)
Regression to the Mean (RtM) is a statistical phenomenon that occurs when repeated measurements are made on the same subject or unit of observation.
During development studies or commercial studies where numerous analytical determinations are being carried out on the same batch or process, isolated unusual observations can be expected and understood as a ‘pretest’ to be mitigated through a ‘posttest’.

10. Case Study of a RtM example in Release Testing of Content Uniformity and how it can be applied to an advanced Control Strategy

11. Harmonized Content Uniformity Test - USP <905> (January, 2007)
Stage 1, 10 Tablets
Yes
PASS No
Stage 2, additional 20 Tablets
FAIL

12. Data Description and Goal
2,000 marketed lots from 12 different groups of sizes
Manufactured prior to 2007 and tested under old USP method
6 lots (A – F) with Stage 2 CU data
New USP test criteria applied to all 2000 lots
1,997 lots passed new USP test <905>,
3 lots failed (X, Y, Z) Stage 1, Stage 2 not available
Issue
Lots (X, Y, Z) failed new Stage 1 test and no Stage 2 data
Objective
Predict Stage 2 test likely outcome based on a RtM analysis

13. Stage 2 RtM Trajectories in 6 Lots
3 lots without S2 samples
S1 (n=10) test criteria
S2 (n=30) test criteria

14. Model Assumptions and Fit Results
2,000 lots with Stage 1 and Stage 2 (only 6 lots) data were used for analysis
A mixed effects model was fit to the data with fixed effects for group and random effects due to lot-to-lot within group, stage-to-stage test (analytical run-to-run) within lot and residuals (dose unit, measurement error, etc.)
The estimated variance components are as follows:
Var(Lot) ( b2 ) = 2.8
Var(Stage) (2 ) = 7.1
Var(Resid) (w2 ) = 3.8

15. Multivariate Normal Distribution
Considering the following two equations where b , 1 , 2, w1 , w2i are independent and

16. Theoretical RtM Effect
Given , And the conditional distribution of +b given is where

17. Simulation of USP Stage 2 Test - Additional 20 Samples given Stage 1 Mean
Adding variability due to stage-to-stage
Adding 20 residual errors
RtM Effect

18. Empirical RtM Effect Obs.S2mean – Group Mean
The empirical RtM effect can be estimated by a linear regression of second stage mean (n=20) on the first stage mean (n=10) from those 6 lots with stage 2 results as follows:
Obs.S2mean – Group Mean
=  × (Obs.S1mean – Group Mean) + error

19. Empirical RtM
Theoretical RtM

20. 10,000 Simulated S2 Results Lot A
Observed S1 result (n=10)
Observed S2 result (n=30)
Simulated S2 results (n=30)

22. Summary Results
Statistical evaluation of 3 lots reporting outlying Stage 1 results shows that on simulated retest, all 3 would pass Stage 2 criterion (> 99%) as a consequence of Regression to the Mean Principle with high certainty
Caveat : Limited information to estimate RTM effect but justified by the overwhelmingly stable behavior of the process
This case study shows the application of the RtM principle to OOS mitigation

23. Recommendations and Future Research
As a consequence of the Clinical Drug Development paradigm, Quality of Materials should be related to the Average as the essential quality statement
Content uniformity should not be inferred from isolated indeterminate OOSs
Re-sampling should be part of a control strategy to mitigate OOS and exploit the RtM principle to achieve a more representative sample for calculating the lot Average
The RtM principle can serve as a basis for pursuing advanced control strategies for OOS mitigation
Directions for future research
Bayesian approach – in a commercial context could be easier to justify priors
Errors-in-variable adjustment