Charles Y. Tan, PhD USP Statistics Expert Committee A Step in the Right Direction? the development of USP chapter <1210> Charles Y. Tan, PhD USP Statistics Expert Committee

Outline Introduction of <1210> Key topics Summary Accuracy and Precision Linearity LOD, LOQ, range Summary

USP <1210> United States Pharmacopeia General Chapters <1210> Statistical Tools for Method Validation Current status: a draft is published in Pharmacopeial Forum 40(5) [Sept-Oct 2014] Seek public comments

Purpose of <1210> A companion chapter to <1225> Validation of Compendial Procedures USP <1225> and ICH Q2(R1) USP <1033> Biological Assay Validation Statistical tools TOST, statistical equivalence Statistical power, experimental design tolerance intervals, prediction intervals Risk assessment, Bayesian analysis AIC for calibration model selection

Recent Framework Life cycle perspective ATP: Analytical Target Profile procedure design performance qualification / validation ongoing performance verification ATP: Analytical Target Profile Pre-specified acceptance criteria Assume established Validation: confirmatory step Statistical interpretation of “validation”

Performance Characteristics Different statistical treatments Tier 1: accuracy and precision Statistical “proof” ATP is met Equivalence test / TOST Sample size / power, DOE Tier 2: linearity, LOD Relaxed evidential standard, estimation Sample size / power optional

Key topics USP General Chapter <1210> Statistical Tools for Method Validation

Accuracy and Precision Separate Assessment Of Accuracy And Precision Confidence interval within acceptance criteria from ATP Combined Validation Of Accuracy And Precision γ-expectation tolerance interval: 100γ% prediction interval for a future observation, Pr (-λ ≤ Y ≤ λ) ≥ γ γ-content tolerance interval: 100γ% confidence of all future observations Bayesian tolerance interval

Experimental Condition Yij = μ + Ci + Eij Ci: experimental condition combination of ruggedness factors: analyst, equipment, or day DOE: experience the full domain of operating conditions As independent as possible Eij: replication within each condition One-way analysis (w/ random factor): why?

Separate Assessment Closed form formulas: Accuracy: classic confidence interval for bias Precision: confidence interval for total variability under one-way layout (Graybill and Wang) Power and sample size calculation Statement of the parameters: bias, variance Eg. CI of bias: [-0.4%, 1.1%], within ±5% (ATP) Eg. CI of total variability: ≤2.4%, within 3% (ATP) Implicit risk level: 95% confidence intervals

Combine Accuracy and Precision Statement of observation(s) Closed form formulas, but a bit more complicate 99%-expectation tolerance interval: eg. [-4.3%, 5.0%] within ±10% (ATP) 99%-content tolerance interval: eg. [-5.9%, 6.6%] within ±15% (ATP) Bayesian tolerance interval “the aid of an experienced statistician is recommended” Simpler Alternative: directly assess the risk with the λ given in ATP Pr (-λ ≤ deviation from truth ≤ λ|data)

Scale of Analysis Pooling variances is central to stat analysis Variance estimates with df=2 are highly unstable Need to pool across samples, levels Variance at mass or concentration scale/unit Increase with level Solutions: Normalize with constants, eg. Label claim Normalizing by observed averages makes stat analysis too complicated Log transformation %NSD and %RSD

Linearity Internal performance characteristic External view: accuracy and precision Transparency => credibility Appropriateness of standard curve fitting A model A range Better than the alternatives (all models are approximations) Proportional: model: Y = β1X + ε Straight line: Y = β0 + β1X + ε Quadratic model: Y = β0 + β1X + β2X2 + ε

Current Practices Pearson correlation coefficient Lack-of-fit F test Anscombe's quartet Lack-of-fit F test independent replicate Mandel’s F-test, the quality coefficient, and the Mark–Workman test Test of significance Evidential standard: low since it gives the benefit of doubt to the model you want Good precision may be “penalized” with a high false rejection rate Poor precision is “rewarded” with false confirmation of the simpler and more convenient model

Anscombe's Quartet

Two New Proposals Equivalence test, TOST, in concentration units Define maximum allowable bias due to calibration in ATP Construct 90% confidence interval for the bias comparing the proposed model to a slightly more flexible model Closed form formula, complex Evidential standard: could be high, depend on allowable bias Akaike Information Criterion, AICc Compare the AICc of the proposed model to a slightly more flexible model (smaller wins) Very simple calculations Evidential standard: most likely among candidates

Different Burden of Proof Hypothesis Testing: Neyman-Pearson Frame the issue: null versus alternative hypotheses Goal: reject the null hypothesis Null hypothesis: protected regardless of amount of data Decision standard: beyond reasonable doubt Legal analogy: criminal trial Information Criteria: Kullback-Leibler Frame the issue: a set of candidate models Goal: find the best approximation to the truth Best: most parsimonious model given the data at hand Decision standard: most likely among candidates Legal analogy: civil trial Stepping-stone or tactical questions: information criteria are apt alternatives to hypothesis tests

IUPAC/ISO LOD (RC and RD)

IUPAC/ISO LOD

LOD: Using Prediction Bounds

Range and LOQ Range LOQ (LLOQ) LOQ versus LOD suitable level of precision and accuracy Both upper and lower limits LOQ (LLOQ) acceptable precision and accuracy lower limit LOQ versus LOD Only one is needed for each use LOQ for quantitative tests LOD for qualitative limit tests LOQ calculation in ICH Q2: candidate starting values

Summary A draft of USP <1210> is published, seeking public comments A step in the right direction? More than a bag of tools Implement modern validation concepts with a statistical structural More tools development needed More statisticians involvement needed in pharmacopeia and ICH development