The equivalence trial Didier Concordet NATIONAL VETERINARY S C H O O L T O U L O U S E

Comparison of two treatments Population of animals  R = 17.8 Treatment effect  T = 16.8 Aim of all trials : to compare treatments on the population of individuals Impossible in practice

A two-steps method Sample of animals Sampling Population of animals

Effect of sampling Sample of animals X R = 16.2 Treatment effect X T = 17.8

A two-steps method Sample of animals Inference Population of animals

Effect of inference X R = 16.2 Observed on the samples X T = 17.8 Truth in the population  R = 17.8  T = 16.8 New Treatment T > Ref Ref > New Treatment T Lead to a wrong conclusion

A good trial Minimize the risk of bias in sampling Minimize the risk of a wrong conclusion in inference - All Randomised Study Animals - Per Protocol Set of Study Animals - Response Variable - Experimental (study) design - Consumer Risk - Producer Risk - Relevant difference

Tree kinds of study     T -  R    Equivalence study  R -     T Non inferiority Superiority  R +     T

Non inferiority  R -   Values of  T RR  R -     T Unacceptable for primary efficacy variable in clinical trial Does not prove that the treatment T works

Superiority  R +   Values of  T RR  R +     T Primary efficacy variable in clinical trials

Equivalence Equivalence range Values of  T  R -   RR  R +   Does not prove that the treatment T works For secondary efficacy variables in clinical trials

Equivalence Equivalence range Values of  T  R -   RR  R +   Clinical effect

Equivalence Values of  T  R -   RR  R +   Clinical effect

Equivalence Values of  T  R -   RR  R +   Clinical effect

Even with a good question, a poor design leads to poor conclusions Superiority clinical trials Cure rate = 83% N = 2400 REFERENCE Cure rate = 79% N = 2100 New TRT Reference < New TRT ( P<0.001)

Even with a good question, a poor design leads to poor conclusions Clinical trial 1 REFERENCE New TRT New TRT< Ref P<0.001 Clinical trial 2 Cure rate = 90% N = 2000 REFERENCE Cure rate = 96% N = 1000 New TRT New TRT < Ref P<0.001 Conclusion : Reference > New TRT Superiority trials Cure rate = 50% N = 400 Cure rate = 63% N = 1100

Even with a good question, a poor design leads to poor conclusions Superiority clinical trials X = 39 N = 100 SD = 1 REFERENCE X = 37 N = 100 SD = 1 New TRT Reference < New TRT ( P<0.001)

Even with a good question, a poor design leads to poor conclusions Clinical trial 1 X = 40 N = 90 SD = 1 REFERENCE X = 42 N = 50 SD = 1 New TRT New TRT< Ref P<0.001 Clinical trial 2 X = 30 N = 10 SD = 1 REFERENCE X = 32 N = 50 SD = 1 New TRT New TRT < Ref P<0.001 Conclusion : Reference > New TRT Superiority trials

Usual statistical tests are not intended to answer to useful questions Efficacy variable on two groups of dogs Ref Test Mean 15.4 SD Student t-test P = 0.23 N33 In the population  R = 14.5 ;  T = 19.7 this difference is clinically important Conclusion : “EQUIVALENCE”

Comparison of two treatments Efficacy variable on two groups of dogs Ref Test Student t-test Mean 16.0 SD N15 P = 0.03 In the population  R = 16.8 ;  T = 17.8 This difference is not clinically important Conclusion : NO EQUIVALENCE Study 1

Comparison of two formulations Efficacy variable on two groups of dogs Ref Test Student t-test Mean 16.0 SD N15 P = 0.26 This difference is not clinically important Conclusion : EQUIVALENCE Study 2 In the population  R = 16.8 ;  T = 17.8

Consequences Large samples size Small variability Small sample size Large variability "Equivalence" Penalty for companies to show equivalence An ill-posed problem that encourages poor trials A bad answer to a wrong question

A wrong question ? H 0 :  T =  R Classical hypotheses for student t-test H 1 :  T   R Treatments are equivalent  T = population mean for test treatment  R = population mean for reference treatment Too restrictive and not relevant  T and  R are close Treatments are not equivalent

A bad answer ? H 0 :  T =  R Classical test of null hypothesis (student t-test) H 1 :  T   R The controlled risk  = risk to wrongly reject H 0 = risk to declare not equivalent formulations that are equivalent = risk for drug companies Not important from a regulatory point of view The consumer risk is uncontrolled Treatments are equivalent Treatments are not equivalent

Bioequivalence : objectives Check that  T and  R are close Control the consumer risk risk to declare equivalent treatments that are not with regard to clinical relevance

Check that  T and  R are close     T -  R    bioequivalence Close in an absolute way Close in a relative way bioequivalence      equivalence range (to be discussed)  T -  R <   or    T -  R bioinequivalence

Control the consumer risk A test controls the risk to wrongly choose the H 1 hypothesis Consumer risk : the risk to wrongly conclude to bioequivalence Bioequivalence H 1   Equivalence range  T -  R Possible values of Bioinequivalence H 0 Bioinequivalence H 0

Hypotheses of a bioequivalence study H 1 :     T -  R    bioequivalence Additive hypotheses Multiplicative hypotheses bioequivalence      equivalence range H 0 :  T -  R <   or    T -  R bioinequivalence H 0 : H 1 :