Critical appraisals: Treatment

CLINICAL TRIAL = a prospective study comparing the effect and value of intervention(s) against a control in human beings (Friedman,et al)

Structure of a Simple Trial Comparing Two Treatments Study sample R= Random assignment of patients to a treatment arm R Treatment “A” Treatment “B” Improved Not Improved Improved

CLASSIFICATION OF CLINICAL TRIAL Based on its objective PRAGMATIC TRIAL EXPLANATORY TRIAL

Pragmatic trials: Attempt to determine cause-effect relationship Assuming the results will be applied in actual clinical practice Preferably: binomial outcome (Yes/ No) Analysis: Intention to treat analysis: All randomized subjects are accounted for the final calculation according to their original allocation

Pragmatic trial: R Y N N Y a b c a, b, & c are accounted as failure of Exp arm Exp Ctrl

Explanatory trials Attempt to explain cause-effect relationship Usually in laboratory investigations (pharmacology, pharmacodynamic, etc) Analysis: On treatment analysis: only subjects completed the trial are accounted in analysis Only minimal drop out is allowed, or replacement for drop outs

Gold standard: Randomized, double blind, placebo controlled clinical trial (Randomized controlled trial, RCT) High rank in hierarchy of evidence

The evidence pyramid

Key elements of RCT: Control Randomization Blinding

1. CONTROLS The essence of any research is comparison = needs to compare the experience of a group of patients on new Tx with a control group of similar patients receiving standard Tx RESULTS OF CLINICAL TRIAL BASED ON DIFFERENT EFFECTS BETWEEN CONTROL AND EXPERIMENT GROUPS

2. RANDOMIZATION = random allocation = random assignment - The process of assigning trial subjects to treatment or control group using an element of chance to determine the assignments in order to reduce bias Random allocation: - aimed to create comparable groups - balances the groups for prognostic factors - if not distributed evenly may exaggerate, cancel or counteract the effects of treatment

Why Is Randomization Important? So all groups are as alike as possible. Provides the best way to prove the effectiveness of a new agent or intervention To exclude CONFOUNDING BIAS Usually = start of trial

3. BLINDING (MASKING) = a procedure in which one or more parties to the trial are kept unaware of the treatment assignment - The primary aim: to reduce or eliminate bias, esp. measurement bias - To avoid patients’ reporting symptoms or clinician’s interpretation being affected by hunches about whether the treatment is effective - Highly recommended whenever applicable

Single blind, double blind - The patient - The treatment team - The evaluator - Data analysts BLINDING (MASKING) Double blind also prevent patient and clinician from adding additional treatment to just one of the groups

Influential factors of measured outcome (OC)

Influential factors Prognostic factors  Risk profile of the patients Extraneous factors  Attention of the physician  Psychology of patient  Adjustment life style  Co-medication Information on outcome

Comparison of 2 treatments OC 1 OC 0 NC 1 = NC 0  Randomization EF 1 =EF 0  Masking/Placebo IE 1 =IE 0  Blinding TE 1 TE 0 OC = TE + NC + EF + IE (OC 1 -OC 2) = (TE 1 -TE 2 )+(NC 1 -NC 2 )+(EF 1 -EF 2 )+(IE 1 -IE 2 )

Three comparability criteria 1.Do the groups have comparable prognosis? 2.Do the groups receive similar treatment other than the drug in question? 3.Is the observation of the outcome identical between the treatment groups?

Appraising Clinical Trial - Valid - Important - Applicable  Methods  Results  Discussion

A.VALIDITY - Primary Guides 1.Was the assignment of patients to treatment randomized? -Was the randomization list concealed? 2. Was follow up sufficiently long and complete? 3. Were all patients analyzed in the groups to which they were randomised?

Allocation Concealment The allocation sequence is concealed from those enrolling participants until assignment is complete. Prevents enrollers from (subconsciously or otherwise) influencing which participants are assigned to a particular treatment group.

A.VALIDITY - Secondary Guides 4.Were patients, health workers, and study personnel “blind” to treatment? 5. Were the groups treated equally apart from the experimental therapy? 6. Were the groups similar at the start of the trial? Some less important points

B. IMPORTANCE 1.Magnitude of the treatment effect -p value -Relative Risk Reduction (RRR) -Relative Risk Increase (RRI) -Absolute Risk Reduction (ARR) -Absolute Risk Increase (ARI) -Number Needed to Treat (NNT) -Number needed to harm (NNH) 2. How precise is the treatment effect construct Confidence Interval (CI)

Search result Klissen TP, Feldman ME, Watters LK, Sutcliff T, Rowe PC. Nebulised budesonide for children with mild to moderate croup. N Eng J Med 1994;331(5): A randomized double blind trial on 3 mo – 5 yr old with mild to moderate croup (laryngotracheobronchitis) Experimental group: 2mg (4 ml) nebulized budesonide Control group: 4 ml nebulized normal saline Event being prevented: hospital admission due to upper-airway obstruction

Budesonide Normal saline N= R HOSPITALIZED NON HOSPITALIZED NON HOSPITALIZED

E C X 2 = ; df = 1; p = 0.04 No Yes B. IMPORTANCE Upper-airway obstruction Results Budesonide NaCL

E C No Yes B. IMPORTANCE Upper-airway obstruction Results Budesonid NaCL CER = 7/27 = 0.26; EER = 1/27=0.04 RRR = (CER-EER)/CER = ( )/0.26 = 85%

Definition Relative Risk Reduction (RRR) is the percent reduction in risk in treated group compared to the control group The RRR is a measure of how much the treatment studied has reduced the frequency of an adverse event Absolute Risk Reduction (ARR) is the difference in risk between the control group and the treatment group

Definition Relative Risk Reduction (RRR) is the percent reduction in risk in treated group compared to the control group The RRR is a measure of how much the treatment studied has reduced the frequency of an adverse event Absolute Risk Reduction (ARR) is the difference in risk between the control group and the treatment group

CEREERRRR Control Event rate Experimental Event rate Relative risk reduction (CER-EER) / CER In the actual 26% 4% 85% trial In the 0.026% 0.004% 85% hypothetical trivial case

CER = 7/27 = 0.26; EER = 1/27=0.04 ARR = [CER-EER] = = 0.22 NNT = 1/ARR = 1/0.22 = 5 No Yes E C Upper-airway obstruction Budesonide NaCL

Absolute Risk Reduction (ARR) is the difference in risk between the control group and the treatment group NNT = 1 / ARR NNT= number needed to treat = number of patients should be treated to avoid 1 bad outcome = number of patients should be treated to have 1 additional good outcome NNH =number needed to harm =number needed to harm one more patient from the therapy NNH = 1 / ARI

CER EERARR Control Event rate Experimental Event rate Absolute risk reduction CER - EER In the actual 26% 4% 22% 5 trial In the hypothetical Trivial case NNT 1/ARR

CER = 6/27 = 0.2; EER = 19/27=0.7 ARI = [CER-EER] = = 0.5 NNH = 1/ARI = 1/0.5 = 2 No Yes E C Mild fever ADVERSE EFFECT IF

NNT= number needed to treat = number of patients should be treated to avoid 1 bad outcome = number of patients should be treated to have 1 additional good outcome NNH =number needed to harm =number needed to harm one more patient from the therapy

Summary of risks and benefits of therapy Outcome CER EER RRRARRNNT Upper-airway obstruction 26% 4%85%22%5 Outcome CER EER RRIARINNH Mild fever 20% 70%250%50%2

NNT = 5 We would need to treat 5 children with croup to prevent one additional child from being hospitalized due to upper-airway obstruction NNH = 2 We would only treat 2 children with croup to harm one more child NNT and NNH provide us a measure of the effort to expend to prevent or cause 1 more bad outcome an effort : yield ratio

How precise is the treatment effect CI for NNT: NNT = 1/ARR First calculate CI for ARR (ARR = diff between proportion) Then calculate 1/(upper CL of ARR) and 1/(lower CL of ARR)

Calculating CI for NNT CER = 6/27 = 0.26; EER = 1/27=0.04 ARR = (CER-EER) = = 0.22 NNT = 1/ARR = 1/0.22 = 5 95%CI ARR = ARR V(p1q1/n1 + p2q2/n2) = V(0.26x0.74)/27+(0.04x0.96/27) = x = = 0.04 ; %CI NNT = 1/0,40 ; 1/0.04 = 3 ; 25

Can you apply this valid, important evidence about a treatment in caring for your patient? Is your patient so different from those in the trial that its results can not help you? Do these results apply to your patient? C. APPLICABILITY Is the treatment feasible in your setting

Can you apply this valid, important evidence about a treatment in caring for your patient? Are your patient’s values and satisfied by the regimen And its preferences consequences? Do your patient and you have a clear assessment of their values and preferences? Consider inconvenience and cost of nebulizer versus hospital admission saved Are they met by this regimen and its consequences?