Uses of Diagnostic Tests Screen (mammography for breast cancer) Diagnose (electrocardiogram for acute myocardial infarction) Grade (stage of cancer) Monitor progression (recurrence) Monitor therapy (blood drug level) and therapeutic response (regression of tumor size) Guide treatments (arteriography for bypass surgery)

TPFP FNTN DefiningTest Performance in 2 x 2 Table Disease Test Sensitivity (TPR) = TP/(TP+FN) Specificity (1-FPR) = TN/(TN+FP) Odds Ratio = LR+/LR-

Changing diagnostic threshold or disease spectrum changes test performance Diseased Not diseased False Positive False Negative Diagnostic Threshold Region of overlapping test results True Negative True Positive Test Performance

Cancer Screening Incidence =.001 Sensitivity = P(test +| disease) = 0.9 Specificity = P(test -| no disease) = 0.95

Cancer Screening What are the positive and negative predictive values? PPV= P(cancer|test +) = P(cancer and test+) / P(test+)‏ = /( ) = NPV = = P(no cancer | test-) = P(no cancer and test -)/P(test -) = /( ) =

Members of a study population are randomly assigned to either a treatment or control group. Treated and untreated groups are prospectively followed to determine whether groups subsequently differ in terms of incidence rates of selected outcome(s). Randomized Controlled Trials (RCTs)

Defined Population RANDOMIZED New Treatment Current Treatment (or placebo)‏ Improved (+) Not Improved (-)‏ Improved (+) Not Improved (-)‏ Design of a Randomized Controlled Trial

Goals of Trials Types of Bias Selection Measurement Confounding Types of Variability Measurement Confounding Fundamental trade-off is between internal and external validity Decreased bias and confounding but less generalizability Efficacy vs. effectiveness Minimize Bias Reduce Variability

Choice of Control Group Three common choices: –No treatment control –Placebo –Active comparator (standard treatment)‏ What other therapies are being given to both groups –Are there any differences between treatments in addition to intervention?

Characteristics of a Well-planned Experiment 1. Units receiving different treatments should respond independently of each other and be alike in all respects except the treatment application 2. Comparisons should be made sufficiently precisely to draw proper conclusions but not so precisely that needless data are collected 3. Conclusions should have a wide range of validity 4. Methods of design and analysis should be simple 5. Uncertainty in conclusions should be assessable

Components of Good Study Design Randomization Replication Blocking Make designs as simple as possible with generalizable conclusions and assessble uncertainty under controlled conditions

Statistical Advantages of Randomization Randomization ensures that the experimental units are independent of each other. This independence gives us: –Estimated treatment effects equal on average to true effects (unbiasedness) –High probability that estimated treatment effect close to true effect in large samples (consistency) –Sample variance equal on average to true variance –Results from t-test like those from randomization test –Eliminates confounding (theoretically) Conclusion: In large experiments, the treatment effect is correct and random error can be assessed.

Blocking (matching/stratification) ‏ Increases precision Removes confounding factors Control of external conditions affecting treatments

Blocking Block is a portion of experimental material that should be more homogeneous than the entire set of material –Location, time, physical properties of units and environmental conditions Compare conditions of interest in experiment within each block –Increases precision –Removes confounding Variation, not level, of block effects is usually more important Block what you can and randomize what you cannot

Covariate Adjustment Serves same purpose as blocking –Increases precision of estimates –Extends validity of experiment Could divide units into blocks by grouping according to values of covariate, but this –Wastes information given by data collected from covariate –May be more expensive than covariate adjustment –Adjustment restricted to covariates available at treatment assignment –Some covariates do not lend themselves well to blocking

Covariate Adjustment Use of covariate values allows estimation of covariate effect and covariate-treatment interaction Blocking more useful with factors that contribute variation affecting groups of units Covariate adjustment better with factors that contribute variation affecting individual units

Intent to treat analysis - “Once randomized, always analyzed” - Interpretation of crossovers - Explanatory versus policy management trial Subgroup Analyses - Hypothesis generating not hypothesis testing - Beware of multiple significance and group testing which increases likelihood of false positive results Analysis of RCT’s

Factors to be Specified in Estimating Sample Size Difference in response rates to be detected Estimate of response rate in one of the groups Level of statistical significance Whether the test should be 1 or 2-sided Power

Plots of Four Regression Lines Set Set Set Set 4 For each set: Mean(X) = 9.0 Mean (Y) = 7.5 Corr (X,Y) = 0.82 R 2 = 0.67 Y= X

Example PTT Serum IL PTT IL Scatterplot What is relationship between PTT and IL-6 in 15 cases?