Presentation is loading. Please wait.

Presentation is loading. Please wait.

Lecture 3 Validity of screening and diagnostic tests

Similar presentations

Presentation on theme: "Lecture 3 Validity of screening and diagnostic tests"— Presentation transcript:

1 Lecture 3 Validity of screening and diagnostic tests
Reliability: kappa coefficient Criterion validity: “Gold” or criterion/reference standard Sensitivity, specificity, predictive value Relationship to prevalence Likelihood ratio ROC curve Diagnostic odds ratio Lecture 3 (Sept 7)

2 Clinical/public health applications
screening: for asymptomatic disease (e.g., Pap test, mammography) for risk (e.g., family history of breast cancer case-finding: testing of patients for diseases unrelated to their complaint diagnostic: to help make diagnosis in symptomatic disease or to follow-up on screening test

3 Evaluation of screening and diagnostic tests
Performance characteristics test alone Effectiveness (on outcomes of disease): test + intervention

4 Criteria for test selection
Reliability Validity Feasibility Simplicity Cost Acceptability

5 Measures of inter- and intra-rater reliability: categorical data
Percent agreement limitation: value is affected by prevalence - higher if very low or very high prevalence Kappa statistic takes chance agreement into account defined as fraction of observed agreement not due to chance

6 Kappa statistic Kappa = p(obs) - p(exp) 1 - p(exp)
p(obs): proportion of observed agreement p(exp): proportion of agreement expected by chance


8 Interpretation of kappa
Various suggested interpretations Example: Lanis & Koch, Fleiss excellent: over 0.75 fair to good: poor: less than 0.40

9 Validity (accuracy) of screening/diagnostic tests
Face validity, content validity: judgement of the appropriateness of content of measurement Criterion validity concurrent predictive

10 Normal vs abnormal Statistical definition Clinical definition
“Gaussian” or “normal” distribution Clinical definition using criterion





15 Selection of criterion (“gold” or criterion standard)
Concurrent salivary screening test for HIV history of cough more than 2 weeks (for TB) Predictive APACHE (acute physiology and chronic disease evaluation) instrument for ICU patients blood lipid level maternal height

16 Sensitivity and specificity
Assess correct classification of: People with the disease (sensitivity) People without the disease (specificity)


18 Predictive value More relevant to clinicians and patients
Affected by prevalence

19 Choice of cut-point If higher score increases probability of disease
Lower cut-point: increases sensitivity, reduces specificity Higher cut-point: reduces sensitivity, increases specificity

20 Considerations in selection of cut-point
Implications of false positive results burden on follow-up services labelling effect Implications of false negative results Failure to intervene

21 Receiver operating characteristic (ROC) curve
Evaluates test over range of cut-points Plot of sensitivity against 1-specificity Area under curve (AUC) summarizes performance: AUC of 0.5 = no better than chance


23 Likelihood ratio Likelihood ratio (LR) = sensitivity 1-specificity
Used to compute post-test odds of disease from pre-test odds: post-test odds = pre-test odds x LR pre-test odds derived from prevalence post-test odds can be converted to predictive value of positive test

24 Example of LR prevalence of disease in a population is 25%
sensitivity is 80% specificity is 90%, pre-test odds = = 1/3 likelihood ratio = = 8 1-0.90

25 Example of LR (cont) If prevalence of disease in a population is 25%
pre-test odds = = 1/3 post-test odds = 1/3 x 8 = 8/3 predictive value of positive result = 8/3+8 = 8/11 = 73%

26 Diagnostic odds ratio Ratio of odds of positive test in diseased vs odds of negative test in non-diseased: a.d b.c From previous example: OR = 8 x 27 = 36 2 x 3

27 Summary: LR and DPR Values: Relationship to prevalence?
1 indicates that test performs no better than chance >1 indicates better than chance <1 indicates worse than chance Relationship to prevalence?

28 Applications of LR and DOR
Likelihood ratio: Primarily in clinical context, when interest is in how much the likelihood of disease is increased by use of a particular test Diagnostic odds ratio Primarily in research, when interest is in factors that are associated with test performance (e.g., using logistic regression)

Download ppt "Lecture 3 Validity of screening and diagnostic tests"

Similar presentations

Ads by Google