1. Introduction to Biomedical Statistics

2. Signal Detection Theory What do we actually “detect” when we say we’ve detected something?

3. Signal Detection Theory What do we actually “detect” when we say we’ve detected something? We say we’ve “detected” when a criterion value exceeds a threshold

4. Signal Detection Theory examples: – the onset of a light or sound –the presence of an abnormality on x-ray

5. Signal Detection Theory There are 4 possible situations Target is: You Respond: PresentAbsent Present Absent

6. Signal Detection Theory There are 4 possible situations Hit Target is: You Respond: PresentAbsent Present Absent

7. Signal Detection Theory There are 4 possible situations Hit Miss Target is: You Respond: PresentAbsent Present Absent

8. Signal Detection Theory There are 4 possible situations HitFalse Alarm Miss Target is: You Respond: PresentAbsent Present Absent

9. Signal Detection Theory There are 4 possible situations HitFalse Alarm MissCorrect Rejection Target is: You Respond: PresentAbsent Present Absent

10. Signal Detection Theory There are 4 possible situations HitFalse Alarm MissCorrect Rejection Target is: You Respond: PresentAbsent Present Absent This is the total # of Target Present Trials

11. Signal Detection Theory There are 4 possible situations HitFalse Alarm MissCorrect Rejection Target is: You Respond: PresentAbsent Present Absent This is the total # of Target Present Trials

12. Signal Detection Theory Hit Rate (H) is the proportion of target present trials on which you respond “present”

13. Signal Detection Theory Notice that H is a proportion, so 1 - H gives you the “miss” rate or…

14. Signal Detection Theory False-Alarm Rate (FA) is the proportion of target absent trials on which you respond “present”

15. Signal Detection Theory Notice that FA is a proportion. 1 minus FA gives you the correct rejections or …

16. Signal Detection Theory Signal Detection can be modeled as signal + noise with some detection threshold

17. Signal Detection Theory Stimulus Intensity Frequency Noise is normally distributed - Target Absent trials still contain some stimulus Target Absent

18. Signal Detection Theory Stimulus Intensity Frequency Noise is normally distributed - Target Absent trials still contain some stimulus Target Present trials contain a little bit extra intensity contributed by the signal Target Absent Target Present

19. Signal Detection Theory Stimulus Intensity Frequency Noise is normally distributed - Target Absent trials still contain some stimulus Target Present trials contain a little bit extra intensity contributed by the signal Target Absent This is the signal’s contribution Target Present

20. Signal Detection Theory We can imagine a static criterion above which we’ll respond “target is present” Stimulus Intensity Frequency Target Absent Target Present Criterion

21. Signal Detection Theory Notice that H, FA, etc thus have graphical meanings Stimulus Intensity Frequency Proportion Hits Criterion

22. Signal Detection Theory Notice that H, FA, etc thus have graphical meanings Stimulus Intensity Frequency Proportion Misses Criterion

23. Signal Detection Theory Notice that H, FA, etc thus have graphical meanings Stimulus Intensity Frequency Proportion False Alarms Criterion

24. Signal Detection Theory Notice that H, FA, etc thus have graphical meanings Stimulus Intensity Frequency Proportion Correct Rejections Criterion

25. Signal Detection Theory Notice that as H increases, FA also increases Stimulus Intensity Frequency H Criterion

26. Signal Detection Theory Notice that as H increases, FA also increases Stimulus Intensity Frequency FA Criterion

27. Signal Detection Theory d’ (pronounced d prime) is a measure of sensitivity to detect a signal from noise and does not depend on criterion - it is the distance between the peaks of the signal present and signal absent curves Stimulus Intensity Frequency d’ is computed by converting from H and FA proportions into their corresponding Z scores and subtracting Zinv(FA) from Zinv(H)

28. Some Common Biomedical Statistics Sensitivity Specificity Positive Predictive Value Negative Predictive Value Likelihood Ratio Relative Risk Absolute Risk Number needed to treat/harm

29. Sensitivity and Specificity Consider a test for a condition –e.g. Pregnancy test –e.g. Prostate-specific Antigen (Prostate Cancer) –e.g. Ultrasound (Breast Cancer) These are all signal detection problems

30. Sensitivity and Specificity Four possible situations: Condition is: Test Result: PresentAbsent Present Absent

31. Sensitivity and Specificity Four possible situations: True Positive Condition is: Test Result: PresentAbsent Present Absent

32. Sensitivity and Specificity Four possible situations: True Positive False Negative Condition is: Test Result: PresentAbsent Present Absent

33. Sensitivity and Specificity Four possible situations: True Positive False Positive False Negative Condition is: Test Result: PresentAbsent Present Absent

34. Sensitivity and Specificity Four possible situations: True Positive False Positive False Negative True Negative Condition is: Test Result: PresentAbsent Present Absent

35. Sensitivity and Specificity Four possible situations: True Positive False Positive False Negative True Negative Condition is: Test Result: PresentAbsent Present Absent This is Total # of Condition Present Cases

36. Sensitivity and Specificity Four possible situations: True Positive False Positive False Negative True Negative Condition is: Test Result: PresentAbsent Present Absent This is Total # of Condition Absent Cases

37. Sensitivity and Specificity Four possible situations: True Positive False Positive False Negative True Negative Condition is: Test Result: PresentAbsent Present Absent This is Total # of “positive” tests

38. Sensitivity and Specificity Four possible situations: True Positive False Positive False Negative True Negative Condition is: Test Result: PresentAbsent Present Absent This is Total # of “negative” tests

39. Sensitivity and Specificity Sensitivity is the proportion of condition present cases on which the test returned “positive” Analogous to the hit rate (H) in Signal Detection Theory

40. Sensitivity and Specificity Specificity is the proportion of condition absent cases on which the test returned “negative” Analogous to the Correct Rejection rate in Signal Detection Theory

41. Sensitivity and Specificity Notice that 1 minus the Sensitivity is analogous to the FA of Signal Detection Theory

42. Sensitivity and Specificity Notice that 1 minus the Sensitivity is analagous to the FA of Signal Detection Theory Recall that in Signal Detection Theory, as criterion were relaxed both H and FA increased and as criterion were more stringent, H and FA decreased

43. Sensitivity and Specificity Notice that 1 minus the Sensitivity is analagous to the FA of Signal Detection Theory Recall that in Signal Detection Theory, as criterion were relaxed both H and FA increased and as criterion were more stringent, H and FA decreased Sensitivity and Specificity have a similar relationship: as a cut-off value for a test becomes more stringent the sensitivity goes down and the specificity goes up…and vice versa

44. Sensitivity and Specificity “For detecting any prostate cancer, PSA cutoff values of 1.1, 2.1, 3.1, and 4.1 ng/mL yielded sensitivities of 83.4%, 52.6%, 32.2%, and 20.5%, and specificities of 38.9%, 72.5%, 86.7%, and 93.8%, respectively.” JAMA. 2005 Jul 6;294(1):66-70.

45. Sensitivity and Specificity Likelihood Ratio is the ratio of True Positive rate to False Positive rate Loosely corresponds to d’ in that Likelihood ratio is insensitive to changes in criterion

46. Sensitivity and Specificity If a test is positive, how likely is it that the condition is present? Positive Predictive Value is the proportion of “positive” test results that are correct

47. Sensitivity and Specificity Negative Predictive Value is the proportion of “negative” test results that are correct

48. Sensitivity and Specificity Consider the influence of exposure to some substance or treatment on the presence or absence of a condition e.g. smoking and cancer e.g. aspirin and heart disease

49. Sensitivity and Specificity A similar logic can be applied AB CD Disease: Exposure: YesNo Yes

50. Sensitivity and Specificity A similar logic can be applied AB CD Disease: Exposure: YesNo This is total # exposure No Yes

51. Sensitivity and Specificity A similar logic can be applied AB CD Disease: Exposure: YesNo This is total non-exposure No Yes

52. Sensitivity and Specificity We can think in terms of “Event Rates” AB CD Disease: Exposure: YesNo Yes No e.g. the proportion of non-smokers who get lung cancer e.g. the proportion of smokers who get lung cancer

53. Sensitivity and Specificity Relative Risk is the ratio of Exposure Events to Non-Exposure Events Often encountered in regard to rate of adverse reactions to drugs AB CD Disease: Exposure: YesNo Yes No

54. Sensitivity and Specificity Often we are interested in whether the chance of an event changes with exposure Relative Risk Reduction is the difference between event rates in the exposure and non-exposure groups, expressed as a fraction of the non-exposure event rate

55. Sensitivity and Specificity Notice that Relative Risk Reduction can be positive or negative: that is, exposure could reduce the risk of some event (e.g. exposure to wine reduces risk of heart disease) or increase the risk (e.g. exposure to cigarette smoke increases risk of heart disease)

56. Sensitivity and Specificity Notice also that these figures do not take into account the absolute numbers e.g. control event rate =.264 and exposure event rate =.198 e.g. control event rate =.000000264 and exposure event rate =.000000198 Both yield the same relative risk reduction of -25%

57. Sensitivity and Specificity Notice also that these figures do not take into account the absolute numbers e.g. control event rate =.264 and exposure event rate =.198 e.g. control event rate =.000000264 and exposure event rate =.000000198 Both yield the same relative risk reduction of -25% Doesn’t discriminate between large and small effects

58. Sensitivity and Specificity The absolute risk reduction conveys effect size

59. Sensitivity and Specificity The absolute risk reduction conveys effect size An intuitive version is to consider the reciprocal - the “number needed to treat or harm” Indicates the number of individuals that would have to be exposed to the treatment in order to cause one to have the outcome of interest