Presentation is loading. Please wait.

Presentation is loading. Please wait.

Curva ROC figuras esquemáticas Curva ROC figuras esquemáticas Prof. Ivan Balducci FOSJC / Unesp.

Similar presentations


Presentation on theme: "Curva ROC figuras esquemáticas Curva ROC figuras esquemáticas Prof. Ivan Balducci FOSJC / Unesp."— Presentation transcript:

1 Curva ROC figuras esquemáticas Curva ROC figuras esquemáticas Prof. Ivan Balducci FOSJC / Unesp

2 Each threshold c corresponds to a point (FPR, DR) on the X-Y plane. A ROC curve is obtained as c sweeps from -  to + . cc Receiver Operating Characteristic (ROC) Curve 2

3 Lab Tests: What is “Abnormal”?

4 Desempenho True Positive False Positive False Negative True Negative AB TP – Classe é A e classificamos como A TN – Classe é B e classificamos como B FP – Classe é B e classificamos como A FN – Classe é A e classificamos como B

5 The Cut-off Value Trade off Sensitivity and specificity depend on the cut off value between what we define as normal and abnormal Assume high test values are abnormal; then, moving the cut-off value to a higher one increases FN results and decreases FP results (i.e. more specific) and vice versa There is always a trade off in setting the cut-off point

6 Each threshold c corresponds to a point (FPR, DR) on the X-Y plane. A ROC curve is obtained as c sweeps from -  to + . cc Receiver Operating Characteristic (ROC) Curve 6

7 ROC Curve

8 Receiver Operating Characteristic (ROC) Curves

9 Goodness-Of-Fit: Other Measures of Model Performance ROC (Receiver Operating Characteristic) Curve Sensitivity and Specificity are dependent on a given cut-point c. An ROC curve is obtained by plotting sensitivity against (1- specificity) for an entire range of possible cut-points. The area under the ROC curve is a measure of the model’s ability to discriminate between event and non-event in the following fashion: »Among all possible pairs (event, non-event), the proportion of pairs for which the event has higher probability than the corresponding non-event is equal to the area under ROC.

10 ROC Curve Interpretation: Area Under ROC Curve –If randomly selected pairs of subjects (one with event and one with non-event) are classified in such a way that the subject with higher estimated probability of the event belongs to the event group and the other subject to non-event group, then the proportion of correctly classified such pairs of subjects would be equal to the area under ROC Generally Accepted Rule: ROC = 0.5: no discrimination (no better than coin toss) 0.7 <= ROC < 0.8: acceptable discrimination 0.8 <= ROC < 0.9: excellent discrimination ROC > 0.9: outstanding discrimination Roc area is often used to compare predictive ability of different models;

11 The ROC The ROC shows the tradeoff between P FP and P TP as the threshold is varied

12

13

14 Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC) Healthy

15 Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC) Healthy Sick

16 Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC) Fals pos= 20% True pos=82%

17 Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC) Fals pos= 9% True pos=70%

18 Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC) Healthy

19 Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC) Healthy Sick

20 Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC) Fals pos= 20% True pos=82%

21 Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC) Fals pos= 9% True pos=70%

22 0.7 Evaluating the results How can we measure the performance of a feature matcher? false positive rate true positive rate # true positives # matching features (positives) 0.1 # false positives # unmatched features (negatives)

23 0.7 How can we measure the performance of a feature matcher? false positive rate true positive rate # true positives # matching features (positives) 0.1 # false positives # unmatched features (negatives) ROC curve (“Receiver Operator Characteristic”) ROC Curves Generated by counting # current/incorrect matches, for different threholds Want to maximize area under the curve (AUC) Useful for comparing different feature matching methods For more info:

24 Cut-off Área sob curva  Curva ROC  especificidade  sensibiidade


Download ppt "Curva ROC figuras esquemáticas Curva ROC figuras esquemáticas Prof. Ivan Balducci FOSJC / Unesp."

Similar presentations


Ads by Google