1. 2/3/04Sacks1 CLINICAL CONCLUSIONS

2. 2/3/04Sacks2 Granting the statistical significance of a gain in A z of 0.02 (95% CI: 0.01, 0.04), what is the clinical significance ?

3. 2/3/04Sacks3 Clinical Utility Clinical Utility... CAD intended to reduce missed nodules; i.e., intended to increase user’s SENSITIVITY... Clinical Utility Clinical Utility... CAD intended to reduce missed nodules; i.e., intended to increase user’s SENSITIVITY...

4. 2/3/04Sacks4 Gain of 0.02 in A Z understatesrelative gain in sensitivity. Why? Gain of 0.02 in A Z understatesrelative gain in sensitivity. Why?

5. 2/3/04Sacks5 When CAD is used according to instruction to retain all judgments of actionability even if unmarked by CAD...

6. 2/3/04Sacks6... user always maintains or increases sensitivity, and always maintains or increases FPF, as well.

7. 2/3/04Sacks7 FPR 00 11 00 11 TPR Unaided Aided ΔSe =ΔFNR ΔFPR FNR U Se U Chance line (guessing) ROC

8. 2/3/04Sacks8 FPR0 1 0 1 TPR Unaided Aided Chance line (guessing) Loss of Se only possible if instruction not followed Realistic ROC

9. 2/3/04Sacks9 So any statistically significant improvement in A z means an even greater relative gain in Se, and one achieved without falling to a lower ROC curve.

10. 2/3/04Sacks10 The real question for judging the safety and effectiveness of a CAD is...

11. 2/3/04Sacks11 Can we infer from improved average user performance (A z ) in a clinical study...

12. 2/3/04Sacks12... that the average user will improve performance (A z ) with CAD in clinical practice...

13. 2/3/04Sacks13... i.e., improve over her/his performance (A z ) in current clinical practice?

14. 2/3/04Sacks14 To put it another way, is the unaided reading in a clinical study a good surrogate for current (CAD-less) clinical practice?

15. 2/3/04Sacks15 AzAz Current clinical practice Unaided reading Clinical study Aided reading CAD-less reading

16. 2/3/04Sacks16 For example, in actual clinical practice with CAD, the unaided A z could be lowered by failure to read first as one would normally read (i.e., with adequate vigilance).

17. 2/3/04Sacks17 If this were to happen, then the aided A z could also be lower than current (CAD- less) practice.

18. 2/3/04Sacks18 AzAz Current clinical practice Unaided reading Clinical study Aided reading CAD-less reading

19. 2/3/04Sacks19 What would be the implications of such lowering of vigilance for judging the safety and effectiveness of the CAD?

20. 2/3/04Sacks20 Can labeling help prevent this?

21. 2/3/04Sacks21 LABELING ISSUE LABELING ISSUE Two rules, if followed by CAD user in future clinical practice, will help prevent missing more nodules than former reading without CAD: LABELING ISSUE LABELING ISSUE Two rules, if followed by CAD user in future clinical practice, will help prevent missing more nodules than former reading without CAD:

22. 2/3/04Sacks22 Always read unaided first, and as carefully as if you had no CAD. Always read unaided first, and as carefully as if you had no CAD. (This would help keep Az of aided reading higher than Az of current CAD-less reading.) (This would help keep Az of aided reading higher than Az of current CAD-less reading.) Always read unaided first, and as carefully as if you had no CAD. Always read unaided first, and as carefully as if you had no CAD. (This would help keep Az of aided reading higher than Az of current CAD-less reading.) (This would help keep Az of aided reading higher than Az of current CAD-less reading.)

23. 2/3/04Sacks23 Never back off from unaided judgment of actionability of a nodule if CAD fails to mark it. Never back off from unaided judgment of actionability of a nodule if CAD fails to mark it. (This would prevent sensitivity from falling below that of current CAD-less sensitivity. I.e., it would prevent missing more, rather than fewer, nodules.) Never back off from unaided judgment of actionability of a nodule if CAD fails to mark it. Never back off from unaided judgment of actionability of a nodule if CAD fails to mark it. (This would prevent sensitivity from falling below that of current CAD-less sensitivity. I.e., it would prevent missing more, rather than fewer, nodules.)