1. 2005 Syndromic Surveillance1 Estimating the Expected Warning Time of Outbreak- Detection Algorithms Yanna Shen, Weng-Keen Wong, Gregory F. Cooper RODS Laboratory, Center of Biomedical Informatics, University of Pittsburgh

2. 2005 Syndromic Surveillance 2 Overview Objective Background Methods Experimental Results Conclusions Future Work

3. 2005 Syndromic Surveillance 3 Objective A new measure for evaluating alerting algorithms, which is called Expected Warning Time (EWT). It is a generalization of the standard AMOC curve.

4. 2005 Syndromic Surveillance 4 Why Useful? Can compare expected clinician detection time to expected computer-based algorithm detection time Can provide a promising new approach for optimizing and comparing outbreak detection algorithms

5. 2005 Syndromic Surveillance 5 Background Maximum meaningful detection time Maximum meaningful WT Time Warning Time Computer Detection Time False Alerts (in red) Incubation Time Release occurs Computers raise alert Clinicians detect outbreak Last outbreak case appears ……… hit

6. 2005 Syndromic Surveillance 6 Model A simple model of clinician outbreak detection Assumes that  People with disease D are diagnosed independently of each other.  The probability of a person with disease D being diagnosed is constant (p).

7. 2005 Syndromic Surveillance 7 Equation Definitions: p – probability that a person with D is diagnosed as having D upon presentation with that disease time(i) – maps patient case i to the time at which that patient presented with D to clinicians M – total # of patient cases with D t – time at which the alerting score first exceeds a given threshold

8. 2005 Syndromic Surveillance 8 Equation x x + WT if clinicians never detect the outbreak Probability that clinicians will never detect the outbreak WT if clinicians first detect the outbreak on the i th case Probability that clinicians will detect the outbreak on the i th case

9. 2005 Syndromic Surveillance 9 Experiment Setup Apply PANDA to simulated cases of inhalational anthrax For various value of p, derive EWT for PANDA

10. 2005 Syndromic Surveillance 10 PANDA An outbreak detection system Uses causal Bayesian networks to model spatio- temporal patterns of a non-contagious disease in a population (Cooper, 2004) Contains a model to detect inhalational anthrax

11. 2005 Syndromic Surveillance 11 BARD BARD simulator produces the simulated cases of anthrax. It models the effects of an outdoor airborne anthrax release using the Gaussian plume model of atmospheric dispersion and a model of inhalational anthrax (Hogan, 2004).

12. 2005 Syndromic Surveillance 12 Performance of Clinicians pP(CD) ECDT (hours) 0.0010.992167.9 0.0050.999104.1 0.010.999988.6 0.050.9999964.5 110  p – Clinician detection proficiency  P(CD) – Probability that clinicians will detect the outbreak at all  ECDT – Expected clinician detection time given that clinicians detect the outbreak

13. 2005 Syndromic Surveillance 13 Experimental Results  p increases, EWT decreases  p = 1, EWT = 0 ?

14. 2005 Syndromic Surveillance 14 Experimental Results Clinician- detection- proficiency (p) Expected Warning Time (EWT) 0.00176 hours 0.00513 hours 0.015 hours 0.0534 minutes 10 False alert rate = 1 per month  If and false alert rate = 1 per month, then EWT minutes.

15. 2005 Syndromic Surveillance 15 Conclusions The Expected Warning Time (EWT) is a useful concept for evaluating outbreak-detection algorithms. We illustrated the general idea of EWT using a simple model of clinician detection and simulated cases of inhalational anthrax. Our example analysis suggests that PANDA is most helpful when clinicians’ detection proficiency < 5%.

16. 2005 Syndromic Surveillance 16 Future Work Extend the model:  Instead of a constant (p), use the function p(t), where t is time  Develop and apply more disease-specific models of clinician detection (please see the poster by Christina Adamou)

17. 2005 Syndromic Surveillance 17 Acknowledgements This research was supported by grants from the National Science Foundation (IIS-0325581), the Department of Homeland Security (F30602-01-2- 0550), and the Pennsylvania Department of Health (ME-01-737). We thank members of the Bayesian Biosurveillance Project for helpful comments.