1. 1 The Use of Control Charts in Health Care Monitoring and Public Health Surveillance William H. Woodall William H. Woodall Department of Statistics Department of Statistics Virginia Tech Virginia Tech Blacksburg, VA 24061-0439 Blacksburg, VA 24061-0439 bwoodall @ vt.edu bwoodall @ vt.edu

2. 2 Topics to be covered:  Some general issues  Risk adjustment  Examples of some of the plots used for monitoring  Detection of clusters of chronic disease  League tables, control charts, and funnel charts for cross-sectional data  Conclusions

3. 3 In 1999 the Institute of Medicine reported the number of deaths due to medical errors in U.S. hospitals to be 44,000 to 98,000 per year. Some prefer the term “preventable adverse events.” In 1999 the Institute of Medicine reported the number of deaths due to medical errors in U.S. hospitals to be 44,000 to 98,000 per year. Some prefer the term “preventable adverse events.”

4. 4 Examples of health care variables  Lab turnaround time  Days from positive mammogram to definitive biopsy  Patient satisfaction scores  Medication error counts  Emergency service response times  Infection rates  Mortality rates  Number of patient falls  Post-operative length of stay  “Door-to-needle” time ……and many others…

5. 5 Control charts are used to understand variation over time and to detect unusual events and trends. They are most effective in a hospital when used as a part of its organized quality improvement program, such as Six-Sigma.

6. 6

7. 7

8. 8

9. 9

10. 10 Some Types of Control Charts  Shewhart X-bar and R charts (n>1)  Normality  Shewhart I and MR charts (n=1)  Normality  Shewhart p-charts for proportions Binomial Binomial  Shewhart c- and u-charts for counts Poisson Poisson  Cumulative sum (CUSUM) charts

11. 11 Our focus will be on the monitoring of chronic diseases, congenital malformations, and mortality rates over time.

12. 12 Health care quality data are often  attribute (yes/no) data with 100% inspection.  counts or times to a “failure” with an assumed underlying Bernoulli, geometric or exponential distribution.

13. 13 Suppose one counts the number of births between successive cases of a specific type of congenital malformation. The sets method of Chen (1978, JASA) signals an increase in the rate if a specified number of consecutive counts are all less than a specified value. For example, signal if 5 consecutive values are less than 1000.

14. 14 Risk-adjustment is often essential in health care applications, where a logistic or other model is used to predict the probability of “failure.” …oooooooooooooooooooo …oD89pkej589Cv0238&*&%#$

15. 15 Examples of Risk Factors  Down’s Syndrome: Age of mother  Heart Surgery: Age, gender, hypertension, diabetic status, renal function, left ventricular mass. (Parsonnet score)  Heart Surgery (Europe): Age, gender, chronic pulmonary disease, extracardiac arteriopathy, neurological dysfunction, previous cardiac surgery, creatinine > 200 µmol/ L, active endocarditis, critical preoperative state. (euroSCORE)

16. 16 Much of the focus and work on mortality rate monitoring for physicians is being done in the UK and Canada.

17. 17

18. 18

19. 19 Shipman Inquiry July 2002: 215 definite victims, 45 probable

20. 20 (Shipman Inquiry: total of definite or probable victims: 189 female > 65, 55 male over 65)

21. 21 Sequential probability ratio test (SPRT) for detection of a doubling in mortality risk: age >64 years and death in home/practice for Dr. Harold Shipman. (Spiegelhalter et al. (2003))

22. 22 Resetting sequential probability ratio test (RSPRT) for detection of a doubling in mortality risk, age >64. (From Spiegelhalter et al. (2003).

23. 23 RSPRT charts have a problem with building up “credit”. An increase in the mortality rate can occur when the SPRT value is below zero. This phenomenon is referred to as “inertia” in the industrial SPC literature.

24. 24 Cumulative risk adjusted mortality (CRAM) chart with 99% control limits for change in mortality in last 16 expected deaths. (From Poloniecki et al. (1998))

25. 25 Example of a two-sided risk-adjusted CUSUM chart (provided by Stefan H. Steiner)

26. 26 The CUSUM chart is the best option.  It can be risk-adjusted.  It has optimality properties in detecting sustained shifts in the process.  It has good inertial properties.  It can be designed based on meaningful performance measures such as average run length (ARL).  It can be used in the background with CRAM charts.

27. 27 Control charts can be used to identify physicians or hospitals with unusually high (or low) mortality rates. The Society of Cardiothoracic Surgeons of Great Britain and Ireland interprets giving the benefit of the doubt to physicians as 9999:1 odds of adverse outcomes being due to chance alone before any alarm. Control charts can be used to identify physicians or hospitals with unusually high (or low) mortality rates. The Society of Cardiothoracic Surgeons of Great Britain and Ireland interprets giving the benefit of the doubt to physicians as 9999:1 odds of adverse outcomes being due to chance alone before any alarm.

28. 28 The Centers for Disease Control and Prevention use CUSUM and other control charting methods in their Early Aberration Reporting System (EARS). www.bt.cdc.gov/surveillance/ears/index.asp

29. 29 Virtually all methods for the detection of clusters of disease are retrospective, based on historical spatial data. There are some new methods for detecting clusters prospectively, i.e., as they are forming.

30. 30

31. 31 Detection of clusters of chronic disease  Aggregation of data by time and location Raubertas (1989, Statistics in Medicine) Raubertas (1989, Statistics in Medicine) Rogerson and Yamada (2004, Statistics in Medicine) Rogerson and Yamada (2004, Statistics in Medicine)  Aggregation of data by location Rogerson (1997, Statistics in Medicine) Rogerson (1997, Statistics in Medicine)  No aggregation Rogerson (2001, JRSS-A) Rogerson (2001, JRSS-A)

32. 32 It is often useful to compare units, e.g., institutions or physicians, using cross-sectional data.

33. 33

34. 34 Example of a League Table from Adab et al. (2002). Example of a League Table from Adab et al. (2002).

35. 35 Example of Proposed “Control Chart” by Adab et al. (2002)

36. 36 "Funnel plot" of emergency re-admission rates following treatment for a stroke in large acute or multi-service hospitals in England and Wales in 2000–1. Exact 95% and 99.9% binomial limits are used. (From Spiegelhalter (2002))

37. 37 Harold Shipman  Killed his patients using morphine overdoses.  Was caught after carelessly revising a patient’s will, leaving all her assets to himself.  His office typewriter was used to type the revised will.  His computer records were doctored to show his patients had needed morphine just after the patients had been killed. The computer software, however, recorded the dates of these modifications.  He hung himself in prison, never confessing to his crimes.

38. 38 Baker, R. et al. British Medical Journal 2003;326: pp. 274-276

39. 39 Highly recommended reference: Michael L. Millenson (1999). Demanding Medical Excellence: Doctors and Accountability in the Information Age, The University of Chicago Press.

40. 40 Recommended References  Sonesson, C. and Bock, D. (2003). “A Review and Discussion of Prospective Statistical Surveillance in Public Health”. Journal of the Royal Statistical Society A 166, pp. 5-21.  Grigg, O. A.; Farewell, V. T.; and Spiegelhalter, D. J. (2003). “Use of Risk-adjusted CUSUM and RSPRT Charts for Monitoring in Medical Contexts”. Statistical Methods in Medical Research 12, pp. 147-170.  Grigg, O. and Farewell, V. (2004a). “An Overview of Risk- Adjusted Charts”. Journal of the Royal Statistical Society A 167, pp. 523-539.  Steiner, S. H.; Cook, R. J.; Farewell, V. T.; and Treasure, T. (2000). “Monitoring Surgical Performance Using Risk- Adjusted Cumulative Sum Charts”. Biostatistics 1, pp. 441- 452.

41. 41 My paper is available at http://filebox.vt.edu/users/bwoodall/

42. 42Conclusions  There are many important applications of control charts in health care.  Improvement of health care is a life-or-death matter.  There are many interesting SPC research opportunities in public health surveillance.  There needs to be a greater transfer of knowledge between the medical and industrial application areas.