1. On the Path to an Aberration Detection System for STD Surveillance
Delicia Carey1, Ranell Myles1, Samuel Groseclose1
1Division of STD Prevention, Centers for Disease Control and Prevention, Atlanta, GA
March 12, 2008
2008 National STD Prevention Conference

2. Presentation Overview
Background
Objectives
Rationale
Methods
Results
Conclusion/Implications
Next Steps
I will first give a brief background, followed by the objectives, rationale, methods (manual threshold, cumulative sums and historical limits), results, conclusion and next steps.

3. What is an “aberration”?
Background
What is an “aberration”?
An aberration in public health surveillance data can be defined as a change in the distribution or frequency of health related outcomes that is statistically significant when compared with historical data.
I just want to cover a couple of things to provide a brief background….
First of all, What is an aberration?
READ SLIDE

4. Aberration Detection Methods
Background
Aberration Detection Methods
Detect changes in the reported occurrence of health events
Assist epidemiologists in the recognition of an unusual case reporting pattern
Stimulate public health investigation
Aberration detection methods are used to
1- Detect changes… by comparing current surveillance data to previously reported, historical (or expected) baseline data.
2- Assist epidemiologists… and may provide timely warning of a true disease outbreak.
3- Further Investigation… to determine why the case reporting trend changed

5. Objectives
To apply selected aberration detection methods to syphilis surveillance data at the federal level and describe their performance characteristics
To implement methods/monitoring system at state and/or local level
The objectives of this work are […read slide].

6. Rationale
Syphilis elimination shifted surveillance focus to early detection and control of outbreaks
Standardized approach for the identification of fluctuations in reported cases
More timely response to emerging outbreaks and support a phased response
1. Syphilis elimination has shifted surveillance efforts to focus on early detection and control of outbreaks
2. Applying statistical aberration detection methods to syphilis surveillance data should provide a standardized approach for the identification of fluctuations in reported cases which may represent real increases or decreases in disease incidence.
3. Routine, standardized, and automated approaches to identification of changes in incidence should translate into more timely response to emerging outbreaks and support a phased response.

7. Research Questions
Do we improve STD surveillance practice and surveillance data quality by applying statistical methods to detect aberrations in routinely-collected disease surveillance data?
Which aberration detection methods perform best when applied to syphilis surveillance data?
Which aberration detection algorithm parameters should be adjusted to provide the correct balance between sensitivity and specificity (fewer false positives) when applied to local and state populations under surveillance?
The research questions for this project are….

8. Project Plan Phase I - Evaluation of Methods
Phase II - Method Exploration/Adaptation
Phase III - Pilot Demonstrations
Phase IV - Implementation
Collaborative project:
-within DSTDP
-between project areas and DSTDP
Our project is divided into 4 phases.
This is a collaborative project within our division. So far, 3 branches and OD have been involved. This is also a collaboration between project areas and DSTDP in that we plan to do pilot demonstrations with 3-5 STD project areas prior to Phase IV implementation. We are currently in Phase 2 of the plan.

9. METHODS Systems Explored
Manual Threshold (for areas with low morbidity)
EARS (Early Aberration Reporting System)
-cumulative sums method
MMWR (Morbidity and Mortality Weekly Report) Figure 1
-historical limits
Early on after reviewing the morbidity data for the various project areas, we discovered that areas with low morbidity could easily set a
manual threshold. For example, an area that on average has fewer than 5 cases of syphilis per year would likely benefit from further investigation if
they all of sudden got 2+ cases in a month or started seeing 1 case for a couple of consecutive months.
After a literature review and the exploration of other analytic methods, we decided to focus on two methods for areas with medium to high morbidity, the cumulative sum method which is embedded in the Early Aberration Detection System (EARS) and the historical limits methods which is used to produce figure 1 in the MMWR report. Now let me tell you a little more about the these two methods and how they are currently being used.

10. EARS (Early Aberration Reporting System)
EARS: used to analyze and visualize public
health surveillance data
EARS: uses 3 limited baseline aberration detection methods (cumulative sums)
* C1- Mild
* C2- Medium
* C3- Ultra
Available at :
EARS is a tool used by state and local epidemiologists to analyze and visualize public health surveillance data. As of Oct , there has been about 350 downloads of the software with 50 of those being outside of the US. EARS is used to assist in the early identification of outbreaks of disease and bioterrorism events.
Cumulative Sums is a method which was designed to detect changes in the mean value of the quantity of interest. This method accumulates deviations between observed & expected. It alarms when cumulative deviations exceed some predefined threshold.
EARS uses 3 variations of the cumulative sums method- C1, C2 and C3, mild, medium and ultra, respectively. The terms mild, medium and ultra refer to the level of sensitivity of the three statistical methods.
When applying these analytic methods to syphilis incidence data, deviations from the mean ‘expected’ case count are ‘flagged’ for further evaluation.

11. EARS- Cumulative Sums Output
This table shows output from the EARS application. Actually the data used for this run is from our national dataset. The dataset consists of P&S cases for 3 counties (pink, green, blue shaded areas) in one of the project areas (purple shaded area). A syphilis dataset was created for Jan December For presentation purposes, I selected to display Jan Dec The red cells show where the aberrations were flagged by at least one of the cumulative sum variations that we just discussed (C1,C2, C3). [Show this with pointer]. Explain: County ‘pink’ incidence was flagged in Aug. 2007,by all three variations, so were cases reported by county ‘green’; however, low case counts reported from county ‘blue’ were not flagged at all between Jan.2007 and Dec., 2007.
The row that reads All Strata is the aggregate of the 3 preceding areas (pink, green, blue). It too had a flag produced in Aug 2007.
Aside:Highlighted cell means new high

12. EARS- Cumulative Sums Output
As you see in this example…
It is not always the case that a flag in a smaller area will mean a flag produced in a larger geographical area, sometimes the smaller areas produce a flag and data from the combined areas do not. Or just simply, analysis of aggregate state level data can mask aberrations at the county level (i.e., the analysis of state level data could miss an aberration that the county level could detect).

13. EARS- Cumulative Sums Output
C1C2C3
Aug.2007
This is a graphical representation of the data in the table we saw earlier(advance one slide). It illustrates where aberrations are being detected. [show with pointer].

14. EARS- Cumulative Sums Output
FOLLOW-UP SLIDE FROM 11.
This table shows output from the EARS application. Actually the data used for this run is the P&S cases for 3 counties (pink, green, blue shaded areas) in one of the project areas (purple shaded area). Syphilis data was entered for Jan December For presentation purposes, I selected to display Jan Dec It shows where the aberrations were flagged by at least one of the cumulative sum variations that we just discussed (C1,C2, C3). [Show this with pointer]. Explain: The cells highlighted in red are the flags. County ‘pink’ incidence was flagged in Aug. 2007,by all three variations, so were cases reported by county ‘green’; however, low case counts reported from county ‘blue’ were not flagged at all between Jan.2007 and Dec., 2007.
The row that reads All Strata is the aggregate of the 3 preceding areas (pink, green, blue). It too had a flag produced in Aug 2007.

15. MMWR Figure 1- Historical Limits
Our second method of focus is the historical limits.
The historical limits method is currently being used at CDC in the Morbidity and Mortality Weekly Report (MMWR) known to most as figure 1. Essentially, it compares the number of reported cases in the current 4-week period for a given health event with historical data reported for similar 4-week periods from the preceding 5 years.
EXPLAIN GRAPH
Ratio of current reports (4-week period) to Historical mean
If Ratio > 1  implies increase
If Ratio < 1  implies decrease
Hatched bar implies statistically significant change from historical pattern.
In line with our objectives, we also applied the historical limits method to syphilis data , as well. We
deciphered the SAS coding used to create this figure and then modified it to fit our needs.
National Notifiable Diseases Surveillance System (NNDSS), MMWR

16. MMWR Figure 1- Historical Limits Output
WYEAR
WEEK
DIS
CAT
RATIO
ULIMIT
LLIMIT
PLOTRAT
PLOTU
PLOTL FX
CSUMX 1
2007 34
-999 8 2 43 3 4
30.8 51 5 6 23 7 74 17 9 10
Here are the variables using our same national STD data with the Figure 1 SAS program (3 counties-P&S cases aggregated). We used results from this table to plot the historical limits figure. Let’s focus on P&S, since this is what we analyzed with EARS. The values plotted are the natural log of the ratio, plotrat=0.504 (0.456 which is the solid part of the graph and which is the hatched part).
DIS is the disease value,
CAT is the category value which distinguishes the solid vs. the hatched part of the graph
RATIO is the ratio of cases current 4 weeks to historical mean
Ulimit is the upper limit of the ratio
Llimit is the lower limit of the ratio
Plotrat is the natural log of the ratio
Plot u is the natural log of the upper limit
Similarly plot l is the natural log of the lower limit
FX is the historical mean
CSUMX is the sum of the cases for the current 4-week period.

17. MMWR Figure 1- Historical Limits Output (Using Aggregate Data for 3 Counties in State X-Week 34, 2007)
STAGE
This is the figure corresponding to the data in the preceding table. Here we see that for P&S cases for the aggregate data of the 3 counties at week 34, there was an increase when you compare the number of cases in the current 4-week period to the historical data. The hatch mark indicates an unusually high reported incidence.
PLOTRAT SUM
PLOTRAT SUM

18. Conclusion/ Implications
Use manual threshold for areas with low
syphilis incidence
Apply cumulative sums and historical limits methods to STD data reported from jurisdictions with medium to high syphilis incidence
- provides numerical and graphical output
Use aberration detection methods as STD program tool to focus the review and application of surveillance data for syphilis prevention
Based on the work that the group has done so far, we have determined that we can use a manual threshold for
areas with low syphilis incidence and can use more sophisticated approaches for areas with medium to high incidence.
We can [read bullet 2]
[Read for bullet 3]
We can definitely use aberration detection methods as an STD program tool to focus the review and application
of surveillance data for syphilis prevention.

19. Next Steps
Test the application at the national level using syphilis surveillance data
- Establish data review and response protocol detailing DSTDP actions in response to
findings
- Develop evaluation protocol to determine effectiveness of aberration detection
analysis for outbreak detection and response
- Demonstrate application of these methods
Develop an application with capabilities of producing output from the cumulative sums and historical limit methods for analysis of state- and county-level data
Develop a user-friendly automated application and user’s manual for project areas and evaluate its use
Try to adapt this application to other STDs
Our next steps include:
[Aside: Effectiveness may be expressed as more timely detection of an ‘outbreak.’]

20. SPECIAL THANKS TO Jim Braxton, NCHHSTP, DSTDP, SDMB
Sharon Clanton, NCHHSTP, DSTDP, SDMB
Alesia Harvey, NCHHSTP, DSTDP, SDMB
Lori Hutwagner, NCPDCID/DBPR/ESRB
Fred Rivers, NCHHSTP, DSTDP, SDMB
NCHHSTP, DSTDP, SDMB
National Center for HIV/AIDS, Viral Hepatitis, STD, and TB Prevention
Division of STD Prevention
Statistics and Data Management Branch
NCPDCID/DBPR/ESRB
National Center for Preparedness, Detection, and Control of Infectious Diseases
Division of Bioterrorism Preparedness and Response
Epidemiology, Surveillance and Response Branch

21. Email Address: DCarey@cdc.gov
Questions???
Contact Information
Address:
Are there any questions???
All questions and comments can be sent to me at
The findings and conclusions in this presentation are those of the author(s) and do not necessarily represent the views of the Centers for Disease Control and Prevention.

22. BACK-UP SLIDE(S)

23. ALL CALCULATIONS ARE BASED ON THE FOLLOWING TABLE:
HISTORICAL LIMITS CALCULATION
ALL CALCULATIONS ARE BASED ON THE FOLLOWING TABLE:

24. HISTORICAL LIMITS CALCULATIONS
3 Counties Aggregated
P&S Week 34, 2007

25. Just in case someone asks about the input parameters for EARS.