Presentation on theme: "Evaluation of surveillance systems"— Presentation transcript:
1Evaluation of surveillance systems Preben Aavitsland
2Information for action! SurveillanceSurveillance is the ongoing systematic collection, collation, analysis and interpretation of data; and the dissemination of information (to those who need to know) in order that action may be takenInformation for action!
3Analysis, interpretation Feedback, recommendations The surveillance loopHealth care systemSurveillance centreEventActionDataInformationReportingAnalysis, interpretationFeedback, recommendations
4Importance of evaluation QualityOften neglectedBasis for improvementsObligationDoes the system deliver?Credibility of public health serviceLearning processEPIET training objective”Do not create one until you have evaluated one”
5General framework A. Engagement of stakeholders B. Evaluation objectiveC. System descriptionD. System performanceE. Conclusions and recommendationsF. Communication
11C. System description 1 Public health rationale (why?) 2 Objectives (what?)3 Operations (how?)4 Resources (how much?)Extreme learning value!!!!
121. Rationale for surveillance The diseaseSeverityFrequencyCommunicabilityInternational obligationsCostsPreventabilitySocietyPublic and mass media interestWill to preventAvailability of data
132. Objectives of system Documented? SMART? If not = trouble Specific MeasurableAction oriented [information] in order to [action]RealisticTime frame specified
14Possible objectives of surveillance Detect outbreaksMonitor trends (by time, place, person)towards a control objectiveas programme performanceas intervention evaluationEstimate future disease impactCollect cases for further studies….in order to [action]
15Objectives”To have a continuous overview of the spread of the disease in Norway in order to target preventive measures and plan resource needs.”
163. Operations of system Health events under surveillance Type of event: exposure -> infection -> disease / outbreaks -> outcomeCase definitionsLegal frameworkOrganisational frameworkComponentsFlow chartDescription
17Analysis, interpretation Feedback, recommendations The surveillance loopHealth care systemSurveillance centreEventActionDataInformationReportingAnalysis, interpretationFeedback, recommendations
19Components of system Population under surveillance Period of data collectionType of information collectedData sourceData transferData management and storageData analysis: how often, by whom, howDissemination: how often, to whom, howConfidentiality, security
204. Resources for system operation Funding sourcesPersonell time (= €)Other costsTrainingMailFormsComputers...
23System performance Does it work? System attributes Is it useful? SimplicityFlexibilityData qualityAcceptabilitySensitivityPositive predictive valueRepresentativenessTimelinessStabilityIs it useful?Use of informationUsersActions takenLink to objectives
32Usefulness Event Action Data Information Health care system Surveillance centreEventActionDataInformation
33Meeting objectives? Was information produced? TrendsOutbreaksFuture impactCases for further studiesWas information used, and by whom?Actions: listConsequences: list
34Usefulness Ex 1 (mid 1990s): Ex 2 (1999): Information: Aid workers infected in AfricaAction: Revision of recruitment policyEx 2 (1999):Information: Men infected in ThailandAction: Publication --> mass media interest --> = public health warning
38Communicating findings To stakeholdersTo data providersTo public health communityReportConference presentationScientific article
39Scientific publication IntroductionEvaluation objective (B)Material and methodsMethods of evaluation (B)ResultsSystem description (C)System performance (D)DiscussionSources of error and biasConclusions and recommendations (E)AcknowledgmentsStakeholders (A)
41LiteratureCDC. Updated guidelines for evaluating public health surveillance systems. MMWR 2001; 50 (RR-13): 1-35WHO. Protocol for the evaluation of epidemiological surveillance systems. WHO/EMC/DIS/97.2.Romaguera RA, German RR, Klaucke DN. Evaluating public health surveillance. In: Teutsch SM, Churchill RE, eds. Principles and practice of public health surveillance, 2nd ed. New York: Oxford University Press, 2000.