Presentation on theme: "Evaluation of surveillance systems"— Presentation transcript:
1 Evaluation of surveillance systems Preben Aavitsland
2 Information for action! SurveillanceSurveillance is the ongoing systematic collection, collation, analysis and interpretation of data; and the dissemination of information (to those who need to know) in order that action may be takenInformation for action!
3 Analysis, interpretation Feedback, recommendations The surveillance loopHealth care systemSurveillance centreEventActionDataInformationReportingAnalysis, interpretationFeedback, recommendations
4 Importance of evaluation QualityOften neglectedBasis for improvementsObligationDoes the system deliver?Credibility of public health serviceLearning processEPIET training objective”Do not create one until you have evaluated one”
5 General framework A. Engagement of stakeholders B. Evaluation objectiveC. System descriptionD. System performanceE. Conclusions and recommendationsF. Communication
11 C. System description 1 Public health rationale (why?) 2 Objectives (what?)3 Operations (how?)4 Resources (how much?)Extreme learning value!!!!
12 1. Rationale for surveillance The diseaseSeverityFrequencyCommunicabilityInternational obligationsCostsPreventabilitySocietyPublic and mass media interestWill to preventAvailability of data
13 2. Objectives of system Documented? SMART? If not = trouble Specific MeasurableAction oriented [information] in order to [action]RealisticTime frame specified
14 Possible objectives of surveillance Detect outbreaksMonitor trends (by time, place, person)towards a control objectiveas programme performanceas intervention evaluationEstimate future disease impactCollect cases for further studies….in order to [action]
15 Objectives”To have a continuous overview of the spread of the disease in Norway in order to target preventive measures and plan resource needs.”
16 3. Operations of system Health events under surveillance Type of event: exposure -> infection -> disease / outbreaks -> outcomeCase definitionsLegal frameworkOrganisational frameworkComponentsFlow chartDescription
17 Analysis, interpretation Feedback, recommendations The surveillance loopHealth care systemSurveillance centreEventActionDataInformationReportingAnalysis, interpretationFeedback, recommendations
19 Components of system Population under surveillance Period of data collectionType of information collectedData sourceData transferData management and storageData analysis: how often, by whom, howDissemination: how often, to whom, howConfidentiality, security
20 4. Resources for system operation Funding sourcesPersonell time (= €)Other costsTrainingMailFormsComputers...
23 System performance Does it work? System attributes Is it useful? SimplicityFlexibilityData qualityAcceptabilitySensitivityPositive predictive valueRepresentativenessTimelinessStabilityIs it useful?Use of informationUsersActions takenLink to objectives
24 Data quality Completeness Proportion of blank / unknown responses Simple countingValidityTrue data?ComparisonRecords inspectionPatient interviews...
32 Usefulness Event Action Data Information Health care system Surveillance centreEventActionDataInformation
33 Meeting objectives? Was information produced? TrendsOutbreaksFuture impactCases for further studiesWas information used, and by whom?Actions: listConsequences: list
34 Usefulness Ex 1 (mid 1990s): Ex 2 (1999): Information: Aid workers infected in AfricaAction: Revision of recruitment policyEx 2 (1999):Information: Men infected in ThailandAction: Publication --> mass media interest --> = public health warning
38 Communicating findings To stakeholdersTo data providersTo public health communityReportConference presentationScientific article
39 Scientific publication IntroductionEvaluation objective (B)Material and methodsMethods of evaluation (B)ResultsSystem description (C)System performance (D)DiscussionSources of error and biasConclusions and recommendations (E)AcknowledgmentsStakeholders (A)
41 LiteratureCDC. Updated guidelines for evaluating public health surveillance systems. MMWR 2001; 50 (RR-13): 1-35WHO. Protocol for the evaluation of epidemiological surveillance systems. WHO/EMC/DIS/97.2.Romaguera RA, German RR, Klaucke DN. Evaluating public health surveillance. In: Teutsch SM, Churchill RE, eds. Principles and practice of public health surveillance, 2nd ed. New York: Oxford University Press, 2000.