1. Unit 9: Evaluating a Public Health Surveillance System #1-9-1

2. Warm Up Questions: Instructions v Take five minutes now to try the Unit 9 warm up questions in your manual. v Please do not compare answers with other participants. v Your answers will not be collected or graded. v We will review your answers at the end of the unit. # 1-9-2

3. What You Will Learn v By the end of this unit you should be able to: © list tasks for evaluating a surveillance system © develop a plan for evaluating your own country’s surveillance system # 1-9-3

4. Why Evaluate? v Evaluation ensures that your HIV/AIDS surveillance system remains effective as the epidemic shifts over time. v If your system is no longer effective, you will not have the right information to control HIV/AIDS. # 1-9-4

5. Purpose of Evaluation v Appraise and prioritise the disease events to be kept under surveillance v Assess how the system can detect and report these diseases  Assess the quality of the epidemiologic information produced # 1-9-5

6. Purpose of Evaluation, Cont. v Assess how the system can respond to these diseases v Assess how surveillance results affect disease control and policy v Identify which elements of the system can be enhanced in order to improve the quality of information # 1-9-6

7. Figure 9.1. Elements of an Evaluation Evaluation  Document current state of the surveillance system  Identify strengths and weak points  Recommend improvements  Define training requirements or gaps  Justify resources  Additional funding  More and better training  Improved surveillance for better disease control possible outcomes # 1-9-7

8. Six Evaluation Tasks 1.Engage the stakeholders in the evaluation. 2.Describe the surveillance system to be evaluated. 3.Focus the evaluation design. 4.Gather credible evidence regarding the performance of the surveillance system. # 1-9-8

9. Six Evaluation Tasks, Cont. 5.Justify and state conclusions and make recommendations. 6.Ensure use of evaluation findings and share lessons learned. # 1-9-9 For more detail, refer to the Updated Guidelines for Evaluating Public Health Surveillance Systems (Centers for Disease Control and Prevention, 2001), available at www.cdc.gov/mmwr/preview/mmwrhtml/rr5013a1.htm www.cdc.gov/mmwr/preview/mmwrhtml/rr5013a1.htm

10. Task 1: Engaging Stakeholders in the Evaluation v Stakeholders include: © Public health practitioners © Healthcare providers © Data providers and users © Community representatives © Governments at the district, provincial and national levels © Governmental, non-governmental, professional and private non-profit organisations # 1-9-10

11. Task 1: Engaging Stakeholders in the Evaluation, Cont. v Examples of ways to engage stakeholders include: © Hold a community meeting to discuss plans for the evaluation. © Hold one-on-one meetings with the key people listed above. © Invite participants to join the evaluation team. # 1-9-11

12. Class Discussion: Task 1 v Task 1 is to ‘engage stakeholders.’ © Who would this be, in your case? © Why is this task important? # 1-9-12

13. Task 2: Describe the Surveillance System to be Evaluated v The evaluation should describe: © The public health importance of the health- related event under surveillance © The purpose and operation of the system © The resources used to operate the system # 1-9-13

14. Class Discussion: Task 2 v Task 2 is to ‘describe the surveillance system.’ © In your case, where would you find the type of information listed in the task? # 1-9-14

15. Task 3: Focus the Evaluation Design v Determine the specific purpose of the evaluation. v Identify stakeholders who will receive findings. v Consider what to do with the information generated from the evaluation. v Specify the questions that will be answered. v Determine the standards for assessing the performance of the system. # 1-9-15

16. Class Discussion: Task 3 v For task 3: © What are some ways you can think of to design an evaluation? © How would you know what to evaluate? # 1-9-16

17. Task 4: Gather Credible Evidence Regarding the Performance of the Surveillance System v Simplicity v Flexibility v Data quality v Acceptability v Positive predictive value v Representativeness v Timeliness v Stability v Data use/ dissemination Describe the surveillance system in terms of its: # 1-9-17

18. Class Discussion: Task 4 v Task 4 concerns gathering data. © Should there be a time limit on gathering data? © What would you do if you found that some clinics didn’t have data or had incomplete data? # 1-9-18

19. Task 5: State Conclusions and Make Recommendations v Conclusions can be justified through appropriate analysis, synthesis, interpretation and judgment of the gathered evidence. v Recommendations should address modifications and continuations of the public health surveillance system. # 1-9-19

20. Class Discussion: Task 5 v In task 5, you are asked to make recommendations. © How is this typically done in your district or region? # 1-9-20

21. Task 6: Ensure Use of Findings and Share Lessons Learned v Develop strategies for communicating the findings from the evaluation. v Tailor recommendations to relevant audiences. v Recommendations for improvements should be distributed to all partners and sites involved in sentinel surveillance. # 1-9-21

22. Class Discussion: Task 6 v Task 6 concerns communications. © How would you decide whom to send your final report to? © In your case, would the report go to subject matter experts, perhaps Ministry of Health HIV/AIDS experts? © If not, how could you make sure that the people reading the report would understand what you were saying? # 1-9-22

23. In Summary v Evaluating your HIV/AIDS surveillance system is important, to ensure that remains effective as the epidemic changes over time. v The tasks that you should implement during this evaluation include: © engaging stakeholders © describing the surveillance system © focusing the evaluation design © gathering evidence on the system’s performance © stating conclusions and recommendations © sharing the lessons learned # 1-9-23

24. Warm Up Review v Take a few minutes now to look back at your answers to the warm up questions at the beginning of the unit. v Make any changes you want to. v We will discuss the questions and answers in a few minutes. # 1-9-24

25. Answers to Warm Up Questions 1. List three stakeholder groups that should be engaged during the evaluation of the surveillance system. # 1-9-25

26. Answers to Warm Up Questions 1. List three stakeholder groups that should be engaged during the evaluation of the surveillance system.  public health practitioners  healthcare providers  government officials  representatives of affected communities  non-profit and donor organizations  etc. # 1-9-26

27. Answers to Warm Up Questions, Cont. 2. If there is a high probability that cases identified by the surveillance system are actually cases of HIV infection, the system is said to have high: a. sensitivity b. representativeness c. acceptability d. positive predictive value # 1-9-27

28. Answers to Warm Up Questions, Cont. 2. If there is a high probability that cases identified by the surveillance system are actually cases of HIV infection, the system is said to have high: a. sensitivity b. representativeness c. acceptability d. positive predictive value # 1-9-28

29. Small Group Discussion: Instructions v Get into small groups to discuss these questions. v Choose a speaker for your group who will report back to the class. # 1-9-29

30. Small Group Reports v Select one member from your group to present your answers. v Discuss with the rest of the class. #1-9-30

31. Case Study: Instructions v Try this case study individually. v We’ll discuss the answers in class. # 1-9-31

32. Case Study Review v Follow along as we go over the case study in class. v Discuss your answers with the rest of the class. #1-9-32

33. Questions, Process Check v Do you have any questions on the information we just covered? v Are you happy with how we worked on Unit 9? v Do you want to try something different that will help the group? # 1-9-33