1. BENCHMARK REPORT

2. AgendaAgenda Beginner Level Basics Beginner Level Basics IntroductionIntroduction Generating reportsGenerating reports Interpreting reportsInterpreting reports Beginner to Intermediate Level Details Beginner to Intermediate Level Details Other report optionsOther report options General tipsGeneral tips Resources Resources 2 REMINDER: the information generated by this report is only as accurate as the data entered into the NCIR!

3. IntroductionIntroduction 3

4. Benchmark:Benchmark: A benchmark is a standard of excellence or achievement that similar things are compared to or measured against (in order to evaluate performance or level of quality). 4

5. NCIR Benchmark Report: The NCIR benchmark report assesses the immunization records of a particular group of your clients and tells you HOW MANY, WHAT PERCENT, and WHICH of these clients have met and not met a particular “benchmark” for immunizations. 5

6. Key concept: Select a group of clients to assess.Select a group of clients to assess. Select the criteria for meeting the benchmark (BM).Select the criteria for meeting the benchmark (BM). The report determines which of these clients have met the benchmark criteria and which have not.The report determines which of these clients have met the benchmark criteria and which have not. within any age or birth date rangewithin any age or birth date range (( clients who have not met benchmark Up-to-date with: specific vaccine(s) By: specific age (for childhood vaccines) Up-to-date with: specific vaccine(s) By: specific age (for childhood vaccines) selected clients clients who have met benchmark (( (( ((

7. Select a group of clients to assess.Select a group of clients to assess. Select the criteria for meeting the benchmark.Select the criteria for meeting the benchmark. Report determines which of these clients have met the benchmark criteria and which have not.Report determines which of these clients have met the benchmark criteria and which have not. within any age or birth date rangewithin any age or birth date range Key concept: select benchmark criteria select clients

8. To learn the number and ercent of clients up- to-date (UTD) with a particular vaccine or group of vaccines:To learn the number and ercent of clients up- to-date (UTD) with a particular vaccine or group of vaccines: by a specific age (for childhood immunizations)by a specific age (for childhood immunizations) by the date the report was runby the date the report was run For a list of clients that indicates (for each selected vaccine) whether the client was/had:For a list of clients that indicates (for each selected vaccine) whether the client was/had: UTD by benchmark age; late UTD; due (and if so, number of doses received); refusedUTD by benchmark age; late UTD; due (and if so, number of doses received); refused Why generate a report? number and percent list of clients

9. Why generate a report? number & percent Info list of clients

10. Generating Reports Beginner Level 10

11. To open: Click “benchmark report” (on the left side of the NCIR screen, under the “reports” section)Click “benchmark report” (on the left side of the NCIR screen, under the “reports” section) Note: Benchmark reports must be generated from an “Administrator” account. 11

12. 12 These are the main sections of concern for beginner level reports. Beginner Reports 66 33 select clients select benchmark criteria Select Evaluation Date 55

13. 33 13 The 1 st /default option generates results that can be confusing.* Until comfortable with report, use another option.The 1 st /default option generates results that can be confusing.* Until comfortable with report, use another option. Birth date range: If you have a specific birth date range, enter dates or select using calendar icon.Birth date range: If you have a specific birth date range, enter dates or select using calendar icon. Age range: If you don’t need a specific birth date range, the age range option is simpler. Be sure to specify “days,” “weeks,” or “years” in the dropdown menus.Age range: If you don’t need a specific birth date range, the age range option is simpler. Be sure to specify “days,” “weeks,” or “years” in the dropdown menus. *For more info on this option, see the benchmark report manual. Select: group of clients to assess (use only one of the three options)

14. 14 Select: evaluation date Type/select the current date (the date that you are running the report). You must enter an evaluation date or the report will not run.Type/select the current date (the date that you are running the report). You must enter an evaluation date or the report will not run. A date that is different from the report generated date can impact the results (designed for advanced level use).*A date that is different from the report generated date can impact the results (designed for advanced level use).* *For more information on evaluation dates, see the benchmark report manual. 55

15. The report will review the immunization status of each client to see if it meets this BM criteria that you selected.The report will review the immunization status of each client to see if it meets this BM criteria that you selected. Select: benchmark criteria Column: Vaccine Rows: Age/Age Group In cell: Number of doses required to meet BM Column: Vaccine Rows: Age/Age Group In cell: Number of doses required to meet BM 66

16. 16 Childhood benchmarks (BM): when you want to know if a group of clients are UTD with vaccines routinely given in childhood.Childhood benchmarks (BM): when you want to know if a group of clients are UTD with vaccines routinely given in childhood. As childhood vaccine recommendations are based on age, the BM report includes age and immunization criteria. (Aka: to meet a childhood benchmark, clients must be UTD by a particular age)As childhood vaccine recommendations are based on age, the BM report includes age and immunization criteria. (Aka: to meet a childhood benchmark, clients must be UTD by a particular age) 66 Select: benchmark criteria Childhood benchmarks

17. 17 Adolescent/Adult Benchmark: when you want to know if a group of clients (within any age or birth date range, but usually clients over age 6) are UTD with vaccines routinely given in adolescence or adulthood.Adolescent/Adult Benchmark: when you want to know if a group of clients (within any age or birth date range, but usually clients over age 6) are UTD with vaccines routinely given in adolescence or adulthood. There is no age criteria for these vaccines. Clients meet the benchmark if they were UTD when the report was generated.There is no age criteria for these vaccines. Clients meet the benchmark if they were UTD when the report was generated. 66 Select: benchmark criteria Note: Both males and females will be assessed for HPV.Note: Both males and females will be assessed for HPV. Adolescent/Adult benchmarks

18. 18 You can select a full row: Click on the first cell of the row that you want to select. The row will become highlighted.You can select a full row: Click on the first cell of the row that you want to select. The row will become highlighted. Select: benchmark criteria In this example: all assessed clients who are UTD with 4DTaP, 3HepB, 3Hib, 1MMR, 3Polio, 4Pnuemo, & 1Var by 24 months of age will meet this benchmark. 66

19. 19 You can select a single cell: Just click on it.You can select a single cell: Just click on it. Select: benchmark criteria In this example: the report will determine which of the assessed clients were UTD with Hep B at 16 months. You can select multiple cells within the same row, but you cannot select cells in more than one row.You can select multiple cells within the same row, but you cannot select cells in more than one row. 66 In this example: clients UTD with 1Mening and 3 doses of HPV will meet BM

20. 20 Select: benchmark criteria To deselect: Just click on the cell and it will no longer be highlighted.To deselect: Just click on the cell and it will no longer be highlighted. In this example: clients must have completed the 4:3:3:1:3:1 series by 24 months of age in order to meet the benchmark 66

21. To process: 21 Click generate (only one time, please) and then wait…Click generate (only one time, please) and then wait…

22. To view report: 22 Report will take a few minutes to complete. Click “refresh” occasionally to determine report status.Report will take a few minutes to complete. Click “refresh” occasionally to determine report status. When status is “DONE,” click on blue benchmark link to view report.When status is “DONE,” click on blue benchmark link to view report.

23. Interpreting Report Results For beginner level reports 23

24. 24 So, this is the example criteria that was selected, and now we will look at the report that would be generated from this criteria. Beginner Reports 66 33 Report run date: 1/9/13 55 66 33 Clients: age 24-36 months Benchmark: UTD with 4:3:3:1:3:4:1 series by 24 months

25. List of selected criteria for report (“filter conditions”)List of selected criteria for report (“filter conditions”) Number & percent InformationNumber & percent Information List of clientsList of clients Key sections of report report criteria number and percent Info list of clients

26. Each selection you made for the report is listed hereEach selection you made for the report is listed here Tells you exactly what the report meansTells you exactly what the report means Selected criteria for report Report generated” and “Evaluation” dates are the same “Report generated” and “Evaluation” dates are the same Selected Clients: 24- 36 months of age Selected benchmark immunizations” 4:3:3:1:3:1 by 24 months

27. Understanding Report Criteria 27 Clients:Clients: Benchmark:Benchmark: Clients:Clients: Benchmark:Benchmark:

28. The primary outcome of the report.The primary outcome of the report. Found on the last two lines of the “filter conditions” section.Found on the last two lines of the “filter conditions” section. The first number is the number of total clients that were assessed.The first number is the number of total clients that were assessed. Number & percent info number and percent info

29. Understanding number and percent info: Line 1: UTD with BM vaccines by BM ageLine 1: UTD with BM vaccines by BM age Line 2: UTD with BM vaccines as of date report generated ( includes clients on the first line plus those who were late UTD)Line 2: UTD with BM vaccines as of date report generated ( includes clients on the first line plus those who were late UTD) Misleading. Pretend this says: “Including late UTD” 4 of 10 clients (40%) UTD by 24 months 5 of 10 clients (50%) UTD by the report generated date (the 4 who met the benchmark and 1 late)

30. Adolescent/Adult benchmarks have no age criteriaAdolescent/Adult benchmarks have no age criteria Clients cannot be late UTDClients cannot be late UTD The two number and percent lines are identicalThe two number and percent lines are identical For Adolescent/Adult BM: 5 of 12 clients UTD/ met all BM criteria Clients cannot be late UTD. So, including late UTD clients, the same 5 of 12 (42%) met all BM criteria

31. . Client list Lists clients who did not meet all benchmark criteria (Note: you can choose which clients are listed- discussed on following slides).Lists clients who did not meet all benchmark criteria (Note: you can choose which clients are listed- discussed on following slides). report criteria Number and percent Info List of Clients

32. Did not meet benchmark N C: UTD (“completed”) vaccines, but late UTDN C: UTD (“completed”) vaccines, but late UTD N # : Not UTD (and the # of doses received)N # : Not UTD (and the # of doses received) N : Not UTD (did not receive any doses of vaccine)N : Not UTD (did not receive any doses of vaccine) Understanding the client list Met benchmark Y : met all benchmark criteria for that vaccineY : met all benchmark criteria for that vaccine Column: Selected BM Vaccine Parenthesis: number of doses required to be UTD) Column: Selected BM Vaccine Parenthesis: number of doses required to be UTD)

33. 33 To view a specific record: This list is an easy way to quickly review client names, addresses, and phone numbers and see if any are obviously missing or wrong (e.g. out of state).This list is an easy way to quickly review client names, addresses, and phone numbers and see if any are obviously missing or wrong (e.g. out of state). You can view a specific client record by clicking on the link on the client’s last name.You can view a specific client record by clicking on the link on the client’s last name.

34. 34 Gender is coded as “U” (unknown). Missing address You can view client info but you cannot make changes.You can view client info but you cannot make changes. When you close out of the screen you’re still able to view report.When you close out of the screen you’re still able to view report. Client comments Reviewing client information

35. 35 Updating client information Please correct or update any info that is incorrect or missing (Note: Changes will not show up on a report until the next day).Please correct or update any info that is incorrect or missing (Note: Changes will not show up on a report until the next day). Steps: copy the Client ID and use it to search for the client through “manage clients.” Make edits.Steps: copy the Client ID and use it to search for the client through “manage clients.” Make edits.

36. Saving/exporting report Be sure to save the report! Once you run the next report, any previously run report will disappear!Be sure to save the report! Once you run the next report, any previously run report will disappear! In order to save the report you must first export it as text or a spreadsheet or display it as a PDFIn order to save the report you must first export it as text or a spreadsheet or display it as a PDF Filter conditions can only be saved and viewed in PDF format.Filter conditions can only be saved and viewed in PDF format. You can use a spreadsheet if you want to sort or further filter the client information.You can use a spreadsheet if you want to sort or further filter the client information. *See user manual for details on why you want to export information and what you can do with it.

37. To re-open a report: Once you leave the BM screen you can still return to the last report that was generated.Once you leave the BM screen you can still return to the last report that was generated. Click on “check benchmark.”Click on “check benchmark.” 37

38. Other options when generating the benchmark Beginner to Intermediate LevelBeginner to Intermediate Level 38

39. 66 33 39 Again, here is the screen where you can build the report. 55 11 22 44 Screen used to build report

40. 40 Sections that can be skipped but may at times be useful. 11 22 44 Screen used to build report

41. 11 41 Select: client population Only relevant if your practice is associated in the NCIR with multiple sites/clinics and you want to assess clients from only one of those sites.Only relevant if your practice is associated in the NCIR with multiple sites/clinics and you want to assess clients from only one of those sites. If the above applies, select the 1 st option and specify a site using the dropdown menu.If the above applies, select the 1 st option and specify a site using the dropdown menu. Otherwise, leave the 2 nd /default option selected.Otherwise, leave the 2 nd /default option selected. Dropdown Menu

42. 11 42 Select: client population (LHDs only) Local Health Dept. accounts display two extra client population options:Local Health Dept. accounts display two extra client population options: 3rd option: Assesses only people who are listed in the NCIR as residents of your county (whether or not they are “active” clients).3rd option: Assesses only people who are listed in the NCIR as residents of your county (whether or not they are “active” clients). 4th option: Assesses your “active” clients (regardless of their county of residence) AND people who are listed in the NCIR as residents of your county,4th option: Assesses your “active” clients (regardless of their county of residence) AND people who are listed in the NCIR as residents of your county,

43. 22 43 Select: clients displayed on the client list This section determines which clients are displayed in the client list. It has no other impact.This section determines which clients are displayed in the client list. It has no other impact. 1 st /default option: Generates a list of clients who did NOT meet the benchmark.1 st /default option: Generates a list of clients who did NOT meet the benchmark. 2 nd option: Generates a list of clients who DID meet the benchmark.2 nd option: Generates a list of clients who DID meet the benchmark. 3 rd option: Generates a list of all assessed clients (those who met benchmark and those who did not).3 rd option: Generates a list of all assessed clients (those who met benchmark and those who did not).

44. You can run the report so that clients who refused a vaccine are assessed as UTD with that vaccine.*You can run the report so that clients who refused a vaccine are assessed as UTD with that vaccine.* 1 st /default option: only considers vaccinated clients as UTD.1 st /default option: only considers vaccinated clients as UTD. 2 nd option: considers both vaccinated clients and those who refused the vaccine as UTD.2 nd option: considers both vaccinated clients and those who refused the vaccine as UTD. *Based on “client comments” recorded in the NCIR. For more info, see user manual. 44 Select: how to assess refusals

45. General Report Tips 45

46. RefusalsRefusals One refusal of a vaccine is counted on the benchmark report as one “dose” of that vaccine.One refusal of a vaccine is counted on the benchmark report as one “dose” of that vaccine. Example: If a client has 2 refusals for HepB, then the client would meet the benchmark for 1Hep but would not meet a benchmark that was run for 3Hep.Example: If a client has 2 refusals for HepB, then the client would meet the benchmark for 1Hep but would not meet a benchmark that was run for 3Hep. You can now record multiple client refusals for the same vaccine in “client comments,” as long as each entry has a different “applies-to” date.You can now record multiple client refusals for the same vaccine in “client comments,” as long as each entry has a different “applies-to” date. You may want to review previous refusals and add “as-of” date.You may want to review previous refusals and add “as-of” date. 46

47. Refusal example You want to know: Clients: age 13-19Clients: age 13-19 Benchmark: UTD with 3HPVBenchmark: UTD with 3HPV 47

48. 48 If refusals do not meet benchmark

49. If refusals meet benchmark

50. 50 Refusals on client list An “R” in the client list indicates that the client is considered UTD for all doses of the vaccine as a result of refusals. Note: “R”s only display on reports run for childhood BMs. For adolescent/adult BMs, UTD as a result of refusals is indicated the same as in UTD as a result of vaccination (“Y”).

51. Childhood BMs: You can assess clients within any age or birth date range.Childhood BMs: You can assess clients within any age or birth date range. Adolescent/Adult BMs: Report will not run if all selected clients are age 6 and under. You can assess clients within any age or birth date range as long as part of the range includes clients who are over age 6 years.Adolescent/Adult BMs: Report will not run if all selected clients are age 6 and under. You can assess clients within any age or birth date range as long as part of the range includes clients who are over age 6 years. 51 Client age and BM age

52. Flu shots for clients age 6 and under You want to know: Clients: ages 24-36 monthsClients: ages 24-36 months Benchmark: UTD with flu vaccineBenchmark: UTD with flu vaccineSteps: Select clients using an age or birth date range that includes people older than 6. For example: age 2 to 7 years.Select clients using an age or birth date range that includes people older than 6. For example: age 2 to 7 years. You could export to Excel, arrange columns in order of date of birth, and quickly narrow the list down to only clients in your desired birth date range.You could export to Excel, arrange columns in order of date of birth, and quickly narrow the list down to only clients in your desired birth date range. 52

53. Flu shots for clients age six and under 53

54. Report size It can be more manageable and useful to run reports for a narrow birth date or age range (to reduce the number of clients included).It can be more manageable and useful to run reports for a narrow birth date or age range (to reduce the number of clients included). Similarly it can be more manageable and useful to run reports on a regular, frequent basis in order to follow-up with clients throughout the year.Similarly it can be more manageable and useful to run reports on a regular, frequent basis in order to follow-up with clients throughout the year.

55. QUESTIONS?QUESTIONS? 55