1. Assessment Implementation
Performance of Routine Information System Management (PRISM)
Assessment Training
Session 8:
Assessment Implementation
MEASURE Evaluation
Date:

2. Session objectives Describe the PRISM assessment implementation steps
Understand the criteria for the adaptation of the PRISM Tools to the local context
Understand the sampling method
Become familiar with electronic PRISM data entry

3. PRISM implementation Pre-assessment steps
Establish country leadership, buy-in, and stakeholder engagement/coordination
Principle outputs:
HIS Advisory Group
Assessment management and monitoring Working Group
Set priorities and plan for the assessment
Principle products:
HIS priorities in country context: health program focus (nation-wide vs selected provinces/states); desired unit of analysis (comprehensive vs focused)
Customized PRISM Tools
Assessment Plan: sampling unit, sampling frame, sample size; key informants; assessment schedule; composition of assessment team; assessment team training schedule

4. Steps to conducting an assessment
PRISM implementation
Steps to conducting an assessment
Manage and monitor data collection
Assemble a core team of data collectors and supervisors
Train the data collectors
Select sites to conduct the assessment; identify key informants to interview
Inform the assessment sites and key informants (preferably through official letters/communication media)
Arrange logistics for data collectors
Use the tools in order (Overview, Performance Diagnostic, eRHIS Performance, MAT, Office/Facility Checklist, OBAT)
Supervise data collection; monitor quality of data collected and/or data entered using an electronic data entry tool
Principle product:
Assessment data (raw data)

5. Post data-collection steps
PRISM implementation
Post data-collection steps
Analyze and assess current RHIS performance
Apply PRISM data analysis plan
Prepare tables, charts, assessment score matrix, etc.
Review, verify, and adjust (if anomalies/inconsistencies found)
Principle product:
PRISM Assessment Report – current RHIS performance and its major determinants; identifying major areas of attention
Translate evidence into policy, strategy, and interventions
RHIS performance improvement plan, policies
Implement the plan and monitor progress

6. Sampling method
Probability samples are better than nonprobability samples.
Probability samples are randomly selected and produce unbiased estimates.
Two commonly used sampling techniques are:
Lot Quality Assurance Sampling (LQAS)
Sample size of 19 for each supervisory area
Five supervisory areas: total sample size of 95
Probability Proportion to Size (PPS)
Minimum sample size of 100

7. What is LQAS?
LQAS is a sampling method with flexibility, rapidity, and inexpensiveness – relative to other probability survey methods – to assess performance using a smaller sample size.
A sampling method that can:
Be used locally, at the level of a “supervision area,” to identify priority areas or indicators that are not reaching average coverage or an established benchmark
Provide an accurate measure of coverage or health system quality at a more aggregate level (e.g., program catchment area or district)
Produce data for local management decision making and for sharing information across supervision areas
Be used for quality assurance using a “minimal sample”, “maximal security” principle:
Most frequently used size = <20 per supervision area
Larger sizes are seldom needed

8. What a sample of 19 can tell us
A sample of 19 provides an acceptable level of error for making management decisions
At least 92% of the time, it is good for:
Identifying whether a coverage benchmark has been reached or whether a supervision area is below the average coverage of a program
Setting priorities in a supervision area
Setting priorities among supervision areas with large differences in coverage
Deciding which are the higher performing supervision areas to learn from
Deciding which are the lower performing supervision areas
Distinguishing knowledge/practices that have high coverage from those with low coverage
Samples larger than 19 have practically the same statistical precision as samples of 19. They do not result in better information and they cost more.

9. What a sample of 19 cannot tell us
A sample of 19 is not good for:
Calculating exact coverage in a supervision area (but can be used to calculate coverage for an entire program)
Setting priorities among supervision areas with little difference in coverage

10. Burundi: 17 health provinces Mali: 8 health provinces
Sampling examples
Burundi
Mali
Burundi: 17 health provinces
Country divided into 5 zones
Random selection of 1 province in each zone (5 provinces)
Selection of 2 districts per province: the provincial capital and 1 random district (10 districts)
Random selection of another 5 districts across the 5 selected provinces (now 10+5=15 districts)
Random selection of 9 health facilities per district and selection of the district hospital (9+1=10 facilities/district)
Total of 150 health units selected
Mali: 8 health provinces
Selection of 6 provinces
Selection of 2 districts per province: the provincial capital and 1 random district (12 districts)
Random selection of another 6 districts across the 6 selected provinces (now 12+6=18 districts)
Random selection of 9 health facilities per district and selection of the district reference center (9+1=10 facilities/district)
Total 180 health units selected

11. PRISM Tools implementation

12. PRISM electronic data entry
Options available for electronic data entry:
SurveyCTO - a survey platform that allows for mobile data collection using phones, tablets, or computers
All of the PRISM Tools have been adapted into electronic data collection forms in SurveyCTO
Locally build data entry modules using Microsoft Excel, Google Sheets, or another platform
Facilitator’s Note:
If the country has decided to use SurveyCTO, refer to the guidance document for guidelines of how to upload form definitions onto SurveyCTO, customize the data entry forms, and download data files from SurveyCTO for further analysis.
Demonstrate (on a big screen) the steps in using SurveyCTO for data collection. Encourage participants to ask clarifying questions during the demonstration.

13. How to access the PRISM Series
This slide deck is one of nine in the PRISM Series Training Kit, which also includes a Participant’s Manual and a Facilitator’s Manual.
Also in the PRISM Series is a Toolkit (the centerpiece of the series) and a User’s Kit.
The PRISM Series is available in its entirety on MEASURE Evaluation’s website, here:

14. MEASURE Evaluation is funded by the United States Agency for International Development (USAID) under the terms of Cooperative Agreement AID-OAA-L It is implemented by the Carolina Population Center, University of North Carolina at Chapel Hill, in partnership with ICF International; John Snow, Inc.; Management Sciences for Health; Palladium; and Tulane University. The views expressed in this presentation do not necessarily reflect the views of USAID or the United States government.