1. Utilizing Evidence to Guide Program Implementation
The Use of Routine Lot Quality Assurance Sampling (LQAS) Surveys at Household Level
Kironde S, Ekochu E, Businge D, Nkwake A, Kiracho G John Snow Inc./Uganda Program for Human and Holistic Development (JSI/UPHOLD)
XVII International Conference on AIDS, August 3-8, 2008

2. Purpose of the Presentation
To briefly introduce the Lot Quality Assurance Sampling (LQAS) survey methodology
To share other illustrative results from LQAS surveys and how these have been utilized in various technical areas JSI/UPHOLD has supported
To share lessons learned, benefits and challenges

3. LQAS is a simple, low cost random sampling survey methodology…
Refers to ‘Lot Quality Assurance Sampling’
Introduced in 1920s as an adaptation for quality control in industry
In Public Health, can be used locally at the level of a ‘supervision area’ as is subsequently illustrated
Can provide an accurate measure of coverage or health system quality at a more aggregate level (e.g., program catchment area, district or refugee camp)
Can enable targeting of interventions within a district

4. What are the LQAS Principles?
County A
County D
County E
County C
County B
Assume a program covers a whole district
Each county is then called a ‘Supervision Area’ and the district a ‘Supervision Unit’
LQAS utilizes a minimum of 19 items (e.g., households, schools, health units) from each ‘supervision area’ in order to assess an indicator

5. What can LQAS tell us? Good coverage Below desired coverage
Indicator: Percent of children under age five who slept under a mosquito net the night before the survey
Good coverage
County A = 80
County D = 20
County E = 45
County C = 85
County B = 35
Below desired coverage

6. What can be learned from the findings?
Good and above established benchmark
Study and learn what is working well
Identify what can be applied to other Supervision areas
Learn causes of low coverage
Focus efforts and resources on these supervision areas
Poor or below the established
Benchmark
Improve coverage of whole program area by improving coverage in these SAs

7. Normally, a sample size of 19 is utilized from each supervision area…
Little is added to the precision of the measure by using a sample larger than 19, notwithstanding the level of coverage to be assessed
Sample sizes less than 19 however, see a rapid deterioration in the precision of the measure. This is particularly problematic when coverage benchmarks vary

8. Why LQAS was chosen in Uganda
Decentralization in the 1990s brought a new set of demands and challenges as planning and implementation of social services became a core mandate of districts
Existing information systems had certain unique challenges e.g.:
The Demographic Health Survey (UDHS) though household based is every five years (long interval for projects) and does not provide district-specific information
The Health Management Information System (HMIS) is mainly facility-based, which presents potential problems with representativeness in the general population

9. LQAS implementation is representative of
all regions in Uganda
UPHOLD Supported District
Key:

10. JSI/UPHOLD has institutionalized LQAS at district level for evidence-based planning and decision-making
District capacity to annually collect data for planning, monitoring and evaluation using LQAS has been enhanced:
303 district staff in 37 districts have been trained to use the methodology
Four LQAS surveys have been carried out (2004, 2005, 2006 and 2007) hence making trend analysis possible for each district
Annual presentation and discussion of district-specific LQAS results is conducted in each participant district
User friendly district and county/sub-county specific data is now available for evidence-based planning and tracking of progress across a wide range of parameters
Over time other Development Partners have also participated in the annual LQAS surveys – something which shows buy-in for the methodology

11. Illustrative Results

12. Uptake of HIV Counselling & Testing Services can be tracked
Percent of adults who tested for and received their HIV test results
Trained ~18,400 service providers
Increased HIV testing sites from 32 to 683 (including outreaches)
Provided grant support to AIC
Awarded 29 grants to CSOs to promote HIV/AIDS related activities
Awarded HIV/AIDS grants to 28 district local governments
Procured and distributed 1,331 bicycles to support community mobilization
Used BCC to mitigate against stigma, gender based violence and other factors that hinder uptake of HCT services
Supported the formation and functioning of over 1,700 Radio Listening Clubs reaching ~11m people
62% of individuals received HCT from outreaches or their homes
Support supervision of HIV/AIDS services with TB, STIs and malaria
Increased from; 982,189 in 2004 to 1,759,941 in 2007

13. Malaria Interventions at Household level can also be tracked
Percent of children under-5 years sleeping under a treated net in the previous night
UPHOLD has supported the re-treatment of 511,990 nets in 6 original UPHOLD supported districts in 4 rounds of net re-treatment ( )*
UPHOLD has distributed 311,311 insecticide treated nets to under fives since 2004
Increased BCC campaigns on use of ITNs through LG and CSO grants as well as Radio Listening Clubs
In 2007, GFATM financed the procurement and distribution of ~1.8m ITNs – some of them n UPHOLD supported districts
Increased from; 262,117 in 2004 to
908,852 in 2007
* DFID and GFATM supported the Net re-treatment Campaign of 2005

14. Fewer Children Under 5 Years Reported Fever (2004-07)
Percent of children under 5 years who had fever in past 2 weeks prior to the survey
Increased BCC campaigns on use of bed nets and other forms of malaria prevention
By 2007, ~54% of households owned a net of any kind in UPHOLD supported districts (c.f. 34% nationally [UDHS, 2006])
UPHOLD supported four net re-treatment campaigns to increase efficacy of bed nets (as per previous slide)
Children with fever have decreased from 1,250,095 in 2004 to 724,597 in 2007

15. Deliveries in Formal Health Sector can be tracked
Percent of deliveries in last two years preceding the survey that took place in a health facility
Trained ~1,862 health workers on Reproductive Health
Awarded grants to civil society (e.g., UPMA) to train private midwives and to provide services
Supported five grantees to mobilize communities for Reproductive Health services
Procured and provided about 26,000 ‘Mama Kits’ to pregnant women in the ‘conflict’ districts of Northern Uganda

16. LQAS can been used to estimate Bed Net requirements at district level

17. Knowledge of population size and ITN coverage can accurately future estimate demand
District
Estimated Population of <5s (2006)
% ITN Coverage by Sept (LQAS)
% ITN Coverage by Sept (LQAS)
# of ITNs distributed by UPHOLD Dec 2005 – Mar 2006
Estimated % increase in bed net coverage in <5s
Estimated % new ITN coverage after distribution
Actual ITN coverage as per 2006 LQAS Survey
p Value
Bugiri
100,104
4.2%
19.6%
13,000
12.1%
31.7%
32.1%
0.980 (ns)
Bushenyi
159,920
9.4%
22,800
14.1%
23.5%
25.8%
0.518 (ns)
Mayuge
75,259
2.1%
4.7%
10,000
13.1%
17.8%
18.4%
0.981 (ns)
Rukungiri
58,993
3.7%
10.5%
9,931
16.7%
27.2%
23.7%
0.513 (ns)

18. Graphical Illustration
The 2006 LQAS Survey validated the accuracy of the intended coverage estimates

19. LQAS results compare well with other standard Survey Methodologies
SAMPLE INDICATORS
LQAS results compare well with other standard Survey Methodologies
Comparison Indicator
Household Survey
HIV/AIDS
LQAS 2004
UHSBS*
% Adults (15 years and above) who reported being tested for HIV and received their results 16 14
Malaria
LQAS
2006
UDHS** 2006
% Children under five with fever in two weeks preceding the survey 43 41
% Children with fever who received timely treatment 77 75
% Households with a bed net of any kind 39 34
* UHSBS = Uganda HIV/AIDS Sero-Behavioural Survey **UDHS = Uganda Demographic and Health Survey, 2006

20. Benefits of LQAS as a Survey Methodology
Low sample size needs (n=19 in most cases)
Simple to apply yet has very specific conclusions
District level people can be trained to entirely ‘own’ this methodology
Provides reliable information at low & affordable cost
Fast – ‘supervision areas’ are able to conduct self-evaluation and obtain results immediately after the survey
Results are locally relevant and can be utilized in district level annual planning and decision-making

21. Benefits of LQAS to Districts
Change of mindsets towards reliance on data for evidence-based planning
Districts have relied on LQAS data to identify priority target areas for implementation and to focus work plans
Districts can use LQAS to do own evaluations in different technical areas e.g., education, agriculture etc.
Strengthened partnership between districts and NGOs in provision of related services

22. Challenges
Initial costs for training and capacity building may be quite high
Being a relatively new methodology, there are concerns about the validity and reliability
May require significant district personnel time and hence requires appropriate pre-planning
Cannot answer the ‘why?’ Follow up studies are required to establish the reasons for low coverage/poor performance

23. Acknowledgements
JSI/UPHOLD would wish to thank the following for their contribution to this work:
USAID for provision of funding
District Local Governments for participation in the surveys
UNICEF, NUMAT for participation and part funding of some of the LQAS Surveys
All respondents at household and facility level