1. Motivation & Background
Evaluating WASH Education and Training for Improved Outcomes Brittany Coff, Tommy Ka Kit Ngai, Paul Earwaker, Candice Young-Rojanschi 7th RWSN Forum, 29 November – 2 December, 2016, Abidjan, Côte d’Ivoire
Motivation & Background
Approach
Results
Centre for Affordable Water and Sanitation
Calgary, Canada; est. 2001
Non-profit consultancy that provides capacity development to WASH program implementers. Services include:
Reviewed existing practices of WASH organizations for measuring and reporting results of education and training.1
33% reported the results of their capacity building activities
All reports focused on outputs
Investigated evaluation metrics.2
Few metrics related to longer term outcomes from capacity building
Developed evaluation methodology for education and training.3
Examined over 20 evaluation frameworks
Selected the Kirkpatrick framework
Piloted and refined methodology in field studies.
The methodology examines not only individual learning but also resulting behaviour change of the learner- and community-level outcomes by using the following steps:
Determine an evaluation goal.
Develop a theory of change for the program and indicators based on the Kirkpatrick model
Develop data collection tools.
Collect data.
Identify gaps and trends from the information.
Feed results back to planning process.
Training workshops
Learning exchanges
Consulting support
Action research
Education resources
Health Workers attend WASH Awareness training
Health workers learn knowledge, attitudes, skills
Health workers more effective at spreading WASH messages in communities
Community members learn about WASH options
Community members practice better WASH
Kirkpatrick
Levels
Reaction
Learning
Behaviour
Key knowledge, skills and attitudes related to WASH that Health Workers had learned
Changes to teaching methods Health Workers in communities as a result of the workshop
Theory of Change Diagram
Changes in the community WASH knowledge and practices where the trained HEWs are working.
Example Indicator Topics for Each Level
Results
How relevant the training was
How useful the tools used in the training were
Biosand filter training workshop in Zambia
CAWST works to transfer knowledge between different levels of the water drop
The Need
Many organizations conduct education and training to improve WASH outcomes
Few understand whether training and education services result in desired outcomes
Nepal & Peru (2013)
Ethiopia (2014)
Haiti (2015)
1 in partnership with Cranfield University, UK, 2012,
2 in partnership with Cambridge University, UK, 2013,
3 in partnership with Mount Royal University, Canada, 2013
Example of theory of change and indicator topics, Ethiopia
Challenges
Next Steps
Creating a methodology that can apply to a wide range of situations
Finding a compromise between research rigour and practicality in the methodology
Accounting for practitioner abilities with respect to qualitative data analysis
Avoiding interpretive bias
Research Objectives
Support partner organisations to use the methodology
Create a training workshop and support materials
Develop a practical methodology for evaluating the effectiveness of WASH training programs
Test and refine this methodology through field pilot studies