1. Allie Davis Dr. Amy Javernick-Will and Dr. Sherri Cook
Avoiding Failure: The Use of Qualitative Comparative Analysis to Identify Pathways to Successful Sanitation Interventions
UNC Water & Health
October 16 – 20, 2017
Allie Davis Dr. Amy Javernick-Will and Dr. Sherri Cook
Department of Civil, Environmental, and Architectural Engineering Civil Systems and Engineering for Developing Communities University of Colorado Boulder

2. Acknowledgments Sridar Selvaraj Cook Research Group 2

3. Overview Failed sanitation systems impede SDGs
Qualitative Comparative Analysis can help identify conditions for success
Pathways to success can refine project planning
Observed Problem
QCA Method
Future Work
Summary
Pathways to Success 3

4. Overview Failed sanitation systems impede SDGs
Qualitative Comparative Analysis can help identify conditions for success
Pathways to success can refine project planning
Observed Problem
QCA Method
Future Work
Summary
Pathways to Success 3

5. What combinations of factors lead to successful sanitation systems?
Understanding factors that lead to success is important to achieving SDGs
Social?
Successful
Failed
Institutional?
30%
Economic?
Technical?
70%
What combinations of factors lead to successful sanitation systems?
Davis, S. (2015). “Statistics on Sanitation Failures.” Improve International.
Observed Problem
QCA Method
Future Work
Summary
Pathways to Success 4

6. India’s ambitious national initiatives and large infrastructure gaps motivate the context selection
Only 21 percent of wastewater is treated1
23 percent lack access to electricity2
50 percent of shared toilets are unused3
60 percent still go for open defecation3
67 percent use solid fuels for cooking2
1Kamyotra, J. S., and Bhardwaj, R. M. (2011). “Municipal wastewater management in India.” India Infrastructure Report, 299.
2World Health Organization, and UNICEF. (2015). Progress on sanitation and drinking water: 2015 Update and MDG Report. Joint Monitoring Programme, World Health Organization, [S.l.].
3Davis, S. (2015). “Statistics on Sanitation Failures.” Improve International.
Observed Problem
QCA Method
Future Work
Summary
Pathways to Success 5

7. Overview Failed sanitation systems impede SDGs
Qualitative Comparative Analysis can help identify conditions for success
Pathways to success can refine project planning
QCA Method
Observed Problem
Future Work
Summary
Pathways to Success 6

8. Fuzzy-set Qualitative Comparative Analysis uses set theory and Boolean algebra to analyze pathways
Define Outcomes
Select Causal Conditions
Select Cases
Data Collection
Calibrate Conditions
Analysis & Minimization
Pathways to Success
Validity Tests
QCA Method
Observed Problem
Future Work
Summary
Pathways to Success 7

9. Success is defined as the presence of use, maintenance, and performance
Define Outcomes
Select Causal Conditions
Select Cases
Data Collection
Calibrate Conditions
Analysis & Minimization
Pathways to Success
Success
Use
Maintenance
Performance
> 75% of community uses correctly & daily
90% of maintenance tasks performed correctly & on time
Treated water meets all local regulations
QCA Method
Observed Problem
Future Work
Summary
Pathways to Success 8

10. Hypothesized causal conditions come from literature and pilot studies
Define Outcomes
Select Causal Conditions
Select Cases
Data Collection
Calibrate Conditions
Analysis & Minimization
Pathways to Success
Community Participation
Training
Priorities
Skilled Operator
Priority Assessment
Community Participation in Planning
Technical Training
Addressed Sanitation Priorities
Community Contributions
Behavior Change Education
Community Participation in Construction
Institutional Support
Income Generation
Community Participation in O&M
Infrastructure
Lack of Vandalism
Ongoing External Support
Water Supply
Lack of Government Barriers
QCA Method
Observed Problem
Future Work
Summary
Pathways to Success 9

11. Resource recovery sanitation systems were selected to ensure variation within conditions
Define Outcomes
Select Causal Conditions
Select Cases
Data Collection
Calibrate Conditions
Analysis & Minimization
Pathways to Success
5 Successful
5 Failed
800 – 1200 users
Low-income
Karnataka
Dalit caste
Tamil Nadu
Peri-urban settlements
QCA Method
Observed Problem
Future Work
Summary
Pathways to Success 10

12. In-depth case studies were completed to understand each system’s story
Define Outcomes
Select Causal Conditions
Select Cases
Data Collection
Calibrate Conditions
Analysis & Minimization
Pathways to Success
Interviews
Documentation
Observations
Technical Evaluations
QCA Method
Observed Problem
Future Work
Summary
Pathways to Success 11

13. Conditions were calibrated to define in-set and out-of-set membership
Define Outcomes
Select Causal Conditions
Select Cases
Data Collection
Calibrate Conditions
Analysis & Minimization
Pathways to Success
Skilled Operator
Fuzzy-Set Score
Condition
No operator exists.
0.33
Unskilled operator exists but is not present daily at the system. Operator cannot diagnose and recognize system problems and is unreliable.
0.67
Operator has some technical knowledge, but needs external assistance to diagnose and recognize system problems. Operator is not present daily at the system. 1
Skilled operator regularly present; demonstrates extensive technical system knowledge; capable of recognizing and rectifying system problems independently; proactive.
QCA Method
Observed Problem
Future Work
Summary
Pathways to Success 12

14. Ongoing External Support
Set values were assigned for all cases and conditions and assembled in a truth table
Skilled Operator
Define Outcomes
Select Causal Conditions
Select Cases
Data Collection
Calibrate Conditions
Analysis & Minimization
Pathways to Success
Fuzzy-Set Score
Condition
No operator exists.
0.33
Unskilled operator exists but is not present daily at the system. Operator cannot diagnose and recognize system problems and is unreliable.
0.67
Operator has some technical knowledge, but needs external assistance to diagnose and recognize system problems. Operator is not present daily at the system. 1
Skilled operator regularly present; demonstrates extensive technical system knowledge; capable of recognizing and rectifying system problems independently; proactive.
Case Number
Skilled Operator
Ongoing External Support
N Conditions
Outcome: Success 1 2
N Cases
QCA Method
Observed Problem
Future Work
Summary
Pathways to Success 13

15. Ongoing External Support
Set values were assigned for all cases and conditions and assembled in a truth table
Skilled Operator
Define Outcomes
Select Causal Conditions
Select Cases
Data Collection
Calibrate Conditions
Analysis & Minimization
Pathways to Success
Fuzzy-Set Score
Condition
No operator exists.
0.33
Unskilled operator exists but is not present daily at the system. Operator cannot diagnose and recognize system problems and is unreliable.
0.67
Operator has some technical knowledge, but needs external assistance to diagnose and recognize system problems. Operator is not present daily at the system. 1
Skilled operator regularly present; demonstrates extensive technical system knowledge; capable of recognizing and rectifying system problems independently; proactive.
Case Number
Skilled Operator
Ongoing External Support
N Conditions
Outcome: Success 1 2
N Cases
QCA Method
Observed Problem
Future Work
Summary
Pathways to Success 13

16. Ongoing External Support
Set values were assigned for all cases and conditions and assembled in a truth table
Skilled Operator
Define Outcomes
Select Causal Conditions
Select Cases
Data Collection
Calibrate Conditions
Analysis & Minimization
Pathways to Success
Fuzzy-Set Score
Condition
No operator exists.
0.33
Unskilled operator exists but is not present daily at the system. Operator cannot diagnose and recognize system problems and is unreliable.
0.67
Operator has some technical knowledge, but needs external assistance to diagnose and recognize system problems. Operator is not present daily at the system. 1
Skilled operator regularly present; demonstrates extensive technical system knowledge; capable of recognizing and rectifying system problems independently; proactive.
Case Number
Skilled Operator
Ongoing External Support
N Conditions
Outcome: Success 1 2
0.33
N Cases
QCA Method
Observed Problem
Future Work
Summary
Pathways to Success 13

17. Ongoing External Support
Set values were assigned for all cases and conditions and assembled in a truth table
Skilled Operator
Define Outcomes
Select Causal Conditions
Select Cases
Data Collection
Calibrate Conditions
Analysis & Minimization
Pathways to Success
Fuzzy-Set Score
Condition
No operator exists.
0.33
Unskilled operator exists but is not present daily at the system. Operator cannot diagnose and recognize system problems and is unreliable.
0.67
Operator has some technical knowledge, but needs external assistance to diagnose and recognize system problems. Operator is not present daily at the system. 1
Skilled operator regularly present; demonstrates extensive technical system knowledge; capable of recognizing and rectifying system problems independently; proactive.
Case Number
Skilled Operator
Ongoing External Support
N Conditions
Outcome: Success 1
0.67 2
0.33
N Cases
QCA Method
Observed Problem
Future Work
Summary
Pathways to Success 13

18. Conditions that were found to be inconsistent were removed and allowed us to minimize analyzed conditions
Define Outcomes
Select Causal Conditions
Select Cases
Data Collection
Calibrate Conditions
Analysis & Minimization
Pathways to Success
Community Participation
Training
Priorities
Skilled Operator
Priority Assessment
Community Participation in Planning
Technical Training
Addressed Sanitation Priorities
Community Contributions
Behavior Change Education
Community Participation in Construction
Institutional Support
Income Generation
Community Participation in O&M
Infrastructure
Lack of Vandalism
Ongoing External Support
Water Supply
Lack of Government Barriers
QCA Method
Observed Problem
Future Work
Summary
Pathways to Success 14

19. Conditions that were found to be inconsistent were removed and allowed us to minimize analyzed conditions
Define Outcomes
Select Causal Conditions
Select Cases
Data Collection
Calibrate Conditions
Analysis & Minimization
Pathways to Success
Community Participation
Training
Priorities
Skilled Operator
Priority Assessment
Community Participation in Planning
Technical Training
Addressed Sanitation Priorities
Community Contributions
Behavior Change Education
Community Participation in Construction
Institutional Support
Income Generation
Community Participation in O&M
Infrastructure
Lack of Vandalism
Ongoing External Support
Water Supply
Lack of Government Barriers
QCA Method
Observed Problem
Future Work
Summary
Pathways to Success 14

20. Pathways reflect the most simplified solution that still captures important case complexity
Community Participation
Training
Priorities
Skilled Operator
Priority Assessment
Community Participation in Planning
Technical Training
Addressed Sanitation Priorities
Community Contributions ?
Behavior Change Education
Community Participation in Construction
Success
Institutional Support
Income Generation
Community Participation in O&M
Infrastructure
Lack of Vandalism
Ongoing External Support
Water Supply
Lack of Government Barriers
QCA Method
Observed Problem
Future Work
Summary
Pathways to Success 15

21. The combination of six conditions led to success
Skilled Operator
Lack of Government Barriers
Ongoing External Support
Water Supply
Lack of Vandalism
Addressed Sanitation Priorities *
Condition
Necessity
Skilled Operator 1
Lack of Government Barriers
Ongoing External Support
0.93
Water Supply
0.87
Lack of Vandalism
0.80
Addressed Sanitation Priorities
0.73
Consistency = 0.93
Coverage = 0.66
Pathways to Success
QCA Method
Observed Problem
Future Work
Summary 16

22. The presence of a skilled operator was important for maintenance and use
Lack of Government Barriers
Ongoing External Support
Water Supply
Lack of Vandalism
Addressed Sanitation Priorities *
Condition
Necessity
Skilled Operator 1
Lack of Government Barriers
Ongoing External Support
0.93
Water Supply
0.87
Lack of Vandalism
0.80
Addressed Sanitation Priorities
0.73
Consistency = 0.93
Coverage = 0.66
Pathways to Success
QCA Method
Observed Problem
Future Work
Summary 17

23. The absence of government barriers was important for continued operation of the systems
Skilled Operator
Lack of Government Barriers
Ongoing External Support
Water Supply
Lack of Vandalism
Addressed Sanitation Priorities *
Condition
Necessity
Skilled Operator 1
Lack of Government Barriers
Ongoing External Support
0.93
Water Supply
0.87
Lack of Vandalism
0.80
Addressed Sanitation Priorities
0.73
Consistency = 0.93
Coverage = 0.66
Pathways to Success
QCA Method
Observed Problem
Future Work
Summary 18

24. The types and quality of external support was more important than the source of external support
Skilled Operator
Lack of Government Barriers
Ongoing External Support
Water Supply
Lack of Vandalism
Addressed Sanitation Priorities *
Condition
Necessity
Skilled Operator 1
Lack of Government Barriers
Ongoing External Support
0.93
Water Supply
0.87
Lack of Vandalism
0.80
Addressed Sanitation Priorities
0.73
Consistency = 0.93
Coverage = 0.66
Pathways to Success
QCA Method
Observed Problem
Future Work
Summary 19

25. Addressed Sanitation Priorities
Higher percentages of weighted sanitation priorities are associated with successful systems
Case Number
Addressed Sanitation Priorities 1
0.78 2
0.60 6
0.86 8
0.74 5
0.67 3
0.40 10
0.32 9
0.23 7
0.19 4
0.14
Skilled Operator
Lack of Government Barriers
Ongoing External Support
Water Supply
Lack of Vandalism
Addressed Sanitation Priorities *
Condition
Necessity
Skilled Operator 1
Lack of Government Barriers
Ongoing External Support
0.93
Water Supply
0.87
Lack of Vandalism
0.80
Addressed Sanitation Priorities
0.73
Consistency = 0.93
Coverage = 0.66
Pathways to Success
QCA Method
Observed Problem
Future Work
Summary 20

26. Future Work Analyze pathways to failure
Perform fsQCA with 20 cases: 10 resource recovery and 10 non-resource recovery systems
Design trainings and practical tools for implementing organizations
Pathways to Success
Future Work
QCA Method
Observed Problem
Summary 21

27. Summary
Institution building should be a focus of project planning since ongoing external support and lack of government barriers are necessary for success
Addressed sanitation priorities suggests that participation and planning must ensure projects meet community priorities
The combination of Skilled Operator, Lack of Government Barriers, Ongoing External Support, Water Supply, Lack of Vandalism, & Addressed Sanitation Priorities lead to successful sanitation systems
Questions?
Pathways to Success
QCA Method
Observed Problem
Future Work
Summary 22

28. EXTRA SLIDES

29. Fuzzy-Set Qualitative Comparative Analysis
Statistical Analyses
Case Studies
Quantitative Methods
Qualitative Methods
Adapted from Gross and Garvin 2011

30. Understanding Pathways
Adapted from Kaminsky and Jordan 2017
Cases w/ Factor X
Cases w/ Outcome
Cases w/ Factor X
Cases w/ Outcome = =
Necessary Condition
≥ 0.9
All (or nearly all) instances of the outcome show the condition
Pathway Coverage
≥ ?
The percent of cases with the outcome explained by a given pathway
Pathway Consistency
≥ 0.8
The pathway consistently leads to the outcome

31. Failure * * * * * * Communities 3, 9, 10 Consistency = 1.0
Lack of Skilled Operator
Lack of Income Generation *
3, 9, 10
Government Barriers
Lack of Ongoing External Support
Vandalism * 4
Lack of Addressed Sanitation Priorities
Lack of Skilled Operator
Lack of Ongoing External Support
Vandalism
Lack of Income Generation
Lack of Water Supply * * * * 7
Condition
Necessity
Lack of Addressed Sanitation Priorities
0.74
Government Barriers
0.68
Lack of Skilled Operator
0.67
Lack of Ongoing External Support
Vandalism
Lack of Income Generation
0.60
Lack of Water Supply
0.47
Consistency = 1.0
Coverage = 0.74

32. Use * Use Communities Community Participation in Planning 10
Ongoing External Support
Water Supply
Community Participation in Planning
Addressed Sanitation Priorities * 10
1, 2, 5, 6, 8
Condition
Necessity
Ongoing External Support
0.90
Water Supply
0.85
Community Participation in Planning
0.75
Addressed Sanitation Priorities
0.68
Consistency = 0.95
Coverage = 0.94

33. Maintenance and Performance
Maintenance and System Performance
Communities
Lack of Government Barriers
Skilled Operator
Water Supply * *
1, 2, 5, 6, 20
Condition
Necessity
Lack of Government Barriers
1.0
Skilled Operator
0.94
Water Supply
0.83
Consistency = 0.95
Coverage = 0.94