1. GSF Results and Financial Monitoring Workshop
Value for Money Study
(2016)
GSF Results and Financial Monitoring Workshop
Dear es Salaam

2. Content What is VFM analysis?
Adapting VFM analysis to sanitation and hygiene programmes
Applying the methodology: Country case studies and desk reviews
Reflections on applying the methodology

3. 08/11/2018
What is VFM?
Value for money (VFM): The optimum combination of whole-life cost and quality (or fitness for purpose) to meet the user’s requirement.
VFM is about striking the best balance between the “three E’s” − economy, efficiency and effectiveness.
A fourth “E” – equity – is now also sometimes used to ensure that value-for-money analysis accounts for the importance of reaching different groups.
“S” – Sustainability as additional dimension for sustained outcomes
VFM is not a tool or a method, but a way of thinking about using resources well.
So why do projects or organisations do VFM -

4. Internal VFM / External VFM
08/11/2018
How can VFM analysis be used in development context?
To help managers better understand and analyse performance issues they see on the ground, and their associated cost
To demonstrate results and attract funding based on evidence
To create a culture of transparency around programme results – what are our costs, what are our results. What do they mean?
So why do projects or organisations do VFM -
Internal VFM / External VFM

5. VfM dimensions across the WASH results chain
The financial costs of inputs
Eg. organizational expenditure at all levels

6. VfM dimensions across the WASH results chain
The resources used in terms of finance and staff time (capital and labour)

7. VfM dimensions across the WASH results chain
Economy = a measure of the expenditure for resources needed for the programme

8. VfM dimensions across the WASH results chain
The process by which inputs are transformed into results. The process can be the object of a programme evaluation
Eg. Pre-triggering, triggering, post-triggering follow-up

9. VfM dimensions across the WASH results chain
Outputs: the direct deliverables of the programme
Eg. number of triggering meetings, number of improved latrines constructed, communities declared ODF
Assumed outcomes: assumed outcomes resulting from the outputs
Eg. number of people using latrines, living in ODF environments

10. VfM dimensions across the WASH results chain
Efficiency = a measure of how well resources (inputs) are being used to achieve programme deliverables (outputs)

11. VfM dimensions across the WASH results chain
Cost-efficiency = a measure of how well costs are transformed into programme deliverables

12. VfM dimensions across the WASH results chain
The actual change in people’s lives over time (appropriately measured)
Eg. As assumed outputs, but measured/verified as opposed to assumed

13. Cost-efficiency = a measure of how well costs are transformed into sustained programme outcomes

14. Objectives of the study
1) Assess mature GSF-funded sanitation and hygiene programmes to gauge current levels of economy, efficiency and effectiveness by applying standard VFM analysis procedures and producing current unit costs of outcomes.
2) Recommend a better cost classification structure and aggregation procedure to facilitate future Value for Money analyses and standardise this for cross-sector benchmarking.
3) Compare findings from GSF programmes with existing data in the sanitation/hygiene sector so as to benchmark GSF performance
Assumptions
Limitations

15. Understanding GSF financing and data sources

16. Cost classification – programme costs
Capital expenditure software
Activity + implementation expenditure on demand creation
Expenditure on direct support
Expenditure directly supporting implementation activities
Expenditure on indirect support
Institutional, or other staff expenditure not directly linked to sub-grantee implementation
Sub-grantee operating costs
UNOPS hosting fee
Sub-grantee activity and programme staff expenditure
CPM costs
EA management fee and EA operating costs
GSF Staff cost & operating costs
Executing agency implementation
Cost classification – non-programme costs
Capital expenditure hardware
i) HH expenditure on latrine construction
ii) Expenditure by other agencies in the implementation area

17. Step 1: Align sub-grantee financial data with indicators of interests.
08/11/2018
Summary of methodology and process
Step 1: Align sub-grantee financial data with indicators of interests.
Coding of a sample of sub-grantee financial reports by expenditure type and activity (overheads, programme staff costs, expenditure on triggering, post-triggering, School WASH, BCC sessions etc…).
Step 2: Attributing global level expenditure to country programmes and activities
Separate processes applied for global expenditure, EA expenditure, and SG overhead expenditures
Step 3: Combining coded financial data with monitoring data for VfM Indicators
Indicators of interest are those which are monitored

18. Cost per ODF village – Cambodia and Madagascar

19. Approximate village size: Cambodia 250 Madagascar 10-20
Cost per ODF village – Cambodia and Madagascar
Approximate village size:
Cambodia 250
Madagascar 10-20
ODF village conversion rates per sub-grantee
Madagascar
Cambodia

20. Example of the output of activity coding (Nepal)
Activity coded expenditure (output of step 1)
Example of the output of activity coding (Nepal)

21. Results of the financial analysis (output of step 2)

22. Results of the financial analysis (output of step 2)

23. Desk reviews – Cost-efficiency

24. Desk reviews – cost- efficiency

25. Desk reviews – findings economy/efficiency
Economy indicators are hugely sensitive: both to context, definitions in the monitoring data and the methodology (codes used).
Dispersion around the mean reduces as you move from economy to cost-efficiency. Partially as monitoring indicators become more closely aligned.
Figures capture key dimension of programme activity but not all. Different programmes have different amounts of programme expenditure ‘on the books’.

26. Reflections on applying the methodology
The method of allocating of different types of expenditure to activities varies in each country, but in all cases we saw SG’s recorded the expenditure types prior to EA aggregation
 the data are hard to get at the moment but there if the GSF want them!
Only an implicit theory of change embedded in the results framework
 but results framework does not necessarily link to the financial reporting or monitoring data; nor harmonised across countries
Potential for two tiered methodology: one analysis tailored to the country context, a more general global methodology
 but there is a tension between the two.

27. Can it be done = yes with some reservations
Reflections on applying the methodology
Can it be done = yes with some reservations ++
Excellent financial reporting mechanisms at sub-grantee level – these are largely activity based, and subject to strong check and balances.
Significant opportunity for sector learning. Initially in terms of internal VFM – assessing the economy and efficiency costs of different innovative approaches to sanitation and hygiene promotion; or the cost of reaching the hard to reach communities – the last 20%. --
At present deriving VFM results from current reporting is difficult and very time consuming – especially on an SG by SG basis.
VFM analysis of EA level is limited by differing and unclear methods of aggregation of SG data.
Finding are very sensitive to variations in how monitoring data is collected and how key results are calculated.

28. 08/11/2018
Recommendations
17 Recommendations, grouped into five thematic areas: i) Three on more clearly establishing causal relationships between programme inputs and outputs/outcomes. ii) Two on the collection and treatment of financial data. iii) Four on the collection and treatment of the financial data. iv) Three on the perspective and purpose of future analyses; internal or external VfM analyses. v) Five on cost classifications for future VfM analyses.

29. Next steps GSF M&E system review & strengthening
08/11/2018
Next steps
GSF
M&E system review & strengthening
Results Framework update
Financial Reporting Tool finalization
Roll-out
Applying more articulated VfM considerations in future programming
WSSCC
VFM of policy advocacy

30. Thank you.
More information: