1. American Evaluation Association Conference
Evaluation Capacity Building Lessons Learned from the Experience of Botswana
Robert Lahey
REL Solutions Inc
American Evaluation Association Conference
San Antonio: November 2010

2. Background on Botswana
Southern African country (to the North of South Africa)
Land-locked; Kalahari Desert (5% arable land)
Small, sparsely populated (less than 2 M)
Independence in 1966 – Significant growth & transformation over last 20 years
2 Key Sectors: Mining (diamonds) & Public Sector
Middle Income country - But still major challenges (poverty, unemployment, HIV/AIDS)
REL Solutions Inc - Nov

3. Some Critical Success Factors for M&E Development & Implementation
Drivers – Is there a ‘real’ need?
Leadership & Commitment – Is there a high-level ‘champion’; sustainability?
Capacity ‘to do’ – Is there infrastructure to supply M&E information?
Capacity ‘to use’ – Are there incentives to demand & use M&E information?
Oversight – Will implementation be monitored?
REL Solutions Inc - Nov

4. Drivers – Is there a ‘real’ need?
Two Key Initiatives
‘Vision 2016’ - Long-Term Vision for Botswana
2. Public Sector Reform
REL Solutions Inc - Nov

5. Drivers – Is there a ‘real’ need?
‘Vision 2016’ - “Towards Prosperity for All”
Developed in 1996, jointly by public sector, private sector & civil society
7 Pillars setting expectations for the Nation
A very public expectation of national performance reporting against Vision pillars
Vision Council has mandate/requirement to monitor and report on ‘progress’
REL Solutions Inc - Nov

6. Drivers – Is there a ‘real’ need?
2. Public Sector Reform
Aim - more accountability in the public sector & management of individual Ministries
Several cross-government initiatives launched over the 2000s
More systematic macro and sector-specific planning
Desire to raise focus from expenditures/activities to ‘results’
Timing – last leg of Vision 2016 & desire to base new National Plan more systematically on Vision Pillars
REL Solutions Inc - Nov

7. Leadership & Commitment – Is there a ‘Champion’ & Sustainability?
Vision 2016
Vision Council - a high-level group representing all 3 sectors – champion for both ‘M’ and ‘E’
Vision Secretariat & Coordinator – Ongoing operational support and M&E advocate
Public Sector Reform
Office of the President setting the tone & rules
Key for introducing concept & ensuring development
New President – reinforced drive to increased accountability & discipline in the public sector
REL Solutions Inc - Nov

8. REL Solutions Inc - Nov. 2010 RELahey@rogers.com
Capacity ‘to do’ – Is there infrastructure needed to supply credible M&E?
Some critical deficiencies:
Insufficient HR with M&E or analytical skills
2. Too little of the ‘right data’ – credible, quality, results-oriented data, and, when needed, at a sub-national level
3. Little/no experience with the conduct of ‘Evaluation’ studies
REL Solutions Inc - Nov

9. REL Solutions Inc - Nov. 2010 RELahey@rogers.com
Capacity ‘to do’ – Is there infrastructure needed to supply credible M&E? (cont)
But, some elements to build on:
1. Some ‘pockets’ of M&E and analytical expertise across the system
2. Some understanding of RBM and role of M&E across public sector
A Central Statistics Office (CSO) – data experts
A number of IPDET-trained public servants - beginnings of a ‘network’
5. M&E unit in a few Ministries (unfilled positions; limited experience)
6. A version of a Vision 2016 M&E system – BUT, not functioning well
REL Solutions Inc - Nov

10. REL Solutions Inc - Nov. 2010 RELahey@rogers.com
Capacity ‘to do’ – Is there infrastructure needed to supply credible M&E? (cont)
Some achievements to date:
Modifications to Vision M&E System
Vision Pillars Key Result Areas (KRAs) expected results
More systematic linking of Sector/Ministry development plans to Vision Pillars
More ‘results’-oriented indicators, (both outputs and outcomes)
4. Intent to develop plans for future ‘Evaluation’ studies
5. Partnering to increase M&E training – U of Botswana; World Bank
REL Solutions Inc - Nov

11. REL Solutions Inc - Nov. 2010 RELahey@rogers.com
Capacity ‘to use’ – Are there incentives to demand & use M&E information?
National Performance Reporting:
All sectors (& media) raising question: ‘Where are we?’ re Vision 2016 Pillars and goals
First comprehensive results-based ‘progress’ report on nation’s performance against Vision ‘Botswana’s Performance’ (April 2009)
High-level audience:
* President and High Level Consultative Committee
National performance report identifies areas for possible future Evaluation (e.g. poverty; youth unemployment; HIV/AIDS)
REL Solutions Inc - Nov

12. REL Solutions Inc - Nov. 2010 RELahey@rogers.com
Capacity ‘to use’ – Are there incentives to demand & use M&E information? (cont)
Ministry-Level Use:
Link of Ministry programs to national goals and KRAs identified in latest National Development Plan (NDP 10)
Mid-term review of NDP 10 to report on ‘results’
‘M&E chapter’ in NDP 10 identifies broad need for Ministries to embrace results-oriented M&E
But, accountability has not been a part of the culture of organizations
* How quickly can/will Ministries embrace M&E?
Capacity constraints (HR and data)
REL Solutions Inc - Nov

13. Oversight – Will implementation be monitored?
Vision Council and Secretariat
Lobbying senior officials to fill M&E capacity gaps – particularly data deficiencies and HR training needs
Plan to formalize role in establishing a ‘centre of excellence for evaluation’
Office of the President
New unit to provide oversight
Progress on M&E implementation will be seen in mid-term review of NDP 10
REL Solutions Inc - Nov

14. Some Lessons for Capacity Building
Importance of senior leadership & political support → a champion to drive change and provide an enabling environment
Position M&E as more than a technical issue
→ a mechanism to support public sector
management (M&E is not an end in itself)
Links to need to build capacity ‘to use’ M&E information within and across organizations
→ key users and uses of M&E need to be clarified
REL Solutions Inc - Nov

15. Some Lessons for Capacity Building
Manage expectations of senior officials → seeking to achieve in 1 or 2 years what other countries have been working at for some 20 to 30 years
Recognize 3 audiences for M&E training & orientation →
(1) Senior Officials
(2) Program Managers
(3) M&E Officers; Analysts
Often, too little focus on (1) and (2)
REL Solutions Inc - Nov

16. Some Lessons for Capacity Building
Clarify major challenges early on (M&E ‘Readiness Assessment’) → will impact pace of roll-out and identify the need for new investment
Moving from ‘concept to reality’ → Need to detail & resource an Implementation Plan (& monitor)
The ‘E’ often overlooked → Evaluation strategy & plan and Evaluation policy centre would help in positioning and resourcing the function
REL Solutions Inc - Nov

17. Some Lessons for Capacity Building
Too few skilled HR → pilot, using a phased implementation rather than government-wide roll-out of M&E
Too little ‘quality’ data → national data development strategy & plan and involve national statistical agency
Limited resources → identify potential partners & network with ‘pockets’ of analytical expertise
Building support for sustainability → need to navigate the political landscape; an awareness beyond technical skills
REL Solutions Inc - Nov

18. REL Solutions Inc - Nov. 2010 RELahey@rogers.com
Contact Coordinates
Robert Lahey
REL Solutions Inc.
Ottawa, Canada
Telephone:
REL Solutions Inc - Nov