1. Practical Issues in Implementing Performance-Based Contracting Health System Innovations Workshop Abuja, Jan. 25-29, 2010

2. A Few Thoughts to Start Don’t Panic: Not everything has to be perfect, be creative! Make new & interesting mistakes – don’t repeat old ones Be Systematic: Write things down in a contract and contracting manual The Limits of Planning: Endless planning and analysis can get in the way of action & learning Humility: a) don’t be too sure of things b) knowledge must be larger than our experience c) give people at local levels sufficient autonomy d) keep learning, evaluating, adapting :

3. Performance-Based Contracting for Health Services in Developing Countries-A Toolkit

4. Outline of the Toolkit 1.Summary of the Toolkit – pages 1-8 2.What is performance-based contracting? Definitions and Concepts – pages 9-17 3.How to Contract? Going through the 7 steps of the contracting cycle – pages 19-66 4.Checklist for Contracting – quick summary of tasks in contracting – pages 67-68 5.Whether to contract? Review of the global experience with contracting – pages 69-96

5. Outline of the Toolkit Appendix A: Example of a Contracting Manual including an example of a contract Appendix B: World Bank procurement approach to contracting for health services Appendix C: Description of Evaluated Contracting Experiences Appendix D: TORs for 3 rd party evaluation Appendix E: TORs for contracts involving: (i) PHC delivery; and (ii) HIV prevention for CSWs WWW.rbfhealth.org - tools and guidelines WWW.rbfhealth.org

6. 2. Define the Services 3. Design the Monitoring and Evaluation 5. Arrange for Contract Management 7. Carry out Bidding Process and Manage the Contracts 1. Dialogue with Stakeholders 4. Decide how to Select Contractors and Establish Price 6. Draft Contract & Bidding Documents The Contracting Cycle: A Systematic Approach (page 20)

7. The 6 most important mistakes 1.Not clearly defining the objectives of the contract and the indicators of success 2.Allowing contractors & purchasers to forget the stated objectives & targets 3.Limiting the managerial autonomy of contractors 4.Not defining the size & location of each “lot” 5.Not having a contracting plan: not indicating how contracts will be managed & M&E will be done 6.Setting prices irrationally and not choosing the best contractors

8. Defining Objectives – More than SMART Big advantage of contracting is results focus so concentrate on outputs not inputs. The purchaser should objectively define: – Quantity of services (e.g. % DTP3 coverage, skilled birth attendance) – Technical Quality (national technical guidelines) – Equity (ensuring the poor receive services)

9. Defining Objectives – More than SMART Need to address the most important challenges – In Ghana 70% of HIV transmitted through FSWs, guess what % of grants went to FSWs? Focus on a Few!! Indicator inflation if > 10  loss of focus, less data collection Bias towards outputs & outcomes not inputs & processes e.g. DPT3 coverage better than vaccine availability or “micro-planning” – the exception is in measuring “quality”

10. Defining Objectives – More than SMART They need to be defined in a measurable way – define numerator & denominator precisely. Avoid undefined terms like “functional” health facility Set targets broadly: 20%  58% DPT3 coverage is good even if the target was 60% Within the “span of control” of the contractor, e.g., measuring availability of staff if contractor not allowed hire, transfer, decide payments to health workers

11. Some Examples of Indicators – What’s Wrong with Them? 1.% of county hospitals with functioning x-ray machines 2.Number of health workers receiving appropriate training 3.% of patients using a PHC Center who are satisfied with the services 4.Low rate of health workers leaving their positions

12. Defining Objectives – More than SMART Refer to Tasks 4 and 5 (pages 26-31) Look at table 3.2 (page 28) Congratulations!! You’ve avoided the first important mistake

13. The 6 most important mistakes 1.Not clearly defining the objectives of the contract and the indicators of success 2.Allowing contractors & purchasers to forget the stated objectives & targets 3.Limiting the managerial autonomy of contractors 4.Not defining the size & location of each “lot” 5.Not having a contracting plan: not indicating how contracts will be managed & M&E will be done 6.Setting prices irrationally and not choosing the best contractors

14. Ensuring Focus on Outputs Regular (quarterly) discussion of progress on indicators between purchaser and contractor Carry out independent M&E Credible threat of sanctions: – Embarrassment works!! Special meetings – Letters to NGO board – Replacement of key managers – Termination of contract RBC!!!! (Read the bloody contract)

15. Ensuring Focus on Outputs – Performance Bonuses Sends signals to contractors about important indicators Challenges of performance bonuses: – Finding indicators that are important & can be measured reasonably frequently – Amount of bonus – enough to signal but not too expensive – about 10% – What contractor can do with the bonus – Design of bonus: (i) improvement from baseline; (ii) absolute target

16. Design of Bonus System Lot ALot BLot C Score year 0406550 Score Year 15070 Score Year 26075 Change from previous best (+10 points) Yes, Yes No, No Yes, No Absolute Target (=70 points)No, No Yes, Yes Yes, Yes Change and/or above 70 pointsYes, Yes Yes, Yes

17. Ensuring Focus on Outputs-Results- Based Financing “Transfer of money or material goods conditional on taking a measurable health related action or achieving a predetermined performance target.” - Eichler and Levine Contractors are provided payments based on the amount of services they actually deliver

18. The 6 most important mistakes 1.Not clearly defining the objectives of the contract and the indicators of success 2.Allowing contractors & purchasers to forget the stated objectives & targets 3.Limiting the managerial autonomy of contractors 4.Not defining the size & location of each “lot” 5.Not having a contracting plan: not indicating how contracts will be managed & M&E will be done 6.Setting prices irrationally and not choosing the best contractors

19. Ensuring Managerial Autonomy Decentralize management to people who are closest to the ground reality - Purchaser can hold contractors accountable for results when managers have responsibility & autonomy - avoids the “blame game” Encourages innovation – RBF, sub-centers Take advantage of private sector’s flexibility – that’s why they’re called Non-Governmental Organizations

20. Impediments to Managerial Autonomy Telling contractors “how” they should deliver services (define objectives, “what”) Line item budgets with reimbursement of actual expenditures – Limit flexibility to move money where it’s needed – Encourages micro-management – Increases transaction costs & arguments – Encourages focus on inputs Lump-sum allows proper FM!!!

21. Example of a Line-Item Budget ItemAmount 1. Staff salaries$150,000 2. Drugs$40,000 3. Medical Equipment$20,000 4. Medical supplies$30,000 5. Maintenance & Repair$10,000 TOTAL$250,000

22. Impediments to Managerial Autonomy Following Government procedures for staff hiring, firing, transfer, & pay. Unclear authority of purchaser’s officials – Will try to force certain approaches – Will claim power over more decisions including staff recruitment Government procures important inputs (allow contractors to do procurement, use private auditors)

23. Drug Availability Index (max = 100) according to who is responsible for drug procurement - Afghanistan

24. A Tale of 2 Countries: Afghanistan & DRC Both severely affected by conflict Poor countries with limited infrastructure Lots of donor money flowing in Lots of NGOs Both started contracting with NGOs to deliver health services around 2004 Used different approaches to contracting

25. Government Controls on Contracted NGOs Type of ControlAfghanistanDRC Plans and Fiduciary Controls Annual work plans approved by GovernmentNOYES Annual procurement plans approved by GovernmentNOYES Centralized procurement of goods by GovernmentNOYES Reimbursement for individual expendituresNOYES Ex ante approval of payments to health workersNOYES Innovations have to be pre-approvedNOYES Monitoring & Evaluation Regular quarterly review of reported resultsYESNO High quality annual health facility surveysYESNO Frequent field supervisionYESNO

26. Change in the utilization rate for curative care (per 100 population per year) during the first year of the contracts 20072008 DRC 20042005 Afghanistan

27. Increasing Managerial Autonomy See tasks 26 – 29 pages 55-59

28. The 6 most important mistakes 1.Not clearly defining the objectives of the contract and the indicators of success 2.Allowing contractors & purchasers to forget the stated objectives & targets 3.Limiting the managerial autonomy of contractors 4.Not defining the size & location of each “lot” 5.Not having a contracting plan: not indicating how contracts will be managed & M&E will be done 6.Setting prices irrationally and not choosing the best contractors

29. Size of “Lots” – Economies of Scale Economies of scale in price per beneficiary – Fixed management and admin. costs. Likely more competition  lower prices Large packages facilitates contract management Easier & cheaper to monitor and evaluate contractor performance with fewer lots Likely to reduce opportunities for corruption – Unscrupulous officials will intimidate small contractors – Easier to avoid “ghost” NGOs

30. Scale of Contracts – Provinces vs. Clusters in Afghanistan Provinces: 300,000 – 900,000 population “Clusters” of districts – 100,000 – 180,000 Issues related to NGO capacity, equity Bid price of provinces = $4.05 per capita per year vs. $7.80 for clusters, high admin. costs MOH decided cluster approach was too expensive

31. Size of “Lots” – Arguments for Smaller Scale Concerns regarding contractor “capacity” – hard to predict & compared to what alternative Increased diversity – no oligopoly No disruption to existing providers – But could suffer from “Swiss cheese”

32. Distribution of NGO HCs in Afghanistan un-served

33. Size of “Lots” - Recommendations Lots should cover at least 0.5 million population 7-20 contracts See task 8, pages 34-36

34. The 6 most important mistakes 1.Not clearly defining the objectives of the contract and the indicators of success 2.Allowing contractors & purchasers to forget the stated objectives & targets 3.Limiting the managerial autonomy of contractors 4.Not defining the size & location of each “lot” 5.Not having a contracting plan: not indicating how contracts will be managed & M&E will be done 6.Setting prices irrationally and not choosing the best contractors

35. Use a Contracting Plan People tend to focus on the contract itself and the recruitment procedures Fail to systematically address: 1.How monitoring and evaluation will be carried out (otherwise indicators in contract are meaningless) 2.How contracts will be managed (client’s activities not described in the contact)

36. Monitoring and Evaluation Ensure that contracts remain output and outcome focused Learn lessons and improve performance Definitions: – “Monitoring” tracking the performance of individual contractors – “Evaluation” tracking the overall progress in service delivery of all contractors (comparison to other service providers)

37. The Process of M&E Collect baseline data early on & provide to all stakeholders (helps contractors identify important issues) Recruit 3 rd party to help with M&E design & data collection while maintaining government oversight & involvement Clear responsibility for analysis of M&E data Ensure there’s data on effectiveness (coverage), equity, quality of care, cost

38. Recommendations M&E key to achieving good results –worth the investment needed. Use different sources of data – none are perfect Make M&E somebody’s job See tasks 10-15, pages 37-44

39. Contract Management Often done poorly Ministries of Health often have limited experience and understanding of contracting Often not clear who in the MOH is responsible for contract management

40. Major Issues and Challenges in Contract Management Paying contractors on time Avoiding corruption Proper supervision, monitoring & evaluation Solving problems – many related to relationships Maintaining government ownership, oversight, & involvement while avoiding micro-management

41. Some Possible General Solutions to Contract Management Recruiting sufficient number of talented people to manage contracts Allow sufficient budget Computerized contract management systems Provide incentives to contract managers based on results achieved by contractors, timely payment, lack of audit objections, etc.

42. The 6 most important mistakes 1.Not clearly defining the objectives of the contract and the indicators of success 2.Allowing contractors & purchasers to forget the stated objectives & targets 3.Limiting the managerial autonomy of contractors 4.Not defining the size & location of each “lot” 5.Not having a contracting plan: not indicating how contracts will be managed & M&E will be done 6.Setting prices irrationally and not choosing the best contractors

43. Determining the Price of a Contract There are basically 3 options, each with benefits and issues: 1.Competition at least partly on the basis of price 2.Negotiation of price with selected bidder 3.Fixed price where client sets price in advance

44. Competition, at least partly on Price Advantages lowest price transparent encourages innovation reflects local realities benefits local NGOs Disadvantages may take longer may end up with prices that are inconsistent may end up with prices that are too low or too high

45. Negotiations with Contractor Advantages fairly quick reflects local realities if purchaser well motivated & savvy can be low cost Disadvantages not transparent!!! may end up with prices that are inconsistent may end up with prices that are too low or too high

46. Fixed Price Advantages fairly quick transparent uniform, hence “fair” (?) Disadvantages rigid, “one size fits all” discourages innovation or cost savings hard to estimate real costs (be humble!)

47. Example of Competition (LCS or QCBS) Bidders submit technical and financial proposals technical proposals scored by evaluation committee financial proposals opened publicly In LCS, lowest price among technically responsive bidders is selected. In QCBS technical “score” combined with financial score using a weight (e.g. 80/20)

48. Example of LCS: Minimum Acceptable Technical Score = 60 Name of NGO Technical Score (St) Financial Proposal ($) Financial Score Sf =100 x Fm/F Total Score (S) = St x T+ Sf x P A*60$2.0 million 10068.0 B70$2.5 million 8072.0 C75$4.0 million 5070.0

49. Example of Competition (QCBS) Name of NGO Technical Score (St) Financial Proposal ($) Financial Score Sf =100 x Fm/F Total Score (S) = St x T+ Sf x P A60$2.0 million 10068.0 B*B* 70$2.5 million 8072.0* C75$4.0 million 5070.0 * 2.0m/2.5m x 100=80 and (70 x 0.8) + (80 x 0.2) = 72.0

50. Contractor Selection – Competition vs. Sole Source Bidders want transparent process with a “level playing field”  competition Competitive process will generally lead to “best” managers, most innovative ideas, “best” organizations, best prices Sole source selection is quick but not fair, not transparent, creates resentment, leads to “fat & happy” contractors, limits innovation & creative thinking

51. Situations Where Sole Source Might Make Sense Where a mission clinic has been providing services for 50 years Limited competition possible due to ongoing armed conflict Contractor bringing significant funds into a partnership

52. Selection Criteria Need to be clearly defined in advance Should not be excessively detailed Should not be excessively difficult Should look at: – Experience of organization (track record) – Key staff (quality of managers) – Work-plan/strategy (lowest % of score)

53. Evaluation Process Independent evaluation committee Committee should include members external to client/purchaser: – technical agency (e.g. WHO, UNICEF) – representative of NGO community (obviously need to avoid conflicts of interest)

54. Maximizing Participation Prior consultations with many NGOs Advertise widely Hold information sessions Don’t make process too daunting Establish reasonable selection criteria Allow NGOs to form consortia (< 3 or 4) Avoid bid & performance bonds, guarantees “Grow” local NGOs by having a few small packages

55. Recommendation Use competition based at least partly on price whenever it’s feasible Modify approach to minimize disadvantages, e.g. fixed price is adjusted to reflect local variation See tasks 16-20 pages 44-51