Practical Issues in Implementing Performance-Based Contracting Health System Innovations Workshop Abuja, Jan , 2010

A Few Thoughts to Start Don’t Panic: Not everything has to be perfect, be creative! Make new & interesting mistakes – don’t repeat old ones Be Systematic: Write things down in a contract and contracting manual The Limits of Planning: Endless planning and analysis can get in the way of action & learning Humility: a) don’t be too sure of things b) knowledge must be larger than our experience c) give people at local levels sufficient autonomy d) keep learning, evaluating, adapting :

Performance-Based Contracting for Health Services in Developing Countries-A Toolkit

Outline of the Toolkit 1.Summary of the Toolkit – pages What is performance-based contracting? Definitions and Concepts – pages How to Contract? Going through the 7 steps of the contracting cycle – pages Checklist for Contracting – quick summary of tasks in contracting – pages Whether to contract? Review of the global experience with contracting – pages 69-96

Outline of the Toolkit Appendix A: Example of a Contracting Manual including an example of a contract Appendix B: World Bank procurement approach to contracting for health services Appendix C: Description of Evaluated Contracting Experiences Appendix D: TORs for 3 rd party evaluation Appendix E: TORs for contracts involving: (i) PHC delivery; and (ii) HIV prevention for CSWs - tools and guidelines

2. Define the Services 3. Design the Monitoring and Evaluation 5. Arrange for Contract Management 7. Carry out Bidding Process and Manage the Contracts 1. Dialogue with Stakeholders 4. Decide how to Select Contractors and Establish Price 6. Draft Contract & Bidding Documents The Contracting Cycle: A Systematic Approach (page 20)

The 6 most important mistakes 1.Not clearly defining the objectives of the contract and the indicators of success 2.Allowing contractors & purchasers to forget the stated objectives & targets 3.Limiting the managerial autonomy of contractors 4.Not defining the size & location of each “lot” 5.Not having a contracting plan: not indicating how contracts will be managed & M&E will be done 6.Setting prices irrationally and not choosing the best contractors

Defining Objectives – More than SMART Big advantage of contracting is results focus so concentrate on outputs not inputs. The purchaser should objectively define: – Quantity of services (e.g. % DTP3 coverage, skilled birth attendance) – Technical Quality (national technical guidelines) – Equity (ensuring the poor receive services)

Defining Objectives – More than SMART Need to address the most important challenges – In Ghana 70% of HIV transmitted through FSWs, guess what % of grants went to FSWs? Focus on a Few!! Indicator inflation if > 10  loss of focus, less data collection Bias towards outputs & outcomes not inputs & processes e.g. DPT3 coverage better than vaccine availability or “micro-planning” – the exception is in measuring “quality”

Defining Objectives – More than SMART They need to be defined in a measurable way – define numerator & denominator precisely. Avoid undefined terms like “functional” health facility Set targets broadly: 20%  58% DPT3 coverage is good even if the target was 60% Within the “span of control” of the contractor, e.g., measuring availability of staff if contractor not allowed hire, transfer, decide payments to health workers

Some Examples of Indicators – What’s Wrong with Them? 1.% of county hospitals with functioning x-ray machines 2.Number of health workers receiving appropriate training 3.% of patients using a PHC Center who are satisfied with the services 4.Low rate of health workers leaving their positions

Defining Objectives – More than SMART Refer to Tasks 4 and 5 (pages 26-31) Look at table 3.2 (page 28) Congratulations!! You’ve avoided the first important mistake

The 6 most important mistakes 1.Not clearly defining the objectives of the contract and the indicators of success 2.Allowing contractors & purchasers to forget the stated objectives & targets 3.Limiting the managerial autonomy of contractors 4.Not defining the size & location of each “lot” 5.Not having a contracting plan: not indicating how contracts will be managed & M&E will be done 6.Setting prices irrationally and not choosing the best contractors

Ensuring Focus on Outputs Regular (quarterly) discussion of progress on indicators between purchaser and contractor Carry out independent M&E Credible threat of sanctions: – Embarrassment works!! Special meetings – Letters to NGO board – Replacement of key managers – Termination of contract RBC!!!! (Read the bloody contract)

Ensuring Focus on Outputs – Performance Bonuses Sends signals to contractors about important indicators Challenges of performance bonuses: – Finding indicators that are important & can be measured reasonably frequently – Amount of bonus – enough to signal but not too expensive – about 10% – What contractor can do with the bonus – Design of bonus: (i) improvement from baseline; (ii) absolute target

Design of Bonus System Lot ALot BLot C Score year Score Year Score Year Change from previous best (+10 points) Yes, Yes No, No Yes, No Absolute Target (=70 points)No, No Yes, Yes Yes, Yes Change and/or above 70 pointsYes, Yes Yes, Yes

Ensuring Focus on Outputs-Results- Based Financing “Transfer of money or material goods conditional on taking a measurable health related action or achieving a predetermined performance target.” - Eichler and Levine Contractors are provided payments based on the amount of services they actually deliver

The 6 most important mistakes 1.Not clearly defining the objectives of the contract and the indicators of success 2.Allowing contractors & purchasers to forget the stated objectives & targets 3.Limiting the managerial autonomy of contractors 4.Not defining the size & location of each “lot” 5.Not having a contracting plan: not indicating how contracts will be managed & M&E will be done 6.Setting prices irrationally and not choosing the best contractors

Ensuring Managerial Autonomy Decentralize management to people who are closest to the ground reality - Purchaser can hold contractors accountable for results when managers have responsibility & autonomy - avoids the “blame game” Encourages innovation – RBF, sub-centers Take advantage of private sector’s flexibility – that’s why they’re called Non-Governmental Organizations

Impediments to Managerial Autonomy Telling contractors “how” they should deliver services (define objectives, “what”) Line item budgets with reimbursement of actual expenditures – Limit flexibility to move money where it’s needed – Encourages micro-management – Increases transaction costs & arguments – Encourages focus on inputs Lump-sum allows proper FM!!!

Example of a Line-Item Budget ItemAmount 1. Staff salaries$150, Drugs$40, Medical Equipment$20, Medical supplies$30, Maintenance & Repair$10,000 TOTAL$250,000

Impediments to Managerial Autonomy Following Government procedures for staff hiring, firing, transfer, & pay. Unclear authority of purchaser’s officials – Will try to force certain approaches – Will claim power over more decisions including staff recruitment Government procures important inputs (allow contractors to do procurement, use private auditors)

Drug Availability Index (max = 100) according to who is responsible for drug procurement - Afghanistan

A Tale of 2 Countries: Afghanistan & DRC Both severely affected by conflict Poor countries with limited infrastructure Lots of donor money flowing in Lots of NGOs Both started contracting with NGOs to deliver health services around 2004 Used different approaches to contracting

Government Controls on Contracted NGOs Type of ControlAfghanistanDRC Plans and Fiduciary Controls Annual work plans approved by GovernmentNOYES Annual procurement plans approved by GovernmentNOYES Centralized procurement of goods by GovernmentNOYES Reimbursement for individual expendituresNOYES Ex ante approval of payments to health workersNOYES Innovations have to be pre-approvedNOYES Monitoring & Evaluation Regular quarterly review of reported resultsYESNO High quality annual health facility surveysYESNO Frequent field supervisionYESNO

Change in the utilization rate for curative care (per 100 population per year) during the first year of the contracts DRC Afghanistan

Increasing Managerial Autonomy See tasks 26 – 29 pages 55-59

The 6 most important mistakes 1.Not clearly defining the objectives of the contract and the indicators of success 2.Allowing contractors & purchasers to forget the stated objectives & targets 3.Limiting the managerial autonomy of contractors 4.Not defining the size & location of each “lot” 5.Not having a contracting plan: not indicating how contracts will be managed & M&E will be done 6.Setting prices irrationally and not choosing the best contractors

Size of “Lots” – Economies of Scale Economies of scale in price per beneficiary – Fixed management and admin. costs. Likely more competition  lower prices Large packages facilitates contract management Easier & cheaper to monitor and evaluate contractor performance with fewer lots Likely to reduce opportunities for corruption – Unscrupulous officials will intimidate small contractors – Easier to avoid “ghost” NGOs

Scale of Contracts – Provinces vs. Clusters in Afghanistan Provinces: 300,000 – 900,000 population “Clusters” of districts – 100,000 – 180,000 Issues related to NGO capacity, equity Bid price of provinces = $4.05 per capita per year vs. $7.80 for clusters, high admin. costs MOH decided cluster approach was too expensive

Size of “Lots” – Arguments for Smaller Scale Concerns regarding contractor “capacity” – hard to predict & compared to what alternative Increased diversity – no oligopoly No disruption to existing providers – But could suffer from “Swiss cheese”

Distribution of NGO HCs in Afghanistan un-served

Size of “Lots” - Recommendations Lots should cover at least 0.5 million population 7-20 contracts See task 8, pages 34-36

The 6 most important mistakes 1.Not clearly defining the objectives of the contract and the indicators of success 2.Allowing contractors & purchasers to forget the stated objectives & targets 3.Limiting the managerial autonomy of contractors 4.Not defining the size & location of each “lot” 5.Not having a contracting plan: not indicating how contracts will be managed & M&E will be done 6.Setting prices irrationally and not choosing the best contractors

Use a Contracting Plan People tend to focus on the contract itself and the recruitment procedures Fail to systematically address: 1.How monitoring and evaluation will be carried out (otherwise indicators in contract are meaningless) 2.How contracts will be managed (client’s activities not described in the contact)

Monitoring and Evaluation Ensure that contracts remain output and outcome focused Learn lessons and improve performance Definitions: – “Monitoring” tracking the performance of individual contractors – “Evaluation” tracking the overall progress in service delivery of all contractors (comparison to other service providers)

The Process of M&E Collect baseline data early on & provide to all stakeholders (helps contractors identify important issues) Recruit 3 rd party to help with M&E design & data collection while maintaining government oversight & involvement Clear responsibility for analysis of M&E data Ensure there’s data on effectiveness (coverage), equity, quality of care, cost

Recommendations M&E key to achieving good results –worth the investment needed. Use different sources of data – none are perfect Make M&E somebody’s job See tasks 10-15, pages 37-44

Contract Management Often done poorly Ministries of Health often have limited experience and understanding of contracting Often not clear who in the MOH is responsible for contract management

Major Issues and Challenges in Contract Management Paying contractors on time Avoiding corruption Proper supervision, monitoring & evaluation Solving problems – many related to relationships Maintaining government ownership, oversight, & involvement while avoiding micro-management

Some Possible General Solutions to Contract Management Recruiting sufficient number of talented people to manage contracts Allow sufficient budget Computerized contract management systems Provide incentives to contract managers based on results achieved by contractors, timely payment, lack of audit objections, etc.

The 6 most important mistakes 1.Not clearly defining the objectives of the contract and the indicators of success 2.Allowing contractors & purchasers to forget the stated objectives & targets 3.Limiting the managerial autonomy of contractors 4.Not defining the size & location of each “lot” 5.Not having a contracting plan: not indicating how contracts will be managed & M&E will be done 6.Setting prices irrationally and not choosing the best contractors

Determining the Price of a Contract There are basically 3 options, each with benefits and issues: 1.Competition at least partly on the basis of price 2.Negotiation of price with selected bidder 3.Fixed price where client sets price in advance

Competition, at least partly on Price Advantages lowest price transparent encourages innovation reflects local realities benefits local NGOs Disadvantages may take longer may end up with prices that are inconsistent may end up with prices that are too low or too high

Negotiations with Contractor Advantages fairly quick reflects local realities if purchaser well motivated & savvy can be low cost Disadvantages not transparent!!! may end up with prices that are inconsistent may end up with prices that are too low or too high

Fixed Price Advantages fairly quick transparent uniform, hence “fair” (?) Disadvantages rigid, “one size fits all” discourages innovation or cost savings hard to estimate real costs (be humble!)

Example of Competition (LCS or QCBS) Bidders submit technical and financial proposals technical proposals scored by evaluation committee financial proposals opened publicly In LCS, lowest price among technically responsive bidders is selected. In QCBS technical “score” combined with financial score using a weight (e.g. 80/20)

Example of LCS: Minimum Acceptable Technical Score = 60 Name of NGO Technical Score (St) Financial Proposal ($) Financial Score Sf =100 x Fm/F Total Score (S) = St x T+ Sf x P A*60$2.0 million B70$2.5 million C75$4.0 million

Example of Competition (QCBS) Name of NGO Technical Score (St) Financial Proposal ($) Financial Score Sf =100 x Fm/F Total Score (S) = St x T+ Sf x P A60$2.0 million B*B* 70$2.5 million * C75$4.0 million * 2.0m/2.5m x 100=80 and (70 x 0.8) + (80 x 0.2) = 72.0

Contractor Selection – Competition vs. Sole Source Bidders want transparent process with a “level playing field”  competition Competitive process will generally lead to “best” managers, most innovative ideas, “best” organizations, best prices Sole source selection is quick but not fair, not transparent, creates resentment, leads to “fat & happy” contractors, limits innovation & creative thinking

Situations Where Sole Source Might Make Sense Where a mission clinic has been providing services for 50 years Limited competition possible due to ongoing armed conflict Contractor bringing significant funds into a partnership

Selection Criteria Need to be clearly defined in advance Should not be excessively detailed Should not be excessively difficult Should look at: – Experience of organization (track record) – Key staff (quality of managers) – Work-plan/strategy (lowest % of score)

Evaluation Process Independent evaluation committee Committee should include members external to client/purchaser: – technical agency (e.g. WHO, UNICEF) – representative of NGO community (obviously need to avoid conflicts of interest)

Maximizing Participation Prior consultations with many NGOs Advertise widely Hold information sessions Don’t make process too daunting Establish reasonable selection criteria Allow NGOs to form consortia (< 3 or 4) Avoid bid & performance bonds, guarantees “Grow” local NGOs by having a few small packages

Recommendation Use competition based at least partly on price whenever it’s feasible Modify approach to minimize disadvantages, e.g. fixed price is adjusted to reflect local variation See tasks pages 44-51