1. IAS Satellite Session, 25 July 2017
Tracking manufacturer performance to ensure the uninterrupted provision of timely, high quality early infant HIV diagnosis test results
Esther Turunga
IAS Satellite Session, 25 July 2017

2. Overview
The point-of-care (POC) early infant diagnosis (EID) and viral load procurement consortium
Defining indicators to monitor manufacturer performance
Preliminary results of key performance indicator (KPI) monitoring
Conclusions and recommendations

3. Convener and facilitator
The global procurement consortium
Step 1: Information sharing and joint negotiations with manufacturers
Convener and facilitator
Step 2: Procurement
Supporting more than
100 countries
Goal: Uninterrupted provision of timely, high-quality test results.
Objective: Catalyze demand for and uptake of fit for purpose point-of-care technologies through affordable, transparent and sustainable pricing arrangements.
Expected outcome: Improved and aligned prices, terms and conditions.
CHAI UNICEF
CHAI
UNICEF
EGPAF
EGPAF
MSF
MSF
The Global Fund
USAID
Consortium
Negotiations
Individual Negotiations based on orders
The Global Fund
Manufacturers
USAID
Orders placed by individual organizations, but aligned with consortium negotiations
Product selection and procurement based on country choice and country operating plans.

4. Defining KPIs Purpose: Two types of KPIs: Procurement indicators
Inform how manufacturer is performing with regards to agreed service levels, quality and cost;
Identify room for improvement; and
Ensure immediate problem-solving and corrective action.
Two types of KPIs:
Procurement indicators
On-Time, In-Full principle
Guaranteed shelf life
Quality of delivered devices and consumables
Service and maintenance costs
In-country device performance indicators
Rate of errors, invalids and undetermined
Reported device and module breakdowns
Response time from receipt of complaint and provision of replacement device
Repair time

5. Early results of KPI monitoring
Procurement indicators
From October 2016 to June 2017, EGPAF monitored procurement indicators for POC EID devices and tests kits ordered for eight project countries: Cameroon, Côte d’Ivoire, Kenya, Lesotho, Mozambique, Rwanda, Swaziland, and Zimbabwe.
Number of devices and cartridges delivered to countries (Pilot phase: November 2016 – June 2017)
Country
Number of devices
Number of cartridges
Cameroon 11
2,510
Cote D’Ivoire 5
700
Lesotho
2,520
Mozambique 13
2,500
Rwanda 2
200
Swaziland 6
950
Zimbabwe 10
2,250
TOTAL 52
11,630

6. Early results: Procurement indicators
Number of orders and time taken until manufacturer acknowledge orders
27 orders
Mean: 7 days
Median: 4 days (range 0 – 50 days)
Number of orders and rate of correct shipping documents provided (documents without errors)
23 orders;
Rate: 35%
Number of orders and rate of on-time delivery against agreed delivery dates
N = 23
Rate: 17%
Number of orders and rate of agreed guaranteed shelf life delivered
Rate: 13%
Number and rate of defective platforms and/or missing and accessories (e.g. modules, power cables, etc.) delivered
6 incidences reported; Rate - 13%
Are repair services available through the manufacturer or a designated distributor and are replacement parts and/or devices available in-country
Out of eight project countries currently implementing POC EID, four do not have designated distributors in-country
Costs of annual service contract and/or increment price per test for service and maintenance
Range – USD 2,250 - USD 2,280 per device per year if purchased with device in single transaction  or USD 2,500 – USD 2,898 per device per year if purchased separately

7. Early results: In-country device performance
From December 2016 to June 2017, EGPAF monitored device performance indicators in six project countries: Cameroon, Cote d’Ivoire, Lesotho, Rwanda, Swaziland, and Zimbabwe.
In-country device performance indicators
Results
Rate of errors, invalid and undetermined tests
8% (231 out of 2,922 tests performed)
Number and rate of platform and module breakdowns reported per year
N=7 breakdowns; Rate: 15%
Median, range, and IQR time from receipt of complaint to response by manufacturer
Median: 0.5 days; Range: 0 to 9 days; IQR: 2 days
Median, range, and IQR time taken by manufacturer to provide replacement platform
Median: 7 days; Range: 0 to 17 days; IQR: 13 days
Median, range and IQR time taken by manufacturer to repair defective platform
Median: 17 days; Range: 7-22 days;
IQR: 10.5 days
Number and proportion of breakdowns covered by the service warranty
7 breakdowns (proportion covered: 100%)
Number and rate of intervention reports provided by manufacturer after repair of platform
1; Rate – 14%

8. Conclusions and recommendations
Monitoring manufacturer performance is necessary to ensure the provision of uninterrupted timely, high-quality EID and viral load test results
There is a need to standardize KPIs across procurement partners so that quality, price, delivery and service levels can be evaluated uniformly
Members of the procurement consortium and other procurement partners should regularly share results of KPI monitoring, and provide regular joint feedback to manufacturers to continuously improve performance
Manufacturers should develop and use service and maintenance tracking tools
Performance indicators should be an explicit part of the manufacturer’s contract
Is there a need to consider incentives and/or penalties for manufacturers who fail to meet expectations?

9. EGPAF gratefully acknowledges the valuable contributions of CHAI, UNICEF and USAID in defining the KPIs.

10. Thank you!