Evaluating and Managing Performance …..it’s not just about Vendors Yukon Procurement Conference February 16, 2015

Provide an overview of:  what “Vendor Performance Management” is, and is not  Some examples from public sector agencies in Canada (and their different objectives) Purpose

It is:  about improving success in meeting contract outcomes  one element of “Vendor Management”  Aligning procurement, contract terms, project management to meet organizational objectives  Monitoring/mitigating vendor risks as well as  Aggregating, assessing and communicating vendor performance information and  Actively managing relationships It is NOT :  code for simply increasing criticism of suppliers  Surprise report cards  Only about ‘bidder barring’ What is “Vendor Performance Management”?

 Increase accountability  Help achieve best value for taxpayers Office of the Procurement Ombudsman, 2010  Increase competitive advantage  Improve stakeholder satisfaction  Increase performance visibility Survey Analytics, 2011 Benefits

 Most effective approaches integrate tools across all phases of the procurement and contract lifecycle  For example, buyers can:  Ensure clear statement of functional and performance requirements  Establish key performance indicators  Use in-contract evaluations, monitoring procedures, and measurement of performance against KPIs  Evaluate and maintain records on supplier performance  Provide feedback to suppliers How is it done?

Each tool:  Has different advantages/disadvantages  Requires a different level of investment to implement and maintain  Has a different likelihood of success, depending on:  Market characteristics (e.g. market size and health, maturity of vendors)  Capacity/maturity of an organization to properly implement and support Which tool(s) to pick?

Recent Survey (Ontario)*  92% of suppliers and buyers think VPM is an important activity  25% of organizations have VPM activities Becoming increasingly common across Canada, some examples include:  PWGSC  Canada Revenue Agency  Correctional Service of Canada  Nfld Department of Transportation and Works  Infrastructure Ontario  Defence Construction Canada  Province of BC  Ministry of Government Services (Ontario) *Malatest & Assoc. (2012), Vendor Performance Management Study Public Sector Activity

 Driver: “train (not penalize) contractors/consultants to meet expectations of DCC”  Overall rating = sum of points from equally weighted criteria  Administration / Contract Management  Execution / Project Management  Quality of Workmanship  Completion / Close Out / Time  Health & Safety  Scale of Unacceptable (0-5), Not satisfactory (6-10); Satisfactory (11-16) and Superior (17-20) Defence Construction Canada (DCC)

 Bidding privileges suspended for any score of 5 or less in one category, or a second occurrence of a total score less than 50%  “failures” are relatively rare  Heavy reliance on documentation – require clear evidence DCC’s own practices didn’t contribute  Generally considered by staff and suppliers to be successful  Staff believe contributes to their ability to be an attractive client in a competitive market DCC - continued

 Driven in part by desire to reduce time spent evaluating high volume of responses to RFPs/tenders  Required for all contracts >$100,000  Vendor Performance Rating = average of that vendor’s Scorecards over a three year period  Vendor performance is rated as :  1 – Consistently falls far below expectations (< 25% of expectations met)  2 – Frequently misses expectations (<50%)  3 – Mostly meets expectations but sometimes misses (<75%)  4 – Consistently meets expectations (100% of requirements)  5 – Exceeds expectations  Ratings applied during final evaluation, worth minimum of 10%  excellent service can lead to extended time on prequalified list Infrastructure Ontario

 Required for services contracts >$10M  Conducted at beginning of a prequalification process, result in pass/fail  Common indicators include:  Performance - contractual standards & service target levels  Cost performance  Schedule performance  Team BC – Internal Reference Check (2011)

 Two new tools: one for “strategic deals”, one for the consultants that support them  Driver – recognition that improving performance requires clear and ongoing communication concerning performance expectations  Rating carried out by both parties and aims to foster discussion about areas needing improvement or differences of opinion  For consultants:  If more than 2 of 8 questions unsatisfactory = 1 demerit  3 demerits = potential removal from RFQ BC – Vendor Scorecard (2014)

Questions?