Presentation on theme: "Effective Contract Monitoring System"— Presentation transcript:
1Effective Contract Monitoring System Components of anEffective Contract Monitoring SystemGeorgia Department of Audits and AccountsPerformance Audit Operations DivisionIntroTopic that most are familiar with; Lot of experience in the room – more in the audiencethis review should serve as a refresher and an opportunity to emphasize the importance of monitoringYou would be surprised at the number of gaps in monitoring that exist, likely in your agencyAbout 45 minutes
2Purpose of the ReportWe frequently found that agencies did not have adequate information on the level/quality of services provided on their behalf by contractors.Mentality was that someone else was responsible for providing the service, not the agency.Agencies cannot contract out responsibility to be a good steward of public funds.Published in July 2003 as a “Best Practices in Government” report, not a traditional performance audit.Reason for the ReportWe routinely found a lack of oversight of the work performed once a contract was signedAgency personnel was busy with tasks in front of them and saw the contract as a way to let someone else do the workPROBLEM – agency is still responsible for ensuring the work is done and will ultimately be held responsible by auditors, legislators, and the public is the work is not done satisfactorily.CONTRACT GUIDANCE AVAILABLE - State law, DOAS policies, as well as agency policies cover the procurement process in some detail; agencies generally understood purchasing authority, when competitive bids were necessary, the GA Procurement RegistryReleased July 2003 – not as a performance audit-Give brief description of each component-Use in other states-Use in Georgia – survey of 25 agencies; review of 20 contracts in 6 agencies
3Overview of Contract Monitoring Process of ensuring that a contractor adequately performs a contracted service.An agency’s contract monitoring system has numerous elements, many of which are instituted through policies and procedures.The responsibility of multiple individuals, within both program offices and administrative offices.Cannot be a consideration only during the contract period. Must be a consideration during the pre- and post-contract periods.Contract monitoring is one part of the larger topic of contract administrationProcess of ensuring that a contractor adequately performs a contracted service.Numerous elements of the systemThis is the responsibility of individuals throughout an agency – within program offices and administrative offices (contracting offices/financial staff).Finally, contract monitoring cannot be a consideration on the first day of the contract term.Begins when the solicitation document is being created. Many elements of a successful monitoring system are in the RFP and contract.Obviously monitoring is done during the contract term, but it also extends to a period after the contract has ended.
4Components of a Monitoring System Identified 15 components of an effective contract monitoring system.Not all contracts are monitored using the same components, but a number of the components are necessary for an effective system.We identified 15 components of an effective contract monitoring system through:- a review of policies and procedures from Georgia and other states;guides from the federal government and other states;other states’ audits; andpublications from the National Institute of Governmental Purchasing.The list was then reviewed by Georgia chapters of the National Contract Management Association, National Institute of Governmental Purchasing, and National Association of Purchasing Management.IMPORTANT – This is not a list of items that are necessary for each and every contract.It is a list of potential controls that management should consider when deciding how to ensure that the contractor fulfills the terms of the contract.
5List of Components Contract Monitoring Training* Use of Incentives and ConsequencesWritten Policies and Procedures*Access to Records/Right to Audit*Contingency Plans*Measuring Customer SatisfactionCommunicating Clear Expectations to Contractors*Dispute Resolution Procedures*Contract Administration Plan*Organized Contract Files*Closeout ProceduresPayments Linked to Performance*Post-Contract ReviewRegular Programmatic Reports from Contractor** NECESSARYOn-Site MonitoringList of componentsI’m sure that most of you are familiar with most, if not all, of theseA number of these are necessary for an adequate contract monitoring system, though the formality of each (e.g., contingency plan) may vary based on the value, risk of the contract.
6Training in Contract Monitoring* Program employees responsible for much of the monitoring are often educated/experienced in their service area – not in contracting.Many of the topics that should be included are listed as components in the report.Ideally, training addresses agency-specific processes (e.g., agency’s policy on what types of contracts require a formal contingency plan).NECESSARYDuring performance audits, we often deal with program staff that are educated/trained in a subject area other than management, especially contract management.--- Educators, social workers, engineers, law enforcement officialsContract monitoring training can be incorporated into larger issue of contract management. But it should be given adequate attention.Training should include the topics in this presentation, but when delivered at the agency level it can be more specific.WE FOUND –Survey - 10 of 25 agencies said staff received some training in contract managementOn-Site Review - 0 of 6 agencies had adequate training in contract monitoring
7Written Policies and Procedures* Serve as a guide to agency personnel in ensuring a consistent, high-quality monitoring process. Policies in place often cover procurement with little attention paid to monitoring.Items that should be covered in policies and procedures include:Responsibilities of agency personnelContract correspondenceContract monitoring reportsDefining conflicts of interestDocumenting contract administration decisionsSubcontractor administrationStandard contract termsDispute resolutionContract closeoutProfessional development of contract personnelNECESSARYWritten policies and procedures help to ensure that agency personnel use a consistent, high-quality monitoring processThey are necessary for any agency process, especially those that are undertaken by multiple individuals across multiple offices/unitsShould include: LIST FROM ABOVE- Correspondence – where maintained; methodsWE FOUND –Survey - 9 of 25 (36%) agencies had formal policies and procedures in placeOn-Site Review – 1 of 6 had policies on monitoring and another 1 had policies that covered only financial aspects
8Contingency Plans*Addresses how the agency would respond in the event of an interruption in service delivery.Detailed, written plans may not be necessary for low-risk contracts. Risk level determines level of detail.Plan options include:Contracting with the next lowest bidder from the original solicitationUsing another current contractorIn-house delivery of the serviceContracting with another government entityNECESSARYHow the agency will respond if contractor quits, firedDoesn’t have to be formal, written plan (though ideally written)High risk – CMO contracts (What happens if one walks away?)Low risk – DOT contracts with ROW appraisersOptions: SEE ABOVEWE FOUND –Survey – 15 of 25 agencies (60%) reported using contingency plansOn-Site Review – 2 of 20 contracts (10%) had contingency plans; 1 had been implemented
9Communicating Clear Expectations* Contract performance improved when agency and contractor are on the same page.Three methods:Detailed statement of work/scope of servicesPerformance measures and goalsPost-award meeting with contractorDetailed Statement of WorkBegins as part of the solicitation documentPurpose of the contract, performance measures/goals, agency’s methods for monitoring performance, deliverables, and other requirementsMust be clear, concise, and free from ambiguityAs part of the contract, it contains responsibilities of the contractor, including those activities in which the contractor will engage, products to be produced, and timetables for the completion of activities.Final SOW is a combination (and often a negotiation) of the SOW from the solicitation document and the contractor proposal.NECESSARYObviously no party is going to be pleased if expectations are different; need to be on the same pageI’ll briefly discuss three methods of communicating clear expectations :Detailed scope of services/statement of workPerformance measures/goalsPost-award meeting with the contractorDetailed Scope of Services/Statement of WorkPart of the solicitation document and later the contractDebate/clarification of scope of services big par t of procurement processSEE SLIDE ABOVE
10Communicating Clear Expectations (continued) Performance Measures and GoalsEstablished to evaluate the quality and quantity of services providedShould be included in the solicitation document; if necessary, additional measures can be negotiated before the contract is signedGenerally should include outputs and outcomesOutputs measure a process and are units of the serviceOutcomes assess the quality or result of the serviceContract should state what data will be collected by the contractor, and how, when and to whom the data will be submittedMeeting with Contractors to Clarify WorkPost-award or kickoff meeting between agency and contractorEspecially important when those individuals involved in the day-to-day execution of services (contractor) and monitoring (agency) are not significantly involved in the procurementTopics include contract scope, reporting requirements, administration procedures, rights and obligations of parties, performance standards, monitoring procedures, and potential problem areasFormal meeting may not be necessary for less complex contractsPerformance Measures and Goals – difference between two (# versus desired #)Needed to evaluate quality and quantity of servicesShould be in the solicitation document; if not, must be in the final contractMeasures should include outputs and outcomes. Output measures units of service (e.g., number served; items provided), while outcomes measure the result of quality of the service (e.g., improvement in test scores)Contract should state what data will be collected by the contractor, how, when and to whom it will be submittedPost-award or Kickoff MeetingLast opportunity to clarify/summarize the decisions made during the pre-contract period. Decisions may have been negotiated over weeks/months by many individuals.Important to ensure the day-to-day players on both sides have a clear understanding.Includes discussion of contract scope of work, reporting, administrative procedures, rights and obligations of parties, performance measures and goals, monitoring procedures, and potential problem areasWE FOUND –Survey – 22 of 25 (84%) agencies reported perf measures and post-award meetingsOn-Site – 5 of 20 contracts (25%) had detailed Statements of Work; 7 (35%) had somewhat detailed Statements but lacked important items such as performance measures; 10 of 20 contracts (50%) had no performance measures and several were vagueEXAMPLE – 4 DJJ contracts for outdoor therapeutic camps. No performance measures for years; when added to contract, data not collected; contracts continued to be renewedEXAMPLE – Medicaid drug rebates (no performance measures, no penalties, no incentives); after instituting better monitoring processes, 41% quarterly increase in rebates ($5.6 million)
11Contract Administration Plan* View of planned and completed activities and can be used as a status report.Contains list of deliverables, milestones, contract modifications, summary of invoices submitted and paid, and renewal dates.Also includes individuals responsible for monitoring each contract aspect, as well as the method of monitoring.May be simple or complex, depending on the complexity of the contracted service, the contract value, and risk to the agency of poor performance.Should consider that more intense monitoring early in the contract period will be beneficial.NECESSARYGA Procurement Manual gives list of items that should be in a CAPDocument that describes the who and how of the monitoring of a particular contract.Ensures that all agency personnel are on the same page regarding responsibilities – between agency and contractor and within the agencyContains items listed above – list and dates for deliverables, milestones, invoices submitted and paidMore intensive monitoring at the beginning of the contract period is often beneficialWE FOUND –Survey – 14 of 25 agencies (56%) reported CAPS for some of their contractsOn-Site Review – 2 of 20 contracts (10%) had written CAPsAt one agency, two different offices stated that the other was checking contractor performance against contract goals. The contractors didn’t even bother to report because no one asked for it. The three contracts had been renewed.
12Organized Contract Files* Necessary to have access to the master contract files.Files should have all information necessary to know what was expected and received under the contract, as well as the basis for any major decisions made related to the contract.At a minimum, files should contain:Signed contract and purchase orderContract modificationsContract administration planContingency planSources solicitedEvaluation method and awardMeeting minutesContract correspondenceReports from any on-site visitsPerformance reportsRecords of complaints and contractor disputesInvoices and vouchersNECESSARYCentral file containing information that allows someone to determine what was expected and received under the contractNeed access to contract files, especially when disputes arise with the contractorFiles should contain: SEE ABOVEWE FOUND –Survey – 19 of 25 agencies (76%) said they maintained a central contract fileSite Visits – 6 of 20 contracts (30%) had a single file maintained in one officeWe regularly find that agencies have difficulty locating a signed contract; contract correspondence is often dispersed.
13Payments Linked to Satisfactory Performance* Payments should not be made unless the agency has assurance that the contractor is making satisfactory progress in fulfilling the contract.Programmatic reports that directly relate to contract terms should be submitted with the invoice.If a cost-reimbursement contract, adequate documentation to support an invoiced amount must be submitted.Appropriate personnel, who may include a program official and an administrative (i.e., contracting) official, should review the invoice and supporting documentation prior to payment.NECESSARYAgency must ensure that contractor has fulfilled all contract requirements for quality, quantity, and timeliness before making a full payment.Invoices should be accompanied by documentation that shows performance measures have been metBefore payment, program and administrative personnel should receive documentation submitted by the contractorWE FOUND –Survey – 22 of 25 agencies (88%) said payments tied to performanceSite Visits – 4 of 20 contracts (20%) had direct correlation, requiring proof of performance. For 13 contracts, program manager’s signature was necessary.Common that admin requires program official signature, but program hasn’t received or adequately reviewed program data to ensure contract terms being met.
14Regular Programmatic Reports from Contractor* Contract should require contractor to provide specific information at designated intervals to ensure that performance goals are being met.Content of the reports should be outlined in the contract, with a template or example as an appendix.Contract must also include:reporting method,the individual who should receive the report,the reporting frequency, and whatactions the agency may take in response to an incomplete report.Weakness – Submitted by the contractor, who has a financial interest in stating that contract terms are being met. Reports can be designed to decrease the likelihood of misinformation (e.g., requiring list of participants instead of just a number).NECESSARYContractor should be required to regularly submit documentation that contains programmatic informationThe reports should address the performance measures in the contract and the report content/format should be dictated by the agency- If format changes, harder to compare over timeShould be in contract: Report method (web-based, , hard copy), individual(s) to receive the report, reporting frequency and deadlines, and actions that can be taken by agency if not submitted/submitted properlyRealize that report data needs to be, at least periodically, verified by the agencyWE FOUND –Survey – 21 of 25 agencies (84%) reported use of programmatic reportsOn-Site – 10 of 20 contracts (50%) reviewed required scheduled programmatic reports sufficient enough to assess performance. Another 8 (40%) were available only upon request or were insufficient to assess contractor performance.
15On-Site MonitoringCan allow agency to verify actual performance against scheduled or reported performance.Reinforces the importance of the contract to the contractor.Provides an opportunity for enhanced communication.Structured reviews are more productive than unstructured.Should include random inspections of contractor records.Can be resource-intensive; therefore, necessity dependent on the complexity and risk associated with the contracted service.Written report detailing findings should be produced.On-site monitoring allows agency to independently verify contractor performance; verify actual performance versus scheduled or reported perfReinforces the importance of the contractReview should be structured; have a review instrument, but also allow for an opportunity for enhanced two-way communicationIt can be more than a simple compliance reviewlisten/observe the contractor – you may find that different areas need your attentionShould have a written reportWE FOUND –Survey – 20 of 25 agencies (80%) reported using method for some contractsOn-Site – On-site reviews that occurred were general in nature with little documentation showing that they were beneficial in assessing contractor compliance with contract terms
16Incentives/Consequences Related to Performance Performance reinforcements are helpful in achieving the contract goals.Two types of incentivesBonus: Superior performance may in the form of high quality work, completion of a project ahead of schedule, or saving the agency money. Agency must define the level of service required, the timeframe, or the amount of savings that must be realized. The incentive must be associated only with the most important contract requirements.Revenue-sharing: If a contractor is operating a revenue-generating operation, it may be allowed to retain a portion of revenue. The portion retained can be increased as revenue levels increase.Performance reinforcements - carrot and the stick approach - can be beneficial in improving contractor performance.When incentives offered, agency must clearly define what level of service will be required in order for the contractor to receive the incentive. Similarly, financial consequences can only be imposed when the corresponding performance levels and consequences are clearly spelled out in the contract.- These are intended to improve performance – not become a point of contention.Two types of incentives –Bonus for superior performance – Can be for exceptional quality, meeting or exceeding deadline, agency savings; Ensure that incentives only tied to most important aspects of the contract.Contractor Retaining % of Revenue – A method when contractor operating a revenue-generating operation.
17Incentives and Consequences (Continued) Three types of consequencesLiquidated Damages: Quantify an agency’s loss for poor performance and deduct that loss from payments to the contractor; must be reasonable to avoid reduced competition and/or higher costs.Partial Takeover of Operations: Agency takes over operations of areas in non-compliance and reduces contractor payments by new operating costs.Contract Termination: Used when contractor has been given clear warning about non-compliance.Examples of financial consequences:Liquidated Damages – Quantifies an agency’s loss that occurs due to inadequate contractor performance; reduction in payment based on a formula, amount set in the contract; Must be reasonable; unreasonably high potential damages lead to increased risk to the contractor and fewer bidders and/or higher costsPartial Takeover of OperationsContract termination – Obviously a last resort, after working with contractor to solve problemsWE FOUND –Survey – Incentives rarely used; 4 of 25 agencies (16%);Financial consequences often used ; 16 of 25 agencies (64%)On-Site – 0 of 20 contracts had incentive clauses; 15 of 20 (75%) had liquidated damages, partial takeover, termination for poor performanceIncentives – Maybe GDOT for quick completion of road construction
18Access to Records/Right to Audit* Must have enough access to contractor files to ensure that services are adequately delivered and/or funds spent appropriately.Cannot simply rely on reports submitted by contractor.Ability to audit contractor records should also extend to the records of any subcontractor.NECESSARYVery common contract clauseAgencies have responsibility to verify data reported by contractor and to ensure that funds are expended properly; this requires access to files/recordsRight should also extend to the records of any subcontractorWE FOUND –Survey – 23 of 25 agencies (92%) reported having access to contractor records and right to auditOn-Site – 17 of 20 contracts (85%) contains clauses related to access to files and right to audit
19Measuring Customer Satisfaction When an agency contracts out a service, the agency remains responsible for the customers’ satisfaction.All service contracts have a customer, which may include the public, private companies, or other government agencies.Satisfaction may be measured through:Surveys: Distributed by contractor or agency, but results should be compiled by agencyForumsComplaint/compliment forms: paper, online, or agency phone numberAll government services have a customer – either citizens, private entities, other government agencies, or all of the aboveEven when service contracted out, government is still responsible for ensuring that customers are satisfiedNumerous methods – surveys, forums, complaint formsCONSIDER: the role of the contractor is measuring customer satisfaction; how much should be done independently by the agency?WE FOUND –Survey – 12 of 25 agencies (48%) said they use customer satisfaction surveys to monitor some of their contractsOn-Site – 5 of 20 contracts (25%) were using surveys to evaluate contractor performance.
20Dispute Resolution Procedures* Clear contract language should limit disputes, but disagreements between parties inevitably result.Dispute resolution is intended to address and solve problems before they escalate.Minor disagreements may be worked out between those individuals providing the service and those doing the regular monitoring (should be documented in files).If issue is significant, policy should require notification to the agency’s procurement office, which should provide the contractor with written notification of problems and a timetable for resolution.If not resolved, DOAS State Purchasing Office should be notified.NECESSARYDisputes between agencies and contractors are expected; generally there are competing financial interestsAgencies want to spend little; contractor wants to make a lotGoal of dispute resolution process is to solve problems as soon as possible – long before they escalate to contract termination or litigation.Minor disputes worked out between day-to-day players (should document)Significant disputes – policy should require that procurement office be notified. That office should ensure that contractor receives a written notice that includes a timetable for resolution.DOAS State Purchasing Office should be notified if still not resolved.
21Dispute Resolution Procedures (continued) Agency policy should detail at what level of non- compliance a contractor will receive a letter threatening contract termination.Dispute resolution clause should detail a procedure for contractor to appeal action taken by the agency. Depending on the action, the appeal may be to a higher ranking agency official or DOAS State Purchasing official.Appeals should be settled by negotiation or arbitration.Contract should detail level of non-compliance that will trigger a letter threatening termination.Contract should contain procedure for contractor to appeal agency actions.WE FOUND –Survey – 16 of 25 agencies (64%) reported written proceduresOn-Site – 5 of 20 contracts (25%) contained dispute resolution clauses
22Closeout ProceduresFormal, written closeout procedures ensure important elements are not overlooked.Procedures related to contract performance but also include other administrative items.Checklist should include verification that:All invoices paidAll property returnedAll deliverables/reports acceptedNo lawsuits pendingContract audit completed, if applicableNo outstanding changes or amendmentsAll security badges/keys returnedAll disallowed costs settledEnsures that important elements are not overlookedDone after all services completed/products deliveredElements such as : SEE ABOVE; some are directly related to monitoring, while others are administrative (e.g., returned badges, keys)WE FOUND –Survey – 15 of 25 agencies (60%) reported having some closeout proceduresOn-Site – 3 of 6 agencies were found to have written closeout procedures
23Post-Contract ReviewTypically done at the end of a contract period, but if the contract is subject to renewal, it can be done late in the contract term.Two purposes:Evaluating contractor performanceEvaluating agency method of contract monitoringEvaluating Contractor PerformanceDetermine if the contractor has adequately fulfilled contract terms. Simple task if the contract has been adequately monitored throughout the contract period.Final product will identify those contractors with negative performanceMay include a financial audit if agency determines risk makes it necessaryAt end of contract period, agency should consider evaluating contractor performance, as well as its own method of monitoringObviously, if the contract is subject to renewal, these evaluations should occur (as best they can) before contract renewalEvaluation of contractorConsider is programmatic review AND final audit are necessaryFinal product should identify if the contractor performed poorlyIf contractor has been monitored, the final task is simple (compilation/review of evidence already gathered)Agency must decide if contract warrants a formal review (will there be renewal; will service be contracted out again;
24Post-Contract Review Evaluating Agency Methods Did the contract and meetings with the agency give the contractor an adequate understanding of expectations?Did agency’s policies and procedures sufficiently address issues that arose during the contract period?Did the contract administration plan allow the agency to properly and quickly assess the contractor?Did agency personnel have the skills to monitor the contractor?If incentives and penalties were used in the contract, did they improve contractor performance?If dispute resolution was necessary, were the procedures adequate?Evaluating agency’s contract monitoring methodsSEE ABOVEWE FOUND –Survey – 10 of 25 agencies (40%) reported post-contract evaluationsOn-Site – 2 of 20 contracts (10%) had a formal post-contract evaluation
25Search: “contract monitoring” QuestionsReport is available online at :Search: “contract monitoring”