1. Effective Contract Monitoring System
Components of an
Effective Contract Monitoring System
Georgia Department of Audits and Accounts
Performance Audit Operations Division
Intro
Topic that most are familiar with; Lot of experience in the room – more in the audience
this review should serve as a refresher and an opportunity to emphasize the importance of monitoring
You would be surprised at the number of gaps in monitoring that exist, likely in your agency
About 45 minutes

2. Purpose of the Report
We frequently found that agencies did not have adequate information on the level/quality of services provided on their behalf by contractors.
Mentality was that someone else was responsible for providing the service, not the agency.
Agencies cannot contract out responsibility to be a good steward of public funds.
Published in July 2003 as a “Best Practices in Government” report, not a traditional performance audit.
Reason for the Report
We routinely found a lack of oversight of the work performed once a contract was signed
Agency personnel was busy with tasks in front of them and saw the contract as a way to let someone else do the work
PROBLEM – agency is still responsible for ensuring the work is done and will ultimately be held responsible by auditors, legislators, and the public is the work is not done satisfactorily.
CONTRACT GUIDANCE AVAILABLE - State law, DOAS policies, as well as agency policies cover the procurement process in some detail; agencies generally understood purchasing authority, when competitive bids were necessary, the GA Procurement Registry
Released July 2003 – not as a performance audit
-Give brief description of each component
-Use in other states
-Use in Georgia – survey of 25 agencies; review of 20 contracts in 6 agencies

3. Overview of Contract Monitoring
Process of ensuring that a contractor adequately performs a contracted service.
An agency’s contract monitoring system has numerous elements, many of which are instituted through policies and procedures.
The responsibility of multiple individuals, within both program offices and administrative offices.
Cannot be a consideration only during the contract period. Must be a consideration during the pre- and post-contract periods.
Contract monitoring is one part of the larger topic of contract administration
Process of ensuring that a contractor adequately performs a contracted service.
Numerous elements of the system
This is the responsibility of individuals throughout an agency – within program offices and administrative offices (contracting offices/financial staff).
Finally, contract monitoring cannot be a consideration on the first day of the contract term.
Begins when the solicitation document is being created. Many elements of a successful monitoring system are in the RFP and contract.
Obviously monitoring is done during the contract term, but it also extends to a period after the contract has ended.

4. Components of a Monitoring System
Identified 15 components of an effective contract monitoring system.
Not all contracts are monitored using the same components, but a number of the components are necessary for an effective system.
We identified 15 components of an effective contract monitoring system through:
- a review of policies and procedures from Georgia and other states;
guides from the federal government and other states;
other states’ audits; and
publications from the National Institute of Governmental Purchasing.
The list was then reviewed by Georgia chapters of the National Contract Management Association, National Institute of Governmental Purchasing, and National Association of Purchasing Management.
IMPORTANT – This is not a list of items that are necessary for each and every contract.
It is a list of potential controls that management should consider when deciding how to ensure that the contractor fulfills the terms of the contract.

5. List of Components Contract Monitoring Training*
Use of Incentives and Consequences
Written Policies and Procedures*
Access to Records/Right to Audit*
Contingency Plans*
Measuring Customer Satisfaction
Communicating Clear Expectations to Contractors*
Dispute Resolution Procedures*
Contract Administration Plan*
Organized Contract Files*
Closeout Procedures
Payments Linked to Performance*
Post-Contract Review
Regular Programmatic Reports from Contractor*
* NECESSARY
On-Site Monitoring
List of components
I’m sure that most of you are familiar with most, if not all, of these
A number of these are necessary for an adequate contract monitoring system, though the formality of each (e.g., contingency plan) may vary based on the value, risk of the contract.

6. Training in Contract Monitoring*
Program employees responsible for much of the monitoring are often educated/experienced in their service area – not in contracting.
Many of the topics that should be included are listed as components in the report.
Ideally, training addresses agency-specific processes (e.g., agency’s policy on what types of contracts require a formal contingency plan).
NECESSARY
During performance audits, we often deal with program staff that are educated/trained in a subject area other than management, especially contract management.
--- Educators, social workers, engineers, law enforcement officials
Contract monitoring training can be incorporated into larger issue of contract management. But it should be given adequate attention.
Training should include the topics in this presentation, but when delivered at the agency level it can be more specific.
WE FOUND –
Survey - 10 of 25 agencies said staff received some training in contract management
On-Site Review - 0 of 6 agencies had adequate training in contract monitoring

7. Written Policies and Procedures*
Serve as a guide to agency personnel in ensuring a consistent, high-quality monitoring process. Policies in place often cover procurement with little attention paid to monitoring.
Items that should be covered in policies and procedures include:
Responsibilities of agency personnel
Contract correspondence
Contract monitoring reports
Defining conflicts of interest
Documenting contract administration decisions
Subcontractor administration
Standard contract terms
Dispute resolution
Contract closeout
Professional development of contract personnel
NECESSARY
Written policies and procedures help to ensure that agency personnel use a consistent, high-quality monitoring process
They are necessary for any agency process, especially those that are undertaken by multiple individuals across multiple offices/units
Should include: LIST FROM ABOVE
- Correspondence – where maintained; methods
WE FOUND –
Survey - 9 of 25 (36%) agencies had formal policies and procedures in place
On-Site Review – 1 of 6 had policies on monitoring and another 1 had policies that covered only financial aspects

8. Contingency Plans*
Addresses how the agency would respond in the event of an interruption in service delivery.
Detailed, written plans may not be necessary for low-risk contracts. Risk level determines level of detail.
Plan options include:
Contracting with the next lowest bidder from the original solicitation
Using another current contractor
In-house delivery of the service
Contracting with another government entity
NECESSARY
How the agency will respond if contractor quits, fired
Doesn’t have to be formal, written plan (though ideally written)
High risk – CMO contracts (What happens if one walks away?)
Low risk – DOT contracts with ROW appraisers
Options: SEE ABOVE
WE FOUND –
Survey – 15 of 25 agencies (60%) reported using contingency plans
On-Site Review – 2 of 20 contracts (10%) had contingency plans; 1 had been implemented

9. Communicating Clear Expectations*
Contract performance improved when agency and contractor are on the same page.
Three methods:
Detailed statement of work/scope of services
Performance measures and goals
Post-award meeting with contractor
Detailed Statement of Work
Begins as part of the solicitation document
Purpose of the contract, performance measures/goals, agency’s methods for monitoring performance, deliverables, and other requirements
Must be clear, concise, and free from ambiguity
As part of the contract, it contains responsibilities of the contractor, including those activities in which the contractor will engage, products to be produced, and timetables for the completion of activities.
Final SOW is a combination (and often a negotiation) of the SOW from the solicitation document and the contractor proposal.
NECESSARY
Obviously no party is going to be pleased if expectations are different; need to be on the same page
I’ll briefly discuss three methods of communicating clear expectations :
Detailed scope of services/statement of work
Performance measures/goals
Post-award meeting with the contractor
Detailed Scope of Services/Statement of Work
Part of the solicitation document and later the contract
Debate/clarification of scope of services big par t of procurement process
SEE SLIDE ABOVE

10. Communicating Clear Expectations (continued)
Performance Measures and Goals
Established to evaluate the quality and quantity of services provided
Should be included in the solicitation document; if necessary, additional measures can be negotiated before the contract is signed
Generally should include outputs and outcomes
Outputs measure a process and are units of the service
Outcomes assess the quality or result of the service
Contract should state what data will be collected by the contractor, and how, when and to whom the data will be submitted
Meeting with Contractors to Clarify Work
Post-award or kickoff meeting between agency and contractor
Especially important when those individuals involved in the day-to-day execution of services (contractor) and monitoring (agency) are not significantly involved in the procurement
Topics include contract scope, reporting requirements, administration procedures, rights and obligations of parties, performance standards, monitoring procedures, and potential problem areas
Formal meeting may not be necessary for less complex contracts
Performance Measures and Goals – difference between two (# versus desired #)
Needed to evaluate quality and quantity of services
Should be in the solicitation document; if not, must be in the final contract
Measures should include outputs and outcomes. Output measures units of service (e.g., number served; items provided), while outcomes measure the result of quality of the service (e.g., improvement in test scores)
Contract should state what data will be collected by the contractor, how, when and to whom it will be submitted
Post-award or Kickoff Meeting
Last opportunity to clarify/summarize the decisions made during the pre-contract period. Decisions may have been negotiated over weeks/months by many individuals.
Important to ensure the day-to-day players on both sides have a clear understanding.
Includes discussion of contract scope of work, reporting, administrative procedures, rights and obligations of parties, performance measures and goals, monitoring procedures, and potential problem areas
WE FOUND –
Survey – 22 of 25 (84%) agencies reported perf measures and post-award meetings
On-Site – 5 of 20 contracts (25%) had detailed Statements of Work; 7 (35%) had somewhat detailed Statements but lacked important items such as performance measures; 10 of 20 contracts (50%) had no performance measures and several were vague
EXAMPLE – 4 DJJ contracts for outdoor therapeutic camps. No performance measures for years; when added to contract, data not collected; contracts continued to be renewed
EXAMPLE – Medicaid drug rebates (no performance measures, no penalties, no incentives); after instituting better monitoring processes, 41% quarterly increase in rebates ($5.6 million)

11. Contract Administration Plan*
View of planned and completed activities and can be used as a status report.
Contains list of deliverables, milestones, contract modifications, summary of invoices submitted and paid, and renewal dates.
Also includes individuals responsible for monitoring each contract aspect, as well as the method of monitoring.
May be simple or complex, depending on the complexity of the contracted service, the contract value, and risk to the agency of poor performance.
Should consider that more intense monitoring early in the contract period will be beneficial.
NECESSARY
GA Procurement Manual gives list of items that should be in a CAP
Document that describes the who and how of the monitoring of a particular contract.
Ensures that all agency personnel are on the same page regarding responsibilities – between agency and contractor and within the agency
Contains items listed above – list and dates for deliverables, milestones, invoices submitted and paid
More intensive monitoring at the beginning of the contract period is often beneficial
WE FOUND –
Survey – 14 of 25 agencies (56%) reported CAPS for some of their contracts
On-Site Review – 2 of 20 contracts (10%) had written CAPs
At one agency, two different offices stated that the other was checking contractor performance against contract goals. The contractors didn’t even bother to report because no one asked for it. The three contracts had been renewed.

12. Organized Contract Files*
Necessary to have access to the master contract files.
Files should have all information necessary to know what was expected and received under the contract, as well as the basis for any major decisions made related to the contract.
At a minimum, files should contain:
Signed contract and purchase order
Contract modifications
Contract administration plan
Contingency plan
Sources solicited
Evaluation method and award
Meeting minutes
Contract correspondence
Reports from any on-site visits
Performance reports
Records of complaints and contractor disputes
Invoices and vouchers
NECESSARY
Central file containing information that allows someone to determine what was expected and received under the contract
Need access to contract files, especially when disputes arise with the contractor
Files should contain: SEE ABOVE
WE FOUND –
Survey – 19 of 25 agencies (76%) said they maintained a central contract file
Site Visits – 6 of 20 contracts (30%) had a single file maintained in one office
We regularly find that agencies have difficulty locating a signed contract; contract correspondence is often dispersed.

13. Payments Linked to Satisfactory Performance*
Payments should not be made unless the agency has assurance that the contractor is making satisfactory progress in fulfilling the contract.
Programmatic reports that directly relate to contract terms should be submitted with the invoice.
If a cost-reimbursement contract, adequate documentation to support an invoiced amount must be submitted.
Appropriate personnel, who may include a program official and an administrative (i.e., contracting) official, should review the invoice and supporting documentation prior to payment.
NECESSARY
Agency must ensure that contractor has fulfilled all contract requirements for quality, quantity, and timeliness before making a full payment.
Invoices should be accompanied by documentation that shows performance measures have been met
Before payment, program and administrative personnel should receive documentation submitted by the contractor
WE FOUND –
Survey – 22 of 25 agencies (88%) said payments tied to performance
Site Visits – 4 of 20 contracts (20%) had direct correlation, requiring proof of performance. For 13 contracts, program manager’s signature was necessary.
Common that admin requires program official signature, but program hasn’t received or adequately reviewed program data to ensure contract terms being met.

14. Regular Programmatic Reports from Contractor*
Contract should require contractor to provide specific information at designated intervals to ensure that performance goals are being met.
Content of the reports should be outlined in the contract, with a template or example as an appendix.
Contract must also include:
reporting method,
the individual who should receive the report,
the reporting frequency, and what
actions the agency may take in response to an incomplete report.
Weakness – Submitted by the contractor, who has a financial interest in stating that contract terms are being met. Reports can be designed to decrease the likelihood of misinformation (e.g., requiring list of participants instead of just a number).
NECESSARY
Contractor should be required to regularly submit documentation that contains programmatic information
The reports should address the performance measures in the contract and the report content/format should be dictated by the agency
- If format changes, harder to compare over time
Should be in contract: Report method (web-based, , hard copy), individual(s) to receive the report, reporting frequency and deadlines, and actions that can be taken by agency if not submitted/submitted properly
Realize that report data needs to be, at least periodically, verified by the agency
WE FOUND –
Survey – 21 of 25 agencies (84%) reported use of programmatic reports
On-Site – 10 of 20 contracts (50%) reviewed required scheduled programmatic reports sufficient enough to assess performance. Another 8 (40%) were available only upon request or were insufficient to assess contractor performance.

15. On-Site Monitoring
Can allow agency to verify actual performance against scheduled or reported performance.
Reinforces the importance of the contract to the contractor.
Provides an opportunity for enhanced communication.
Structured reviews are more productive than unstructured.
Should include random inspections of contractor records.
Can be resource-intensive; therefore, necessity dependent on the complexity and risk associated with the contracted service.
Written report detailing findings should be produced.
On-site monitoring allows agency to independently verify contractor performance; verify actual performance versus scheduled or reported perf
Reinforces the importance of the contract
Review should be structured; have a review instrument, but also allow for an opportunity for enhanced two-way communication
It can be more than a simple compliance review
listen/observe the contractor – you may find that different areas need your attention
Should have a written report
WE FOUND –
Survey – 20 of 25 agencies (80%) reported using method for some contracts
On-Site – On-site reviews that occurred were general in nature with little documentation showing that they were beneficial in assessing contractor compliance with contract terms

16. Incentives/Consequences Related to Performance
Performance reinforcements are helpful in achieving the contract goals.
Two types of incentives
Bonus: Superior performance may in the form of high quality work, completion of a project ahead of schedule, or saving the agency money. Agency must define the level of service required, the timeframe, or the amount of savings that must be realized. The incentive must be associated only with the most important contract requirements.
Revenue-sharing: If a contractor is operating a revenue-generating operation, it may be allowed to retain a portion of revenue. The portion retained can be increased as revenue levels increase.
Performance reinforcements - carrot and the stick approach - can be beneficial in improving contractor performance.
When incentives offered, agency must clearly define what level of service will be required in order for the contractor to receive the incentive. Similarly, financial consequences can only be imposed when the corresponding performance levels and consequences are clearly spelled out in the contract.
- These are intended to improve performance – not become a point of contention.
Two types of incentives –
Bonus for superior performance – Can be for exceptional quality, meeting or exceeding deadline, agency savings; Ensure that incentives only tied to most important aspects of the contract.
Contractor Retaining % of Revenue – A method when contractor operating a revenue-generating operation.

17. Incentives and Consequences (Continued)
Three types of consequences
Liquidated Damages: Quantify an agency’s loss for poor performance and deduct that loss from payments to the contractor; must be reasonable to avoid reduced competition and/or higher costs.
Partial Takeover of Operations: Agency takes over operations of areas in non-compliance and reduces contractor payments by new operating costs.
Contract Termination: Used when contractor has been given clear warning about non-compliance.
Examples of financial consequences:
Liquidated Damages – Quantifies an agency’s loss that occurs due to inadequate contractor performance; reduction in payment based on a formula, amount set in the contract; Must be reasonable; unreasonably high potential damages lead to increased risk to the contractor and fewer bidders and/or higher costs
Partial Takeover of Operations
Contract termination – Obviously a last resort, after working with contractor to solve problems
WE FOUND –
Survey – Incentives rarely used; 4 of 25 agencies (16%);
Financial consequences often used ; 16 of 25 agencies (64%)
On-Site – 0 of 20 contracts had incentive clauses; 15 of 20 (75%) had liquidated damages, partial takeover, termination for poor performance
Incentives – Maybe GDOT for quick completion of road construction

18. Access to Records/Right to Audit*
Must have enough access to contractor files to ensure that services are adequately delivered and/or funds spent appropriately.
Cannot simply rely on reports submitted by contractor.
Ability to audit contractor records should also extend to the records of any subcontractor.
NECESSARY
Very common contract clause
Agencies have responsibility to verify data reported by contractor and to ensure that funds are expended properly; this requires access to files/records
Right should also extend to the records of any subcontractor
WE FOUND –
Survey – 23 of 25 agencies (92%) reported having access to contractor records and right to audit
On-Site – 17 of 20 contracts (85%) contains clauses related to access to files and right to audit

19. Measuring Customer Satisfaction
When an agency contracts out a service, the agency remains responsible for the customers’ satisfaction.
All service contracts have a customer, which may include the public, private companies, or other government agencies.
Satisfaction may be measured through:
Surveys: Distributed by contractor or agency, but results should be compiled by agency
Forums
Complaint/compliment forms: paper, online, or agency phone number
All government services have a customer – either citizens, private entities, other government agencies, or all of the above
Even when service contracted out, government is still responsible for ensuring that customers are satisfied
Numerous methods – surveys, forums, complaint forms
CONSIDER: the role of the contractor is measuring customer satisfaction; how much should be done independently by the agency?
WE FOUND –
Survey – 12 of 25 agencies (48%) said they use customer satisfaction surveys to monitor some of their contracts
On-Site – 5 of 20 contracts (25%) were using surveys to evaluate contractor performance.

20. Dispute Resolution Procedures*
Clear contract language should limit disputes, but disagreements between parties inevitably result.
Dispute resolution is intended to address and solve problems before they escalate.
Minor disagreements may be worked out between those individuals providing the service and those doing the regular monitoring (should be documented in files).
If issue is significant, policy should require notification to the agency’s procurement office, which should provide the contractor with written notification of problems and a timetable for resolution.
If not resolved, DOAS State Purchasing Office should be notified.
NECESSARY
Disputes between agencies and contractors are expected; generally there are competing financial interests
Agencies want to spend little; contractor wants to make a lot
Goal of dispute resolution process is to solve problems as soon as possible – long before they escalate to contract termination or litigation.
Minor disputes worked out between day-to-day players (should document)
Significant disputes – policy should require that procurement office be notified. That office should ensure that contractor receives a written notice that includes a timetable for resolution.
DOAS State Purchasing Office should be notified if still not resolved.

21. Dispute Resolution Procedures (continued)
Agency policy should detail at what level of non- compliance a contractor will receive a letter threatening contract termination.
Dispute resolution clause should detail a procedure for contractor to appeal action taken by the agency. Depending on the action, the appeal may be to a higher ranking agency official or DOAS State Purchasing official.
Appeals should be settled by negotiation or arbitration.
Contract should detail level of non-compliance that will trigger a letter threatening termination.
Contract should contain procedure for contractor to appeal agency actions.
WE FOUND –
Survey – 16 of 25 agencies (64%) reported written procedures
On-Site – 5 of 20 contracts (25%) contained dispute resolution clauses

22. Closeout Procedures
Formal, written closeout procedures ensure important elements are not overlooked.
Procedures related to contract performance but also include other administrative items.
Checklist should include verification that:
All invoices paid
All property returned
All deliverables/reports accepted
No lawsuits pending
Contract audit completed, if applicable
No outstanding changes or amendments
All security badges/keys returned
All disallowed costs settled
Ensures that important elements are not overlooked
Done after all services completed/products delivered
Elements such as : SEE ABOVE; some are directly related to monitoring, while others are administrative (e.g., returned badges, keys)
WE FOUND –
Survey – 15 of 25 agencies (60%) reported having some closeout procedures
On-Site – 3 of 6 agencies were found to have written closeout procedures

23. Post-Contract Review
Typically done at the end of a contract period, but if the contract is subject to renewal, it can be done late in the contract term.
Two purposes:
Evaluating contractor performance
Evaluating agency method of contract monitoring
Evaluating Contractor Performance
Determine if the contractor has adequately fulfilled contract terms. Simple task if the contract has been adequately monitored throughout the contract period.
Final product will identify those contractors with negative performance
May include a financial audit if agency determines risk makes it necessary
At end of contract period, agency should consider evaluating contractor performance, as well as its own method of monitoring
Obviously, if the contract is subject to renewal, these evaluations should occur (as best they can) before contract renewal
Evaluation of contractor
Consider is programmatic review AND final audit are necessary
Final product should identify if the contractor performed poorly
If contractor has been monitored, the final task is simple (compilation/review of evidence already gathered)
Agency must decide if contract warrants a formal review (will there be renewal; will service be contracted out again;

24. Post-Contract Review Evaluating Agency Methods
Did the contract and meetings with the agency give the contractor an adequate understanding of expectations?
Did agency’s policies and procedures sufficiently address issues that arose during the contract period?
Did the contract administration plan allow the agency to properly and quickly assess the contractor?
Did agency personnel have the skills to monitor the contractor?
If incentives and penalties were used in the contract, did they improve contractor performance?
If dispute resolution was necessary, were the procedures adequate?
Evaluating agency’s contract monitoring methods
SEE ABOVE
WE FOUND –
Survey – 10 of 25 agencies (40%) reported post-contract evaluations
On-Site – 2 of 20 contracts (10%) had a formal post-contract evaluation

25. Search: “contract monitoring”
Questions
Report is available online at :
Search: “contract monitoring”