2. 2 Breakout Session # 1411 Robert F. Watts, Senior Consultant, ESI International Date: April 16 Time: 2:40 – 3:40 PM Buying Results Through Service Level Agreements: Best Practices in Performance- Based Acquisition

3. 3 Performance Based Acquisition -- Still a Challenge Recent Acquisition Advisory Panel study determined that more than half the contracts designated as PBA in the Federal Procurement Data System-Next Generation were not performance- based

4. 4 It’s All About Mindset… MindsetCharacteristics Process Focused on process and activities -- how requirements need to be accomplished Often lacks confidence in contractor’s quality control practices Prefers to micromanage the company's technical processes Personnel Focused on personnel - who needs to accomplish requirements Often lacks confidence in contractor’s human resource and management practices Prefers to micromanage the company's personnel Performance Focused on results and mission accomplishment - what needs to be accomplished Has confidence in contractor’s technical, quality, human resources, and management practices Prefers to manage outcomes and customer satisfaction and not to micromanage the company’s processes and personnel

5. 5 PBA Incentive Plan Quality Assurance Surveillance Plan Performance Work Statement or SOO Performance Requirements Summary

6. 6 What is a Service Level Agreement (SLA)? A performance-based technique where a written agreement is established between the customer and service provider that defines key service objectives, metrics, and acceptable quality levels (AQLs) –Primarily used in information technology (IT) procurements –Expanding into other sustainment-type services

7. 7 Example SLA Outline Service Level Agreement (Title of Agreement) 1.0 Introduction 1.1 Statement of Need 1.2 Background 2.0 Service Objectives 4.0 Service Level Metrics/AQLs 3.0 Applicable Directives 3.1 Compliance 3.2 Reference 4.0 Constraints and Assumptions 5.0 Data Deliverables 6.0 Government Furnished Property (GFP) Equivalent to PWS/SOO and Performance Requirements Summary Equivalent to PWS/SOO and Performance Requirements Summary

8. 8 Service Objectives Service Desk Provide, implement, operate and maintain a fully-functional, reliable Service Desk… ….for successfully recording, triaging, troubleshooting and resolving/ redirecting customer support requirements and ensuring minimum user downtime. Do what? To accomplish what? Example

9. 9 Service Level Metrics/AQLs Title: Service Desk Objective: Provide, implement, operate and maintain a fully-functional, reliable Service Desk for successfully recording, triaging, troubleshooting and resolving/ redirecting customer support requirements and ensuring minimum user downtime. Metrics: Response: Percentage of inbound calls answered within 20 seconds. First Call Resolution: Percentage of inbound calls handled to resolution at Tier-1. Quality: Percentage of customers surveyed expressing a “satisfactory” or better quality of service rating. Acceptable Quality Levels: Response: 80% First Call Resolution: 70% Quality: 80%

10. 10 Service Level Metrics: Possible Areas to Measure Network PerformanceHostingApplicationUser Support Availability: Percent of time the network is available to users (percentage) Server Availability: Percent of time the server is available to users (percentage) User Interactions: Percent of successful user interactions (downloads, data requests, and so forth) Help Desk: Response time and accuracy in resolving trouble tickets Throughput: Capacity of the backbone connections within the network’s core (bits/sec) Administration of Servers: Acceptable response times for restoring failed servers (period of time) Data Transfer: Acceptable time lapse between user request for data and moment data screen appears (seconds) User Training: Quality of user training, such as classroom, online, and so forth (customer survey) Connection: Acceptable data losses and data latency, plus bandwidth provisioning (percent of packet delivery rates) Data Backup and Storage: Scheduled data backups that will be conducted (percent of completion) Ease of Use: Quality of customer’s experience in using the application (customer survey) Needs Assessment: Quality and thoroughness of program for assessing ongoing user needs Security: Protection of applications, data, and services in transit (levels of encryption, number of breaches, and so forth)

11. 11 What is the QASP? An internal plan used by the customer that describes the performance monitoring methods they will use to evaluate the contractor’s progress and success in meeting service level metrics and AQLs contained in the SLA

12. 12 Some Techniques to Monitor Contractor Performance Example Performance Monitoring Approaches

13. 13 Contractor Monitoring (Example) Title: Service Desk Objective: Provide, implement, operate and maintain a fully-functional, reliable Service Desk for successfully recording, triaging, troubleshooting and resolving/ redirecting customer support requirements and ensuring minimum user downtime. Metrics: Response: Percentage of inbound calls answered within 20 seconds. First Call Resolution: Percentage of inbound calls handled to resolution at Tier-1. Quality: Percentage of customers surveyed expressing a “satisfactory” or better quality of service rating. Acceptable Quality Levels: Response: 80% First Call Resolution: 70% Quality: 80% Method of Government Monitoring: Response: Weekly audit of Service Desk reports First Call Resolution: Weekly audit of Service Desk reports Quality: Automated email customer satisfaction survey Monitoring Officials: COTR; Performance Monitors

14. 14 Automated Email Survey How satisfied are you that the Service Desk responded quickly to your problem? Very DissatisfiedDissatisfiedSatisfiedVery Satisfied OOOO Comments: How satisfied are you that the Service Desk was professional and “customer- oriented” in resolving your problem? Very DissatisfiedDissatisfiedSatisfiedVery Satisfied OOOO Comments: How satisfied are you in the overall quality of the Service Desk’s final resolution of your problem? Very DissatisfiedDissatisfiedSatisfiedVery Satisfied OOOO Comments:

15. 15 Incentive Plan Rewards and/or penalties assessed against the contractor based on their success in achieving service level metrics and AQLs in the SLA

16. 16 Possible Incentive Structures Award Fee Pool of dollars awarded at specified periods to contractor based on performance of SLA Award Term Additional contract terms awarded to contractor based on performance of SLA Past Performance Ratings Formally documented ratings of contractor performance of SLA Milestone Payments Linkage of contractor payments to successful completion of milestones contained in SLA Options Linkage of follow-on options to contractor based on successful performance of SLA Invoice Adjustments Adjustments to contractor billings based on performance of SLA

17. 17 Some Best Practices in Developing SLAs Keep it simple! –Look at the overall mission and develop a small set of key service level metrics—find things that are measurable, meaningful and have real impact on the bottom line Stay focused on mission success and customer satisfaction –Leave measuring of activities and processes to the contractor Manage AQL expectations – be realistic –The closer to 100% perfection you require, then the higher the price -- be practical and realistic Performance standards cannot exist in a vacuum; there must be a forum for discussion and negotiation

18. 18 Case Study Chicago adjusts on the fly to make its contract perform