Chapter 13 Measuring, Reporting, and Controlling Managing the Information Technology Resource Paula Goulding ICT622

Chapter Outline Importance of measurement What and how to measure Quality and efficacy of measures Reporting to different audiences Controlling IT through effective SLAs Negotiating a service level agreement Making SLAs work – metrics Introducing internal SLA to firm SLA myths Who owns measurements and what/when to do them Can enterprise succeed with or without measurements? ICT622

Why Is There IT? ICT622

Productivity Study of IT and Business Executives Source: InformationWeek research productivity study of 300 IT and business executives ICT622

Measuring IT IT expenditure as percent of revenue IT expenditure per employee Revenue per IT dollar Total IT expense budget vs. actual Total IT capital budget vs. actual Employees supported per IT employee ICT622

Importance of Measurement Business wants to know if IT is being run efficiently and effectively Identify opportunities for improving effectiveness and efficiency Effectiveness Doing things right Efficiency Doing things the right way ICT622

Measuring IT – Samples Overall – qualitative measures Overall – quantitative measures Network Data center ICT622

What and How to Measure Need to measure Business impact Customer relationships Internal organization impact Investment impact Value chain impact Know how processes are changing between any measurements, and explain any changes in results No ideal set of metrics ICT622

10-Step Approach to Developing Measurements Commitment of resources, time, and cooperation from senior business and IT management Have and review formal strategic plan, mission statements, and goals Committees need to be established for each of 38 IT processes Each committee meets and defines goals of each IT process as first objective Identify critical success factors ICT622

10-Step Approach to Developing Measurements Establish guidelines to measure if, and to what degree, critical success factors have been met Complete pilot should be done and members jointly participate in the pilot Recommendation to install these measurements should be raised to senior management for endorsement Distribution and implementation of new measures Analyze results and review with all partners ICT622

5 Stages of Benchmarking Planning Analysis Integration Action Maturity Source: Robert C. Camp and http://www.apqc.org ICT622

Benchmarks Planning Analysis Identify what is to be benchmarked Identify comparative companies Determine data collection method and collect data Analysis Determine current performance “gap” Project future performance levels ICT622

Benchmarks Integration Action Maturity Communicate benchmark findings and gain acceptance Establish functional goals Action Develop action plans Implement specific actions and monitor progress Recalibrate benchmarks Maturity Leadership position attained and practices fully integrated into process ICT622

Iterative Process for Benchmarking Source: Dataquest, Gartner Group. ICT622

Quality of Measures Often best when forecasting trends Can lead to misleading results Gain correct insight as to what measurement indicates Good quality may be significant indicator of good performance ICT622

Dimensions of Service Quality Reliability Consistency of performance and dependability Responsiveness Willingness/readiness of employees to provide service in timely manner Competence Possession of required skills to perform service Access Approachability and ease of contact Courtesy Politeness, respect, consideration, friendliness ICT622

Dimensions of Service Quality Communications Keeping customers informed in language they understand Credibility Trustworthiness, believability, honesty Security Freedom from danger, risk, doubt Understanding/Knowing the Customer Making effort to understand customer’s needs Tangibles Physical evidence of service ICT622

Dimensions of Successful IT Functions Service quality System quality Information quality Use User satisfaction Individual impact Work group impact Organizational impact ICT622

Source: Anthony, R.N. Planning and Control Systems: A Framework for Analysis, Cambridge, MA: Harvard University Press (1965). ICT622

Variety in Reporting Historical Data Statistical Data Assist managers in determining repairs Each has different characteristics ICT622

Characteristics of Effective SLAs Based on current “user” expectation Defined by location, function, and service type Uses benchmarks and baselines for measuring performance Specifies minimums, penalties, and incentives Specifies reports and tracking tools to be monitored Agreement not too few, not too many Gets adjusted periodically to reflect changes ICT622

Negotiating a SLA Get a Baseline Define Responsibility Allow for Modifications Ensure Reports Are in Terms Anyone Can Understand Don’t Forget the End-User Experience ICT622

Examples of Service Level Metrics Application availability Average application response time Number of application crashes per unit of time Average throughout Network availability and bandwidth Server availability ICT622

Examples of Service Level Metrics Elapse time to repair hardware failure Mean time between server failure Number of operating system failure per unit of time Number of middleware failure per unit of time Number of production jobs not completed during batch night shift ICT622

Questions Concerning Who Owns Measurement Who should own measurement? Is measure a process that should be owned by the people doing the measurement? Is it a corporate asset or cost? Is this question similar to who should own payroll? Should the owners of internal processes that are being measured simply own it? Does it make a difference? ICT622

What to do with Measurements Establish report card for performance Create effective and understandable benchmarks Senior management needs to know current status and effectiveness of IT ICT622

Performance Measures for Efficient Order Management Short order lead times In-stock availability Order accuracy Access to order status information Response time to customer inquiries ICT622

Performance Measures for Efficient Customer Management Lower inventory carrying cost Reduced human intervention Greater order accuracy Improved production planning/forecasting Lower order administration Lower number of complaints Improved customer decision making based on preferences Increase number of automatic restocking customers Lower stock-out occurrences ICT622

When Measurements Should be Done Continuously Daily Weekly Monthly Semiannually Annually ICT622

Can Enterprise Succeed Without Measurement? Only 42% of companies prepare business case for CRM project Only 45% have centralized CRM responsibility 57% can’t justify investments because they don’t know how to measure customer profitability Only 10% measure ROI ICT622

How Does Enterprise Benefit from Measurement? Performance is related to effectiveness, efficiency, and quality To understand process it must be measured and compared Part of monitoring project performance Determine if vendors are performing to service level agreements ICT622