1. Chapter 13 Measuring, Reporting, and Controlling
Managing the Information Technology Resource
Jerry N. Luftman
© 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 1

2. Chapter Outline Importance of measurement What and how to measure
Quality and efficacy of measures
Reporting to different audiences
Controlling IT through effective SLAs
Negotiating a service level agreement
Making SLAs work – metrics
Introducing internal SLA to firm
SLA myths
Who owns measurements and what/when to do them
Can enterprise succeed with or without measurements?
© 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 2

3. Why Is There IT?
© 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 3

4. Productivity Study of IT and Business Executives
Source: InformationWeek research productivity study of 300 IT and business executives
© 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 4

5. Measuring IT IT expenditure as percent of revenue
IT expenditure per employee
Revenue per IT dollar
Total IT expense budget vs. actual
Total IT capital budget vs. actual
Employees supported per IT employee
© 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 5

6. Importance of Measurement
Business wants to know if IT is being run efficiently and effectively
Identify opportunities for improving effectiveness and efficiency
Effectiveness
Doing things right
Efficiency
Doing things the right way
© 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 6

7. Measuring IT – Samples Overall – qualitative measures
Overall – quantitative measures
Network
Data center
© 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 7

8. What and How to Measure Need to measure
Business impact
Customer relationships
Internal organization impact
Investment impact
Value chain impact
Know how processes are changing between any measurements, and explain any changes in results
No ideal set of metrics
© 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 8

9. 10-Step Approach to Developing Measurements
Commitment of resources, time, and cooperation from senior business and IT management
Have and review formal strategic plan, mission statements, and goals
Committees need to be established for each of 38 IT processes
Each committee meets and defines goals of each IT process as first objective
Identify critical success factors
© 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 9

10. 10-Step Approach to Developing Measurements
Establish guidelines to measure if, and to what degree, critical success factors have been met
Complete pilot should be done and members jointly participate in the pilot
Recommendation to install these measurements should be raised to senior management for endorsement
Distribution and implementation of new measures
Analyze results and review with all partners
© 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 10

11. 5 Stages of Benchmarking
Planning
Analysis
Integration
Action
Maturity
Source: Robert C. Camp and
© 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 11

12. Benchmarks Planning Analysis Identify what is to be benchmarked
Identify comparative companies
Determine data collection method and collect data
Analysis
Determine current performance “gap”
Project future performance levels
© 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 12

13. Benchmarks Integration Action Maturity
Communicate benchmark findings and gain acceptance
Establish functional goals
Action
Develop action plans
Implement specific actions and monitor progress
Recalibrate benchmarks
Maturity
Leadership position attained and practices fully integrated into process
© 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 13

14. Iterative Process for Benchmarking
Source: Dataquest, Gartner Group.
© 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 14

15. Quality of Measures Often best when forecasting trends
Can lead to misleading results
Gain correct insight as to what measurement indicates
Good quality may be significant indicator of good performance
© 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 15

16. Dimensions of Service Quality
Reliability
Consistency of performance and dependability
Responsiveness
Willingness/readiness of employees to provide service in timely manner
Competence
Possession of required skills to perform service
Access
Approachability and ease of contact
Courtesy
Politeness, respect, consideration, friendliness
© 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 16

17. Dimensions of Service Quality
Communications
Keeping customers informed in language they understand
Credibility
Trustworthiness, believability, honesty
Security
Freedom from danger, risk, doubt
Understanding/Knowing the Customer
Making effort to understand customer’s needs
Tangibles
Physical evidence of service
© 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 17

18. Dimensions of Successful IT Functions
Service quality
System quality
Information quality
Use
User satisfaction
Individual impact
Work group impact
Organizational impact
© 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 18

19. Source: Anthony, R.N. Planning and Control Systems: A Framework for Analysis, Cambridge, MA: Harvard University Press (1965).
© 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 19

20. Variety in Reporting Historical Data Statistical Data
Assist managers in determining repairs
Each has different characteristics
© 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 20

21. Characteristics of Effective SLAs
Based on current “user” expectation
Defined by location, function, and service type
Uses benchmarks and baselines for measuring performance
Specifies minimums, penalties, and incentives
Specifies reports and tracking tools to be monitored
Agreement not too few, not too many
Gets adjusted periodically to reflect changes
© 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 21

22. Negotiating a SLA Get a Baseline Define Responsibility
Allow for Modifications
Ensure Reports Are in Terms Anyone Can Understand
Don’t Forget the End-User Experience
© 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 22

23. Examples of Service Level Metrics
Application availability
Average application response time
Number of application crashes per unit of time
Average throughout
Network availability and bandwidth
Server availability
© 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 23

24. Examples of Service Level Metrics
Elapse time to repair hardware failure
Mean time between server failure
Number of operating system failure per unit of time
Number of middleware failure per unit of time
Number of production jobs not completed during batch night shift
© 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 24

25. Questions Concerning Who Owns Measurement
Who should own measurement?
Is measure a process that should be owned by the people doing the measurement?
Is it a corporate asset or cost?
Is this question similar to who should own payroll?
Should the owners of internal processes that are being measured simply own it?
Does it make a difference?
© 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 25

26. What to do with Measurements
Establish report card for performance
Create effective and understandable benchmarks
Senior management needs to know current status and effectiveness of IT
© 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 26

27. Performance Measures for Efficient Order Management
Short order lead times
In-stock availability
Order accuracy
Access to order status information
Response time to customer inquiries
© 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 27

28. Performance Measures for Efficient Customer Management
Lower inventory carrying cost
Reduced human intervention
Greater order accuracy
Improved production planning/forecasting
Lower order administration
Lower number of complaints
Improved customer decision making based on preferences
Increase number of automatic restocking customers
Lower stock-out occurrences
© 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 28

29. When Measurements Should be Done
Continuously
Daily
Weekly
Monthly
Semiannually
Annually
© 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 29

30. Can Enterprise Succeed Without Measurement?
Only 42% of companies prepare business case for CRM project
Only 45% have centralized CRM responsibility
57% can’t justify investments because they don’t know how to measure customer profitability
Only 10% measure ROI
© 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 30

31. How Does Enterprise Benefit from Measurement?
Performance is related to effectiveness, efficiency, and quality
To understand process it must be measured and compared
Part of monitoring project performance
Determine if vendors are performing to service level agreements
© 2004 Managing the Information Technology Resource, Jerry N. Luftman Chapter 13 - Slide 31