1. By: Satchanawit Phongkriangkrai
Chapter 9: Evaluating the Outcomes of Information Systems Plans Managing information technology evaluation – techniques and processes*
By: Satchanawit Phongkriangkrai
* An earlier version of this chapter appeared in the European Management Journal, Vol. 10, No.2. June, pp. 220–229.

2. Overview: Information systems planning

3. Agenda Evaluation: emerging problems Strategy and information systems
Evaluating feasibility: findings
Linking strategy and feasibility techniques
CODA: From development to routine operations

4. Introduction
The size and continuing growth in IT investments from the early 1990 served to place IT issue above the ranks in most organization .
Many organizations have encountered IT investment problems, e.g., high risks, and hidden cost processes.
At least 20% of IT investment is wasted and between 30% and 40% of IT projects realize no net benefits.
Major problems occur in how the IT investment is evaluated and controlled.

5. Evaluation: emerging problems
Evaluation brings into question the notion of cost, benefit, risk, value, and process.
There are many problems that impact on IT evaluation:
inappropriate measures
understating human and organizational costs
overstating cost
neglecting intangible benefits, etc.
The fundamental and common failure relates to strategy alignment.

6. Strategy and information systems
Organizational investment climate is affected by:
The financial health and market position of the organization
Industry sector pressures
The organizational business strategy and direction
The management and decision-making culture
Decision-making culture
Conservative
Innovative
Decision-making style
Directive
Consensus-driven

7. Alignment
Alignment creates strategic climate in which IT investment can be related to business/organizational direction.
Lack of alignment is a common problem in public sector information.
When the organization lacks alignment of IT evaluation practice, they tend to become separated from business needs and plans, and from organizational realities.
Another critical alignment is that between:
what is done with IT
how it fits with the information needs of the organization

8. IT Strategic grid Developed by McFarlan and McKenney in 1983
Classifies systems where IT investment has been made and where it should be applied.
Demonstrates whether the IT vestments are being made:
into core system,
business growth,
competitiveness.
Identifies four classes of firms.

9. Figure Strategic grid analysis

10. Value chain Established by Porter and Millar in 1991
Consists of the primary and support activities.
Primary activities of a typical manufacturing company:
inbound and outbound logistics
operations
marketing and sales
support service
Support activities:
firm infrastructure
human resource management
technology development
procurement

11. IT investment mapping Developed by Peters in 1993
The basic dimensions of map were arrived at after reviewing the main investment concerns.
Two dimension over 50 IT projects
investment orientation
Infrastructure – software/hardware environment
Business Process – finance and accounts
Market Influence – increasing repeat sales
and benefit:
Business Expansion
Risk Minimization
Enhance Productivity

12. Figure IT investment mapping

13. Figure Investment map comparing business and IT plans

14. Multiple methodology Developed by Earl in 1989
A multiple methodology inquires IT investment more closely with the strategic aims and direction of the organization, and its key needs.
Three strategy formulas are:
A top-down approach – use critical factor success (CFC) to establish objectives;
A bottom-up evaluation – evaluate current systems;
Inside-out innovation – create new strategic options.
The purpose is to identify through internal and external analysis of business needs and opportunities to relate the development of IS applications to business strategy.

15. Evaluating feasibility: findings
Evaluating IT projects at its feasibility stage are based on the right strategic climate.
Feasibility evaluation provides results of interest and to sponsor of research.
Value of IT/IS are justified by understating cost and using notational figure for benefit realization.
Willcocks and Lester looked at 50 organizations from both private and public section of manufacturing and service.
There is little evidence of a concern for assessing risk in any formal manner.

16. Figure IT evaluation: feasibility findings

17. Linking strategy and feasibility techniques
A method uses evaluation techniques to
measure the type of IT project,
develop techniques relating the IT investment to business/organization value.
The method of evaluation needs to be
reliable,
consistent in its measurement over time.
The three evaluation techniques are:
Return on management (ROM),
Matching objectives, projects and techniques,
From cost-benefit to value.

18. Linking strategy and feasibility techniques (Return on management (ROM))
Developed by Strassman in 1990
ROM is modern IT investment evaluation.
ROM measures the performance based on the added value – the difference between net revenues and payment external supplier – to an organization provided by management.
However, ROM is not widely used in UK.
The problem is whether it really represents what IT has contributed to business performance.

19. Linking strategy and feasibility techniques (Matching objectives, projects and techniques)
Butler Cox (1990) suggests five main purposes:
surviving and functioning as a business
improving business performance by cost reduction/increasing sales
achieving a competitive leap
enabling the benefits of other IT investments to be realized
being prepared to compete effectively in the future
The matching of IT investment can now be categorized as follows:
Mandatory investments
Investments to improve performance
Competitive edge investments
Infrastructure investments
Research investments

20. Figure Classifying IT investments

21. Figure Matching projects to techniques

22. Linking strategy and feasibility techniques (From cost-benefit to value)
Known as the information economics approach
Developed by Parker et al. 1988
This technique copes with many problems of IT evaluation, both at the level of methodology and of process.
Information economics looks beyond benefit to value.
Value is the sum of:
Enhanced return on investment,
Business domain assessment,
Technology domain assessment.

23. Figure The information economic approach

24. Linking strategy and feasibility techniques (From cost-benefit to value (Cont’d))
Enhanced ROI
Value Linking – assesses indirect benefits added to other departments.
Value Acceleration – reduced time-scales for operations.
Value Restructuring – measures the benefits of restructuring a department, or job.
Innovation valuation – considers the value of competitive advantage and calculate risks, pioneer cost, or failing cost.
Business Domain Assessment
Strategic Match: matches the business goals to the proposed project/IT investment.
Competitive Advantage – assesses the degree of an advantage in the market.
Management Information – assesses the contribution toward the management need for information.
Competitive Response – assesses the degree of corporate risk associated with not undertaking the project.
Organization Risk – implements the project in terms of personnel, skills, and experience.

25. Linking strategy and feasibility techniques (From cost-benefit to value (Cont’d))
Technology Domain Assessment
Strategic IS Architecture – measures the degree to which the project fits into the overall information system plan.
Definitional Uncertainty – assesses the complexity of the area and probability of non-routine change.
IS Infrastructure Risk – evaluates a project’s dependence on new or untried technologies.

26. CODA: From development to routine operations

27. CODA: From development to routine operations (Cont’d)
The evaluation cycle is useful for
controlling a specific project,
building organizational know-how on IT and management.
However, some of limitations in evaluation techniques and processes are:
Weak linkage between evaluations carried out different stages,
Projects shelved,
Stakeholders not included in feasibility stage,
Continual problems from feasibility stage.

28. CODA: From development to routine operations (Cont’d)
Suggestion to improve evaluation:
Linking evaluation across stages and time
Determining stakeholders who are participants in evaluation at all stages
Avoiding the fall-of interest in evaluation at later stage
Providing adequate evaluation techniques for the long term process.

29. Conclusions The profile of IT evaluation has increased
high expenditure,
disappointed expectations,
underdeveloped and undermanaged area.
Need more effective evaluation practice:
traditional techniques,
tailor modern techniques.
In the past of the evaluation practice, value is price.
The future challenge is to continue the measurement of value and build techniques and processes.

30. Thank you Q&A