1. SPPMG Event 18th February 2010 Why (IT) Projects Fail
Malcolm Bronte-Stewart
School of Computing
University of the West of Scotland
Malcolm Bronte-Stewart UWS

2. Objectives This presentation will:
Discuss some of the costs and causes of technology project failure
Challenge the way in which project failure is judged
Suggest pre and post project evaluation techniques that may be useful to SMEs
M B-S

3. Cobb’s Paradox
“We know why projects fail, we know how to prevent their failure – so why do they still fail?”
Martin Cobb, Treasury Board of Canada Secretariat in Unfinished Voyages, a follow to the CHAOS report, The Standish Group
M B-S

4. Lyytinen and Robey (2000)
“Organisations fail to learn from their experience in systems development because of limits of organisational intelligence, disincentives for learning, organisational designs and educational barriers. Not only have many organisations failed to learn they also have learned to fail. Over time they accept and expect poor performance while creating organisational myths that perpetuate short-term optimisation.”
M B-S

5. Project Failure
IT projects tend to go over budget and / or over time schedule and / or do not meet expectations
Research by the BCS, Royal Academy of Engineering, OASIG, Oxford University and others suggest that only about 15% to 30% are successful
Conservative estimates put the cost of IT project failure at tens of billions of Euros across the EU Jaques, 2004 (142 billion euros in 2004 McManus & Wood-Harper, 2008) and around $500 billion wasted on IT purchases that fail to reach their objectives worldwide Feld & Stoddard, 2004
We go on making expensive mistakes
M B-S

6. M B-S

7. M B-S

8. M B-S

9. M B-S

10. M B-S

11. M B-S

12. M B-S

13. M B-S

14. M B-S

15. M B-S

16. IT Development Project Costs
$55 billion wasted on cancelled and over-run IT projects in the USA in 2002 (Standish, March 2003)
“Public sector IT expenditure over past 12 months is in excess of £12.4 billion with a significant proportion at risk of being wasted” according to a House of Commons select committee (published July 2004)
“This year (2006) organisations and governments will spend and estimated $1 trillion on IT hardware, software and services worldwide”. (Charette, IEEE)
M B-S

17. IT project success statistics
75% of IT projects are challenged and 10% are abandoned (Oxford University, August 2003)
Only 7% of the 1000 firms Critical Research surveyed thought they were using IT effectively and 75% of these firms think that their IT systems are not providing a return on investment
Of 3,682 projects in 365 firms – 31% were cancelled, 53% had cost overruns and poor functionality and only 12% were on-time and budget (Johnson)
Three quarters of large systems may be considered failures (Laudon & Laudon)
M B-S

18. For example - some Headlines
New HMRC inland revenue tax system producing wrong codes “ we’ve heard it all before”.
Labour accused of wasting 26bn on failed IT projects “stupendous incompetence”.
Rural payments scheme put out to grass “a display of scant regard for protecting public money”.
C-nomis offender management system “a master class in sloppy project management”
Edinburgh Fringe Box-office system “weak”, “fundamentally flawed” “insufficient planning, lack of risk management, inadequate communications and no authorised business case”.
Strathclyde Police Computer System “a complete disaster”
Libra the IT system for magistrates courts “One of the worst projects I have ever seen”.
M B-S
Malcolm Bronte-Stewart UWS

19. “Prison IT system guilty of 'basic' project management failures” (2009)
The £234m C-Nomis IT system for Prisons failed in almost every possible way .
The NAO concluded that the technical complexity had been “significantly underestimated”. C-Nomis was treated as an IT project and not a business-change programme, project management was poor, and contracts with suppliers were weak. 
Edward Leigh, chairman of the Public Accounts Committee, said of C-Nomis that “kindergarten mistakes” had been repeated: “This Committee hears of troubled government projects all too frequently. But the litany of failings in this case are in a class of their own. All of this mess could have been avoided if good practice in project management had been followed.”
M B-S

20. IT project failures
The Defence Information Infrastructure project to incorporate 150,000 terminals for 300,000 users at over 2000 defence sites is 18 months late and running more than £180m over budget.
The official cost of the controversial national identity system has soared to over £5bn.
The NHS IT scheme [NPfIT] was initially estimated to cost around £2.3bn. The most recent estimate is £12.7bn.
M B-S

21. IT Project Failure
Sainsbury’s, the supermarket giant, installed an automated fulfilment system to increase efficiency and streamline operations in 2003.
The system ran into "horrendous" barcode-reading errors. Yet in 2005 the company claimed the system was operating as intended.
Two years later, the entire project was scrapped, and Sainsbury's wrote off £150 million* in IT costs.
* (One report claims it was a $526m investment)
M B-S

22. Avon County Council installed a new computer program to pay staff wages.
The spree started in a small way paying a caretaker £75 an hour.
Then it didn’t pay canteen workers at all for 7 weeks.
Next it paid a janitor £2,600 for a week’s work.
A deputy headmistress received her year’s annual salary once a month.
Heads of department earned less than their assistants.
Some people had more tax deducted in a week than they earned all year.
By February 280 council employees were out on strike.
(Stephen Pile)
M B-S

23. The misguided torpedo assumption
To prevent the possibility of a torpedo returning and exploding against the ship it was fired from
a safety system built so that it would self-destruct if it turned 180 degrees
Unfortunately, during trials, a torpedo jammed in its tube on board the ship, the test was abandoned and the ship turned round to go home …. BANG!
M B-S

24. London Ambulance System
Hilary Kane
London Ambulance System
New Software “Put Lives At Risk!”
Main Findings of investigation
Inexperienced procurement team
Staff mistrust and opposition
Over-ambitious timetable
Price put before quality
Incomplete and untested software
Andersen Consulting report suppressed
Management failed to identify and solve problems
Users “did things wrong”
“...a faulty system implemented with undue haste, by a management determined to impose its will...”
M B-S
Malcolm Bronte-Stewart UWS

25. IT Project Research
Jones, in a study of 250 large software projects at or above 10,000 function points* between 1995 and found that about 25 were successful, 50 had minor delays and 175 had major delays and overruns or had been terminated without completion.
He notes that the main problems were not technical but were due to aspects of poor project management.
*(equivalent to around1,250,000 statements in the C programming language)
M B-S

26. Wider effects of IT project failure
Perceptions of poor success rates and wasted resources affect decision making
The more IT projects are seen to go wrong the more:
the public become cynical
staff learn to expect problems and delays
Developers wonder if a lot of their work is likely to be wasted effort
business people become nervous of technology change
those holding the purse strings may view IT as a worry and a poor return on investment
M B-S

27. What are the main causes of failure?
6 major research studies reach remarkably similar conclusions about the significant causes that threaten project success:
OASIG
Standish (CHAOS)
Select Committee
OGC / NAO
Schmidt, Lyytinen, Keil & Cule,
Fortune and White
M B-S

28. OASIG Report (Clegg et al,1996)
Outcomes from IT investment :
80% to 90% do not meet goals
80% delivered late and over budget
40% fail or are abandoned
Under 40% address training and skills enough
Less than 25% integrate business and technological objectives properly
Only 10% - 20% meet all success criteria
M B-S

29. IT Project Failure (OASIG)
Main reasons why IT projects fail
Management agenda is too limited
most IT investments are technology led
main investment motive is to only cut costs
This narrow focus on technical capabilities and efficiency goals means that inadequate attention is given to the human and organisational issues that can determine a project’s ultimate success.
Users don’t influence development enough
Senior managers don’t understand the links between technical and organisational change
Project management techniques and IT approaches are too technical
Companies fail to organise work or design jobs/roles properly
M B-S

30. The Standish Group (1995)
Of the $250 billion spent each year on IT application development
31.1% of projects cancelled before completion (failed).
52.7% of projects were challenged: over-budget, over the time estimate, and offers fewer features and functions than originally specified.
16.2% of software projects were successful - completed on time, on-budget and to specification, (but some of the largest of which have only approximately 42% of the originally-proposed features and functions).
Recently these results have improved somewhat but remain disappointing.
M B-S

31. Standish Group CHAOS Survey Results
M B-S

32. M B-S

33. CHAOS Project Success Factors
User involvement
Executive management support
Clear and firm statement of requirements
Proper planning and formal methodology
Realistic expectations
Minimised scope and smaller project milestones
Competent, skilled staff
Ownership
Clear vision and business objectives
Hard-working, focussed staff
Experienced project managers
Reliable estimates
M B-S

34. Select Committee on Public Accounts
Report on Improving the delivery of Government IT projects (Study of 25 projects 1990 to 2000)
“Implementing IT systems has proved difficult”
Frequent cases of delay, confusion and inconvenience for the citizen and poor value for money for the tax payer.
M B-S

35. Select Committee Report conclusions
Decisions about IT are Business not technical, they have profound effects on customer service and must involve senior management.
End users and their business needs must be identified before the project commences so that clear objectives are taken into account fully during design and development
Scale and complexity – how ambitious? Can it be undertaken in one go?
Skilled Project Managers are essential
Sound methodologies and well conceived risk management are called for
Need for a high degree of professionalism in the definition, negotiation and management of IT contracts
Training can take up considerable resources but must address the needs of the users and of those maintaining the systems if the anticipated benefits are to be realised
Contingency plans should be in place
Organisations should learn lessons from project undertaken. A post implementation review should establish the extent to which they secured the proposed business benefits, user expectations and technical requirements.
M B-S

36. OGC and NAO Best Practice 2005 Common causes of project failure
1. Lack of clear links between the project and the organisation's key strategic priorities, including agreed measures of success.
2. Lack of clear senior management and Ministerial ownership and leadership.
3. Lack of effective engagement with stakeholders.
4. Lack of skills and proven approach to project management and risk management.
5. Too little attention to breaking development and implementation into manageable steps.
6. Evaluation of proposals driven by initial price rather than long-term value for
money (especially securing delivery of business benefits).
7. Lack of understanding of, and contact with the supply industry at senior levels in the organisation.
8. Lack of effective project team integration between clients, the supplier team and the supply chain.
M B-S

37. Keil, Cule, Lyytinen & Schmidt (1998)
These researchers recruited 3 panels of experienced project managers in different places – Finland, Hong Kong & U.S.A. – and asked them to identify and rank specific risk factors.
Lack of top management commitment to the project
Failure to gain user commitment
Misunderstanding the requirements
Lack of adequate user involvement
Failure to manage end user expectations
Changing scope / objections
Lack of required knowledge / skills in project personnel
Lack of frozen requirements
Introduction of new technology
Conflict between user departments
M B-S

38. Fortune and White (2006)
Fortune and White reviewed 63 publications that focus on project Critical Success Factors
The top ten (in order of count of citations) of the 27 they quote are:
Support from senior management
Clear realistic objectives
Strong / detailed plan kept up to date
Good communications / feed back
User / client involvement
Skilled / suitably qualified / sufficient staff / team
Effective change management
Competent project manager
Strong business case / sound basis for project
Sufficient / well allocated resources
M B-S

39. M B-S Select Committee OASIG NAO / OGC Keil et al
CHAOS (Top Requirements:)
Fortune and White (CSFs)
IT projects are driven by business (not technical) decisions
Many IT investments are seen only as technology led and aimed at cost cutting
Evaluation of proposals driven by initial price rather than long term value, especially securing delivery of business benefits
Misunderstanding user requirements
Clear business objectives and Realistic expectations
Strong business case / sound basis for project
Insufficient involvement from users
Users do not influence development enough
Lack of effective engagement with stakeholders
Lack of user involvement or commitment
User involvement
User / client involvement
Clear objectives should be set from the start
Need to set and review strategic objectives for change
Lack of clear link between the project and the organisation’s key strategic priorities, including agreed measures of success
Unclear and changing scope and objectives
Clear and firm statement of requirements
Clear realistic objectives
Lack of commitment from senior management
Management agenda is often too limited or narrow
Lack of clear senior management ownership and leadership
Lack of top management commitment
Executive management support
Support from senior management
Large projects may be overambitious
Inadequate attention is given to human and organisational issues
Too little attention to breaking development and implementation into manageable steps
Number of organisational units involved
Minimised scope and smaller project milestones
Project size, complexity, number of people involved, and duration
Skilled project managers are essential to keep to time and budget and appropriate deliverables
Senior managers do not understand the link between technical and organisational change
Lack of skills and proven approach to project management and risk management
Lack of required knowledge and effective project management skills
Experienced project managers
Competent project manager and effective change management
Success depends on good risk analysis and sound methodologies
Some project management techniques and IT approaches are too technical
Lack of effective project management methodology
Proper planning and formal methodology
Correct choice / past experience of project management methodology / tools
Contingency plans should be in place
Must work to detailed implementation plans
Not managing change properly
Reliable estimates
Strong / detailed plans kept up to date
User and operator training must be planned and designed
Failure to organise changes in work and roles properly
Inadequate resources and skills to deliver the total delivery portfolio
Inappropriate staffing and ill defined responsibilities
Competent, skilled and focussed staff
Skilled / suitably qualified and sufficient staff / team
There should be a post-implementation review
Introduction of new technology
Standard software infrastructure
Planned close down / review, acceptance of possible failure
Need professionalism in the definition, negotiation and management of IT contracts
Lack of understanding of , and contact with, the supply industry at senior levels in the organisation
Ownership
Good communication / feedback
M B-S

40. Are IT Projects Different?
IT and software-based projects have certain characteristics which make them different
specification
requirements creep
invisibility
complexity
flexibility
estimation
training / churn
changing technologies
communication and conflict
resistance to change
testing
M B-S

41. Risk Analysis
Consideration of these recognised common causes of project failure, taken with reports, case studies, project management experience and previous research, build a picture of the variety and scope of important risk factors.
Instead of expecting individuals to guess the likely risks for each project we can specify a standard, comprehensive collection of a variety of aspects and issues that threaten successful IT project outcomes.
M B-S

42. Risk Analysis Technique
A technique has been developed that is intended to help project managers and others to gauge and take account of these risk factors.
It requires little training and gives a structured list of relevant questions that invite the user to consider the extent to which each element threatens the project under review .
These can be separated into 3 (or 4) groups:
Some concern the firm itself and are slow to change
Some concern the project in question
Some concern the details of the options being assessed and compared
To create a series of Risk Tables
This Organisation
This Project
This Option
(This Environment)
M B-S

43. M B-S

44. M B-S

45. M B-S

46. Risk or Threat Level
Each factor may be awarded its own weighting (or ignored) according to its perceived significance
Multiplying the weights by the scores gives a total value for each factor and these can be summed to arrive at the overall value
Comparing your risk analysis values with those of others provides a focus and common language to bring out concerns, specific issues, enlightened evaluation and assessment of likely dangers and potential problems
The technique may become more valuable the more it is used as experience of the results and the model’s inadequacies develops
M B-S

47. Other issues
There are at least two other aspects that might usefully be taken into account:
The wider environment, trends and market conditions, ie the level above the first table that is outside the control of the organisation in question but influences project choice
The degree of uncertainty in the estimates of risk factors
So these tables provide an IT project risk analysis ready-reckoner in a standardised and quantifiable form that can be applied before the project begins, or during a feasibility study
But how do we judge the finished project?
M B-S

48. Measuring Success
Three factors are used to decide if a project is a success or a failure soon after its closure:
Time / on schedule?
Money / within budget?
Delivered product / to specification?
Fail any one of these tests and it may be labelled a failed project (even though it passes the other two) }
Yes
or No
M B-S

49. Three factors – “The Iron Triangle”
Good?
Quality / Scope / Functionality
Cost
Time
Did the project come in on or under budget and schedule?
AND did the project meet customer requirements?
Cheap?
Quick?
M B-S

50. Failures? What about the extent and type of the failure?
Some are abandoned, some are incomplete but in use
Some just overrun budget or time, others are total disasters
The project management process may be a mess
AND / OR
The delivered product may be useless or unwanted
How do we differentiate and categorise the relative success / failure of IT projects?
M B-S

51. Maybe graphics would help?
Budget Time Scope
400%
300%
200%
100%
50%
25%
10%
Over
Original
estimate
Under
M B-S

52. Failure?
Was failure an event or a process – did it suddenly appear or take time to become evident?
Was / Were there:
Undisclosed needs and requirements creep?
Over-optimistic budget and timescale?
Weak methodology, changing team and power struggles?
Does blame lie more with the clients who expected too much, or the planner who set the parameters, or the project manager who had to please the clients and abide by the parameters?
How can we tell?
M B-S

53. So What is Project Failure?
Should a project be regarded as a failure if it
overruns budget or timescale slightly but the customer is happy with the results?
Comes in under budget and time but gains little acceptance
Judging overall success by using just the three factors is arguably too short-sighted and simplistic
“the operation was a success but the patient died”
We may be ignoring many of the richer outcomes and avoiding a more systemic view
Perceptions change over time
Project success may not mean Product success
M B-S

54. Famous Project Failures?
Sydney Opera house
18 times over budget
yet world renown icon of Australia
Titanic Movie 1997
one of the most expensive films ever made at the time, lots of problems during shooting, went well over schedule and budget and the producers wanted to cut it by 1 hour yet it was the highest grossing movie ever (till Avatar) won critical acclaim and 11 Oscars.
M B-S

55. Famous Project Failures?
Millennium Dome hit financial difficulties and found itself needing an additional £179 million of National Lottery funding during 2000 but it attracted 5.5 million paying visitors: twice as many as any other UK visitor attraction in the previous year. The vast majority of visitors (87%) were satisfied with their day out and it opened its doors on time. But what now?
M B-S

56. Empire State Building
Objectives: in 1929, to be the highest building in New York (to beat the Chrysler Building)
It achieved that objective, was completed well ahead of schedule and well below budget. ... So a Success!
But for the same reason that anticipated costs had been halved - the great depression - rental rates were low and it was called the “Empty State Building”
It did not turn a profit until So a Failure!
Today it is 97% occupied, world famous and tallest
M B-S

57. Famous Project Failures?
The channel tunnel was initially budgeted at $7 billion but it entered service in 1994 with a price tag of over $13 billion.
In 2002 it was still burdened by $9.3 billion in debt
By 2004 it was £10billion....
But now carries millions of passengers...
M B-S

58. Famous Project Failures?
Thames Barrier
begun in 1974, opened in 1984 way over budget (£55m estimate / £435m actual) and time schedule
But what if it had not been built?
M B-S

59. Passport Agency IT system
Initial assessments were unflattering, long delays and costs over £12m (which included £16,000 on umbrellas for those waiting in queues)
Yet 4 years later the systems was working well with 99.5% of straightforward applications turned around within 10 days.
M B-S

60. Sauer (1988) comments: “Some systems never work.
Some are never made operational because they will be unacceptable to the user.
Some work but come in cripplingly over budget, very late or both.
Others are pared down while still others are literally forced into place in their host organisation, despite their being either inappropriate or unacceptable.
Some perform to specification but turn out to be so inflexible that maintenance and enhancement assume nightmarish proportions.
Others are thought to work but turn out not to.”
M B-S

61. Who says it is a failure? Views may differ
Did it produce unwelcome outputs for some?
“A human system fails if it does not succeed in doing what it was designed to do; or if it succeeds but leaves everyone wishing it had never tried.” (Vickers, 1981)
Unintended impacts and consequences may emerge
M B-S

62. Perceptions of Failure
Lyytinen and Herscheim (1987) view IS failures simply as problems that stakeholders perceive
But how many stakeholders do we ask and when is any problem too trivial to count?
Robinson (1994) suggests that a project’s success or failure is often determined by particular groups in the context of a political and social environment.
M B-S

63. Short and long term objectives
Peterson and Kim, 2000, suggest a 4 way classification:
System
User
Organisational
Strategic
Short-term Objectives
Reliable (bug-free) system
Satisfying user needs
Improving the effectiveness of business operations
Improving customer service
Long-term Objectives
Easily maintainable system
Improving productivity of managers
Generating operational benefits
Enabling cooperative partnership
M B-S

64. Awkward balances and compromises
“The management of IS projects is a difficult and complex task and there are no magic bullet solutions. IS projects are intrinsically uncertain. The management complexity arises from the necessity to deal simultaneously with several tensions:
Innovation versus risk
Learning versus control
The need for organisational change to deliver business benefit versus stakeholder resistance to change
Multiple stakeholder perceptions of the purpose of the project
The need to deliver value to the organisation versus managing effectively to satisfy time, quality and cost objectives
Managing detail and the big picture”. Roger Elvin
M B-S

65. Balanced Scorecard Performance Measurement
Financial
Customer
Internal
Business
Process
Vision and strategy
Learning and Growth
(Based on Norton and Kaplan original model)
M B-S

66. Failure - Success Continuums
We need to extend and amplify our evaluation to include impressions of:
Long-term effects, benefits and drawbacks, ROI, process improvements, efficiencies, cost reductions, income generated, money well spent?
Stakeholder acceptance, to what extent are the customers and sponsors pleased with results?
How successful was the team and the methodology that was used, lessons on approach and structure
M B-S

67. 1 Long-term effects – The investment
How has it changed the business?
Business and organisational benefits may not be realised quickly
It will often take time for the IT system to produce improvements
Unexpected drawbacks and advantages may appear once the system is in operation
Failed projects provide useful lessons
M B-S

68. Financial Services Firm BPR
A New Zealand firm instigated a consultancy led BPR project to improve customer service, reduce costs and improve the quality of work performed.
They implemented a standard ERP system (SAP)
Initially the project was regarded as a success, with 64% reduction in staff and projected savings of $2million per annum.
Two months later however the unintended consequence of the BPR project was that all in-house expertise had disappeared from the accounting group and no one could operate the SAP system properly.
M B-S

69. 2 Stakeholders - The Requirements
There are normally several parties interested in, dependant upon and maybe hostile to the implications and outcomes of any IT project.
It will often cause change and disrupt the status quo
Different stakeholders may hold differing views about what they want from (and fear about) the project.
Each group of stakeholders may judge the success of the project according to different feelings, beliefs and measures.
M B-S

70. M B-S

71. Stakeholder analysis and management
High
Keep informed,
Show consideration,
Consult
Key players,
Manage closely,
Partner & collaborate
Level of concern and interest in project
Keep channels open,
Minimal effort,
Monitor
Keep satisfied,
Meet their needs,
Involve
Low
High
Low
Power, influence, importance
Based on Gardener et al (1986)
M B-S

72. 3 Team and Approach - The Performance
Review effectiveness and productivity of techniques
Team performance – group size and composition, abilities, technical strengths and weaknesses, training, leadership, team work
Organisational structure
Project management procedures and methods
Administration – collecting, recording, monitoring and controlling
Problems, errors and correction effort
Appropriateness of methodology - Over 300 (Avison & Fitzgerald) maybe 1,000 (Jayaratna) different approaches identified
Commercial World Academic World
M B-S

73. Methodology selection & starting point
Unclear and undefined
Interpret
requirements
Messy
Terms of
Reference
Appreciateviews
Clear, Static
Decided and
defined
Known, understood, agreed view of needs, limited scope
Complex – a variety of attitudes and opinions on nature and size of problems
Situation or Context
M B-S

74. The Singer or the Song?
Who/what is to blame when the methodology seems to go wrong?
Perceptions of IT development success depends on a number of interdependent issues
The chosen approach / methodology - appropriateness, strengths and weaknesses, application and tailoring (the Play or concert)
The project manager and the team’s knowledge, experience, training, personalities, attitude and ability to use the methodology properly (the Interpreters – actors or orchestra)
The client and user, values, involvement from the start, requirements well defined, realistic expectations? (the Audience)
The context, appreciation of the richness of the project situation (the venue, background, economic climate, culture, history, politics, market conditions)
M B-S

75. Maybe graphics would help?
Business Stakeholder Method
Benefits Views &Team
Budget Time Scope
400%
300%
200%
100%
50%
25%
10%
Over
Original
estimate
Under
Short-term
Long-term
M B-S

76. Conclusions
This presentation looked at some costs, causes and examples of IT project failure.
It introduced a set of risk analysis tables intended for SME IT projects.
Having criticised current either/or fail/success measurement models it described a broader approach that may help to differentiate among, reach a richer appreciation of, and assist learning about projects.
M B-S

77. Questions?
M B-S