Presentation is loading. Please wait.

Presentation is loading. Please wait.

SPPMG Event 18 th February 2010 Why (IT) Projects Fail Malcolm Bronte-Stewart School of Computing University of the West of Scotland.

Similar presentations


Presentation on theme: "SPPMG Event 18 th February 2010 Why (IT) Projects Fail Malcolm Bronte-Stewart School of Computing University of the West of Scotland."— Presentation transcript:

1 SPPMG Event 18 th February 2010 Why (IT) Projects Fail Malcolm Bronte-Stewart School of Computing University of the West of Scotland

2 2 Objectives This presentation will: Discuss some of the costs and causes of technology project failure Challenge the way in which project failure is judged Suggest pre and post project evaluation techniques that may be useful to SMEs M B-S

3 3 Cobb’s Paradox “We know why projects fail, we know how to prevent their failure – so why do they still fail?” Martin Cobb, Treasury Board of Canada Secretariat in Unfinished Voyages, a follow to the CHAOS report, The Standish Group M B-S

4 Lyytinen and Robey (2000) “ Organisations fail to learn from their experience in systems development because of limits of organisational intelligence, disincentives for learning, organisational designs and educational barriers. Not only have many organisations failed to learn they also have learned to fail. Over time they accept and expect poor performance while creating organisational myths that perpetuate short-term optimisation.” 4M B-S

5 5 Project Failure IT projects tend to go over budget and / or over time schedule and / or do not meet expectations Research by the BCS, Royal Academy of Engineering, OASIG, Oxford University and others suggest that only about 15% to 30% are successful Conservative estimates put the cost of IT project failure at tens of billions of Euros across the EU Jaques, 2004 (142 billion euros in 2004 McManus & Wood-Harper, 2008 ) and around $500 billion wasted on IT purchases that fail to reach their objectives worldwide Feld & Stoddard, 2004 We go on making expensive mistakes M B-S

6 6

7 7

8 8

9 9

10 10M B-S

11 11M B-S

12 12M B-S

13 13M B-S

14 14M B-S

15 15M B-S

16 16 IT Development Project Costs $55 billion wasted on cancelled and over-run IT projects in the USA in 2002 (Standish, March 2003) “Public sector IT expenditure over past 12 months is in excess of £12.4 billion with a significant proportion at risk of being wasted” according to a House of Commons select committee (published July 2004) “This year (2006) organisations and governments will spend and estimated $1 trillion on IT hardware, software and services worldwide”. (Charette, IEEE) M B-S

17 IT project success statistics 75% of IT projects are challenged and 10% are abandoned (Oxford University, August 2003) Only 7% of the 1000 firms Critical Research surveyed thought they were using IT effectively and 75% of these firms think that their IT systems are not providing a return on investment Of 3,682 projects in 365 firms – 31% were cancelled, 53% had cost overruns and poor functionality and only 12% were on-time and budget (Johnson) Three quarters of large systems may be considered failures (Laudon & Laudon) 17M B-S

18 18 For example - some Headlines 2010 New HMRC inland revenue tax system producing wrong codes “ we’ve heard it all before” Labour accused of wasting 26bn on failed IT projects “stupendous incompetence” Rural payments scheme put out to grass “a display of scant regard for protecting public money” C-nomis offender management system “a master class in sloppy project management” 2008 Edinburgh Fringe Box-office system “weak”, “fundamentally flawed” “insufficient planning, lack of risk management, inadequate communications and no authorised business case” Strathclyde Police Computer System “a complete disaster” 2003 Libra the IT system for magistrates courts “One of the worst projects I have ever seen”. M B-S

19 “Prison IT system guilty of 'basic' project management failures” (2009) The £234m C-Nomis IT system for Prisons failed in almost every possible way. The NAO concluded that the technical complexity had been “significantly underestimated”. C-Nomis was treated as an IT project and not a business-change programme, project management was poor, and contracts with suppliers were weak. Edward Leigh, chairman of the Public Accounts Committee, said of C- Nomis that “kindergarten mistakes” had been repeated: “This Committee hears of troubled government projects all too frequently. But the litany of failings in this case are in a class of their own. All of this mess could have been avoided if good practice in project management had been followed.” M B-S19

20 IT project failures The Defence Information Infrastructure project to incorporate 150,000 terminals for 300,000 users at over 2000 defence sites is 18 months late and running more than £180m over budget. The official cost of the controversial national identity system has soared to over £5bn. The NHS IT scheme [NPfIT] was initially estimated to cost around £2.3bn. The most recent estimate is £12.7bn. 20M B-S

21 IT Project Failure Sainsbury’s, the supermarket giant, installed an automated fulfilment system to increase efficiency and streamline operations in The system ran into "horrendous" barcode-reading errors. Yet in 2005 the company claimed the system was operating as intended. Two years later, the entire project was scrapped, and Sainsbury's wrote off £150 million* in IT costs. * (One report claims it was a $526m investment) M B-S21

22 Avon County Council installed a new computer program to pay staff wages. The spree started in a small way paying a caretaker £75 an hour. Then it didn’t pay canteen workers at all for 7 weeks. Next it paid a janitor £2,600 for a week’s work. A deputy headmistress received her year’s annual salary once a month. Heads of department earned less than their assistants. Some people had more tax deducted in a week than they earned all year. By February 280 council employees were out on strike. 22M B-S (Stephen Pile)

23 23 The misguided torpedo assumption To prevent the possibility of a torpedo returning and exploding against the ship it was fired from a safety system built so that it would self- destruct if it turned 180 degrees Unfortunately, during trials, a torpedo jammed in its tube on board the ship, the test was abandoned and the ship turned round to go home …. BANG! M B-S

24 24 London Ambulance System New Software “Put Lives At Risk!” Main Findings of investigation –Inexperienced procurement team –Staff mistrust and opposition –Over-ambitious timetable –Price put before quality –Incomplete and untested software –Andersen Consulting report suppressed –Management failed to identify and solve problems –Users “did things wrong” “...a faulty system implemented with undue haste, by a management determined to impose its will...” M B-S

25 IT Project Research Jones, in a study of 250 large software projects at or above 10,000 function points* between 1995 and 2004 found that about 25 were successful, 50 had minor delays and 175 had major delays and overruns or had been terminated without completion. He notes that the main problems were not technical but were due to aspects of poor project management. *(equivalent to around1,250,000 statements in the C programming language) 25M B-S

26 Wider effects of IT project failure Perceptions of poor success rates and wasted resources affect decision making The more IT projects are seen to go wrong the more: – the public become cynical –staff learn to expect problems and delays –Developers wonder if a lot of their work is likely to be wasted effort –business people become nervous of technology change –those holding the purse strings may view IT as a worry and a poor return on investment M B-S26

27 27 What are the main causes of failure? 6 major research studies reach remarkably similar conclusions about the significant causes that threaten project success: –OASIG –Standish (CHAOS) –Select Committee –OGC / NAO –Schmidt, Lyytinen, Keil & Cule, –Fortune and White M B-S

28 28 OASIG Report (Clegg et al,1996) Outcomes from IT investment : –80% to 90% do not meet goals –80% delivered late and over budget –40% fail or are abandoned –Under 40% address training and skills enough –Less than 25% integrate business and technological objectives properly –Only 10% - 20% meet all success criteria M B-S

29 29 IT Project Failure (OASIG) Main reasons why IT projects fail –Management agenda is too limited most IT investments are technology led main investment motive is to only cut costs –This narrow focus on technical capabilities and efficiency goals means that inadequate attention is given to the human and organisational issues that can determine a project’s ultimate success. –Users don’t influence development enough –Senior managers don’t understand the links between technical and organisational change –Project management techniques and IT approaches are too technical –Companies fail to organise work or design jobs/roles properly M B-S

30 The Standish Group (1995) Of the $250 billion spent each year on IT application development 31.1% of projects cancelled before completion (failed). 52.7% of projects were challenged: over-budget, over the time estimate, and offers fewer features and functions than originally specified. 16.2% of software projects were successful - completed on time, on-budget and to specification, (but some of the largest of which have only approximately 42% of the originally-proposed features and functions). Recently these results have improved somewhat but remain disappointing. 30M B-S

31 31 Standish Group CHAOS Survey Results M B-S

32 32M B-S

33 CHAOS Project Success Factors 1. User involvement 2. Executive management support 3. Clear and firm statement of requirements 4. Proper planning and formal methodology 5. Realistic expectations 6. Minimised scope and smaller project milestones 7. Competent, skilled staff 8. Ownership 9. Clear vision and business objectives 10. Hard-working, focussed staff 11. Experienced project managers 12. Reliable estimates 33M B-S

34 Select Committee on Public Accounts Report on Improving the delivery of Government IT projects (Study of 25 projects 1990 to 2000) “Implementing IT systems has proved difficult” Frequent cases of delay, confusion and inconvenience for the citizen and poor value for money for the tax payer. 34M B-S

35 Select Committee Report conclusions 1.Decisions about IT are Business not technical, they have profound effects on customer service and must involve senior management. 2.End users and their business needs must be identified before the project commences so that clear objectives are taken into account fully during design and development 3.Scale and complexity – how ambitious? Can it be undertaken in one go? 4.Skilled Project Managers are essential 5.Sound methodologies and well conceived risk management are called for 6.Need for a high degree of professionalism in the definition, negotiation and management of IT contracts 7.Training can take up considerable resources but must address the needs of the users and of those maintaining the systems if the anticipated benefits are to be realised 8.Contingency plans should be in place 9.Organisations should learn lessons from project undertaken. A post implementation review should establish the extent to which they secured the proposed business benefits, user expectations and technical requirements. 35M B-S

36 OGC and NAO Best Practice 2005 Common causes of project failure 1. Lack of clear links between the project and the organisation's key strategic priorities, including agreed measures of success. 2. Lack of clear senior management and Ministerial ownership and leadership. 3. Lack of effective engagement with stakeholders. 4. Lack of skills and proven approach to project management and risk management. 5. Too little attention to breaking development and implementation into manageable steps. 6. Evaluation of proposals driven by initial price rather than long-term value for money (especially securing delivery of business benefits). 7. Lack of understanding of, and contact with the supply industry at senior levels in the organisation. 8. Lack of effective project team integration between clients, the supplier team and the supply chain. 36M B-S

37 Keil, Cule, Lyytinen & Schmidt (1998) These researchers recruited 3 panels of experienced project managers in different places – Finland, Hong Kong & U.S.A. – and asked them to identify and rank specific risk factors. –Lack of top management commitment to the project –Failure to gain user commitment –Misunderstanding the requirements –Lack of adequate user involvement –Failure to manage end user expectations –Changing scope / objections –Lack of required knowledge / skills in project personnel –Lack of frozen requirements –Introduction of new technology –Conflict between user departments 37M B-S

38 Fortune and White (2006) Fortune and White reviewed 63 publications that focus on project Critical Success Factors The top ten (in order of count of citations) of the 27 they quote are: –Support from senior management –Clear realistic objectives –Strong / detailed plan kept up to date –Good communications / feed back –User / client involvement –Skilled / suitably qualified / sufficient staff / team –Effective change management –Competent project manager –Strong business case / sound basis for project –Sufficient / well allocated resources 38M B-S

39 39 Select Committee OASIGNAO / OGC Keil et al CHAOS (Top Requirements:) Fortune and White (CSFs) IT projects are driven by business (not technical) decisions Many IT investments are seen only as technology led and aimed at cost cutting Evaluation of proposals driven by initial price rather than long term value, especially securing delivery of business benefits Misunderstanding user requirements Clear business objectives and Realistic expectations Strong business case / sound basis for project Insufficient involvement from users Users do not influence development enough Lack of effective engagement with stakeholders Lack of user involvement or commitment User involvement User / client involvement Clear objectives should be set from the start Need to set and review strategic objectives for change Lack of clear link between the project and the organisation’s key strategic priorities, including agreed measures of success Unclear and changing scope and objectives Clear and firm statement of requirements Clear realistic objectives Lack of commitment from senior management Management agenda is often too limited or narrow Lack of clear senior management ownership and leadership Lack of top management commitment Executive management support Support from senior management Large projects may be overambitious Inadequate attention is given to human and organisational issues Too little attention to breaking development and implementation into manageable steps Number of organisational units involved Minimised scope and smaller project milestones Project size, complexity, number of people involved, and duration Skilled project managers are essential to keep to time and budget and appropriate deliverables Senior managers do not understand the link between technical and organisational change Lack of skills and proven approach to project management and risk management Lack of required knowledge and effective project management skills Experienced project managers Competent project manager and effective change management Success depends on good risk analysis and sound methodologies Some project management techniques and IT approaches are too technical Lack of effective project management methodology Proper planning and formal methodology Correct choice / past experience of project management methodology / tools Contingency plans should be in place Must work to detailed implementation plans Not managing change properly Reliable estimates Strong / detailed plans kept up to date User and operator training must be planned and designed Failure to organise changes in work and roles properly Inadequate resources and skills to deliver the total delivery portfolio Inappropriate staffing and ill defined responsibilities Competent, skilled and focussed staff Skilled / suitably qualified and sufficient staff / team There should be a post- implementation review Introduction of new technology Standard software infrastructure Planned close down / review, acceptance of possible failure Need professionalism in the definition, negotiation and management of IT contracts Lack of understanding of, and contact with, the supply industry at senior levels in the organisation Ownership Good communication / feedback M B-S

40 40 Are IT Projects Different? IT and software-based projects have certain characteristics which make them different –specification –requirements creep –invisibility –complexity –flexibility –estimation –training / churn –changing technologies –communication and conflict –resistance to change –testing M B-S

41 Risk Analysis Consideration of these recognised common causes of project failure, taken with reports, case studies, project management experience and previous research, build a picture of the variety and scope of important risk factors. Instead of expecting individuals to guess the likely risks for each project we can specify a standard, comprehensive collection of a variety of aspects and issues that threaten successful IT project outcomes. M B-S41

42 42 Risk Analysis Technique A technique has been developed that is intended to help project managers and others to gauge and take account of these risk factors. It requires little training and gives a structured list of relevant questions that invite the user to consider the extent to which each element threatens the project under review. These can be separated into 3 (or 4) groups: Some concern the firm itself and are slow to change Some concern the project in question Some concern the details of the options being assessed and compared To create a series of Risk Tables M B-S This Organisation This Project This Option (This Environment)

43 43M B-S

44 44M B-S

45 45M B-S

46 46 Risk or Threat Level Each factor may be awarded its own weighting (or ignored) according to its perceived significance Multiplying the weights by the scores gives a total value for each factor and these can be summed to arrive at the overall value Comparing your risk analysis values with those of others provides a focus and common language to bring out concerns, specific issues, enlightened evaluation and assessment of likely dangers and potential problems The technique may become more valuable the more it is used as experience of the results and the model’s inadequacies develops M B-S

47 47 Other issues There are at least two other aspects that might usefully be taken into account: 1.The wider environment, trends and market conditions, ie the level above the first table that is outside the control of the organisation in question but influences project choice 2.The degree of uncertainty in the estimates of risk factors So these tables provide an IT project risk analysis ready- reckoner in a standardised and quantifiable form that can be applied before the project begins, or during a feasibility study But how do we judge the finished project? M B-S

48 Measuring Success Three factors are used to decide if a project is a success or a failure soon after its closure: –Time / on schedule? –Money / within budget? –Delivered product / to specification? Fail any one of these tests and it may be labelled a failed project (even though it passes the other two) 48M B-S } Yes or No

49 49 Three factors – “The Iron Triangle” Quality / Scope / Functionality CostTime Good? Quick? Cheap? Did the project come in on or under budget and schedule? AND did the project meet customer requirements? M B-S

50 Failures? What about the extent and type of the failure? Some are abandoned, some are incomplete but in use Some just overrun budget or time, others are total disasters The project management process may be a mess AND / OR The delivered product may be useless or unwanted How do we differentiate and categorise the relative success / failure of IT projects? M B-S50

51 Maybe graphics would help? M B-S51 Over Under 400% 300% 200% 100% 50% 25% 10% Budget Time Scope Original estimate

52 Failure? Was failure an event or a process – did it suddenly appear or take time to become evident? Was / Were there: –Undisclosed needs and requirements creep? –Over-optimistic budget and timescale? –Weak methodology, changing team and power struggles? Does blame lie more with the clients who expected too much, or the planner who set the parameters, or the project manager who had to please the clients and abide by the parameters? How can we tell? M B-S52

53 So What is Project Failure? Should a project be regarded as a failure if it 1.overruns budget or timescale slightly but the customer is happy with the results? 2.Comes in under budget and time but gains little acceptance Judging overall success by using just the three factors is arguably too short-sighted and simplistic “the operation was a success but the patient died” We may be ignoring many of the richer outcomes and avoiding a more systemic view Perceptions change over time Project success may not mean Product success 53M B-S

54 Famous Project Failures? Sydney Opera house 18 times over budget yet world renown icon of Australia Titanic Movie 1997 one of the most expensive films ever made at the time, lots of problems during shooting, went well over schedule and budget and the producers wanted to cut it by 1 hour yet it was the highest grossing movie ever (till Avatar) won critical acclaim and 11 Oscars. M B-S54

55 Famous Project Failures? Millennium Dome hit financial difficulties and found itself needing an additional £179 million of National Lottery funding during 2000 but it attracted 5.5 million paying visitors: twice as many as any other UK visitor attraction in the previous year. The vast majority of visitors (87%) were satisfied with their day out and it opened its doors on time. But what now? M B-S55

56 Empire State Building Objectives: in 1929, to be the highest building in New York (to beat the Chrysler Building) It achieved that objective, was completed well ahead of schedule and well below budget.... So a Success! But for the same reason that anticipated costs had been halved - the great depression - rental rates were low and it was called the “Empty State Building” It did not turn a profit until So a Failure! Today it is 97% occupied, world famous and tallest 56M B-S

57 Famous Project Failures? The channel tunnel was initially budgeted at $7 billion but it entered service in 1994 with a price tag of over $13 billion. In 2002 it was still burdened by $9.3 billion in debt By 2004 it was £10billion.... But now carries millions of passengers... M B-S57

58 Famous Project Failures? Thames Barrier begun in 1974, opened in 1984 way over budget (£55m estimate / £435m actual) and time schedule But what if it had not been built? M B-S58

59 Passport Agency IT system Initial assessments were unflattering, long delays and costs over £12m (which included £16,000 on umbrellas for those waiting in queues) Yet 4 years later the systems was working well with 99.5% of straightforward applications turned around within 10 days. M B-S59

60 Sauer (1988) comments: “Some systems never work. Some are never made operational because they will be unacceptable to the user. Some work but come in cripplingly over budget, very late or both. Others are pared down while still others are literally forced into place in their host organisation, despite their being either inappropriate or unacceptable. Some perform to specification but turn out to be so inflexible that maintenance and enhancement assume nightmarish proportions. Others are thought to work but turn out not to.” M B-S60

61 Who says it is a failure? Views may differ Did it produce unwelcome outputs for some? “A human system fails if it does not succeed in doing what it was designed to do; or if it succeeds but leaves everyone wishing it had never tried.” (Vickers, 1981) Unintended impacts and consequences may emerge M B-S61

62 Perceptions of Failure Lyytinen and Herscheim (1987) view IS failures simply as problems that stakeholders perceive –But how many stakeholders do we ask and when is any problem too trivial to count? Robinson (1994) suggests that a project’s success or failure is often determined by particular groups in the context of a political and social environment. M B-S62

63 Short and long term objectives M B-S63 SystemUserOrganisationalStrategic Short-term Objectives Reliable (bug- free) system Satisfying user needs Improving the effectiveness of business operations Improving customer service Long-term Objectives Easily maintainable system Improving productivity of managers Generating operational benefits Enabling cooperative partnership Peterson and Kim, 2000, suggest a 4 way classification:

64 Awkward balances and compromises “The management of IS projects is a difficult and complex task and there are no magic bullet solutions. IS projects are intrinsically uncertain. The management complexity arises from the necessity to deal simultaneously with several tensions: –Innovation versus risk –Learning versus control –The need for organisational change to deliver business benefit versus stakeholder resistance to change –Multiple stakeholder perceptions of the purpose of the project –The need to deliver value to the organisation versus managing effectively to satisfy time, quality and cost objectives –Managing detail and the big picture”. Roger Elvin M B-S64

65 Balanced Scorecard Performance Measurement M B-S65 Vision and strategy Internal Business Process Customer Financial Learning and Growth (Based on Norton and Kaplan original model)

66 Failure - Success Continuums We need to extend and amplify our evaluation to include impressions of: 1.Long-term effects, benefits and drawbacks, ROI, process improvements, efficiencies, cost reductions, income generated, money well spent? 2.Stakeholder acceptance, to what extent are the customers and sponsors pleased with results? 3.How successful was the team and the methodology that was used, lessons on approach and structure M B-S66

67 1 Long-term effects – The investment How has it changed the business? Business and organisational benefits may not be realised quickly It will often take time for the IT system to produce improvements Unexpected drawbacks and advantages may appear once the system is in operation Failed projects provide useful lessons M B-S67

68 Financial Services Firm BPR A New Zealand firm instigated a consultancy led BPR project to improve customer service, reduce costs and improve the quality of work performed. They implemented a standard ERP system (SAP) Initially the project was regarded as a success, with 64% reduction in staff and projected savings of $2million per annum. Two months later however the unintended consequence of the BPR project was that all in-house expertise had disappeared from the accounting group and no one could operate the SAP system properly. 68M B-S

69 69 2 Stakeholders - The Requirements There are normally several parties interested in, dependant upon and maybe hostile to the implications and outcomes of any IT project. It will often cause change and disrupt the status quo Different stakeholders may hold differing views about what they want from (and fear about) the project. Each group of stakeholders may judge the success of the project according to different feelings, beliefs and measures. M B-S

70 70M B-S

71 Stakeholder analysis and management Level of concern and interest in project Power, influence, importance Low High Keep informed, Show consideration, Consult Keep channels open, Minimal effort, Monitor Keep satisfied, Meet their needs, Involve Key players, Manage closely, Partner & collaborate Based on Gardener et al (1986) 71M B-S

72 3 Team and Approach - The Performance Review effectiveness and productivity of techniques Team performance – group size and composition, abilities, technical strengths and weaknesses, training, leadership, team work Organisational structure Project management procedures and methods Administration – collecting, recording, monitoring and controlling Problems, errors and correction effort Appropriateness of methodology - Over 300 (Avison & Fitzgerald) maybe 1,000 (Jayaratna) different approaches identified Commercial World Academic World M B-S72

73 73 Terms of Reference Unclear and undefined Known, understood, agreed view of needs, limited scope Decided and defined Complex – a variety of attitudes and opinions on nature and size of problems Situation or Context Methodology selection & starting point Clear, Static Messy Appreciate views Interpret requirements M B-S

74 74 The Singer or the Song? Who/what is to blame when the methodology seems to go wrong? Perceptions of IT development success depends on a number of interdependent issues –The chosen approach / methodology - appropriateness, strengths and weaknesses, application and tailoring (the Play or concert) –The project manager and the team’s knowledge, experience, training, personalities, attitude and ability to use the methodology properly (the Interpreters – actors or orchestra) –The client and user, values, involvement from the start, requirements well defined, realistic expectations? (the Audience) –The context, appreciation of the richness of the project situation (the venue, background, economic climate, culture, history, politics, market conditions) M B-S

75 Maybe graphics would help? M B-S75 Over Under 400% 300% 200% 100% 50% 25% 10% Budget Time Scope Long-termShort-term Business Stakeholder Method Benefits Views &Team Original estimate

76 Conclusions This presentation looked at some costs, causes and examples of IT project failure. It introduced a set of risk analysis tables intended for SME IT projects. Having criticised current either/or fail/success measurement models it described a broader approach that may help to differentiate among, reach a richer appreciation of, and assist learning about projects. M B-S76

77 77 Questions? M B-S


Download ppt "SPPMG Event 18 th February 2010 Why (IT) Projects Fail Malcolm Bronte-Stewart School of Computing University of the West of Scotland."

Similar presentations


Ads by Google