1. Current reality in Norwegian public investment projects Ole Jonny Klakegg Research Director Concept Research Programme www.concept.ntnu.no

2. What do we need to invest in? Source: State of the Nation 2010 (RIF – Association of consulting engineers) Municipal buildings Health buildings Railway Airports State roads Municipal roads Water supply Waste water Waste disposal Energy production Energy distribution SUM: Current develop- ment Securing the future Current state Value [Million NOK]

3. Research into Current Status 88 Projects subject to external quality assurance in Norway (2001-2007). Sample includes: + 29 Road and railway projects + 12 Building projects + 10 Defence projects = 51 Projects in total (58% of population) Do we define and design major public investment projects well? Ring 3 Ulven-Sinsen Double track Asker-Sandvika Leopard 2 tanks New Opera-house Oslo

4. Well Defined? Category ABCDEF Characteri sation Well defined No purpose No effect No content Parallel effects Top heavy Symbolic represent ation Number of projects 15732212 Share of projects 29,4%13,7%5,9%3,9%41,2%3,9% No! There is a lack of fundamental logic Result of analysis: fundamental logic in defining project objectives. N = 51 projects. Source: O J Klakegg, PhD dissertation 2010

5. Well Designed? No! They are not up to best practice Best practice reference: Logical Framework Approach. The number of projects meeting the best practice criteria. Total of 51 projects evaluated. Source: O J Klakegg, PhD dissertation 2010

6. Governance of Projects The Norwegian State needs to invest in physical infrastructure We have financial ability But we struggle to find economically viable projects (few people to share the cost) And we struggle to define and design the projects well

7. How do our projects perform? Objectives achievedNo. of ProjectsShare of Projects Within cost frame2295% Not significantly beyond approved expected cost 2191% Without significant delay2191% Quality as expected2295% Result of 23 major Norwegian public investment projects finished after going through QA2. They reach objectives and deliver within approved frames. We see a high degree of professionalism in project management in all agencies with projects in this sample. Source: Concept Research Programme 2010

8. Actual cost compared to approved expected cost Results placed on the timeline according to finishing date of the project. Average: -4%

9. Mandate of the research team Purpose: Check immediate results from the projects Compare delivered to what was predicted at QA2 and the decision to finance Support learning/improvement Scope: Project-/execution perspective

10. Ministry and QA-consultant Ministry QA-consultant TransportCultureKnow- ledge Finance and customs DefenceWork- force Dovre511 Holte421 Metier/PTL2121 Terramar41 No. of QA2’s1512151 One road project has been through QA2 twice with the same QA-consultant, One defence project has been through QA2 twice with different QA-consultants.

11. Agency and type of project Agency Type of project NPRANNRAStatsbyggForsvars- bygg FLONAV Transport infrastructure 121 Buildings 41 Materiel ICT-system 131 No. of projects 1224131 Several projects include more than one element (e.g. both ICT-system and materiel) and could be classified differently. NPRA = Norwegian Public Roads Administration NNRA = Norwegian National Railway Administration

12. List of projects NrAgencyProject 1NPRAE6 Skjerdingstad – Jaktøyen 2E6 Riksgrensen – Svingenskogen 3E6 Klemetsrud – Assurtjern 4 Rv 653 Eiksundsambandet 5 E18 Høvik – Frydenhaug 6 Ringvei Nord, Tønsberg 7 E10 Lofast 8 E18 Kopstad – Gulli 9 Rv 2 Kløfta – Nybakk 10 E6 Åsgård – Halmstad 11 E6 Svingenskogen – Åsgård 12 E18 Momarken – Sekkelsten NrAgencyProject 13NNRA Dobbeltspor Sandvika-Asker 14 GSM-R samband for jernbanen 15Statsbygg Nytt operahus 16 Høgskolen i Østfold 17 Nybygg Svalbard forskningspark 18 Tollområde Svinesund 19FB ISL-prosjektet 20FLO FIS Basis 21 GOLF, leveranseprosjekt 1 22 LOS ØP 23NAV NAV-IKT

13. Sources of information QA2-reports (from independent QA consultants) Public documents (from Ministries and Agencies) Copy of original documents from execution Final report from the project management/agency – Ordinary reports from agencies – Specially compiled for the Concept research programme Interviews with key personnel Other sources

14. The execution perspective CategoryAssessment (of the project as a whole)No. of projects Very successful All objectives achieved. Appears as specially positive and/or has established new best practice 6 SuccessfulAll immediate objectives achieved. Well executed, active management, good handling of communication and conflicts. No significant problems. 12 Average successful As expected, some objectives are achieved, others not. Partly well executed, but weak on certain points. Moderate problems with limited consequence. 5 Less successful Most of the immediate objectives are only partly achieved. Management was passive. Major conflicts during execution or after. 0 Not successful A lot of things has gone wrong, at the start and during execution. The objectives are achieved only to a limited degree. 0

15. Themes in our analysis Governance (follow-up by the owner – the Ministry) Agency planning and cost estimation in the front-end Goal achievement Management and organizing (people) Project control (systems) Contract management Uncertainty management

16. Marking scale for each theme in our analysis CategoryAssessment of system/performance Green (OK)Procedures according to guidelines and best practice standards. Performance according to procedures. Procedures and praxis can be used as basis for improvement. Amber (warning)Procedures not in accordance with guidelines, or is missing. Performance mainly according to procedures, but some unwanted cultures or inadequate praxis is observed. Corrective measures should be implemented. Red (stop)The project lacks important procedures normally part of approved practice. Performance not according to approved procedures/ guidelines or is unacceptable for safety, ethical or other reasons. Procedures and/or praxis needs to change ASAP (if not already changed after this project was finished).

17. Result of detailed assessment NoProjectGovernanceAgency plan. cost. estim. Goal achievement Management organizing Project- control Contract management Uncertainty management 1E6 Skjerdingstad – Jaktøyen 2E6 Riksgrens. – Svingensk. 3E6 Klemetsrud – Assurtjern 4 Rv 653 Eiksundsambandet 5 E18 Høvik – Frydenhaug 6 Ringvei Nord, Tønsberg 7 E10 Lofast 8 E18 Kopstad – Gulli 9 Rv 2 Kløfta – Nybakk 10 E6 Åsgård – Halmstad 11 E6 Svingenskogen – Åsgård 12 E18 Momarken – Sekkelsten 13 Dobbeltspor Sandvika-Asker 14 GSM-R samband for jernbanen 15 Nytt operahus 16 Høgskolen i Østfold 17 Nybygg Svalbard forskningsp. 18 Tollområde Svinesund 19 ISL-prosjektet (nybygg FD) 20 FIS Basis 21 GOLF, leveranseprosjekt 1 22 LOS ØP (økonomiprosjektet) 23 NAV-IKT Roads Railw Building ICT OK Warning Stop

18. Governance Project typeObserved potential for improvement Road projects:Decisions / clarification of financing, lack of predictability concerning cost. Railway projects:Management of contingency not working as expected. Building projects:Unclear organization on the owner side, partly unwanted influence in the project’s decision making processes. ICT-projects:Governance tend to contribute to hide real costs. All types of projects: No agencies are good at assisting transfer of experience and learning between projects. Lack of criteria for accepting use of contingency, too much variation i praxis.

19. Agency planning and cost estimation in the front-end Project typeObserved potential for improvement Road projects:Unrealistic (high) cost estimates. Some cases of unmature plans and unrealistic premises, preliminary studies not adequate. Railway projects:Unrealistic (low) cost estimates, estimating errors. Building projects:Unrealistic cost estimates (high and low), partly inadequate preliminary studies. ICT-projects:Unrealistic assumptions for planning. All types of projects: Significant need for improvement of basis for planning and cost estimating.

20. Goal achievement Project typeObserved potential for improvement Road projects:Large degree of goal achievement, some projects did not reach all objectives. Railway projects:Large degree of goal achievement. Building projects:Large degree of goal achievement, some minor exceptions. ICT-projects:Large degree of goal achievement. All types of projects: Goal formulations have large potential for improvement, but these projects where established early, before this was focussed. Important! Only the execution perspective is reviewed here. Objectives on sosiety level and for the users are not an issue in this study.

21. Management and organizing Project typeObserved potential for improvement Road projects:Lack of resources/Low on staffing in several projects. Railway projects:Good project support, this has high priority. Building projects:Strong project managers put their strong mark on and dominate their projects. ICT-projects:These projects require special skills on behalf of the project managers and their project team, close relations between the parties has potential for challenges and good results. Traditional project thinking is challenged. All types of projects: Projects either in a strong and independent position, or in a close and good collaboration to the mother organisation seem to succeed. Projects in a middle position struggle.

22. Project management (system) Project typeObserved potential for improvement Road projects:Not bad, but signs of unclear guidelines, especially concerning change management. Railway projects:Adequate project control. Building projects:Adequate project control, but depends on individuals. ICT-projects:Adequate project control. Defence projects:Not bad, but show signs of some problems with progress control. All types of projects: Basis-systems in all agencies was not adequate at the start in the early 2000’s, but we see significant improvement and development effort in several of these projects.

23. Contract management Project typeObserved potential for improvement Road projects:Varying praxis and partly inadequate contracts for design work. Partly inadequate follow-up (due to capacity problems). Bad experience with horizontal contract interfaces. Railway projects:Partly inadequate contracts for design work. Building projects:Partly inadequate contracts for design work. ICT-projects:Variation in the quality of design basis for follow-up, and the actual follow-up of contracts – from very good to very bad. All types of projects: The totality of available options for contract strategies not exploited, partly because the contract strategy is considered too late. Praxis vary a lot, it may be purposeful to consider strengthening the support from the agency.

24. Uncertainty management Project typeObserved potential for improvement Road projects:System for uncertainty management not adequate, some projects show lack of understanding the phenomenon. Railway projects:Good systems are under development and implementation. Building projects:Variable praxis and some lack of adequate systems in this sample. Partly lack of ability to identify and act on early warning signs, but also examples showing the opposite. ICT-projects:Partly good systems and praxis, partly some lack of ability to identify and act on early warning signs. All types of projects: Uncertainty management is not implemented. Lack of clear procedures for handling contingencies. We know new systems and praxis is developed and implemented in later projects.

25. Uncertainty analyses in QA2 The following slides show the result of an assessment of an extended sample of projects (N=75). www.concept.ntnu.no

26. Uncertainty and size of project Size of project (million NOK) Number of projects Standard deviation in % Average contingency in % 300 – 750 2810,8 % 9,6 % 750 – 1 500 329,8 % 9,3 % 1 500 and more 1510,7 % 10,2 % An analysis of 75 QA2-projects indicates that the size (measured in cost) has little influence on the degree of uncertainty and size of the contingency as perceived by the QA consultants.

27. Uncertainty and Contingency 75 QA2-analyses show the uncertainty of cost as average standard deviation in %. Average contingency for uncertainty recommended by QA-consultant measured as % of total cost. 33 13 7 4 4 14 No. of projects Other Railroad ICT Road Building Equipment Other Railroad ICT Road Building Equipment

28. Assessment of uncertainty Different QA-consultants assessment of uncertainty (measured as average standard deviation in % of total cost).

29. Probability for exceeding cost frame The cost frame is calculated as upper limit (defined as P85) for cost minus the value of potential cuts identified. Cuts are actually subtracted in 32 of 75 projects. (The diagram includes only projects from the period after this praxis was established.)

30. Uncertainty analysis CategoryShort description User Issues like user demands and user involvement during the project process. Transfer to operations Challenges concerned with maintenance, testing and implementation of results, typical for ICT-systems. Organization Issues like quality and skills in the project organization, capacity, competence, continuity in the organization. Organization structure, responsibility, roles etc. Leadership Leadership, kcmmunication and motivation. Framework conditions Framework conditions given by the project owner especially concerning allocation of resources, necessary permissions and premises for the project organization. Technical issues/ cost elements Technical challenges related to large, complex elements in the projects (large bridges, tunnels etc.) and cost variation in these elements. I addition technological development and technical solutions. CategoryShort description Contract strategies and acquisition Aspects of acquisition processes ans procedures, contract models and compensation formats. Project control Structure, systems, routines, especially in connection with cost-, progress-, and quality control, systems for planning and management of the project, change control etc. Technical -/ System interfaces Interfaces between projects and project elements. Stakeholders Issues concerning stakeholders and environment. Given by nature Geotechnical and geological challenges, whether conditions etc. SHA and environment Safety, Health and Environment, Archeological discovery etc. Market Market conditions and development of the market. Scope management Scope creep and changes in quality standards. Categories for ”normalizing” the analysis results

31. Focussing the most important? Almost 70% of the uncertainty elements on ”top three” represent project internal issues (controlled by the project itself). Based on 237 identified uncertainty elements in 75 project analyses.

32. Observed in real life Internal issues in projects (controllable by the project) is important, but the external issues and interfaces are more important in real life than acknowledged in analyses. Based on observations in 23 finished projects.

33. Analyses and Reality Some issues dominate in analyses (based on 75 project analyses). Observations in real life show that reality is more complex (based on 23 finished projects).

34. Do the type of project matter? Uncertainty elements in different types of projects. Included here is only categories with significant differences between project types.

35. Concept Symposium 2010

36. Highlights of the Symposium Decision makers, doers and advisors: Joining forces to enhance utility of investments 32 international presenters 100 participants By invitation from the Norwegian Ministry of Finance Want an invitation? Look at: www.concept.ntnu.no