1. NETLIPSE Infrastructure Project Assessment Tool Professor Hans-Rudolf Schalcher, ETH Zürich Amsterdam, 21 April 2009

2. Motivation (1) The European Union and (new) member states:  address the need for a Trans European Transport Network (TEN-T)  have already invested billions in the construction of several Large Infrastructure Projects (LIPs) to create the TEN-T. But, these organisations:  have limited possibilities for forecasting and monitoring the effectiveness of these projects;  face large delays and cost overruns on the supported projects and experience local opposition;  notice that knowledge exchange between Large Infrastructure Projects (LIPs) is scarce.

3. Motivation (2) These organisations have a need for: 1.Improvement of the current management and organisation of LIPs. 2.Insight in ‘the vitality’ of projects on certain moments, e.g. financing (gate review):  to have a reliable insight in risks and opportunities before deciding; and if decision is ‘go’: as a basis to manage risks and opportunities  to allocate budgets to the most vital projects. 3.Better insight in the progress of LIPs (risks, opportunities). 4.Benchmark projects. For this NETLIPSE will develop: 1.Knowledge exchange programme 2.Infrastructure Project Assessment Tool 3.Training programmes

4. IPAT: Infrastructure Project Assessment Tool

5. 1.Measurement  The IPAT is an assessment method of ‘the vitality’ of the project for the client/sponsor and project delivery organisation. When can the assessment take place? 1.On moments of ‘go/no go-decisions’ by sponsors: ex ante ‘gate review’. Gate reviews can indicate weaknesses and strengths in the organisation and management in advance. 2.During execution: monitoring. The ‘in between’ assessments can indicate the ability of the management to execute the planning and their adaptive abilities towards changes taking place during implementation. 3.Evaluation afterwards: ex post, in order to expand our knowledge about vital forms of organisation and management approaches of project planning and implementation. Ex post evaluation generates a comparative perspective on weaknesses and strengths on different implementation strategies. 4.Benchmarks. The assessments generates the ability to benchmark projects in different stages.

6. 2.Improvement  The IPAT can also be used as a method by the organisation itself to improve their management.  The organisation can improve themselves:  As a stimulus from the IPAT assessment, to improve themselves to have a better review next time  To use the IPAT as a basis for improvement of strategy, organisation and processes.  In that way the organisation will facilitate their learning process.  The organisation can use the IPAT as a ‘self assessment’. A self assessment can be an important first step in an external peer review.

7. Assessment process Effective assessments will be based on professional use of the IPAT: 1.In order to get comparable information, the IPAT will be based on a questionnaire. 2.Self assessment and facts & figures as a starting point. 3.Essential is the team of assessors,  with assessors that have a track record:  in implementation to give professional judgement  in project evaluation and analysis to give a scientific judgement.  Assessors will be facilitated by an IPAT-training. The training will focus on the IPAT itself and the use of it.

8. Focus of the IPAT  The external reviewers will not decide on which approach is most appropriate in a certain situation.  Based on past information we know that each new project will ask for unique answers, that suit the objectives and requirements of that specific project and environment.  What we will evaluate is the quality of thoughts and reflections of sponsors/clients and programme managers on how they deal with the four elements of their challenge and they are going to realise their project within scope, budget and time.

9. Control & Interaction Interaction Control  Mature management of LIPs asks for a hybrid approach combining control and interaction.

10. IPAT Research Framework (1) 3 Levels Objectives and Scope Hardware and software Project and Context Risks and Opportunities

11. IPAT Research Framework (2) Hardware of projects: Planning & Control finance, objectives, risk, contracts, technology Software of projects: Shareholder management, Team culture, HRM Hardware of contexts: Legal consents, Policy dynamics, Market dynamics Software of contexts: Stakeholder management, Quality of external relations, Societal relation mgt. (SRM) CommitmentControl AnticipationSupport

12. 11 Themes (draft) T 1 Objectives, Purpose and Business Case T 2 Functional Specification and Scope T 3 Interfaces T 4 Stakeholders T 5 Finances T 6 Legal T 7 Technology T 8 Knowledge T 9 Organisation and Management T10 Contracting (PPP) T11 Risks (Threats and Opportunities)

13. Milestones M 1Initiation of the project M 2Funding assembly M 3Official approval official planning authority M 4Start of execution M 5Completion M 6Start operation M 75 years after start of operation

14. M 1M 2M 3M 4M 5M 6M 7 T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T 9 T10 T11 Levels of Importance Prioritisation: Level 1: Minimal importance Level 2: Little importance Level 3: Medium importance Level 4: Important Level 5: Crucial

15. Target Groups  European Commission  Member States (Parliaments and Ministries)  Rail, road, waterway and airport authorities  Programme and project organisations  Financial institutes (e.g. EIB)  Insurance companies

16. Criteria IPAT 4 scientific criteria: 1.Practical Can we get the data? 2.Reliable Is the measurement consistent? 3.Validity Do we actually measure what we are supposed to measure? 4.Utility Is it useful, worth doing? 2 practical criteria: 5.Understandable: Is the outcome understandable for clients and project managers? 6.Presentable: Are we able to present the outcome unambiguously in an easy way?

17. Development of the IPAT  The IPAT needs acceptation of a sufficient number of (member) states.  We need to find a balance between effective development of the IPAT and involvement and commitment of (member) states.

18. Development of the IPAT Further steps:  Until September 2009:  Headlines of the IPAT  Concept manual, including questionnaires.  October 2009 – January 2010:  Test the IPAT on 3 to 4 projects.  February – April 2010:  Evaluation and sharpening the IPAT.  Complete and deliver the IPAT Assessors manual.  IPAT-assessors Course.