The spirit of the CIVITAS mobility measure evaluation CIVITAS Forum Umeä Transport & Mobility Leuven Dirk Engels

Objectives of the CIVITAS 2020 Evaluation Understand the process and impact of these mobility measures Learn what works, what does not And the reasons for this What is a measure in the CIVITAS context? A measure is a mobility related action implemented by a city (by the government or other stakeholders) e.g.: New infrastructure A new service A new organization of the way to work Activities to change awareness, acceptance and behaviour   Optimising the measures Upscaling the measures in the city Take-up the measures by other cities (transferability) The spirit of the CIVITAS mobility measure evaluation• CIVITAS Forum Umeä •

The core of CIVITAS 2020 Evaluation 2 focuses of evaluation: core CIVITAS 2020 EVALUATION What is the impact of a measure or an integrated package of measures ? What a the important elements in the implementation of the measure (also barriers and drivers) ? How does the measure interact with other measures ? PROJECT EVALUATION To monitor and to check whether a project fulfills its objectives, whether the project has delivered the outputs promised in the proposal To monitor and to check whether a Work Package fulfills its objectives What is the effect of specific activities in the project ? What is the effect of the project on the take-up of the measures ? Examples: of project evaluation Evaluation of the general dissemination activities of a IA or RIA project: was the dissemination of the project results effective to reach the target audience? Evaluation of the innovation activities (e.g. the workpackage ‘Innovation’) to increase the quality of the measures: evaluation of whether the activities brought a measure on a higher level of innovation (a higher impact evaluated as part of the core CIVITAS 2020 Evaluation can be one aspect to assess this level) Did a RIA approached the planned number of cities to validate the measure they developed? The spirit of the CIVITAS mobility measure evaluation• CIVITAS Forum Umeä •

The core of CIVITAS 2020 Evaluation 2 focuses of evaluation: Focus of the CIVITAS 2020 Evaluation core CIVITAS 2020 EVALUATION What is the impact of a measure or an integrated package of measures ? What a the important elements in the implementation of the measure (also barriers and drivers) ? How does the measure interact with other measures ? PROJECT EVALUATION To monitor and to check whether a project fulfills its objectives, whether the project has delivered the outputs promised in the proposal To monitor and to check whether a Work Package fulfills its objectives What is the effect of specific activities in the project ? What is the effect of the project on the take-up of the measures ? Examples: of project evaluation Evaluation of the general dissemination activities of a IA or RIA project: was the dissemination of the project results effective to reach the target audience? Evaluation of the innovation activities (e.g. the workpackage ‘Innovation’) to increase the quality of the measures: evaluation of whether the activities brought a measure on a higher level of innovation (a higher impact evaluated as part of the core CIVITAS 2020 Evaluation can be one aspect to assess this level) Did a RIA approached the planned number of cities to validate the measure they developed? The spirit of the CIVITAS mobility measure evaluation• CIVITAS Forum Umeä •

The key elements in the CIVITAS 2020 evaluation approach before after design-planning implementation operational IMPACT evaluation Before and after measurements. What is the impact of a measure or an integrated package of measures in the 6 CIVITAS Impact categories? PROCESS evaluation Barriers and drivers in the implementation of the measure? What was the effect of specific activities in the implementation of the measure? To facilitate the implementation? To increase the envisaged impact? To avoid/reduces not-wanted impacts? Understanding of the impact? Why? What are key elements in it? The story behind the figures The spirit of the CIVITAS mobility measure evaluation• CIVITAS Forum Umeä •

Impact evaluation – basic approach CIVITAS IMPACT CATEGORIES People-society People-governance Transport system Energy Economy Environment Impact categories - sub-categories – aspects - indicators IMPACT CATEGORY & sub-category KEY IMPACT ASPECTS INDICATORS DESCRIPTION SOCIETY-PEOPLE Acceptance Awareness Key Indicator no. 1 Awareness level The percentage of the target population with knowledge of a measure on account of provided information. This indicator is used to assess the awareness of the general public or a particular target group on CIVITAS measures. Acceptance/ Attitude Key Indicator no. 2 Acceptance level The percentage of the population who favourably receive or approve of the measure. The spirit of the CIVITAS mobility measure evaluation• CIVITAS Forum Umeä •

Impact evaluation – basic approach Impact categories - sub-categories – aspects - indicators CIVITAS IMPACT CATEGORIES People-society People-governance Transport system Energy Economy Environment IMPACT CATEGORY & sub- category KEY IMPACT ASPECTS INDICATORS DESCRIPTION TRANSPORT SYSTEM Walking Opportunity for walking Key Indicator no. 13 Quality of pedestrian infrastructure Percentage of the total distance of the city's streets (including squares: the “distance” of a square is the sum of the length of its sides) with a good quality for walking on the total length of the city road network (excluding motorways) Additional indicator Quality of sidewalks Calculation of the walkability index of all streets in an area describing in detail all aspects of quality for a sidewalk. Number of pedestrians Number of pedestrians Number of pedestrians passing at set of reference points in area during specific hours a day or during the whole day. Walking perception Aditional indicator Image on the walking conditions (subjective)   Attitude towards walking conditions based on the answers of a survey among citizens and visitors or pedestrians on the street. The spirit of the CIVITAS mobility measure evaluation• CIVITAS Forum Umeä •

Impact evaluation Think also of … A measure is not alone – we don’t work in a laboratory Structuring measures in clusters for evaluation Qualitative interpretation of impacts and influences of other measures Qualitative interpretation of the importance of different measures and supporting activities The spirit of the CIVITAS mobility measure evaluation• CIVITAS Forum Umeä •

City context Mobility figures For the city For the CIVITAS demonstration area/corridor The spirit of the CIVITAS mobility measure evaluation• CIVITAS Forum Umeä •

Process Evaluation Objectives Evaluation of the processes of planning, implementation and operation, aiming to understand why measures have succeeded or failed, understanding the roles of supporting activities e.g. information, communication and participation (citizen engagement, stakeholder involvement) Validation of the impact of the measures Correct interpretation of the impact indicators Understanding the importance of sub-measures in the overall impact of measures Understanding the importance and influence (on the impact) of supporting activities The spirit of the CIVITAS mobility measure evaluation• CIVITAS Forum Umeä •

Process Evaluation activities Pre-analysis of measures (combined with preparation impact evaluation) Responsible stakeholders (CIVITAS project partners and other important actors) and their roles Specific target groups and/or effected part of the city or region Identification of supporting activities to approach the target group(s) The spirit of the CIVITAS mobility measure evaluation• CIVITAS Forum Umeä •

Process Evaluation activities Monitoring and logging of activities, events, … Discussion Measure Leaders – Site Coordinator – Local Evaluation Manager – (External Experts) Learning History Sessions Stakeholder survey or interview Focus Group meeting Expert (validation) meeting … The spirit of the CIVITAS mobility measure evaluation• CIVITAS Forum Umeä •

Process Evaluation activities Final reporting Listing of important events for the measure Identification of process barriers Identification of process drivers Reporting on activities to overcome barriers and/or to make use of the drivers to reach the measure objectives Supporting activities: activities – penetration – quality – impact on the measure – drivers - barriers Focus Group meeting ? Discussion of different aspects of the measure With specialists, with stakeholders Expert meeting The spirit of the CIVITAS mobility measure evaluation• CIVITAS Forum Umeä •

CIVITAS 2020 evaluation and more … A crucial aspect in the evaluation approaches built up by all CIVITAS projects is that all extended and additional evaluation analyses should be constructed based on the core CIVITAS 2020 evaluation approach; using the same terminology, the same categorisation of impacts, the same type of characteristics of the implementation processes, etc., eventually completed with extra terms, categories, characteristics, indicators, etc. The spirit of the CIVITAS mobility measure evaluation• CIVITAS Forum Umeä •

Thank you! Dirk Engels Gitte Van Den Bergh Contact Details Transport & Mobility Leuven Diestsesteenweg 57, 3010 Leuven dirk.engels@tmleuven.be http://www.civitas.eu The spirit of the CIVITAS mobility measure evaluation• CIVITAS Forum Umeä •