1. The spirit of the CIVITAS mobility measure evaluation
CIVITAS Forum Umeä
Transport & Mobility Leuven
Dirk Engels

2. Objectives of the CIVITAS 2020 Evaluation
Understand the process and impact of these mobility measures
Learn what works, what does not
And the reasons for this
What is a measure in the CIVITAS context?
A measure is a mobility related action implemented by a city (by the government or other stakeholders) e.g.:
New infrastructure
A new service
A new organization of the way to work
Activities to change awareness, acceptance and behaviour
Optimising the measures
Upscaling the measures in the city
Take-up the measures by other cities (transferability)
The spirit of the CIVITAS mobility measure evaluation• CIVITAS Forum Umeä •

3. The core of CIVITAS 2020 Evaluation
2 focuses of evaluation:
core CIVITAS 2020 EVALUATION
What is the impact of a measure or an integrated package of measures ?
What a the important elements in the implementation of the measure (also barriers and drivers) ?
How does the measure interact with other measures ?
PROJECT EVALUATION
To monitor and to check whether a project fulfills its objectives, whether the project has delivered the outputs promised in the proposal
To monitor and to check whether a Work Package fulfills its objectives
What is the effect of specific activities in the project ?
What is the effect of the project on the take-up of the measures ?
Examples: of project evaluation
Evaluation of the general dissemination activities of a IA or RIA project: was the dissemination of the project results effective to reach the target audience?
Evaluation of the innovation activities (e.g. the workpackage ‘Innovation’) to increase the quality of the measures: evaluation of whether the activities brought a measure on a higher level of innovation (a higher impact evaluated as part of the core CIVITAS 2020 Evaluation can be one aspect to assess this level)
Did a RIA approached the planned number of cities to validate the measure they developed?
The spirit of the CIVITAS mobility measure evaluation• CIVITAS Forum Umeä •

4. The core of CIVITAS 2020 Evaluation
2 focuses of evaluation:
Focus of the CIVITAS 2020 Evaluation
core CIVITAS 2020 EVALUATION
What is the impact of a measure or an integrated package of measures ?
What a the important elements in the implementation of the measure (also barriers and drivers) ?
How does the measure interact with other measures ?
PROJECT EVALUATION
To monitor and to check whether a project fulfills its objectives, whether the project has delivered the outputs promised in the proposal
To monitor and to check whether a Work Package fulfills its objectives
What is the effect of specific activities in the project ?
What is the effect of the project on the take-up of the measures ?
Examples: of project evaluation
Evaluation of the general dissemination activities of a IA or RIA project: was the dissemination of the project results effective to reach the target audience?
Evaluation of the innovation activities (e.g. the workpackage ‘Innovation’) to increase the quality of the measures: evaluation of whether the activities brought a measure on a higher level of innovation (a higher impact evaluated as part of the core CIVITAS 2020 Evaluation can be one aspect to assess this level)
Did a RIA approached the planned number of cities to validate the measure they developed?
The spirit of the CIVITAS mobility measure evaluation• CIVITAS Forum Umeä •

5. The key elements in the CIVITAS 2020 evaluation approach
before
after
design-planning
implementation
operational
IMPACT evaluation
Before and after measurements.
What is the impact of a measure or an integrated package of measures in the 6 CIVITAS Impact categories?
PROCESS evaluation
Barriers and drivers in the implementation of the measure?
What was the effect of specific activities in the implementation of the measure?
To facilitate the implementation?
To increase the envisaged impact?
To avoid/reduces not-wanted impacts?
Understanding of the impact? Why? What are key elements in it?
The
story behind the figures
The spirit of the CIVITAS mobility measure evaluation• CIVITAS Forum Umeä •

6. Impact evaluation – basic approach
CIVITAS IMPACT CATEGORIES
People-society
People-governance
Transport system
Energy
Economy
Environment
Impact categories - sub-categories – aspects - indicators
IMPACT CATEGORY
& sub-category
KEY IMPACT ASPECTS
INDICATORS
DESCRIPTION
SOCIETY-PEOPLE
Acceptance
Awareness
Key Indicator no. 1
Awareness level
The percentage of the target population with knowledge of a measure on account of provided information.
This indicator is used to assess the awareness of the general public or a particular target group on CIVITAS measures.
Acceptance/ Attitude
Key Indicator no. 2
Acceptance level
The percentage of the population who favourably receive or approve of the measure.
The spirit of the CIVITAS mobility measure evaluation• CIVITAS Forum Umeä •

7. Impact evaluation – basic approach
Impact categories - sub-categories – aspects - indicators
CIVITAS IMPACT CATEGORIES
People-society
People-governance
Transport system
Energy
Economy
Environment
IMPACT CATEGORY
& sub- category
KEY IMPACT ASPECTS
INDICATORS
DESCRIPTION
TRANSPORT SYSTEM
Walking
Opportunity for walking
Key Indicator no. 13
Quality of pedestrian infrastructure
Percentage of the total distance of the city's streets (including squares: the “distance” of a square is the sum of the length of its sides) with a good quality for walking on the total length of the city road network (excluding motorways)
Additional indicator
Quality of sidewalks
Calculation of the walkability index of all streets in an area describing in detail all aspects of quality for a sidewalk.
Number of pedestrians
Number of pedestrians
Number of pedestrians passing at set of reference points in area during specific hours a day or during the whole day.
Walking perception
Aditional indicator
Image on the walking conditions (subjective)
Attitude towards walking conditions based on the answers of a survey among citizens and visitors or pedestrians on the street.
The spirit of the CIVITAS mobility measure evaluation• CIVITAS Forum Umeä •

8. Impact evaluation Think also of …
A measure is not alone – we don’t work in a laboratory
Structuring measures in clusters for evaluation
Qualitative interpretation of impacts and influences of other measures
Qualitative interpretation of the importance of different measures and supporting activities
The spirit of the CIVITAS mobility measure evaluation• CIVITAS Forum Umeä •

9. City context Mobility figures For the city
For the CIVITAS demonstration area/corridor
The spirit of the CIVITAS mobility measure evaluation• CIVITAS Forum Umeä •

10. Process Evaluation Objectives
Evaluation of the processes of planning, implementation and operation, aiming to understand
why measures have succeeded or failed,
understanding the roles of supporting activities e.g. information, communication and participation (citizen engagement, stakeholder involvement)
Validation of the impact of the measures
Correct interpretation of the impact indicators
Understanding the importance of sub-measures in the overall impact of measures
Understanding the importance and influence (on the impact) of supporting activities
The spirit of the CIVITAS mobility measure evaluation• CIVITAS Forum Umeä •

11. Process Evaluation activities
Pre-analysis of measures (combined with preparation impact evaluation)
Responsible stakeholders (CIVITAS project partners and other important actors) and their roles
Specific target groups and/or effected part of the city or region
Identification of supporting activities to approach the target group(s)
The spirit of the CIVITAS mobility measure evaluation• CIVITAS Forum Umeä •

12. Process Evaluation activities
Monitoring and logging of activities, events, …
Discussion Measure Leaders – Site Coordinator – Local Evaluation Manager – (External Experts)
Learning History Sessions
Stakeholder survey or interview
Focus Group meeting
Expert (validation) meeting …
The spirit of the CIVITAS mobility measure evaluation• CIVITAS Forum Umeä •

13. Process Evaluation activities
Final reporting
Listing of important events for the measure
Identification of process barriers
Identification of process drivers
Reporting on activities to overcome barriers and/or to make use of the drivers to reach the measure objectives
Supporting activities: activities – penetration – quality – impact on the measure – drivers - barriers
Focus Group meeting ?
Discussion of different aspects of the measure
With specialists, with stakeholders
Expert meeting
The spirit of the CIVITAS mobility measure evaluation• CIVITAS Forum Umeä •

14. CIVITAS 2020 evaluation and more …
A crucial aspect in the evaluation approaches built up by all CIVITAS projects is that all extended and additional evaluation analyses should be constructed based on the core CIVITAS 2020 evaluation approach; using the same terminology, the same categorisation of impacts, the same type of characteristics of the implementation processes, etc., eventually completed with extra terms, categories, characteristics, indicators, etc.
The spirit of the CIVITAS mobility measure evaluation• CIVITAS Forum Umeä •

15. Thank you! Dirk Engels Gitte Van Den Bergh Contact Details
Transport & Mobility Leuven
Diestsesteenweg 57, 3010 Leuven
The spirit of the CIVITAS mobility measure evaluation• CIVITAS Forum Umeä •