1. Tools and Technologies in eParticipation: Insights from Project Evaluation Melanie Bicking and Maria A. Wimmer University of Koblenz (UKL) {bicking | wimmer}@uni-koblenz.de Fourth International Conference on Online Deliberation (OD2010) July 1st, 2010

2. Fourth International Conference on Online Deliberation (OD2010), July 1st, 2010 Table of Content Problem scope eParticipation Projects Evaluation Methodology Insights from Project Evaluation Outlook -2-

3. Fourth International Conference on Online Deliberation (OD2010), July 1st, 2010 Overall scope of eParticipation Project Evaluation Evaluate eParticipation project results (platforms) in terms of Processes supported Policies supported Topics discussed Tools and technologies used Assess success and potential impact of project results Investigate dialogue between stakeholders and the public over time and its outcomes Receive feedback from each interested stakeholder who is interested in using respective eParticipation platforms of the projects -3-

4. Fourth International Conference on Online Deliberation (OD2010), July 1st, 2010 Table of Content Problem scope eParticipation Projects Evaluation Methodology Insights from Project Evaluation Outlook -4-

5. Fourth International Conference on Online Deliberation (OD2010), July 1st, 2010 Assets of eParticipation projects to be assessed Tools and Technology: ICT deployed in the field of application and used for participation Processes supported: different stages within the legislation process Topics tackled: different topics processed in the legislation process Policies supported: strategic directions and policies addressed in the project -5-

6. Fourth International Conference on Online Deliberation (OD2010), July 1st, 2010 eParticipation Projects Evaluation Methodology Assets to be assessed Key Success and Impact Evaluation criteria Point of departure Platforms of eParticipation projects Processes supported Tools / Technologies deployed Topics discussed Policies supported Usability Perceived usefulness Policies met Interest Perceived Importance Appropriateness Appeal Attractiveness -6-

7. Fourth International Conference on Online Deliberation (OD2010), July 1st, 2010 Assessing the impact By addressing different target groups via Internal assessments External assessments By assessing The quality of reaching the target groups Whether sustainable interest has been reached -7-

8. Fourth International Conference on Online Deliberation (OD2010), July 1st, 2010 The two parts of evaluation Internal assessment External assessment Self- Assess- ment Structured: Questionnaire End user Question- naire Structured Experts in the field Semi- structured: SWOT-Analysis Peer Review Evaluation reports Reports Un- structured -8-

9. Fourth International Conference on Online Deliberation (OD2010), July 1st, 2010 Overall view on evaluation framework Assets to be assessed Key evaluation criteria Impact assessment features Addressing target groups through instruments of evaluation Point of departure What to evaluate? How to evaluate? Main target of evaluation: impact towards target groups Instruments of evaluation -9-

10. Fourth International Conference on Online Deliberation (OD2010), July 1st, 2010 Table of Content Problem scope eParticipation Projects Evaluation Methodology Insights from Project Evaluation Outlook -10-

11. Means to establish contacts with target group Fourth International Conference on Online Deliberation (OD2010), July 1st, 2010-11-

12. Means applied to establish contacts with the most impact Fourth International Conference on Online Deliberation (OD2010), July 1st, 2010-12-

13. Degree of innovation of tools and technologies deployed Fourth International Conference on Online Deliberation (OD2010), July 1st, 2010-13-

14. Usability of tools and technologies Fourth International Conference on Online Deliberation (OD2010), July 1st, 2010-14-

15. Web Content Accessibility Guidelines (WCAG) Fourth International Conference on Online Deliberation (OD2010), July 1st, 2010-15-

16. Overview of the eParticipation tools deployed and used by target users Fourth International Conference on Online Deliberation (OD2010), July 1st, 2010-16-

17. Lessons learnt and best practices Aspects hindering eParticipation projects to succeed Lack of support from elected representatives resulting not only but also in wanting acceptance of the citizens Past oriented thinking and acting, as well as culture of elimination of errors thereby remaining in tried and tested/trusted routines (this is in particularly true for the use of ICT) Confusing information, lack of transparency and accessibility Lack of sharing information with other projects to learn from their experiences or to create synergies Lessons learnt from executing eParticipation projects evaluation An evaluation of the impact is methodologically difficult Engaging end users in the assessment of the projects was problematic Better using mail surveys than online surveys Fourth International Conference on Online Deliberation (OD2010), July 1st, 2010-17-

18. Fourth International Conference on Online Deliberation (OD2010), July 1st, 2010 Table of Content Problem scope eParticipation Projects Evaluation Methodology Insights from Project Evaluation Outlook -18-

19. Toward further analysis The paper bases on parts of ongoing monitoring and evaluation of eParticipation projects funded under the eParticipation Preparatory Action of the EC Evaluation passes through an iterative process including internal and external assessments of the projects projects started and finished or will finish at different points in time At the moment the 2 nd round of expert evaluation is running Results will be available in a few months including a comparative analysis of 1 st and 2 nd round of Expert Evaluation the projects that started at the beginning of 2009 have now advanced to the point that their pilots are live and evaluation of their pilots will start soon involving Self-Assessment, Expert Evaluation, and Peer Review Fourth International Conference on Online Deliberation (OD2010), July 1st, 2010-19-

20. Expected outcomes At the end, a comprehensive and wide-ranging evaluation of the projects funded under the eParticipation Preparatory Action of the EC will be provided The study provide feedback on the single projects and calls, as well as on the eParticipation Preparatory Action itself From this, profound insights into eParticipation and the design and conduction of eParticipation projects are expected thereby providing helpful lessons learnt and recommendations for future eParticipation initiatives to the EC and other designated stakeholders. Fourth International Conference on Online Deliberation (OD2010), July 1st, 2010-20-

21. Fourth International Conference on Online Deliberation (OD2010), July 1st, 2010 Thank you for your attention! Melanie Bicking and Maria A. Wimmer University of Koblenz (UKL) {bicking | wimmer}@uni-koblenz.de -21-