EVAL Practical Introduction May 2018 DEVCO IT Training

Agenda The EVAL module: context, overview Users, process flowchart, functionalities Latest and current developments Questions and Answers Resources

The e-Evaluation Tool & Library - EVAL Was launched in May 2016 Available for DG DEVCO, DG NEAR and FPI staff, in HQ and Delegations, with your "EU Login" Commission credentials Available through Firefox and Chrome in English and in French Since September 2016, it is to be used for all project and programme evaluations managed by DG DEVCO, DG NEAR and FPI As of April 2018, it is also used to encode Country sector level, Thematic and Budget support evaluations Currently EVAL covers more than 600 evaluations (217 on-going) On average, more than 150 evaluations per year are managed in EVAL

Users: Roles and Permissions Internals Evaluation Managers - DG DEVCO and DG NEAR staff, as well as designated FPI staff, can launch and manage an evaluation through EVAL. Evaluation Correspondents Head of Section, Head of Unit All DEVCO, NEAR and FPI staff for consultation Reference Group Members (internals and externals) Internal: HQ Thematic experts, International Aid / Cooperation Officers (Geo-coordinators) External with limited access: following the evaluation manager's decision, a partner country, civil society organisations Externals Contractors, Team Leader, Evaluation experts : with a limited and secure access during the evaluation process

Process flowchart Business Layer EVAL phases EM: Evaluation Manager ECTR: Evaluation Contractor Italics: step taken out of the IT system Business Layer Preparation phase Inception, Desk & Field phase Synthesis Phase Quality Assessment Dissemination & follow-up EM: "New Evaluation" "Plan Evaluation" Fill in the General tab: SAVE Stakeholders' tab: SAVE "Start TOR preparation" EM: Prepare ToR, have it appro-ved Management tab: Upload ToR Set as final "Next Phase" ECTR: Library tab: Consult QAG final report Download EM: Update web form 'QAG on Final report' Set as final ECTR: Create an EU login ECTR: CRIS and EVAL Library: download P/P documents Upload Inception/Desk/ Field Report Submit for Approval ECTR: Upload final report & executive summary (web form) Submit for Approval EM: Discuss with relevant EC services Fill in web form "Response of the services" Set as final "Next Phase" "Skip Fu+1 phase: Y or N?" EM: Discuss with relevant EC services Fill in web form "Follow-up one year later form" Set as final "Complete" EM: Management tab: Retrieve evaluation CTR Evaluation stakeholders tab: add contractor, team leader and experts: SAVE "Next phase" EM: Notification on first draft Fill in web form "QAG on draft report" Comment, approve / send back reports EM: Notification on report first version Share it with reference group Approve it or send back (Ares reg.) Add comments "Next phase" ECTR: If comments: revised draft of the final report/executive summary Submit for Approval EM: "Next Phase" ECTR: no more access Evaluation CN: Here the secretary sets the persons in charge before the evaluation can begin and as of this point the approvals are given by the chairperson not the VA Planned TOR Contract InceptionDesk Field Phases Synthesis QAG Response of Services Follow-up 1 year EVAL phases

EVAL is a knowledge-based management tool, acting as… An "evaluation coach": EVAL teaches you the step-by-step evaluation process; You can use previous evaluations as a starting point (e.g. by looking for ToR templates by sectors). A management tool: The module sends automated notifications to help meet deadlines; It facilitates communication and interactions between the different stakeholders; It enables documents sharing and interactions with other IT modules; It offers a tool for improving the quality of the final reports and executive summaries; It provides a communication system to follow-up on recommendations.

EVAL is also a tool that supports decision-making, by means of its library and search functionality: Giving quick access to findings and conclusions of all past and on-going evaluations sorted for instance by regions, sectors, keywords, EU Delegation; Highlighting the good practices and lessons learned from recent available evaluation results, to better define your next programmes or interventions.

An interactive platform for Contractors: Access to documentation of evaluated actions is made easier; Deadlines can be better followed based on the evaluation schedule; Use of templates and delivery of reports is done online; The approval/rejection of deliverables is automatically notified to the Contractors.

Recent developments New functionality: 'Operational Evaluation Plan'; Better communication with the contractor when assessing the quality of the Draft Final Report; New Evaluation levels (country-sector level, thematic, budget support); Improvements in the 'Search' functionality.

A. 'Operational Evaluation Plan' (OEP): An integrated planning system for evaluations; Flexibility: Possibility to update the OEP at any given time and to export the file with just one click; Classification: possibility to save the OEP in ARES; Information: Possibility to consult OEPs for all Units/EU Delegations.

B. 'Draft QAG': a tool that allows better quality on the assessment of the Final Report and its summary: Clarity: the QAG template has been changed: ease of use and judgment; Timing: when drafting the Final Report; Communication: a communication tool with the Contractor who can comment the assessment online; Transparency: the Contractor automatically receives the Final version of the QAG.

On-going developments Integration with OPSYS; ARES Integration; Evaluations possible for procedures not linked to a CRIS Contract (e.g. Strategic evaluations, EU Trust Funds); Multiple DAC sectors linked to a single evaluation; Library for "Evaluations done by partners". Suggestions?

Q&A

Resources and Support EVAL Knowledge base Illustrated user manuals by role Virtual classroom training courses (sign up on EU Learn) E-Learning videos (available on the Knowledge base) P/P Evaluation Methodological Approach on Capacity4Dev For IT issues: DEVCO APPLICATION SUPPORT NEAR APPLICATION SUPPORT or Business Manager for questions and/or suggestions For methodological support: DEVCO staff: helpdesk@evaluationsupport.eu NEAR staff: NEAR-EVAL-MONITORING@ec.europa.eu