1. ENTITIES FOR A UN SYSTEM EVALUATION FRAMEWORK 17th MEETING OF SENIOR FELLOWSHIP OFFICERS OF THE UNITED NATIONS SYSTEM AND HOST COUNTRY AGENCIES BY DAVIDE TONINI IMO, LONDON, 17-19 NOVEMBER 2008

2. JOINT INSPECTION UNIT (JIU) The Inspectors shall provide an independent view through Inspection and evaluation aimed at improving management and methods at achieving greater coordination between organizations. Without prejudice to the principle that external evaluation remains the responsibility of appropriate intergovernmental bodies, the Unit, with due regard to other responsibilities, may assist them in carrying out their responsibilities for external evaluation of programmes and activities. On its own initiative or at the request of the executive heads, the Unit may also advise organizations on their methods for internal evaluation, periodically assess these methods, and make ad hoc evaluations of programmes and activities.

3. UNITED NATIONS EVALUATION GROUP (UNEG) UNEG, as one of the main interagency bodies providing strategic guidance in evaluation, plays a very important role in strengthening the objectivity, effectiveness, and visibility of the evaluation function across the UN system by bringing together the professional units and individuals responsible for evaluation within the UN system.. UNEG is comprised of 43 member organizations and chaired by UNDP, aiming to provide a forum for the discussion of evaluation issues within the UN system and to promote simplification and harmonization of evaluation reporting practices among UNDP and the executing agencies.

4. JIU STANDARDS & GUIDELINES

5. THE GOAL OF EVALUATION Formative/Process Evaluation: Appraising processes of ongoing programme in order to propose modifications and/or improvements Impact/Outcome Evaluation: Finding out how well programme works in order to provide info for continuation/expansion/reduction decisions

6. CONCLUSION OF DESIGN PHASE Identify existing criteria related to expected outputs Identify relevant executive or legislative actions Understand interest/concern of Member States Clarify purpose of evaluation Understand programme objectives & legislative history Identify applicable rules & regulations for implementation of mandate Identify relevant and related internal controls

7. STRUCTURED DESIGN PLAN FOR EVALUATION Description & justification of sampling methodology details (if sampling used) Specification & description of measurement methods and instruments and estimate of their reliability & validity Justification that appropriate procedures & instruments have been specified

8. UNEG STANDARDS FOR EVALUATION IN THE UN SYSTEM

9. INSTITUTIONAL FRAMEWORK UN organizations should: have an adequate institutional framework develop an evaluation policy and regularly update it ensure submission of evaluation plans for approval ensure evaluation follow-up mechanisms

10. MANAGEMENT OF THE EVALUATION FUNCTION Head of Evaluation: has a lead role in ensuring that the evaluation function is fully operational and that evaluation work is conducted according to the highest professional standards is responsible for ensuring the preparation of evaluation guidelines should ensure that the evaluation function is dynamic, adapting to new developments and changing needs both within and outside the organization

11. COMPETENCIES Persons engaged in designing, conducting and managing evaluation activities: Should possess core evaluation competencies Evaluators should have: Relevant educational background, qualification and training in evaluation Relevant professional work experience Specific technical knowledge of methodology/approach applied

12. ETHICS Evaluators should: be sensitive to beliefs, manners and customs and act with integrity and honesty in their relationships with all stakeholders ensure that their contacts are characterized by respect protect anonymity and confidentiality of informants be responsible for their performance and products

13. CONDUCTING EVALUATIONS DESIGN (1) The evaluation design should: ensure relevant, timely, valid and reliable information include TORs that provide the purpose and description of process & product include clearly stated purpose and context ensure clear description of subject ensure realistic and achievable objectives & clearly defined scope

14. DESIGN (2) The evaluation design should: clearly specify the evaluation criteria ensure methodologies are sufficiently rigorous and ascertain a complete, fair and unbiased assessment assess cost effectiveness include considerations regarding the extent of UN systems commitment to the human rights- based approach incorporated in the design

15. PROCESS Relationship between evaluator and commissioner(s) must be characterized by mutual respect and trust Stakeholders should be consulted in the planning, design, conduct and follow-up A peer review/reference group composed of external experts may be useful

16. SELECTION OF TEAM Evaluations should be: conducted by well-qualified evaluation teams composed of gender balanced and geographically diverse teams IMPLEMENTATION conducted in a professional and ethical manner

17. REPORTING The final evaluation report should: contain evidence-based findings contain conclusions contain lessons and recommendations be free of irrelevant information be presented in an accessible and comprehensible manner FOLLOW-UP Evaluation requires an explicit response by the governing authorities and management addressed by its recommendations

18. EVALUATION REPORTS Key basic info in title and opening pages Executive Summary Clear description of subject evaluated Role & contributions of UN & stakeholders Purpose & context Explanation of evaluation criteria Explanation of evaluation objectives/scope Indication of extent to which gender issues & human rights considerations were incorporated Transparent description of applied methodology Complete description of stakeholders participation/contributions

19. EVALUATION REPORTS (2) Discussion of extent to which design included ethical safeguards Inputs, outputs, outcomes/impacts should be measured Reasons for accomplishments and difficulties of subject evaluated Conclusions substantiated by findings consistent with data & methodology Recommendations based on evidence & analysis, relevant & realistic, with priorities for action made clear Lessons should be generalized beyond immediate subject Complete and relevant annexes

20. Lunchtime!