1. Mexico 8th Meeting of the Steering Committee of INTOSAI Committee on
Knowledge Sharing and Knowledge Services
Mexico
7-9 September 2016
Cour des Comptes - Evaluation Working Group - 15/16 october 2015

2. Cour des Comptes - Evaluation Working Group – 7-9 September 2016
OUR AMBITION
Long term : «help SAIs which practice evaluation or wish to do so by providing them with methodological tools and practical recommendations » (creation of EWG in 1992) Immediate : approval by KSC Steering Committe of the endorsement version of the guidelines
Cour des Comptes - Evaluation Working Group – 7-9 September 2016

3. Cour des Comptes - Evaluation Working Group – 7-9 September 2016
I – Where we are
Achievements
Major issues
II – Next steps
Cour des Comptes - Evaluation Working Group – 7-9 September 2016

4. Cour des Comptes - Evaluation Working Group - 15/16 october 2015
I. Where we are
Cour des Comptes - Evaluation Working Group - 15/16 october 2015

5. Cour des Comptes - Evaluation Working Group – 7-9 September 2016
Achievements (1/3)
Sept 2012– 4th KSC
Decision of the 4th KSC in Sept. 2012
To aim at a definition of what is evaluation and what could be common principles of Public policies evaluation, based on different experiences
To build and to strengthen links with the Performance Audit WG, because the process of elaborating standards of audit performance is far more ahead
To draft guidelines for the SAI’s willing to improve their capabilities in this field
Cour des Comptes - Evaluation Working Group – 7-9 September 2016

6. Cour des Comptes - Evaluation Working Group – 7-9 September 2016
June 2013 – EWG
Achievements (2/3)
Elaboration of a preparatory document
Discussion of its content
Agreement on some key points :
It is neither desirable not possible to achieve an ISSAI, at least in a short-term
It is necessary to go further into the definition and the methodology of evaluation :
Some aspects of evaluation clearly don’t belong to performance audit, for example the impact of a policy : how can we define and determine what is an impact ?
The process of evaluation involves a wide range of stakeholders : how shall we manage this complexity ?
Cour des Comptes - Evaluation Working Group – 7-9 September 2016

7. Cour des Comptes - Evaluation Working Group – 7-9 September 2016
2014/2016 – meetings in Paris
Achievements (3/3)
Three meetings in Paris respectively in November 2014, June 2015 and June 2016
Objective of the meetings : reach a consensus on the distinctive characteristics and methodology of evaluation; modify the preparatory document and summarize it (Nov 2014)
Ultimate Goal : establish guidelines (not a « standard ») to be endorsed by KSC, GB 2015 and INCOSAI 2016 (June 2015 for the project proposal and June 2016 for the endorsement version)
Several amendments both in form and substance were adopted resulting in the current endorsement version
The document, primarily intended to be a methodological guide of the different types of evaluations conducted by member countries now focuses mainly on the evaluation of public policies.
Cour des Comptes - Evaluation Working Group – 7-9 September 2016

8. Crucial Importance of knowledge sharing
Major issues (1/3)
A questionnaire to collect case studies and practical information on Evaluation methodology and experiences (elaborated by a sub-group composed of 5 SAIs) was and sent to 191 SAIs.
The main lessons learnt (44 SAIs have answered) were presented to KSC in 2012.
One SAI out of two deals somehow
with evaluation
Cour des Comptes - Evaluation Working Group – 7-9 September 2016

9. Crucial Importance of knowledge sharing
Major issues (2/3)
A field where definitions are complex, practices are different, comparisons are specifically useful.
Some pitfalls to avoid : focus only on definition problems (even if they are important) ; set wrong ambitions ; lose ourselves in never-ending discussions between standard and guidelines ; complexify instead of clarify.
Cour des Comptes - Evaluation Working Group – 7-9 September 2016

10. Crucial Importance of knowledge sharing
Major issues (3/3)
Guidelines are even more necessary since evaluation of public policies could involve specific risks:
close to politics, even to political philosophy
far from SAIs usual methods
has to deal with new and not always robust techniques
Cour des Comptes - Evaluation Working Group – 7-9 September 2016

11. Cour des Comptes - Evaluation Working Group - 15/16 october 2015
II. NEXT STEPS
Cour des Comptes - Evaluation Working Group - 15/16 october 2015

12. AND NOW THE NEXT STEPS – Four stages in developing INTOSAI GOV
Project proposal
Drafting of project proposal and preliminary draft of INTOSAI GOV by relevant working group (i.e. EWG)
Approval of project proposal and preliminary draft of INTOSAI GOV by relevant steering committee (i.e. KSC)
Exposure draft
Drafting of exposure draft by EWG
Approval of exposure draft by KSC Steering Committee
Exposure period of exposure draft
Endorsment version
Assessment of comments received on exposure draft
Drafting of endorsement version and executive summaries by EWG
Approval of endorsement version by KSC Steering Committee
Approval of endorsement version by Governing Board
Final INTOSAI GOV
Endorsement of final INTOSAI GOV by INCOSAI
Cour des Comptes - Evaluation Working Group – 7-9 September 2016

13. STAGE 1 –PROJECT PROPOSAL
Drafting of the project proposal by EWG
15 June 2015: EWG meeting in Paris (discussions on the last drafts of the project proposal and the guidelines on the evaluation of public policies).
16 June to July  : drafting of the project proposal and the guidelines according to the received comments during the last EWG meeting, approval of the resulting document by the members of the group.
Approval of the Project Proposal by KSC
15 July 2015: project proposal and guidelines sent to KSC for approval.
15 October 2015: KSC Steering Committee (approval of the project proposal).
Cour des Comptes - Evaluation Working Group – 7-9 September 2016

14. Cour des Comptes - Evaluation Working Group – 7-9 September 2016
STAGE 2 : exposure draft
Drafting of the exposure draft by EWG
16 October to 31 October 2015 : drafting of the exposure draft by EWG
Approval of the exposure draft by KSC
1 November 2015 : exposure draft sent to KSC for approval
17 February 2016: exposure draft approved by the KSC, document posted on the issai.org website
17 February - 14 May 2016: exposure period : 149 comments were received, from SAIs and international organizations. Out of the 6 international organizations that were contacted, 3 of them sent us comments: the OECD (67 comments), the UN (23 comments) and the World Bank (6 comments).
Cour des Comptes - Evaluation Working Group – 7-9 September 2016

15. STAGE 3 : ENDORSEMENT VERSION (1/2)
Overview of comments received by EWG
14 May to 31 May 2016 : analysis of the comments received during the exposure period
1 June 2016 : new draft of the guidelines to be sent to EWG members for comment with table including comments and responses to these comments.
20 June 2016 : EWG meeting (discussions on the new draft of the guidelines): 10 members of the group present, minor changes on the document, technical rather than substantial.
Drafting of the endorsement version of the guidelines by EWG
21 June to 31 June 2016 : drafting of the endorsement version according to comments received during the last EWG meeting
1 July 2016 : Draft endorsement version sent to EWG members for comment.
11 July 2016 : drafting of the endorsement version according to comments received from EWG members (done by exchange)
13 July to 31 August 2016: Translation of the final endorsement version in the 5 official languages of INTOSAI.
Cour des Comptes - Evaluation Working Group – 7-9 September 2016

16. Stage 3 : endorsement version (2/2)
Approval of the endorsement version by KSC and by the Governing Board
15 August 2016: Endorsement version sent to KSC for approval
December 2016 : Endorsement version to be sent to Governing Board for approval
Stage 4 : endorsement of final intosai gov
XXI INCOSAI, Abu Dhabi, 2016
Cour des Comptes - Evaluation Working Group – 7-9 September 2016

17. Cour des Comptes - Evaluation Working Group – 7-9 September 2016
Thank you for your attention. Stéphane gaillard, auditor, deputy director, international relations departement
Cour des Comptes - Evaluation Working Group – 7-9 September 2016