1. How does the ECA assess Member States’ internal control systems? Workshop on Audit/Evaluation of Public Internal Financial Control Systems (PIFC) Ankara, 8-9 July 2008 Alan Findlay (CEAD Directorate B - Audit and Audit Supervision) European Court of Auditors

2. 2 Agenda The Court’s “Assurance Model” The DAS audit process  Understanding the environment  Planning audit procedures  Performing the audit & evaluating results  Forming audit opinion Questions / Discussion

3. 3European Court of Auditors The Assurance Model (1)

4. 4European Court of Auditors Assurance Model as a tool to manage audit risk: Preliminary assessment of inherent and control risk (combined risk assessment) to determine:  Reasonable expectations concerning level of confidence (controls assurance) that can be drawn from supervisory and control systems  Resulting extent and scope of substantive testing to be carried out, aiming to reach required overall level of confidence (95%) Revision and / or final confirmation of preliminary assessment on the basis of audit results achieved The Assurance Model (2)

5. 5European Court of Auditors Agenda The Court’s “Assurance Model” The DAS audit process  Understanding the environment  Planning audit procedures  Performing the audit & evaluating results  Forming audit opinion Questions / Discussion

6. 6European Court of Auditors UNDERSTAND THE AUDIT ENVIRONMENT Audit Environment Audit Evidence Legal Basis International Audit Standards THE DAS Art. 248 of the Treaty Reliability Legality & regularity (Art.129(4) of Financial Regulation) published with financial statements F ORM AUDIT CONCLUSIONS AND AUDIT OPINION P LAN AUDIT PROCEDURES PERFORM AUDIT AND EVALUATE RESULTS International Standards of Auditing (ISA) published by IFAC INTOSAI auditing standards COSO Framework DAS audit process

7. 7European Court of Auditors Agenda The Court’s “Assurance Model” The DAS audit process  Understanding the environment  Planning audit procedures  Performing the audit & evaluating results  Forming audit opinion Questions / Discussion

8. 8European Court of Auditors Audit environment Gain understanding of the audit environment Collect /update and analyse information about the audit environment:  Legal framework  Design and functioning of the supervisory and control systems  Nature of payments  Audit standards to be applied, etc.

9. 9European Court of Auditors Agenda The Court’s “Assurance Model” The DAS audit process  Understanding the environment  Planning audit procedures  Performing the audit & evaluating results  Forming audit opinion Questions / Discussion

10. 10European Court of Auditors Audit procedures Plan audit procedures Plan audit procedures allowing to obtain reasonable assurance Preliminary assessment of:  Inherent risk  Quality of supervisory and control systems Determine materiality

11. 11European Court of Auditors Audit procedures (2) Determine materiality Errors are material if, individually or aggregated with other errors, they would reasonably affect decisions of addressees of opinion  General practice: between 0,5% and 2%  Possibility to set different level(s) taking into account addressees requirements (“tolerable or residual risk”)  Issues material by context/nature to be disclosed Materiality rules not (always) directly applicable to systems weaknesses (often only risk)  Qualitative aspects (seriousness of deficiency or frequency)  Quantitative aspects (possible financial impact)

12. 12European Court of Auditors Preliminary assessment of inherent risk Economic and technical environment Type and operating of activities, programmes, etc. Characteristics of beneficiaries Complexity of rules Management policies and practices Other information to be taken into account:  Previous work and knowledge / experience  Annual activity reports and declarations of Commission’s Directors-general and their Synthesis  Other obligatory reports of different services of Commission and Member States  Relevant other information (National declarations, national certificates,...)  Reports of other auditors Audit procedures (3)

13. 13European Court of Auditors Audit procedures (4) Preliminary assessment of quality of supervisory and control systems Central question: Are supervisory and control systems adequate to manage the risk insofar as compliance with relevant laws, regulations, contracts, etc. is concerned?  Strong link between inherent risk and control risk  Different implementation levels to be considered

14. 14European Court of Auditors Agenda The Court’s “Assurance Model” The DAS audit process  Understanding the environment  Planning audit procedures  Performing the audit & evaluating results  Forming audit opinion Questions / Discussion

15. 15European Court of Auditors Performing the audit Perform audit procedures and evaluate results Evaluate audit results:  (Re-)examine / (Re-)assess preliminary expectations  Modify initial planning on the basis of preliminary audit results, if needed  Perform supplementary audit work, if necessary Perform audit procedures:  Examination of supervisory and control systems  Substantive testing of payments  Analysis of management representation (National Declarations Summaries, Annual Activity Reports, etc.)  Possibly work on other sources of audit evidence (e.g. other auditors)

16. 16European Court of Auditors Audit objectives for supervisory and control systems Assessment of their capacity to Supply management with information needed to ensure that legal, regulatory and contractual provisions have been complied with and, where applicable, corrective action is taken Give reasonable assurance concerning compliance of revenue and expenditure with laws, regulations, contracts, etc.

17. 17European Court of Auditors Auditors should first identify the management departments, national/regional/local authorities or other third parties having a role of supervision or control Preparation of a programme of tests to check systems’ compliance with three key control objectives  Design according to regulation or other provisions  Transposition by parties concerned  Continuous and effective functioning Main steps of evaluating supervisory and control Systems

18. 18European Court of Auditors Substantive testing samples may also be used for tests of control to limit number of on-the-spot checks Examination of payments affected by errors  Identification of systems which should have prevented or identified and corrected them  Analysis of control deficiencies at their origin Overall analysis of the results Assessment of level of assurance obtained The role of substantive testing in evaluating systems

19. 19European Court of Auditors Agenda The Court’s “Assurance Model” The DAS audit process  Understanding the environment  Planning audit procedures  Performing the audit & evaluating results  Forming audit opinion Questions / Discussion

20. 20European Court of Auditors Forming audit conclusions & opinions Form audit conclusions and audit opinion Form audit conclusions at the level of specific assessments for different areas Form audit opinion

21. 21European Court of Auditors Straightforward forming of audit conclusions requires that  Actual audit results confirm reasonable expectations formed on the basis of preliminary combined risk assessment  Findings for different sources are compatible with each other Forming audit conclusions & opinions

22. 22European Court of Auditors Evaluation of supervisory and control systems Professional judgement and materiality Qualitative evaluation of results on work on systems Quantitative evaluation of results of substantive testing Analysis of coherence of audit results Audit conclusions Audit opinion – the DAS Analysis of annual activity reports and declarations Substantive testing Examination of work of other auditors Form audit opinion Audit conclusions

23. 23European Court of Auditors Agenda The Court’s “Assurance Model” The DAS audit process  Understanding the environment  Planning audit procedures  Performing the audit & evaluating results  Forming audit opinion Questions / Discussion