1. Results orientation: audit perspective Jiri Plecity, Head of Unit H1, Relations with Control Authorities, Legal Procedures, Audit of Direct Management and EGF ESF TWG, Rome, 16/10/2014 1

2. PART I Audit-related aspects of the performance framework 2

3. Performance Framework Building Blocks Priority level broken down by cat. of region & fund Milestones (intermediate targets) Targets To be achieved by the end of 2018 Formally reviewed in 2019 To be achieved by the end of 2023 Formally reviewed at closure Indicators: Financial, Output, Result (ESF), Key implementation steps 3 Cohesion Policy

4. Performance review In 2019, the performance of the programme priorities is assessed by EC against the milestones set for the end of 2018. Values reported in 2018 Annual Implementation Reports submitted by 30 June 2019. Commission decision by 31 August 2019 on priorities which have achieved milestones, broken down by cat. of region & fund. All indicators of a priority to reach 85% of milestones to gain access to its reserve. 4 Cohesion Policy

5. Allocation of the Reserve Priorities which achieve their milestones: reserve is allocated definitively. Priorities which do not achieve their milestones: MS proposes the attribution of their pre-allocated reserve among priorities which did achieve their milestones (respecting thematic concentration and Fund and category of region allocations). Consequent amendment of OPs, including targets. 5 Cohesion Policy

6. Suspensions or corrections I EC may suspend after the 2019 performance review all or part of interim payment(s) of a priority if:  There is serious failure to achieve the milestones due to a clearly identified implementation weakness  EC has communicated previously this clearly identified implementation weakness and the MS has failed to take the necessary corrective action  Articles 22.6 and 142.1.f 6 Cohesion Policy

7. Suspensions or corrections II At the end of programming period (after final implementation report in 2025), the EC may apply financial corrections if:  There is serious failure to achieve the targets set out in the performance framework (art. 144.4) and in addition if  no significant socioeconomic, environmental developments or implementation delays beyond control of the MS prevented the achievement of targets (art.22.7/3) 7 Cohesion Policy

8. Tasks of the Managing authorities I Collect, record and store reliable data on indicators and milestones in a computerised system Report the progress of the operational programmes in achieving its objectives, indicators and milestones in the Annual Implementation Report (hereafter "AIR"). 8

9. Tasks of the Managing authorities II Establish a system to record and store in computerised form data on each operation Collect, record and store the data in the system including "through" beneficiaries and IBs. Report data relating to progress of the operational programme in achieving its objectives and relating to its indicators and milestones, annually as from May 2016 in the AIR and notably for 2018 and 2023. 9

10. Failure to accomplish the tasks I – designation Designation criterion: the systems in the MA should ensure that it is in a position to fulfil its responsibilities under Articles 72 and 125 including those concerning monitoring systems and those required for indicators 10

11. Failure to accomplish the tasks II – serious deficiency Suspension possible if "Serious deficiency in functionning of management and control system" -KR4 on management verifications (indicators reported) -KR5 on audit trail documenting progress in achieving outputs and results and monitoring reports -KR 6 on reliable system for collecting, recording and storing data for monitoring, evaluation, financial management, verification and audit purposes 11

12. Failure to accomplish the tasks II – serious deficiency Suspension possible if "Serious deficiency in the quality and reliability of the monitoring system or of the data on common and specific indicators" -Quality, integrity and ability of the underlying data management and IT systems to store, collect, aggregate and report these indicators and milestones. -This includes microdata on beneficiaries (ESF reg) 12

13. Role of auditors Assess the quality, integrity and ability of the data management and computerised system to store, collect, aggregate and report the data. Give assurance on the reliability of the reported performance data and of their aggregation. And determine the necessary improvements and their timing and/or corrective measures, if any. 13

14. Role of auditors - Processes and contexts Quality of performance data is part of the designation process, system audits and audit of operations It is reflected in the audit opinion of audit authorities References: guidance notes on designation, system assessment and on audit strategy 14

15. PART II Commission audit of systems for collecting, recording, storing and reporting performance data 15

16. Scope of the audit Managing Authority; Intermediary Body, where applicable; Beneficiaries; Operations; Partners; Projects; 16

17. Objective Assess the quality, integrity and ability of the underlying data management and IT systems to store, collect, aggregate and report these indicators and milestones. Enquiry and audit work is limited to the reliability of performance data reported, and does not extend to performance in general. Objective is to discover deficiencies so that they can before it is too late 17

18. Scope of the audit 18 Accuracy, Completeness, Reliability, Timeliness, Confidentiality, Precision, Integrity Requirements of the Data Management System needed to ensure Performance Indicator Quality 1. M.A. structure, functions and capabilities 2. Indicator definitions and Reporting Guidelines 3. Data collection and reporting documents/tools 4. Data Management Processes 5. Links with National Reporting System

19. System assessment 19

20. Timing? In total 20 audit mission scheduled before 2016; 1 audit missions foreseen in 2014 (YEI France in November); Missions selected on basis of risk assessment; First phase: system assessment; Where possible…also data verification; First focus on YEI 20