Possibilities of evaluation of SAI performance efficiency and effectiveness and evaluation criteria Mindaugas Macijauskas, Head, Strategic Planning and Analysis Division Audit Development Department 4 September 2008

2 Why do we need evaluation?  To know where we are  To measure the progress  Show the public outputs achieved using budgetary money

3 Stakeholders.  Parliament  Public  Media What is important to them?  Auditees NAOL

4 How to set the criteria?  Internal resources  External surveys

5 Criteria originated from internal resources  State budget share audited  Audit reports finalized  Implementation of recommendations  Audit reports discussed in the Parliament  Audit impact  Other

6  Survey of Parliament Members opinion  Feedback from auditees  Survey of public opinion Criteria originated from external surveys

7 Value set: 80 %, increase tendency Linked with financial (regularity) audit The share of state budget audited

8 Number of audit reports finalized Value set: about 25 audit reports annually Linked with performance audit planned finalized

9 Implementation of recommendations plannedachieved Financial (regularity) audit8790 Performance audit8183 Value set: Proportion of fully or partly implemented recommendations Linked to both financial and performance audit

10 Other possible criteria of SAI performance evaluation  Staff turnovers  Staff trainings (money spent, training hours)  Job satisfaction level  Time spent to complete the audit  Average cost of an audit

11 Survey of Parliament Members opinion  Do you read the SAI‘s report in full?  Are you currently using SAI‘s reports as an information source in your work as a MP?  Do you think that SAI‘s reports are of assistance to you in your work as a MP?  Does SAI effectively contribute to ensuring accountability of the public sector?  Does SAI have enough powers to perform functions assigned to it by the Constitution?

12 Feedback from auditees  Financial (regularity) audit - survey of undersecretaries during the presentation of Summary report  Performance audit - survey of auditees after audit is finished

13 Public trust in the Lithuania SAI Survey of public opinion conducted by mass media Survey of Public opinion Source – the Magazine “Veidas”

14 Other criteria  Ranking of public institutions  Ranking of state officers  Peer review  …

15 Audit impact Financial (regularity) audit: 532 violations of legal acts, more than a half of which were eliminated; LTL 6 million of funds recovered to the State Budget till the end of the budgetary year, which were used not as intended or breaching legal acts; LTL 8 million worth property was recovered. Performance audit

16 Challenges evaluating performance of SAI  No common methodology how to assess social and economical impact of audit  Evaluation using the system of criteria, not the single criteria  Criteria should be set taking into consideration their practical usefulness  Exchange of experience among the SAI’s

17 Thank you for attention