1. Ombudsman Western Australia Serving Parliament – Serving Western Australians Evaluation in the Western Australian Ombudsman’s Office Kim Lazenby & Jane Burn Ombudsman Western Australia Australasian Evaluation Society 2011 International Conference Sydney, Australia 1 September 2011

2. Ombudsman Western Australia Serving Parliament – Serving Western Australians Overview Introduction to the Western Australian Ombudsman Exploration of the theme of the conference Evaluation and Influence, focussing on: –Making effective use of evaluation techniques in work of the Ombudsman’s office –Plus, some thinking about making the results of evaluations more influential

3. Ombudsman Western Australia Serving Parliament – Serving Western Australians Introduction to the Western Australian Ombudsman Officer of Parliament Independent of government of the day First and foremost, role is to receive, investigate and resolve complaints from community members about the administration, or alleged maladministration, by government agencies in Western Australia

4. Ombudsman Western Australia Serving Parliament – Serving Western Australians Introduction to the Western Australian Ombudsman Jurisdiction includes Western Australian government departments, statutory authorities, local government authorities, and public universities operating in Western Australia In 2010-2011, Western Australian Ombudsman’s office dealt with more than 2000 complaints Ombudsman is part of the arrangements for oversighting the work of government agencies - heightens accountability of government agencies to citizens

5. Ombudsman Western Australia Serving Parliament – Serving Western Australians Introduction to the Western Australian Ombudsman 4 main functions: –Investigate and resolve complaints about public administration –Improve the standard of public administration over time –Review investigable child deaths –Inspectorate functions, and specialist investigations

6. Ombudsman Western Australia Serving Parliament – Serving Western Australians Improving the standard of public administration Generally occurs by identifying common sources of complaints, analysing underlying causes of these complaints, and assisting agencies address these causes Commonly undertaken as a major project (or an ‘own motion investigation’) focussing on one area of public administration

7. Ombudsman Western Australia Serving Parliament – Serving Western Australians Improving the standard of public administration 2 recent major projects (or ‘own motion investigations’): –2009-10 Survey of Complaint Handling Practices in the Western Australian State and Local Government Sectors –The Management of Personal Information – Good Practice and Opportunities for Improvement Reports available at www.ombudsman.wa.gov.au

8. Ombudsman Western Australia Serving Parliament – Serving Western Australians Reviewing investigable child deaths Ombudsman reviews investigable child deaths to: –Review the circumstances in which and why the deaths occurred (undertaken by Child Death Review team) –Identify any patterns or trends in relation to the deaths –Make recommendations to any department or authority about ways to prevent or reduce investigable deaths

9. Ombudsman Western Australia Serving Parliament – Serving Western Australians Through analysis of patterns and trends in investigable child deaths, 2 topics for further investigation were identified 2 major projects (or ‘own motion investigations’), which focus on these topics are currently underway: –Investigation into the care planning process for children in the care of the Chief Executive Officer of the Department for Child Protection –Investigation into the prevalence, risk factors, and current preventative strategies for sleep related infant deaths Reviewing investigable child deaths

10. Ombudsman Western Australia Serving Parliament – Serving Western Australians

11. Ombudsman Western Australia Serving Parliament – Serving Western Australians Making effective use of evaluation techniques Characteristics of major projects: –Detailed project proposals, based on preliminary research to assist in planning the scope and objectives of the investigation –Approved by Corporate Executive –Once approved, detailed project plan also developed for project management and governance purposes Techniques applied: –Evaluation plans, defining key evaluation questions –Also, techniques drawn from project management discipline

12. Ombudsman Western Australia Serving Parliament – Serving Western Australians Making effective use of evaluation techniques Characteristics of major projects: –Projects are evidence-based –Preliminary research phase includes research into good practice, latest developments, approaches by other jurisdictions etc –Fieldwork to collect primary and secondary evidence Techniques applied: –Evaluation criteria –Monitoring and evaluation –Indicators –Program logic –Research methods, including interview, document review, survey –Audit methods, including audit testing, interrogation of data already held by agencies

13. Ombudsman Western Australia Serving Parliament – Serving Western Australians Making effective use of evaluation techniques Characteristics of major projects: –Analysis phase, including testing evidence against relevant legislative requirements, approved policy and procedures –Use of working papers for quality and accountability –Quality assurance Techniques applied: –Data analysis: Statistical Qualitative Financial –Compliance audit –Working paper and quality assurance techniques drawn from audit discipline

14. Ombudsman Western Australia Serving Parliament – Serving Western Australians Making effective use of evaluation techniques Characteristics of major projects: –Projects result in recommendations for improving public administration –Recommendations are: evidence-based proportionate practical consider costs and benefits of implementing Techniques applied : –Evaluation as research for decision-making purposes –Utilisation-focussed evaluation –Policy development theory and practice –Cost-benefit analysis

15. Ombudsman Western Australia Serving Parliament – Serving Western Australians Making effective use of evaluation techniques Characteristics of major projects: –‘No surprises’ approach –Natural justice principles –Procedural fairness process Techniques applied: –Evaluation’s attention to ethical issues –Utilisation-focussed evaluation

16. Ombudsman Western Australia Serving Parliament – Serving Western Australians Increasing the influence of evaluation Topic selection - topics for major projects are selected on the basis of a number of criteria that include: –the number of complaints received by the Ombudsman –the likely public interest in the identified issue of concern –the number of people likely to be affected –whether other reviews of the issue have been done recently or are in progress –the potential for our investigation to improve administration across the public sector –whether investigation of the chosen topic is the best and most efficient use of our resources

17. Ombudsman Western Australia Serving Parliament – Serving Western Australians Increasing the influence of evaluation Evidence-based Relationship building and ‘no surprises’ approach Dissemination of results and recommendations

18. Ombudsman Western Australia Serving Parliament – Serving Western Australians Summary Western Australian Ombudsman Office’s major reviews and investigations Incorporation of evaluation techniques, and combination with techniques from other disciplines Ensure our major projects help improve public administration in ways that are: –Evidence-based –Proportionate –Practical –Based on consideration of costs and benefits