1. An online clearinghouse for evidence-based research on overcoming disadvantage for Indigenous Australians Dr Fadwa Al-Yaman Australian Institute of Health and Welfare Presentation to Australasian Evaluation Society International Conference 2011

2. Outline Overview of the Clearinghouse Scope and process Available resources Limitations of current approach Future directions

3. Purpose – to identify programs/interventions that work to overcome Indigenous disadvantage (evidence-based evaluations) COAG initiative, jointly funded by all governments and delivered by the AIHW in collaboration with AIFS Primary audience is policy makers Governed by the Clearinghouse Board, with support from a Scientific Reference Group Clearinghouse overview

4. Focus The Clearinghouse’s efforts are directed across 7 building blocks which support the 6 COAG targets COAG Targets (6) Related to: 1. Life expectancy 2. Mortality rates for children 3. Early childhood education 4. Reading, writing and numeracy 5. Year 12 attainment 6. Employment outcomes Building Blocks (7) Health Early childhood Schooling Economic participation Healthy homes Safe communities Governance and leadership

5. Scope of the clearinghouse collections Indigenous Australians Non-Indigenous Australians Indigenous populations in USA, NZ and Canada Non- Indigenous populations in USA, NZ and Canada Other countries

6. Clearinghouse collections General collection Assessed collection- create an assessment tool General info Levels of evidence Adaptability of the evidence to Indigenous Australians Advice on rating of evidence Other relevant info Research & Evaluation Register

7. Clearinghouse publications Issues papers and resource sheets Final topics set by the Board following discussions with jurisdictions Key learnings and gaps in evidence An annual paper that summarises and synthesises the evidence and identifies the major gaps in the research Annual report An overview of the Clearinghouse’s achievements.

8. Clearinghouse communications e-news - Quarterly newsletter about Clearinghouse activities Presentations and discussions of Clearinghouse resources in each jurisdiction Conference presentations Helpdesk

9. Clearinghouse products

10. General collection 4,757 items Note: Items can be counted in one or more building blocks Building BlockNo.Per cent Early childhood 1,41330 Schooling 78416 Health 1,76437 Economic participation 92920 Healthy homes 58812 Safe communities 1,40830 Governance and leadership 1804 Total 4,757100

11. Assessed items Building BlockNo.Per cent Early childhood 8814 Schooling 14023 Health 19432 Economic participation 8814 Healthy homes 539 Safe communities 9215 Governance and leadership 529 Total 607100

12. Characteristics of assessed collection by building block (%) – evaluation component

13. Characteristics of assessed collection by building block (%) – cost component

14. Characteristics of the assessed collection by building block and type of study (%)

15. Characteristics of assessed collection by building block (%) – country

16. Assessed items by country and Indigenous status and research type

17. Characteristics of assessed collection by building block (%) – program/activity

18. Research and Evaluation Register Note: Items can be counted in one or more building blocks Building Block Total Research and Evaluation items Completed Research and Evaluation items No.Per centTotal completed Per cent publically available Early childhood 118185859 Schooling 145237954 Health 3505516252 Economic participation 128207555 Healthy homes 118186650 Safe communities 147239156 Governance and leadership 72115165 Total 63910058255

19. Clearinghouse publications Five resource sheets: early childhood, schooling, alcohol and other drugs, employment, child safety and service delivery co-ordination Three issues papers: on school readiness, school attendance and retention and employment Annual paper: What works to overcome Indigenous disadvantage: key learnings and gaps in the evidence

20. Limitations of the approach Assessed collection –Assessors bias in terms of ranking the evidence and issues with a ranking approach especially in social sciences –Confusion over assessment of peer review articles –Not enough rigorous evaluations to sustain the agreed number of items for assessed collections (600) –Too much time spent screening assessed items by Clearinghouse team

21. Limitations of the approach Resource sheets and issues papers –Too many topics to cover each year –Topics not fully scoped with policy makers upfront –The mismatch between what policy makes wanted and what academics produced –Focus by many authors on the process and listing of programs not on program outcome –Little incentive for academics and significant delays –Some were ideologically driven rather than evidence driven

22. Limitations –communications Policy makers are the main focus Authors and the public are not a central part of the dissemination strategy

23. Main implications Little linkages between the different products Variable quality of evidence Not enough synthesis of evidence Many small-scale Australian programs across the building blocks Not enough rigour in the evaluations

24. Way forward Which evidence? What works in what context? How generalizable is the evidence? When evidence is available? what are the barriers? Adapting the evidence- local context is very relevant –Small versus large jurisdictions –Remote versus urban –Other context

25. An example – home visitation Long time to establish program effectiveness Cost effective if targeted Returns across many areas - many years later Issues are around successful implementation –Sustainable and intensive (almost 3 years) –Flexible –Holistic –Trained staff –Partnership- Indigenous and non- Indigenous service delivery

26. Future directions cont. More focus on the synthesis of evidence – less topics –With a focus on the quality and strength of evidence No of studies Research design Quality of research Consistency of findings (strength of evidence) Greater linkages between policy makers and those who synthesize the evidence Process to ensure evidence is used by policy makers Better dissemination of evidence to the public to improve uptake (authors seminars)

27. Thank you Any questions?