Evaluation of the Community Advisory Committees to Boards of Victorian Public Health Services Health Outcomes International Pty Ltd HEALTH AND COMMUNITY CARE CONSULTANTS Jim Hales, Bart O’Brien, Andrew Alderdice Health Outcomes International Pty Ltd May 2008

Evaluation Objectives The primary objective for the evaluation of CACs has been to undertake an independent evaluation of the processes by which CACs operate, as well as the immediate and long term outcomes of the CACs.

Evaluation Objectives Secondary objectives include: Ensuring that the evaluation contributes to the Department’s knowledge of the effects of consumer, carer and community participation on decision-making at the health service organisational level; Ensuring that the evaluation is grounded in a philosophy of consumer, carer and community participation and is conducted in a manner respectful and promoting of the contribution consumers, carers and communities make to the operation of heath services; and Ensuring that relevant and appropriate data and inputs are accessed and used in the evaluation process.

Evaluation Questions Three key questions were addressed through the evaluation process: Do CACs operate in accordance with the legislation, guidelines and its terms of reference? Is consumer, carer and community participation happening throughout each health service? Have CACs made a difference?

Expectations of the Evaluation These questions have been configured against timeframes for achievement. Immediate Impacts Q1 (1 year+) Short Term Impacts Q2 & Q3 (2 - 5 years) Long Term Impacts Q3 (6 – 10 years) Do CACs operate in accordance with the legislation, guidelines and their terms of reference? Have the following changed:  Acceptability  Access  Appropriateness Has a culture of participation been created? Has health care improved from participation initiated or advocated for by CACs?

Methodology  Detailed project planning.  Refine and finalise the evaluation and consultation strategies.  Initiate contact with participating Health Services and the HIC.  Undertake individual health service (and HIC) evaluations.  Synthesise findings to develop an overall evaluation.  Develop final report.

Consultations  Initial contact.  Achieve consensus on site visit logistics.  Circulate self-evaluation instrument.  Site visits: Validate self-evaluation against evidence provided. Conduct interviews with stakeholders.  Prepare draft reports and circulate for feedback.  Prepare and return final reports (copy to DHS).

Key Findings – State Level  CACs are generally compliant with the spirit of the Legislation.  There are significant differences in both the level and method of attaining community engagement, influenced by: Competing local priorities; Health service history/culture of community engagement and engagement of CAC with this philosophy; CAC resourcing; and Definition of CAC role and scope of activity.

Key Findings – State Level  Audits of consumer participation are the most poorly actioned item from the guidelines: These audits have the potential to inform CACs of other consumer and community engagement activity - between the health service and its community - that the CAC may not be aware of, unwittingly seek to duplicate or miss opportunities to work with.  Need for recognition (at State and local levels) that CAC’s are part of the community and consumer engagement process.

A Model of Community Engagement

CAC Engagement CAC Board Business Units Service Delivery Areas Community issues and need for engagement Strategic engagement Operational engagement

Key Findings – State Level  CACs have differing degrees of awareness of the extent and nature of consumer, carer and community engagement that exists within each health service.  CACs may be positioned in a variety of ways within the AHS’ strategy for community engagement.

CAC Positioning Piece of a jig saw Strategic and/or operational but not necessarily engaged with the other pieces. Part of a network Strategic and operational – part of a ‘web’ of engagement. Toward the top of a hierarchy Strategically placed and focused.

Key Findings – State Level  The HIC provides valuable support and focus for CACs and key CAC stakeholders: Through facilitating networking forums: oRO network working best at present oConsumer forums very well received oCAC Chair’s network being revised oNo forum for Executive Sponsors Through education and support activities (individually and collectively)  HIC’s interface with CACs should be negotiated, (with CACs and DHS) to ensure its scope of activity is well understood & agreed (by CACs & DHS)

Key Findings – Health Service Level  CACs have put community engagement on the Board’s agenda.  Health service history of community engagement can have a contributory or restrictive influence on CAC development, focus and scope of activity depending upon where the CAC is seen to ‘fit in’.  Effective, respectful and collegiate links between the CAC, the Board and Senior Executive contribute to CAC’s impact on health service functioning.

Key Findings – Health Service Level  The capacity and experience of the Resource Officer and support for this role are major factors influencing the level and nature of support this role brings to the CAC.  The contributions of CAC Chairperson, Executive Sponsor and CEO are also key drivers for CAC performance i.e. Chairperson’s style of chairing meetings. Executive Sponsor’s relationship to the RO. CEO commitment to principles of community engagement through a CAC.

Key Findings – Health Service Level  Effective CAC recruitment, membership diversity and member continuity contribute to CAC stability and performance: Peak bodies are seldom used as sources of recruitment. Diversity is clearly important to all CACs. Most CACs manage member turnover effectively.

Facilitators to CAC Effectiveness  Culture of community/consumer engagement.  Mutual agreement/confirmation regarding CAC role, focus and scope of activity from: CAC. Board. Health Service Senior Executive.  Resourcing of the CAC: Membership continuity and diversity. Resources: oRO, Executive Sponsor & Chairperson. oTiming of meetings, reimbursement of members & budget to achieve goals.

Barriers to CAC Effectiveness  Discord and/or lack of common understanding regarding an agreed role for the CAC between: CAC Board Health Service Executive  Lack of adequate, reliable resourcing of the CAC to achieve mutually agreed goals  Lack of, or discontinuous support for the CAC from: Board. CAC Chairperson. Resource Officer. Executive Sponsor and/or CEO.

Future Directions  Consideration of the Guidelines and CACs as a baseline for community, consumer and carer engagement.  Further consideration of how best to use HIC input for CAC development and support.  Undertake and maintain baseline audits of the nature and extent of community, carer and consumer engagement.  Consider more expansive models of engagement: That confirm CACs as part of the process. With consideration of NSW and Canadian (local area health networks – LAHNs) models.

Future Directions   Our next step is to complete our analysis and prepare a draft Final Report for review by DHS.   Provide a Final Report that addresses the feedback received.   Individual Health Services and CACs can action findings from their individual reports.