Community-Based Rehabilitation (CBR) Evaluation Framework Manjula Marella Co-authors: Ecosse Lamoureux and Jill Keeffe Centre for Eye Research Australia University of Melbourne

Community-based rehabilitation (CBR)  The World Report on Disability identifies CBR as an important component of health systems  Rehabilitation within one’s own community using local resources  The World Health Organization (WHO) guidelines promote a comprehensive system addressing Health, Education, Livelihood, Social participation and Empowerment of people with disabilities

Effectiveness of CBR services  Evaluations published on CBR programs lack evidence-based research  Published evaluations are mainly descriptions of services  Different methodologies and indicators used for evaluations  Common indicators and a systematic framework for evaluation that are valid for different settings of CBR are essential

Evaluation framework  To address the gap in the literature related to effectiveness of CBR programs, the Centre for Eye Research Australia (CERA)-CBR Framework (CCF) has been developed  The CCF includes indicators to comprehensively evaluate CBR programs

CERA – CBR Framework (CCF) Service Delivery System Availability of Services Utilisation of Services Quality of Services Program Management Program Planning Human Resources InfrastructureCoverageSustainabilityNetworkingLinkages Referral System Advocacy Support from families Support from community Outcomes Intended outcomes Functional independence Psycho-socialEducation Economic Independence Empowerment Changes in Community Satisfaction with services Components Key Areas Manjula Marella, CERA November 2010

Utilisation of services Enrolments Waiting list Dropouts Eligibility to enrol Client demographics Average number of clients enrolled per year Waiting time Number of clients on waiting list Number of clients who discontinue annually Gender ratio of dropouts Reasons for dropouts Socio-demographic profile of dropouts Themes/QuestionsKey areaIndicators

Developing questionnaires  Questions for each indicator  Data collection methods and sources  Key informant questionnaires – CBR staff – Clients and Family members – Government (Health and Social Welfare) – Eye care practitioners – Disabled People Organisations (DPOs) – Non-government organisations (NGOs)

Face validity of the CCF  Reviewed by an expert panel (n=7) – CBR – Evaluation methods  To assess the appropriateness of the content and structure of the CCF  The CCF was modified based on the feedback from the panel  Modified version of the CCF included 230 indicators

Field-validation of the CCF  Fiji Society for the Blind (FSB)  Cambodian Development Mission for Disability (CDMD)

Field-validation of the CCF FijiCambodia Total Population 944,72014,494,293 Location of services 4 areas of Viti Levu Island4 provinces (29 districts) in the south Number of CBR workers 420 Types of services  Functional rehabilitation  Referrals to hospitals, social welfare department and special education.  Inclusive education program is only for high school students  Functional, vocational and social rehabilitation  Empowerment  Referrals to hospitals and inclusive education Sample size38148

Analysis  Collated responses from various stakeholders  NVivo 8  Qualitative analysis  Strengths and limitations of the CBR programs  Usefulness of indicators – Applicable (relevant to the context of services) – Measurable (feasible to obtain data)  Triangulation – information obtained from different sources and compared

1. Service delivery system FijiCambodia Strengths Medical model of rehabilitation: - functional skills training and - referrals to school for the blind Social model of rehabilitation addressing all components in the WHO CBR matrix: health, education, social, livelihood and empowerment Regular follow-up and monitoring systems Limitations Lack of systematic follow-ups and internal monitoring protocols Unable to manage children <5 years

2. Program planning FijiCambodia Strengths Saving costs associated with office space and transport by working in collaboration with Social Welfare and Eye departments Involvement of external stakeholders to plan activities Limitations Activities do not match with the goals of the program Poor financial sustainability

3. Networking FijiCambodia Strengths Collaboration with Social Welfare and Eye departments for resources Involving families in rehabilitation of the clients Good compliance with referral services Involving families in rehabilitation of the clients Advocating for inclusion of PWDs Limitations Poor collaboration with DPO and government Lack of community involvement in CBR program Accessibility to referral services Poor collaboration with DPO and government Lack of community involvement in CBR program Negative attitudes of staff at referral centres

4. Outcomes FijiCambodia Strengths Improvement in daily living skills Changes in community attitudes Inclusion of clients in community as a result of self- help groups and income generation High client satisfaction - 35 (81%) clients and 52 (88%) family members interviewed felt their needs were met Limitations Client satisfaction - 4 (36%) clients and 7 (50%) families interviewed felt their needs were met Lack of systems to evaluate outcomes of services

Usefulness of the CCF FijiCambodiaBoth Applicable201 (87%)219 (95%)197 (85%) Measurable167 (83%)178 (81%)156 (79%) Overall 230 indicators in the framework

Discussion  The CCF is comprehensive and useful to evaluate the effectiveness of CBR programs  First study to compare two models of CBR  Comprehensive model of CBR is feasible and more effectively meets the needs of the clients  Recommendations provided based on the findings using the CCF helped the organisations to plan new services and strategies – Training of staff – Plans for financial sustainability

Application of the CCF  144 (63%) indicators were derived from the literature where they were not proposed for any specific disability  The majority (197, 85%) indicators were applicable to two different settings  The framework is likely to be applicable to different programs irrespective of model and type of disability

Limitations and recommendations for future research  The component ‘outcomes’ needs further validation  Only two case-studies in this study  Does not include economic analysis  Application in other settings including developed countries

Summary  The CCF has been developed to evaluate the effectiveness of vision-related CBR programs  Field-tested in two different CBR settings in Fiji and Cambodia  Practical and useful to investigate the strengths and limitations of the two CBR programs

Acknowledgements  Supervisors: A/Prof Ecosse Lamoureux and Prof Jill Keeffe  Fiji Society Blind and Cambodian Development Mission for Disability  Vision Cooperative Research Centre (CRC)  University of Melbourne

Thank you