Community-Based Rehabilitation (CBR) Evaluation Framework

Manjula Marella

Co-authors: Ecosse Lamoureux and Jill Keeffe

Centre for Eye Research AustraliaUniversity of Melbourne

Community-based rehabilitation (CBR)

The World Report on Disability identifies CBR as an important component of health systems

Rehabilitation within one’s own community using local resources

The World Health Organization (WHO) guidelines promote a comprehensive system addressing Health, Education, Livelihood, Social participation and Empowerment of people with disabilities

Effectiveness of CBR services Evaluations published on CBR programs lack

evidence-based research Published evaluations are mainly descriptions of

services Different methodologies and indicators used for

evaluations Common indicators and a systematic framework

for evaluation that are valid for different settings of CBR are essential

Evaluation framework

To address the gap in the literature related to effectiveness of CBR programs, the Centre for Eye Research Australia (CERA)-CBR Framework (CCF) has been developed

The CCF includes indicators to comprehensively evaluate CBR programs

CERA – CBR Framework (CCF)

Service Delivery System

Availability of Services

Utilisation of Services

Quality of Services

Program Management

Program Planning

Human Resources

Infrastructure

Coverage

Sustainability

Networking

Linkages

Referral System

Advocacy

Support from families

Support from community

Outcomes

Intended outcomes

Functional independence

Psycho-social

Education

Economic Independence

Empowerment

Changes in Community

Satisfaction with services

Components

Key Areas

Manjula Marella, CERA November 2010

Utilisation of services

Enrolments

Waiting list

Dropouts

Eligibility to enrol

Client demographics

Average number of clients enrolled per year

Waiting time

Number of clients on waiting list

Number of clients who discontinue annually

Gender ratio of dropouts

Reasons for dropouts

Socio-demographic profile of dropouts

Themes/QuestionsKey area Indicators

Developing questionnaires Questions for each indicator Data collection methods and sources Key informant questionnaires

– CBR staff– Clients and Family members– Government (Health and Social Welfare)– Eye care practitioners– Disabled People Organisations (DPOs)– Non-government organisations (NGOs)

Face validity of the CCF

Reviewed by an expert panel (n=7)– CBR– Evaluation methods

To assess the appropriateness of the content and structure of the CCF

The CCF was modified based on the feedback from the panel

Modified version of the CCF included 230 indicators

Field-validation of the CCF Fiji Society for the Blind (FSB) Cambodian Development Mission for Disability

(CDMD)

Field-validation of the CCFFiji Cambodia

Total Population

944,720 14,494,293

Location of services

4 areas of Viti Levu Island 4 provinces (29 districts) in the south

Number of CBR workers

4 20

Types of services

Functional rehabilitation Referrals to hospitals, social welfare department and special education.

Inclusive education program is only for high school students

Functional, vocational and social rehabilitation

Empowerment Referrals to hospitals and inclusive education

Sample size 38 148

Analysis Collated responses from various stakeholders NVivo 8 Qualitative analysis Strengths and limitations of the CBR programs Usefulness of indicators

– Applicable (relevant to the context of services)– Measurable (feasible to obtain data)

Triangulation – information obtained from different sources and compared

1. Service delivery systemFiji Cambodia

Strengths • Medical model of rehabilitation:

- functional skills training and - referrals to school for the

blind

• Social model of rehabilitation addressing all components in the WHO CBR matrix: health, education, social, livelihood and empowerment

• Regular follow-up and monitoring systems

Limitations • Lack of systematic follow-ups and internal monitoring protocols

• Unable to manage children <5 years

2. Program planning

Fiji CambodiaStrengths • Saving costs associated with

office space and transport by working in collaboration with Social Welfare and Eye departments

• Involvement of external stakeholders to plan activities

Limitations • Activities do not match with the goals of the program

• Poor financial sustainability

• Poor financial sustainability

3. NetworkingFiji Cambodia

Strengths • Collaboration with Social Welfare and Eye departments for resources

• Involving families in rehabilitation of the clients

• Good compliance with referral services

• Involving families in rehabilitation of the clients

• Advocating for inclusion of PWDs

Limitations • Poor collaboration with DPO and government

• Lack of community involvement in CBR program

• Accessibility to referral services

• Poor collaboration with DPO and government

• Lack of community involvement in CBR program

• Negative attitudes of staff at referral centres

4. OutcomesFiji Cambodia

Strengths • Improvement in daily living skills

• Changes in community attitudes

• Inclusion of clients in community as a result of self-help groups and income generation

• High client satisfaction - 35 (81%) clients and 52 (88%)

family members interviewed felt their needs were met

Limitations • Client satisfaction - 4 (36%) clients and 7

(50%) families interviewed felt their needs were met

• Lack of systems to evaluate outcomes of services

• Lack of systems to evaluate outcomes of services

Usefulness of the CCF

Fiji Cambodia Both

Applicable 201 (87%) 219 (95%) 197 (85%)

Measurable 167 (83%) 178 (81%) 156 (79%)

Overall 230 indicators in the framework

Discussion The CCF is comprehensive and useful to evaluate the

effectiveness of CBR programs First study to compare two models of CBR Comprehensive model of CBR is feasible and more

effectively meets the needs of the clients Recommendations provided based on the findings using

the CCF helped the organisations to plan new services and strategies– Training of staff– Plans for financial sustainability

Application of the CCF

144 (63%) indicators were derived from the literature where they were not proposed for any specific disability

The majority (197, 85%) indicators were applicable to two different settings

The framework is likely to be applicable to different programs irrespective of model and type of disability

Limitations and recommendations for future research

The component ‘outcomes’ needs further validation

Only two case-studies in this study Does not include economic analysis Application in other settings including

developed countries

Summary

The CCF has been developed to evaluate the effectiveness of vision-related CBR programs

Field-tested in two different CBR settings in Fiji and Cambodia

Practical and useful to investigate the strengths and limitations of the two CBR programs

Acknowledgements

Supervisors: A/Prof Ecosse Lamoureux and Prof Jill Keeffe

Fiji Society Blind and Cambodian Development Mission for Disability

Vision Cooperative Research Centre (CRC) University of Melbourne

Thank you