Using Indicators for Quality Improvement; Overview of the IMCI Bulletin Tool CORE Spring Meeting May 2012

Overview

• Context of tool development

• Methodology of use - Tool - Information produced

• Tool platform and components

• Benefits

• Challenges

• Relevance in other contexts

Tool Context•Expanded Impact Project (EIP) 2006 – 2011 focused on CCM scale-up, quality of care and community mobilization

• Designed within a systems strengthening and quality assurance framework to provide an overarching view of district and health center C-IMCI performance

• Enables decision makers to assess progress in real-time across critical indicators/standards to trigger quality improvement Rwanda

Data compiling and reporting, meetings

Health center CHW supervisor

Bulletin analysis in feedback meetings• Cell coordinators• HC CHW Supervisor • Data Manager

Cell coordinator

C-IMCI Quality Improvement Landscape in Rwanda

Hospital

Health Center

Information & Feedback

Supervision

Sector

1-2 hours motorcycle / car ride

District

Village

Cell1 hour to 1 day walk;bicycle or motorcycle

CHWs

• Community Case Management

• Health Promotion Activities

Medical Director

CHW groups

Methodology

Who?

• Data managers (HC)

• Cell coordinators

• CHW supervisors (HC)

• CHW supervisors (hospital)

• EIP QA and M&E staff (HC)

• District planners

• District authorities

MethodologyWhat?

• Tool explores elements of quality improvement; with corresponding indicators and performance standards

• Designed to collect and analyze data from multiple sources

• HIS database from the HC

• Monthly reports and minutes

of CHW cooperatives meetings

• OTP cards at nutrition unit of

HC• Medication stock cards at HC

• Monthly reports of Care

Groups

• Supervision checklist reports

of cell coordinators and HC

CHW supervisors

• Hospital register of

supervision visits

• Copies of individual patient

cards of CHWs submitted to HC

supervisor

Supervision Checklist

CHW Patient and Drug Register CHW

Compiled Report

CCM Health Facility Report

Excel database

• Based on existing CCM data flow• And more…

MethodologyWhere and When?

• Community, ongoing data collection

• Implementation of corrective measures

• Health Center, monthly data aggregation and quality checks

• Analysis and feedback

• Hospital and District, quarterly data consolidation

• Results dissemination, feedback and planning

• National, six-monthly data review

MethodologyHow is the tool and its information useful?

1. Real-time review of automated results

2. Feedback and planning with stakeholders and decision makers

3. Best performing HC per district publically acknowledged

4. Analysis at community health desk and technical working group

5. Barriers to quality improvement identified, workplans adapted, new targets developed

Methodology

Why incorporate?

• To identify strengths and weaknesses by comparing achievements to agreed norms

• To provide a practical guide for further planning and decision making

• To share strengths, replicate best practices and move quickly to improve on weaknesses

Tool Components• Design• Indicators• Standards

Design

•”Scorecard” i

n

excel

•District le

vel

•Spreadsheet

format

•Distric

t level •Poste

r in health

center

•Summary of H

C data

by month •Form

ats inadequate

•Heavy centra

l level

input

•Not timely

enough

• 2008: district scorecard pilot in one HC

• 2010: scorecard revised into IMCI Bulletin allowing all HCs (across six districts) to collect data monthly

• Six district data consolidated every six months (June 2010, January 2011, and June 2011)

• Final tool: home page, help page, data entry platform, automated analysis

Finally…

IMCI Bulletin Home Page

Help Page

1. Database overview

2. How to input data

3. How to save data4. How to export

data for analysis

IMCI Bulletin Analysis

IMCI Bulletin Analysis

Master Spreadsheet

Indicators based on –• Global C-IMCI technical

reference materials• MoH guidance• Performance-based

financing (PBF)• Data simplicity and

accessibility

Standards based on –• Evidence (expected

incidence…)• Consensus (feasible

targets)

• Design•

Indicators

• Standards

Tool Components

17 indicators and standards across six

focus areas –

• Utilization of services• Medication stock

management• Community participation• Community case management• Human resources for health• Reporting systems

Component 3: Targeted CHWs providing CCM

Component 4: Medicine and diagnostic availability

Component 5: Treatment coverage

Component 7: Routine

supervision coverage

Component 7: Correct case management practice

So did this “Beta Benchmark”

Bulletin Tool help the program?

Bulletin Standards and Indicators

vs. CCM Benchmarks

and Indicators

Bulletin Benefits to EIP• Did quality improve?

– Contributed to quality of care

– HC quality of monitoring evolved and improved

• Handwritten data displays on the wall• Automated data tables• Auto-generated analysis tools: bar

charts and trend lines

– Increased episodes of information feedback, ‘flowing downstream’

• Supplied reliable data to C-IMCI stakeholders

Bulletin Benefits to EIP

• Is it sustainable?

– Better inclusion of C-IMCI data in HIS

• Secondment of staff• Bulletin indicators integrated into national

HIS

– “I am certain that the bulletin will continue. It will be led by the health center Data Manager, and I will ensure this during my supervision visits to the health center, as well as the Data Manager at the hospital level.”

Bulletin Benefits to EIP• Contribute as advertised?

Served technical + motivational purposes

Provided reasonable set of standards, comparable overtime and between areas

Serves as “evolving learning tool” on producing and using information to support performance and quality of care

Met critical need for timely information usable at the local level

But…was not as ‘stand alone’ as originally hoped

Challenges

• Clear need for further improvement – Tool logic is okay – Redesign necessary for use at national scale

• Duplication of HIS – Is that outweighed by the rapid feedback of usable

information at the local level?

EIP Final Evaluation Findings

• Establishes local standards

• Information locally relevant– Utilization– Drug supply– Human resources– Coverage– Quality of treatment– Reporting

• “Real-time” automated signals

• Basis for advancing culture of quality

• “Mirror” of performance referred to by MOH

• Developed at low cost

• Duplication with HIS outweighing complementarity?

• Replication needs adaptation and re-linking to HIS

Strengths Weaknesses

Applicable for other contexts?

• Relevance – Where there are champions for improving the

quality of C-IMCI data and monitoring systems– Culture of learning at many levels

• Incentives

– One size/format does not fit all

• Manageability– Considerable demands on HC staff

• MoH and/or NGO training and ongoing technical support

– Hardware and electricity

• Adaptability– User-friendly format– Low cost– Start small, then scale

Relevance to CCM Benchmarks

1. Coordination and Policy Setting

2. Financing3. Human Resources4. Supply Chain Management5. Service Delivery and Referral6. Communication and Social

Mobilization7. Supervision and Performance

Quality Assurance8. M&E and Health Information

Systems

Advocacy and Planning

Pilot and Early Implementation

Expansion and Scale-Up

Different in

dicators,

simila

r platfo

rm

Take Away

Thank you

justine.landegger@rescue.org