crc, public affairs foundation, public health watch (phw) community monitoring workshop, march...

CRC, Public Affairs Foundation, Public Health Watch (PHW) Community Monitoring Workshop, March 23-25, 2010

HOLDING A MIRROR TO THE HOLDING A MIRROR TO THE GOVERNMENT!GOVERNMENT!

EXPERIENCES WITH CITIZEN REPORT CARDS

1


OVERVIEW OF PRESENTATIONOVERVIEW OF PRESENTATION

The Context – Why Citizen Report Cards?

The Concept – What is a Citizen Report Card?

The Method – How Citizen Report Cards are designed?

The Impact – Major outcomes of Citizen Report Cards

The Learning – Critical lessons & experiences

2


WHY CITIZEN REPORT CARDS?WHY CITIZEN REPORT CARDS?

Budget Formulation – How public resources are allocated

Budget Review & Analysis – Diagnosing the implications of the budget when formed

Expenditure Tracking – Seeing where the money goes

Performance Monitoring – Even after the money is spent, see how the output/service is performing

3


A School Report CardA School Report Card

Name: Gopa; Class: IXSUBJ ECT MAX.

Marks HighestMarks

Actual marks

English 100 82 82

Math 100 85 53

Science 150 123 76

Social Studies 150 125 125

Total 500 415 336

Rank in class: 16/30

4


LEARNINGS FROM A SCHOOL LEARNINGS FROM A SCHOOL REPORT CARD…REPORT CARD…

Power of MEASUREMENTPower of COMPARISON

An Opportunity for REFLECTIONA trigger for CHANGE & IMPROVEMENTS

5


Citizen Report Cards: Defining Citizen Report Cards: Defining FeaturesFeatures

Pioneered by Public Affairs Centre as an independent assessment in 1993

Credible user feedback on public services

Uses the power of measurement & comparison

Communicates findings in a very easy and focused manner

Report always in PUBLIC DOMAINPUBLIC DOMAIN

Not a one-off effort – continued benchmarking

6


CONTENT OF CITIZEN REPORT CONTENT OF CITIZEN REPORT CARDSCARDS

FEEDBACK FROM ACTUAL USERS OF SERVICES REGARDING

AVAILABILITY, ACCESS & USAGE OF SERVICES QUALITY & RELIABILITY INCIDENCE OF PROBLEMS & RESPONSIVENESS OF

SERVICE PROVIDERS COSTS - CORRUPTION & FORCED INVESTMENTS SATISFACTION WITH SERVICE REASONS FOR DISSATISFACTION SUGGESTIONS FOR IMPROVEMENTS

7


STAGES IN CITIZEN REPORT STAGES IN CITIZEN REPORT CARDCARD

DEFINING SCOPE OF ACTION

PREPARATIONS AND COLLECTING CITIZEN FEEDBACK

RATING OF SERVICES

DIALOGUE AND RESPONSE OF AGENCIES

CITIZEN ENGAGEMENT IN REFORM

PERIODIC BENCHMARKING AND PUBLIC REVIEW

8


THE “3” BIG QUESTIONS…THE “3” BIG QUESTIONS…

WHAT DO WE WANT TO KNOW?WHAT DO WE WANT TO KNOW? ASSESSMENT OF SERVICES WHAT COMMUNITY HAS TO SAY SINGLE SERVICE OR COMPARISON OF SERVICES

HOW CAN WE OBTAIN THE REQUIRED INFORMATION?HOW CAN WE OBTAIN THE REQUIRED INFORMATION? WHICH POPULATION CAN GIVE REQUIRED INFO WHAT QUESTIONS WE NEED TO ASK WHAT WILL BE THE SAMPLING METHOD WHAT LEVEL OF SKILL SHOULD FIELDWORKERS HAVE

HOW WILL WE USE THE INFORMATION?HOW WILL WE USE THE INFORMATION? WHAT CONCLUSIONS WILL BE DRAWN WHO WILL SEE THE REPORT WHAT ACTIONS WILL FOLLOW

9


METHODOLOGY FOR CARRYING OUT METHODOLOGY FOR CARRYING OUT CITIZEN REPORT CARDSCITIZEN REPORT CARDS

IDENTIFYING ISSUES THROUGH FGDs IDENTIFYING ISSUES THROUGH FGDs (PROVIDERS & USERS)(PROVIDERS & USERS)

DESIGNING THE SURVEY INSTRUMENTDESIGNING THE SURVEY INSTRUMENT FRAMING A SCIENTIFIC SAMPLEFRAMING A SCIENTIFIC SAMPLE CONDUCT OF SURVEYCONDUCT OF SURVEY CODING, ANALYSIS & INTREPRETATIONCODING, ANALYSIS & INTREPRETATION PRESENTATION OF FINDINGSPRESENTATION OF FINDINGS ADVOCACY & PARTNERSHIPS FOR SERVICE ADVOCACY & PARTNERSHIPS FOR SERVICE

IMPROVEMENTSIMPROVEMENTS

10


CITIZEN REPORT CARDS ARECITIZEN REPORT CARDS ARE……

BLENDING THE “SCIENCE” OF SURVEYS

WITH

THE “ART” OF ADVOCACY/REFORMS

11


CITIZEN REPORT CARDS IN BANGALORECITIZEN REPORT CARDS IN BANGALORE

THE POWER OF MEASUREMENT THE POWER OF MEASUREMENT

& COMPARISON & COMPARISON

12


First Report Card (1993): small experiment with focus on survey & minimal advocacy made quality of service a key governance issue

Second Report Card (1999): planned effort of PAC, survey followed by advocacy which enabledwide range of agency and govt. responsesgrowth in scope/scale of civil society action

Third Report Card (2003): PAC effortmajor improvement in service qualitymajor advocacy in progress

CITIZEN REPORT CARDS IN BANGALORECITIZEN REPORT CARDS IN BANGALORE

13


9

34

49

0

20

40

60

1994 1999 2003

Year

OVERALL SATISFACTIONOVERALL SATISFACTIONAll round overall improvement

% S

ati

sfi

ed

14


34 36

49

62

0

20

40

60

80

1999 2003

General Households Slum

OVERALL SATISFACTION – Poor Vs RestOVERALL SATISFACTION – Poor Vs Rest%

Sati

sfi

ed

15


6

45

71

4

4654

9

30

73

25

43

37

1

16 15 1418

65

0

20

40

60

80

POWER WATER TELECOM Govt Hospital URBAN ATH TRANSPORT

1994 1999 2003

AGENCY WISE SATISFACTIONAGENCY WISE SATISFACTION %

Sati

sfi

ed

agency16


31

65

33

57

41

55 52

74

29 29

0

20

40

60

80

POWER WATER Govt Hospital TRANSPORT MUNICIPAL

SERVICES

1999 2003

AGENCY WISE SATISFACTION - SLUMAGENCY WISE SATISFACTION - SLUM%

Sati

sfi

ed

agency

17


29

24

1211

0

20

40

1999 2003

Slum General Households

PROBLEM INCIDENCEPROBLEM INCIDENCE%

Sati

sfi

ed

Significant reduction in problems while using services

18


STAFF BEHAVIOURSTAFF BEHAVIOUR

Major improvement in satisfaction with staff behaviour

Trends in Satisfaction with Behaviour of staff

40 4448 56

0204060

1999 2003

Year

Per

cen

tag

e o

f re

spo

den

ts

sati

sfie

d

General Household

Slum

19


CORRUPTIONCORRUPTION

20

Коррупция по трем оценочным листам

19

9

2214

3225

0

10

20

30

40

1994 1999 2003

Год

% п

лат

ивш

их

Домохозяйства в целом Трущобы


APPLICATIONS OF CRCsAPPLICATIONS OF CRCs

As a “Diagnostic Tool”

As an “Accountability Tool”

As a “Benchmarking Tool”

21


22

BackgroundBackground

The pilot CRC project in Tajikistan was designed and implemented by United Nations Development Programme (UNDP) through a local partner –Zerkalo.

Public Affairs Foundation (PAF), Bangalore, provided technical assistance for the pilot project, which was funded by the UNDP.

Following a workshop with participation of UNDP Democratic Governance Programme, government agencies, and selected research company (El-Pikir Center for Public Opinion Study), a task force was set up to carry out the study in Kyrgyzstan.

Tajikistan- Opinions of Health Tajikistan- Opinions of Health Services by AspectServices by Aspect


23

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

Очень хорошо

Скорее хорошо

Скорее плохо

Очень плохо

Tajikistan- Overall Opinions on Tajikistan- Overall Opinions on Health ServicesHealth Services


Percent

Very good48.8

Somewhat good28.8

Somewhat poor2.0

Very poor20.5

24

Kyrgyzstan- The Overall Scorecard Kyrgyzstan- The Overall Scorecard on the Services of on the Services of

the Oblast Hospitalsthe Oblast Hospitals


4.1 4.193.98 4.13 3.98

4.16 4.2

3.593.78

4.1 4.03

0

1

2

3

4

5

Staff

Attitu

de

Cleanlin

ess of

war

d

Cleanlin

ess of

conv

enien

ces

Availa

bility o

f equipm

ents

Availa

bility o

f Med

icines

Care f

or pat

ients

Skill of

per

sonn

el

Distan

ce to

facil

ityCos

ts

Effecti

vess

of T

raet

ment

Overa

ll sco

re

Score out of 5

25

Bangalore- Public Hospitals Bangalore- Public Hospitals 1999 (general households)1999 (general households)


Client satisfaction with agency interactions

Percent satisfied (1994) 25Percent satisfied (1999) 43

Percent dissatisfied (1994) 19Percent dissatisfied (1999) 6

The bribery matrixProportion in sample claiming to have paid a bribe (%)

(1994)17

Proportion in sample claiming to have paid a bribe (%) (1999)

24

Average payment per transaction in Rupees (1994) 396

Average payment per transaction in Rupees (1999) 289

26


Bangalore- Public Hospitals Bangalore- Public Hospitals 19991999 (general households) (general households)

27

Улучшение услуг за последние три года

61

62

64

59

60

61

62

63

64

65

Аспект услуг

% о

твет

ивш

их,

что

ест

ь ул

учш

ени

е

Общее качество услуг Поведение персонала

Легкость взаимодействия

Bangalore- Public Hospitals Bangalore- Public Hospitals 2003 (general households)2003 (general households)


Indicator Service providerType of hospital visited in last I year State

governmentCorporation Private

% respondents (multiple responses possible)

89 10 12

Government hospitals (% respondents)Quality of staff service % satisfied Time to attend 83Overall behaviour of doctors 86Overall behaviour of nurses 100Overall behaviour of other staff 83Helpfulness of staff 75Service quality for in-patients % satisfiedCleanliness of room 31Quality of food 12Cleanliness of linen 12Overall satisfaction with service % satisfied Completely satisfied 37Partially satisfied 36Dissatisfied 2Bribe demanded 100%*Bribe paid 2%

28

Bangalore- Public Hospitals Bangalore- Public Hospitals 1999 (slum households)1999 (slum households)


29

Service provider (Public Hospitals) IndicatorSpeed Money Phenomenon

Percentage of those who paid 53Average amount paid (in Rupees) 640

Percentage of persons from whom bribe was demanded * 63Willingness to pay more for improved service

Percentage willing to pay more for better services 53Percentage willing to pay over and above the current payment 32

Improvement in services over last three years –peoples’ perception

Overall Quality of Services (%) 79Behaviour of Staff (%) 74Ease of interaction (%) 67

Client satisfaction with agency interactionsPercent satisfied 1994 80Percent satisfied 1999 73Primary health centre available

Percentage of respondents saying it is available 56Percentage of respondents saying they use it 84

Bangalore- Public Hospitals Bangalore- Public Hospitals 2003 (slum households)2003 (slum households)


30

Indicator Service providerType of hospital visited in last I year State

governmentCorporation Private

% respondents (multiple responses possible)

86 16 14

Government hospitals (% respondents)Quality of staff service % satisfiedTime to attend 69Overall behaviour of doctors 72Overall behaviour of nurses 73Overall behaviour of other staff 70Helpfulness of staff 67Service quality for in-patients % satisfiedCleanliness of room 80Quality of food 73Cleanliness of linen 70Overall satisfaction with service % satisfiedCompletely satisfied 55Partially satisfied 33Dissatisfied 12Bribe demanded 85%*Bribe Paid 9%


Bangalore Government Bangalore Government Hospitals-Quality/ReliabiliHospitals-Quality/Reliabili

tytySome Findings 99% slum respondents report doctor’s chamber as

clean. 34% report presence of doctors at the time of visit. 37% said all the required medicines were available. 11% report issue of expired medicines. 69% completely satisfied with the time taken. by the

government hospital staff to attend to them.Follow up Board of visitors constituted. Help desk for patients. Constant internal reform from the health department and

health authorities. 31


POWER OF DIAGNOSISPOWER OF DIAGNOSIS

SECTOR Governance was better this year

Governance was worse this year

Cannot Comment

Farmers/ Cultivators

19 6644 17

Labourers 14 47 39

Entrepreneurs/ Industrialists

7744 19 07

Rural 20 60 20

Overall 24 52 24

32


Corruption & The Urban PoorCorruption & The Urban Poor(SURVEY IN BANGALORE, 1999)(SURVEY IN BANGALORE, 1999)

SERVICE / AGENCY

PERCENT PAYING BRIBES

AVERAGE AMOUNT (RUPEES)

PERCENT CLAIMING BRIBE WAS DEMANDED

WATER SUPPLY

04 200 50

MUNICIPAL SERVICES

34 270 73

ELECTRICITY 10 250 100

RATION SHOPS

11 84 83

POLICE 46 4193 88

PUBLIC HOSPITALS

53 640 63

33


POST SURVEY ACTIONSPOST SURVEY ACTIONS

Pre-launch presentations to the providers & elected representatives

Inter agency workshop to stimulate learning and sharing good practices

Public release of findings & public forum – “open house” with agency heads

Sustained media advocacySupporting local champions

34


INSTITUTIONAL FORMSINSTITUTIONAL FORMS

Individual Civil Society Organizations – PAC & TIB

Civil Society Partnerships – People’s Voice Project, PANE

Independent Multi-Stakeholder Consortiums – Kenya, Tanzania

Governments – Internal (Vietnam) & Open (Delhi)

35


Counting and discussing instead of just shoutingPresenting strengths and weaknesses – “pat” &

“slap”Enables policy makers set policy prioritiesHelps agency managers assess service efficiencyProvides a bridge for civil society to dialogue on

citizens’ priorities Transmits the voice of the poor without

intermediation by representatives

MAJOR POINTS OF INFLUENCEMAJOR POINTS OF INFLUENCE

36


VARIETY IN IMPACT OF CRC’SVARIETY IN IMPACT OF CRC’S AGENCIES DISCUSS PERFORMANCE WITH CITIZENS IN OPEN

FORA IN BANGALORE SYSTEMATIC CITIZEN WATCH DOG ROLE IN LOCAL GOVT. IN

TERNOPIL, UKRAINE FORCING POLITICAL ACCOUNTABILITY IN MUMBAI SLUMS INDEPENDENT APPROACH TO MONITOR PRO-POOR SERVICES

IN ZANZIBAR, ETHIOPIA & TAJIKISTAN LOWER LEVEL OFFICIALS CITE FINDINGS TO SEEK FUNDS &

SUPPORT IN MUMBAI, INDIA POLITICAL LEADERSHIP ASKS FOR MORE DIRECT FEEDBACK

IN DELHI, INDIA REGULATORY BODIES SEEK INDEPENDENT VOICES IN KENYA BENCHMARK LOCAL SELF GOVERNMENTS IN INDIA

37

PLANNING FOR A CRCPLANNING FOR A CRCACTIVITY REQUIRED COMPETENCY TIME FRAME

Identifying Scope Knowledge of service provisionAccess to technical resources

2 Months

Conducting the survey Field work managementTrained pool of enumeratorsSupervising quality of survey

2 weeks - 2 months

Post survey analysis Data Entry & AnalysisAnalytical Report writing

2-3 months

Dissemination of findings & Advocacy

Stakeholder analysisCommunicating to different groups

1-2 months

Improving Services Ability to work with different stakeholdersImagination & creativity

2-6 months


38

crc, public affairs foundation, public health watch (phw) community monitoring workshop, march...

Documents

public services

public affairs foundation

public review

public resources

public affairs centre

performance monitoring

comparison of services

assessment of services