monitoring &...

Monitoring&Evalua/on

AriProbandari

LearningObjec/ves

•  Studentsareabletoexplaintheimportanceofmonitoringandevalua/onaprogrammanagement

•  Studentsareabletoapplyconceptsofmonitoringandevalua/oninaprogrammanagement

WhatisMonitoringandEvalua/on(M&E)?

GroupWork1

Dimension Monitoring Evalua2on

Frequency Periodic,occursregularly Episodic

Func2on Tracking/oversight Assessment

Purpose Improveefficiency,provideinforma/onforreprogrammingtoimproveoutcome

Improveeffec/veness,impact,valueformoney,futureprogramming,strategyandpolicymaking

Focus Inputs,outputs,processes,work-plans(opera/onalimplementa/on)

Effec/veness,relevance,impact,cost-effec/veness(popula/oneffect)

Methods Rou/nereviewofreports,registers,administra/vedatabases,fieldobserva/ons

Scien/fic,rigorousresearchdesign,complexandintensive

Informa2onsource

Rou/nesurveillancesystem,fieldobserva/onreports,progressreports,rapidassessment,programreviewmee/ng

SamePopula/onbasedsurveys,vitalregistra/on,specialstudies

Cost Consistent,recurrentcostsspreadacrossimplementa/onperiod

Episodic,oSenatthemidpointandendofimplementa/onperiod

TheGlobalFund.(2011).MonitoringandEvalua/onToolkits.

Monitoringgivesinforma/ononwhereapolicy,program,orprojectisatanygiven/me(andover/me)rela/vetorespec/vetargetsand

outcomes.Itisdescrip/veinintent.Evalua/ongivesevidenceofwhytargetsandoutcomesareorarenotbeingachieved.Itseekstoaddress

issuesofcausality.

Kusek&Rist.(2004).TenStepstoResultsBasedMonitoring&Evalua/onSystem.WorldBank.

M&E

Monitoring•  Clarifiesprogramobjec/ves•  Linksac/vi/esandtheir

resourcestoobjec/ves•  Translatesobjec/vesinto

performanceindicatorsandsetstargets

•  Rou/nelycollectsdataontheseindicators,comparesactualresultswithtargets

•  Reportsprogresstomanagersandalertsthemtoproblems

Evalua2on•  Analyzeswhyintendedresults

wereorwerenotachieved•  Assessesspecificcausal

contribu/onsofac/vi/estoresults

•  Examinesimplementa/onprocess

•  Exploresunintendedresults•  Provideslessons,high-lights

significantaccomplishmentorprogrampoten/al,andoffersrecommenda/onsforimprovement


WhyM&Eareneeded?

•  Toensurethatresourcesareusedeffec/veandefficient(input)

•  Toensurethattheprogramisimplementedasplanned(process)

•  Toinformaboutthelevelofachievedoutput,thereforecorrec/onsmaybeconductedifneeded(output/outcome)

•  Toassessimpactofprogram(impact)

HowdowedesignM&Esystemsinaprogrammanagement?

WhataspectsofM&Esystemsshould

beplanned?

PlanningofM&ESystems

Indicators,baseline,target

Datasource+Methodsofdata

collec/on

Analysis+Synthesis

Communica/on+Useof

informa/on

ResultsbasedM&E

Goal

Outputs

Outcome

Ac/vi/es

Input

Results

Implem

enta/o

n

Longterm,widespreadimprovementinsociety

Intermediateeffectofoutputsonclients

Productsandservicesproduced

Taskperssonelundertaketotransforminputsintooutputs

Financial,human,andmaterialresources


Example:ResultsbasedM&E

Goal

Outputs

Outcome

Ac/vi/es

Input

Results

Implem

enta/o

n

ToendTBepidemicinIndonesia

Increasethecaseno/fica/onrateannualytoachieveaminimuncasedetec/onrateof70%,by2019,as

comparedto33%in2014

34provincialleveltraining1na/onalworkshopontheguideline

IntensifiedCaseFinding(ICF)isimplementedinall34provinces

Guidelines,training,implementa/onofICF

Trainers,Trainee,DraSofguideline,etc

GoalofProgram

GlobalContext

Na/onalContext

OtherRelevantContext

Outcome

Situa/onanalysis

3

Situa/onanalysis

2

Situa/onanalysis

1

Indicators,Baseline,Target


IndikatoryangBaik

Clear Relevant Economic

Adequate Monitorable


M&EDataSource

AnnualReport

MonthlyReport

DailyRegister

IndividualMedicalRecords

Agregate

Detail


Designanevalua/onOvretveitJ.2003.Evalua/ngHealthInterven/on.OpenUniversityPress

Purpose

Whentheevalua/onis

undertaken:before/during/aSer?

Focusofevalua/onScope:

limitedorcomprehensive?

Methods

Evaluatorrole:internal/external

evaluator?HowoSen

Focusofevalua/on

Needs Demands Inputs Processes Outputs Outcome

Feasibility Process/Forma/ve Lookattheeffect

Methods

Quan/ta/ve Qualita/ve

Let’slearnfromaprogramevalua/onresearch

AssessmentofDOTSstrategyimplementa/oninhospitalsinIndonesia:2005-2007

AdiUtarini,AriProbandari,TrisasiLestari,HarySanjoto,KtutAs/,Agus/na,MohammadArifin

32

TheStudy

BurdenofTB TBservicesinhospital

DOTSstrategyimplementa/on

DOTSCentre Lab Districtwasor

Pa/ent Provider

ISTC

Prescrip/on

ThePerspec/ves

Service

Provider

Pa/-ent

Focusofevalua/on

Needs Demands Inputs Processes Outputs Outcome

Process/Forma/ve

Methods

Quan/ta/ve Qualita/ve

Processinexternallinkage

35

FreeTBdrugsandchoicesoffacili/esfortreatment

36

Sputumcollec/onandprac/cesintheward

37

“HIVpa/entswereputinthesameroomwithTB,TB-HIVorotherpa/entswithinfec/ousdiseases”(Fieldnotes)

38

UseofsecondlinedrugsinTBtreatment

Common pattern : -HRE-Cipro (Cat 1) -HRZE-Cipro (Cat 1) -Cipro (Cat II)

•  CategoryI,IIandExtra-pulmonaryTBCases:

– firstlinegenericdrugs(41-64%)

– FDCprovidedbyNTP(21-40%)

•  Children:

– firstlinegeneric(82%)

– first-linebranded(18%)

39

MostCommonSourceofTBdrugsusedbyTBpa/ents

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1 2 3 4 5 6 7 8

TypeDosageSecondline drug

DeficienciesinTBdrugsprescrib/onanddosageinTBAdultCases

ISTCStandard Criteria Hosp(%)

ChestClinic(%)

Standard1 Coughfor2-3weeksasTBsuspects 88.9 50.0Standard2 SputummicroscopyforTBdiagnosis:

Sputumtestcarriedout 94.4 100.0Atleast2specimensfordiagnosis 94.4 100.0

Standard3 Microscopyexamina/onforEP-TB 66.7 50.0Standard4 Sputumexamina/onforsugges/veofTB 94.4 100.0Standard5 DiagnosisofSS(-)TB:

Repeatsputumexamina/on 22.2 50.0Repeatsputumexamina/on&XRay 22.2 0.0Clinicaljudgement&XRay 33.3 0.0

Standard6 ScoringsystemforpediatrictTB 11.1 100.041


C.Clinic(%)

Standard7 Treatmentmonitoring 77.8 100.0Standard8 TBtreatment:

Firstlinedrugs 94.4 100.0Dossageconformtointerna/onalrecommenda/on

94.4 100.0

Standard9 Treatmentadherence:Treatmentsupporter 72.2 50.0Referralforpooradherence 83.3 100.0

Standard10 SStestformonitoringtreatment 88.9 100.0Standard11 Recordingallmedica/ons 88.9 100.0Standard12 TB-HIV:

HIVriskassessed 50.0 50.0ReferraltoVCT 77.8 50.0 42


C.Clinic(%)

Standard13 ARVtreatmentini/a/on:ARVandTBdrugsgivenconcurrently 55.6 50.0Cotrimoxazoleprofilaxis 50.0 50.0

Standard14 Monitoringofdrugresistance 44.4 50.0Standard15 DrugresistanceTB:

ConsulttoMDRexpert 72.2 100.0Effec/vefourdrugregimen 44.4 50.0

Standard16 Contacttracing 50.0 0.0Standard17 Repor/ngallTBcasestohealth

authori/es94.4 100.0

43

44

ISTCPublicHealthResponsibili/es


ChestClinic(%)

Standard16 Contacttracing 50.0 0.0Standard17 Repor/ngallTBcasestohealth

authori/es94.4 100.0

45

Findings

STRUCTURE

1.  Ownershipofhospitals

2.  Qualityassurancesysteminhospital

3.  TBcaseload4.  Trainedstaffs5.  DOTSteam6.  DOTSunit7.  NTPGuidelines8.  Financialincen/ves

fromDHOtohospitals

PROCESS

1.  Commitmentfromhospital

2.  Adherencetostandardofdiagnosisandtreatment

3.  Conversionrate4.  Caseholding

process

OUTCOME

1.  Treatmentsuccessrate

2.  Treatmentcomple/onrate

3.  Defaultrate4.  CoverageofDOTS

withinahospital

ProbandariA,UtariniA,Hur/gAK.AchievingqualityintheDOTSstrategyimplementa/onprocess:achallengeforhospitalPPMinIndonesia.GlobalHealthAc/on2008.DOI:10.3402/gha.v1i0.1831

Outpa/entUnit

TB suspects

Outpa/entUnit

HospitalLaboratory

DOTSunit

TBsuspectsdonotperformsputumtest

TBsuspectsperformsputumtest

TBcases NotTBcases

TreatedwithinthehospitalnotunderDOTS

unit

Refertootherhealthservicefacili/es

MedicalRecordReportICDXA.15-A.19

DOTSpaBentregister

Labregister

ReferralratefromIrawa/et.al.(2007)20-53%

LossoffollowupamongSS(+)TBcases:8-18%

46

ProbandariA,LindholmL,StenlundH,UtariniA,Hur/gAK.MissedopportunityforstandardizedtreatmentamongadultTBpa/entsinhospitalsinvolvedinPPM-DOTSinIndonesia.BMCHealthServicesResearch2010;10:113

Repor/ngM&EResults

•  Fullwrirenreport•  Execu/vesummary(1-4pages)•  Oralpresenta/on•  Policybrief

M&EResultU/liza/on

•  Mass-Media•  Internet•  E-government•  AnnualReport•  PublicHearingwithHouseofRepresenta/ve•  PublicDiscussion

Con/nuingM&E

•  Demand•  Clearrolesandresponsibility•  Trustworthyandcredibleinforma/on•  Accountability•  Capacity•  Incen/ves

•  Whatyouhavelearnedfromthissession?

monitoring &...

Documents