design an “evaluable project” - cosvdesign an “evaluable project” ... 9defines project logic...

In collaboration with

Development of integrated project and financial

management tools in conformity with the 2003 FPA

Design an “evaluable project”

ECHO and the evaluability

ECHO and the ECHO and the evaluabilityevaluability

CarryingCarrying out of the mission in out of the mission in coco--ordinatedordinated way way bybyECHO and the NGOECHO and the NGOThe The CommissionCommission shallshall makemake the the draftdraft of the report of the report availableavailable to the NGO to the NGO forfor eventualeventual commentscomments

IfIf agreedagreed on, the on, the commentscomments of the NGO of the NGO shallshall bebeinsertedinserted in the Final Reportin the Final Report

Include Include evaluationevaluation costcost in budgetin budget

InsertInsert genericgeneric ToRToR in in Single FormSingle Form

PresentPresent specificspecific ToRToR to ECHO to ECHO beforebefore the mission the mission withwith Curriculum VitaeCurriculum Vitae of of evaluatorsevaluators

The The Commission shall carry Commission shall carry out out evaluations evaluations of the of the humanitarian operations financedhumanitarian operations financed (art. 22.1. (art. 22.1.

ProvisionsProvisions; ; art. 8 CG)art. 8 CG)

The The NGOs must make NGOs must make ex ante/ex post ex ante/ex post evaluations evaluations wherewhere possiblepossible (art. 22.2 (art. 22.2 ProvisionsProvisions))

PresentPresent EvaluationEvaluation Report Report withwith Final ReportFinal Report

Keep Keep a a constant monitoringconstant monitoring

ECHO and the ECHO and the evaluationevaluation

Control of Control of results based results based on the promotion on the promotion of best of best practicepractice

Application Application of HOLIS program of HOLIS program –– Standard Standard Operative Operative ProceduresProcedures

Possibility Possibility of of ““dede--selectionselection”” of underof under--performing partnersperforming partners ((monitoringmonitoring & & evaluationevaluation))

Introduces Introduces the the principle principle of of responsability towards responsability towards beneficiarybeneficiary, , donors donors and and partnerspartners

Introduces Introduces the the principle principle of of measurability measurability and and monitorabilitymonitorability

Obligatory insertion Obligatory insertion PCM, LF and PCM, LF and indicators indicators (project (project formsforms))

Allocation of Allocation of aid aid in a in a balancedbalanced and and verifiableverifiable way way for beneficiaryfor beneficiary

ACCOUNTABILITY ACCOUNTABILITY ((FromFrom ECHO ECHO AidAid StrategyStrategy 20032003))

Allocation of Allocation of aid aid in in balanced balanced way way for beneficiaryfor beneficiary

ConceptualConceptual work on a work on a methodologymethodology to allocate to allocate aidaid in a more in a more balancedbalanced way per way per beneficiary has progressedbeneficiary has progressed in 2002, in 2002, highlightinghighlighting, , howeverhowever, , problems withproblems with data data qualityquality. Data . Data availabilityavailability and and quality will therefore be addressed asquality will therefore be addressed as a a prioritypriority in the in the 2003 HOLIS work 2003 HOLIS work programmeprogramme..

Control of Control of results using results using ““best best practicepractice”” approachapproach

AdoptionAdoption of of measures focusingmeasures focusing on control of on control of results usingresults using a a ““best best practicepractice”” approachapproach. . CapacityCapacity building through the building through the disseminationdissemination of best of best practicespractices..

Application Application of HOLIS program of HOLIS program –– Standard Operative Standard Operative ProceduresProcedures

The establishment of final Standard Operating The establishment of final Standard Operating Procedures for crisis responseProcedures for crisis response in in humanitarian emergencies will also contributehumanitarian emergencies will also contribute to ECHOto ECHO´́s s qualityquality--based approachbased approachand and itsits operationaloperational management management capacitycapacity..

Obligatory insertion Obligatory insertion LF and LF and indicatorsindicators

In In orderorder to facilitate the to facilitate the monitoringmonitoring of of resultsresults, the , the objectivesobjectives of of an operationan operation and and the performance the performance indicators applicableindicators applicable to to measure achievements will become measure achievements will become constituent partsconstituent parts of the new project of the new project proposalproposal standard standard formsforms..

Possibility Possibility of Partner of Partner ““dede--selectionselection””

The The conceptconcept of of qualityquality partnership partnership also involvesalso involves a regular a regular monitoringmonitoring of of performance, performance, withwith the concrete the concrete possibilitypossibility of of ““dede--selectionselection”” of underof under--performingperformingpartnerspartners..

ACCOUNTABILITY ACCOUNTABILITY ((FromFrom ECHO ECHO AidAid StrategyStrategy 20032003))

Project Project presentationpresentation: : mandatory mandatory LF of LF of maximum maximum 3 3 pagespages

Final Report: the LF Final Report: the LF is adapted is adapted in light of the in light of the completed operationcompleted operation

ObligationObligation to to draw draw up a Log up a Log frameframe

Interim ReportInterim Report: the LF : the LF is adapted is adapted in light of in light of what what has been donehas been done

The partner The partner shall guarantee an internal monitoring shall guarantee an internal monitoring of the project, of the project, informing informing ECHO through the ECHO through the Interim Interim ReportReportThe base of the The base of the monitoring monitoring system system is given by is given by the the Work PlanWork Plan

Evaluation Evaluation

Monitoring Monitoring

AccountabilityAccountability of ECHO on money of ECHO on money spentspentLessonsLessons learnedlearned

ACCOUNTABILITY ACCOUNTABILITY ((FromFrom ECHO ECHO ManualManual Project Project CycleCycle

Management 2003Management 2003))

The “evaluable”

quality

The The ““evaluableevaluable””

qualityquality

Key pointsKey points expressedexpressed byby ECHO in ECHO in ““Qualitative Qualitative ApproachApproach””

CLEARNESS

Precise identification of the beneficiaries and the objective

Identify needs, problems, priorities and risks

Detailed analysis – elements defined (better if supported by Report)

Expected results in relation to defined indicators (SMART)

Realistic planning – efficient expense organization

TRUSTWORTHINESS

The Project(the Operation)

RESPONSABILITY

Continuous monitoring, ability to adapt, no over budgeting

Transparency of procedure, correctness of work

Attention to priority themes (Gender, Human Rights)

Quality control

CROSSOVER

Visibility and General Protocols and safety details

Project phases

Identification Feasability … Evaluation Passages project Logic

PCM PCM phasesphases withwith emphasisemphasis on qualitative on qualitative aspectsaspects of the of the constructionconstruction of the of the proposalproposal::

•• ProblemProblem analysis, single out analysis, single out objectivesobjectives//strategiesstrategies

•• Work plan, Work plan, monitoringmonitoring

•• EvaluationEvaluation, , best best practicespractices

Approval Comunication … LiquidationPassages ECHO procedures

PhasesPhases relative to project iter relative to project iter foreseenforeseen byby GC and GC and documentsdocuments::

•• Responsability Responsability and and maturity towardsmaturity towards ECHOECHO

•• Procedure, Procedure, statements statements and and mandatory documentsmandatory documents

Real definition phases and project management:

• Predisposition of statements

• Reference system (sinergies)

• Supervision e monitoring

Preparation Start … ClosurePassages Internal Projects

The project The project cyclecycle shallshall keepkeep in account the in account the variousvariousneedsneeds accordingaccording to to pointspoints of of viewview whichwhich mustmust bebe

consideredconsidered::

RespectRespect the the orthodoxyorthodoxy of the projectof the project

RespectRespect the ECHO the ECHO proceduresprocedures

RespectRespect internalinternal HO HO proceduresprocedures

FocalFocal pointspoints

FocalFocal pointspoints in in practicepracticeThe project The project

implicatesimplicates

From the proposal to the end focus on method

Qualitative approach

Demonstrate know how and knowledge of procedures

Evaluationand Monitoring

Plan the work from the initialphase

Preventive approach

The basic idea The basic idea isis……

Qualitative approach

Evaluationand monitoring

Preventiveapproach

The The ““perfectperfect””ProjectProjectperh

aps

Quality of Reports

QualityQuality of of ReportsReports

Qualitative Qualitative approachapproachbasic basic elementselements

The The ““FiveFivepointspoints””

RelevanceRelevance

EffectivenessEffectiveness

EfficiencyEfficiency

Impact Impact

SustainabilitySustainability


StakeholderStakeholderAnalysisAnalysis

PotentialPotential of of beneficiariesbeneficiaries ((numbernumber, , categorycategory, , typestypes), ), institutionsinstitutions, , locallocalgroupsgroups, , etcetc……

Limits (Limits (obstaclesobstacles))

InterestsInterests

PossiblePossible rolerole of stakeholders in the of stakeholders in the projectproject

StakeholdersStakeholders Analysis Analysis tooltool


ProblemProblem and and needsneeds analysisanalysis

IdentifyIdentify generalgeneral problemsproblems

IdentifyIdentify particularparticular problemsproblems and and needsneeds

IdentifyIdentify needsneeds of of beneficiariesbeneficiaries ((byby categoriescategories))

CorrelationCorrelation and and articulationarticulation of of mainmain elementselements((logiclogic of analysis)of analysis)

ProblemProblem TreeTree

Macro contest Macro contest shownshownbyby ECHO/CEECHO/CE

InterventionInterventionfocusfocus


Analysis of Analysis of objectivesobjectives

DefineDefine ((strategicstrategic) ) generalgeneral objectivesobjectives ((connectingconnectingstrategystrategy EU and HO)EU and HO)

IdentifyIdentify quantitative and qualitative quantitative and qualitative beneficiariesbeneficiaries

IdentifyIdentify objectiveobjective of the of the opertionopertion ((servicesservices givengiven to to target group)target group)

DefineDefine observtionobservtion criteriacriteria ((typetype of of needneed) and ) and selectionselection((inclusioninclusion--exclusionexclusion))

ObjectiveObjective TreeTree

Qualitative Qualitative approachapproachbasic basic elementselementsAnalysis of Analysis of

resultsresults and and activitiesactivities

DefineDefine generalgeneral frameworkframework of of servicesservices ((expectedexpectedresultsresults in line in line withwith budget)budget)

IdentifyIdentify activitiesactivities ((descriptiondescription, management, , management, humanhuman resourcesresources, , timestimes, , detaileddetailed descriptiondescription of of costscosts))

General General frameworkframework of of mainmain pointspoints relative to relative to implementationimplementation ((purchasepurchase/tender, /tender, safetysafety, , visibilityvisibility, , monitoringmonitoring plan)plan)

Qualitative Qualitative approachapproachbasic basic elementselementsAnalysis of Analysis of risksrisks

and and assumptionsassumptions

IdentifyIdentify ((evaluateevaluate) ) risksrisks inherentinherent to the to the actionaction

IdentifyIdentify ((evaluateevaluate) ) assumptionsassumptions

IdentifyIdentify mitigationmitigation meausuresmeausures

Intervention Logic

Objectively Verifiable Indicators

Sources of Verification

Risks and Assumptions

Principal Objective

Specific Objective

Results

Activities

Means

Costs

Pre-conditions

ReferRefer to the EC/ECHO to the EC/ECHO strategicstrategic frameworkframework

Log Log FrameFrame

ConsiderConsider costscosts and and organizationorganization of a of a monitoringmonitoring planplan

ReferRefer to the budget to the budget lineslines

CannotCannot bebe modifiedmodified, , referreferto ECHO decisionto ECHO decision

Imp!

Imp!

Imp!

Imp!

WorkflowWorkflow


QualityQualityfactorsfactors

OwnershipOwnership byby beneficiariesbeneficiaries

Policy Policy supportsupport (positive/negative)(positive/negative)

Appropriate Appropriate technologytechnology

SocioSocio--cultural cultural issuesissues

EqualityEquality of of allall ((especiallyespecially the the mostmost vulnerablevulnerable group)group)

InstitutionalInstitutional and management and management capacitycapacity

CoCo--ordinationordination betweenbetween the the differentdifferent actorsactors


Cross cutting Cross cutting issuesissues

LRRD LRRD ((towardstowards self self sufficiencysufficiency))

Gender (Gender (rolesroles, impact of project, , impact of project, womenwomen’’s s rolerole asas developmentdevelopment factorfactor))

HumanHuman RightsRights (impact on HR)(impact on HR)

SafetySafety

VisibilityVisibility

Qualitative Qualitative approachapproachbasic basic elementselementsIndicatorsIndicators and and

sourcessources of of verificationverification

ClearClear definitiondefinition

((quantityquantity, , relevancerelevance, , thresholdsthresholds, , minimum minimum standardsstandards))

FeasabilityFeasability of controlof control

RelevanceRelevance of project of project variablevariableobservedobserved

ReferenceReference DocumentsDocuments

Define and plan steps;

General indication of operative formalities and more critical points (personnel, tender, means, safety);

Experience, ideas, strategies.

Indication on UE/ECHO strategies on regional, national and local levels.

PCM synthesis (with attention to preliminaryanalysis and log frame construction);

Terms and synthesis of criteria for defining, monitoring and evaluating the projects.

PCM ECHO and PCM ECHO and Solint Solint manualmanual

Data Base and HO Data Base and HO praxispraxis

Country Country StrategyStrategyPaperPaper ECHO/UEECHO/UE

Defines procedures, times, and focal elements of work;

Elegible expences, purchases, visibility, responsability.

Indicates main variables for the defining, monitoring and evaluation of the activity(performance e benchmark);

Formation of log frame.

FPA FPA documentsdocuments

Sphere Sphere ManualManual

InstrumentsInstruments and and methodsmethods

Collects preliminary data;

Meetings with key actors;

Meetings (Focus Group) with privileged actors andstakeholders analysis.

RapidRapidsurveysurvey

Clear identification of beneficiaries and project components;Clear identification of critical points;Coherent base of project logic (attention to objectives, not activity)

Lists minimum elements for defining the proposal.

ProblemProblemTreeTree

ChekChek ListList

Defines project logic and analysis (deviation fromplan, hiring, indicators);

List key elements (objectives, results activity and indicators).

Log Log frameframe

Realistic analysis of times;

Initial Assessment and verification of responsabilityand focal moments of intervention.

WorkWork--flowflow((activityactivity plan)plan)

The end

The endThe end