transforming the market for efficient lighting in …20te_… · transforming the market for...
TRANSCRIPT
TRANSFORMINGTHEMARKETFOREFFICIENTLIGHTINGINUKRAINE(TMEL)
UNDP/GEFPROJECT00076692(PIMS4175)
TERMINALEVALUATIONREPORTMarch31,2017
Preparedby:
MyK.Ton
PetroPavlychenko
TMELTerminalEvaluationReport–31March2017
2
ProjectData:TransformingtheMarketforEfficientLightinginUkraine
TitleofUNDPSupportedGEFFinancedProject
TransformingtheMarketforEfficientLightinginUkraine(TMEL)
UNDPandGEFProjectIDs UNDPProjectID:00076692
GEFProjectID:4175
EvaluationTimeframeandDateofEvaluationReport
Content Time
MeetingatthePMO 15December,2016
TEVisit 23January–February2017
PresentationofinitialTEfindings 3February2017
Reportdraftingandanalysis 20January–20February2017
SubmissionofDraftTEReport 21February2017
RegionandCountriesincludedintheProject
Region:CIS
Country:Ukraine
GEFOperationalProgram/StrategicProgram
GEFStrategicProgramNo.1,PromotingEnergyEfficiencyinResidential
andCommercialBuildings(SP-1)
GEFOperationalProgram:5:Removalofbarrierstoenergyefficiencyand
energyconservation.
ImplementingPartnerandOtherProjectPartners
UNDPUkraine
Government:MinistryofEcologyandNaturalResourcesUkraine,Private
Sector:MaxusFund(NGO),Epicentre(Retail)
DateofInceptionWorkshop 28November2011
DateofFirstDisbursement 10June2011
OriginalClosingDate 31January2016
RevisedClosingDate 31March2017
EvaluationTeam International:MyTon
National:PetroPavlychenko
TMELTerminalEvaluationReport–31March2017
3
AcronymsandAbbreviationsEE EnergyEfficient
CDR CombinedDeliveryReport
CFL CompactFluorescentLamp
CMU CabinetofMinistersofUkraine
CO2 CarbonDioxide
CTA ChiefTechnicalAdvisor
DehrzStandard StateStandardofUkraine
EA ExecutingAgency
GDP GrossDomesticProduct
GEF GlobalEnvironmentFacility
GHG GreenhouseGas
GOC GovernmentoftheCountry
GWh GigaWatt-hour
IA ImplementingAgency
IL IncandescentLamp
IO ImplementingOrganisation
kWh Kilowatt-hour
LED LightEmittingDiode
MDG UNMilleniumDevelopmentGoals
M&E MonitoringandEvaluation
MENR MinistryofEcologyandNaturalResources
MTR Mid-TermEvaluationReport
Mtn MillionTons
MWh Megawatt-hour
NAER StateAgencyonEnergyEfficiencyandEnergySaving
NEFCO NordicEnvironmentFinanceCorporation
NGO Non-GovernmentalOrganisation
NOx OxidesofNitrogen
OECD OrganisationforEconomicCooperationandDevelopment
PM ProjectManager
PMO ProjectManagementOffice
PB ProjectBoard
PSC ProjectSteeringCommittee
QA/QC QualityAssurance/QualityControl
RTA RegionalTechnicalAdvisor
SEIA StateEnvironmentInvestmentAgency
SanPin SanitaryRegulationsandStandards
SNiP ConstructionNormsandRegulationsofUkraine
SOx OxidesofSulfur
TE TerminalEvaluation
TMEL TransformingtheMarketforEfficientLighting
ToR TermsofReference
TWh TeraWatt-hour
UAH UkrainianHrivnya(localcurrency)
UNDP UnitedNationsDevelopmentProgramme
UNEP UnitedNationsEnvironmentProgramme
TMELTerminalEvaluationReport–31March2017
4
TableofContentsAcronymsandAbbreviations.......................................................................................................................3
TableofContents.........................................................................................................................................4
Acknowledgments........................................................................................................................................6
ExecutiveSummary......................................................................................................................................7
ProjectDescription...................................................................................................................................7
SummaryofConclusions,LessonsLearned,andRecommendations........................................................8
1. INTRODUCTION...................................................................................................................................13
1.1. PurposeoftheEvaluation............................................................................................................13
1.2. ScopeandMethodology...............................................................................................................13
1.3. StructureoftheEvaluationReport...............................................................................................14
2.PROJECTDESCRIPTIONANDDEVELOPMENTCONTEXT.........................................................................16
2.1. ProjectStartandDuration...........................................................................................................17
2.2. ProblemsthattheProjectSoughttoAddress...............................................................................17
2.3. ImmediateandDevelopmentObjectivesoftheProject...............................................................17
2.4. MainStakeholders........................................................................................................................18
2.5ExpectedResults...............................................................................................................................19
3. FINDINGS.............................................................................................................................................21
3.1. ProjectFormulation&Design......................................................................................................22
3.2. ProjectImplementation................................................................................................................26
3.3. ProjectResults..............................................................................................................................38
4. CONCLUSIONS,RECOMMENDATIONS&LESSONS.............................................................................66
4.1. Conclusions...................................................................................................................................66
4.2. LessonsLearned...........................................................................................................................68
4.3 Recommendations........................................................................................................................70
Annexes................................................................................................................................72
ListofFiguresFigure1.StakeholdersintheProjectDesignProcess__________________________________________________19
Figure2:ProjectManagementStructureofTMEL ___________________________________________________27
Figure3.TMELManagementStructure(Organigram)________________________________________________28
Figure4.EstimatedProjectCO2Impacts___________________________________________________________65
TMELTerminalEvaluationReport–31March2017
5
ListofTablesTable1.Year-WiseDistributionofSub-Contracts,byComponent..............................................................................32
Table2.CommittedProjectFundingandSources.......................................................................................................33
Table3.ApprovedProjectBudgetandDistribution,byYearandbyComponent.......................................................34
Table4.ActualBudgetExpenditure,byYearandbyComponent...............................................................................34
Table5.ActualExpendituresComparedtoOriginalBudgetAllocation(in%)............................................................35
Table6.ProjectImplementation/StaffingCosts,perComponent...............................................................................36
Table7.Component1:Outputs,Indicators,Accomplishments,andRatings..............................................................39
Table8.Outputs,Indicators,Accomplishments,andRatings.....................................................................................45
Table9.Component3:Outputs,Indicators,Accomplishments,andRatings..............................................................49
Table10.Component4:Outputs,Indicators,Accomplishments,andRatings............................................................52
Table11.Component5:Outputs,Indicators,Accomplishments,andRatings............................................................58
Table12.SummaryandOverallProjectRating...........................................................................................................61
TMELTerminalEvaluationReport–31March2017
6
AcknowledgmentsTheEvaluationTeamwishestoexpressourgratitudetoUNDPUkrainestafffortheirinput,aswellas
hospitalityandsupportoftheTeaminthepreparationofthisTerminalEvaluationreport.Wealso
appreciatetheirusefulcommentsandsuggestionstoimprovethequalityofthisfinalversion.
TheProjectOfficeStaffmembersrespondedtotheTeam’smultiplerequestsforinformationduringthe
courseofthemissionandafterwardsinaprofessionalandtimelymanner,andtheircooperationisvery
muchvalued.Inaddition,thanksareextendedtotheUNDPRegionalTechnicalAdvisor,UNDPUkraine
HeadofEECluster,ChiefTechnicalAdvisor,andtheNationalEvaluator,whoprovidedcomments,
guidance,andinputtoenhancedetailsofthereport.
ThanksarealsoduetovariousrepresentativesoftheCitiesofMykolayivandSumy,OdessaOblast,
OdessaRegionalStateAdministration,CityandMunicipalGovernmentsofOdessa,theiradvisors,
stakeholdersfromthePrivateSector,NGOs,aswellasotherswhohavemetwiththeEvaluationTeam–
wearegratefulforyourtime.
TMELTerminalEvaluationReport–31March2017
7
ExecutiveSummary
ProjectData:“TransformingtheMarketforEfficientLightinginUkraine”
GoaloftheProject
ToreducetheannualgrowthrateoflightingelectricityuseandresultantGHG
emissionsfromtheUkrainianpublicandresidentialsectors.
ObjectiveoftheProject
Theenhancedpromotionandimplementationoftheutilizationofenergyefficient
lightinginUkrainethroughthetransformationofthelocallightingmarket,products,
andthephasing-outofincandescentlampproductionandsale.
MajorComponentsand
Focus
TMELconsistsoffivespecificcomponentsandoutcomes:
• Component1:Prepareandset-upnationalpolicyframeworktopromoteEE
lighting:focusesonimprovingthenationalpolicyframeworkforpromoting
energyefficiencyandenergyefficientlightingandthehandlinganddisposalof
spentmercurycontaininglamps.
• Component2:ImproveQA/QCframeworkforEElightingmarket:focuseson
settinguptheframeworkfornationalqualityassuranceandqualitycontrol
systemsforimportedanddomesticlightingproductsinUkraine.
• Component3:Efficientlightingdemonstrationinmunicipaleducational
sector:focusesonprovidingtechnicalassistanceandincreasingknowledgeof
EElightinginthissector.
• Component4:ImproveEELightingproductpenetrationintheResidential
Sector:addressestheinformationandavailabilitygapforEElightingamong
consumers.
• Component5:ReplicationandDisseminationoftheProjectResults:focuses
onprojectsustainabilitymeasures.
ProjectBudget GEFFund USD$6,500,000
GovernmentofUkraineCo-financing USD$19,375,000(Direct)
USD$1,600,000(in-kind)
PrivateSectorCo-financing USD$3,275,000
OtherSources USD$250,000(UNDP)
TotalCommittedFunding USD$31,000,000
ProjectDescriptionTheoverallgoaloftheTMELprojectwastoreducetheannualgrowthrateofGHGemissionsfromthe
Ukrainianpublicandresidentialsectors.Theprojectobjectivewastheenhancedpromotionand
implementationoftheutilizationofenergyefficientlightinginUkraine,throughthetransformationof
thelocallightingproductsmarketandthephasing-outofincandescentlampproductionandsale.The
TMELprojectrepresentedaninnovativeapproachtoenergyefficiencyprojects,andcomprisedofan
importantpartofUNDP-GEFportfolio,whichwouldcomplementandbuilduponthelessonslearned
fromothersimilarUNDP-GEFmarkettransformationprojectsinRussia,RomaniaandVietnam.
TMELTerminalEvaluationReport–31March2017
8
SummaryofConclusions,LessonsLearned,andRecommendationsNote:TheEvaluationTeambelievesthatthechallengesencounteredbytheTMELprojectover
thecourseofitsimplementationandthelessonslearnedwarrantalongernarrativethanisusual
withanexecutivesummary.
CONCLUSIONS:Ingeneral,acasecanbemadethattheTMELprojectwasabletodelivermostofits
plannedactivitiesandoutputs.However,evenwithinitialinterestandcommitmentbytheUkrainian
Government,andtheproject’soutputs,notalldesiredoutcomeswereachieved.Challengesremainin
thefinalstagesofimplementationfortheproject’sfulloutcomestoberealized.
Overthecourseofitsimplementation,theTMELprojectexperiencedsignificantchallenges,including:
• Changingnationalpoliticalandeconomicconditions,includingachangeingovernment,
• Worseningsecuritysituationsinsomeareasofthecountry(armedconflictintheeasternareas),
• Acollapsedinternational(andnational)carbonmarket,and
• Rapidtechnologicalshiftstowardsmoreadvancedandefficientlightingproducts(and
correspondingpricereduction).
Despitetheseexternalchallenges,theTMELprojectwasabletoexerciseadaptivemanagement,
achievedmostofitsoutputs,anddeliveredonanumberofareasnotcoveredbytheprojectdocument,
includingclosecoordinationwithmunicipalitiesandcitiestoeffectthephaseoutofinefficientlamps.
Possiblyduetothescaleandambitionoftheproject’sobjectives,aswellasthefactthatTMELwasthe
firstfull-scale,markettransformationprojectinUkrainewithcomplex,interlinkedoutputs,therewere
alsointernalandimplementationchallenges.Factorscontributingtoreduceoperationaleffectiveness
includedinconsistentadministrativesupport,lackoftechnicalinput,andchangesinkeystaff:TMELwas
managedbythreedifferentprojectmanagersandwasadvisedbyatleasttwoInternational
Consultants/TechnicalAdvisors(CTA)overthespanofsixyears.NeitherCTAwasengagedfromthe
outset,norweretheyfullyutilizedinallprojectaspectsasenvisionedbytheprojectdocument.
Thesizeoftheprojectteam,aswellascompositionandskillsetsoftheteamresultedinincreasedcosts
andreducedefficiencyfortheproject(22%ofprojectbudgetwasspentonstaffing).Inaddition,the
teamresponsibleforPartnershipsandRelationswithGovernmentandNationalAuthoritieshada
separatereportingstructure.Whilethisstructuremayhaverecognizedtheimportanceandsensitivityof
thisteam’stasks,itallowedforlesscoordinationbetweentheteamleaderandprojectmanager,and
limitedinteractionswiththeCTAaswelltherestoftheprojectteamonmattersrelatedtoproduct
testing,qualityassuranceframework,aswellasinternationaloutreach,amongothers.WithoutaCTA,
theprojectcommissionedextensiveresearchonitsownonawidearrayoftopicsratherthanbuilding
from,andcoordinatingwith,internationallyavailablework.Theconductionofthesestudiesdelayed
neededactivities,andreducedoverallprojectcosteffectiveness.
Withrespecttoprojectexpendituresandbudgetmanagement,theprojectexercisedadaptive
managementbyredirectingspendingfromtheoriginallyplannedbudget.Thecontinuedemphasison
increasingawarenessandmarketpenetrationshowedgoodresults.However,thefactthatspendingon
pilotsanddemonstrationswasmuchlessthanoriginallyplanned(6%actualvs.28%planned)wasa
TMELTerminalEvaluationReport–31March2017
9
significantshift.Italsoindicatedthattheprojectneededbetterfeedbackmechanismsforusingmarket
intelligence,asthereremainhighlevelsofinterestforpublicdemonstrationsandpilotprojectsamong
municipalitiesnotaffectedbythesecuritysituation,andadditionalspendingcouldhavebeenjustified.
Theproject’sinformationalcampaignswereitsbrightpoints,andresultedinthecreationofanumberof
highlyvisiblemessagesforUkrainianconsumersonenergyefficientlightingandclimatechange.
However,itmustbenotedthattheprojectinvestedsignificantresourcesforgeneralawarenessraising,
andcontinuedtodosoevenafterithasachievedsignificantgainsandadvisedbytheMid-term
Evaluationtorefocus(spendingonawarenessandeducationaccountedfor42%oftotalcomponent
activities).Intermsofmessagelongevityandprojectsustainability,itmayhavebeenmoreeffective
onceincreasedawarenesswasachievedtocreatemorelinkageswithotherprojectoutputsanddesired
outcomes,suchasqualitycontrolmechanismsorframework,orconnectingthecharacteristicsofenergy
labelsandproductperformancewiththeneedfortestingandverification.Thisisespeciallyimportant
sincethetechnologicalshifttolight-emittingdiode(LEDs)occurredmuchfasterthananticipated,
resultinginincreasedavailability,penetration,andmorecompetitivepricing,butalsoissuesofquality,
andhowtoproperlymeasureprojectimpactsinthisarea.
Inconclusion,theEvaluationTeamhasdeterminedthattheTMELprojectdesignhasremainedhighly
relevanttothedevelopmentcontextofUkraineandtheprioritiesofvariousstakeholders,includingGEF,
UNDP,municipalgovernments,cities,schools,testlaboratories,andtheprivatesector.Itscombined
outputshavemettheGEF’sguidelinesforaModeratelySatisfactoryproject(theprojectincurredsome
moderateshortcomings,includinglackofoutputprogressoncertaincomponents,andinconsistent
projectprogress,resultingintheneedforanextension).
LESSONSLEARNED:Basedonconsultationswithkeystakeholdersandtheconclusionsdrawnbythe
EvaluationTeam,somekeylessonslearnedfromtheTMELprojectdesignandimplementationinclude:
• Interest/supportbytheappropriategovernmentagency(ies):Theprojecthasdemonstrated
thatfullsupportbyrecipientGovernmentoftheCountry(GOC)andcooperationbetween
relevantministries/departmentsarenecessarytoachievetheintendedoutcomes(forexample,
morethanonegovernmentministriesisneededtofacilitatelegislationadoption);
• Privatesectorengagement:Engagementwithprivatesectorisnecessaryforachievingmarket-
relatedgoals(forexample,workingwithEpicentreonproductpromotion,includingtrainingof
salesstafftorealizeincreasedpenetrationandawarenessofefficientlightingproductsatretail);
• Supportawarenesswithavailability:Inmediumandsmallcitiesandruralareas,having
productsavailabletoconsumersinconjunctionwithawarenesscampaignscansignificantly
increasemarketshare.
• Interlinkmessages:Effortstoincreaseawarenessofenergyefficientlightingcanbecombined
with,orfollowedbyfocusedmessagesonprojectoutputsanddesiredoutcomes,suchasquality
controlmechanismsorframework,orconnectingthecharacteristicsofenergylabelsand
productperformancewiththeneedfortestingandverificationtoincreaseeffectiveness.
• Active,engaged,andcomprehensivePSC:AnactiveandengagedPSCrepresentingawide-
ranginggroupofstakeholdersisneededtoappropriatelyaddressthechallengesandrisks,as
TMELTerminalEvaluationReport–31March2017
10
wellasdeterminingwhentoreviseoradoptnewobjectivesbasedonchangingmarketor
politicalconditions.
• Updateprojectobjectives:Projectsmaytakelongerthanexpectedtobeapproved,asareproposedlegislation,andtechnologicalorpoliticaldevelopmentsmayhappenalmostovernight.
Suchsituationsmayrequiretheprojectplanstobemodifiedtoaddressnewrealities.
• Betterfeedbackmechanismsforusingmarketintelligence:Ahighlevelofinterestforpublic
demonstrationsandpilotprojectsexistedduringthecourseoftheprojectimplementation,yet
thislevelofinterestwasnotreflectedintheannualplanningprocess,whichcouldhelpto
increasefocusadditionalspendingforpilotsinplaceofawarenessraisingactivities.
• Administrativecommunicationandcoordination:Communicationandcoordination
arrangementsareessentialtosupportprojectplanningandimplementation,especiallyunder
changingmarketandpoliticalsituations;
• Administrativesupport:Projectsstartingupmayrequirestaffingadjustmentsorother
administrativesupport,suchasdevelopmentoftendersorsecuringshort-termconsultants.A
goodrelationshipbetweentheprojectadministrationandimplementationteamscanbe
valuableinhelpingaprojecttoachieveitsinitialsuccesses;
• ComprehensiveM&E:AnM&Esystemthatfocusesonallkeyprojectaspects,includingco-
financing,sub-contracts,andimpactindicatorsisessentialtoassessingaproject’sprogressand
impacts;
• Teamcompositionandskillsetsareimportantfactors:Itisimportanttomatchtheteam
memberskillsetswiththeprojectrequirements.Markettransformationprograms,forexample,
tendtorequireteammemberswithentrepreneurialtraits,whocanunderstandoradaptquickly
tochangingmarketsituations.Technically-focusedprojects,ontheotherhand,canrequire
teammemberswithdeeptechnicalknowledgeandexperience.
• Clearreportingstructure:Aclearreportingstructureshouldbeapparenttoallteammembers,
andopportunitiesforcoordinationandcooperationcross-sector/objectivesshouldbe
encouraged.
• Adaptivemanagementpracticesshouldbeencouraged:Projectstendtoexperiencemarket
shifts,butafewmaybeaffectedbymorethanone“game-changing”factors.Inthesecases,
projectsareforcedtoadaptinordertoremainrelevant.Suchexamples,whereavailable,should
besharedandreviewedtolearnfrom,ifpossible.
• Cleardocumentation:Instancesofsignificantadjustmentstobudgetoroutputdeliveriesneed
tobeclearlydocumented,withjustificationsandapprovals.Thisiscritical,andcanbehelpful
especiallywithchangesinprojectmanagement.
• Technicalsupportbyinternationalexperts:Forprojectsthataimtotransformthemarketfor
productsorservices,especiallytopioneeranapproachthatcanbeusedbyotherprojects,itis
essentialtosecuretheservicesofanexperiencedinternationalexpertfromtheoutset.Suchan
expertshouldhaveexperienceintechnicalissuesaswellasprevioushands-onexperiencein
markettransformationprojectsandcanbevaluableforimplementationinput.
• Drawfrominternationalbest-practicesfromtheoutset:Securingtheservicesofexperiencedinternationalexpertscanalsoprovideprojectswithaccessandunderstandingtobest-practices
TMELTerminalEvaluationReport–31March2017
11
internationally.WhileprojectevaluationandotherdocumentationavailablefromUNDPand
elsewherecanprovideusefulinformation,findingandaccessingthemcanbeachallengetonew
projectteamslearningnovelapproaches,andunderstandingthisinformationwithoutthehelp
ofaninternationalexpertcanaddanotherlayerofcomplexity.
• Marketresearch:Internationalbest-practiceinformationcanbefurthersupplementedwith
researchfocusingonthelocalmarket’sparticularcharacteristicsorculturalpreference,which
canincreaseeffectivenessandreach.
RECOMMENDATIONS:Basedontheaboveconclusionsandthelessonslearned,theEvaluationTeam
recommendsthefollowingactions:
1. ReplicationandUp-Scaling:Theprojecthasmadesignificantcontributionstotheawarenessof
energyefficiencyandenergyefficientlightinginUkraine’sconsumer,localandmunicipal
governments.Toensurethesustainabilityofthiscontribution,theoutputsinthisareaneedto
bemaintainedandcontinuetobedisseminated.Theconversionofretailersandmanufacturers
toenergyefficientlighting,aswellasadoptionofdecreesbymunicipalandcitygovernments
needtobefurtherpromotedandadoptedbyothers.
2. DocumentationandDisseminationofResults:Theprojecthasmadesignificantcontributionsto
thedevelopmentoftheEElightingindustryandlightingknowledgebyundertakingconsumer
research,developingpilots,andfacilitatingtechnologytransfer,etc.Forfutureeffortsand
projectstofullyutilizetheseproductsaswellasthelessonslearned,itisimportantthatthe
projectcandocumentanddisseminatesitsresults,achievements,andlessonslearnedinmarket
transformation.UNDPshouldconsiderkeepingtheprojectwebsiteortransferittothe
maintenanceofanotherproject.
3. StakeholdersCollaboration:Toensureeffectiveplanningandimplementation,itisimportantto
haveopencommunicationlinesbetweenkeystakeholders.Toavoiddelaysinimplementationin
thefuture,theUNDPandPMOneedcommunicateopenlytoaddressissuesrelatedto
implementation,suchasunfamiliarapproachortime-sensitiveactivities.
4. TrackingCo-FinancingandImpacts:Duetochangingpoliticalsituationsandotherchallenges,thePMOwasnotabletofullytrackandjustifyprojectco-financing,orfullyassessedproject
GHGimpacts.Thissituationshouldbeaddressedinotherfutureprojects.
5. AttainmentofOutcomes:GiventhemultiplechallengesfacedbytheTMELprojectduringthe
implementationprocess–whichmaybeunlikelytobefacedbyanotherproject–wesuggest
thatUNDPtaketheseconditionsintoconsideration,andconsiderthedevelopmentofamore
exhaustivelistingofrisksforUkraine,aswellasmoredetailedriskmitigationstrategiesfor
designofprojects,andconsiderwaysforactiveinvolvementbythePSCinthesecasestohelp
addressanyrequiredchangesinprojectdesignsoroutputsinamoretimelymanner.
TMELTerminalEvaluationReport–31March2017
12
SpecifictothedesiredlegislativeandQAframeworkoutcomes,wesuggestthattheprojectandUNDP
exploreallvenuesforcooperationwiththeMinistryofRegionalDevelopment,Construction,Housing
andCommunityServicessothattheseobjectivescanbesustained.Giventhattheprojecthasprovided
allofthenecessarylegislativeoutputsandNationalRoadmap,aswellassupportfortheQAframework,
itwouldbebeneficialforallinvolvediftheMRDCHCScanbepersuadedtocontinuetopursueprogress
towardstheseimportantoutcomes.
EvaluationRatings
1.MonitoringandEvaluation Rating 2.IA&EAExecution Rating
M&EDesignatEntry S QualityofUNDPImplementation MU
M&EPlanImplementation MS QualityofExecution–ExecutingAgency MS
OverallQualityofM&E MS OverallQualityofImplementation/Execution MS
3.AssessmentofOutcomes Rating 4.Sustainability Rating
Relevance R FinancialResources MU
Effectiveness S Socio-Political ML
Efficiency MS InstitutionalFrameworkandGovernance L
OverallProjectOutcome MS Environmental L
Overalllikelihoodofsustainability ML
TMELTerminalEvaluationReport–31March2017
13
1. INTRODUCTION
1.1. PurposeoftheEvaluationInaccordancewithUNDPandGEFM&Epoliciesandprocedures,allfullandmedium-sizedUNDP-
supportedGEF-financedprojectsarerequiredtoundergoaterminalevaluationuponcompletionof
implementation.
ThisTerminalEvaluation(TE)seekstofulfillthefollowingoverarchingobjectivesofthemonitoringand
evaluationofGEFprojects:
• PromoteaccountabilityfortheachievementofGEFobjectivesthroughtheassessmentofresults,
effectiveness,processesandperformanceofthepartnersinvolvedinGEFactivities.GEFresults
willbemonitoredandevaluatedfortheircontributiontoglobalenvironmentalbenefits;
• Promotelearning,feedbackandknowledgesharingonresultsandlessonslearnedamongthe
GEFanditspartners,asbasisfordecision-makingonpolicies,strategies,programmanagement,
andprojectsandtoimproveknowledgeandperformance.
1.2. ScopeandMethodologyThisTEcoverstheentireUNDP/GEF-fundedprojectnumber00076692,knownasTransformingthe
MarketforEfficientLighting(TMEL)forUkraine,anditscomponentsaswellastheco-financed
componentsoftheproject.
TheTerminalEvaluationoftheTMELProjectwascarriedoutattheprojectlevelandcomponentlevel.
TheTEisanassessmentoftheprogresstowardsachievementoftheprojectoutcomesandoutputs,the
relevanceofthevariousprojectoutputs,andeffectivenessandefficiencyofthedifferentactivities
undertakentoachievetheoutputs.Inaddition,theprojectinputsintheformofcontributionsmadeby
theUNDPanditsimplementingpartners,wereassessedfortheappropriatenessandeffectivenessof
thepartnershipstrategyutilized,andsustainabilityoftheproject’soutcomesandoutputs.
TheEvaluationTeam(theTeam),consistingoftheInternationalEvaluatorandNationalEvaluator,
carriedoutvariousactivitiesduringthecourseoftheevaluation,including:
• Literaturereview
o Reviewandassessmentofselectmajorresearchstudiescommissionedbytheproject1
• DevelopmentofanInceptionReportandevaluationtools
• Meetingswithprojectstakeholders.
I. DevelopmentofEvaluationTools
Inordertounderstandofthescope,objectives,andcomplexityofthisproject,theEvaluationTeam
begantheTEwithadetailedreviewoftheproject-relateddocuments,andanevaluationinceptionvisit
tookplaceinDecember2016.2Duringtheinceptionmission,theTeammetwiththePMO,theRegional
1AlistingoftheresearchstudiescommissionedbytheprojectisprovidedinAnnexG.2AcompletelistofdocumentsreviewedduringthecourseoftheassignmentisprovidedinAnnexA.
TMELTerminalEvaluationReport–31March2017
14
TechnicalAdvisor,andprojectstaffresponsibleforthevariousprojectoutputsandactivitiesinorderto
obtainanoverviewoftheproject’simplementationmechanismsandassociatedchallengesand
opportunities.Basedonthedocumentreviewandinceptionvisit,theTeamdevelopedadetailed
programmaticandgeographicscopeoftheevaluationactivities,evaluationvisits,aswellassample
interviewguidesforinterviews.
Theproposedevaluationmethodology,developedinterviewtools,andscheduleofevaluationwere
sharedwiththeUNDPandPMOintheformofanInceptionReport.
II. CountryMissionandFieldVisits
TheTECountryMissionandFieldVisitswereconductedfrom23Januaryto3February2017tovarious
locationsinandaroundKyiv,aswellasMykolaiv,Odessa,Potlava,andSumy.DuringthisMission,the
Teamworkedtogethertoreviewadditionaldocuments,conductinterviews,sitevisits,andpreliminary
analyses.3
TheTeamdevelopedtwointerviewguides(listofquestions)foruseduringthecourseoftheevaluation
visits.Ashorterguidewasdevelopedforusewithvariousprojectstakeholders,partners,andsub-
contractors,etc.AlongerinterviewguidewasdevelopedforinterviewswiththeProjectManagersand
otherrelevantprojectstaff.4Theseguideswereusedininterviewswithprojectstaff,stakeholders,
includingUNDP,MinistryofEcologyandNaturalResourcesofUkraine,andMinistryofRegional
Development,Construction,HousingandCommunityServices,UNDPSmallGrants,sub-contractors,and
others.5
III. MissionDebriefing
AttheendoftheTEmission,theTeammetwiththeUNDPPMOon3Februarytodeliveranoverviewof
theinitialfindings.AnInceptionReport,andInceptionReportBriefwithinitialkeylessonslearnedwere
alsodeliveredtoUNDPPMOon20December2016.
1.3. StructureoftheEvaluationReportAdraftreportwasdevelopedforUNDPreviewusingtheoutlineprovidedbyGEF’sEvaluationGuidance
document.6Thedraftreportpresentedaninitialanalysisoftheinformationgatheredfromliterature
review,interviews,discussions,andsitevisits.
PerUNDP’sguidanceonterminalevaluations,thedraftreportcoveredthekeycriteriaofrelevance,
effectiveness,efficiency,sustainabilityandimpact.Inaddition,ratingsbasedontheobligatoryrating
scalesareprovidedforthefollowing:
• MonitoringandEvaluation
• IA&EAexecution
3AdetailedmissionscheduleispresentedinAnnexC.4TheinterviewguidesareincludedinAnnexB.5AcompletelistofstakeholdersinterviewedduringtheTEisincludedinAnnexC.6GuidanceforConductingTerminalEvaluationofUNDP-Supported,GEF-FinancedProject.UNDPEvaluationOffice2012
TMELTerminalEvaluationReport–31March2017
15
• Assessmentofoutcomes
• Sustainability.
ThedraftreportalsoincludedananalysisoftheProjectFinanceandCo-finance,Mainstreaming,and
Impacts.Toassessprojectfinances,theprojectcostandfundingdatawasanalyzed,theplannedand
actualexpenditureswerepresentedandthevariancesbetweenthetwowereassessedandexplained.
ThereportincludesdetailedConclusions,Recommendations,andLessonsLearnedfromtheproject
implementationexperienceinordertoinformUNDP,GEF,andvariousstakeholdersaswellastobenefit
futureprojects.InadditiontothisTEreport,theprojectmanagerandinternationalCTAarealso
producingalessonslearnedstudytohelpbetterinformfutureprojects.
TMELTerminalEvaluationReport–31March2017
16
2.PROJECTDESCRIPTIONANDDEVELOPMENTCONTEXTUkrainehasoneofthehighestGHGemissionslevelperunitofGDPamongCIScountries,withaper
capitaemissionof9,45tonsofCO2equivalentperyear.7Consequently,Ukraineranks19thamongthe
world’slargestemittersofGHGs.Inaddition,UkraineisoneofthecountriesinEuropewiththelowest
energy-efficiency.TheUkrainianenergysectorcontributes69%ofoverallGHGemissions(299,7million
tons),includingtheemissionsfromelectricityproduction,whichin2007amountedto101,7milliontons
ofCO2equivalent.Thehighshareofcoal-firedthermalpowerplantsinelectricityproductionandhigh
lossesinelectricitydistributiongridcombinestogiveUkrainearelativelyhighCO2emissionfactorper
MWhofproducedpower1.031tonsCO2/MWh2.
ThehighlevelsofenergyconsumptioncomparedtoGDPcanbetiedtothelackofinvestmentsforthe
modernizationofindustrysincethecollapseoftheSovietUnion.Insufficientimplementationofenergy
efficienttechnologiesinUkrainecanlimitindustrialglobalcompetitiveness,andcausessevereimpacts
onthelocalandglobalenvironment.Itshouldbepointedoutthatthelowenergyefficiencyisoneofthe
majorfactorsthathaveexacerbatedthefinancialcrisisintheUkrainianeconomy.Theeconomiccrisis
andnaturalgascrisesbetweenRussianandUkrainethattookplaceinrecentyearshasalsohadastrong
impactonenergypolicyformationandgovernmentalgoalsinUkraine.Asaconsequence,the
GovernmentofUkrainehasestablishedaroadmaponEnergyStrategyforUkraineto2030thatplansto
saveupto470milliontonsofequivalentoilby2020,whichwilllessenimportofenergyresourcesbyup
to38billionUSD.
Inresponsetoglobalclimatechangeandinrecognitionofanumberofphasing-outactionsbeginning
aroundtheworldUNDP,GEF,andtheGovernmentofUkraine,viatheMinistryofEnvironment
Protectionagreedtoco-operatetoenabletheimplementationoftheTMELproject.TMEListhefirst
UNDP/GEF-supportedprojectsinefficientlightingandmarkettransformationforUkraine,andis
expectedtoparallelwithanotherUNDP/GEFlightingprojectbeingimplementedinRussia.Theprimary
contextoftheTMELprojectcanbedescribedinbroadtermsasfollows:
DevelopmentandCO2savingspotentialsforUkraine:Thereisuntappedpotentialforthedevelopment
andimplementationofnewenergyefficienttechnologiesinUkraine,includingenergy-efficientlighting.
Energy-efficientlightinghasbeengivenalowerpriorityinUkrainecomparedtomeasuresforenergy-
efficiencyrelatedtoenergyintensiveapplicationssuchasheatingsupply.Unlikeheatingsupply,which
tendtousegasorcoal-fireddistrictnetworks,energyconsumptionfromlightingimpactselectricity
productionanddistribution,andnotasseasonal.BecauseEElightinginitiativesimpactadifferent
industrialcomplexthanotherenergy-savingsprograms,thesemeasuresareanimportantand
complementaryenergyefficiencytool.
SupportforUkraine’senergypriorities:InlinewiththeGovernment’spriorities,thisprojectaddressed
anoverlookedissueinthereductionofgreenhousegasemissionsthroughlarge-scaleimprovementsin
energyefficiencyforUkraineusingafive-partapproach:
7Pertheprojectdocument,annualpercapitaemissionsin2007forGermanyis10.40tons,andforRussia12.00tons.
TMELTerminalEvaluationReport–31March2017
17
1. Improvethenationalpolicyframeworkforpromotingenergy-efficient(EE)lighting
2. Improvethenationalquality-assurance(QA)&quality-control(QC)systemsforimported
andproducedlightingproductsinUkraine
3. DesignandimplementEElightingdemonstrationsinthemunicipalsectorfocusingonpublic
schools
4. ImproveEELightingproductpenetrationintheResidentialSector
5. Replicateanddisseminatetheprojectresults
WithinthecontextoftheUNDPMillenniumDevelopmentGoals,otherthanthemajorcontributionto
environmentalsustainability(Goal7)bythisprojectwithinandoutsideofUkraine,theTMELprojectis
alsoexpectedtocontributetotheempowermentofwomen(Goal3)andreducingpoverty(Goal1).
2.1. ProjectStartandDurationTheprojectdocumentwassignedon31March2011,andthefirstdisbursementoffundstotheproject
wasmadeinmid2011(10June).Theoriginalprojectdurationwasfiveyears(60months),withan
expectedkickoffdateofSeptember2010andclosureinSeptember2015.Theoriginalproject
documentsubmittedfortheGEFCEOendorsementindicatedthefive-yeartimingasJanuary2011to
January2016.Duetothedelayinprojectapproval,theInceptionWorkshopwasheldon18November
2011,followedbystart-upactivitiessuchasorganizationofthePSCandPMO.Withthefive-year
plannedduration,theprojectwastobeimplementedfrom2011to2016.However,ano-costextension
wasappliedforbytheprojectandgrantedbyUNDP/GEF,extendingtheimplementationforafurther15
months,toMarch2017.
2.2. ProblemsthattheProjectSoughttoAddressMajorproblemsthattheprojectsoughttoaddressincludethefollowing:
• Inadequatelegislationtopromoteenergyefficiencyandenergyefficientlighting
• Inadequateabilitytorealizethetransformationbythepublicandprivatesectors
• Lackofqualitycontrolandsupervisionsystemforenergysavinglamps;
• Inadequatecontrolofthepollutantsfromspendanddiscardedlamps;
• Theneedforpromotingenergyefficiencylightingincitiesandruralareas.
2.3. ImmediateandDevelopmentObjectivesoftheProjectTheprojectaimedtoachievetheobjectivesetoutintheGEFStrategicProgramNo.1,whichis
PromotingEnergyEfficiencyinResidentialandCommercialBuildings(SP-1).
TheoverallgoaloftheTMELprojectwastoreducetheannualgrowthrateofGHGemissionsfromthe
Ukrainianpublicandresidentialsectors.Theprojectobjectivewastheenhancedpromotionand
implementationoftheutilizationofenergyefficientlightinginUkraine,throughthetransformationof
thelocallightingproductsmarketandthephasing-outofincandescentlampproductionandsale.Itwas
expectedtocontributetothereductionofGHGemissionsthroughthetransformationofthelighting
marketinUkrainetowardsmoreenergy-efficientlightingproducts,awarenessofnewtechnologiesand
bestpractices.
TMELTerminalEvaluationReport–31March2017
18
InadditiontoreflectingnationalprioritiesinUkraine,theprojectwasexpectedtobuilduponthe
existinggoalsandactivitiesofUNDP,withenvironmentalsustainabilitybeingoneoftheeight
millenniumdevelopmentgoals(MDGs)thatUNDPisplayingacentralroleinhelpingtopromote.With
respecttoUNDPactivities,theTMELprojectrepresentedaninnovativeapproachtoenergyefficiency
projects,andcomprisedofanimportantpartofUNDP-GEFportfolio,whichwouldcomplementand
builduponthelessonslearnedfromothersimilarUNDP-GEFprojectsinRussia,RomaniaandVietnam.
2.4. MainStakeholdersIngeneral,thestakeholdersoftheTMELprojectencompassorganizationsandgroupsinvolvedinthe
locallightingindustry,supplychain,marketdemandaswellaseconomyandsocialissuesofthephase
outofthemanufactureandsalesofincandescentlamps(ILs).Themandatesofthesestakeholdersare
directlyorindirectlylinkedtotheimpactsofILphaseoutandenergyefficientlightingpromotion,onthe
lightingindustryandtheusersoflightingproductsinUkraine.
TheTMELprojectdocumentwasdevelopedwithextensivestakeholderinvolvement,includingMinistry
ofEnvironmentalProtection(nowMinistryofEcologyandNaturalResources),MinistryofEconomyand
EuropeanIntegration,StateCommitteeforMunicipalHousing,OblastAdministrations,Municipalities,
Association“EnergyEfficientCitiesofUkraine”,AssociationofLocalandRegionalAuthorities,etc.
Discussionshavealsobeenheldwithpotentialprojectpartnersfromtheprivatesector,e.g.Phillips
Lighting,OSRAM,Kosmos,Gazotron-Lux,etc.Atthenationallevel,theprojecthasthefullsupportofthe
StateCommitteeforMunicipalHousing,theMinistryofEnvironmentalProtection,andbenefittedfrom
theparticipationoftheAssociation“EnergyEfficientCitiesofUkraine,”andAssociationofLocaland
RegionalAuthorities–anNGOveryactiveintheenvironmentfield.Amorecomprehensivelistofthe
stakeholdersinvolvedduringthepreparationphaseisshownbelow.
TMELTerminalEvaluationReport–31March2017
19
Figure1.StakeholdersintheProjectDesignProcess
2.5ExpectedResultsTheproject’smarketandinstitutionalinterventionsweredesignedtoleadtoanumberofsignificant
outcomes,includingimprovedlawsandregulationspromotingEElighting,increasedcapacitiesfor
lightingtestinglaboratoriesandthequalityassuranceframework,andcapacitybuildingfor
municipalitiesonthebenefitsanduseofEEtechnologies,helpingUkrainetoachieveitsenergysaving
TMELTerminalEvaluationReport–31March2017
20
objectives.Itisexpectedthattheimplementationoftheprojectwouldbringtogetherthehumanand
financialresourcesnecessarytodraftlawsandregulationssupportingEElighting,whichcanfeedinto
thewidernationalactionsonenergy-efficiency,andtomakeavailablethetechnicalassistanceneeded
toinitiatepilotprojectsinordertodemonstratethebenefitsofEElightingintheresidentialand
municipalsectors.Theseactivitieswereexpectedtodevelopbothdemandandsupplyofefficient
lightingequipment,andinparalleltransformthemarket.
ActionstopromoteEElightingbythisprojectwereexpectedtoeffectivelydoublethegrowthofEE
lightingmarketpenetration.Thiswouldincreasetheannualgrowthrateto25%yearonyearduringthe
projectperiod.Basedonthisincrease,theprojectwasprojectedtocontributeanetCO2reductionover
itslifetimeof4.15milliontonsfrom2011to2015fromdirectelectricityreductionintheresidential
sector.Nationalbenefitswouldincludeimprovedlocalenvironmentairquality(reductionsinSOx,NOx,
andparticulateemissions),long-termsavingsforconsumers,andabetterbalanceofpaymentsfor
electricityproducersandthestate,whichsubsidizedresidentialelectricitycosts.
Forthemunicipalsector,theprojectgoalwas100%compliancewithCMUOrder#1337-rrby2020,i.e.
100%replacementofmunicipalILswithEElightingproducts.Theshareofenergyefficientlightbulbsin
themunicipalsectorbuildingswasassumedas25%forthebaselinein2010.Thisfigurewasexpectedto
increaseatafasterrateduetoprojectactivitiesaimedatimprovingcomplianceandgiving
municipalitiesaccesstofinancingforlargeEElightingrenovations.Bytheendof2015,thecompliance
rateswereprojectedtobe63%(from25%today),and100%by2020respectively(fullcompliancewith
theCMUorder).TheprojectedCO2reductionsduetotheprojectactivitiesinthemunicipalsectorfrom
theincreasedcomplianceratesby2015wasestimatedtobe900,000tons.
Inadditiontoemissionsreduction,anumberofoutcomesresultingfromprojectactivitiesand
achievementofoutputswereexpected.Theseinclude:
• Improvedavailabilityandaccessibilityofenergyefficientlampsandlightingsystemsinthe
Ukrainianmarket;
• Improvedavailabilityandqualityoflocallyavailableenergyefficientlamps;
• Reducedhazardouswastepollutionfrommercury-containinglamphandlinganddisposal;
• Improvedcapacityofstakeholderstopromoteenergyefficiency,energyefficientlampsand
systems;
• Expandedmarketingchannelsforenergyefficientlampsandlightingsystemsattheretaillevel
acrossthecountry;
• Increasedsalesofenergyefficientlampsandreductioninthesalesofincandescentlampsinall
areas(includingsmalltowns&villages);
• Improvedpublicawarenessonthebenefitsandapplicationofenergyefficientlampsinallareas.
TMELTerminalEvaluationReport–31March2017
21
3. FINDINGSDetailedfindingsoftheTEarepresentedinthissection,andincludeanassessmentoftheTMELproject
FormulationandDesign,ProjectImplementationApproachandModality,aswellasasummaryof
projectresults.Wherepossible,detailedexamplesareincludedtosupportthefindings.
ThestatedgoaloftheTMELprojectistohelptransformtheUkrainianlightingmarkettowardsmore
energyefficientlightingproducts,technologiesandpractices.Itproposedtoachievethisgoalby
promotingagradualphase-outofinefficientlightingproductsintheresidentialandpublicbuilding
sectors.Specifically,theprojectproposedtoreduceCO2emissionsbyuptofivemilliontonsoverits
lifetimeoffromdirectreductioninelectricityconsumption.Theprojectdesigncomprisedofthe
followingfivemajorcomponentsandcorrespondingactivities.
• Component1:Prepareandset-upnationalpolicyframeworktopromoteEElighting–This
componentfocusesonimprovingthenationalpolicyframeworkforpromotingenergyefficiency
andenergyefficientlighting.
• Component2:ImproveQA/QCframeworkforEElightingmarket–Thiscomponentfocuseson
settinguptheframeworkfornationalqualityassuranceandqualitycontrolsystemsfor
importedanddomesticlightingproductsinUkraine.
• Component3:Efficientlightingdemonstrationinmunicipaleducationalsector–This
componentafocusesonprovidingtechnicalassistanceandincreasingknowledgeofEElighting
inthissector.
• Component4:ImproveEELightingproductpenetrationintheResidentialSector–This
componentaddressestheinformationandavailabilitygapforEElightingamongconsumers.
• Component5:ReplicationandDisseminationoftheProjectResults–Thiscomponentfocuses
onprojectsustainabilitymeasures.
Activitiesunderthesecomponentsweredesignedtobeimplementedinanintegratedfashion,andto
producerealanddemonstrableresultswithoutcomesthatcanbemonitored.
Itshouldbenotedthatmarkettransformationprojectsbytheirnature,areambitious,complexprojects
thatsetouttofacilitatetheintroductionofenergyefficientproductsintothemarket,andtoaddress
existingoranticipatedbarriersinordertoaccelerateandsolidifysuchintroductions.Well-designed
projectsseektoaddressallaspectsofthemarkettransformationprocess,includingtheeconomic,
information,institutional,technical,andsustainabilitybarriersbyapplyingtheappropriatepolicies,
supportingactions,andfinancialinstrumentsinanintegratedandcomplementaryfashion.Theyalso
attempttominimizetheassociatedriskstypicallyencounteredthroughtheincorporationofexperiences
andbestpracticesfromenergyefficiencyand/orclimatechange/markettransformationprojects
implementedinotherpartsoftheworld.8
8AchievingtheGlobalTransitiontoEnergyEfficientLightingToolkit,UNEP2012.
Note:althoughmuchoftheinformationcontainedinthisToolkitwasavailablefromothersources,theToolkit itselfwasnot
availableuntil2012,severalyearsafterTMELwasdevelopedandimplemented.
TMELTerminalEvaluationReport–31March2017
22
Overthecourseofitsimplementation,TMELexperiencedsignificantchallenges,includingchanging
nationalpoliticalandeconomicconditions(changesingovernmentandpolicies),worseningsecurity
situations,acollapsedinternationalcarbonmarket,andrapidtechnologicalshiftstomoreenergy
efficienttechnologies.Alongwiththeseexternalchallenges,theprojectalsoexperiencedanumberof
internalchallenges.Atitsinception,theprojectoperatedwithoutaninternationalCTAexperiencedin
markettransformation.Overitssix-yearduration,theTMELprojectwasmanagedbythreedifferent
projectmanagersandwasadvisedbyatleasttwoInternationalConsultants/TechnicalAdvisors.9Neither
CTAwasfullyutilizedinallprojectaspectsasenvisionedbytheprojectdocument.10
Generally,mostprojectsofthisscaleandtypecanexpecttoencounteroneortwomajoridentifiedrisks.
Inmostcases,theprojectdesignprocess,riskmanagementstrategies,aswellastheprojectadvisory
frameworkcanhelptomitigatetheimpactsofthesechallenges.Similarly,UNDPcountryofficesareset
uptoprovideadministrativesupporttoprojectsandtominimizeimplementationdisruptions.
ItisalsoworthnotingthattheTMELprojectwastheinitialeffortbyUNDP/GEFtointroduceafull-scale
countryprojectusingthemarkettransformationapproachtoUkraine,workingwithnationalinstitutions
andagenciestoaddressbarriersinthemarket.11TMELwasinitiatedshortlyafteranothersimilar
UNDP/GEFprojectworkingonintroducingfull-scalemarkettransformationtotheRussianFederation.
Whilebothproject’sdesignrelieduponpastinternationalexperienceinthisarea,Ukrainehasless
nationalexperienceintheimplementationofsuchaproject,andthereforetheremayhavebeenless
availableknowledge,familiarityandinputthatcanbecalleduponduringimplementation,especiallyin
theinitialstages.
InevaluatingtheTMELproject,itmaybenecessarytoviewtheprojectactivities,outputs,andoutcome
fromtwodifferentperspectives.Theprojectcanbeevaluatedforitsachievementsagainsttheoriginal
objectivesaslaidoutbytheProjectDocument,astheseincorporatedallindicatorsandmetricsfor
markettransformationbytheoriginalprojectdesign.12Considerationsmayalsobegiventothe
evaluationoftheprojectagainstitsrevisedpath,asthiscanalsoprovideanindicationofproject
adaptation,achievementsandoutputsgiventhevariouschallengesthattheprojectencounteredover
thecourseofitsimplementation.
3.1. ProjectFormulation&DesignAnexpertteamofinternationalandnationalconsultantsconductedtheTMELprojectdesignand
preparedtheProjectDocument,followingtheguidelinesforGEF-UNDPprojectformulation.Thedesign
wasinformedbyUNDPandGEF’sexperienceofefficientlightingandclimatechangeprojects
9Theprojectmanagersandtheirserviceduration:
• SergeiVarga:June2011–December2014
• OlexanderSeverin(Acting):January2015–July2015
• AndriyBuriakovskiy:July2015–March2017
TheCTAs:
• VladimirGabrielyan(NationalStrategyonly):2014
• SteveCoyne:Jan2015–March201710TheTORfromtheprojectdocumentwasnotfullyappliedtoeitherCTA.
11ThefirstUNDPfull-scalecountryprojectwasESCORivne.
12Itcanalsobesaidthattheoriginalprojectobjectivesbecameoverlyambitiousgiventhechallengingconditionsencountered.
TMELTerminalEvaluationReport–31March2017
23
implementedinotherpartsoftheworld.AtthetimeoftheprojectDocumentpreparation,thedesign
teamwassupportedbybaselineresearchoftheUkrainianlightingandenergymarkets,andtherefore
hadasolidfoundationfortheplannedprojectactivitiesbasedonconditionsthatexisted.However,as
notedabove,Ukrainiansecurity,economicandpoliticalconditionschangedsignificantlyoverthecourse
oftheprojectimplementation,asdidthetechnicalprogressofenergyefficientlighting.
Findings:TheEvaluationTeamfoundthat,basedontheconditionsthatexistedatthetimeof
conception,theproject’slogicalframeworkprovidedadetailed,cohesive,andapplicablecoursefor
projectimplementation,reflectinginternationalbestpractices.Withrespecttotherelevancyofthe
logicalframework,theevaluationteamfoundthatthelogicalframeworkremainedrelevantforthe
projectintermsofcreatingthenecessaryconditionstosustainmarkettransformation.
TheEvaluationTeamalsofoundthattheactivitiescontainedunderthedifferentcomponentswere
coherent,replicable,canbesustainable.Theywerealsodesignedasanintegratedapproach,andina
cost-effectivemanner.Inaddition,theimplementationarrangementsandresponsibilitiesofthevarious
stakeholderswereoutlinedclearlyintheprojectdocument.However,theTeamnotedthatspecificco-
financingfromthevariousstakeholders,includingtheprivatesector,wasnotspecifiedindetail,which
madethetrackingofthesedetailschallengingfortheM&Eprocess.
3.1.1. StakeholderParticipationinProjectDesignPertheTeam’sinterviewwiththeProjectDocumentconsultant,keystakeholderswereconsultedand
theirexperiencesandrecommendationswereintegratedintotheprojectdesignandlogicalframework.
StakeholderinvolvementinthedesignstageincludedtheMinistryofEnvironmentalProtection,Ministry
ofEconomyandEuropeanIntegration,StateCommitteeforMunicipalHousing,OblastAdministrations,
Municipalities,Association“EnergyEfficientCitiesofUkraine”,AssociationofLocalandRegional
Authorities,etc.).Discussionswereheldwithpotentialprojectpartnersfromtheprivatesector,e.g.
PhillipsLighting,OSRAM,Kosmos,Gazotron-Lux,etc.Atthenationallevel,theprojecthasthefull
supportoftheStateCommitteeforMunicipalHousing,theMinistryofEnvironmentalProtectionand
benefitsfromtheparticipationoftheAssociation“EnergyEfficientCitiesofUkraine”andAssociationof
LocalandRegionalAuthorities,forexample.
Findings:UponreviewoftheProjectDocument,theTeamfoundthatstakeholdersatvariouslevels
wereconsultedatthetimeofprojectformulation,andasmuchaspossible,stakeholders’commitments
andbuy-inwereobtainedatthedesignstage.TheTeamalsonotedthatchanginginstitutionaland
marketconditionsresultedinnewstakeholders,someofthesestakeholderswereidentifiedbythe
projectandsubsequentlyconsultedorcooperatedwiththeproject,butotherpotentialstakeholders
werenotrecognizedorfullyengagedbytheproject.
OneexampleofchangingconditionsandstakeholdersistherecentlyorganizedMinistryofRegional
Development,Construction,HousingandCommunalServices(MRDCHCS),whosepurviewnowincludes
energyefficiencyforbuildingsandlighting.13DuetothefactthattheTMELprojectishousedunderthe
13PreviouslyMinistryofConstruction,Architecture,andCommunalLiving.
TMELTerminalEvaluationReport–31March2017
24
MinistryofEcologyandNaturalResources,cooperationbetweenministrieshasbeenminimal.14
However,exploringhowtheprojectcanworkwiththenewministrycouldhavebeenoneofthekey
tasksfortheteamresponsibleforPartnershipsandRelationswithGovernmentandNationalAuthorities.
Inaddition,theprojecthadverylimitedinteractionwithotherdonorsveryactiveintheareaofenergy
efficiencyinUkrainesuchasUSAID,GIZ,EBRD,WorldBank,IFC,orNEFCO.Itwasdifficulttoascertain
whetherthisnon-engagementofnewpotentialstakeholderswasaresultofprojectconsideration,or
lackofwillingnessofthesestakeholderstocooperateandengagewiththeproject,astherewasnowork
planorPIRitemsindicatingtheneedforthistask.
3.1.2. ManagementArrangements(ProjectDesign)TMELwasdesignedtobeaNationalImplementationModality(NIM)projectbytheUkraine
Government.KeymanagementarrangementsoutlinedinthedesignincludedtheroleoftheMinistryof
EcologyandNaturalResources(originallyMinistryofEnvironmentalProtectionpertheproject
document)astheImplementingPartner(orExecutingAgency),aPMOresponsibleforday-to-day
managementoftheprojectactivitiesaswellasforfinancialandadministrativemanagementservices
(UNDPUkraine).Inaddition,thedesigncalledfortheestablishmentofaProjectSteeringCommittee
(PSC)withrepresentationfromallkeystakeholders.Theprojectdocumentalsopresentedadetailed
stakeholderinvolvementplanwhilespecifyingtheroleofeachstakeholder.15Notethataccordingtothe
RTA,theTMELprojectconvertedtoaDonorImplementationModality(DIM)projectwhenconflicts
aroseinEasternUkraine,butthereisnodocumentationregardingthisconversion.
Findings:TheEvaluationTeamconcludedthattheoriginalprojectdesignwouldhaveprovidedacost-
effectiveapproach,whileincorporatinginter-agencyandinter-stakeholdercollaborationandoversight
atvariouslevelsofmanagement.Therolesandresponsibilitiesofthevariousstakeholdersinvolvedin
theproject’smanagementhadbeenclearlydefinedintheprojectdesigndocument.
3.1.3. ReplicationofApproachReplicationanduptakehasbeenakeypartofUNDPprojectdesign.ThedesignforTMELfacilitated
replicationbyincludingstakeholdersthathavethecapacityandstakeinthepromotionofenergy
efficiency,productquality,adoptionofenergyefficientlighting,andmanagementofhazardouswaste
management.KeyexamplesinthisincludeworkingwiththeImplementingPartner(andExecuting
Agency)–theMinistryofEcologyandNaturalResourcesofUkraine,aswellasnon-governmental
organizations.
AnumberofprojectactivitiesoutlinedintheLogicalFrameworkwerespecificallyaimedattechnology
transferanddemonstrationtosupportreplication.Theseactivitiesincludedthedevelopmentofa
NationalRoadmapandpolicyforconversiontoenergyefficientlighting,supporttotestinglaboratories
inenergyefficientlightingtechnologyandhandlingofhazardoussubstances.Alsoincludedwerethe
establishmentofacollectionnetworkforspentlampscontainingmercury,demonstration,pilots,and
awarenessraisingactivities,aswellasthepromotionandestablishmentofmarketingvenues.Moreover,
14According to some stakeholders, this limited interaction allowed forMinistries to actmore independently, andwith less
externalinfluence.15AlistingofthePSCisincludedinAnnexF
TMELTerminalEvaluationReport–31March2017
25
theprojectdesignincludedthedevelopmentofvariousdocuments,includingmarketstudies,
documentationofpilots,productstandards,andguidelines,etc.,aswellaslinkagesandknowledge
sharingwithinternationalactivities.
Theprojectdesignalsoincorporatedrecentlessonslearned.Forexample,theprojectdesignaddressed
policyimprovementsinparallelwithconcreteactionstoraisethequalityofEElightingproductsonthe
market,whileprovidinginitiativesforovercomingthelargerupfrontcostofqualityEElightingproducts.
InpreviousGEFEEprojects,overcomingtheupfrontcostbarrierwasseenasmorecriticalforsuccess
thanonlyprovidingawarenessraisingactivities.Thesewerealsothesamerecommendationsmadeby
theMTRandthenewCTAupontheirreviewofprojectactivitiesin2014and2015.Bothexperts
recognizedprogressmadeinawarenessraisingbytheprojectuptothatpoint,andsuggestedarenewed
focusonotheroutcomeswithlessprogress.16
Findings:TheEvaluationTeamconcludedthattheoriginalprojectdesignwouldhaveprovidedacost-
effectiveapproach,withinter-agencyandinter-stakeholdercollaboration,andoversightatvarious
levelsofmanagement.Therefore,theapproachusedisreplicable.
3.1.4. AssumptionsandRisksTheprojectdesignwascognizantofthesomeofthemajorpotentialrisksassociatedwith
implementationofthefivecomponents,includingweakgovernmentsupport,ineffectiveQA/QC
enforcementmeasures,lowlevelofparticipationfromtheprivatesector,ineffectivelong-term
financingprogramsforthepilotprojectsandtheirreplication,andlowlevelofinvolvementofregional
authoritiesindemonstrationprojectactivities.Accordingly,practicalmitigationactionsweresuggested
foreachoftheserisks,whichincludedsecuringfirmcommitmentsfromresponsibleinstitutionsduring
theprojectdesignstage,workingwithsuggestionsmadebyDehrzStandardforimprovementsinthe
QA/QCsystem,involvingtheprivatesectorduringtheprojectdesignstage,etc.Thedesignalso
stipulatedfortheconstantmonitoringandrevisionoftheserisksinaccordancewiththe
implementationrealitiesduringkeystages,e.g.arevisionattheinceptionstageaswellasatthetimeof
submissionofAnnualWorkPlans.
Findings:TheTeamfoundthatalthoughanumberofrisksandriskmitigationstrategieswereidentified,
someofthechallengesoccurringduringtheprojectimplementationwerenotanticipated,andcouldbe
classifiedasoutsideofthenorms.Anticipatedrisksincludeweakgovernmentsupport,andlowlevelof
involvementfromtheprivatesector.Unanticipatedrisksincludedchangingnationalsecuritysituations
andacollapsingcarbonmarket.AnotherpointtoconsideristherapidpaceofadvancementinLED
technologies,whichresultedinlowerpricingandfasterpenetrationofLEDsthananticipated.Thisfaster
penetrationhelpedtoincreasetheadoptionrates,butalsopresenteditsownchallengesinissuesof
quality,userfamiliarity,andhowtoproperlymeasuretheirimpacts.Therefore,notallpractical
mitigationactionswerecovered,orcoveredinsufficientdetailsbytheProjectDocument,andneeded
actionsorguidancewerenotavailabletotheProjectSteeringCommitteeortheProjectTeamto
16PIMS4175Mid-TermEvaluationReport,March2015;MissionReport,October2015.
TMELTerminalEvaluationReport–31March2017
26
effectivelyaddressallchallengesastheyarose.Inaddition,giventhechallenges,amorebroadand
activeorengagedPSCmayhavebeenrequired.
3.1.5. UNDPComparativeAdvantageUNDPregionalofficehasprovidedtechnicalsupporttonumerousmarkettransformationandclimate
changeprojectsinvariouscountriesacrosstheregion.Inaddition,UNDPhasimplementedover30
technicalassistanceprojectsrelatedtoenergyefficiencyinthepastdecade,withestimatedvalueof
overUSD$100million.ThisexperienceenabledtheUNDPandtheRegionalTechnicalAdvisortoprovide
technicalsupporttotheprojectformulationandinputintothedevelopmentofthelogicalframework,
recruitmentofinternationalexpertsfortheprojectformulationandimplementation,andidentification
ofkeystakeholders,etc.
Findings:TheTeamconcludedthatbasedonpriorexperience,theUNDPwasabletoprovideguidance
forestablishmentofinstitutionalcoordinationmechanisms,andforleveragingtheprojectactivities
throughcollaborationbetweenpublicandprivatesectors.
Inconclusion,theEvaluationTeamfoundtheprocessofprojectformulationandtheprojectdesigntobe
satisfactoryforatypicalmarkettransformationproject.GiventhechallengesfacedbytheTMELproject
duringtheimplementationprocess–whichmaybeunlikelytobefacedbyanotherproject–wesuggest
thatUNDPtaketheseconditionsintoaccount,andconsiderthedevelopmentofamoreexhaustivelisting
ofrisksforUkraine,aswellasmoredetailedriskmitigationstrategiesfordesignofprojects,and
considerwaysforactiveinvolvementbythePSCinregionsthatmayexperiencerapidchangesforother
projectstoreferenceshouldtheyencounterthesamesituations.
3.2. ProjectImplementation
Theoriginalprojectdurationwasfiveyears(60months),withanexpectedkickoffdateof2010and
closurein2015.TheoriginalprojectdocumentsubmittedfortheGEFCEOendorsementindicatedthe
five-yeartimingasJanuary2011toJanuary2016.Theprojectdocumentwassignedon31March2011,
andthefirstdisbursementoffundstotheprojectwasmadeinJune2011.Asaresult,theInception
Workshopwasheldon18November2011,followedbystart-upactivitiessuchasorganizationofthe
PSCandPMO.Withthefive-yearplannedduration,theprojectwastobeimplementedfrom2011to
2016.However,duetochallengesanddelaysinprojectstart-up,ano-costextensionwasappliedforby
theprojectandgrantedbyGEF,extendingtheimplementationafurther15months,toMarch2017.
3.2.1. UNDPandImplementingPartnerImplementation/ExecutionCoordination,andOperationalIssues
TMELTerminalEvaluationReport–31March2017
27
Figure2:ProjectManagementStructureofTMEL
Theimplementationandcoordinationroleplayedbythevariousstakeholdersisdetailedbelow:
§ GovernmentofUkraine(NationalImplementingPartner-NIP):MinistryofEcologyandNatural
Resources,Ukraine.MENRactedastheleadimplementingagency,andwastherecipientof
legislativeoutputsfromtheproject,andcoordinatedgovernmentresponse.MENRalso
providedofficespaceandin-kindservicesforthedurationoftheproject.
§ UNDP:UNDPUkraineandtheUNDP-GEFRegionalTechnicalAdvisorforClimateChangeinthe
regionhaveprovidedGEFoversight.Inthiscapacity,UNDPhasbeenresponsibleforoverall
M&E,organizingprojectreviews,providingsupportintherecruitmentofinternational
consultantsandtechnicalexperts,approvingprojectimplementationreviews(PIRs),budgets,
andprovidingfeedbacktoensurethatallreportingiscarriedoutinlinewithstandardUNDP-
GEFprocedures.TheUNDPUkraineofficehadbothanoversightroleaswellasamemberofthe
PSC.Inaddition,theUNDP-GEFRegionalAdvisorprovidedongoingtechnicalsupportand
guidancetotheproject.
§ ProjectSteeringCommittee(PSC):APSCwasestablishedattheonsetoftheprojectandcomprisedofrepresentativesfromkeystakeholders,includingGovernmentofUkraineand
UNDPUkraine.ThePSCwasintendedtomeetonceayearsincetheprojectinception,but
convenedatotaloftwotimes(December2012,February2014).17AlistofthePSCmembersis
providedinAnnexF.ThePSCcontributedtotheimplementationasanadvisorycommitteeand
providedguidancetoprojectplanning.
§ ProjectManagementOffice(PMO):ThePMO,workingfirstundertheNIPwasresponsiblefor
performedeffectiveimplementation,M&E,andstakeholdercollaboration.Theplacementofthe
PMOwithinthegovernmentMinistryalsohelpedtoelevatethevisibilityoftheprojectwith
stakeholders.
17AthirdmeetingscheduledinFebruary2016wascancelledduetolackofparticipants.
TMELTerminalEvaluationReport–31March2017
28
Theaboveentitiesworkedtogethertoimplementandmonitortheproject.
However,therewereanumberofinternalissuesthattogether,contributedtoalessthansmooth
projectexecutionduringcertainperiods.Someissuesspecifictoprojectadministrationwereidentified
bytheMid-TermReview,whichwasconductedbetweenFebruaryandMarch2014.Themajorityof
administrativeissuesidentifiedbytheMTRhasbeen,orintheprocessofbeingcorrectedduringtheTE
missions,exceptionsincludestaffsizeandunevenfocusonprojectcomponents.
Thereremainanumberofadministrativechallengesthatwarrantmention,including:
• LessthanidealcommunicationsorcoordinationbetweentheprojectteamandUNDP
administrativesupport,whichcontributedtothedelaysinhiringofsomesub-contractors,staff,
orsecuringneededservices,andreducedeffectiveness.18
• Adherencetotheprojectdocumentstrategywasalsoinconsistent,leadingtoanumberof
implementationissues,including,forexample:
o ThelackofconsistentengagementbyanexperiencedCTAfromtheoutset,whoseregular
visitsandinputcouldhavereducedtheinitialresearchneeds,thetimeneededforthe
rolloutofawarenesscampaigns,andQA/QCframework.19
o DelaysinthepreparationoftheNationalRoadMapforEELightingMarketTransformation,
whichmayhavehelpedwithsettinguptheframeworkforQualityAssurancesooner;
o Delaysinthepreparationoflegislationtoimprovethequalityofelectricitysuppliedto
publicandresidentialconsumers,whichremainsunfinished.
o Confusionovermanagementreporting,includingadualreportingstructure.20
Figure3.TMELManagementStructure(Organigram)
18Staffindicatedwaittimeofuptoninemonthsforhiring,althoughtheEvaluationTeam’scontractswereputinplacewithina
fewweeks.19ThefirstCTAhiredlackedinternationalimplementationexperience,andproducedunsuitabledocumentsfromothers’work.
20TherewasconfusionduetothefactthatsomeTORswerenotconsistentintheirtranslationfromtheprojectdocument.For
example,theEnglishversionoftheProjectAssistant’sTORindicatedthatthepositionreportsto“AdvisoronNewPartnerships
andRelationswithGovernmentandNationalAuthorities,”insteadoftheProjectManager.
TMELTerminalEvaluationReport–31March2017
29
Ascanbeseenabove,theproject“organigram”doesnotshowthepositionofaCTA,whichisakey
positionpertheprojectdocument.Italsoshowsamixedprojectmanagementandreportingstructure,
whichwasnotanoptimalandcostefficientsetup,duetothefactthat:
• TheteamresponsibleforPartnershipsandRelationswithGovernmentandNationalAuthorities
hadaseparatereportingstructure.21
• Thesizeandcompositionoftheteammaybeoverlylarge,asdiscussedbytheMTR.
WhiletheabovestructuremayhaverecognizedtheimportanceandsensitivityofthePartnershipsand
RelationswithGovernmentandNationalAuthoritiesteam’stasks(anditssuccesses,includingsecuring
supportbyMPs),itcreatedaseparate“silo,”allowingforlesscoordinationbetweentheteam’sleader
andprojectmanager.22ItalsoallowedforlittleornointeractionsbetweenthisteamandtheCTAon
mattersrelatedtoproducttesting,qualityassuranceframework,andinternationaloutreach.Infact,
SteveCoyne(theCTA)reportedthattherewasnocommunicationsbetweenthe2ndPMandhimself
beforethetransitiontothe3rdPMduetothelanguagebarrier,andthewaytheteamwasstructured.
Pertheprojectdocument,TMELwasintendedtocoordinateandshareinformationwiththeUNEP’s
en.lightenprogram,cooperatewiththeUNDP/GEFEElightingprojectintheRussianFederation,and
throughthetranslationanddistributionoftheUNEPglobalprogram’snewsletterattheNationalLevel.
Conversely,theactivitiesandlessonslearnedfromthePartnershipsTeaminworkingwithUkraine’s
ParliamentandsecuringsupportofanMP,aswastherationaleforthisstructure,werenotfully
documentedorcapturedbytheproject.
TheTMELprojectdidnotretaintheservicesofanexperiencedinternationalCTAuntil2014(4yearsinto
a6yearsproject),whichwasoneofthekeyprojectpositionsasenvisionedbytheprojectdocument,
includingfrequentvisitstoUkraine,withmissionsonceper1-2months.Thetermsofreference(TOR)
foraCTAforaprojectsuchasthistypicallyincludestrategicguidance,workplanning,reviewofproject
outputs,andparticipationinpreparationofToRandselectionofsub-contractorstohelpimprovethe
overallqualityoftheproject.ThiswasnotthecaseforTMELfromtheoutset.Themainassessmentof
theinternationalCTAontheprojectisthatahighlyqualifiedexpertwashiredtoolateintheproject
withinsufficientvisitstoUkraine(twiceperyear)tohaveanysignificantimpactsonalloftheproject’s
outputs.WhiletheinvolvementoftheinternationalCTAhelpedtomoveanumberofissuesforward,
suchasathedevelopmentofaqualityassuranceframework,includinginter-laboratorytestingand
trainingoflaboratorytechnicians.However,thissupportcouldhavehappenedearlierandincluded
moretopics,suchascollectionanddisposal,aswellaspilotprojectdevelopments,trainingforretail
staff,andoutreachtocitiesandmunicipalities,aswellasinternationally.
Basedontheabove,theEvaluationTeamconcludedthattheoverallprojectmanagementstructureand
arrangementswereapartofaparticipatoryandconsultativeapproachtoimplementationand
monitoringofprojectresults.However,thepace,internal(andinternational)coordinationand
21Pertheprojectorganigram,theteamresponsibleforPartnershipsandRelationswithGovernmentandNationalAuthorities
reportedtotheHeadofEEcluster.22TheteamleaderwasalsonotbasedintheprojectofficeatthetimeoftheTEvisit.
TMELTerminalEvaluationReport–31March2017
30
consistentdeliveryofprojectimplementation,aswellasthecoststructureclearlysufferedfromthe
dualreportingarrangementandthelargesizeoftheteam(whichremainedunchangedaftertheMTR).
Asindicatedearlier,thefactthatthisprojectrepresentedtheinitialintroductionofafull-scale,market
transformationprojecttoUkraine,administrativecautionorinexperiencewithimplementationmay
havecontributedtosuchissues.23
Therefore,theevaluationteamfoundtheUNDPandImplementingPartnermanagementoftheproject
tobeModeratelySatisfactory,anddeterminedthattheimplementation/executioncoordinationon
operationalissues(specificallyprojectcommunications,planningprocessandPSCinvolvement)has
beenModeratelyUnsatisfactory.
3.2.2. AdaptiveManagementOverthecourseofitsimplementation,theTMELprojectexperiencedsignificant,ifnotunprecedented
challenges:changingpoliticalandeconomicconditions,aworseningsecuritysituation(anarmedconflict
inEasternregionfrom2014),rapidtechnologicalshifts,andinternalissues.Itwasalsotheinitialeffort
byUNDP/GEFtointroduceafull-scaleprojecttoUkraine,workingwithnationalinstitutionsand
agenciestotransformthemarket,whosedesignandapproachmayhavebeenunfamiliarincomparison
tootherfull-scaleUNDPProjects.
Becauseofthesignificantimplementationchallenges,thehallmarkoftheTMELproject’s
implementationhasbeenadaptivemanagement.Theprojecthasimplementedanumberof“work-
arounds”initseffortstomaintainforwardmomentumtowardsitsobjectives.Theseadjustments
allowedtheprojecttocontinueandmadesomeimportantcontributionstotheoverallmarket
transformationefforts.Asindicated,theproject’sachievementsshouldbeconsideredagainstits
progresstowardsitsobjectivesaswellastheapproachtakentoaddresschallengesandbarriers.
Infact,theprojectmetanumberofconditionsforwhichadaptivemanagementisrequiredasdefined
bytheGEFGuidancedocument,whichinclude:24
• Originalobjectiveswerenotsufficientlyarticulated;
• Exogenousconditionschanged,duetowhichachangeinobjectiveswasneeded;
• Projectwasrestructuredbecauseoriginalobjectiveswereoverambitious;
• Projectwasrestructuredbecauseofalackofprogress;
Asignificantexampleofadaptivemanagementwasdemonstratedintheproject’sfocusonoutreach
andsupportformunicipalitiesandcitiestophaseoutincandescentlamps.Whiletheprojecthadmade
progressindevelopingoutputssupportingnationallegislationstointroduceenergyefficiencyand
energyefficientlightingforUkraine,theselegislationshavenotbeenpassed.Theoutreachtomunicipal
andcitygovernmentsenabledtheprojecttotakeadvantageofthedecentralizationofauthority(which
23For example, it was not entirely clearwhy the project did not hire an experienced CTA from the outset, but it could be
attributed to unfamiliarity with this type of projects, and the possibility for applying best international practice to local
messagingwasnotrecognized.24GuidanceforConductingTerminalEvaluationsofUNDP-Supported,GEFFinancedProjects,UNDP2012.
TMELTerminalEvaluationReport–31March2017
31
happenedafterprojectinitiation)toachieveitsgoalseffectivelythroughpartneringwithawide-range
ofhighlyrelevantstakeholdersthatwerenotinitiallyspecifiedintheprojectdocument.
AdaptiveManagementandtheMTR
Therewereotherprojectareasthatdidnotnecessarilyrequirethesameadaptivetreatment.For
example,theprojecthadmadesignificantprogressintheareaofincreasinggeneralpublicawareness,
yet,theprojectresourcesremainedfocusedonthisarea,perhapstocompensateforlessprogressin
otherareas.ThisfocuscontinueddespiterecommendationsfromtheMTRtoreduceemphasison
marketing,orrecommendationsfromtheCTAtotargetmorespecificconsumertopics,suchas
increasingawarenessofdifferencesinproductquality.TheUNDPRTA’srecommendationstofocuson
ESCOmarketdevelopmentforstreetlightingandworkingwithEBRDasameansofpromotingadditional
investmentinefficientlightinginUkrainewerealsonotincludedasapartofadaptivemanagement.
TheMTRwascarriedin2014reviewedtheproject’sprogressuntilthenwithrespecttotheoutput
intendedbytheprojectdocument.TheMTRmadeaseriesofrecommendationsintermsofproject
administrativeandimplementationareas.Intermofprojectimplementation,theMTRproposedthat
theprojecthassucceededinadaptivemanagementbyconcentratingonincreasingenergyefficiency
awarenessbyUkrainiansthroughmarketingandoutreachefforts.TheMTRrecommendedgoing
forwardthattheprojectfocusedlessonmarketingandinsteadfocusingonareaswerenotasadvanced,
includinglegislation,qualitycontrolframework,demonstrations,andsupportforlocalproducers.In
termsofadministrativerecommendations,theMTRrecommendedthehiringofaCTA,asmallerteam
andreportingstructure,alongwithM&Eimprovements.However,whilesomerecommendationswere
adopted,thespendingremainedonmarketingandoutreach,andtheteamsizewasnotadjusted,
affectingimplementationcosteffectiveness.
Theprojectdocumentremainedhighlyrelevantthroughtheoftheimplementationperiod.However,to
adjusttheactivitiesaccordingtotheon-the-groundrealityandtoensureefficientachievementof
projectgoals,afewoutputsandactivitiesweremodifiedinitiallybythePSC.Ofthese,significant
changesincludechangesOutcome3(implementationofefficientlightingdemonstrationsinthe
municipalsector).ThisOutcomewasoriginallydesignedtoimplementlightingdemonstrationsinthe
municipaleducationalsector,suchasschools.However,duringtheInceptionWorkshop,adecisionwas
madetoincreasethelocations,andexpandcoveragebeyondschoolstoincludeothermunicipaltargets,
suchasbuildings,streetlightingandtrafficlights.
Theevaluationteamconcludedthat,whiletheoutputsandindicatorsstatedintheprojectdesign
documentarerelevantandappropriate,aspartoftheadaptivemanagementapproach,the
managementteamcouldandhadproposedtoadoptorchangesomeapproachesandoutput
implementingdetails,whennecessary.
3.2.3. PartnershipArrangementsOverthecourseofimplementation,theprojecthaspartneredwithgovernmentagencies,industry
associations,enterprises,researchinstitutes,testinglaboratories,certificationbodies,consultants,and
TMELTerminalEvaluationReport–31March2017
32
mediaoutlets.Majorpartnershipactivitiesincludedresearch,policydevelopment,demonstration
projects,testing,awarenessraising,andtrainingprograms.
Thesepartnerswereengagedusingasub-contractingmodality,withtheprojecthavingissued30sub-
contracts,includingtotwoormoreorganizationsforjointimplementation.Thetotalvalueofthesub-
contractsawardedwasapproximatelyUSD$3.83million.Ayear-wisedistributionofsubcontracts
awardedispresentedintheTablebelow:
Table1.Year-WiseDistributionofSub-Contracts,byComponent
Contracts Year1 Year2 Year3 Year4 Year5 Extended AdjustedTotal
%ofTotal
Component1 2,274 92,036 178,618 118,901 519,319 237,579 1,148,727 30%
Component2 145 27,835 190,109 267,462 461,813 215,061 1,162,426 30%
Component3 0 15,124 4,590 32,846 53,605 46,632 152,797 4%
Component4 145 137,132 145,036 83,154 308,940 171,061 845,469 22%
Component5 145 202,856 44,847 164,127 44,897 68,112 524,984 14%
Total 2,710 474,983 563,202 666,490 1,388,574 738,444 3,834,403
%ofTotal 0.1% 12% 15% 17% 36% 19%
AsshowninTable1above,thehighestproportionoffundingforsub-contractswasspentduringYears3,
4andespeciallyYear5oftheProject(36%oftotal).Thisisduetothefactthatthepaceofproject
implementationacceleratedinresponsetotheMTR,afterathirdfull-timeProjectManagerwas
retained,andaninternationalCTAwashiredtofocusontheQAframeworkandothertechnicalareas
suchasESCOdevelopmentandaddstreetlighting.Thisisdifferentthantheusualprojectpace,asthe
firstandlastyearsofaprojectwouldbeexpectedtohavecomparativelylesseractivitiestoduetothe
focusoninitiatingorclosingoftheproject.
Theevaluationteamconcludedthattheproject’spartnershipswithstakeholdersservedtocreateor
developedthesynergiesneededtoachieveprojectgoals.Asshowninotherrelevantsections,thesub-
contractingalsoplayedaroleoncostefficiency,effectiveness,andsustainabilityofprojectactivities.
3.2.4. MonitoringandEvaluation(M&E)Accordingtotheprojectdesigndocument,UNDPUkraineandtheprojectwerejointlyassigned
responsibilitiesofM&E.Inaddition,theprojectdocumentprovidedaclearM&Eplanandbudget,
includingannualoutcomeleveltargetsandadetailedM&Eplan,amonitoringplantogetherwith
concisetargets,asimplelogicalframeworkwithSMARTindicators,andabudgetforM&Eactivities.The
UNDPRegionalTechnicalAdvisor(RTA)forClimateChangealsoprovidedperiodicoversightforproject
implementation,includingpromptingtimelyreporting,providingguidanceaboutreportingtoensure
thattheprogressisimplementedinlinewithUNDP-GEFguidelines,andprovidingfeedbackonproject
planning,includingencouragingtheprojecttofollowandimplementandfollowtheMTR
TMELTerminalEvaluationReport–31March2017
33
recommendations.25Similarly,theprojecthasundertakenimpactassessmentstodemonstrateprogress
towardsitsgoalsandobjectives.
ThePMOhadtheresponsibilityofproject-levelmonitoring.Forthispurpose,thePMOhasdevisedand
implementedanM&Eplanthatisresponsivetotheproject’slogicalframework.Theplancomprisesof
thefollowingkeyelements:
• Projectmanagementrules
• Sub-contractbiddingevaluationmanagementrules
• M&Erules
• PMOlogisticadministrationrules
• DutiesandresponsibilitiesofPMOstaff
• Websitemaintenancerules
TheplanwasdevelopedbythePMOteamatthestartoftheprojectandwasapprovedbythePSC.The
M&EplancompliedwithUNDP-GEFprojectreportingguidelines.Inaddition,thePMOhasdeveloped
andmadeappropriateuseofaProjectManagementSystemthathelpsinmonitoringactivitiesand
trackingresults.
Findings:Theevaluationteamconcludedthattheproject’sM&Eplanwaswelldesignedandadequately
implemented,withafewexceptions,forexample:co-financingfromthevariousstakeholders,including
theprivatesector,whichwasnotspecifiedindetail,andthereforewasnotfullytrackedbytheproject;
theimpactsmonitoringwasalsoproblematic,duetosecuritysituations,aswellaslackofreliabledata
onsalesandemissions.Theproject’sM&EisModeratelySatisfactory.
3.2.5. ProjectFinance
Table2.CommittedProjectFundingandSources
GrantFund Committed(USD) PercentofCommitments
GEF 6,500,000 21%
GovernmentofUkraineCo-financing• Direct:19,375,000• Indirect:1,600,000
24,500,000 67%
PrivateSectorCo-financing 3,275,000 10%
UNDP 250,000 1%
TotalBudget 31,000,000 100%
25TheRTAalsoraisedCTAengagementandbudgetmanagementissuesduringthecourseofprojectimplementationthrough
thePIRsandperiodicinputs.ItisnotclearwhytheRTA’srecommendationsontheseissueswerenottakenintoaccount.
TMELTerminalEvaluationReport–31March2017
34
TheTMELprojectwasdesignedtobefundedbyvarioussources.Theoriginaltotalprojectbudgetwas
USD$31,000,000,consistingof$6,500,000fromGEF,$20,975,000commitmentfromtheGovernment
(bothcentralandmunicipal),$250,000formUNDP,and$3,275,000fromtheprivatesector
($1,125,000fromLtd.STK-Ukraineand$2,150,000fromLLGGazotron-Lux,respectively).Perthe
discussiontofollow,itshouldbenotedthattheUSD$20.9millioncommitmentisasignificantamount–
forcomparison,theUSD$14millionUNDPChinaEElightingproject(PILESLAMP)receivedatotal
commitmentfromtheChineseGovernmentofUSD$27million.
I. UtilizationofGEFFunds
Thissub-sectionprovidesdetailsabouttheutilizationofallocatedGEFfundsamountingtoUSD6.5
million.
Table3.ApprovedProjectBudgetandDistribution,byYearandbyComponent
Year1 Year2 Year3 Year4 Year5 Total %ofBudget
Component1 262,500 190,000 160,000 105,000 69,800 787,300 12%
Component2 497,700 239,000 140,000 95,000 95,000 1,066,700 16%
Component3 385,000 394,000 350,000 340,000 330,000 1,799,000 28%
Component4 480,000 330,000 285,000 264,000 265,000 1,624,000 25%
Component5 144,000 124,000 155,000 175,000 175,000 773,000 12%
OtherCosts 130,920 81,020 81,020 79,020 78,020 450,000 7%
Total 1,900,120 1,358,020 1,171,020 1,058,020 1,012,820 6,500,000
%ofBudget 29% 21% 18% 16% 16%
TheTableaboveshowsthesummaryoftheapprovedbudget,originalallocationpercomponent,andby
year.TheTablebelowshowstheactualexpendituresanddeliveryrateoftheprojectonayear-to-year
basis,includingtheactualamountandpercentageofbudgetexpenditureonaper-componentbasis.
Theactualexpenditurereflectedthefactthattheprojecttookstepstoadapttopoliticalandsecurity
challengesduringimplementation.
Table4.ActualBudgetExpenditure,byYearandbyComponent
Year1 Year2 Year3 Year4 Year5 Extended AdjustedTotal
%ofBudget
Component1 5,373 96,731 194,348 139,953 535,083 253,299 1,234,841 19%
Component2 145 33,956 221,673 292,522 754,787 356,380 1,724,224 27%
Component3 0 65,442 39,141 59,436 116,583 98,977 388,579 6%
Component4 145 149,316 410,704 410,704 352,439 130,336 1,238,900 19%
Component5 2,170 227,514 690,641 187,800 292,823 72,612 1,481,060 23%
OtherCosts 118,399 61,555 68,226 94,968 56,789 18,702 418,640 6%
Total 126,232 634,515 1,359,265 1,185,383 2,090,899 930,306 6,500,000
%ofBudget 2% 10% 21% 18% 32% 14%
TheactualspendingforComponents1,4and5weremuchhigherthanplannedforbytheoriginal
budget.Specifically,assummarizedinthetablebelow,Component5accountedfor23%oftheoverall
budget,versus12%oftheplannedbudget,a92%increase(spendingforthiscomponentincluded
TMELTerminalEvaluationReport–31March2017
35
expensesforschoolsandothereducationalactivities).Conversely,spendingforpilotprojects
(Component3)wasmuchlowerthanplanned:6%vs.28%,a78%decrease.
Table5.ActualExpendituresComparedtoOriginalBudgetAllocation(in%)
%ofOriginalBudget
%ofActualExpenditure
%ChangefromOriginal
Component1 12% 19% 57%
Component2 16% 27% 62%
Component3 28% 6% -78%
Component4 25% 19% -24%
Component5 12% 23% 92%
OtherCosts 7% 6% -7%
ItisunderstoodbytheEvaluationTeamthatthechangestotheexpenditurespercomponentwere
necessaryduetotheeconomicandsecuritysituations.Forexample,anumberofthepilotprojects
developedbytheprojectinitially(ormunicipalitiesthathaveshowninterest)arenowinconflictzones,
andthatthecarbonmarketexperiencedsignificantsetbacks.However,thereareotherareasofthe
countrythatarenotlimitedbyon-goingconflict,withhighlyinterested,motivatedlocalofficialsthat
wouldbenefitfromacombinationofhands-onexperiencewithnewtechnologies,alongwiththe
guidebooksandothertechnicalinformationdeliveredbytheproject.GiventhattheMTRhadalready
recommendedadditionaldemonstrationprojects,itwasnotclearwhyadditionalprojectswerenot
pursuedinparallelwithothermunicipaloutreachefforts.Therewasnodocumenteddecision,nor
supportinganalysistoindicatethatdemonstrationprojectswerenotsuccessfulornotcosteffective.26
Areviewoftheexpendituresindicatedthatinadditiontotheoutreachandmarketingcampaign
implementationcosts,theprojectinvestsignificantlyinanumberofstudies,marketanalyses,legal
analysesandsupportrelatedtolegislationactivities(Components1,2&4).Forexample,astudywhich
analyzedthestateofproductqualityintheUkrainianmarketinsupportofComponent2costoverUSD
$500000(forreference,thisismorethanseventimesthecostoftheinter-laboratorycomparisonstudy
thatwascommissionedin2016).27Yet,activitiestosupportaQA/QCframeworkhavebeenidentifiedby
theprojectdocuments,andwerenotimplementeduntilthelasttwoyearsoftheproject.Similarly,an
analysisofelectricitysupplycostoverUSD$175000,whichmayormaynotbenecessary,giventhat
theseissueswerealsoidentifiedbytheprojectdocument.Asummaryoftheactivitiescommissionedby
theprojectisincludedinAnnexG.
Thetablebelowshowsoverallprojectmanagementandstaffingimplementationcostsoversixyears,
averagingaboutUSD$233,000peryear,whichtotaledUSD$1.4million,orabout22%oftheoverall
projectbudget.Thisisahigherpercentageofimplementationandstaffcoststhananumberofother
26SupporttomunicipalitiesaftertheMTRcameprimarilyintheformofthedraftphase-outdecreeandtechnicalsupportfor
localdecreeadoption.27Streetlightinganalysis-originalinUkrainian“Oцінкаякостіенергоефективнихвуличнихлампі
світильниківдоступнихнаринкуукраїни”
[https://drive.google.com/file/d/0B0ENyYqfz4jedzU2Q1l6YTIwWHM/view?usp=sharing]
TMELTerminalEvaluationReport–31March2017
36
lightingprojects.28Onereasonofthishighimplementationcostwasduetothelargenumberofstaff
retainedbytheprojectovertheimplementationperiod.AspointedoutbytheMTR,thislevelofstaffing
maynothavebeennecessary,andcouldhavebeenchanged,giventhesuccessoftheawareness
campaignsandthelackofprogressonothercomponentsby2014.Itispossiblethatthediscontinuous
projectmanagementresultedinlessattentionfocusedonthishighpercentage.
Regardingtheper-componentimplementationandstaffingcosts,itcanbeseeninthetablebelowthat
staffcostsforComponents1and4accountedforthemajorityofthestaffingandprojectmanagement
budget(27%and28%,respectively).Combined,thesecomponentsaccountedforoverhalfofthe
budget.ItcanbeseenthatspendingonComponent2ceasedafter2013,indicatingthattheprogress
thatwasmadeinthisareafrom2014onwasmostlyduetoactivitiessupportedbytheCTA.Thefocus
onpublicoutreachandawarenessoverdemonstrationsisalsoreflectedhere,withtheexpenditureson
Component4(productpenetration)accountingforthehighestportionoftheoverallbudget,while
Component3(demonstrations)accountedfor8%,oraboutone-thirdofComponent4.
Table6.ProjectImplementation/StaffingCosts,perComponent
Component 2011 2012 2013 2014 2015 2016 Total
Component1 2,274 42,441 72,698 73,983 53,324 132,750 377,471
Component2 145 27,813 27,525 - - - 55,484
Component3 - 15,111 - 28,645 36,409 39,445 119,609
Component4 145 45,960 97,657 79,280 85,747 90,740 399,529
Component5 145 38,106 44,271 33,675 35,926 38,751 190,874
Other 24,559 58,356 57,816 53,484 53,355 17,211 264,781
Sub-total 27,269 227,787 299,967 269,066 264,761 318,897 1,407,748
AsofJanuary2017,theprojectwasprojectedtohaveutilized97%oftheGEF-fund.ThePMOplansto
expendtheremainingfundsbyprojectclosureon31March2017.
TheEvaluationTeamconcludedthattheGEFfundinghasbeenadaptivelyreallocatedwithinthefive
projectComponents,andthePMOhadnotifiedUNDPofsuchadjustmentswhentheyoccurred.
However,asdemonstratedbythelowdeliveryrateinYears1and2,theover-allocationforComponent
4and5,andhighstaffingcosts,theprocessoffinancialplanningwasnotsmooth.Thesituationalso
resultedintheneedforaone-yearextension.Therefore,theproject’sfinancialplanningisratedas
ModeratelySatisfactory.
II. Co-Financing
Accordingtotheprojectdesign,co-financingaccountedfor79%oftotalresourcesexpectedforthe
project,ineithercashorin-kindcontributionsfromstakeholders,includingtheGovernmentofUkraine
28Project management costs for China’s PILESLAMP, for example, was less than 10% of overall project budget of USD
$14,000,000.
TMELTerminalEvaluationReport–31March2017
37
(67%),privatesector(10.5%)andothers(1%).Duetochangesineconomicandpoliticalsituations,co-
financingfromstakeholdershavenotbeenfullyrealizedforanumberofcategories,including:
• Co-FinancefromGovernmentofUkraine(PerComponent)
• Co-FinancefromthePrivateSector(PerComponent)
• CommittedCo-financingInputsfromotherPartners(PerComponent)
Specifically,itwasestimatedbytheMTRthatonly$234,000hasmaterializedtodatefromthetotalof
almostUSD$19millioncommittedbytheGovernmentby2014,withnodirectadditionalfundingsince
thenorintootherEElightingrelatedareasasadirectresultofprojectactivities.
However,in2008theUkrainianGovernmentacceptedandapprovedimplementationofaStatetarget
scientificandtechnicalprogram“DevelopmentandimplementationofenergysavingLEDlightsources
andlightingsystemsbasedonthem.”29ThisLEDprogramwasoriginallyintendedforimplementation
during2009-2015,inparalleltotheproject.Thisprogramwasdiscontinuedin2014bytheUkrainian
government.From2009to2014,theUkrainianGovernmentspentatotalof187,477,700UAH
(approximatelyUSD$23.4million),ofwhich177,417,700UAH(approximatelyUSD$22.2million)came
fromthestatebudgetand10,060,000UAH(approximatelyUSD$1.2million)fromothersources.Itcan
beassumedthatthismayhavebeenoneoftheco-financingcommitmentactivities.Todate,no
reportableresultsareavailablefromthisprojecttosupporttheTMELprojectobjectivesoroutcomes.30
TheUNDPRegionalTechnicalAdvisoronClimateChangemitigationnotedthatinhisview,thisshould
notbecountedasco-financingasitwasnotlistedintheproject,anditwasinitiatedbeforetheproject
started(duetothefactthattheprojectdidnotworkwiththeGovernment’sLEDprogram).
ThecommittedamountsfromSTK-UkraineandGazotron-Luxdidnottranslateintoactualfunding,as
bothcompanieswentoutofbusinessduringthelifetimeoftheproject.Therewasonlyoneremaining
localproducer(ISKRA)bytheendoftheproject,andnocooperationtookplace.Nevertheless,there
havebeenco-financingfromothersintheprivatesectorforvariousprojectactivities.Forexample,a
numberofprivatesectorentities(MaxusFund,Epicenters,otherretailersandNGOs)providedco-
financingfortheproject’spolicy,awarenessandmercuryre-cyclingactivities.Anumberofretailersalso
investedinthedevelopmentofEElightingretaildisplays,supplyandservice,whicharenotfully
capturedintheco-financingreporting.
Giventhelackofrealizationfromthecommittedco-financingfromstakeholders,andeventakinginto
accountthegovernment’sLEDprogram,theEvaluationTeamconcludedthattheproject’sco-financing
resultsareModeratelyUnsatisfactory.
29http://zakon2.rada.gov.ua/laws/show/632-2008-п#n54
30Referencing GEF guidelines, which defines co-financing as “resources that are additional to the GEF grant and that are
provided by the GEF Partner Agency itself and/or by other non-GEF sources that support the implementation of the GEF-
financedprojectandtheachievementof itsobjectives,”thentheStatetargetscientificandtechnicalprogram"Development
andimplementationofenergysavingLEDlightsourcesandlightingsystemsbasedonthem”metthedefinitionofco-financing
TMELTerminalEvaluationReport–31March2017
38
3.3. ProjectResultsThissectionprovidesasummaryoftheprojectresultsandanassessmentoftherelevance,effectiveness
andefficiency,countryownership,mainstreaming,sustainability,andimpactoftheTMELproject.
Evaluationratingsforoverallresults,effectiveness&efficiency,andsustainabilityarealsoprovided.
3.3.1. OverallResults(AttainmentofObjectives)TheoverallgoaloftheTMELprojectisthereductionintheannualgrowthrateofGHGemissionsfrom
electricitygeneratedandusedforlightinginUkraine.Theprojectwasdesignedtotargetfivespecific
areasinanintegrated,coordinatedmannerinordertoaccomplishitsgoals:
1. Residential/ConsumerLighting
2. PublicSectorLighting
3. DomesticLightingSuppliers
4. ForeignLightingSuppliers
5. DomesticTestingFacilitiesforQualityControl
Thesefiveareasweretargetedbyfiveprojectcomponents,listedpreviously.Activitiesunderthese
componentsweredesignedtocomplementeachother,toproducerealanddemonstrableresultswith
outcomesthatcanbemonitored.Thefollowingsectionscontaindetailsofprogressachievedby
activitiesundereachcomponent.
Component1:Prepareandset-upnationalpolicyframeworktopromoteEElighting
UnderthisComponent,itwasplannedthattheprojectwouldassistinthedevelopmentoflegislations
tophaseoutincandescentlamps;improvethequalityofelectricsupply;introduceanenergyefficient
lightingschemethatisharmonizedwithEuropeanstandardsandnormsforEElighting;andimprovethe
collection,disposal,andwaste-handlingofCFLsandotherpotentiallyhazardouslightingproducts.
Table7containstheaccomplishmentsunderComponent1,alongwiththeevaluationratings,amore
detailedsummaryfollows.
TMELTerminalEvaluationReport–31March2017
39
Table7.Component1:Outputs,Indicators,Accomplishments,andRatings
Component1:ImprovetheNationalPolicyFrameworkforpromotingEElighting
Output SuccessIndicator EndofProjectTarget ProjectAccomplishments AchievementofTarget
Rating
1.1ImprovetheNationalPolicyframeworkforpromotingEElighting
NationalRoadmapforEElighting&markettransformationisdeveloped
TodevelopanEElighting-specificRoadmapthatisintegratedwithoverallGovernmentprioritiesforenergysecurity&savings
TheNationalRoad-mapwasdrafted,discussedandsubmittedtotheMENRon30June2015byLetter№26-45
100% Satisfactory
1.2DevelopandprepareforgovernmentalacceptancedraftlegislationforimprovingtheelectricitysupplyforUkrainianconsumers
Draftlegislationispreparedandsubmitted
Toimprovetherightofconsumersvis-à-visOblenegrosandtoprovideenforcementofelectricitysupplystandards
Theprojectcommissioned"StudyofUkrainianandinternationallegislationasitpertainstothequalityofelectricityinordertoimprovetheconsumerrightsprotection"(2015).TheresultsofthestudywerepresentedtotheMENRon30June2015byLetter№26-45.
ThestudydidnotincludesuggestionsimprovingtheLawsofUkraineduetothefactthatcurrentlyapplicableLawofUkraine"OnElectricity"regulatesprotectionofrightsofelectricityconsumersinfull.Thestudyproposedtobringinlineonlytheelectricitystandardssystem(GOST).
75% ModeratelyUnsatisfactory
1.3DevelopandsubmitforgovernmentalacceptanceanenergyefficientlightingschemethatisharmonizedwithEuropeanstandardsandnormsforEElightingandtheusageofsuchproducts
Draftlegislationispreparedandsubmitted
Toimprovetherightofconsumersvis-à-visOblenegrosandtoprovideenforcementofelectricitysupplystandards
Compendium“EULegislationintheEnergyEfficientLighting”waspublishedandformallysubmittedtotheMoJforregistration.
AcomparativeanalysisofUkrainianlegislationandEUDirectivesonthesubjectwillsubmittedtoMENRduring2017reportingperiod.
DraftLaw“OnbasicsofEnergy-efficientLightninginUkraine”(#3245)wasregisteredon7October2015.
DraftLaw“OnIntroductionAmendmentstotheLegislationofUkraine(regardingimprovingEnergy-efficiencyinLighting)”wasfiledon4May2016.
DraftResolutionontheadoptionofregulationswithregardtolabelingoflampsandluminaireswassubmittedtotheMENRinNovember2013.
100% Satisfactory
TMELTerminalEvaluationReport–31March2017
40
1.4Improvewastehandlingdirectivesforlightingproducts
Draftlegislationispreparedandsubmitted
ToproperlyclassifyandpromoteprogramstorecycleCFLsandothermercurycontainingequipment
A“DraftResolutiononcollection,disposalandutilizationofelectricalequipmentwaste”hasbeensubmittedtotheMENRinSeptember2012.Thisdraftresolutiondealswiththecollection,handlinganddisposalofCFLsandotherpotentiallyhazardouslightingproducts.
In2014thisdraftResolutionweretransformedtoDraftLawbyProjectICandpassedandregisteredinVerhovnaRada(DraftLaw3374).ThereweretwopositiveconclusionsfromVRCommitteesandonefromGNEUreceived.
ATechnicalRegulationoncollection,disposalandutilizationofelectricalequipmentwastewasdevelopedandsubmittedtoMENRon30June2015togetherwithLetter#26-45.LaterthisTechnicalRegulationwasincludedintoNationalStrategyonWasteHandling.
100% Satisfactory
OverallRating:Component1 ModeratelySatisfactory
TMELTerminalEvaluationReport–31March2017
41
Outcome1:ImprovedNationalpolicyframeworkforpromotingEElighting.
Output1.1:Developandsubmitanationalroad-mapforEElightingmarkettransformation.
CurrentStatus:TheNationalRoad-mapwasdraftedbytheexpertswithdirectassistancefromtheproject,discussedandsubmittedtotheMENRon30June2015byLetter№26-45.
Output1.2:DevelopandprepareforgovernmentalacceptancedraftlegislationforimprovingtheelectricitysupplyforUkrainianconsumers.
CurrentStatus:Theprojectcommissioned"StudyofUkrainianandinternationallegislationasitpertainstothequalityofelectricityinordertoimprovetheconsumerrightsprotection"(2015).TheresultsofthestudywerepresentedtotheMENRon30June2015byLetter№26-45.
• ThestudydidnotincludesuggestionsimprovingtheLawsofUkraineduetothefactthatcurrentlyapplicableLawofUkraine"OnElectricity"regulatesprotectionofrightsofelectricityconsumersinfull.Thestudyproposedtobringinlineonlytheelectricitystandardssystem(GOST).
• RecommendationsprovidingfortheimprovementofapplicablelegislationhavebeendevelopedandsubmittedtotheCabinetofMinistersforafinaldecision.On22.09.2016,theParliamentapprovedinprinciplethebilloftheCabinet"OnelectricenergyinUkraine."(Http://w1.c1.rada.gov.ua/pls/zweb2/webproc4_2?pf3516=4493&skl=9).
Output1.3:DevelopandsubmitforgovernmentalacceptanceanenergyefficientlightingschemethatisharmonizedwithEuropeanstandardsandnormsforEElightingandtheusageofsuchproducts.
CurrentStatus:Theprojectwasresponsiblefor,ordirectlysupportedthefollowinglegislativeactivitiesonenergyefficientlighting:
• TheprojectcommissionedthetranslationofDirective2012/27/EUof25October2012onenergyefficiency,amendingDirectives2009/125/ECand2010/30/EUandrepealingDirectives2004/8/ECand2006/32/EC.Compendium“EULegislationintheEnergyEfficientLighting”waspublishedandformallysubmittedtotheMoJforregistration(seetheletterfromMoJinAnnexG.31
• AcomparativeanalysisofUkrainianlegislationandEUDirectivesonthesubjectmatterwasundertakenin2016,tobesubmittedtoMENRduring2016reportingperiod.ThecompilationwillbesubmittedtoMENRpriortoprojectcompletion.
• DraftLaw“OnBasicsofEnergy-efficientLightninginUkraine”(#3245)wasregisteredon7October2015.
• DraftLaw“OnIntroductionAmendmentstotheLegislationofUkraine(regardingimprovingEnergy-efficiencyinLightning)”wasfiledon4May2016toreplaceapreviousdraft.ItreceivedpositiveResolutionfromtheCommitteesofVerhovnaRada,butstillispendingParliament
31Alsoavailablefromhttp://lampochki.org.ua/?p=791&lang=en
TMELTerminalEvaluationReport–31March2017
42
• DraftLaw“OnIntroductionAmendmentstotheLegislationofUkraine(regardingimprovingEnergy-efficiencyinLightning)”wasfiledon4May2016toreplaceapreviousdraft.ItreceivedpositiveResolutionfromtheCommitteesofVerhovnaRada,butstillispendingParliamentadoption.ThisdraftproposeschangestotheLawofUkraineonEnergy-savingandtotheCodeofUkraineonAdministrativeOffences.
• DraftResolutionontheadoptionofregulationswithregardtolabelingoflampsandluminaireswassubmittedtotheMENRinNovember2013.In2015thisDraftwasusedasbasisforTechnicalRegulation“Energylabelingoflampsandluminaries”(CMUResolution#340dated27/5/2015,aswellasDraftResolutiononLEDequipment.
Theprojectalsofocusedonpromotionenergyefficientlightningschemeattheregionallevel.TheDBN(ConstructionNormsandRegulations)on“NaturalandArtificialLighting”thatincludesspecificminimumenergyperformancerequirementsoflightingsystemsinmunicipalbuildings,newresidentialconstructionandstreetlightingweredevelopedbytheprojectandtransformedintoeditionoftheStateConstructionNorms(DBN)andpassedtotheMRDCHCSforapproval.TheseamendmentstoDBNarestillunderconsideration.
Beforethedevelopmentofthespecificminimumenergyperformancerequirements,theprocurementofLEDlightingequipmentbymunicipalitiesfacedsignificantdrawbacksduetothelackofrigorousrequirements.Thislackofminimumrequirementsledtothecancellationoftendersorprocurementoflow-qualityproductsthatdidnotperformasexpected.InordertoensurecompliancewiththeEuropeanandinternationalstandardsintheprocessofprocurementoflightingequipmentforpublicpurposes,thetechnicalrequirementshavebeendevelopedforcertainclassesofLEDlightingequipment.
TheTMELprojectdevelopedaDraftInstructiononpromotionenergyefficientlightingequipmentinthecitiesandproposedittothemayorsandexecutivecommitteesoflocalcouncils.Recommendationsforprocurementandfortechnicalspecificationsfortheenergyefficientlightningweredevelopedanddisseminatedamong150municipalities.Astheresult,24municipalitiesandOdessaOblastadoptedrespectiveregulationsandstartedplanningandprocurementaccordingtoInstructionsandRecommendationsabovementioned(seeexamplesathttp://lampochki.org.ua/wp-content/uploads/2015/11/All-Local-Regulatory-Acts.1.pdf.Itwasrecommendedthattheserequirementswereadoptedwhenprocurementismadewithuseofthestatebudget,municipal,investmentfundsorprivateone.Specificrequirementshavebeendesignedindividuallyforoutdoorlighting(roadsofdifferentcategories,parks,greenareas)andindoorillumination(schools,hospitals,administrativebuildings).TheguidelinesdevelopedbytheTMELprojectwereusedbytheabovementionedmunicipalitieswhentheypreparedtenderdocumentationsandorganizedprocurementprocessforlightningequipment.32
32TheserecommendationsweredevelopedbytheprojectwiththeassistanceoftheLightingAssociation’sDr.Sorokinin2015.
TMELTerminalEvaluationReport–31March2017
43
Output1.4:Improvewaste-handlingdirectivesforlightingproducts
CurrentStatus:Theprojectwasresponsiblefor,ordirectlysupportedthefollowinglegislativeactivitiesonhandlingofwastesfromlightingproducts:
• A“DraftResolutiononcollection,disposalandutilizationofelectricalequipmentwaste”hasbeensubmittedtotheMENRinSeptember2012.Thisdraftresolutiondealswiththecollection,handlinganddisposalofCFLsandotherpotentiallyhazardouslightingproducts.In2014thisdraftResolutionweretransformedtoDraftLawbyProjectICandpassedandregisteredinVerhovnaRada(DraftLaw3374).ThereweretwopositiveconclusionsfromVRCommitteesandonefromGNEUreceived.33
• ATechnicalRegulationoncollection,disposalandutilizationofelectricalequipmentwastewasdevelopedandsubmittedtoMENRon30June2015togetherwithLetter#26-45.LaterthisTechnicalRegulationwasincludedintoNationalStrategyonWasteHandling.34
MTRRecommendations&implementationstatus:
TheMTRidentifiedthattherewasnotmuchthattheprojectcandotogetUkrainianGovernmenttopromptlyapproveandimplementtherequireddecrees/regulations,thusdevelopmentofthenationalroadmapforEElightningmarkettransformationwasrecommended.Theprojectproperlyimplementedthisadviceanddevelopedrespectiveroadmap,whichwassubmittedtotheMENRon30June2015.ProvisionsofthisRoadmapwereusedbyMENRduringtheplanningprocessfortheministry,proposingrespectivedraftby-lawsbytheministry,andproposingdraftlegislationbytheMPs.TheHeadofDepartmentoftheMENRconfirmedthatMENRusedfindingsandrecommendationsfromtheRoadmap,andforcoordinationofawarenessandeducationcampaignsonthesubjectmatter.
MTRidentifiedthatUkrainehasnotyetjoinedUNEP’sen.lighteninitiative;however,inviewofen.lighten’sfocusonmarkettransformation,thissituationshouldbereassessed.Thisitemremainsoutstanding.
Component2:ImproveQualityAssurance&QualityControlsystemsforEElightingmarket
UnderthisComponent,itwasplannedthattheprojectwouldsupporttheNAERandDerzhStandardinthedevelopmentofimprovedenvironmental,energy-efficiencyandqualitystandardsandnormsforlightingproducts;improvetheUkrainianinstitutionalcapabilitiesforauditingandassessingthequalityofimportedEElightingproducts;supportlocaldevelopmentofEElightingproductsandmodernizationofnationallightingindustry;andtocreateasystemforCFLcollection,recyclingand/ordisposal.
IncourseoftheProjectimplementationTheDerzhstandardwasliquidatedandnowpartofitsfunctionsconductstheDepartmentonTechnicalRegulationinMinistryofEconomicDevelopment,seehttp://www.me.gov.ua/Documents/Detail?lang=uk-UA&id=d71e145f-452e-4412-b1f8-33http://w1.c1.rada.gov.ua/pls/zweb2/webproc4_1?pf3511=5689334 http://menr.gov.ua/garbage/5632-tekhnichna-redaktsiia-proektu-natsionalnoi-stratehii-povodzhennia-z-vidkhodamy-dlia-podalshoho-hromadskoho-obhovorennia
TMELTerminalEvaluationReport–31March2017
44
247504f403d1&title=DepartamentTekhnichnogoReguliuvannia.TheStateAgencyonEnergyEfficiencyandEnergySavingofUkrainehadbecomeasuccessorofNAERfunctionin2014.TheStateAgencyonEnergyEfficiencyandEnergySavingofUkraine(Derzhenerhoefektyvnosti)wasestablishedbyResolutionoftheCabinetofMinistersofUkraine#676dated26November2014asacentralexecutivepowerinstitution,whoseactivitiesaredirectedandcoordinatedbytheCabinetofMinistersofUkrainebyVicePrimeMinisterofUkraine-MinisterofRegionalDevelopment,ConstructionandHousingandimplementsstatepolicyintheefficientuseofenergyresources,energyefficiency,renewableenergyandalternativefuels.Thischangecreatedanumberofchallengestotheproject.Forexample,NAERhadbranchesthroughoutUkraineand400employees.In2015thebranchesofNationalagencywereliquidatedandonlythecentralbodynowexists.TheTMELprojectmadeanumberofunsuccessfulattemptstocoordinateeffortswithDerzhenerhoefektyvnosti.35
Table8containstheaccomplishmentsunderComponent2,alongwiththeevaluationratings,amoredetailedsummaryfollows.
35For example, the Project in cooperation with Institute of Semi-conductors of National Academy of Sciences suggestedcooperation with Derzhenerhoefektyvnosti to update the Resolution 992 of CMU on requirements for LED equipmentpurchased frombudgetcosts (see letter fromDCD)as theexistingrequirementsare in linewith2012LEDtechnologywhichresultsinpurchaseoflowestqualityLEDs.TheinvitationwasdeclinedbyDerzhenerhoefektyvnosti.
TMELTerminalEvaluationReport–31March2017
45
Table8.Outputs,Indicators,Accomplishments,andRatings
Component2:ImproveQualityAssurance&QualityControlSystemsforimportedandlocallyproducedlightingproductsinUkraine
Output SuccessIndicator EndofProjectTarget ProjectAccomplishments AchievementofTarget
Rating
2.1Supportthedevelopmentofimprovedenvironmental,energy-efficiency,andqualitystandardsandnormsforlightingproducts
NewstandardsforEElightingaredevelopedforDehrstandard&NEA
Anincentiveschemeisimplementedforproducts&apenaltyschemeisimplementedforsub-standardproducts
StateStandardsonLEDtechnologydeveloped(fivehaveenteredintoforce).
DSTU-PIEC/PAS62717:2014LEDModulesforGeneralLighting–PerformanceRequirements
DSTU-P7732:2015LightEmittingDiodes.CIE127-2007MeasurementofLEDs
DSTUXXXX:201XGeneralPurposeLuminairesw/LEDLightSources–PerformanceRequirements
DSTU-PІЕС/PAS62722-1:2014LuminairePerformance–Part1:GeneralRequirements
DSTU-PIEC/PAS62722-2-1:2014LuminairePerformance–Part2-1:ParticularRequirementsforLEDluminaires
DSTUIECTR62778:2014ApplicationofIEC62471fortheAssessmentofBlueLightHazardtoLightSourcesandLuminaires
75% ModeratelyUnsatisfactory
2.2ImprovetheUkrainianinstitutionalcapabilitiesforauditingandassessingthequalityofimportedEElightingproducts
1.EquipmentdeliveredtoDehrstndard
2.IndependenttestingofEElightingsamplesinstoresallowedinUkraine
Testingandcertificationofproductsisstartedandmaintained
TheNationalAccreditationAgencyofUkraine(NAAU)issuedaccreditationdocumentsfor18laboratoriesthatareequippedtotestlightingproducts,anddelivercertificationthattheymeetstandardsthatarepresentlyinplace,noneofthemrequiresequipmentupgrade.
ThreegovernmentlaboratorieswerevalidatedbytheInternationalConsultanthiredbytheproject,andthreelaboratoriesreceivedcertificatesfromILAC.
Testingofproductsinitiated.
90% ModeratelySatisfactory
TMELTerminalEvaluationReport–31March2017
46
2.3SupportforlocaldevelopmentofEElightingprojects&modernizationofnationallightingindustry
1.Workshopsheldatinterestedlightingmanufacturers
2.Businessplansdevelopedforselectedcompanies
AtleastoneILmanufacturerhasnewEElightingproductline
Duetothefactthatthereisonlyonelocalproducerofincandescentlamps(Iskra)andproductionoftheincandescentlampsisnotbannedinthecountryuntilthelegislationsareadopted,theprojectdidnotpursueactivitiesunderthisoutput.
0% N/A
2.4CreateanimprovedsystemforCFLcollection,recyclingand/ordisposal
AnewmunicipalsystemforCFLcollection,recyclingand/ordisposal
Alocallyadaptedsystemthatissustainableisdevelopedfrombestpracticesinothercountries
TheprojectconductedareviewofCFLcollectionandrecyclingschemesinWesternEuropeancountries,andafundingschemeforcollectionandrecyclingwasformulated.3NGOswereawardedbygrantstoaddresstheissueofcollectionanddisposalofmercury-containinglamps.Theyare:
• DzherelaRadosti• ZelenyiParus• Eco-clubBabylon
100% Satisfactory
OverallRating:Component2 ModeratelySatisfactory
TMELTerminalEvaluationReport–31March2017
47
Output2.1:Supportthedevelopmentofimprovedenvironmental,energy-efficiencyandqualitystandardsandnormsforlightingproducts.
CurrentStatus:SixnewStateStandardsonLEDtechnologyweredevelopedwithdirectsupportfromtheproject,fiveofthemhaveenteredintoforce.Thesestandardswillenabletheremovalofsub-standardproductsfromthemarket.Theyare:
• DSTU-PIEC/PAS62717:2014LEDModulesforGeneralLighting–PerformanceRequirements• DSTU-P7732:2015LightEmittingDiodes.CIE127-2007MeasurementofLEDs• DSTUXXXX:201XGeneralPurposeLuminairesw/LEDLightSources–Performance
Requirements• DSTU-PІЕС/PAS62722-1:2014LuminairePerformance–Part1:GeneralRequirements• DSTU-PIEC/PAS62722-2-1:2014LuminairePerformance–Part2-1:ParticularRequirementsfor
LEDluminaires• DSTUIECTR62778:2014ApplicationofIEC62471fortheAssessmentofBlueLightHazardto
LightSourcesandLuminaires
AnincentiveschemeaccordingtoMTRwassupposedtobedesignedinthesecondhalfof2014,neverthelessnoprovenrecordisavailable.
Output2.2:ImprovetheUkrainianinstitutionalcapabilitiesforauditingandassessingthequalityofimportedEElightingproducts
CurrentStatus:TheNationalAccreditationAgencyofUkraine(NAAU)issuedaccreditationdocumentsfor18laboratoriesthatareequippedtotestlightingproducts,anddelivercertificationtheymeetstandardsthatarepresentlyinplace.36Accordingtoassessmentmadebytheproject,noneofthemrequiresequipmentupgrade.ThreegovernmentlaboratorieswerevalidatedbytheInternationalConsultanthiredbytheproject,andthreelaboratoriesreceivedcertificatesfromILAC(InternationalLaboratoryAccreditationCooperation).
Aninter-laboratorycomparisontest(IC)andstafftraininginitiativewasinitiatedbytheproject,called“ConductingInter-laboratoryComparisonTestandLaboratoryStaffTrainingCourseforTestingofEfficientLightingProducts.”TheICwasconductedtoinvestigateandunderstandthemeasurementcapabilitiesoffiveselectedUkrainianlaboratorieswithregardtoSSLlightingproducts.TheICactivitywasorganizedincompliancewithISO/IEC17043ConformityAssessment–Generalrequirementsforproficiencytesting,andservedasa“startype”ICforthefiveparticipatinglaboratoriesfromUkraine.TheNationalLightingTestCentre(NLTC)inBeijing,Chinaisthereferencelaboratory.ThefinalresultsfromtheICtestingwillbepresentedattheProjectClosureConferenceinMarch,2017.
36http://naau.org.ua/reyestr-akreditovanix-oov/
TMELTerminalEvaluationReport–31March2017
48
Output2.3:SupportforlocaldevelopmentofproductionofEElightingequipment&modernizationofnationallightingindustry
CurrentStatus:Duetothefactthatthereisonlyonelocalproducerofincandescentlamps(Iskra)andproductionoftheincandescentlampsisnotbannedinthecountryuntilthelegislationsareadopted,theprojectdidnotpursueactivitiesunderthisoutput,onlytofocusonlegislativeactivities.37ItwasnotclearthatattemptsforcooperationweremadetoIskrawithoutsuccess,ortheProjectwasnotconsistentinitsapproachtorecruitpartners.ALEDequipmentproducerinPotlavaindicatedthatGazotronLuxwithdrewfromthemarketasnoneofthedomesticproducerswasabletocompetewithproductioninChina.38
Output2.4:Tocreateanimprovedsystemforcollection,recyclingand/ordisposalofmercury-containinglightingequipment
CurrentStatus:TheprojectconductedareviewofCFLcollectionandrecyclingschemesinWesternEuropeancountries,andafundingschemeforcollectionandrecyclingwasformulated.3NGOswereawardedbygrantstoaddresstheissueofcollectionanddisposalofmercury-containinglamps.Theyare:
• DzherelaRadosti(45530USD),• ZelenyiParus(45585USD)• Eco-clubBabylon(45255USD).
MTRRecommendations&Implementationstatus:
TheMTRrecommendedtherecruitmentofalocalorinternationalconsultantwithexpertiseintestinglaboratoriestovalidatetheproject’sfindingsthatall3Government-accreditedlaboratoriesinUkrainedonotrequireanyequipmentupgrade.ThiswasconfirmedforOutput2.3above.
Component3:Efficientlightinginthemunicipaleducationalsector
UnderthisComponent,itwasplannedthattheprojectwouldprovidebankablemunicipalprojectsforco-financingbaseduponmunicipalEElightingprojectsinthetertiarysector;designandimplementpilotdemonstrationprojectsin7participatingmunicipalitiesthattargetsschoolbuildings;andpayforindependentEAofthefinalpilotprojectEElightingsolutionandperformanceagainstrelevantlightingstandardsandthedesignrequirementsstatedintheservicecontracts.NotethattheseoriginaloutputsfromtheprojectdocumentweremodifiedwithinputfromthePSCatprojectinception.
Table9containstheaccomplishmentsunderComponent3,alongwiththeevaluationratings,amoredetailedsummaryfollows.
Outcome3:Implementefficientlightingdemonstrationsinthemunicipaleducationalsector
37IskraestablishedanewlineofLEDlampsduringthecourseofprojectimplementation.38The Evaluation Team were told by the project staff that Iskra by itself suspended cooperation. A number of industryrepresentativesalsoindicatedthatonlyviablewaytostayonthemarketistobuyLEDcomponentsandequipmentfromChina,orplaceproductionofcomponentpartsofLEDequipmentinChina,andonlyassembletheminUkraine.
TMELTerminalEvaluationReport–31March2017
49
Table9.Component3:Outputs,Indicators,Accomplishments,andRatings
Component3:Implementefficientlightingdemonstrationsinthemunicipaleducationalsector
Output SuccessIndicator EndofProjectTarget ProjectAccomplishments AchievementofTarget
Rating
3.1Providebankableprojectsforco-financingbyexistingcreditfacilities
Municipalitiesreceiveco-financing/creditlinestosupportEElightingprojectsdevelopedbytheproject
Municipalitiesthatneedco-financingforEElightingprojectscanreceiveitonacase-by-casebasisand/orprogrammaticbasisfromWorldBank,Ukreximbank,NEFCO,etc.
Theprojecthasco-financedafeasibilitystudyinGorlovkaforexpandingstreetlightingunderaproposedUAH3million(approx.$345,000)creditfromNEFCO.However,duetothefactthatthecityisnowinanareanotcontrolledbytheUkrainianGovernment(ithasbeenoccupiedsince2014),thecreditwasnotsettled.
50% N/A
3.2Designandimplementpilotdemonstrationprojectsin7participatingmunicipalitiesthattargetschoolbuildings
1.Numberofschoolswithsuccessfulupgrading/refurbishmentofschoollightingsystems.
2.Energysavingsperschools
Revised:Designandimplementpilotdemonstrationprojectsinatleast7participatingmunicipalities
Atleast9pilotprojectshavebeeninitiatedin8participatingmunicipalitiessinceprojectinitiation.
100% Satisfactory
3.3Provideindependentperformanceauditsofthepilotprojects
1.Numberofauditsperformed
2.Allauditnon-conformitiesareresolvedsatisfactorily
50auditsat50schoolsatleastonceperprojectlifetime
The“investment-grade”auditsnecessaryforundertakingtechno-economicfeasibilitystudiesrelatedtolightingretrofitshavebeenconductedinearly-2015.
100% ModeratelySatisfactory
OverallRating:Component3 ModeratelyUnsatisfactory
TMELTerminalEvaluationReport–31March2017
50
Output3.1:Providebankablemunicipalprojectsforco-financingbyexistingcreditfacilities
CurrentStatus:Theprojecthasco-financedafeasibilitystudyinGorlovkaforexpandingstreetlightingunderaproposedUAH3million(approx.$345,000)creditfromNEFCO.However,duetothefactthatthecityisnowinanareanotcontrolledbytheUkrainianGovernment(ithasbeenoccupiedsince2014),thecreditwasnotsettled.
Output3.2:Designandimplementpilotdemonstrationprojectsinatleast7participatingmunicipalities
CurrentStatus:Atleast9pilotprojectshavebeeninitiatedin8Municipalitiessinceprojectinitiation.Thelocationsare:39
• SadzhavaVillage:15LEDlampsequippedwithhighqualityLEDmodulesfromoneoftheworld’sLEDlightingleadershavebeenpurchasedfromoneofthelocalproducersinordertolightthemainstreetandtheschoolyardofSadzhavavillageofIvano-Frankovskregion.
• Berdyansk:Theprojectprovidedforreplacementofexistingstreetlightswithup-to-dateandefficientfixturescapableofsavingenergyandbeoperableinadverseweatherconditions.
• Lugansk:AsaresultofimplementationoftheprojectforreplacementofoutdoorlampsjointlyimplementedbytheTMELprojectandMiskSvitloPublicUtilityCompany,Luganskinstalled1100energy-savinglamps.Thecitypurchasedandinstalledanother1,200energy-savinglampsforadditionallocations.
• Dobrotvir:Theprojecthelpedtoreplaceincandescentlightbulbsinanumberofmunicipalresidentialapartmentbuildings(entrancelobbies,staircases)withLEDlightingsources.Estimatedtotallifetimeenergysavingsforthisactivityisabout1387.6MWh,withfinancialsavingsestimatedtobe107,1076UAH,andCO2emissionsreductionof1387tons.
• NesherivVillage:Thevillagereplaced75150Wmercury-containingstreetlightsourcesfor7530WLEDlights.ItalsoreplacedincandescentaccentlightingsystemontheCathedralofSaviour’sTransfigurationMonasteryCompoundwith2212WLEDlightsources.Estimatedannualenergysavingsisabout34.7MWh,whichequalstoUSD$3,225.Energysavingsbasedupontheusefullifeoflightbulbsinstalledisestimatedtobe866.7MWhover25years,ReductionofCO2emissionsbasedupontheusefullifeoflightbulbsinstalledisestimatedtobe867tons.
• Gorlovka:TheTMELprojectprovidedforengineeringcontractorservicesinordertosupportacomplexupgradeoftheoutdoorlightingsystemandretrofittingofthetrafficlightsinthecity.Forthepurposesofthisproject,NEFCOwasplanningtopartiallycovertheengineeringdesignandfurtherinstallationofsmartlightingmanagementsystem.Thecityalsoplannedtoreplace22street-lightluminairesandprovidefreeWi-FiaccesswithinthesectionlimitedbyRudakovandLeninstreets.
• UNDP,Kiev:Thisprojectincludedinstallationoften35WattLEDdownlightsandthree(3)600х600mm56WattLEDlightingfixtures.
39NotethatLuganskandGorlovkaareareasnolongerundercontroloftheUkrainianGovernment.
TMELTerminalEvaluationReport–31March2017
51
• UNDP,Kiev:Thisprojectincludedinstallationoften35WattLEDdownlightsandthree(3)600х600mm56WattLEDlightingfixtures.
RecommendationsforprocurementandtechnicalspecificationsfortheEElightingwereplacedintheHandbookstitled“EnergyEfficientstreetlighting,buildingandexploitationand“EnergyEfficientLighting”disseminatedamongmunicipalities(seedetailsunderOutput5.2).Itresultedinadoptionby15municipalitiesofrespectiveregulationsforprocurementandtenderprocessthatprovidedthemwithturn-keyEElightingsolutions.
Output3.3:Provideindependentperformanceauditsofthepilotprojects
CurrentStatus:Theprojecttargeted50auditsfor50projectsforthisactivity.The“investment-grade”auditsnecessaryforundertakingtechno-economicfeasibilitystudiesrelatedtolightingretrofitshavebeenconductedinearly-2015.40
MTRRecommendations&implementation:
Recommendationsweremadetoensurethatmunicipalstakeholdersarethoroughlybriefedontheobjectivesofthepilots,aswellasthattheirlearningprocessisfacilitatedinawaythattheyareabletoreplicatepilotsonthebasisoftheirownresources.WithissuinganddisseminationaHandbookthefirstrecommendationwasexecuted.Astotheorganizationofthelearningprocessformunicipalstakeholders,intheframeworkof2014and2015conferencestherewasessentialpartoftheactivitydedicatedtodemonstrationoftheprojectactivitiesonpilotingincludingthemunicipalstakeholders(seepresentationhttps://drive.google.com/drive/folders/0B06VW9ErMx4EQXczY1hSQThaeD).
Component4:ImproveEElightingproductpenetrationintheResidentialSector
UnderthisComponent,itwasplannedthattheprojectwouldcollaboratewithlocalandinternationalEElightingproducers,consumerbankslikePlatinumBank,andnationalretailernetworkstodesignandimplementEElightingdisseminationprogramforresidentialconsumers;introduceEElightingandGreenLightLabelcomponentineducationalcurriculaforschoolchildren;designandimplementmunicipalPRcampaignsonEElightinginparallelwithimplementationofdemonstrationprojects;andtailorselectedglobalCFLspromotionalactivitiestoUkrainianconsumers.
Table10containstheaccomplishmentsunderComponent4,alongwiththeevaluationratings,amoredetailedsummaryfollows.
Outcome4:ImprovedEELightingproductpenetrationintheResidentialSector
40Detailsoflocationsandissuesaddressedhaveyettobeprovided.
TMELTerminalEvaluationReport–31March2017
52
Table10.Component4:Outputs,Indicators,Accomplishments,andRatings
Component4:ImproveEElightingproductpenetrationintheResidentialSector
Output SuccessIndicator EndofProjectTarget ProjectAccomplishments AchievementofTarget
Rating
4.1DesignandimplementtheCFLdisseminationprogramforresidentialconsumers
1.EstablishmentofretailchainEElightingpromotionprogram
2.Amount/volumeofEElightingpurchasedviatheprogram
EstablishmentofretailchainEElightingpromotionprogramavailableformostconsumers
Target2municipalitiesinitiallyandspreadtotop5populationcentersbyendofproject
ConsumerawarenesssurveytobeconductedbytheprojecttoestablishbaselinelevelofawarenessforEElightingtechnologies
Threepromotionalcampaignshavebeendevelopedandconductedintheretailstores:
1.All-UkrainianLEDpromotionalcampaign(2016):Covered35retailstores(Epicentre)in23cities.
2.All-UkrainianawarenesscampaignonEElighting(2ndphase)(2015):Thiscampaignfocusedprimarilyonbroadcastandprintedmedia,publictransitandout-of-homeadvertising.
3.UkraineretailchainEpicenter(largestinUkraine)andsomeother(e.g.Foxtrot)areimplementingamicro-financefacilitywiththesupportofUkrsibbank,PlatinumBankforresidentialcustomers
• Followupsurveyswereconducted,withthefollowingresults:
• All-Ukrainianawarenesscampaignswasestimatedtocovermorethan27millionUkrainiansoftheage17andolder(73,8%),wholiveinruralareas,townsandcities.
• Advertisingeventsintheretailstoreswasestimatedtocover23,4%ofUkrainiansoftheage17andolder(about7,5millionpeople).
• IncreasesalesofEElightingproductsusedinhouseholdsto53,3%.
• MarketshareofEEbulbsusedinhouseholdshasgrownby22,5%
100% HighlySatisfactory
4.2IntroduceEElightingandGreenLightcomponentsineducationalcurricula
1.Creationofeducationalmaterials
2.NumberofinstitutionsacceptingtheeducationalcurriculathroughoutUkraine
DatafromUNDP/MinistryofEducationregardingdisseminationofmaterialsandincorporationintoschoolcurriculum
TheAllUkrainianEducationalAwarenessCampaignonEnergyEfficientLightinginSchoolswasconductedfromMarch2014untilFebruary2015.
6896schoolsofall24Ukrainianoblasts/regions,includingcities,townsandruralareas,wereinvolvedinthecampaign.12000lessonsforschoolchildrenand25seminarsforteachershavebeenheld.
Thecampaignisestimatedtocover55,3%ofallschoolstudentsin
100% Satisfactory
TMELTerminalEvaluationReport–31March2017
53
Ukraine(2334766schoolchildren),19,6%ofalladultsinUkraine(6199532teachersandparents).
4.3DesignandimplementmunicipalPRcampaignonEElighting
1.ScopeofPRCampaign
2.EffectivenessofPRCampaign
Campaignvisibleinthetop10municipalitiesinUkrainebypopulation
Ameasureableincreaseof20%inawarenessbyconsumersinthemunicipalities
Fournation-widepromotionalandPRcampaignshavebeendevelopedandconducted:
1. All-UkrainianawarenesscampaignsonEElighting(1stphase):ConductedfromDecember2012throughJune2013coveringcities,townsandruralareasofall25regionsofUkraine.
2. AllUkrainianEducationalAwarenessCampaigninSchoolsonEnergyEfficientLighting(2014-2015):seeoutput4.2
3. All-UkrainianawarenesscampaignsonEElighting(2ndphase):ConductedfromMay2015throughFebruary2016coveringcities,townsandruralareasofall24regionsofUkraine.
4. All-UkrainianLEDpromotionalcampaign(2016):Seeoutput4.1.
Post-campaignsurveysbytheprojectshowedthatthelevelofpublicawarenesshasincreasedby7%,with3.6%directlyattributedtotheAll-UkrainianawarenesscampaignsonEElighting(1stphase)and3.4%asaresultofcampaignsofEElightingequipmentproducersincourseofyears2012-2013,andby35,7%in2016comparedto2013dueto2014-2016project’scampaigns.Ingeneral,incomparisonwith2012theawarenesslevelontheEElightingbenefitshasgrownby42,7%asof2016.
100% HighlySatisfactory
4.4TailorselectedglobalEElightingpromotionalactivitiestoUkrainianconsumers
1.ScopeofPRCampaign
2.Volumeofsales(data)oftargetedproducts
IncreaseinsalesofEElightingproductsbyatleast20%peryearattheparticipatingoutlets
Fournation-widepromotionalandPRcampaignshavebeendevelopedandconducted(seeoutput4.3)resultingin:
§ SalesofEEbulbsusedathouseholdshaveincreasedby53,3%in2016(vs.2012)
§ MarketshareofEEbulbsusedathouseholdshasgrownby22,5%in2016comparedto2012.
§ MarketshareofLEDbulbsusedathouseholdshasmade48,5%incomparisonwithCFLsandincandescentbulbsin2016.
100% Satisfactory
OverallRating:Component4 HighlySatisfactory
TMELTerminalEvaluationReport–31March2017
54
Output4.1:DesignandimplementEElightbulbdisseminationprogramsforresidentialconsumers
CurrentStatus:Thefollowingthreepromotionalcampaignshavebeendevelopedandconductedintheretailstores:
1.All-UkrainianLEDpromotionalcampaign(2016):Covered35retailstores(Epicentre)in23cities.Thecampaignincluded:
• GivingeventparticipantsdiscountsforbuyingaLEDbulbintheEpicentreinexchangeforausedmercury-containingbulb.
• Disseminatingthebooklet“ABCBookofLightBulbs”developedbytheprojectamongthecustomersintheEpicentre
• ProvidingtheEpicentreshopassistantswithtrainingsessionsonhowtoproperlychooseenergyefficientLEDbulbsfortheconsumersfordifferentroomsandpurposes
2.All-UkrainianawarenesscampaignonEElighting(2ndphase)(2015):Thiscampaignfocusedprimarilyon:
• Broadcastandprintedmedia,publictransitandout-of-homeadvertising.Italsocoveredretailstores(viaairingofvideos/commercialsonEElightingonvideoscreens).19retailstores(Epicenter)in9citieshavebeencovered.
• Demonstrationcampaignoncollectionanddisposalofusedmercury-containingbulbsandEElightingpromotion(2015).Itcovered5retailstoresEpicenterin5cities.
Followupsurveyswereconducted,withthefollowingresults:
• All-Ukrainianawarenesscampaignswasestimatedtocovermorethan27millionUkrainiansoftheage17andolder(73,8%),wholiveinruralareas,townsandcities.
• Advertisingeventsintheretailstoreswasestimatedtocover23,4%ofUkrainiansoftheage17andolder(about7,5millionpeople).
• IncreaseinsalesofEElightingproductsusedinhouseholdsreached53,3%.• MarketshareofEEbulbsusedinhouseholdshasgrownby22,5%from2012to2016.
Asthesurveysarebasedondeskreview,thereisnoprovenrecordsonthefactthatsignificantincreaseinsalesofEElightingproductsandgrowthofmarketshareofEEbulbscanbeattributedsolelytosuccessoftheawarenesscampaignheldbytheProject.41
DecreaseofpricesofEElightingproductsandgrowingtariffsforelectricityalsoweresubstantialeffectfactors.Atprojectstart,mostEElampscarriedaone-yearwarranty,todaythewarrantiesaretypicallytwotothreeyears.Thesetwoabovefactors(availabilityofcredithelpingtomakehigherpricedlampsmoreaffordable;andlongertermofwarrantyforEElamps)arebelievedtohelpresolvepartiallythe
41Notethatoneofthesurveyssupposedlycoveredover40,000subjects.Thisvolumeofsurveyresponsewasnotkeptintheprojectoffice,butwiththecontractor–whichtheEvaluationTeambelievestobeacontractualoversight.
TMELTerminalEvaluationReport–31March2017
55
problemoflow-qualitybulbsfloodingthemarket,whichwascitedanissueatthestartoftheprojectbytheprojectdocument.
Aspointedoutearlier,therapidpaceofadvancementinLEDtechnologiesresultedinlowerpricingandfasterpenetrationofLEDsthananticipated(andreducedthevolumeofCFLsfordisposal).Thisfasterpenetrationdefinitelyhelpedtoincreasetheadoptionratesintheresidentialsector.However,theincreasedavailabilityofLEDsalsopresenteditsownchallengesinissuesoftestingforquality,userfamiliarity,andhowtheprojectcanproperlymeasuretheirimpact–currently,resultsprovidedbytheprojectassumedthatgainsmarketpenetrationweretheresultsoftheproject’sactivities,however,thisassumptionunderestimatestheeffectoftheoverallrapiddeclineofproductpricingandincreasedavailabilityovertheprojectimplementationperiod.
3.UkraineretailchainEpicenter(largestinUkraine)andsomeother(e.g.Foxtrot)areimplementingamicro-financefacilitywiththesupportofUkrsibbank,PlatinumBankforresidentialcustomerstopurchasegoodsincludingEElightlampsoncredit.
Output4.2:IntroduceEElightingandGreenLightLabelcomponentineducationalcurricula
CurrentStatus:TheAllUkrainianEducationalAwarenessCampaignonEnergyEfficientLightinginSchoolswasconductedfromMarch2014untilFebruary2015,withthefollowingelements:
Theprojectcommissionedastudyofthecorecurriculum,standardandfunctionalcurricula,andsyllabusfortheschoolyear.Thesubjectsandtopicsdirectly/indirectlyrelatedtoEElightinginprimaryschool(1-4grades),secondaryschool(5-9grades)andhighschool(10-11grades)wereidentifiedaswellasthelessons’synopsesforeachschoolgradesweredeveloped.
Theprojectalsocommissionedlogoandsloganforusewiththiscampaign.Campaignmaterialsincluded:
• Informationalbookletsforpupilsofsecondaryandhighschools,bookmarksforpupilsofprimaryschool,thematictransformablebookletsforpupilsofsecondaryschool),
• Promotionalgiftsforschoolchildren(kitsofthematicstickersforpupilsofprimaryschool,thematicfridgemagnets),andprintedmaterialsandpromotionalgiftsforteachers(bookletsandthematicfridgemagnets),
• EducationalanimatedcartoonandfilmonEElightingaredesigned,producedanddeliveredtoschools.PhysicsClassroomdemonstrationstandsforschoolbulletinboardsaredeveloped,producedanddeliveredtoschools,aswellasEELightingandinformationalleafletsforusingduringlessons.
• 10000textbooksEnergyEfficientLightingand5000practicaltextbooksonEnergyEfficientOutdoorLightinghavebeendeveloped,produced,anddisseminatedin1500establishments(universities,colleagues,libraries,localmunicipalities,etc.).
Generally,6896schoolsofall24Ukrainianoblasts/regions,includingcities,townsandruralareas,wereinvolvedinthecampaign.12000lessonsforschoolchildrenand25seminarsforteachershavebeen
TMELTerminalEvaluationReport–31March2017
56
held.Thecampaignisestimatedtocover55,3%ofallschoolstudentsinUkraine(2334766schoolchildren),19,6%ofalladultsinUkraine(6199532teachersandparents).Numberofeffectivecontactswithtargetedaudiencewas29,4%ofallUkrainiansattheageof6andolder(8534298childrenandadults).DuringtheAll-UkrainianawarenesscampaignsonEElighting(1stphase)conductedin2013,informationalbookletsforstudentswerealsodevelopedanddisseminated.
Output4.3:DesignandimplementmunicipalPRcampaignsonEElighting
Thefollowingfournation-widepromotionalandPRcampaignshavebeendevelopedandconducted:
1. All-UkrainianawarenesscampaignsonEElighting(1stphase):ConductedfromDecember2012throughJune2013coveringcities,townsandruralareasofall25regionsofUkraine.Itincluded:designofcampaignsloganandlogo;broadcastingdevelopedcommercialsonTVandRadiochannels;placementofthematicpagesonsocialnetworksandonspecializedInternetportals;placementofinformationalpostersonbillboardsandcity-lightsandinformationalblocksoninvoicesformunicipalservices;placementofthematicarticlesinprintedissuesandinformationleafletsinresidentialblocks/elevators;distributionofdevelopedinformationalbookletsforstudents,environmentalservicesandmunicipalservicesdepartments.
2. AllUkrainianEducationalAwarenessCampaigninSchoolsonEnergyEfficientLighting(2014-2015):seeoutput4.2
3. All-UkrainianawarenesscampaignsonEElighting(2ndphase):ConductedfromMay2015throughFebruary2016coveringcities,townsandruralareasofall24regionsofUkraine.Thesocialevent“TheDayofLightinYourCity”wasconducted,andthethematicmagnets,fansand‘HatandScarf’setshavebeenproducedanddistributed.Theprojectalsocommissionedaneducationalfilm,children’scartoon,andpromotionalcommercialstobeairedonTVchannels,videoscreensinretailchainstoresandundergrounds,aswellasonout-of-homeboards.Publictransportationfacilities(busesandundergrounds)werealsousedascommunicationchannelsforplacementofinformationalpostersandcity-lights,andbranding.
4. All-UkrainianLEDpromotionalcampaign(2016):Seeoutput4.1.
Post-campaignsurveysbytheprojectshowedthatthelevelofpublicawarenesshasincreasedby7%,with3.6%directlyattributedtotheAll-UkrainianawarenesscampaignsonEElighting(1stphase)and3.4%asaresultofcampaignsofEElightingequipmentproducersincourseofyears2012-2013,andby35.7%in2016comparedto2013dueto2014-2016project’scampaigns.Ingeneral,incomparisonwith2012theawarenesslevelontheEElightingbenefitshasgrownby42.7%asof2016.
NOTE:Samecaveatsapplytothestatementsaboveregardingdiscountingthecontributionofreductioninpriceandincreaseinproductavailabilitytowardsincreasedawareness.
TMELTerminalEvaluationReport–31March2017
57
Output4.4:TailorselectedglobalEElightingpromotionalactivitiestoUkrainianconsumers
CurrentStatus:Fournation-widepromotionalandPRcampaignshavebeendevelopedandconducted(seeoutput4.3)thathaveledtothefollowingresults:incomparisonwith2012salesofEEbulbsusedathouseholdshaveincreasedby53,3%in2016.Moreover,marketshareofEEbulbsusedathouseholdshasgrownby22,5%in2016comparedto2012andhasmade72,6%in2016.ThemarketshareofLEDbulbsusedathouseholdshasmade48,5%incomparisonwithFCLsandincandescentbulbsin2016.
TheMTRmentioneddevelopmentbytheprojecttogetherwithMaxusasecondphaseoftheAll-UkrainianPublicAwarenessCampaignonEELighting.ThisphaseofCampaignincludedTVcommercials,postersanddistributionofleaflets/calendarsinsupermarkets.TheprojecttargetforthisoutputwasincreaseinsalesofEElightingproductsby20%attheparticipatingoutlets.
NOTE:Samecaveatsapplytothestatementsaboveregardingdiscountingthecontributionofreductioninpriceandincreaseinproductavailabilitytowardsincreasedawareness.
MTRRecommendations&implementation:
TheMTRmadetherecommendationtohaveawarenessactivitiesgoinghandinhandwithotheractivitieswasonlypartiallyimplemented(asistheoutput’sobjectiveof“tailorselectedglobalEElightingpromotionalactivitiestoUkrainianconsumers–itwasnotclearfromthematerialsthatwhatglobalEElightingpromotionalactivitieswereadoptedandtailored).Atthesametime,therecommendationforprojecttodeveloparoadmapthatshouldensuresustainedawarenessraisingaftertheprojecthascometoanendwasnotimplementedtofullscale.Partially,thisrecommendationcanbeconsideredasimplemented,asNGOs,includingAll-UkrainianEnvironmentalLeagueconfirmtheircommitmentandcapacitytocontinuetheawarenesscampaignusingmaterialspreparedbytheProject.
Component5:DisseminationandReplicationoftheProjectResults
UnderthisComponent,itwasplannedthattheprojectwoulddevelopprojectwebsite;developtrainingbookletforschools,incooperationwiththeschoolswhichparticipatedintheproject,anddevelopatrainingbookletforschoolchildrenonenergy-efficientlighting;designthesecondstageofEElightingdemonstrationprojectsinmunicipalbuildingscoveringatleast20municipalitiesacrossUkraineforusewiththesustainablefinancingmechanism(s);developandconductseminarsformunicipalitiesonthesustainablefinancingmechanismoutlinedunder3.1&5.1andoncarbonfinance,andotheralternativefinancing;andsupportorganizationsthatfocusonenergyefficiencyinthepublicsector/municipalitiesofUkraine.
Table11containstheaccomplishmentsunderComponent5,alongwiththeevaluationratings,amoredetailedsummaryfollows.
Outcome5:DisseminationandReplicationoftheProjectResults
TMELTerminalEvaluationReport–31March2017
58
Table11.Component5:Outputs,Indicators,Accomplishments,andRatings
Component5:DisseminationandReplicationofProjectResults
Output SuccessIndicator EndofProjectTarget ProjectAccomplishments AchievementofTarget
Rating
5.1Implementationofprojectwebsite
Websiteupandmaintainedregularly
Websiteupdatedwithhistorical,currentandplannedprojectactivities&progress.
Aprojectwebsite(www.lampochki.org.ua)wasdevelopedandisbeingmaintainedbytheprojectwithregularupdateofinformationonallon-goingpilots,contacts,newdevelopmentsetc.,thewebsitewasupgradedandiscurrentlyupdatedonby-weeklybasis.
100% Satisfactory
5.2Designthesecondstageofdemonstrationprojectcoveringatleast20municipalitiesacrossUkraine
Municipalapplicationstoparticipateinnewprojects/program
Selectionresultsfromtheprocessselecting100+schools
Initiallightingsite-auditsattheparticipatingschools
Over40applicationsfrommunicipalitiestoparticipateintheprogram
Applicationsresultin20municipalitiesparticipatingandsupportingtheprogram
100to200schoolsparticipating,dependingonfunding
Theprojecthasdevelopedthefollowinghandbooks:“EnergyEfficientStreetLighting,Building,andDevelopment”;and“EnergyEfficientLighting”withintroductionandrecommendationsonEElightningschemesonamunicipallevelandforeducationalpurposes.Over10,000copiesofthesepublicationshavebeendistributedtomunicipalities.ThisoutputiscloselyinterlinkedwithOutput3.2.
90% ModeratelySatisfactory
5.3Supportandworkwithlocalorganizationsthatfocusonenergyefficiencyinthepublicsector
ThenumberandtypeoforganizationswhichcooperatewithGEF/UNDPprojects
Cooperateandsupportatleast2regional/nationalorganizations(staffinseverallocations)
Cooperatewithatleast1organizationinthe12+pilotmunicipalities
Cooperationhasbeeninitiatedandfurtherdevelopedwithseveralorganizations,regardingenergyefficientlighting,andbestpracticesoncollectionanddisposalofmercury-containingCFLlamps.Theyinclude:
• All-UkrainianEcologicalLeague,• StateEcologicalAcademyofPost-GraduateEducationand
Management,• Eco-clubBabylon,• InternationalEconomicCommittee,• FoundationonDevelopmentofEnvironmentandEnergy
Market,• DzherelaRadosti,• ZelenyiParus,
100% HighlySatisfactory
TMELTerminalEvaluationReport–31March2017
59
5.4DevelopandconductseminarsformunicipalgovernmentsregardingEEprojectsandopportunitiesforleveragingcarbonfinance,andotheralternativefinance
Developmentofthetrainingprogram
Numberofseminars
Participantsattheseminars
Newprojectideas&financingproposalsdevelopedattheoutcomefromseminars
Atleast10seminarsin10municipalities
Atleast2newprojectsdeveloped&implementedasaresultoftheseminars
CarbonfinanceseminarswereheldwithEco-Forumin2012andNEFCOinApril2013.Duetothefactthatthereisnolongercarbonfinancemarketexistssince2014,thetargetforthisoutputtohave10seminarsin10municipalitiesheldwasnotachieved.
N/A N/A
OverallRating:Component5 Satisfactory
TMELTerminalEvaluationReport–31March2017
60
Output5.1:Implementprojectwebsite
CurrentStatus:Aprojectwebsite(www.lampochki.org.ua)wasdevelopedandisbeingmaintainedbytheproject.PertheMTR’srecommendationsonregularupdateofinformationonallon-goingpilots,contacts,newdevelopmentsetc.,thewebsitewasupgradedandiscurrentlyupdatedonby-weeklybasis.
Output5.2:Designthesecondstageofdemonstrationprojectcoveringatleast20municipalitiesacrossUkraine
CurrentStatus:Theprojecthasdevelopedthefollowinghandbooks:“EnergyEfficientStreetLighting,Building,andDevelopment”;and“EnergyEfficientLighting”withintroductionandrecommendationsonEElightningschemesonamunicipallevelandforeducationalpurposes.Over10,000copiesofthesepublicationshavebeendistributedtomunicipalities.ThisoutputiscloselyinterlinkedwithOutput3.2.
Output5.3:Supportandworkwithlocalorganizationsthatfocusonenergyefficiencyinthepublicsector
CurrentStatus:Cooperationhasbeeninitiatedandfurtherdevelopedwithseveralorganizations,regardingenergyefficientlighting,andbestpracticesoncollectionanddisposalofmercury-containingCFLlamps.Theyinclude:
• All-UkrainianEcologicalLeague,• StateEcologicalAcademyofPost-GraduateEducationandManagement,• Eco-clubBabylon,• InternationalEconomicCommittee,• FoundationonDevelopmentofEnvironmentandEnergyMarket,• DzherelaRadosti,• ZelenyiParus,
Output5.4:DevelopandconductseminarsformunicipalgovernmentsregardingEEprojectsandopportunitiesforleveragingcarbonfinance,andotheralternativefinance
CurrentStatus:CarbonfinanceseminarswereheldwithEco-Forumin2012andNEFCOinApril2013.DuetothefactthatthereisnolongercarbonfinancemarketexistsinUkrainesince2014,thetargetforthisoutputtohave10seminarsin10municipalitiesheldwasnotachieved.
MTRRecommendations&implementation:
TheMTRrecommendedtomorefrequentlyupdatethewebsitewithlatestinformationontheprojectingeneralandwitheasytounderstand,forthegeneralpublic,informationonprojectachievements.Thewebsiteiscurrentlyupdatedonaregularbasis.However,theanalyticaltoolsfortrackingwebsite“hits”arenotavailableontheregularwebsite,butonlyonsocialmediaaccounts.UNDPCO,asamatterofpolicy,doesnotfavortheideaofhavingprojectsites,butratherconcentratingallprojectinformationonUNDPCOwebsite.
TMELTerminalEvaluationReport–31March2017
61
SummaryofResults:
Overall,theprojecthadimplementedactivitiestosupport19outputsfrom5projectcomponents,completingactivitiesfornearly16,andachieving“Satisfactory”or“HighlySatisfactory”ratingson11outputactivities.Basedonanevaluationoftheactivities,outputs,andachievements,theevaluationteamconcludedthattheoverallresultsoftheTMELprojectModeratelySatisfactory.Thetablebelowprovidesasummaryoftheresultsandratings:
Table12.SummaryandOverallProjectRating
ProjectComponent #ofSub-components OverallRating
Component1 4 ModeratelySatisfactory
Component2 4 ModeratelySatisfactory
Component3 3 ModeratelyUnsatisfactory
Component4 4 Satisfactory
Component5 4 Satisfactory
OverallRatingforTheProject ModeratelySatisfactory
3.3.2. RelevanceFromthedevelopmentperspective,UNDPandGEFarecommittedtoclimatechangeadaptationandmanagementinternationally,andcontinuetosupportenergyefficiencyprojectsasanimportanttooltowardsreductioninGHGemissions.Further,internationalexperiencehasshownthatcomprehensiveenergyefficientlightingprogramshaveproventobeeffectiveintheintroductionofinstitutionalandmarketchangesthatcanhelptopavethewayforotherenergyefficientproductsandinitiatives.
Pertheprojectdocument,theGovernmentofUkrainehasestablishedaroadmaponEnergyStrategyforUkrainetill2030.AccordingtothisStrategy,in2020Ukraineplanstosaveupto470milliontonsofequivalentoil,whichwillreduceimportofenergyresourcesbyabout38billionUSD.Therefore,energy-efficiencymeasuresareapolicyfocusforthedrivetoimproveenergyindependenceandsecurityforUkraine.Furthermore,theeconomiccrisisandnaturalgascrisesbetweenRussianandUkrainethattookplaceinrecentyearshasalsohadastrongimpactonpolicyformationandgovernmentalgoalsinUkraine.
AlthoughtheUkrainiangovernmenthassupportedenergyefficiencythroughsomepolicymeasuresfundsarelackingforimplementinglarge-scaleenergy-efficiencyprograms,ingeneral.Consequently,thereremainsuntappedpotentialforthedevelopmentandimplementationofnewenergyefficienttechnologiesinUkraine,includingenergy-efficientlighting.InlinewiththeGovernment’spriorities,theTMELprojecthelpedtoaddressanoftenover-lookedissueinthereductionofgreenhousegasemissionsthroughlarge-scaleimprovementsinenergyefficientlightinginUkraine'sresidentialandcommunalsector.
TMELTerminalEvaluationReport–31March2017
62
ThegovernmentofUkraineanditsimplementationagency–theMinistryofEnvironmentalProtection(TheMENRwasnottheGEFFocalPointatthattime)–hadalsoindicatedthatenergyefficiencywasoneofthegovernment’sprioritiesfromtheprojectoutset.Certainlevelsofgovernmentcommitmentremainedhigh,especiallyregardingeducation,increasingawareness,anddisseminatingresultsoftheproject.Giventhatgovernmentagenciesandtheirfocushaveshiftedsincetheprojectinception,theMinistryofEcologyandNaturalResourcesremainstobeamajordesignatedbodyfortheproject.Inaddition,theMinistryofCommunalServices,whichwasdefinedintheprojectdocumentasastakeholder,hasunderwentseveralreformsandrestructuringsinceprojecthasbeenlaunched.AsofnowthereisaMinistryofRegionalDevelopment,Construction,HousingandCommunalservices(MRDCHCS)withextendedscopeofauthorityandstaffchanges.MRDCHCShasindicatedrenewedinterestinenergyefficientlighting.
Consequentlyitcanbeconcludedthatsinceitsinception,theTMELprojecthasbeenrelevanttotheUkrainiandevelopmentcontextandneedsofallkeystakeholdersinvolved.However,thechallengesremainonanumberoffronts,inparticularonthelegislativeside,withanumberofdraftlawsyettobepassedbytheParliament.
3.3.3. EffectivenessandEfficiencyIngeneral,acasecanbemadethattheprojecthasbeeneffectiveindeliveringmostofitsplannedactivitiesandoutputs.However,challengesremaininthefinalstagesofimplementation,duetothevariousexternalfactors,andtheUkrainegovernment’swillingnessinadoptingthekeyprojectoutputsinorderfortheproject’sfulloutcomestobeachievedinthenearfuture.
Amongtheproject’seffectiveactivitieshaveincludedthedevelopmentofaNationalRoadmap,thedevelopmentofstandardsandspecifications,comparativeanalysisofUkrainianlegislationandEUDirectives,providingaccreditationsupportfortestinglaboratories,andawareness-raisingthroughoutreachactivitiesandpilots.
Forexample,activitiesontheNationalRoadmapwereinitiatedasoneoftherecommendedactionsbytheMTRinMarch2014.TheRoadmapwascompleted,discussedandsubmittedtotheMENRon30June2015byLetter№26-45.ImplementationofthisRoadmapwouldhaveprovidedencouragementtotheprivatesectortoconverttoenergyefficientlightingtechnologies,andwillalsoleadtoaprogressivereplacementofinefficienttechnologiesbyconsumers.Similarly,thesupporttovariouslaboratoriesfortestingandaccreditationonlyoccurredafteraCTAwascontractedbytheprojectin2015.Yet,activitiesinthisareahaveincreasedawarenessoftestingmethodologies,needforlaboratoryaccreditation,andimprovedthecountry’scapacityontestingofefficientandsafelightingproducts.Finally,thesuccessiveawarenessandoutreachcampaignshavecombinedtoincreaseconsumerawarenessandacceptanceofnewlightingtechnologies,andgainedrecognitionfortheproject’seffortsnationallyandperhapsinternationally.42
42There has been little or no contact between the project and other UNDP and UNEP lighting andmarket transformationactivitiesasenvisionedintheprojectdocument.
TMELTerminalEvaluationReport–31March2017
63
Intermsofoperationalissuesandimplementation,theproject’sexternalimplementationarrangementsreliedonexistingorganizationalstructuresinthelightingindustry,aswellasNGOsthataretypicalinothermarkets.However,thiswasnotentirelyapplicable,asthelackofastronglightingindustryassociationandanindependentconsumerorganization,aswellasamarketsurveillanceandenforcementinfrastructuremadeitdifficulttopromoteandmaintainproductqualityassurance.Internally,factorscontributingtoreduceoperationalefficiencyincludedinconsistentadministrativesupport,changesinkeystaff,andhighcosts.
TheEvaluationTeamalsonotedthatthecoordinationandcooperationamongthestaffresponsibleforkeyprojectoutcomescouldhavebeenimproved,whichmayhelptoincreaseimplementationefficiency(forexample,theintegrationofQAintoproductawarenesscampaign’smessaging).Thecombinationofchallengesandissuescontributedtothedelayofprojectclosurefrom2016to2017.However,throughthesupportofanexperiencedprojectmanagerandCTAinthelasttwoyears,theprojectwasabletoutilizetheextensiongrantedbyUNDPtodeliveranumberofneededoutputs.
Overall,itisconcludedthattheTMELproject’seffectivenesswasSatisfactory,whiletheproject’sefficiencyhasbeenModeratelySatisfactory.
3.3.4. CountryOwnershipAttheproject’soutset,boththeUkrainianGovernmentandtheprivatesectorhaveshownstrongcommitmentandownershipoftheTMELproject.However,astheprojectprogressed,theGovernmentwasfacingmanyissues,andwhileitcommitmentmaystillbestrong,ithasnotbeenabletotakefulladvantageoftheproject’sdeliveries,orleveragedtherapidchangesintheinthemarket.Theprivatesectorhasbeenmoreproactive,andwasabletoleveragetheproject’sawarenessraisingefforts.Forexample,keycontributionsfromretailersincludedusing/displayingeducational/outreachmaterials,allowing/providingtrainingtotheiremployees,aswellasoptingtoincreasethestockandvarietyofenergyefficientlightingproducts.
Thesectorthatbenefittedmostfromtheproject’sactivitiesandprovidedthestrongestoutcomeshasbeenmunicipalandlocalgovernments.Theyenthusiasticallyparticipatedinpilots,auditsandutilizedothertechnicalassistancefromtheproject.Forexample,recommendationsforprocurementandtechnicalspecificationsfortheEElightingwereplacedintheHandbookstitled“EnergyEfficientStreetLighting,BuildingandDevelopment”and“EnergyEfficientLighting”disseminatedamongmunicipalities(seedetailsunderOutput5.2).Itresultedinadoptionby15municipalitiesofrespectiveregulationsforprocurementandtenderprocessthatprovidedthemwithturn-keyEElightingsolutions.Themunicipalandcitygovernmentsalsosupportedandadoptedthedraftdecreeonthephasingoutofinefficientlamps,helpingtoprovideforwardprogressinthisareawhenthecentralgovernmentcouldnot.
3.3.5. MainstreamingandSustainabilityThesustainabilityofprojectinterventionswasincorporatedintotheproject’sdesignformainstreamingandreplicationpotentialbasedoninternationalexperience.Theseincludedcertainprojectimplementationpractices,contributions,andoutcomesthatcanhavewide-reachinglongtermimplicationsforprivateandpublicsectorsaswellastheconsumerstocontinueswitchingtoenergy
TMELTerminalEvaluationReport–31March2017
64
efficientlighting,includingoutputssuchtheNationalRoadmap,legislationsonenergyefficiency,standardsandlabeling,wastecollectionandhandling,aswellasqualityassurance/controlmechanismsandinstitutionalframework.
Anumberofsub-contractsawardedbytheprojecttovariouspublicandprivatestakeholders,specificallytheMaxusFund,schools,testlaboratories,aswellasagreementswithretailers,haveresultedincapacitybuildingoftheseorganizationsforfuturesupportforenergyefficientlightingactivities.Demonstrations,pilots,technicalassistance,developmentofguidelines,standardsandbuildingnorms,havealsocontributedtoawidearrayofdocumentsonenergyefficientlighting.Itisexpectedthatthisinformationwillbereplicatedorbuiltuponbytheimplementingsub-contractorsorbeneficiaries,e.g.theuseofbuildingnormsorguidelinesandadoptionofdecrees.However,fortheprojecttoensurelong-termreplicabilityandsustainabilityoftheseefforts,itwillbeimportanttohaveaplantodeliverthesuiteofinformationdevelopedbytheprojecttoallpotentialindividualandorganizationalstakeholders,includingresearchers,academics,entrepreneurs,policymakers,andconsumers,etc.
Consideringthepolicysupport,positiveresponseofthelocalandmunicipalgovernments,technologicalshifts,andtheglobaltrendsinILphase-out,theEvaluationTeamconcludesthattheTMELprojectisModeratelyLikelytobesustainable.
3.3.6. ImpactsActivitiesbythisprojectwereexpectedtoeffectivelydoublethegrowthofEElightingmarketpenetration,increasingtheannualgrowthrateto25%yearonyearduringtheprojectperiod.Basedonthisincrease,theprojectwasprojectedtocontributeanetCO2reductionoveritslifetimeof4.15milliontonsfrom2011to2015fromdirectreductioninelectricityconsumption.Forthemunicipalsector,projectactivitiesaimedtoprovidemunicipalitieswithtechnicalresourcesandaccesstofinancingforlargeEElightingrenovationsinordertoachievecompliancewithCMUOrder#1337-rrby2020.TheprojectedCO2reductionwasestimatedtobe900,000tonsforthissectorbytheendoftheproject.TheprojecthascommissionedaspreadsheettooltotracktheimpactsoftheprojectgoalsusingGEF-developedmethodologies.43Basedonthetool’scalculations,theproject’sactivitieswillresultindirectgreenhousegasemissionreductionsduringtheproject’simplementationperiodoffiveyearstotaling3,308,750tonsofCO2.UsingtheGEFbottom-upmethodology,indirectemissionreductionsattributabletotheprojectareestimatedtobe9,926,249tonsofCO2.44Basedontheseestimates,theprojecthashadSignificantimpactonitsemissionreductiongoal.
43ManualforCalculatingGHGBenefitsofGEFProjects:EnergyEfficiencyandRenewableEnergyProjects.GEF200844UsingtheGEFbottom-upmethodology,indirectemissionreductionsattributabletotheprojectare9926249tonnesofCO2.Thisfigureassumesareplicationfactorof3.0.UsingtheGEFtop-downmethodology.Inaddition,indirectemissionreductionsattributabletotheprojectare37011750tonnesofCO2.ThisfigureassumesthattotaltechnologicalandeconomicpotentialforGHGemissionreductionsover10yearsis46264687tonnesofCO2,andaprojectcausalityfactorof80percent.
TMELTerminalEvaluationReport–31March2017
65
Note:theestimationofdirectemissionsreductionbytheprojectincludeddemonstrationprojectsandlampdistribution,whileindirectreductionsarefromincreasedpenetrationofEElightinginthemarket.Inthiscase,thedirectemissionsestimatesincludedprojectsinareasthatarenolongerundergovernmentcontrol,andthereforemayoverestimatetheimpacts.However,thisisbalancedbytheindirectestimates,whichusedabottom-upmethodologyandrepresentanunderestimationoftheimpacts.TheGEFtoolalsoproducedanestimatedCO2savingsfromatop-downapproachofnearly37milliontons,duetoincreasedproductpenetrationrates.Figure4.EstimatedProjectCO2Impacts
TMELTerminalEvaluationReport–31March2017
66
4. CONCLUSIONS,RECOMMENDATIONS&LESSONS
4.1. ConclusionsIngeneral,acasecanbemadethattheTMELprojectwasabletodelivermostofitsplannedactivitiesandoutputs.However,fortheproject’sfulloutcomestobeachievedinthefinalstagesofimplementation,challengesremainduetothevariousexternalfactors,andtheUkraineGovernment’swillingnessinadoptingandimplementingthekeyprojectoutputs.
Toreiterate–overthecourseofitsimplementation,theTMELprojectexperiencedsignificantchallenges,including:
• Changingnationalpoliticalandeconomicconditions,includingachangeingovernment,• Worseningsecuritysituationsinsomeareasofthecountry(armedconflictintheeasternareas),• Acollapsedinternational(andnational)carbonmarket,and• Rapidtechnologicalshiftstowardsmoreadvancedandefficientlightingproducts(and
correspondingpricereductionofLEDs).
Alongwiththeseexternalchallenges,theprojectalsoexperiencedsomeinternalchallenges.Theprojectwasmanagedbythreedifferentprojectmanagersoverthespanofsixyears,andwasadvisedbyatleasttwoInternationalConsultants/TechnicalAdvisors,neitherwasengagedfromtheoutset,norweretheyfullyutilizedbytheproject,asenvisionedbytheprojectdocument.45Thechallengesexperiencedbytheprojectrepresentedtherangeofrisksthatallmarkettransformationprojectscanface.
Despitethesechallenges,theTMELprojectwasabletoexerciseadaptivemanagement,achievedamajorityofitsoutputs,anddeliveredonanumberofareasnotcoveredbytheprojectdocument,includingclosecoordinationwithmunicipalitiesandcitiestoeffectthephaseoutofinefficientlampsatthelocallevels.However,evenwithinitialinterestandcommitmentbytheUkrainianGovernmentandtheproject’sdeliveryonkeyoutputs,notalldesiredoutcomeswereachieved.Acaseinpoint:althoughtheprojecthasdevelopeddetaileddraftlegislationssupportinganumberofimportantenergyefficientlightingtopics,aswellascultivatedaworkingrelationshipwithanUkrainianMemberofParliament,theseproposedlegislationshaveyettobeadopted.
Theprojectsufferedfrominconsistentdeliveryanddelaysinoutputsinitially,andduringtransitionperiods,perhapsduetothescaleandambitionoftheproject’sobjectives.But,theprojectwasalsoabletorespondtoanumberofrecommendationsfromtheMTR,andwasabletousetheadditionaltimefromtheextensioneffectively.Intermsofoperationalissues,theproject’sexternalimplementationarrangementsreliedonexistingorganizationalstructuresinthelightingindustry,aswellasNGOsthataretypicalinothermarkets.However,thiswasnotentirelyapplicable,asthelackofastronglightingindustryassociationandaviableindependentconsumerorganization,aswellasnon-existentmarketsurveillanceandenforcementinfrastructuremadeitdifficulttopromoteaproductqualityassuranceandqualitycontrolframework.
45FromwhattheEvaluationTeamcoulddiscern,thefirst InternationalConsultantwasonly involvedwiththepreparationofspecifictasks,suchastheRoadmap.ThesecondICwasmoreengaged,butmostlyontechnicalaspectsoftheproject,suchasComponent2.Infact,theTORoftheCTAcallsforbroadprojectengagement.
TMELTerminalEvaluationReport–31March2017
67
Internally,factorscontributingtoreduceoperationaleffectivenessincludedinconsistentadministrativesupportandchangesinkeystaff.Inaddition,thesizeoftheprojectteam,aswellascompositionandskillsetsoftheteammaynothavebeenoptimalandresultedinreducedcosteffectivenessfortheproject,aspointedoutbytheMTR(theEvaluationTeamconcurswiththosefindingsbytheMTR).Inaddition,wenotedthattheteamresponsibleforPartnershipsandRelationswithGovernmentandNationalAuthoritieshadaseparatereportingstructure.Whilethisstructurerecognizedtheimportanceandsensitivityofthisteam’stasks,itmayhaveallowedforlesscoordinationbetweentheteamleaderandprojectmanager,andlessinteractionswiththeCTAandtheprojectteamonmattersrelatedtoproducttesting,qualityassuranceframework,aswellasinternationaloutreach,amongothers.
Aspointedoutearlier,theinitiallackofconsistentprogresscouldbeduetothefactthatTMELwasthefirstfull-scale,markettransformationprojectinUkraine,withambitiousgoalsandcomplex,interlinkedoutputs,whilehavinglessactualMTimplementationexperiencetodrawfromlocally.Thislackoflocalimplementationexperiencecouldhavebeenpartiallyalleviatedwithavailableinternationalexperience,especiallyifanexperiencedinternationalCTAwasavailableandinvolvedwithallaspectsoftheprojectfromtheoutset.WithouttheCTA,theprojectconductedextensiveresearchonitsownonawidearrayoftopicsratherthanbuildingfrom,andcoordinatingwith,internationallyavailablework,asspecifiedunderOutput4.4.Becauseofthis,theprojecthadnotfullytakenadvantageincoordinationwith,orsharedinformationwiththeUNEP’sen.lightenprogramorotherlightingprogramsoperatingconcurrently,asoriginallyintended.Theconductionofthesestudiesalsodelayedneededactivities,andreducedoverallprojectcosteffectiveness.
Withrespecttoprojectexpendituresandbudgetmanagement,itisunderstoodthattheprojectexercisedadaptivemanagementandwasabletoredirectspendingfromtheoriginallyplannedbudget.AshasalreadypointedoutbytheMTR,therefocusonincreasingawarenessandmarketpenetrationhasshowngoodresults.However,thefactthatspendingonpilotsanddemonstrationswasmuchlessthanoriginallyplanned(6%actualvs.28%planned)wasasignificantshift.Italsoindicatedthattheprojectneededbetterfeedbackmechanismsforusingmarketintelligence.AspointedoutbytheMTRandconfirmedthroughtheEvaluationTeam’smeetingswithmunicipalandcitygovernments,thereremainsahighlevelofinterestforpublicdemonstrationsandpilotprojects,andadditionalspendingcouldhavebeenjustified.
Finally,regardingtheproject’sinformationalcampaigns–theywereoneoftheproject’sbrightpoints,andresultedinthecreationofanumberofhighlyvisiblemessagesforconsumersonenergyefficientlighting.However,itmustalsobenotedthattheprojectinvestedsignificantresourcesforgeneralawarenessraising,andcontinuedtodosoevenafterithasachievedsignificantgains,possiblytocompensateforlackofprogressinotheroutputareas–againsttheadviceofferedbytheMTR.Intermsofmessagelongevityandprojectsustainability,itmayhavebeenmoreeffectivetocreatesomelinkageswithotheraspectsoftheprojectoutputsanddesiredoutcomes,suchasqualitycontrolmechanismsorframework,orconnectingthecharacteristicsofenergylabelsandproductperformancewiththeneedfortestingandverification,asitsownextensiveresearchshouldhaveshown.Thisisespeciallyimportantsincethetechnologicalshifttolight-emittingdiode(LEDs)occurredmuchfasterthan
TMELTerminalEvaluationReport–31March2017
68
anticipated,resultinginincreasedavailability,penetration,andmorecompetitivepricing,butalsoissuesofquality,andhowtoproperlymeasureprojectimpactsinthisarea.
Inconclusion,theEvaluationTeamhasdeterminedthattheTMELprojectdesignhasremainedhighlyrelevanttothedevelopmentcontextofUkraineandtheprioritiesofvariousstakeholders,includingGEF,UNDP,municipalgovernments,cities,schools,laboratories,andtheprivatesector,anditscombinedoutputshavemettheGEF’sguidelinesforaModeratelySatisfactoryproject(theprojectincurredsomemoderateshortcomings,includinglackofoutputprogressoncertaincomponents,andinconsistentprojectprogress,resultingintheneedforanextension).
4.2. LessonsLearnedBasedonconsultationswithkeystakeholdersandtheconclusionsdrawnbytheEvaluationTeam,somekeylessonslearnedfromtheTMELprojectdesignandimplementationinclude:
• Interest/supportbytheappropriategovernmentagency(ies):TheprojecthasdemonstratedthatfullsupportbyRecipientCountryGovernment(GOC)andcooperationbetweenrelevantministries/departmentsarenecessarytoachievetheintendedoutcomes,especiallywherelegislativeactions,buildingnormsorproductstandardsmaybeconcerned(forexample,theneedformorethanonegovernmentministriestocooperateandcoordinateonlegislationsupport);
• Privatesectorengagement:Engagementwithprivatesectorisnecessaryforachievingmarket-relatedgoals(forexample,workingwithEpicentreonproductpromotion,includingtrainingofsalesstafftorealizeincreasedpenetrationandawarenessofefficientlightingproductsatretail);
• Supportawarenesswithavailability:Inmediumandsmallcitiesandruralareas,havingproductsavailabletoconsumersinconjunctionwithawarenesscampaignscansignificantlyincreasemarketshareofEElightingintheseareas.
• Interestsalsoremainhighamonglocalandmunicipalgovernmentsforpilotsanddemonstrations.
• Updateprojectobjectives:Projectsmaytakelongerthanexpectedtobeapproved,asareproposedlegislation,andtechnologicalorpoliticaldevelopmentsmayhappenalmostovernight.Thesesituationscancausetheprojectobjectivesoroutcomestobeoutdated,ornolongerneeded.Suchsituationsmayrequiretheprojectplanstobemodifiedtoaddressnewmarketorpoliticalrealities.
• Activeengaged,andcomprehensivePSC:AnactiveandengagedPSCrepresentingawide-ranginggroupofstakeholderscanhelptomoreappropriatelyaddressthechallengesandrisks,aswellasdeterminingwhentoreviseoradoptnewobjectivesbasedonchangingmarketorpoliticalconditions.
• Betterfeedbackmechanismsforusingmarketintelligence:Ahighlevelofinterestforpublicdemonstrationsandpilotprojectsexistedduringthecourseoftheprojectimplementation,yetthislevelofinterestwasnotreflectedintheannualplanningprocess,whichcouldhelptoincreasefocusadditionalspendingforpilotsinplaceofawarenessraisingactivities.
TMELTerminalEvaluationReport–31March2017
69
• Administrativecommunicationandcoordination:Communicationandcoordinationarrangementsareessentialtosupportprojectplanningandimplementation,especiallyunderchangingmarketandpoliticalsituations;
• Administrativesupport:Projectsstartingupmayrequirestaffingadjustmentsorotheradministrativesupport,suchasdevelopmentoftendersorsecuringshort-termconsultants.Agoodrelationshipbetweentheprojectadministrationandimplementationteamscanbevaluableinhelpingaprojecttoachieveitsinitialsuccesses;
• ComprehensiveM&E:AnM&Esystemthatfocusesonallkeyprojectaspects,includingco-financing,sub-contracts,andimpactindicatorsisessentialtoassessingaproject’sprogressandimpacts;
• Teamcompositionandskillsetsareimportantfactors:Itisimportanttomatchtheteammemberskillsetswiththeprojectrequirements.Markettransformationprograms,forexample,tendtorequireteammemberswithentrepreneurialtraits,whocanunderstandoradaptquicklytochangingmarketsituations.Technically-focusedprojects,ontheotherhand,canrequireteammemberswithdeeptechnicalknowledgeandexperience.
• Clearreportingstructure:Aclearreportingstructureshouldbeapparenttoallteammembers,andopportunitiesforcoordinationandcooperationcross-sector/objectivesshouldbeencouraged.
• Adaptivemanagementpracticesshouldbeencouraged:Projectstendtoexperiencemarketshifts,butafewmaybeaffectedbymorethanone“game-changing”factors,includingsignificantpoliticalortechnologicalchanges,orevenboth.Inthesecases,projectsareforcedtoadaptinordertoremainrelevant.Suchexamples,whereavailable,shouldbesharedandreviewedtolearnfrom,ifpossible.
• Technicalsupportbyinternationalexperts:Forprojectsthataimtotransformthemarketforproductsorservices,especiallytopioneeranapproachthatcanbeusedbyotherprojects,itisessentialtosecuretheservicesofanexperiencedinternationalexpertfromtheoutset.(TheprojecthadthisintheformoftheRTA’sinputonimplementationandcostmanagement).However,anexpertshouldhaveexperienceintechnicalissuesaswellasprevioushands-onexperienceinmarkettransformationimplementation.UNEPshouldbeabletodrawonavailableexpertswithsuccessfultrackrecordsinimplementationsupport,notjustdocumentproduction.
• Drawfrominternationalbest-practicefromtheoutset:Securingtheservicesofexperiencedinternationalexpertscanalsoprovideprojectswithaccessandunderstandingtobest-practicesinternationally.WhileprojectevaluationandotherdocumentationavailablefromUNDPandelsewherecanprovideusefulinformation,findingandaccessingthemcanbeachallengetonewprojectteamslearningnovelapproaches,andunderstandingthisinformationwithoutthehelpofaninternationalexpertcanaddanotherlayerofcomplexity.
• Marketresearch:Internationalbest-practiceinformationcanbefurthersupplementedwithresearchfocusingonthelocalmarket’sparticularcharacteristicsorculturalpreference,whichcanincreaseeffectivenessandreach.
TMELTerminalEvaluationReport–31March2017
70
4.3 RecommendationsBasedontheaboveconclusionsandthelessonslearned,theEvaluationTeamrecommendsthefollowingactions:
1. ReplicationandUp-Scaling:TheprojecthasmadesignificantcontributionstotheawarenessofenergyefficiencyandenergyefficientlightinginUkraine’sconsumer,localandmunicipalgovernments.Toensurethesustainabilityofthiscontribution,itwillbeimportantfortheprivateandpublicsectorstocontinuetosupportthereplicationoftheseactivities.Theoutputsoftheprojectinthisareaneedtobemaintainedandcontinuetobedisseminated.Theconversionofretailersandmanufacturerstoenergyefficientlighting,aswellasadoptionofdecreesbymunicipalandcitygovernmentsneedtobefurtherpromotedandadoptedbyothers.
2. DocumentationandDisseminationofResults:TheprojecthasmadesignificantcontributionstothedevelopmentoftheEElightingindustryandlightingknowledgebyundertakingconsumerresearch,developingpilots,andfacilitatingtechnologytransfer,etc.ForfutureeffortsandprojectstofullyutilizetheseproductsaswellasthelessonslearnedintheimplementationoftheTMELproject,itisimportantthattheprojectcandocumentanddisseminatesitsresults,achievements,andlessonslearnedinmarkettransformation.Inthisregard,UNDPshouldconsiderkeepingtheprojectwebsiteortransferittothemaintenanceofanotherprojectthatalsoaddressesmarkettransformation.Hostingandmaintenancerightsshouldberetained.Instead,hostingrightsfor10yearsshouldbepurchasedandtheinformationmentionedaboveaswellasanyotherresultstheprojectsshouldbepostedonthewebsitetobenefitfutureprojects/efforts.ThewebsitecouldalsobelinkedtootherkeynationalandinternationalEElightinginitiatives.
3. StakeholdersCollaboration:Toensureeffectiveplanningandimplementation,itisimportanttohaveopencommunicationlinesbetweenkeystakeholders.Toavoiddelaysinimplementationinthefuture,theUNDPandPMOneedcommunicateopenlytoaddressissuesrelatedtoimplementation,suchasunfamiliarapproachortime-sensitiveactivities.
4. TrackingCo-Financing:Duetochangingpoliticalsituationsandotherchallenges,thePMOwasnotabletofullytrackandjustifyprojectco-financing.Thissituationshouldbeaddressedinotherfutureprojects.
5. AttainmentofOutcomes:GiventhemultiplechallengesfacedbytheTMELprojectduringtheimplementationprocess–whichmaybeunlikelytobefacedbyanotherproject–wesuggestthatUNDPtaketheseconditionsintoconsideration,andconsiderthedevelopmentofamoreexhaustivelistingofrisksforUkraine,aswellasmoredetailedriskmitigationstrategiesfordesignofprojects,andconsiderwaysforactiveinvolvementbythePSCinthesecasestohelpaddressanyrequiredchangesinprojectdesignsoroutputsinamoretimelymanner.Thisapproachmayalsobedocumentedforregionsthatmayexperiencerapidchangesforotherprojectstoreferenceshouldtheyencounterthesamesituations.
TMELTerminalEvaluationReport–31March2017
71
SpecifictothedesiredlegislativeandQAframeworkoutcomes,wesuggestthattheprojectandUNDPexploreallvenuesforcooperationwiththeMinistryofRegionalDevelopment,Construction,HousingandCommunityServicessothattheseobjectivescanbesustained.GiventhattheprojecthasprovidedallofthenecessarylegislativeoutputsandNationalRoadmap,aswellassupportfortheQAframework,itwouldbebeneficialforallinvolvediftheMRDCHCScanbepersuadedtocontinuetopursuetheseimportantoutcomes.
TMELTerminalEvaluationReport–31March2017
72
AnnexesAnnexA:ListofDocumentsReviewed
AnnexB:InterviewGuides
AnnexC:DetailedTEMissionSchedules&Interviews
AnnexD:PSCMembersandMeetings
AnnexE:Letterofsubmission
AnnexF:TEReportOutline
AnnexG:ListofResearchStudies&SummaryEvaluationofMajorStudiesCommissionedbytheProject
AnnexH:LessonsLearnedfromRTA
AnnexesAnnexA:ListofDocumentsReviewed
AnnexB:InterviewGuides
AnnexC:DetailedTEMissionSchedules&Interviews
AnnexD:PSCMembersandMeetings
AnnexE:Letterofsubmission
AnnexF:TEReportOutline
AnnexG:ListofResearchStudies&SummaryEvaluationofMajorStudiesCommissionedbytheProject
AnnexH:LessonsLearnedfromRegionalTechnicalAdvisor
TMELTerminalEvaluationReport–31March2017
73
AnnexA:ListofDocumentsReviewedGeneraldocumentation
§ UNDPProgrammeandOperationsPoliciesandProcedures§ UNDPHandbookforMonitoringandEvaluatingforResults§ GEFMonitoringandEvaluationPolicy§ GEFfocalareastrategicprogramobjectives
Projectdocumentation
§ GEFapprovedprojectdocumentandRequestforCEOEndorsement§ ProjectInceptionReport§ ProjectMid-TermEvaluationReport§ Annualworkplans(PIRs)§ Financialauditreports§ ProjectSteeringCommitteemeetingminutes§ Updatedrisklog
Otherrelevantdocumentation
§ Reportsonmarketmonitoringoflightingproducts§ ResultsofOutreachCampaigns§ MarketSurveys§ Textbooksforspecialists§ Promotionmaterials§ Pressarticles§ Website
TMELTerminalEvaluationReport–31March2017
74
AnnexB:InterviewGuidesStaff,KeyConsultants&MainStakeholders
Date:
Name(s):
Position(s)/Role(s)inProject:
ContactInfo:
QUESTIONS
I. PROJECTDESIGN&MANAGEMENT
1. Wereyouinvolvedorawareoftheprojectdesignprocess?Canyouprovidesomedetails?(e.g.whowasonthedesignteam,whichstakeholderswereconsulted?AretherelinkagestosimilarUNDPorinternationalprojects,etc?
2. Hastheprojectdesignandlogframebeenrelevantacrosstheprojectduration?Orhavetherebeenchangestotheoriginalprojectdesign/LogFrame?
3. TheprojectexperiencedsignificantchallengesingoalssetbytheprodocbeforetheMTRandalsoaftertheMTR.Doesthismeanthattheprojectdesigndocumentunderestimatedthechallengesoftheproject?
4. Duetothechallenges,could/shouldgoalsbechangedorshouldotheractivitiesaddedtotheproject?
II. EXTENSION
1. Whatfactorsledtotheextensionofprojectclosurefrom2015to2016?2. Whatmeasuresweretakenbykeystakeholderstoavoidanyfurtherdelays?3. Howbeneficialhasitbeentoextendtheprojectto2016?
III. MONITORING&EVALUATION
1. DoestheprojecthaveanM&EFramework/Plan?Ifyes,whatarethekeyelementsforyourrespectiveareofresponsibilities/Objectives?
2. Whatarethetoolsusedtotrackprogressonprojectgoal,outcome,andoutputlevels?3. Howhelpful/effectivehastheM&EplanbeeninhelpingtheProjecttomeetits
Goals/Objectives?4. WastheProDocorthelogframeusedforM&E,andReporting?5. Wastheriskassessmentandmanagementmatrixbeingupdatedannually?Ifyes,howandby
whom?Copyofallreviewedmatrices
IV. PROGRESS,OUTPUTS&ACTIVITIES
1. Areanythereanyoutstandingprojectoutputsoractivitiesatthistime?Ifyes,whatarethereasons?
2. Whenwilltheseactivitiescloseout?3. TowhatextenthavetherecommendationsoftheMTRbeenimplemented?Ifsomewerenot
implemented,whatwasthereason?
TMELTerminalEvaluationReport–31March2017
75
V. SUBCONTRACTS&CONSULTANTS
1. Howmanysubcontractsandconsultancieshavebeenissuedundereachprojectcomponent(year,topic,andbudget)?WeretheybasedonProDoc?MTR?
2. Haveallsubcontractsbeencompleted?Ifno,whichonesareoutstanding?Whenaretheyexpectedtocomplete?Whatarethereasonsfordelay?
3. Whatwasthegeneralprocessofselectingthesub-contractorsandconsultants?4. Werethereproblemswithmanagingthecontractors?5. Whichofthecontractshavecontributedmostpositivelytotheproject’soutcomes/goals?What
werethecontributions?6. Whichofthecontractshadtheleastcontributionorwereineffective?Why?
VI. TRAINING,CAPAPCITYBUILDING,COMMUNICATIONS&OUTREACH
1. Listofvarioustrainingandoutreachactivities(includingbudget,andpeoplereached)undereachoftheprojectcomponents
2. Wasthereaprocesstoselectparticipants?3. Isthereaprocesstoassesstheimpactsofthetraining/outreach?Whathavebeenthe
outcomes?4. Whatweresomekeychallengesfacedbythetraining/outreachprograms?Howwerethey
addressed?5. Whatarethekeyelementsofthewebsite?6. Howoftenisthesiteupdated?7. Whoaccessesthewebsite?Isthereatracking/countingmechanism?8. Isthewebsitepromotedamongprojectstakeholders/participants?9. Isthereaprojectnewsletter?Ifyes,howoftenandtowhomisitdistributed?Istherefeedback
fromthosereceivingnewsletter?10. Havetheintellectualproductsoftheproject(e.g.studies,research,etc.)beensaved(indigital
format)sothattheycanbeaccessibleandforotherprojectstobuildontheproject’slessons?
VII. PERSONNEL&STAFFING
1. HowisthePMO/PMUorganized?Isthiscleartoall?2. Hastheprojectfacedanystaffing/resourcechallenges?Whatarethey?3. Howhavethesestaffing/resourcesbeenresolved?
VIII. PARTNERSHIPS
1. Whoarethekeystakeholdersunderthedifferentoutcomesandwhatistheirrole?Whenweretheyaddedtotheproject?
2. Whichstakeholdersundereachcomponenthavemadethemostproductivecontributiontowardstheprojectgoal?
3. Whichstakeholdershavemadetheleastproductive/leastactivecontribution?4. WhatistheliaisonmechanismbetweentheProjectandotherinstitutionalstakeholders?5. Whatchallengeshavebeenfacedwithmanagingthepartnerships?6. HowdoestheProjectinteractwiththevariousstakeholdersandpartnerstoensure
communicationandlinkagesbetweentheirrespectiveactivities?
IX. UNDP/GEF
TMELTerminalEvaluationReport–31March2017
76
1. WhatsupportwasprovidedbyUNDPUkraine?2. Werekeystakeholderssuccessfulincollaboratingwitheachother?3. DidthePSCmetregularly?4. HoweffectivehasbeenthePSCbeenperformingitsdutiesofoversight(e.g.reviewofAnnual
WorkPlans,AnnualProgressReports)?
X. FINANCE
1. Havetherebeenanydelaysorproblemsfacedwiththeproject’sfinancialdisbursementsfromthedifferentstakeholders?
2. Haveregularprojectfinancialauditsbeenundertaken?Weretheseauditssatisfactory?
XI. EFFECTIVENESS
1. Towhatextenthastheprojectachieveditsgoalsandobjectives?2. WhatkeyGovernmentpolicies/strategiesweretheprojectabletocontributeto?How?3. Whatfactorshavebeencriticalforthesuccessoftheprojecttoachieveitsgoalsandobjectives?4. Whathavebeentheproject’skeysuccesses?5. Whathavebeentheproject’skeychallenges?
XII. IMPACTS
1. Whatmeasureshavebeenundertakentoassesstheproject’simpact?2. Whathavebeentheresultsofthesemeasures?3. Whichoftheprojectactivities/componentshavehadthehighestimpact?Why?4. Whichoftheprojectactivities/componentshavehadtheleastimpact?Why?5. HastheGHGreductioncalculationmethodologybeenfinalizedbasedonempiricalresults
gatheredduringtheprojectimplementationandbeevaluatedintheTerminalEvaluation?ThesameforreductioningrowthrateofCO2emissions
XIII. SUSTAINABILITY
1. Whathavebeenthekeymeasuresofsustainability/replicabilityembeddedintheprojectdesignanddelivery?
2. Whichelements/resultsoftheprojectareparticularlysustainable?Why?3. Whichelements/resultsoftheprojectareleastsustainable?Why?4. Whatpotentialchallengescantheproject’ssustainabilityface?5. Whatistheproject’sexitstrategy?Hasthisbeendocumented?
XIV. LESSONS&RECOMMENDATIONS
1. Whathavebeensomeoftheproject’skeylessons?2. Whatareyourrecommendationsforthesustainabilityofprojectinterventions?3. WhatareyourecommendationsfordesignofsimilarfutureprojectsforUkraine?
TMELTerminalEvaluationReport–31March2017
77
SUBCONTRACTORS/OTHERSKEYPARTNERS
Date:
NameofInterviewee: OrganizationName:
Title: ContactInfo:
BACKGROUND1. DateanddurationofyourinvolvementintheProject?2. Whatparticularrole/mandatedoesyourorganizationplaywiththeproject?3. Inyouropinion,whathavebeenthekeysuccessesoftheproject?Why?4. Inyouropinion,whathavebeenthekeychallengesoftheproject?Why?Couldtheybe
mitigated?STAKEHOLDERCOLLABORATION1. Whichprojectstakeholdersdoyoudealwithdirectly?2. Whatisthemechanismforcollaborationwiththeproject?3. Inyouropinion,whichstakeholdershaveplayedakeyroleintheproject?4. Whatweretheopportunities/positiveoutcomesinworkingwithstakeholders?5. Whatweresomeofthechallengesinregardtocollaborationamongstakeholders?STEERINGCOMMITTEE1. AreyouawareofPSC?2. Ifyes,whatisyourimpressionofthePSC’seffectivenessinperformingitsdutiesofoversight
(e.g.reviewofAnnualWorkPlans,AnnualProgressReports),andguidanceRELEVANCE1. Whatisthekeyrolethatyourorganizationhasplayedintheproject’sactivitiesanditssuccess?2. Howdoestheprojectfitintothestrategicprioritiesofyourorganization?3. Howcan/willtheproject’ssuccesses/activitiesfeedintofuturestrategyoractivitiesofyour
organization?4. WhatotherEEorlightingprogramshasyouragencybeeninvolvedin?5. Ifyes,howwouldyouratethecomparativecontributionsandchallengesofTMELwiththese
otherprograms?CAPACITYBUILDING&SUPPORT1. Howhavetheprojectactivitiescontributedtobuildingthecapacityofyourorganization?(e.g.
trainingofpersonnel,technologytransfer,policysupport,marketmapping,etc)2. Areyousatisfiedwiththelevelofadministrativeandtechnicalsupportprovidedbytheproject
toyourorganizationortootherstakeholders?Pleasegivedetails3. Werethereproblemsfacedbyyourorganizationinreceivingsupportfromtheproject?How
weretheseproblemsresolved?REPLICATION&UP-SCALING1. Arethereanymechanismsputinplacebytheprojectfortheup-scalingoftheactivities
implementedbytheproject?
TMELTerminalEvaluationReport–31March2017
78
2. Theprojecthasimplementedanumberofsuccessfulpilots.Havethesepilotsbeenreplicatedbyotherstakeholders?
3. Whatarethepotentialopportunitiesforfurtheractivities?4. Whatarethepotentialchallenges?SUSTAINABILITY1. Willtherebeopportunityfortheprojectstakeholdersfromthebusinessand/orpublicsectorto
continuecollaborationafterprojectend?2. Cantheprojectinstitutionalizesuchcollaborationplatformsbeforeitcloses?3. Aretherekeyprojectactivitiessustainableinthemediumandlongterm?4. Whatcanbedonetoincreasethechancesofsustainabilityofsomeoftheseactivities?LESSONSLEARNED&RECOMMENDATIONS1. Whatarekeylessonslearnedfromtheprojectfromyourperspective?2. Basedontheprojectimplementationexperience,whatareyoursuggestionsforimprovements
infutureprojects?
TMELTerminalEvaluationReport–31March2017
79
AnnexC:DetailedTEMissionSchedules&StakeholdersInterviewedEvaluationMissionProgramme
GEFfundedUNDPImplementedProject:“TransformingtheMarketsforEfficientLighting”
January23–February03,2017InternationalEvaluator:Mr.MyTonNationalEvaluator:Mr.PetroPavlychenko
Date Day Time Activity Participants Venue
1/23/17Monday
13:15 Arrival 16:00 MeetingwithProjectteam(optional) MT,PP ProjectOffice
1/24/17 Tuesday
9:30 MeetingwithVadimPozharskiy(exNPD,exGEFFocalpoint)
MT,PP,AB Lipskaya10
11:00MeetingwithLesyaKarnaukh(HeadofDepartmentofMinistryofEcologyandNaturalResources)
MT,PP,AB MENR
12:00 MeetingwithMarketingcompanyUSP-projectsubcontractor
MT,PP,OB ProjectOffice
15:00 MaxximumMediaCommunicationsGroup-projectsubcontractor
MT,PP,OBDmitrievskaya48-D
1/25/17 Wednesday allday
TriptoMykolayiv(meetingswith:OleksandrOmelchuk,DeputycityMayor,CityCouncil;YuriyKharitonov,DeanoftheNationalUniversityofShipbuilding,ChairoftheEnergyandinfrastructure;AnatoliyShiyaniuk,DirectoroftheMunicipalServiceCompany“InstituteofSocial-EconomicDevelopmentoftheCity”)
MT,PP,AB,IC
Mykolayiv,tbd
1/26/17 Thursday allday
TriptoOdessaregion(meetingswith:OblastAuthorities,visitingcitiesofdemoprojects);MeetingwithSergeiLeivikov,Independentconsultant,formerDirectoroftheMunicipalServiceCompany”EnargysavingandEnergyefficiency”;MeetingwithVolodymyrLevytskyi,DirectoroftheDepartmentofInfrastructure,HousingandUtilitiesDevelopmentofOdessaRegionalStateAdministration
MT,PP,AB,IC Odessa,tbd
1/27/17 Friday
10:00MeetingwithRomanRadchenko(representativeofMinistryofRegionalDevelopment)
MT,PP ProjectOffice
11:30 MeetingwithOlegNedava,MemberofParliament,DeputyHeadofCommittee MT,PP MPsOffice
12:30 MeetingwithOleksandrSeverin(Projectstaff)
MT,PP MPsOffice
TMELTerminalEvaluationReport–31March2017
80
15:00
MeetingwithAndriiRybalochka(HeadofSemiconductorLightingTestingCenter,NationalAcademyofSciencesofUkraine,LighttestinglaboratoryinKyiv)
MT,PP LaboratoryatNauki44
1/30/17 Monday
10:00 MeetingwithTatianaTymochkoHeadofAll-UkrainianEnvironmentalLeague(NGO)
MT,PP ProjectOffice
11:30MeetingwithSvetlanaNigorodova(GEF-UNDPSmallgrantsprogram)-Projectpartner
MT,PP UNDPoffice
16:00 MeetingwithEconica(Recyclingcompany)-Projectsubcontractor/partner
MT,PP ProjectOffice
1/31/17 Tuesday allday TriptoSumy(meetingswithMayor,CityCouncil)
MT,PP,IC Sumy,tbd
2/1/17 Wednesday allday TriptoPoltava(meetingwithNataliaSmirnova,HeadofLighttestingLaboratory)
MT,PP Poltava,tbd
2/2/17 Thursday allday Reportdrafting 2/2/17 Thursday 14:00 UNDPUkraine:Briefing 2/3/17 Friday 5:35 departureday
TMELTerminalEvaluationReport–31March2017
81
AnnexD:ListofProjectPSCMembers&MeetingSchedules
PSCMembers(AsofMarch,2017)
1. Mr.JanthomasHiemstra,CountryDirector,UNDPinUkraine2. Mr.SergeiVolkov,SeniorProgrammeManager,UNDPinUkraine3. Ms.AllaTynkevych,ProgrammeAssociate4. Ms.AnnaVronska,MinisterofEcologyandNaturalResourcesa.i.5. Mr.OlegNedava,People’sDeputy6. Mr.AndriyBuryakovskyy,ProjectManager,TransformingtheMarkettowardstheEfficient
LightingProject
PSCMeetingsHeld• 7December2012,• 7February2014,• 26February2016
TMELTerminalEvaluationReport–31March2017
82
AnnexE:LettertoMinistryofJusticeforRegistrationofProjectOutput
TMELTerminalEvaluationReport–31March2017
83
AnnexF:TEReportOutlineOpeningpage:
• TitleofUNDPsupportedGEFfinancedproject• UNDPandGEFprojectID#s.• Evaluationtimeframeanddateofevaluationreport• Regionandcountriesincludedintheproject• GEFOperationalProgram/StrategicProgram• ImplementingPartnerandotherprojectpartners• Evaluationteammembers
AcknowledgementsExecutiveSummary
• ProjectSummaryTable• ProjectDescription(brief)• EvaluationRatingTable• Summaryofconclusions,recommendationsandlessons
AcronymsandAbbreviations1.Introduction
• Purposeoftheevaluation• Scope&Methodology• Structureoftheevaluationreport
2.Projectdescriptionanddevelopmentcontext
• Projectstartandduration• Problemsthattheprojectsoughttoaddress• Immediateanddevelopmentobjectivesoftheproject• BaselineIndicatorsestablished• Mainstakeholders• ExpectedResults
3.Findings
• (Inadditiontoadescriptiveassessment,allcriteriamarkedwith(*)mustberated)3.1ProjectDesign/Formulation
• AnalysisofLFA/ResultsFramework(Projectlogic/strategy;Indicators)• AssumptionsandRisks• Lessonsfromotherrelevantprojects(e.g.,samefocalarea)incorporatedintoprojectdesign• Plannedstakeholderparticipation• Replicationapproach• UNDPcomparativeadvantage• Linkagesbetweenprojectandotherinterventionswithinthesector• Managementarrangements
TMELTerminalEvaluationReport–31March2017
84
3.2ProjectImplementation• Adaptivemanagement(changestotheprojectdesignandprojectoutputsduring
implementation)• Partnershiparrangements(withrelevantstakeholdersinvolvedinthecountry/region)• FeedbackfromM&Eactivitiesusedforadaptivemanagement• ProjectFinance:• Monitoringandevaluation:designatentryandimplementation(*)• UNDPandImplementingPartnerimplementation/execution(*)coordination,andoperational
issues3.3ProjectResults
• Overallresults(attainmentofobjectives)(*)• Relevance(*)• Effectiveness&Efficiency(*)• Countryownership• Mainstreaming• Sustainability(*)• Impact
4.Conclusions,Recommendations&Lessons
• Correctiveactionsforthedesign,implementation,monitoringandevaluationoftheproject• Actionstofollowuporreinforceinitialbenefitsfromtheproject• Proposalsforfuturedirectionsunderliningmainobjectives• Bestandworstpracticesinaddressingissuesrelatingtorelevance,performanceandsuccess
Annexes
TMELTerminalEvaluationReport–31March2017
85
AnnexG(Part1):CommissionedResearchActivitiesbytheTMELProjectNr. Title Purpose Output# Date Completedby Cost(USD)
1 StrategyforMercurycontainedwastes
handling//originalinUkrainian:
«Стратегіяповодженнязвідходами,що
містятьртуть»
[https://drive.google.com/file/d/0BzlTNjcOQ
wWuVnZ2RkplZWFrQWo5Y2V0MW00aVF2Z
nMtOTVj
/view]
Purpose-Analysisofthelegalframeworkandpracticeof
handlingthemercurycontainedwastesinEU,USAandRussian
Federation.Proposalstothenationalstrategy.
1.Forimplementationofoutput1.4.
2.ResultswereusedfordevelopmentofNationalStrategyon
HazardousWasteHandling
3.SubmittedtoMinEnvatJun2015
4.Strategynowisinpublichearingsstageandcanbereached
byMinEnvsitehttp://menr.gov.ua///Component#1
1.4 Nov,
2014
TOV"MittalServis"
–Strategyon
hazardouswaste
handling
$47,147
2 DraftLawOnthehandlingofusedmercury
containinglightingequipment(fluorescent
lamps)//originalinUkrainian:
«Проповодженнязвідпрацьованими
ртутьвміщуючимиосвітлювальними
приладами(люмінесцентнимилампами)»
[https://drive.google.com/file/d/0B06VW9Er
Mx4EakliSlJuaHhEeVU/view]
Purpose–
1.Forimplementationofoutput1.4.
2.DevelopedforGovernmentandregisteredinVR(№3374),it
haspassedallrespectiveVRcommitteesandisincludedonlists
forthefirstreading
http://w1.c1.rada.gov.ua/pls/zweb2/webproc4_1?pf3511=5689
3s//Component#1
1.4 Oct,
2014VygovskaGanna
Pavlivna$4,890
3 DraftLawOnamendinglegislativeacts//
OriginalinUkrainian«ПроектЗаконупро
внесеннязміндодеякихзаконодавчих
актівУкраїни(щодополіпшення
енергоефективностівосвітленні)»
[http://w1.c1.rada.gov.ua/pls/zweb2/webpr
oc4_1?pf3511=56712]
Purpose–
1.Forimplementationofoutput1.3
2.DevelopedforGovernmentandregisteredinVR(№3345),it
haspassedallrespectiveVRcommitteesandisincludedonlists
forthefirstreadingВРУ
http://w1.c1.rada.gov.ua/pls/zweb2/webproc4_1?pf3511=5671
2//Component#1
1.3 Dec,
2014
LEBEDEVAAlina
Volodymyrivna$4,920
TMELTerminalEvaluationReport–31March2017
86
4 StudyofexistingLabellingschemesof
lightingsources,theirimplementationand
outcomes,proposingsuitableforUkraine
//originalinRussian«Отчетисследования
национальныхимеждународныхнорм,
стандартов,способствующихвнедрению
энергоэффективныхтехнологийв
освещении»
[https://drive.google.com/file/d/0BzlTNjcOQ
wWuRjJQcmNEc2d2Z2VkaUVFZXNULTEyeF9
2aUhV/view]
Purpose-
1.Forimplementationofoutput1.4.
2.ThestudycontainstherecommendationonImprovementof
TechnicalregulationonEnergyLabelling.
3.SubmittedtoMinEnv(letter№2645dt15.01.2015)4.Adopted
byCMUresolution#34027.05.2015)/Component#1
1.4 Oct,
2013
LegalfirmLexPlus $23,870
5 ReportonReviewofthenationaland
internationallegislationonutilizationand
recyclingoffluorescentlamps//originalin
Ukrainian«ЗвітДослідження
національноготаміжнародного
законодавстващодоутилізаціїта
переробкифлуоресцентнихламп»
[https://drive.google.com/file/d/0BzlTNjcOQ
wWucXhadXFhNmttVW1RejB3TFRaOEFuWG
9XRG93/view]
Purpose–
1.Forimplementationofoutput1.4.
2.ThematerialswasusedtodesignDraftLawonCFLdisposal
3.DraftLawisinVR//Component#1
1.4 Nov,
2012
LegalfirmLexPlus $20,619
6 ReportonReviewofthenationaland
internationalnorms,standardsand
legislationonpromotionenergy-efficient
lightningtechnologies//originalinUkrainian
«ЗвітДослідженнянаціональнихта
міжнароднихнорм,стандартіві
законодавства,щосприяють
енергоефективнимтехнологіям
освітлення»
[https://drive.google.com/file/d/0BzlTNjcOQ
wWub25SNmZZczBsR1ZtaGlZX21YVjVOTlgxS
EpF/view]
Purpose–
1.Forimplementationofoutput1.1
2.StudywasusedtoestablishabaselineinthesphereofEE
legislationandtosupportthedesignofRoad-Map//Component
#1
1.1 Dec,
2012
LegalfirmLexPlus $8,345
TMELTerminalEvaluationReport–31March2017
87
7 IConRoadMap–[
https://drive.google.com/file/d/0B0ENyYqfz
4jeSkZoUjdTY0hIM0E/view?usp=sharing]
Purpose
1.Forimplementationofoutput1.1.
2.Road-mapdevelopedfortheGovernmenttoincludein
Ukrainianstrategicdocuments.3.SubmittedtoMinEnv(letter
№2645dt30.06.2015)//Component#1
1.1 Dec,
2014
LightingBusiness
Consulting,Ltd.
$24,800
8 IConEULawanalysis–originalinUkrainian
“АналіззаконодавстваЄвропейського
Союзутаміжнародногодосвідуйого
імплементаціївчастині,щостосується
поетапноївідмовивідламп
розжарюванняівпровадження
енергоефективногоосвітленнявУкраїні”
[https://drive.google.com/drive/folders/0B0
ENyYqfz4jeZ1NHR0RpVTg4ZzQ]
Purpose
1.Forimplementationofoutput1.3.(createalawfortotal
phaseoutofIBs)
2.Thestudycontainsthebestpracticesandrecommendations
forUkraineonbulbphase-out.
3.ResultssubmittedtoMPat12.12.2016//Component#1
1.3 Nov,
2016
SamborskaYuliia
Leonidivna
$6,800
9 IConBudgetimpactofILCphase-out-
originalinUkrainian“Аналізекономічної
ефективності,отриманоївідпоетапної
відмовивідлампрозжарювання,
включаючивпливнадержавнийбюджет
України
“[https://drive.google.com/drive/folders/0B
0ENyYqfz4jeZ1NHR0RpVTg4ZzQ]
Purpose
1.Forimplementationofoutput(createalawfortotalphase
outofIBs)1.3
2.StudycontainstheassessmentofbudgetimpactoftotalIB
phase-out
3.ResultssubmittedtoMPat12.12.2016//Component#1
1.3 Nov,
2016
DanchukAnna $8,600
10 IConFinancingforLighting-originalin
Ukrainian“Аналізефективностірізних
формфінансуваннядлякомплексної
модернізаціїосвітлення”
[https://drive.google.com/drive/folders/0B0
ENyYqfz4jeZ1NHR0RpVTg4ZzQ]
Purpose
1.Forimplementationofoutput1.3.(createalawfortotal
phaseoutofIBs)
2.Thestudycontainstheanalysisofdifferentfinancialtoolsfor
EELandrecommendationforUkraine,
3.ResultssubmittedtoMPat12.12.2016//Component#1
1.3 Nov,
2016
DanchukIgor $13,000
11 Electricitysupplyanalysis-originalin
Ukrainian“Разработкановыхтехнических
нормпокачествуэлектроснабжения”
[https://drive.google.com/file/d/0B0ENyYqfz
4jeb2RIdWQzM2UxUW8/view?usp=sharing]
Purpose
1.Forimplementationofoutput1.2.
2.ThestudyshowedthatoneofthemajorissuewithEELislack
1.2 Aug,
2014
NT-Project,Ltd $175,324
TMELTerminalEvaluationReport–31March2017
88
ofqualityforelectricitysuppliedwhichcreatesabadpromotion
ofEELtechnologies.
3.Basedonthisstudythestudyoflegislationonprotectionof
consumerrightsweremade//Component#1&2
12 Streetlightinganalysis-originalinUkrainian
“Oцінкаякостіенергоефективних
вуличнихлампі
світильниківдоступнихнаринкуукраїни”
[https://drive.google.com/file/d/0B0ENyYqfz
4jedzU2Q1l6YTIwWHM/view?usp=sharing]
Purpose
1.Forimplementationofoutput2.2
2.Thestudyshowedthepresenceofsub-parqualityproducts
onUkrainianmarket
3.itsresultswerepresentedin2015oninternational
conference"ChangetheBulb-ChangetheWorld"//Component
#2
2.2 Jul,
2015
SGSUkraine $526,935
13 ReviewonDevelopmentoftheStrategieson
implementationofeconomicallyviable
energy-savinglightningtechnologiesin
Ukraine//originalinUkrainian–
Дослідження«Розробкастратегій
впровадженняекономічнодоцільних
енергозберігаючихтехнологійосвітлення
вУкраїні»
[https://drive.google.com/file/d/0BzlTNjcOQ
wWuYndYUHpnekFtc2c/view]
Purpose
1.Forimplementationofoutcome4
2.Thisbaselinestudyshowedthesectorsinwhichrespective
technologieshavethebestpotentialtodecreaseofelectricity
consumption
3.ThiswasusedduringPRcampaignsandpilots//Component
#4
4.2 Oct,
2014IK"SferaPlus" $9,577
14 Consumermarketanalysis-originalin
Ukrainian“ЗВІТздослідженняспоживачів
ринкупродукціїенергоефективного
освітленняУкраїни”
[http://lampochki.org.ua/wp-
content/uploads/2015/11/Report-on-
consumer-market-research-2012.pdf]
Purpose
1.Forimplementationofoutput4.3
2.ThestudyshowedthebaselineofEELconsumersmarket
3.itwasusedtoformulateToRfor1stPromotional
Campaign//Component#4
4.3 Apr,
2012
IK"SferaPlus" $15,992
15 Producermarketanalysis-originalin
Ukrainian“Проведеннядослідження
товарногоринкуенергоефективного
освітленняУкраїнитафакторіввпливуна
йогорозвиток”
Purpose
1.Forimplementationofoutput4.3
2.ThestudyshowedthebaselineofEELproducersmarket3.it
Nov,
2012
KvitenV.S.Ltd. $7,722
TMELTerminalEvaluationReport–31March2017
89
[http://lampochki.org.ua/wp-
content/uploads/2015/11/Report-on-
product-market-research-2012.pdf]
wasusedtoformulateToRfor1stPromotional
Campaign//Component#4
16 AnalysisofimpactofAwarenessCampaign-
originalinUkrainian“Проведення
дослідженнявпливупроведеної
всеукраїнськоїінформаційноїкампаніїз
енергоефективногоосвітленняна
інформаційнийрівеньнаселенняукраїни
тайоговідношеннядоенергозберігаючих
ламп”[http://lampochki.org.ua/wp-
content/uploads/2015/11/2013-Report-on-
assessment-efficiency-of-a-Campaign1.docx]
Purpose
1.Forimplementationofoutput4.3
2.Thestudyshowedtheresultsof1stawarenessCampaign3.It
wasusedtoformulateToRforsecondCampaign//Component
#4
4.3 Dec,
2013
KvitenV.S.Ltd. $11,854
17 StudyonStandardsinthesphereoflighting-
originalinUkrainian“Разработка
санитарныхнормиправилосвещения
жилыхиобщественныхзданий»”
[https://drive.google.com/file/d/0B0ENyYqfz
4jedEVlX2t5X1BDR28/view?usp=sharing]
Purpose
1.Forimplementationofoutput1.3.
2.Thestudycontainssomerecommendationtosupplement
implementationofDBN"Artificialandnaturallighting"
3.SubmittedtoMinEnvatJun2015//Component#1
1.3 Dec
2015
TOV"MittalServis" $122,366
18 ICondesignofToRforRoad-mapcreation Purpose–
1.Forimplementationofoutput1.1.
2.TheToRforIConRoad-mapdesignisdraftedandtenderwas
conducted//Component#1
1.1 Nov
2013
RachukOleg
Oleksandrovych
$4,100
19 SupportofDraftLawonPhase-outin
Parliament
Purpose–
1.Forimplementationofoutput1.3.
2.AnexperthasbeenhiredtosupporttheDraftLaw3345inVR
2.Theindependentassessmentsofdraftlawwasacquired:1.
FromNAS2.GovernmentalofficeonEurointegration.
3.CommitteofVRonEurointegration//Component#1
1.3 Feb
2017
SydorenkoSvitlana
Victorivna
$4,990
TMELTerminalEvaluationReport–31March2017
90
20Studyoflegislationonprotectionof
consumerrightsatelectricitymarket-
originalinUkrainian“Остаточнийзвітпро
дослідженнящодоукраїнськогота
міжнародногозаконодавствазякості
електроенергії,щопостачається”
[https://drive.google.com/file/d/0B0ENyYqfz
4jebVFORWpOTFhCbVk/view?usp=sharing]
Purpose
1.Forimplementationofoutput1.2.
2.Recommendationsonlegislationimprovementwasacquired
andsubmittedtoMinEnv4.TheVRhasapprovedinfirstreading
theLaw"OnElectricitysupply(Http://w1.c1.rada//
Component#1.gov.ua/pls/zweb2/webproc4_2?pf3516=4493&sk
l=9).//Component#1
1.2 May
2015
UPF"Gryshchenko
taPartnery"
$12,040
21 PublishingofEUEElegislationCompendium Purpose
1.Forimplementationofoutputs1.2,1.3,1.4.
2.CompendiumonEUlegislationinEEsubmittedto
governmentalandlocalauthoritiesandscience//Component#1
1.2,1.3,
1.4
Oct
2016
Vydavnych.dim
TsentrUchbovoi
literatury
$4,444
22 TranslationofEUDirectives Purpose
1.Forimplementationofoutput1.3.
2.ThetranslationofEUdirectivesforGovernmentsconducted
andisincludedinCompendium//Component#1
1.3 Dec
2015
FOPTimachov
SergiyMykhailovych
$777
23 ProofreadingoftranslationofEUDirectives Purpose
1.TechnicalverificationofEUDirectivestranslationbefore
publishing.
2.Toconductdefinitionverificationaftertranslation//
Component#1
1.3 Feb
2016
PavliukSviatoslav
Kostiantynovych
$370
24 ICfordevelopmentofTechnicalregulation
towardutilizationofMCLSinUkraine
Purpose
1.Forimplementationofoutput1.4.
2.DesignedDraftTechnicalRegulationandsubmittedtoMinEnv
(letter№2645,30.06.15)//Component#1
1.4 Oct
2014
Kozachenko
TetyanaPetrivna
$4,770
25 Designof1steditionofDBN"Artificialand
Natural"lighting-originalinUkrainian“пр
ДБНВ.2.5-28:2016Природнеіштучне
Purpose
1.Forimplementationofoutput1.3.
1.3 Dec
2014
TOV
KiyvPromElektroPro
ekt
$4,259
TMELTerminalEvaluationReport–31March2017
91
освітлення” 2.1steditionofDBN"Artificialandnaturallighting"isdesigned
3.SubmittedtoMinRegionforassessmentandapproval(letter
fromPromElektroKomplekt06.12.16)//Component#1
26 Designof2ndeditionofDBN"Artificialand
Natural"lighting-originalinUkrainian“пр
ДБНВ.2.5-28:2016Природнеіштучне
освітлення”
[http://dbn.at.ua/load/normativy/dbn/dbn_
v_2_5_28_2015/1-1-0-1188]
Purpose
1.Forimplementationofoutput1.3.
2.2ndeditionofDBN"Artificialandnaturallighting"isdesigned
3.SubmittedtoMinRegionforassessmentandapproval(letter
fromPromElektroKomplekt06.12.16)//Component#1
1.3 Aug
2015
TOV
KiyvPromElektroPro
ekt
$5,000
27 ProjectCTA Purpose
CTAsharedwithProjecthisrecommendationsonfurther
implementation//Component#1,2,3,4,5,6
Mar
2017
GelamiPTYLTDATF
SJCTRUST
$60,000
28 TechnicalassessmentoftheBulbEater
device
Purpose
1.Forimplementationofoutput2.4
2.TheBulbEaterwasinterestingdeviceforsafedisposalofCFLs
3.Howeverassessmentshowedthatithasn’tcapabilitiesin
demercurization//Component#2
2.4 Jul
2012
SpecialneByuroz
Eksperementalnum
Vyrob
$1,296
29 AssessmentofqualityofhouseholdEE
lightingsource-originalinUkrainian
“Порівняльнетестуванняпобутових
енергоефективнихджерелосвітлення”
[https://drive.google.com/file/d/0B0ENyYqfz
4jeam1MU0pUZGxFelk/view?usp=sharing]
Purpose
1.Forimplementationofoutput2.2
2.Thestudyshowedthepresenceofsub-parqualityproducts
onUkrainianmarket
3.Itsfindinsareavailbleonwell-knownconsumerrights
organizationTEST//Component#2
2.2 Dec
2014
Fundatsiya
RozvytkuEko.i
Energ.Rynkiv
$43,942
30 Inter-laboratorycomparisontest Purpose
1.Forimplementationofoutput2.2
2.ThisassessmentwillallowUkrainianlabstobeinternationally
2.2 Mar
2017
NationalLighting
TestCentre
$76,189
TMELTerminalEvaluationReport–31March2017
92
recognizedatobuilduptheircapacitiesinlamptesting//
Component#2
31 Organizingofmercurycontainingbulbs
disposalpointsincitiesofKyiv,Rive,Lviv,
Cherkasi,Vinnitsa-originalinUkrainian
“Організаціяпунктівзборувіднаселення
відпрацьованихенергоефективних
освітлювальнихпродуктів,щомістять
ртуть(такихякКЛЛ,ТЛЛ,тощо),зметою
подальшоговідпрацюваннямеханізмуїх
утилізаціїтапереробкиумістах:Київ,
Рівне,Львів,Черкаси,Вінниця”
[https://drive.google.com/file/d/0B0ENyYqfz
4jeR0ZhV3BCYmxsMTg/view?usp=sharing]
Purpose
1.Forimplementationofoutput2.4and4.1
2.ThegranteeorganisedcollectionpointsanddisposedCFLs
3.Theactivitydemonstratedpossibilitiesinbottom-upscheme
oforganizinghouseholdCFLdisposal//Component#2
2.4,4.1 Nov
2016
KlubVAVILON,GO $45,255
32 Organizingofmercurycontainingbulbs
disposalpointsincitiesofIvano-Frankivsk,
Poltava,Sumy,Lutsk,Zhitomyr-originalin
Ukrainian“Організаціяпунктівзборувід
населеннявідпрацьованих
енергоефективнихосвітлювальних
продуктів,щомістятьртуть(такихякКЛЛ,
ТЛЛ,тощо),зметоюподальшого
відпрацюваннямеханізмуїхутилізаціїта
переробкиумістах:Івано-Франківськ,
Полтава,Суми,ЛуцьктаЖитомир”
[https://drive.google.com/file/d/0B0ENyYqfz
4jeZ0R5bDNwVEozRU0/view?usp=sharing]
Purpose
1.Forimplementationofoutput2.4and4.1
2.ThegranteeorganisedcollectionpointsanddisposedCFLs
3.Theactivitydemonstratedpossibilitiesinbottom-upscheme
oforganizinghouseholdCFLdisposal//Component#2
2.4,4.1 Nov
2016
ZalenyiParus,GO $45,585
33 Organizingofmercurycontainingbulbs
disposalpointsincitiesofKharkiv,Odessa,
Dnipro,Kherson,Zaporozhe,Mykolaiv,
Khmelnitskiy,Chernivtsi-originalin
Ukrainian“Організаціяпунктівзборувід
населеннявідпрацьованих
енергоефективнихосвітлювальних
продуктів,щомістятьртуть(такихякКЛЛ,
ТЛЛ,тощо),зметоюподальшого
відпрацюваннямеханізмуїхутилізаціїта
переробкиумістахХарків,Одеса,
Дніпропетровськ,Херсон,Запоріжжя,
Purpose
1.Forimplementationofoutput2.4and4.1
2.ThegranteeorganisedcollectionpointsanddisposedCFLs
3.Theactivitydemonstratedpossibilitiesinbottom-upscheme
oforganizinghouseholdCFLdisposal//Component#2
2.4,4.1 Nov
2016
Ts.ek.usv.batk-va
"Dzherelaradosti"
$45,530
TMELTerminalEvaluationReport–31March2017
93
Миколаїв,Хмельницький,Чернівці”
[https://drive.google.com/file/d/0B0ENyYqfz
4jeREt1b19zdEJFWEE/view?usp=sharing]
34 ConsultantonevaluationofCO2emissions
reductionsbytheGEFmethodology
[https://drive.google.com/file/d/0B0ENyYqfz
4jeRDFIaUw1cHZuNzA/view?usp=sharing]
Purpose
1.Forimplementationofoutput3.3andglobalProjecttarget
2.ThereportreceivednegativefeedbackfromRTAandstudy
willbeconductedonemoretimein2017//Component#3
3.3 Mar
2015
GROZAIevgenii
Georgiiovych
$5,555
35 Energyauditoftheimplementedpilot
projects-originalinUkrainian“Проведення
енергоаудитипілотнихпроектів”
[https://drive.google.com/file/d/0B0ENyYqfz
4jeRnloZzNCQTJ5WFU/view?usp=sharing]
Purpose
1.Forimplementationofoutput3.3
2.Thestudyshowedthatlightingmanagementsystemscreate
bestopportunitiesintermsofelectricityconsumptiondecrease
//Component#3
3.3 Mar
2015
NTUUKPI $2,629
36 AnalysisofUkrainianlightingmarket2014 Purpose
1.Forimplementationofoutput4.3
2.ThestudyshowedthedevelopmentofEELmarketinUkraine
in2014//Component#4
4.3 Nov
2014
RBK-UKRAINE,TOV $300
37 AnalysisofUkrainianlightingmarket2015 Purpose
1.Forimplementationofoutput4.3
2.ThestudyshowedthedevelopmentofEELmarketinUkraine
in2015//Component#4
4.3 Sep
2015
TOV"Ukraiinskyi
Biznestehnologii"
$500
38 AnalysisofimpactofAwarenessCampaigns
2013-2016-originalinUkrainian
“Дослідженняринкуенергоефективного
освітленняпобутовогопризначенняв
україніза2014–іпівріччя2016років”
[http://lampochki.org.ua/wp-
content/uploads/2015/11/2016-Report-on-
EEL-market.pdf]
Purpose
1.Forimplementationofoutput4.3
2.ThestudyshowedthedevelopmentofEELmarketinUkraine
andProjectimpactduring2012-2016
3.ItwillbeusedtocalculateindirectCO2reductionscausedby
awarenessrisingactivities//Component#4
4.3 Nov
2016
Smerichevska
SvitlanaVasylivna,
FOP
$4,500
TMELTerminalEvaluationReport–31March2017
94
39 DesignofTextBookonEEL-originalin
Ukrainian
Purpose
1.Forimplementationofoutput4.2
2.TheTextBookisaimedonstudentsandteachersofrelevant
studycourses
3.Itwassharedwitheducationalinstitutions//Component#4
4.2 Apr
2015
PISTUNYevhen
Pavlovych
$1,500
40 DesignofGuidebookonEEstreetligthing Purpose
1.Forimplementationofoutputs3.13.2,5.2and5.4
2.TheguidebookcontainsthevaluablelessonslearntonEEL
implementationduringfromProjectexperience
3.Itwassharedwithmunicipalauthorities,scienceandcity
lightingmanagementstructures//Component#5
3,5 Oct
2015
PatrulYevgenii
Valeriiovych,FOP
$1,925
41 Mid-TermReview Purpose
1.MTRconductedreviewandprovidedtherecommendations
onfurtherProjectimplementation//Component#6
Feb
2014
HURRYSuresh $24,468
42 ProjectAudit Purpose
1.TheauditshowedthatfinancesideoftheProjecthasn’t
majorissues//Component#6
Apr
2015
Audytorska
KompaniiaPSP
Audyt,TOV
$1,200
43 ProjectAudit Purpose
1.TheauditshowedthatfinancesideoftheProjecthasn’t
majorissues//Component#6
Mar
2016
Audytorska
KompaniiaPSP
Audyt,TOV
$1,200
TMELTerminalEvaluationReport–31March2017
95
AnnexG(Part2):SummaryEvaluationofMajorStudiesCommissionedbytheTMELProjectNr. Title Purpose/InsupportofComponent#&Relevance Date Completedby Economic
effectImpact
1 Strategyforhandlingwastes
containingMercury//
originalinUkrainian:
«Стратегіяповодженняз
відходами,щомістять
ртуть»
(126pages)
[https://drive.google.com/fil
e/d/0BzlTNjcOQwWuVnZ2R
kplZWFrQWo5Y2V0MW00a
VF2ZnMtOTVj
/view]
-Purpose:ProvidingbasicsfortheStrategyofhandlingthemercurycontainingwastesinUkraine.Analysesofthelegal
frameworkandmethods,technologies,practiceofhandlingthe
mercurycontainingwastesinEU,USAandRussianFederation.
AnalysisofUkrainianlegislationoncollectionofused
fluorescentlamps,onfinancialpenaltiesfornon-compliance
withlegislationontheircollectionandutilization,oncostsfor
servicesoncollection,transportationandutilizationofCFLs.
-Component#1;foroutput1.4//
-Relevance:ProposalstotheNationalstrategyaredevelopedbasedonassessmentandcomparisonofthepoliticalandlegal,
environmental,economicandsocialfactors.Theconclusion
madeontopicalitytoadoptspeciallawincollectionand
utilizationofthemercurycontainingwastesintheresidential
sector,ascurrenttariffsestablishedbylocalself-government
bodiesandnon-liabilityforindividualsdonotstimulatefor
separatecollectionandutilizationoftheFLinresidentialsector.
Thisisaseriousshortcomingincurrentnationallegislationand
practice.ProposalstotheStructureoftheStrategy,its
monitoringinstruments,financialresourcesandmeansof
enforcementaremadeaswell.Thestudyishighlyrelevantand
topical.
ThisStrategywassubmittedtoMENRinJune2015
Nov,
2014
TOV
"MittalServis"
–Strategyon
hazardous
waste
handling
(UAH
1272970)
Sufficient Thisworkhassufficient
potentialforfurtherapplication
anduseinprocessof
developmentandadoptionthe
NationalStrategyOnWaste
Treatmentmanagednowbythe
MENR.
Also,argumentsmadeinthe
Strategycanbeusedforfurther
improvementofrelevant
legislation,inparticular,one
thatregulatestreatmentofthe
mercurycontainingwastesin
theresidentialsector.
2 DraftLaw“Onthehandling
ofusedlightingequipment
containingmercury
(fluorescentlamps)”//
originalinUkrainian:«Про
поводженняз
-Purpose:Toensuredetailedregulationofthepropertreatment
oftheFL.Itshouldmitigateshortcomingofcurrentlegislation.
-Component#1;foroutput1.4
-Relevance:Currentlytheissueisregulatedonlybygeneral
Oct,
2014Vygovska
Ganna
Pavlivna
Sufficient Theimpactislimitedbypolitical
willornon-willingnessto
promoteEElightning.Itmaybe
areasonableexplanationofthe
factthatproposedDraftlaw
wasnotregisteredasitis,but
TMELTerminalEvaluationReport–31March2017
96
відпрацьованимиртуть
вміщуючими
освітлювальними
приладами
(люмінесцентними
лампами)»
(15pages)
[https://drive.google.com/fil
e/d/0B06VW9ErMx4EakliSlJ
uaHhEeVU/view]
normsofArticle34oftheLawonWastesthatledtomisuseof
authoritybylocalself-governments,non-properutilizationofFL
andabsenceofliabilityofindividualsinresidentialsectorfor
non-compliancetolaw.ProposedDraftlawintroduces
utilizationfee,systemofseparatecollectionofFLand
establishmentaSpecialFundforFLutilization.
Onlysomeprovisions,mostlyregardingso-called“passporton
wastetreatment”ofproposeddraftwereusedinanotherDraft
lawmodifiedandregisteredintheParliament(№3374).Ithas
passedallrespectiveVRcommitteesandisincludedonlistsfor
thefirstreading
http://w1.c1.rada.gov.ua/pls/zweb2/webproc4_1?pf3511=5689
3s
(USD
4890)
onlysomeprovisionsareused
toimproveLawonWastesin
force.
IfproposedDraftlawis
registeredandthenenactedit
willpromoteproperhandlingof
usedFLandpromoteuseofLED
technologies.
3 DraftLaw“Onamending
legislativeactsofUkraine
(regardingimprovementof
theenergy-efficiencyin
lighting)”
OriginalinUkrainian
«ПроектЗаконупро
внесеннязміндодеяких
законодавчихактівУкраїни
(щодополіпшення
енергоефективностів
освітленні)»(19pages)
[http://w1.c1.rada.gov.ua/pl
s/zweb2/webproc4_1?pf351
1=56712]
-Purpose:TobringnationallegislationinlinewithEuropeanstandardsinenergyefficiency,creatingconditionsforphasing
outofincandescentlamps,aswellaspromotionoftransitionto
energyefficientlampsandlightinguseinmunicipalitiesand
households.Thesemeasureswillleadtoreducedenergy
consumption,ensureeconomicaluseofenergyresourcesand
willimprovethestateoftheenvironmentbyreducingCO2
emissions,alsowillimprovelivingstandardsduetoreduced
costsforenergy.Draftlawistointroduceintonational
legislationdefinitionsregardingenergysavingandenergy-
efficiencyinlighting,stimulateenergy-efficiencymyenergy
labelinganddefineasoneofastatepolicyprinciplereducing
greenhousegasemissionsintotheatmospherethroughthe
implementationoftheenergy-efficiencypolicy.
-Component#1;foroutput1.3
-Relevance:ThisdraftlawproposesamendmentstotheLaw
“OnEnergySaving”.Inparticular,itproposes:
• Tointroducenewdefinitionstothelegislationsuchas
Dec,
2014
Lebedeva
Alina
Volodymyrivn
a
(USD
4920)
Sufficient Whenadoptedthislawwillhave
extremelypositiveimpacton
promotingenergy-efficiency
policyinthecountry,reducing
CO2emissionsandensuring
compliancetorespectiveEU
policy.
TMELTerminalEvaluationReport–31March2017
97
energy-efficiency,lightingequipment,energy
intensity,eco-design,improvementsofenergy-
efficiency.Italsoextendsdefinitionoftheenergy
savinginlegislation;
• TointroducesomeprovisionsoftheDirective
2009/125/EU,andDirective2012/27/EU;
• Toestablishthatoneofthebasicprinciplesofthe
statepolicyonenergysavingistheintroductionof
energylabelingofhouseholdelectricappliancesby
class(А+++,А++,А+,А,В,С,D,E,F,G);
• Tointroduceanewprincipleofstatepolicyofenergy
security-"reducinggreenhousegasemissionsinto
theatmosphereasaresultofenergysavingpolicy";
• Tocreatealegalbasisforenergysavingpolicyin
housingandpublicsector;
AlloftheabovecurrentlymissinginUkrainianlegislation,thus
proposeddraftishighlyrelevant.
TheDraftlawwasdevelopedforGovernmentandregisteredin
theParliament(№3345),ithaspassedallrespectiveVR
committeeshearingsandisincludedintothelistofthedrafts
forthefirstreading
(http://w1.c1.rada.gov.ua/pls/zweb2/webproc4_1?pf3511=567
12)
4 Reportonstudyofexisting
labelingoflightingsources,
theirimplementationand
outcomes,proposing
-Purpose:Toprovidewithbackgroundinformationinorderto
promotelabelingoflightingsourcesinlinewithDirective
2010/30/EU.
Oct,
2013
LegalfirmLex
Plus
Sufficient Recommendationsareusedin
theTechnicalRegulationsfor
energylabelingoflightbulbs
andunitsadoptedbytheCMU
TMELTerminalEvaluationReport–31March2017
98
suitableforUkraine.Original
inRussian«Отчет
исследования
национальныхи
международныхнорм,
стандартов,
способствующих
внедрению
энергоэффективных
технологийвосвещении»
(15pages)
[https://drive.google.com/fil
e/d/0BzlTNjcOQwWuRjJQcm
NEc2d2Z2VkaUVFZXNULTEy
eF92aUhV/view]
-Component#1;foroutput1.4
-Relevance:Thisstudycontainstherecommendationson
ImprovementofTechnicalRegulationsonEnergyLabelingbased
onEUrequirementsandpreviousUkrainianexperience.Italso
arguesforvoluntarylabeling.
(UAH
644515)
Resolution#340dated
27.05.2015
5 ReportonReviewofthe
NationalandInternational
LegislationonUtilization
andRecyclingofFluorescent
Lamps.OriginalinUkrainian
«ЗвітДослідження
національногота
міжнародного
законодавстващодо
утилізаціїтапереробки
флуоресцентнихламп»
(14pages)
[https://drive.google.com/fil
e/d/0BzlTNjcOQwWucXhadX
FhNmttVW1RejB3TFRaOEFu
WG9XRG93/view
-Purpose:Toprovideoverviewofthebestexamplesof
internationallegislationandpracticesonFLcollectionand
recyclingthatmaybereplicatedtotheUkrainianrealm.
-Component#1;foroutput1.4//
-Relevance:TheReportsummarizedgoodpracticesand
legislationfromUSA,EUcountriesandRussianFederation.In
absenceofsufficientlegislationandoperatinginstrumentsfor
collectionandutilizationofFLfromhouseholds,thisreview
mightbecomeanextremelyrelevantstudytoproposeefficient
modelforthis.
Thereviewsuffersfrombadstructure,absenceofthe
referencestothesourcesofinformation,non-
comprehensivenessofmaterialspresented,absenceofanalysis
ofcomparativeadvantagesofvariousmodels,aswellas
structuredrecommendations.
Nov,
2012
LegalfirmLex
Plus
(UAH
556731)
Moderately
sufficient
Certainimpactisachievedas
theinformationfromReport
wasusedtodesignDraftLaw
“Onthehandlingofused
mercurycontaininglighting
equipment(fluorescentlamps)”
Thematerialspresentedinthe
Reportwerenotusedin
mentionedDraftlawtofull
extent,inparticulardescription
ofvariousinstrumentsfor
collectionandutilizationofthe
FL.Itmaybeexplainedby
weaklystructuredinformation
intheReportandabsenceof
clearconclusionsand
recommendations.
TMELTerminalEvaluationReport–31March2017
99
6 ReportonReviewofthe
NationalandInternational
Norms,Standardsand
LegislationonPromotionof
Energy-efficientLighting
Technologies.Originalin
Ukrainian«Звіт
Дослідженнянаціональних
таміжнароднихнорм,
стандартівізаконодавства,
щосприяють
енергоефективним
технологіямосвітлення»
(10pages)
[https://drive.google.com/fil
e/d/0BzlTNjcOQwWub25SN
mZZczBsR1ZtaGlZX21YVjVOT
lgxSEpF/view]
-Purpose:StudywasundertakentoestablishabaselineinthesphereoflegislationonEnergy-efficiency.
-Component#1;foroutput1.1
-Relevance:Proposedrecommendationsonbasicsfor
developmentNationalprogramsforpromotionEElighting
technologies,includingMEPSstandards,labeling,financialand
taxstimulus,publicprocurementpolicy,awarenessraising
policy,partnerships.Policyrecommendationsaremadeaswell.
Dec,
2012
LegalfirmLex
Plus
(UAH
225337)
Moderately
sufficient
Therecommendationswere
usedinprocessofdesignof
Draftphase-outprogramfor
Ukraineandoverallroadmapas
describedunder#7ofthistable
7 Developmentofthe
roadmaptowardsphasing-
outenergyinefficient
lightinginUkraine
(141pages)
[https://drive.google.com/fil
e/d/0B0ENyYqfz4jeSkZoUjdT
Y0hIM0E/view?usp=sharing]
-Purpose:ToproposeistoprovidewithanalysisofinternationalexperienceandrecommendationstoUkraineregarding
developmenttheroadmaptowardsphasing-outenergy
inefficientlightingandpromotionofEElighting
-Component#1;foroutput1.1
-Relevance:Thereportconsistofthreeparts,inparticular,1.Phase-outcomplianceprogramandpromotionprograms,2.
Complianceprogram,3.Nationaltestlaboratories.Basedon
internationalexperiencethisreviewpresentstheideathatas
soonasalternativetotraditionalandmoreEEtechnologiesare
availableinthemarket,thetransitionstrategyhastobe
developedtospeeduptheprocessofphasing-outofinefficient
lightingproductsusedinacountry.Theenergysavingsarenot
Dec,
2014
Lighting
Business
Consulting,
Ltd.
(USD
24800)
Sufficient Road-mapproposedforthe
Governmenttobeincludedinto
Ukrainianpolicyplanning
documents.
TMELTerminalEvaluationReport–31March2017
100
theonlyaspectsthatshouldmatterwhenitcomestochoosing
analternativeinlighting(forbothindoorandoutdoorlighting).
Therequirementsforqualityperformanceshouldbeestablished
tocutbackthecostsofenergyusedforlighting.Thereview
presentsstatusofcompliancewithEErequirementsinUkraine,
reviewofstimulationprograms,generalmarketsurveillance
procedure,structureofthenationalcomplianceregimeand
risksforthecomplianceprogramimplementationetc.
Prioritylistofenergyperformancestandardsforlighting
equipmentandsystemsisconsidered,respectiveproposalsfor
legislativechangesaremade.TheInternationalEnergy
ConservationCode(IECC)asamodelcodeadoptedbymany
statesandmunicipalgovernmentsintheUnitedStatesforthe
establishmentofminimumdesignandconstruction
requirementsforenergyefficiencyisconsidered.TheEC
245/2009Ecodesignrequirementforfluorescentlampsis
consideredassourceformandatoryperformancestandardsfor
luminairesinEurope.Recommendationsfordevelopmentofan
indicativebudgetfortheenergy-efficientlightingaremade.
Evaluationoftheroleofnationaltestlaboratoriesispresented
togetherwiththeorganizationalstructureofnational
accreditationsystemofUkraine.Anoverviewofpossibleways,
descriptionofmechanismsanddevelopmentofproposalsfor
nationaltestlaboratoriesonobtaininginternational
accreditationandonbecomingapartoftheinternational
verificationsystemismade.Thedraftphase-outprogramin
Ukraineproposedforlightingequipment.
Thestudyishighlyrelevanttothecontextofprojectareasof
activityandtothedevelopmentfurtherrecommendationsto
theNationalGovernment.Thedocumentwassubmittedto
MENRtogetherwithletter№2645dated30.06.2015
TMELTerminalEvaluationReport–31March2017
101
8 EULegislationand
internationalexperience
analysisonphaseoutof
incandescentlampsand
promotionofEElightingin
Ukraine.Originalin
Ukrainian“Аналіз
законодавства
ЄвропейськогоСоюзута
міжнародногодосвіду
йогоімплементаціїв
частині,щостосується
поетапноївідмовивід
лампрозжарюванняі
впровадження
енергоефективного
освітленнявУкраїні”
(59pages(235together
withannexes)
[https://drive.google.com/d
rive/folders/0B0ENyYqfz4jeZ
1NHR0RpVTg4ZzQ]
-Purpose:ToevaluateefficiencyoftheprocessofUkrainianlegislationadaptationtoEUasrequiredbyUkraine-EU
AssociationAgreement,identifyproblemsandevaluatelevelof
complianceatcurrentstage.
-Component#1;foroutput1.3
-Relevance:Thedocumentdevelopedistopical.Itprovided:
criticalreviewofUkrainianlegislationtogetherwith
identificationofshortcomingsandoutdatednorms;
comparativeanalysisofUkrainianlegislationregardingenergy-
efficiency,energylabeling,emissionsetc.fromtheperspective
ofphasingoutIL;Recommendationsforadaptationand
implementationoflegislation;descriptionofbestpracticesand
recommendationsforUkraineonILphase-out.
ReviewofrelevantEUDirectives,inparticular2002/91/EU-
EPBD,2010/31/EU,2005/32/EU,2009/125/EU,2010/30/EU,
2009/28/EU,2012/27/EUwithdescriptionoftheirrelevance,
threatsforimplementationandperspectivewaystoimplement
isundertaken.ThetechnicalregulationsonlightingEN12464–1
andEN15193–1arereviewedaswell.Impactofprovisionsof
theEUDirectives2006/32/EC,2009/72/EUand2009/73/EUon
energysavingcalculationisconsidered.Nationalenergy-
efficiencyactionplansandreportsareproposedasinstrument
tonationalpolicymakingprocess.RespectiveUSAexperienceis
consideredaswell.Anissueofmisinterpretationofsome
standardtermsoflawinnationallegislationisdiscussedand
proposalstotackletheissuearemade.
Anapproachtopromoteenergy-efficientlightingequipment
throughadoptionrelevantlegislationonEco-designisproposed.
Detailedrecommendationsonlegislationdevelopment,
technicalregulationsimprovement,creationnewinstruments
forelectricitytradingareproposed.
Nov,
2016
Samborska
Yuliia
Leonidivna
(USD
6800)
Sufficient Thisanalysisprovideswithup-
datedinformationontheEU
legislationonthesubjectmatter
thathavebeenchangedsince
2012,thusrecommendations
madeinthisstudymayserveas
aroadmapforlegislatorsto
amendUkrainianlegislation
accordinglyandcomplywithEU
normsandrequirements.
Theanalysisalsowassubmitted
toMPat12.12.2016inorderto
supportprocessofadoptionof
alreadyregisteredDraftsLaw.
TMELTerminalEvaluationReport–31March2017
102
9 EvaluationofBudget
impactsofILCphase-out.
OriginalinUkrainian“Аналіз
економічноїефективності,
отриманоївідпоетапної
відмовивідламп
розжарювання,включаючи
впливнадержавний
бюджетУкраїни
“[https://drive.google.com/
drive/folders/0B0ENyYqfz4j
eZ1NHR0RpVTg4ZzQ]
-Purpose:1.Forimplementationofoutput(createalawfor
totalphaseoutofIBs)2.Studycontainstheassessmentof
budgetimpactoftotalIBphase-out3.ResultssubmittedtoMP
on12.2016
-Component#1:foroutput1.3
Nov,
2016
Danchuk
Anna
USD8600
10 ResearchonFinancingfor
Lighting.Originalin
Ukrainian“Аналіз
ефективностірізнихформ
фінансуваннядля
комплексноїмодернізації
освітлення”
[https://drive.google.com/d
rive/folders/0B0ENyYqfz4jeZ
1NHR0RpVTg4ZzQ]
-Purpose:1.Forimplementationofoutput1.3.(createalawfor
totalphaseoutofIBs)2.Thestudycontainstheanalysisof
differentfinancialtoolsforEELandrecommendationfor
Ukraine,3.ResultssubmittedtoMPat12.12.2016.
-Component#1:foroutput1.3
Nov,
2016
DanchukIgor
USD13000
11 DevelopmentofaNew
TechnicalStandardon
ElectricPowerQuality.
OriginalinRussian
“Разработкановых
техническихнормпо
качеству
электроснабжения”
(156pages)
-Purpose:Thepurposeofthisstudyisnotdefinedinthereport.Basedonthecontent,theconclusionsmaybedrawnthatmain
subjectmatterofstudyis:(i)definitionofqualityparametersfor
electricpower;(ii)definitionofthewaystocontrolaqualityof
electricpower;(iii)criticismofcurrentStateStandardand
proposingnewonethatsupposedtoensuresufficientcontrol
overqualityofelectricpowerforpromotionnewenergy-
efficienttechnologies.
-Component#1;foroutput1.2
-Relevance:
Aug,
2014
NT-Project,
Ltd
(USD
175324)
Non-
sufficient
Majorityoftheinformation
presentedinthestudyregarding
problemwithqualityofelectric
powerinthegridinUkrainecan
befoundinopensourcesof
information.
Evaluationoftheinnovative
characterandrelevanceofthe
textofproposednewState
Standard(ГОСТ)requires
specificknowledgeandskills.
TMELTerminalEvaluationReport–31March2017
103
[https://drive.google.com/fil
e/d/0B0ENyYqfz4jeb2RIdW
QzM2UxUW8/view?usp=sha
ring]
Inthecontextofthisreportabriefdescriptionofthemain
problemswiththequalityofelectricpowersupplyandcontrolis
provided.Aplanofactiononimprovementofthequalityof
electricpowerisproposed.Itsuggestsimprovementoftechnical
charactersofthecurrentbyissuingrecommendationsfor
consumersandforhousingandcommunalservices;
improvementoftheregulatoryenvironment;publichearings.
Thereportonpublichearingheldin2014onqualitycontrolwith
participationofrepresentativesfromVatra,Iskra,Philips,Maxus
andUkrtestisprovided.Resolutionofthishearingisthat
currentStateStandard(ГОСТ13109-97)isincompliancewith
InternationalStandards(МЭК868,МЭК1000—3—2,МЭК1000-
3-3,МЭК1000-4-1,МЭК1000-2-1,МЭК1000-2-2)butsince
year1997onEUlevelandinternationallevelnewstandards
wereadoptedthatnationalonesdon’tcomplytofullextend.
The29Recordsonmeasuringqualityoftheelectricpowerin
electricgridarepresented.Theseprotocolsdemonstratedthe
non-compliance.Consideredelectricpowerascommoditywith
certainsetofcharacteristicsofquality.TheLawOnElectric
PowerIndustryisconsidered,aswellasTechnicalRegulationsas
basicnormativeactstoensurequalitycontrolofelectricpower
inUkraine.Currentfactorsthatnegativelyimpactthequalityof
electricpowerinUkrainearebrieflyconsidered.
Methodological,organizationalandtechnicalmeasuresare
proposedtotackletheproblemofnon-stablequalityofelectric
power.Asoneofinstrumenttoresolvetheissue,thesystemof
permanentmeasurement/monitoringofthequalityofelectric
powerisproposed.Thedeviceisproposedtobeincorporatedin
parallelwiththeelectricsupplymeterontheentities.Itwill
complywiththeISO9000requirementsasauthorsconsider.
Datacollectedinprocessofqualitymeasurementcanbeused
forenergyauditofenterprises.Authorsarguedforplacementof
thehub/groupqualitycorrectionequipmentratherthan
individualones.
IncasethisnewStateStandard
isinnovative,itshouldhavea
positiveimpact(whenadopted)
onensuringqualitycontrolover
electricpowersupplyandon
promotionnewEEtechnologies.
TMELTerminalEvaluationReport–31March2017
104
Asaresult,authorsconsiderthatcurrentStateStandard(ГОСТ
13109-97)isnotsufficienttoensurequalityofelectricpowerin
grid,thustheyproposednewStateStandard(ГОСТ13109-14)
called“Electricenergy.Electromagneticcompatibilityof
technicalequipment”.Powerqualitylimitsinpublicelectrical
systems”.ThecurrentStateStandardshouldbereplacedby
proposednewonethatisinlinewithrequirementsofrelevant
internationalstandards(МЭК868,МЭК1000—3—2,МЭК1000-
3-3,МЭК1000-4-1;МЭК1000-2-1,МЭК1000-2-2).Thetextof
newproposedstandardisintegralpartofthestudy.
12 FinalReportonTestingfor
AssessmentofQualityof
EnergyEfficientStreet
LampsandLuminaires
AvailableontheMarketof
Ukraine.Originalin
Ukrainian“Oцінкаякості
енергоефективних
вуличнихлампі
світильниківдоступнихна
ринкуУкраїни”
(73pages)
[https://drive.google.com/fil
e/d/0B0ENyYqfz4jedzU2Q1l
6YTIwWHM/view?usp=shari
ng]
-Purpose:Toundertakeassessmentbasedontestingresultsof
streetlampsandluminairesavailableonUkrainianmarketin
ordertofacilitatethetransitiontoenergy-efficientlighting
marketandtopromoteenergyefficiencylighting,aswellas
ensuringcompliancewithnewstandards.Thetestingwasheld
intheperiodof21.12.2014-24.04.2015.
-Component#2;foroutput2.2
-Relevance:
Thistestingwasheldfordischargelamps(mercuryarclampsof
highpressure,sodiumlampsofhighpressure,metal-halide
lamps)offivemarketbrandsandLEDluminairesoffourbrands
forlightingthestreetsandroads.Totalnumberoftestedlamps
is1500pcs,ofluminaires-200pcs.
LampsofOptima,DeLux,Iskra,Osram,Philipsweretested
againststandards:ДСТУEN60662-2001;ДСТУIEC62035:2005;
ДСТУIEC60662-2001;ДСТУEN61167:2014;ДСТУIEC
60188:2003,aswellasluminairesofBellson,Philips,CETOand
НПВ«Радій»againststandards:ДСТУEN60598-2-3:2014;ДСТУ
ІЕС60598-1:2002;ДСТУCISPR15:2007;ДСТУІЕС61000-3-
2:2004;ДСТУEN61000-3-3:2004;ДСТУIEC61547-2001).
Jul,
2015
SGSUkraine
(USD
526935)
Non-
sufficient
Resultsoftestingmayserveas
guidelinesforconsumerson
qualityofsomelampsand
luminairespresentedon
Ukrainianmarket.
(resultslooksimilartotheones
representedunderReport#23,
butlessuniversal)
TMELTerminalEvaluationReport–31March2017
105
Finalsummaryoncomplianceoftheselampsandluminairesto
selectedstandardsispresented.Also,proposalismadeto
undertakeperiodicalamendmentoftheResolutionofCMU№
992,dated15.10.2012inordertoensureitsupdateaccordingto
European(EN)andinternationalstandards(IEC).
Thestudyisrelevant,evenit’snotcomprehensiveanditshows
thatproductsofbelowestablishedparametersarepresentedon
Ukrainianmarket.Theresultsofthistestingwerepresentedin
2015oninternationalconference"ChangetheBulb-Changethe
World"
13 ReviewonDevelopmentof
theStrategieson
Implementationof
EconomicallyViableEnergy-
SavingLightingTechnologies
inUkraine.Originalin
Ukrainian–Дослідження
«Розробкастратегій
впровадженняекономічно
доцільних
енергозберігаючих
технологійосвітленняв
Україні»
(254pages)
[https://drive.google.com/fil
e/d/0BzlTNjcOQwWuYndYU
HpnekFtc2c/view]
-Purpose:TodemonstrateadvantagesoftheLEDtechnologies
promotiontoUkrainianmarketcomparingtoordinarylight
sourcessuchasincandescent,halogen,fluorescentand
dischargelamps.Thereviewproposeseconometricmodelsfor
Ukrainianmarketdevelopmenttill2030basedoncomparing
energyconsuming,savingofenergyandpricedifferences.
-Component#4;foroutput4.2.
-Relevance:
ThisreviewpresentsananalysisofUkrainianlightingmarket
dynamics;analysisofthelegalframework;reviewoftechnology
developmentandimprovementofLEDlighting;prognosisofthe
electricenergysavingpresentedbysectors(households,street
lighting,industry)basedonthemodelingapproach.
Thesummaryofresultsmaybepresentedasfollowing:
assumingthatLEDlampsandluminariesmeettheirexpected
efficacy,lifetime,andpricetargetstheLEDlightingwillgain
significantmarketpenetration.By2020,LEDlightingisexpected
torepresent36percentoflumen-hoursalesonthegeneral
illuminationmarket.By2030,itisexpectedtogrowto74
percentoflumen-hoursales.In2030,theannualenergysavings
duetotheincreasedmarketpenetrationofLEDlightingis
estimatedtobeapproximately30terawatt-hours,orthe
Oct,
2014IK"Sfera
Plus",
(UAH
258600)
Sufficient Theresultsofthisreviewwere
usedduringawareness
campaignsandinprocessof
pilotsimplementation
TMELTerminalEvaluationReport–31March2017
106
equivalentannualelectricaloutputofaboutfive1,000-
megawattpowerplants.Attoday’senergyprices,thatwould
equatetoapproximately$3billioninenergysavingsin2030
alone.Assumingthecurrentmixofgeneratingpowerstations,
theseenergysavingswouldreducegreenhousegasemissionsby
21millionmetrictonsofcarbon.Thetotalelectricity
consumptionforlightingwoulddecreasebyroughly46percent
relativetoascenariowithnoadditionalpenetrationofLED
lightinginthemarket.
Overthe20-yearanalysisperiod,spanning2011-2030,the
cumulativesiteenergysavingsareestimatedtototal
approximately2700terawatt-hoursrepresentingapproximately
$25billionattoday’senergyprices.Assumingtheelectricpower
plantgeneratingmixisheldconstantoverthenexttwodecades,
thesesavingswouldreducegreenhousegasemissionsby180
millionmetrictonsofcarbontheresidentialandcommercial
sectorsprovidethegreatestopportunityforenergysavings.The
formerisprimarilycomposedofinefficientincandescentlamps,
towhichLEDsprovideacost-effectivealternative.The
commercialsectorcontributed60percentoflightingservicein
theUkrainein2011,thuspresentsanopportunityforsignificant
energysavings.By2030,thecommercialsectorenergysavings
potentialwillbe35percentofthebaselineenergyconsumption.
14 ConsumerMarketAnalysis.
OriginalinUkrainian“ЗВІТз
дослідженняспоживачів
ринкупродукції
енергоефективного
освітленняУкраїни”
(527pages).
Purpose:ThestudyestablishedthebaselineofEELconsumers
market.
Component#4:foroutput4.32.
-Purpose:thestudywasusedtoformulateToRfor1st
PromotionalCampaign//Component#4
Apr,
2012
IK"Sfera
Plus",TOV
(UAH
431800)
TMELTerminalEvaluationReport–31March2017
107
[http://lampochki.org.ua/wp
content/uploads/2015/11/R
eport-on-consumer-market-
research-2012.pdf]
15 Producermarketanalysis-
originalinUkrainian
“Проведеннядослідження
товарногоринку
енергоефективного
освітленняУкраїнита
факторіввпливунайого
розвиток”
[http://lampochki.org.ua/wp
-
content/uploads/2015/11/R
eport-on-product-market-
research-2012.pdf]
Purpose:ThestudyestablishedthebaselineofEELconsumers
market.
Component#4:foroutput4.32.
-Purpose:thestudywasusedtoformulateToRfor1st
PromotionalCampaign//Component#4
Nov,
2012
Kviten
V.S.,Ltd(UAH
208516)
16 Analysisofimpactof
AwarenessCampaign-
originalinUkrainian
“Проведеннядослідження
впливупроведеної
всеукраїнської
інформаційноїкампаніїз
енергоефективного
освітленняна
інформаційнийрівень
населенняукраїнитайого
відношеннядо
енергозберігаючихламп”
(482pages)
Purpose:Thestudyestablishedthebaselineofconsumer
awarenessasaresultofthefirstconsumerawareness
campaign.
Component#4:foroutput4.32.
-Purpose:thestudywasusedtoformulateToRfor2st
PromotionalCampaign//Component#4
Dec,
2013
Kviten
V.S.,Ltd
(UAH
320066)
TMELTerminalEvaluationReport–31March2017
108
[http://lampochki.org.ua/wp
-
content/uploads/2015/11/2
013-Report-on-assessment-
efficiency-of-a-
Campaign1.docx]
17 ReportonDevelopmentof
SanitaryStandards/Codes
andRulesforLightingin
ResidentialandPublic
Buildings.OriginalinRussian
“Разработкасанитарных
нормиправилосвещения
жилыхиобщественных
зданий»”
(86pages)
[https://drive.google.com/fil
e/d/0B0ENyYqfz4jedEVlX2t5
X1BDR28/view?usp=sharing]
-Purpose:Thepurposeisnotidentifiedinthereport.
-Component#1;foroutput1.3.
-Relevance:
Therelevanceisdifficulttodefine,asnogoalofthestudyis
identifiedandnocomparisonforstandardsofsanitarycodeis
presented.The42pagesofreportpresentthegooglemap
photosofthebuildingschosentomeasurelevelofillumination.
AlsotheauthorsproposedneweditionoftheSanitarycodewith
twotablesreflecting:(i)indexesfornatural,artificialand
combinedlightinginresidentialbuildings,and(ii)indexesof
natural,artificialandcombinedlightinginpublicbuildings.
Dec,
2015
TOV
"MittalServis"
(USD
122366)
Non-
sufficient
Projectreportedthat“thestudy
containssomerecommendation
tosupplementimplementation
ofDBN"Artificialandnatural
lighting",butnoprovenrecords
forthisareprovided.The
informationofverygeneral
characterthatisavailablefrom
opensourcesispresentedinthe
report.Nosourcesof
verificationthatthisreportwas
usedduringprocessof
developmentDBNisprovided.
18 SupportofDraftLawon
Phase-outinParliament
-Purpose:AnexpertwashiredtosupporttheDraftLaw3345inVR.
-Component#1:foroutput1.3.2
-Relevance:
SupportforGovernmentalOfficeonEuroIntegrationand
CommitteeofVRonEuroIntegration.
Feb
2017
Sydorenko
Svitlana
Victorivna
USD4990
TMELTerminalEvaluationReport–31March2017
109
19 TheFinalReportofthe
StudyontheUkrainianand
InternationalLegislationon
theQualityofElectricity
Supplied.Originalin
Ukrainian“Остаточнийзвіт
продослідженнящодо
українськогота
міжнародного
законодавствазякості
електроенергії,що
постачається”
(32pages)
[https://drive.google.com/fil
e/d/0B0ENyYqfz4jebVFORW
pOTFhCbVk/view?usp=shari
ng]
-Purpose:Toproviderecommendationsonthestrengthening
accountabilityofenergyserviceproviders,protectconsumer
rightswithregardofelectricitypowerquality.
-Component#1;foroutput1.2
-Relevance:
TheanalysisUkrainelegislationontheprotectionofconsumer
rightsandliabilityofsupplierforinadequatequalityof
electricityinthelightofexistingpracticeisundertaken.A
considerationoflegislationandpracticestoensureprotectionof
consumerrightsonsupplyofelectricpowerofdefinedqualityis
madebasedonexperienceofsomeselectedEUmemberstates.
ConclusionmadethatinUkraineonlysomeprovisionsofEU
legislationonsubjectmatterareenforced.Respective
recommendationsforlegislationimprovementareproposed.
AnalysisofthecurrentpracticeinUkraineshowsthatit’s
possibletoimplement:(i)voluntaryenforcementofthe
decisionsadoptedbyNKREKPwhenitadoptslegallybinding
decisiontothepartiesondisputeregardingqualityofelectric
powersupplied;and(ii)adoptionoflegalactstoensure
approximationofnationalstandardstoEuropeanonesunder
requirementsoftheThirdDirective.
May,
2015
UPF
"Gryshchenko
taPartnery"
(UAH
325080)
Marginally
sufficient
Therecommendationsarenot
comprehensive
Recommendationsonlegislation
improvementweresubmittedto
MENR
TheParliamenthasapprovedin
firstreadingthedraftLaw"On
themarketofElectricPower”
(http://w1.c1.rada.gov.ua/pls/z
web2/webproc4_2?pf3516=449
3&skl=9).//
20 TechnicalSupportfor
DevelopmentofTechnical
RegulationsToward
UtilizationofMCLSin
Ukraine
-Purpose:Toproviderecommendationsonregulationsfor
MCLS.
-Component#1;foroutput1.4
-Relevance:
DesignedDraftTechnicalRegulationandsubmittedtoMENR
(letter№2645,30.06.15)
Oct
2014
Kozachenko
Tetyana
Petrivna-USD
4770
TMELTerminalEvaluationReport–31March2017
110
21 Designof1steditionofDBN
"ArtificialandNatural
Lighting.”Originalin
Ukrainian“прДБНВ.2.5-
28:2016Природнеіштучне
освітлення”
-Purpose:Toproviderecommendationsonlightingstandards.
-Component#1;foroutput1.3
-Relevance:
1steditionofDBN"Artificialandnaturallighting"designedand
submittedtoMRDCHforassessmentandapproval(letterfrom
PromElektroKomplekt06.12.16).
Dec
2014
TOV
KiyvPromElek
troProekt-
UAH115000
22 Designof2ndeditionof
StateConstruction
Regulations(DBN)"Artificial
andNaturalLighting.
OriginalinUkrainian“пр
ДБНВ.2.5-28:2016
Природнеіштучне
освітлення”
(136pages)
[http://dbn.at.ua/load/norm
ativy/dbn/dbn_v_2_5_28_2
015/1-1-0-1188]
-Purpose:TodraftStateConstructionRegulationsthatincorporaterequirementsfortheEElighting/
-Component#1;foroutput1.3
-Relevance:
FulltextoftheStateConstructionRegulationsonsubjectmatter
isdrafted
Aug,
2015
TOV
KiyvPromElek
troProekt
(UAH
135000)
Sufficient 2ndeditionofDBN"Artificial
andNaturallighting"isdesigned
andsubmittedtoMinistryof
RegionalDevelopment,
Construction,Housingand
communalservicesfor
assessmentandapproval(letter
fromPromElektroKomplekt
dated06.12.16)
23 AssessmentofQualityof
HouseholdEELighting
Sources.Originalin
Ukrainian“Порівняльне
тестуванняпобутових
енергоефективнихджерел
освітлення”
[https://drive.google.com/fil
e/d/0B0ENyYqfz4jeam1MU0
pUZGxFelk/view?usp=sharin
g]
-Purpose:purposeisnotidentified.
-Component#2;foroutput2.2
-Relevance:
Bulbsoffollowingbrands:Iskra,NashaSyla,EcoEnergy,Global,
Lexman,Maxus,Osram,Philips,Raflight,Visson,Eurolamp,SLS,
Uniel,Electrum,Jazzway,ELM,Lummax,Extra,weretested
against21parameters.
Thestudyprovidedwithcomparativecharactersforbulbstested
andsomeofthemdoesn’tmeetrequiredstandards
Dec,
2014
Fundatsiya
RozvytkuEko.
iEnerg.
Rynkiv
(UAH
1186446)
Resultsof
testsare
presented
inone
Exceltable,
thusit’s
impossible
tomeasure
economic
effect,in
particular
whether
thistesting
Resultsoftestingmayserveas
guidelinesforconsumerson
qualityofthesomelampsand
luminairespresentedon
Ukrainianmarket.
(resultslookssimilartotheones
representedunderReport#12,
butmoreuniversal)
TMELTerminalEvaluationReport–31March2017
111
Thetable(asthereportofstudyshowsthatthetestingwasheld
byCenterofExpertiseTEST(ЦентрЭкспертизТЕСТ
(www.test.org.ua)inperiodMarch,2014–March,2015
shouldcost
equivalent
ofUSD
78800
24 Energyauditofthe
implementedpilot
projects//originalin
Ukrainian«Проведення
енергоаудитупілотних
проектівврамкахпроекту
ПРООН/ГЕФ
«Трансформаціяринкув
напрямку
енергоефективного
освітлення»
(63pages)
[https://drive.google.com/fil
e/d/0B0ENyYqfz4jeRnloZzN
CQTJ5WFU/view?usp=sharin
g]
-Purpose:Themainpurposeofworkperformedwastoconduct
anenergyaudit,inparticular,toevaluatetechnicalparameters
ofthepilotprojectslaunchedbyUNDP/GEFProject,aswellas
determinetheefficiencyofpilotprojectslaunchedandidentify
feasibilityoftheirfurtherreplication.
-Component#3;foroutput3.3
-Relevance:
Practicalrecommendationsdevelopedhavehighrelevancefor
furtherpromotionoftheapproachproposedbytheproject.The
studyshowedthatlightingmanagementsystemscreatebest
opportunitiesintermsofelectricityconsumptiondecrease.
Mar,
2015
NTUUKPI
(UAH
71002)
Sufficient Fivecitiesandtwovillages
installedEEstreetlightingand
gainedadvantagesand
economiceffectasresultof
implementationpilotprojects
25 AnalysisofUkrainianlighting
market2014
Purpose-1.Forimplementationofoutput4.32.Thestudy
showedthedevelopmentofEELmarketinUkrainein2014//
Component#4
Nov
2014
RBK-
UKRAINE,TOV
-UAH1500
26 AnalysisofUkrainianlighting
market2015
Purpose-1.Forimplementationofoutput4.32.Thestudy
showedthedevelopmentofEELmarketinUkrainein2015//
Component#4
Sep
2015
TOV
"Ukraiinskyi
Biznes
tehnologii"-
UAH2300
TMELTerminalEvaluationReport–31March2017
112
27 Researchonthemarketfor
energy-efficientlightingfor
domesticpurposesin
Ukraineinyears2014-2016.
OriginalinUkrainian
“Дослідженняринку
енергоефективного
освітленняпобутового
призначеннявУкраїніза
2014–іпівріччя2016
років”
(121pages)
[http://lampochki.org.ua/wp
-
content/uploads/2015/11/2
016-Report-on-EEL-
market.pdf]
-Purpose:Tocollectinformationandundertakeananalysisand
evaluationofthedynamicsofthemarkettheEElightingfor
household(CFLsandLEDlamps)inUkrainefor2013-2015and
thefirsthalfof2016comparedto2013.Thisresearchisalso
supposedtomakeananalysisofthedynamicsofpublic
awarenessaboutthebenefitsofEElighting,butalsoevaluation
oftheefficiencyoftheAll-Ukrainianpublicawareness
campaignsheldbytheprojectinyears2014-2016.Themain
researchmethodswere:a)collectionofinformationandDesk
reviewofsecondaryinformationonthecurrentstateand
dynamicsofEElightingmarketinUkraine,aswellasb)analysis,
evaluationandinterpretationofinformationfromsecondary
sourcesoncurrentstateanddevelopmentofEElightingmarket
inUkraine.
-Component#4;foroutput4.3
-Relevance:
Evaluationofnationwideinformationcampaignheldin
frameworkoftheprojectandanalysisofthemarketofenergy-
efficienthouseholdlightingheldinUkraineincourseof2014-
2016showedthefollowingresults:
(i)NumberofeffectivecontactsduringAll-Ukrainianeducational
campaigninschoolsonenergy-efficientlightingreached
8534298persons(or55,3%)oftargetaudienceof
schoolchildren,theirparentsandteachers;
(ii)2ndstageofAll-Ukrainianinformationcampaignonenergy-
efficiencyheldinyear2015wasfocusedonadultsaged17+
years.Thiscampaignreached17955663personsrepresenting
56.6%oftargetaudience;
(iii)3-rdstageofthecampaignwasconductedonpromotion
LEDlampsinthenationwideretailnetwork"EpicenterK";
(iv)Alltogetherthesethreestagesofinformationcampaign
Nov,
2016
Smerichevska
Svitlana
Vasylivna,
FOP
(UAH
121800)
Sufficient Theresultswereusedto
calculateindirectCO2
reductionscausedbyawareness
risingactivities.
Theconclusionmadeon
efficiencyofawareness
campaignsheldrequire
additionalsourcesofverification
TMELTerminalEvaluationReport–31March2017
113
resultedinthelevelofawarenessonEElightingof74,8%ofthe
populationofUkraineaged17/18+and55.3%ofpopulation
aged6-17/18;
(v)IncaseifmarketpricesonCFLandLEDaredeclined,the
commoditymarketforEElightingtechnologieswilldemonstrate
growthandincreaseofthesalesinvolume;
(vi)TheshareamongmajorplayersonUkrainianEElighting
marketasof2016arelistedasfollowing:MAXUSwithamarket
shareof18,0%,EUROLAMP-16,4%,ELECTRUM-14,3%,OSRAM
-12,3%,FERON-10,5%,PHILIPS-10.1%,SVETKOMPLEKT-9.0%,
ESTARES-3.3%;
(vii)Lowqualityproductsareavailableonthemarket;(viii)Since
year2013to2016asignificantreductioninpricesfordomestic
LEDlampsisobserved.Pricesdeclinedin2016comparedto
2013fromUSD43.8toUSD5.8perunitinsegmentofhigh
prices,fromUSD22.5toUSD2.3insegmentofmediumprices
andfromUSD14.4toUSD1.3insegmentoflowprices.Thisfact
standssignificantincentivetoincreasedemandforLEDlampsas
themostenergy-efficientones;
(viii)In2014therewasadeclineindemandforEElamps
comparedtoyear2013by16.6%duetoasharpincreasein
inflation.Atthesametime,in2015thesalesofEElampshave
increasedby53.5%comparedtoyear2014.In2016,thevolume
ofdomesticsalesofCFLandLEDlampswillincreaseby11.4%
comparedtotheyear2015,andby42.6%comparedtoyear
2013andby53.3%comparedtotheyear2012;
(ix)Inactualsalesplusforecastforyear2016ashareofEE
lampsforhouseholdis72.6%(thatdemonstratesincreaseon
20.5%comparingtoyear2013and22.5%to2012),whilethe
incandescentlampshadshareof27.4%(20.5%lessthanin2013
and22.5%thanin2012);(xi)DemandforLEDlampsincreased
sharply,thusthemarketshareoftheselampsinkindincreased
TMELTerminalEvaluationReport–31March2017
114
in2014and2015comparedtopreviousyearsby1.9%and
27.2%respectively.Atthesametime,incomparisonwithyear
2013theshareofsalesforthislampsincreasedin2015on
29.1%,andforecastforyear2016is38.9%;
(x)Comparedtoyear2013thesalesofhouseholdEElamps
increasedin-kindin2015and2016respectivelyon28%and
42.6%.Thedemandfortheselampsgrewby53.3%in2016
comparedtotheyear2012.
Overallconclusionismadethat:thedemandonconsumerand
commoditymarketofEEhouseholdlightinginUkrainewas
stimulateddueto:
-UNDPandGEFinformationcampaignsonEElighting;
-decreaseinretailpricesforhouseholdEElamps,
-increasedinflation,
-reducingthesolvencyofthepopulation,
-raisingtariffstopayforelectricityconsumption,
-highlevelofcompetitioninthemarketofhouseholdEElamps.
TMELTerminalEvaluationReport–31March2017
115
AnnexH:SummaryofLessonsLearnedfromtheRegionalTechnicalAdvisor
TWOMAINLESSONSLEARNED
(1)LessonsLearned#1-Inordernottoreceivedthebeststrategicguidanceandremainontrackthe
projectshouldhavehiredastrongtechnicallyqualifiedinternationalCTAwithregularvisitstoUkraine
(oncepermonth)fromtheoutsetbackin2011or2012.WhathappenedwasthatnointernationalCTA
washireduntiloneyearafterthe2014midtermreview,inearly2015,forthelasttwoyearsofthe
project.TheprojectdidhireastrongtechnicallyqualifiedinternationalCTAbutitwastoolittleandtoo
latewithonlyinfrequentvisitstoUkraineforafewdaysatatimeandverylittletimetogo.Asaresult
activitiesthatshouldhavehappenedatthestartoftheproject(e.g-interlabtesting)happenedrightat
theendoftheprojectandadaptivemanagementhappenedonlyverylateinthisproject.Thelesson
learnedhereisclear.AllUNDPGEFprojectsneedstrongtechnicallyqualifiedinternationalCTAonboard
fromthestartwithregularvisits.
(2)LessonLearned#2-theprojectteamonthisprojectwasclearlytoolargeasidentifiedbythemid-
termreview(butnotfixed)andclearlyhavingmultiplecomponentleadersorfulltimestaffwhoprepare
ToRsandsuperviseworkswithoutactuallyimplementingtheworksorperformingtechnicalanalysis
themselvesisnotwhattheprojectdocumenthadenvisagedanditleadstoaveryhighoverallproject
managementcostwhichwasalsounforeseen.Thelessonlearnedhereisthatthefull-timeprojectteam
needstobesmallerandmorecompact(saymaximum4to5fulltimestaffincludingtheproject
manager)andrelymoreonexpertsandshorttermconsultantssuchasaninternationalCTA,tohelpthe
projectandallprojectteamneedstobeselectedbyandreporttotheprojectmanager.Havingstaffon
theprojectwhodonotreporttotheProjectManagerandstaffwhoreporttoanotherpersononthe
projectwhoisnottheProjectManagerisnotagoodapproachandshouldbeavoidedinfutureatall
costs.