terminal evaluation report of pakstran project 11-05-2017 · brt bus rapid transit cd component...

Terminal Evaluation Report of Pakistan Sustainable Transport Project

1

TerminalEvaluationReportofPakistanSustainableTransportProject(PAKSTRAN)

UNDPProjectID:3953,GEFProjectID:3539

PreparedBy

NisarAhmadKhanIndependentConsultant

January2017

Commissionedby

UnitedNationsDevelopmentProgramme(UNDP)PakistanTABLEOFCONTENTSACRONYMS…………………………………………………………………………………………………………………………….4


2

EXECUTIVESUMMARY…………………………………………………………………………………………………………….5Projectoverallsummary……………………………………………………………………………………………………......5ProjectDescription…………………………………………………………………………………………………………………5ProjectRatings……………………………………………………………………………………………………………………….6Summaryofconclusions,recommendations……………………………………………………………………………61.INTRODUCTION………………………………………………………………………………………………………………….101.1Purposeoftheevaluation…………………………………………………………………………………………........101.2Scope&Methodology……………………………………………………………………………………………………...102.PROJECTDESCRIPTIONANDDEVELOPMENTCONTEXT…………………………………………………….....122.1Projectstartandduration………………………………………………………………………………………………..122.2Problemsthattheprojectsoughttoaddress…………………………………………………………………….122.3Immediateanddevelopmentobjectivesoftheproject………………………………………….............122.4Mainstakeholders……………………………………………………………………………………………………………133.FINDINGSOFTHEEVALUATIONEXERCISE…………………………………………………………………...........143.1ProjectDesign/Formulation………………………………………………………………………………………........143.1.1AnalysisofLFA/ResultsFrameworks………………………………………………………………………………143.1.2AssumptionsandRisks………………………………………………………………………………………………….163.1.3Lessonsfromotherrelevantprojectsincorporatedintoprojectdesign…………………………..163.1.4Replicationapproach…………………………………………………………………………………………………….173.1.5UNDPComparativeAdvantage………………………………………………………………………………………173.1.6Linkagesbetweenprojectandotherinterventionswithinthesector……………………………..173.1.4Managementarrangements………………………………………………………………………………………….183.2ProjectImplementation……………………………………………………………………………………………………203.2.1Adaptivemanagement…………………………………………………………………………………………………..203.2.2ProjectFinanceandExpenditures…………………………………………………………………...................213.2.3Monitoringandevaluation…………………………………………………………………………………………….233.2.4Overallprojectimplementation/execution,coordination,andoperationalissues…………..233.3ProjectResults………………………………………………………………………………………………………………….253.3.1OverallResults/Outcomes……………………………………………………………………………………………..253.3.2RelevanceofProjectinterventionsandresults………………………………………………………………303.3.3EfficiencyofProjectInterventions………………………………………………………………………………….323.3.4CountryOwnership……………………………………………………………………………………………………….363.3.5Mainstreaming……………………………………………………………………………………………………………..373.3.6EffectivenessofProjectResults………………………………………………………………………………………373.3.7SustainabilityofProjectInterventionsandResults…………………………………………………………413.3.8ImpactofProjectInterventionsandResults…………………………………………………………………..424.MAINCONCLUSIONSANDRECOMMENDATIONS………………………………………………………………..44AnnexesAnnex-1:Listofpersonsmet…………………………………………………………………………………………………..47Annex-2:ToR………………….………………………………………………………………………………………………….....48Annex-3:SummaryofFieldVisits…………………………………………………………………………………………..55Annex4:ListOfDocumentsReviewed……………………………………………………………………………………55Annex-5:EvaluationMatrix…………………………………………………………………………………………………....56


3

Annex-6:EvaluationQuestionsforTEofPASTRAN…………………………………………………………………57Annex-7:RatingScales..………………………………………………………………………………………………………....59Annex-8:EvaluationConsultantAgreementForm…………………………………………………………………..60Annex-9:EvaluationReportClearanceForm…………………………………………………………………………..61ACRONYMS


4

BAU BusinessasusualBRT BusRapidTransitCD ComponentDirectorCESTAC CentreforexcellenceinsustainabletransportandclimatechangeCIU ComponentImplementationUnitCM ComponentManagerCP CountryProgrammeCPAP CountryProgrammeActionPlanCPEC ChinaPakistanEconomicCorridorEAD EconomicAffairsDivisionENERCON NationalEnergyConservationCentre,MinistryofWater&PowerFJWU FatimaJinnahWomenUniversityRawalpindiGCF GreenClimateFundGEF GlobalEnvironmentFacilityGHG GreenhouseGasGoP GovernmentofPakistanGoPb PunjabProvincialGovernmentIP ImplementingPartnerIUCN InternationalUnionfortheConservationofNatureLoA LetterofAgreementM&E MonitoringandEvaluationMoC MinistryofCommunicationsMoW&P MinistryofWater&PowerNED NadirshawEduljiDinshawUniversityofEngineeringandTechnology,KarachiNIM NationalImplementationModalityNPDNPM

NationalProjectDirectorNationalProjectManager

NTRC NationalTransportResearchCentrePAKSTRANPCOMPB

PakistanSustainableTransportprojectProjectCycleandOperationsManualProjectBoard

PMU ProjectManagementUnitPro-Doc UNDPProjectDocumentR&D ResearchandDevelopmentRP ResponsiblePartySTD SindhProvincialTransportDepartmentTE TerminalEvaluationToR TermsofReferenceToR TermsofReferenceUET UniversityofEngineeringandTechnologyLahoreUNDP UnitedNationsDevelopmentProgrammeUU UrbanUnit,Planning&DevelopmentDepartment,PunjabEXECUTIVESUMMARY ProjectSummaryTable


5

ProjectDescriptionPakistanSustainableTransport(PAKSTRAN)projectwasaninitiativeofUNDP,theGEFandtheGovernment of Pakistan that aimed to provide technical assistance to reduce the growth ofenergyconsumptionandrelatedgreenhousegas(GHG)emissionsfromthetransportsectorinPakistan, while simultaneously improving urban environmental conditions and improvingPakistan’s trade competitiveness. The project’s original implementation timeframewas from2010to2015,however,actual implementationstarted in2012and itstermwasextendedbyoneyeartillDecember2016.TheglobalobjectiveofPAKSTRANistoreducetheGHGemissionsfromthetransportsectorinPakistan. The developmental objectives of PAKSTRAN are: to improve urban environmentalconditions(i.e.improvedairquality,urbanmobility,equityandcityaesthetics);andtoimproveenergy security for Pakistan. Moreover, the project has four components with specificoutcomes,asoutlinedinthefollowing:Outcome1:AnoperationalsustainableurbantransportsysteminPunjabprovince(PunjabP&DDepartmentistheResponsibleParty-RPtoachievethisoutcome);


6

Outcome 2: An operational sustainable urban transport system in Sindh province (SindhTransportDepartmentistheResponsibleParty-RPtoachievethisoutcome);Outcome3: Improvedfuelefficiencyintruckfreighttransport(MinistryofCommunications istheResponsibleParty-RPforthisoutcome;&Outcome 4: Increased public awareness and institutional capacity on sustainable transportconcepts(IUCN-PakistanistheResponsibleParty-RPforthisoutcome)

PleaseseedetailsofratingscalesasAnnex-4

SummaryofConclusionsandRecommendationsBased on the detailed analysis and findings of the evaluation exercise following are thesummaryconclusionsandrecommendations:a)OverallRelevanceandContinuityofTechnicalSupportPAKSTRANprojectwashighlyrelevantandinstrumentalinbringingforthandaddressingissuesof sustainable transportandprovided technical support in implementationofBRTsystems inlargeurban centers.However, the issuesof the transport sector are very complex, timeandresourceconsuming.Ontheotherhand,itisalsoimportanttohighlightthatthetruckingsectoralso remains one of the neglected sectors. The project has handsomely contributed to itspromotion,buttheworkhasjuststartedandthereisalongroadahead.Recommendation1: It isrecommendedthattechnicalandcapacitybuildingsupporttourbantransport (BRT)and trucking sector should continue. In this regard soundconceptpapers forfutureprojectsshouldbepreparedandsharedwithrelevantdonorsandinstitutionslikeGreenClimate Fund (GCF) etc. to secure necessary funding. For the trucking sector, possibilities of

ProjectEvaluationRatingTable


7

securingfundsfromChinaPakistanEconomicCorridorProgrammeandprivatesectoralsoneedtobeexplored.b)ProjectDesignandResultsFrameworkPAKSTRANProjectwasoriginallydesignedin2010,withtotalbudgetofaroundUSD78Million,includingUSD 70Million from theGovernment and other partners. However, the stipulatedcontributionsdidnotmaterializeandtheprojectwasimplementedwithbudgetaryresourcesofUSD 7.8 Million, provided by GEF and UNDP. In addition, during project implementationcircumstances of the urban transport sector also kept continuously changing. Overall, theoriginalprojectresultsframeworkwaswellformulatedandexhibitedclearlinkagesamongsub-outputs, outputs and outcomes. However, the changes in budgetary resources andcircumstanceshaditsownimplicationsforprojectdesignandachievementofprojectoutputsandoutcomes.Theproject’sResultsFrameworkwasaccordinglyrevisedandadjusted,buttheoutcomesandoutputsremainedthesame.Recommendation2:Itisrecommendedthatfuturesuchprojectsneedtobedesignedusingatheoryofchangeapproachthroughrigorouslyinvolvingallstakeholdersinallstagesofprojectformulation. Stakeholders’ financial commitments and roles and obligations also need to beclearlydefinedandagreeduponwell inadvance.Projectdesignshouldalsoduly incorporatetheelementsofflexibilitytoallowfordesiredchangesduringimplementation.c)OverallProjectEfficiency,ManagementandImplementationGiventhecomplexityofcircumstances,theprojecthasmaderigorouseffortsandconsiderableprogresstowardsachievingitsoutputsandoutcomes.Asatechnicalassistanceproject,projectinterventionsweremainlygearedtowardsprovidingdesiredsupport intheareasoftechnicalresearch,capacitybuilding,coordinationandawarenessraisingetc.However,theprojectalsofacedanumberofmanagementandimplementationissueswhichconsiderablydelayedprojectstartandimplementationofactivities.Main issues included changes in theproject’s original ImplementingPartner andResponsibleParties,frequentturnoveroftheNationalProjectDirector(NPD)andprojectstaff,lackofclarityand understanding of execution and organizational arrangements, lack ofcoordination/understandingespeciallybetweenIPandRPinPunjab,latereleasesoffundsandcumbersomeworkplanning,managementandapprovalprocesses.Recommendation 3: It is recommended that in future such projects, partners should beselected carefully keeping in view their relevance, expertise, interest and commitment and,their roles and obligations should be very clearly defined and agreed upon. Similarly,agreement/contracts(LOAs)withallpartnersshouldalsobesignedbeforeprojectinception.Recommendation 4: It is recommended that in the timeframes of future such projects,adequate time should be designated for formation of governing bodies/structures,establishmentofoffices,recruitmentofstaff,procurementofgoodsandservices,formulationof necessary implementation processes and procedures. Needless to emphasize that project


8

schedules should also provide necessary allowance and flexibility for unforeseen delays androadblocks.Recommendation5: It isalsorecommendedthat insuchprojects,withmatrixorganizationalstructures, where multiple and independent partners are involved, coordination amongstakeholders should be further strengthened. Regular six-monthly and annual review andcoordination meetings/workshops can greatly help in improving coordination amongstakeholders.Ifneeded,technicalcommitteesshouldbeestablishedatthecomponentleveltofurtherfacilitatecoordinationamongstakeholders.Recommendation 6: It is recommended that UNDP project management and operationalprocesses likeworkplanning, recruitment,procurement,monitoringandevaluation,progressreporting and fund releases etc. need to be further simplified and made more swift andefficient.Moreover,UNDPshoulddulytrainandbuildthecapacitiesofstakeholdersinUNDP-Project Cycle and Operations Manual (PCOM) procedures at the time of project start andprovideperiodicrefreshers.d)EffectivenessofProjectResultsThrough implementation of awide range of interventions, project contributed to achieve itsfour outcomes related to sustainable urban transport in Punjab and Sindh, fuel efficiency intruck freight transport and awareness raising. Support was provided in development and/oroperationalizationofBRTsystemsinthecitiesofLahoreandRawalpindi-IslamabadandKarachi.Researchandpolicyworks, institutionalcapacitybuildingandawarenessraisingremainedthehallmarkinterventions.Inaddition,sustainabletransportresearchcenterswereestablishedatFatima Jinnah Women University (FJWU) Rawalpindi and University of Engineering andTechnology(UET)Lahore,tocaterforfutureneeds.Recommendation 7: It is recommended that project research works and all knowledgeproductsshouldbewidelysharedanddisseminatedtorelevantstakeholdersfortheirreferenceanduse.Presently these resourcesareavailableonlineonPAKSTRANwebsite,howeveraftertheterminationofprojecteitherthesiteneedstobemaintainedorthesevaluableresourcesshouldbemovedeithertoIUCNorUNDPwebsitetomakethemavailablefortimestocome.Recommendation 8: As mentioned, the project has contributed handsomely to transportrelatedpolicywork.However,thereisalotmoretobedone,thereforeit isrecommendedtorigorously peruse Sindh Transport Policy approval process and finalization of strategies fortruckingpolicy2007.Similarly,developmentoftransportpoliciesforotherprovincesalsoneedstobedulyconsideredandprioritized.e)SustainabilityofprojectinterventionsandbenefitsIn view of the high level of relevance and demand and keen interest, will and significantinvestmentsoftheFederalandrespectiveProvincialGovernmentsinBRTinLahore,Islamabad,Rawalpindi and Karachi it can be easily concluded that the completed and forthcoming BRTsystemswillbeadequatelysustainedandthebenefitswillcontinuetoflow.Similarly,thepolicy


9

works also enjoy high level of ownership from the Government. However, in the wake ofupcoming BRT systems in other cities, there is still a good demand and need for projectinterventions related to research work, capacity building and awareness raising in times tocome. Though it is not very clear that in the absence of PAKSTRAN support howwill theseinterventionsbefinanced.Recommendation9:ProjecthasestablishedtwosustainabletransportresearchcentersatFJWUniversityRawalpindiandUETLahore,tocaterforfutureresearchneeds.Thesecentersarestillinveryearlystageofdevelopment.Therefore, it isstronglyrecommendedthatthesecentersshouldbesupportedtechnicallyandfinanciallyintheshorterruntomakethemproductiveandsustainable in the longer run. Furthermore, these centers need to be connected to otherdonors,institutionsandindustryforsustainabilitypurposes.f)OverallImpactofProjectPAKSTRAN project overall outcome was to reduce GHG emissions associated with urbantransportation.Estimatesfrom2016projectstudyongreenhousegasemissionsstatusbeforeandaftermetrobusproject implementationalongtheMetroBusCorridorRWP-ISB,suggeststhatthenetreductioninCO2fromRWP-ISBMetroBusisaround4,486.76tonsannually.For20yearBAUscenarioitcomestoaround89.7ktonnes.IfthesameestimatesareappliedtoLahoreMetro, then the assumed cumulative total reduction in CO2will be around179.4 ktonnes ascomparedtoBAUscenarioovera20-yearperiod. It is importanttohighlightthattheoriginalprojecttargetofCO2reduction,fromthetwoprojectenvisionedBRTdemonstrations,was608ktonnes for 20 year BAU. Results of the mentioned study shows that project targets werecalculatedtooambitiously.Recommendation 10: It is recommended that in future such projects, targets related tooutcome and impact indicators need to be realistically determined and should be based onsoundcalculationsandrealisticestimation,keepinginviewitsachievabilitystatusandeaseofmeasurability.1. INTRODUCTION


10

1.1PurposeofTerminalEvaluationAs outlined in the ToR, the main objectives of the terminal evaluation are to assess theachievementsofprojectresults,andtodrawlessonsthatcanbothimprovethesustainabilityofbenefitsfromthisproject,andaidintheoverallenhancementtoUNDPprogramming.1.2Scope&MethodologyIn view of the objectives, scope and duration of the Terminal Evaluation, a semi structuredmixedmethodapproachhasbeenadoptedusingmainlyqualitativedatacollectionandanalysismethodsandtools.Theevaluationprocesshasbeenconductedinlinewiththeguidance,rulesandproceduresestablishedbyUNDPandGEFasreflectedintheUNDPEvaluationGuidanceforTerminalEvaluationofGEFFinancedProjects.Broadlytheoverallevaluationprocessconsistsoffive standard steps i.e. 1) Evaluation Questions, 2) Evaluation Design, 3) Data CollectionMethods, 4) Data Analysis and 5) Presentation and Reporting. Following is the summary ofproposedevaluationmethodology:

a) MainEvaluationCriteriaandQuestionsIn linewith theToRandGuidelines forConductingTerminalEvaluationsofUNDP-Supported,GEF-Financed Projects, the TE adhered to the standard assessment criteria of Relevance,Effectiveness, Efficiency, Impact and Sustainability to assess the overall progress andperformanceoftheproject.Followingisthesummaryofmainquestionsrelatedtoassessmentcriteria:• Relevance:HowdoestheprojectrelatetothemainobjectivesoftheGEFfocalarea,andto

theenvironmentanddevelopmentprioritiesatthelocal,regionalandnationallevels?• Effectiveness: Towhat extent have the expected outcomes and objectives of the project

beenachieved?• Efficiency:Wastheprojectimplementedefficiently,inlinewithNationalandinternational

normsandstandards?• Sustainability: To what extent are there financial, institutional, social-economic, and/or

environmentalriskstosustaininglong-termprojectresults?• Impact: Are there indications that the project has contributed to, or enabled progress

toward,reducedenvironmentalstressand/orimprovedecologicalstatus?In addition to the above evaluation criteria, efforts have also been made to assess projectdesign, results frameworks, implementationarrangements,project financing/co-financingandoverallmainstreamingetc.


11

b) EvaluationCriteria&RatingsIn line with UNDP and GEF terminal evaluation guidelines, various project performancedimensions and criteria were rated in accordance with the obligatory rating scales. Thesedimensions include project design, implementation and execution, M&E mechanisms andrelevance, effectiveness, efficiency and sustainability of project outcomes and interventions.The completed table is included in the executive summary. The obligatory rating scales areoutlinedinAnnex-2.c) Datacollectionmethods/tools• DeskReviewofofficialrecordsanddocumentsAgooddealof relevance,efficiency,effectiveness, impactand sustainability relateddatahasbeenobtainedfromreviewofprojectdocuments,officialrecordsandsecondarysources.TheseincludebutarenotlimitedtotheProjectDocument,ProjectProgressReportsincludingAnnualProjectReview/PIRs,AnnualWorkPlans,FinancialReports,ProjectBudgetRevisions,MidtermReview Report, Technical Reports/Publications, Project Board Meeting Minutes, NationalStrategicandLegalDocuments,andsecondarysourcesetc.• KeyInformantsinterviewsandfocusgroupdiscussionsKeyinformants interviewsremainedtheprimarysourceof information.Relevantrespondentsamong project partners were identified in consultation with UNDP and project staff andmeetings were held with concerned partners and project staff in Islamabad, Lahore andKarachi.Detaileddiscussionswereheldkeepinginviewvariousoutlinedevaluationquestions.PleaseseelistofpeoplemetasAnnex1.d) DataAnalysis,PresentationandReportingIn view of the nature of evaluation questions and use of qualitative assessment approach,collected data has been processed using validations, triangulations, interpretations andabstractions techniques and on the other hand, where applicable, quantitative data wasanalyzedusingsimplestatisticalmethodstodetermineprogressandtrends.Thedetailed findingsof theevaluationexercisehavebeenoutlined in this EvaluationReportusingprescribed formatasoutlined in theToR. Inaddition tomajor findings, the reportalsoprovidesoverallconclusions,lessonslearntandspecificrecommendations.Apresentationwasheldsoonafterfurnishingofthedraftreport.Accordingly,theReportwasfinalizedafterdulyaddressing and accommodating the comments received on the Draft Report and during thepresentation.2.PROJECTDESCRIPTIONANDDEVELOPMENTCONTEXT2.1Projectstartandduration


12

According to project document, the proposed implementation timeframe of PakistanSustainableTransport (PAKSTRAN)projectwas for fiveyears i.e. from1stOctober2010 to31December 2015. However, the project document was formally signed by the GovernmentaroundMarch 2011. Projectmobilization and identification and arrangements with partnerstook further time and project inception workshop was organized inMarch 2012. Down theroad,establishmentofPMUsandCIUs,organizationofprojectboard1stmeeting,recruitmentofproject staff and implementationarrangementswithpartnersalso consumedconsiderabletime.Finally, from2013onwardsproject interventionswere implementedwith full steamtillprojectclosureon31December2016.2.2ProblemsthattheprojectsoughttoaddressTheprojectmainly intendedtoaddress theproblemsof increasinggreenhousegasemissionsarisingfromgrowingurbantransportcongestionespeciallyinbigcities.ItwashighlightedthatPakistanisfacedwithrisingurbanizationandincreaseinprivateownershipofmotorvehicles,whichhaveadirectbearingonsustainabledevelopment intermsofdeteriorating livabilityofurbanareasandconstrainedeconomicgrowthbyurbantransportinefficiencies.Furthermore,theurbantransportcongestionhasbeennegativelyimpactingthepoorandlowincomeclasseswholargelyrelyonpublictransport.Itwasalsonotedthatatthetimeofprojectdesign,therewasnoorganizedpublicBusTransitsystemsavailableinanyofthebigcities.On the other hand, the project also intended to address the issues of fuel inefficiency incommercialvehiclesespeciallyoldandoutdatedtrucks,whichhavebeenputtingheavyburdenon foreign exchange reserves due to increased and inefficient consumptionof imported fuelandarealsosourceofexcessiveGHGemissions.Barrier analysis in the project document also highlighted a number of contributing issuesincludinglackofsustainedinstitutionalcoordinationforpublictransportdevelopment,absenceof a comprehensive urban transport policy, lack of capacities andplanning for urban transit,weaktrafficmanagement,lackofmoderntransportvehiclesandlowawarenessofsustainableurbantransport.2.3ImmediateanddevelopmentobjectivesoftheprojectPakistan Sustainable Transport (PAKSTRAN) project aimed to provide technical assistance toreducethegrowthofenergyconsumptionandrelatedgreenhousegas (GHG)emissions fromtransport sector in Pakistan,while simultaneously improvingurbanenvironmental conditionsandimprovingPakistan’stradecompetitiveness.The global objective of PAKSTRAN is to reduce the GHG emissions from transport sector inPakistan. The developmental objectives of PAKSTRAN are: to improve urban environmentalconditions i.e. improved air quality, urban mobility, and equity and city aesthetics; and toimproveenergysecurityforPakistan.Moreover,theprojecthasfourcomponentswithspecificoutputs/results,asoutlinedinthefollowing:Component1:AnoperationalsustainableurbantransportsysteminPunjabprovince


13

Component2:AnoperationalsustainableurbantransportsysteminSindhprovinceComponent3:ImprovedfuelefficiencyintruckfreighttransportComponent4: Increasedpublic awarenessand institutional capacityon sustainable transportconcepts2.4MainstakeholdersThemainstakeholdersforthePAKSTRANprojectinclude:• UnitedNationsDevelopmentProgramme(UNDP)PakistanandGlobalEnvironmentFacility

(GEF) – Financiers and Overseers. UNDP also provided needed technical and projectmanagementsupport.

• TheMinistryofWaterandPower,GovernmentofPakistanthroughENERCON(theNationalEnergyConservationCentre)–ImplementingPartnerattheFederalLevel

• The Planning and Development Department (P&DD), Government of Punjab Province,throughTheUrbanUnit(UU)–ResponsiblePartyforPunjab(Component-1)

• The TransportDepartment (STD),Government of Sindh Province–Responsible Party forSindh(Component-2)

• TheMinistryofCommunication,GovernmentofPakistanthroughtheNationalTransport• ResearchCentre(NTRC)–ResponsiblePartyforSindh(Component-3)• InternationalUnionforConservationofNature(IUCN)–ResponsiblePartyfor(Component

-4)In addition to the above main partners, a number of relevant stakeholders were engagedduring implementation of project interventions which includes but not limited to CapitalDevelopmentAuthority,PunjabTransportDeportment,TrafficPolice,KarachiMassTransitCell,and Transport Associations etc. It is important to highlight that project has established verygood collaboration and remained closely engaged with a number of reputable academicinstitutions including: NUST University Islamabad, UET Lahore, Fatima Jinnah WomenUniversity,Rawalpindi,NEDUniversityKarachi, SindhUniversity, Jamshoro,BahriaUniversity,Islamabad,Quaid-e-AzamUniversity,Islamabad,CUSTUniversity,Islamabad,IslamicUniversity,IslamabadandAridUniversity,Rawalpindi.3.FINDINGSOFTHEEVALUATIONEXERCISE3.1ProjectDesign/Formulation3.1.1AnalysisofLFA/ResultsFramework


14

AdetailedprojectLogicalFrameworkAnalysis(LFA)/ResultsFrameworkwasformulatedatthetime of project design consisting of overall UNDP Country Programme outcomes, projectspecificoutcomesandcomponentoutputsandsub-outputsrelatedtofourcomponents.TheRFalsoprovidesoutcomesandoutputlevelIndicators,Baselines,Targets,DataSourcesandRisksand Assumptions. Analysis suggests that the original results frameworkwaswell formulatedandexhibitedclearlinkagesamongsub-outputs,outputsandoutcomes.ItisimportanttonotethattheResultsFrameworkwasformulatedkeepinginviewtheoriginalprojectbudgetaryresourcesofaroundUSD78Million,whichincludedalargeshareofUSD64Million from Government of Pakistan and another 6 Million contribution from otherinternational agencies likeWorld Bank and JICA etc. However, down the road the stipulatedcontributions from Government and other donors did not materialize and thus the projectspecificbudgetaryresourceswereconstrainedonlytoUSD7.8Million,whichincluded4.8and3.0MillionasGEFandUNDPsharerespectively.Itisimportanttoquestionthatwhatcouldbetheimplicationsofthesesubstantialchangesinavailability of project resources on the project design and especially achievability status ofprojectoutcomesandoutputstargets.AnalysisofResultsFrameworksuggestthattheproject’smainoutcomesofreductioninGHGemissionsfromurbantransportandtruckingsectorwerelargelydependentonthedevelopmentandoperationalizationoftwoBRTdemonstrations,oneeach in Lahore andKarachi, andpilots formodernizationof trucks.Discussion and reviewofrecordssuggestthattheproposedBRTdemonstrationsandpilotsformodernizationoftruckscouldnotbeimplementedassuchasenvisagedbytheprojectdesign.However,theGovernmentofPunjabwasverykeentoimplementalargescaleBRTprojecti.e.the Lahore Metro Bus. The Metro Bus project was successfully completed and madeoperational in early 2013. In view of the success of the Lahore Metro, the interest ofGovernment in BRT further enhanced and another state-of-the-art Metro Bus Service wassuccessfully completed and operationalized in mid-2015 in the cities of Rawalpindi andIslamabad.DuetothesetwolargescaleBRTprojectsofGovernmentofPakistan,theneedforBRT demonstration proposed under PAKSTRAN project in Lahore was diminished and therequiredprojectoutcomesofreductioninGHGemissionswouldbefulfilledbythesetwoBRTprojectsinLahoreandIslamabad-Rawalpindi.InKarachi,workonproposedOrangeandGreenLinesofMetroBusisinprogressandwillhelpinreductioninGHGemissionsoncecompleted.It can be concluded that circumstances during project implementation kept continuouslychanging,whichposed greater challenges for theoriginal design and results framework. TheprojecttookthenecessarymeasurestoreviseandadjustitsResultsFrameworkinMarch2014withtheapprovalofProjectboard.Effortsweremadetoadjustprojectindicatorsandtargets,however the broader outcomes and outputs remained the same, posing challenges ofattributionandcontributionforthepurposeofmeasurability.

ProjectResultsFramework


15

3.1.2AssumptionsandRisksThe revised Results Framework of the project outlines a number of assumptions for thesuccessful implementation of the project and achievability of results. Major assumptionsincluded:


16

1.Reliabledataobtainedfromcarryingoutphysicalsurveysrelatedtoridershipswitchingfromcartopublictransport;Inthisregard,theprojectcompletedastudyonestimationofgreenhousegasemissionsstatusbefore and after metro bus project implementation along the Metro Bus CorridorRawalpindi/Islamabadusingsurveyrelatedtoridershipswitchingfromcartopublictransport.2.FirmandcontinuedsupportfromGovernmentandallrelatedstakeholdersinimplementationandmonitoringandevaluation;Overall support from Ministry of Water and Power, Ministry of Communication, SindhTransport Department, various academic institutions, IUCN and other relevant stakeholdersremained forthcoming and productive. However, at times, collaborationswithUrbanUnit ofGovernmentofPunjab remainedconstraineddue todifferences inunderstandingsofprojectinterventionsandimplementationmodalities.3.BRTdesignandconstructionstandardsareadoptedandimplementedbytheGovernmentofPunjabintrueletterandspirit;This assumptionwasmade in the contextof projectproposedBRTdemonstration.However,since Government of Punjab has already adhered to high standards in the design andconstruction of Lahore and Rawalpindi Metro Bus service, therefore this assumption issufficientlytakencareof.4.Private-publicpartnerships(PPPs)supportfromGovernmentandallrelatedstakeholders;This assumption was made in the context of modernization of trucks. Discussions withstakeholders suggest that so far very limited progress has beenmade to secure any PPP fortruckingsectors.3.1.3LessonsfromotherrelevantprojectsincorporatedintoprojectdesignPAKSTRANprojectwasuniquein itskindandtheprojectdesigndrawsfromanumberofBRTandtruckingrelatedstudiesand initiativescarriedout inthepast.Theprojectdocumenthadhighlighted that given the history of failed initiatives to develop and implement sustainabletransport in Pakistan, the project designwill focus on demonstrating best practices that areapplicable to the Pakistani urban concept and take into account the lessons learned andexperiences from previous initiatives in developing and implementing BRT systems and tomodernize trucking sector. The main lessons drawn from previous experiences andincorporated intoprojectdesignwere related toadoptionofaholisticplanningapproach forBRT, streamliningof institutionalmechanismsandpolicy frameworks, holistic approaches forimplementationoftruckingpolicy,raiseawarenessandknowledgelevelsandsustainabilityofinterventionsandbenefits.3.1.4ReplicationapproachSincesuccessfulcompletionandoperationalizationofMetroBusServiceinLahore,Rawalpindiand Islamabad,NationalandProvincialGovernmentsareverykeentoextendBRTsystemstoothermajorcities likeMultan,KarachiandPeshawar.WorkonGreenandOrangeLineMetro


17

Bus inKarachi is inprogressand isexpectedtobecompletedby2018.Similarly,planningforBRT in Peshawar is also in progress. Overall, it can be concluded that demand andgovernmentalwillforestablishingBRTsystemsinurbancentersisgainingmomentumanditisexpectedthattheBRTwillbegraduallyreplicatedintimestocome.Regarding trucking component, work on truckmodernization pilots could not be expedited.However,inviewoftheincreasedimportanceofthetruckingsectorduetotheChinaPakistanEconomic Corridor, it is expected that good practices from the project will be replicated intimestocome.3.1.5UNDPComparativeAdvantageOverall,UNDPhasplayedanimportantroleinfacilitation,coordinationandimplementationofPAKSTRAN Project. Discussions with stakeholders suggest that they highly valued theircollaborationwithUNDP.Overall UNDP enjoys very sound reputationwith stakeholders andespeciallywith governmental institutionswhichhas greatlyhelped in facilitationof access topartnersandimplementationoftheproject interventions.UNDPcloseinteractionwithallthefederalandprovincialgovernmententitieswasfoundveryinstrumentaltostreamlineactions,boostdeliveryandensureownershipofthegovernment.UNDPremainedoneoftheco-financersandprovidedashareofprojectfinancialresources.Theproject was implemented using UNDP’s National Implementation (NIM) modality andaccordinglyallneededadministrative,operational,technicalandcapacitybuildingsupportwasprovidedbyUNDPduringimplementation.UNDPalsodiligentlyexerciseditsroleinthequalityassurance andmonitoring and evaluation of the project interventions and results, includingregularprogressreportingandmid-termevaluationetc.ItisimportanttohighlightthatUNDPplayedan important role in facilitation toaddress issues related toprojectmanagementandimplementation.3.1.6LinkagesbetweenprojectandotherinterventionswithinthesectorFederalandProvincialGovernmentshavebeenverykeenandinterestedinimplementinglargescale BRT systems in the major cities of Pakistan and have recently completed andoperationalizedMetroBusSystemsinLahoreandRawalpindi-Islamabad.FurthermoreworkonOrangeLineandGreenLineMetroBus inKarachi isalsounderwaywith full steam.PlansarealsobeingpreparedforextendingMetroBusSystemstoothercitieslikeMultan,FaisalabadandPeshawaretc.InthewakeofthesemegaMetroBusProjects,theprojectmadestrenuouseffortstoadjustitsdesignand interventionsandhasestablished linkageswith relevant institutionsandprovidedtechnicalsupportduringandaftercompletionofMetroBusSystemsinLahoreandIslamabadand forthcoming Metro Bus Lines in Karachi. Accordingly, in trucking sector Project alsoestablished linkages with relevant academic and governmental institutions to promote fuelefficiency andmodernization of trucks in thewake of enhanced interests of Government ofPakistanduetoChinaPakistanEconomicCorridor(CPEC).


18

3.1.7ManagementarrangementsPAKSTRANprojectwasimplementedusingNationalImplementation(NIM)Modality.Followingisthesummarydescriptionofmainimplementingandresponsiblepartners:• UNDP Pakistan and GEF, provided overall funding, technical assistance, oversight and

supporttoprojectmanagementandoverallcoordination.• TheMinistryofWaterandPower,GovernmentofPakistan throughENERCON, remained

the ImplementingPartnerat thenational levelanddirected, facilitatedandcoordinatedoverallprojectimplementationthroughtheProjectManagementUnit,headedbyNationalProjectDirector

• PlanningandDevelopmentDepartment(P&DD),GovernmentofPunjabProvince,throughTheUrbanUnit(UU),remainedresponsiblepartyforComponent-1anddirected,facilitatedand coordinated overall project implementation in Punjab through ComponentImplementationUnit(CIU),headedbyComponentDirector

• SindhTransportDepartment (STD),GovernmentofSindhProvince, remainedResponsibleParty for Component-2 and directed, facilitated and coordinated overall projectimplementation in Sindh through Component Implementation Unit (CIU), headed byComponentDirector.

• TheMinistryofCommunication,GovernmentofPakistan,remainedResponsiblePartyforComponent-3anddirected,facilitatedandcoordinatedoverallprojectimplementationfortrucking sector through Component Implementation Unit (CIU), headed by ComponentDirector.

• International Union for Conservation of Nature (IUCN), remained Responsible Party forComponent -4 and directed, facilitated and coordinated overall project implementationregardingawarenessandcapacitybuildingthroughComponentImplementationUnit(CIU),headedbyComponentDirector.

PAKSTRAN project was guided, advised and overseen by a Project Board (PB), chaired bySecretary, Ministry of Water and Power and members including senior officials of UNDP,PlanningCommissionofPakistan,EconomicAffairsDivision,EnvironmentalProtectionAgency,MinistryofCommunication,HDIP,PMUandrespectiveCIUsetc.ReviewofPBmeetingminutessuggests that 1st PBmeetingwas organized on 4thOctober 2012, after a considerable delaysince initiationofproject,howeverafterwardsPBmeetingswereheld regularlymostlyonanannualbasisanddiscussedindetailtheprogressofprojectinterventionsandsimilarlyreviewedand approved annual work plans. PB also provided strategic advice and oversight and wasinstrumental in revising the project results framework. (Please see PAKSTRAN organizationalstructurebelow)

OrganizationalStructureofPAKSTRANProject


19

Theprojectwasimplementedthroughamulti-tieredandmatrixorganizationalsetupinvolvinga number of partners and project staff and at the national and provincial level. Overall, theprojectwas led byNational ProjectDirector, theAdditional SecretaryMinistry ofWater andPower. A Project Management Unit was established in Islamabad, led by National ProjectManagerandconsistingofMonitoringandEvaluationOfficerandadministrativestaff.ThemainfunctionsofPMUincludedoverallcoordination,monitoringandevaluation,progressreporting,facilitation of annual work plans and day-to-day management of the project. Discussionssuggest that operationalization of PMU took considerable time, however afterwards itfunctionedveryefficientlyandnegotiateditsrolesinthebestpossiblemanner.Toimplementprojectinterventionsatthecomponentlevel,ComponentImplementationUnits(CIUs)wereestablished in Lahore (atUrbanUnit), Karachi (at STD), Islamabad (atNTRC) andIslamabad(atIUCN).Overall,therespectivecomponentswereledbyComponentDirectorsi.e.senior officials of respective Responsible Parties. CIUs were led by project appointedComponent Managers and consisted of limited administrative staff. Main functions of CIUsincluded work planning, implementation and monitoring of component interventions andprogressreporting.Discussionswithpartnerssuggestthatinitiallytheorganizationalandexecutionsetupwasnotvery clearly defined. In the project start, CIUs received funds directly from UNDP againstapproved work plans, making them relatively independent of the PMU. In addition, the


20

relationshipamongtheinvolvedMinistriesatthenationallevelandGovernmentDepartmentsat theprovincial levelwasalsonotveryclearespecially the reporting linesand roleofPMU.Matterswere further compoundedby lack of proper agreements or signed contracts amongimplementingresponsibleparties.To streamlineproject implementationand coordination, PMUmade considerableefforts andformulated and secured the signatures on official Letters of Agreement (LoA) betweenImplementing partner and responsible parties. For example, LoA for Component-1 betweenMoWP and P&DD Punjab was signed inMay 2014 and LoA for Component-3 was betweenMoWP and MoC was signed as late as March 2015. These organizational and contractualarrangements consumed considerable time and resulted in delays in implementation ofinterventions.Afterwardsthestreamliningofroles,responsibilitiesandprocessesweregreatlyhelpful in improving implementation of project interventions and coordination amongstakeholders and PMUs and CIUs regarding preparation of work plans, implementation ofprojectactivitiesandmonitoringandreportingofprogress.3.2ProjectImplementation3.2.1AdaptivemanagementSinceprojectdesignandinception,sectoralcircumstancesandgovernmentalprioritiesrelatedto urban transport considerably changed from time to time. In recent years, Government ofPunjab’s interestswere greatly enhanced in implementing large scale BRT project in Lahore.Their efforts culminated in the completion and operationalization of state-of-the-art LahoreMetroBusService inFebruary2013. Inviewof thesuccessofLahoreMetro, the interestsofGovernment of Pakistan in BRT further enhanced and another state-of-the-art Metro BusServicewassuccessfullycompletedandoperationalizedinmid-2015inthecitiesofRawalpindiandIslamabad.InKarachitoo,workonOrangeLineandGreenLineMetroBusisunderwaywithfullsteamandissponsoredbySindhGovernmentandFederalGovernmentrespectively.Review of project documents and results framework suggests that project design did notanticipatethecomingofthese largescaleMetroBusprojects.Therefore, itposedsubstantialchallengesforprojectoutputsandsub-outputsrelatedtoestablishingtwoBRTdemonstrationsin Lahore and Karachi, as in the wake of these Metro Bus Projects, the need for a projectproposed BRT demonstration was already diminished. Therefore, project made strenuousefforts to adjust its interventions to the changing situations and accordingly revised itsindicators and targets in the results framework with the approval of PB in March 2014.Subsequently,ittriedtoadjustitsprojectdesignandinterventionstosupportandcomplementtheongoingMetroBusSystemsinLahoreandIslamabadandforthcomingMetroBusLines inKarachi. Additionally, project also adopted its interventions to the needs of stakeholders intermsofcapacitybuildingandawarenessraising.In addition, the project alsomade necessary adaptations to its initial design regardingmainimplementing partner and responsible party for Component-3. Originally, the project designidentified Ministry of Environment, Government of Pakistan through ENERCON as the mainImplementing Partner. Down the road, the Ministry of Environment was devolved to the


21

TotalResourcesasoutlinedintheProdoc

provincesandsubsequentlyENERCONwasattachedtoMinistryofWaterandPowerandhencetheMinistryofWaterandPowerbecamethenewImplementingPartner.Ontheotherhand,accordingtoinitialdesign,theMinistryofIndustrieswasidentifiedasResponsiblePartyfortheComponent-3 related to trucking. However, later it was replaced by the Ministry ofCommunicationasResponsiblePartyforthementionedcomponent.3.2.2ProjectFinanceandExpendituresAccording to the original Projectdocument, the total proposed budgetaryresources of the project were around USD78.0Million, includinga largeshareofUSD64.0 Million from Government of Pakistanand around 6 Million from otherinternational agencies and 7.8Million fromGEFandUNDP.However,downtheroadthestipulated contributions from Governmentand other donors did not materialize andthus the project specific budgetaryresourceswereconstrainedonlytoGEFandUNDPfinancesi.e.USD7.8Million.Discussion with stakeholders suggests thatitwasnotveryclearhowthese largesumsof contributions, especially fromgovernment,werederived at and includedin the overall project resource frameworkand why these resources did notmaterialize for project use. In a nutshell,the project had to rely only on theresourcescommittedandprovidedbyUNDPandGEF.


22

Accordingtosummaryfinancialstatements,as of December 2016, project has utilizedaroundUSD5.4Million i.e.70%of its totalcommitted resources of USD 7.8 Million.Outofthese,around4.7Millionwerespentfrom GEF funding and the rest 0.7Millionwere spent from UNDP resources. It isimportant to highlight that GEF financingwas up to the mark almost all of itscommitted resources were duly providedandutilized.UNDPfundingremainedlowerascomparedtototalcommitmentandwasallocatedonanannualbasisinlinewiththepaceoftheproject.Overall,thelowerspendingoftheprojectcanbeattributedtotheverylatestartoftheprojectandslowerpaceininitialyears.Component wise expenditures details show that activities under Component -1 consumedaround 33%of the total resources, followed by Component-4 at 22%, Component-2 at 17%,Component-3at15%and13%wasspentonoperationsandmanagement.Comparisonofthecomponent wise allocation (as per Prodoc) and actual expenses suggest that component-1nearlyconsumedmostoftheallocatedresources,howeverspendingonallothercomponentsremainedlowerthanplanned.Yearwiseanalysissuggeststhatprojectexpendituresgrewsteadilysince2012andwereatthehighest in 2015. It is also important to note that Component-3was started as late as 2015.Componentwiseallocationsweremadebasedonannualworkplansandbudgets,whichweredulyapprovedbytheProjectboard.FundswerereleasedbyUNDPtoPMUandthentoCIUsonquarterlybasisaftersubmissionandapprovalofquarterlyworkplanbyImplementingPartner.PleaseseethefollowingTablefordetailsofyearlyandcomponentwiseexpenditures.Overall,itisconcludedfromanalysisthatprojectfinancialresourcesandinputsweremanagedandspent inanefficient,transparentandaccountablemanner,usingUNDPstandard(PCOM)proceduresforfinancialmanagementandprocurementandrecruitmentprocesses,keepinginviewthebestvalueformoney.

ComponentwiseProjectExpendituresduring2012-2016 2012 2013 2014 2015 2016 TotalOperational sustainableurban transport system inPunjabprovince

Comp-1

23203 79045 488226 544879 651260 1,786,613

Operational sustainableurban transport system inSindhprovince

Comp-2

0 0 293395 397577 238235 929,206

Improved energy efficiencyintruckfreighttransport

Comp-3

0 0 0 354159 462193 816,352


23

Increased public awarenessand institutional capacityon sustainable transportconcepts

Comp-4

10643 117609 297849 600584 141575 1,168,260

Project Operations &Management

ProjectMgt

46157 194336 155066 237176 93090 725,825

Total 80,003 390,989 1,234,535 2,134,375 1,586,353 5,426,256

3.2.3MonitoringandevaluationTheProjectdocumenthasputgreateremphasisonandhasoutlinedanumberofmonitoringandevaluationmeasuresandplans toeffectivelymonitor and report theprogressofprojectinterventionsandresults.Atthehighestlevel,theprojectprogresswasrigorouslymonitoredbytheProjectBoard.ProjectcomponentwiseprogresswasregularlypresentedanddiscussedinPBmeetings and correctivemeasureswere suggested to streamline project interventions. Inaddition,theprojectalsomonitoreditsprogressthroughinternalmonthlyandquarterlyreviewmeetingsfromtimetotime.PMUremainedresponsibleforoverallmonitoringandevaluationofproject interventionsandresults and regular compilation of progress reports. The project has employed a dedicatedMonitoringandEvaluationOfficer,whowasresponsibleforplanningandimplementationofallM&E related activities including collection and processing of timely data and compilation ofprogressreportsetc.ProjectResultsFrameworkwasrevisedanditsoutcomeandoutputlevelindicatorsand targetswereadjustedandmonitoringplanwaspreparedand implemented tomeasureprojectprogressandresults.ProjectprogresshasbeenregularlyreportedonquarterlyandannualbasisthroughfurnishingaseriesofQuarterlyandAnnualProgressReports.Preparedonstandardformatstheseprogressreportsdescribeindetailtheprogressofimplementationofactivities.Theyalsohighlightmajorissues, risks and lessons learnt during the project implementation. UNDP CO was regularlyengaged in oversight and quality assurance of project and has closelymonitored the projectinterventionsonquarterlyandannualbasisthroughregularprogressreviewandreporting. In2016,UNDP introducedanonlineprogress reporting system (STAR) formonitoringofprojectprogress. Accordingly, feedback from Progress Reports/PIRs and tracking tools was used forcourse correction from time to time. UNDP CO was also involved and facilitated thecommissioningofindependentMid-termreviewandTerminalEvaluationoftheProject.Anindependentmid-termreviewoftheprojectwascarriedoutinmid-2015,whichrigorouslyreviewed project design and progress and provided operational recommendations forimprovementofproject interventions.Forexample,amongothers, itcalledagainforrevisionand adjustment of project results framework especially of outcome-1 and outcome-2. TheProjectdocumentalsoenvisagedan independentTerminal Evaluationof theproject towardstheendoftheproject.TheobjectivesofthisTEistoassesstheachievementsofprojectresults,andtodrawlessonsthatcanbothimprovethesustainabilityofbenefitsfromthisproject,andaidintheoverallenhancementtoUNDPprogramming.3.2.4Overallprojectimplementation/execution,coordination,andoperationalissues


24

Giventhecomplexityofcircumstances,theprojecthasmaderigorouseffortsandconsiderableprogresstowardsachievingitsoutputsandoutcomes.However,likemanyotherdevelopmentprojects, it also faced a number of execution and coordination issueswhich delayed projectstart and full scale implementation. According to project document, the proposedimplementation timeframe of project was for five years i.e. from 1st October 2010 to 31December2015.However, theproject started its actual implementation from late2012withtheoperationalizationofProjectBoard,PMUandCIUs.Implementationstartedwithslowpaceandsteadilygatheredsteamupfrom2013to2016.Anumberoffactorswereresponsibleforthedelaysinimplementationincluding;a)ImplementingpartnerandResponsibleparties’selectionissuesOriginally, the project design identified Ministry of Environment, Government of PakistanthroughENERCON,asthemainImplementingPartnerandsubsequentlytheProjectDocumentwasalsosignedbytheMinistryofEnvironmentandUNDP.However,soonafter,thefunctionsof Ministry of Environment at the Federal level were devolved to the provinces under theconstitutionalamendment.Withthis,ENERCON,previouslyasubsidiaryofMoE,wasattachedtoMinistryofWaterandPowerandhencetheMinistryofWaterandPowerbecamethenewImplementing Partner. These changes took considerable time to take effect, thus delayingproject implementation. Furthermore, frequent turnover of the National Project Directors,designatedofficialsofIP,alsoposedmanagementchallengesanddelaysinimplementation.RegardingComponent-3, the initialdesign identifiedtheMinistryof IndustriesasResponsiblePartyfortheComponent-3relatedtotrucking.However,theyexpressedtheirinabilitytotakeupthecomponentduetorelevanceissues.Afternecessarysearchanddeliberations,MinistryofCommunicationwas selectedas relevantResponsibleParty for theComponent-3,andLoAwas signed inMarch 2015. The late start of interventions under this component also has itsownimplicationsforachievementofdesiredprojectresults.b)LackofclarityandunderstandingofexecutionarrangementsDiscussionswithpartnerssuggestthatinitiallytheorganizationalandexecutionsetupwasnotvery clearly defined. Project was executed through NIMmodality, which basically takes intoconsiderationasingleImplementingPartneratthenationallevel.Sinceprojectalsoinvolvedanumber of Responsible Parties, especially provincial governments, therefore in the start theproject hierarchy and line of responsibilities and reporting among IP and RPs were unclear.InitiallyRPs-CIUsreceivedfundsdirectlyfromUNDPagainstapprovedworkplans,makingthemrelativelyindependentoftheIP-PMU.Matterswere further compoundedby lack of proper agreements or signed contracts amongimplementing responsible parties. To streamline project implementation and coordination,PMUmade considerable efforts and formulated and secured signatures on official Letters ofAgreement (LoAs) between IP and RPs. These organizational and contractual arrangementsconsumedsubstantialtimeandresultedindelaysinimplementationofprojectinterventions.


25

DiscussionswithpartnersalsosuggestthatoverallcollaborationbetweenIPandRPsremainedoptimal,exceptforComponent-1,whereattimes,collaborationsbetweenIP-PMUandUrbanUnit of Government of Punjab remained strained due to differences in understandings ofprojectpriorities,interventionsandimplementationmodalities.Overall,thesedifferencesalsoconsiderably hampered the trust between IP and RP and resulted in delays of projectinterventions.c)ProceduralandprocessdelaysOrganization of Project board and recruitment of PMU and CIUs staff also consumedconsiderable time till required project staff came on board and 1st PB meeting was held inOctober2012.Furthermore,preparationandapprovalofannualworkplansalsoremainedverytimeconsumingandcumbersome.Forexample,Mid-termreviewnotesthatthefinalizationofAWPfor2013tookalotofeffortanddraftplanswererevised38timesbeforefinalapproval.ThelateapprovalofAWPsinturnresultedindelaysintransferoffundsfromUNDPtoPMUandCIUs,subsequentlydelayingprojectinterventions.Theprojectundertookanumberofstudiesandresearchworks.BiddingprocessesandselectionofcontractorsingeneralandespeciallyinPunjab,consumedextratimeduetoissueslikegapsin budgets foreseen for a study as compared with the bids received. In such cases, call forproposals were re-advertised and the selection process was re-done, resulting in delays incompletionofproposedstudies.3.3ProjectResults3.3.1OverallResults/Outcomes(EvaluationRating:4-ModeratelySatisfactory)The following table provides a summary of achievements against Indicators and Targets ofProjectRRF(RevisedMarch2014).DetailedanalysisisprovidedunderthefollowingsectionsonRelevance,Efficiency,Effectiveness,SustainabilityandImpact.

SUMMARYTARGETSANDACHIEVEMENTSREVISEDRESULTSANDRESOURSESFRAMEWORK(RRF)FORPAKISTANSUSTAINABLETRANSPORTPROJECT

Results (Outcomes and Outputs)

Sub-Output and Indicators Targets Summary Achievements

Overall Project Objective/ Atlas Output Reduction of greenhouse gas (GHG) emission associated with

Overall Project Objectives Reduction of greenhouse gas (GHG) emission associated with urban transportation Indicator 1: Cumulative, direct GHG emission reductions in the urban transport sector compared to BAU scenario over a 20-year period, ktonnes

Target 1: 608 ktonnes CO2 (direct reduction from BRT demonstrations)

• 179.4 ktonnes CO2 estimated reduction

from Lahore and Islamabad Metro Bus Systems (Source: Project Study) (For details please see section on 3.3.8 related to Impact)


26

urban transportation

Indicator2: Cumulative, direct GHG emission reductions from a pilot scheme to modernize the trucking fleet compared to BAU scenario over a 10-year period, ktonnes

Target 2: 5 ktonnes CO2 (direct reduction from truck modernization program)

• Pilot on Truck modernization did not materialize (For details please see section on 3.3.6 on Effectiveness )

Output -1 Operational Sustainable Urban Transport System in Punjab Province

Sub- Output 1.1: Feasibility plans for a demonstration integrated BRT system in selected cities in Punjab Province Indicator 1: Number of completed feasibility plans prepared for BRT in Punjab Indicator 2: No of public private financing secured based on the selected feasibility study (IPDF and ECF)

Target 1: 2 feasibility plans augmented for BRT in Lahore and Rawalpindi Target 2: 1 Public private financing secured based in one city in Punjab

• In the wake of Governmental interests in implementation of Lahore and Islamabad Metro. The project proposed/envisaged BRT demonstration related feasibility plans did not materialize and project support was diverted to the two Metro Systems.

• The proposed securing of public-private for

BRT did not materialize.

Sub-Output 1.2: Plans for implementing demonstration integrated BRT system Indicator 1: Extent to which effective capacity building programmes for UU developed and implemented Indicator 2: Number of approved integrated BRT implementation plans for selected cities in Punjab Province.

Target 1: No of feasibility studies from output 1.1 being upgraded to holistic BRT implementation plan for the BRT system. Target 2: 2 approved implementation plans for integrated BRT system in 2 cities of Punjab

• As mentioned earlier the project proposed/envisaged BRT demonstration related feasibility plans did not materialize.

• Read as above

Sub-Output 1.3 Infrastructure for demonstration BRT system Indicator 1: Number of capacity development programmes developed and implemented for city district government/govt. agencies and local engineering firms Indicator 2: Extent to which the capacity development programmes have been effective.

Target 1: No of capacity development programmes for city government/govt. agencies and local engineering firms in engineering, construction, operations and management of BRT system

• Following is a short list of capacity development events organized by the project. (For detail list please see table of interventions under section 3.3.3 related to Efficiency);

• Workshop on “The Provision of Sustainable Public Transport System” • Seminar on “GIS Applications in the Sustainable Urban Transport System, Lahore • CDM Training in Lahore • 'Training of Police Wardens' on Bus rapid Transit during the construction phase" in Rawalpindi • Traffic Police training on Bus Rapid Transit (Batch-I) in Multan • City Traffic Police training on Bus Rapid Transit (Batch-II) in Multan • Training course on “Nationally Appropriate Mitigation Measures” • Consultative Sessions for Academic Institutions/Universities of Rawalpindi and Islamabad on Sustainable Urban Transport.


27

Sub-Output 1.4: An operational demonstration BRT system

Indicator 1: Number of institutions with enhanced capacity to operate, maintain, and manage a BRT system Indicator 2: Cumulative GHG reductions from the BRT demos in 02 cities of Punjab- ktonnes CO2

Indicator 3: Cumulative energy savings generated by BRT pilot demonstration

Indicator 4: % increase in public transit ridership

Indicator 5: Methodology and M&E plan designed for the measurement of the specific energy and emission parameters

Target 1: One operational BRT system with no. of institutions having enhanced capacity to operate, maintain, and manage a BRT system

Target 2: 20,280 tonnes of CO2 reduced by the BRT demonstration

Target 3: 1000 toe of energy saved from BRT demonstration Target 4: 8% increase in public transit ridership Target 5: M&E plan (including methodology) development for calculating energy & emission savings

• As mentioned earlier the project proposed/envisaged BRT demonstration related feasibility plans did not materialize.

• BRT demonstrations were not perused.

However for cumulative CO2 reduction (from Lahore and Islamabad Metro Bus) please see in above Target 1 for overall Project objectives

• Project has conducted a Study on

Assessment of the Vehicular Emissions’ Status in the Pre and Post BRT Project Implementation Scenario at the Twin Cities, Rawalpindi and Islamabad

Sub- Output 1.5: Strengthened institutional framework that enables holistic urban transport development. Indicator 1: Number of institutional framework developed to facilitate holistic urban transport planning in Punjab

Target 1: New policy framework proposing & stream-lining the reporting lines, responsibilities and accountability for each relevant agencies (Punjab Govt, and other institutions)

• Work on desired new policy framework in Punjab did not materialize.

Sub- Output 1.6: Punjab Provincial Government integrated urban transport plan Indicator 1: Number of strategic integrated urban transport plans Indicator 2: Number of Provincial policy for integrated Sustainable Urban Transport

Target 1: 1 strategic plan for holistically planned integrated urban transport Target 2: Provincial policy for integrated Sustainable Urban Transport formulated, approved and implemented.

• Work on any such strategic plan did not materialize.

• Work on provincial policy for sustainable

transport in Punjab did npt materialize.

Output- 2 Operational sustainable urban transport system in Sindh Province

Sub- Output 2.1: Feasibility plans and approved financing with “integrated BRT plans” for cities in Sindh Province Indicator 1: Number of completed feasibility plans prepared for BRT in Sindh Indicator 2: Number of public

Target 1:1 bankable integrated BRT feasibility with implementation plan Target 2: 02 financial institutions with commitment to

• Work on any such bankable BRT feasibility did not materialize.

• Govt. of Sind and Govt. of Pakistan has

committed to provide finances for the two Metro Systems in Karachi.


28

private financing secured based on the selected feasibility study (ECF/IPDF)

finance BRT systems

Sub- Output 2.2: Strengthened institutional framework that enables holistic urban transport development. Indicator 1: Number of institutional framework developed to facilitate holistic urban transport planning in Sindh

Target 1: New policy framework proposing & stream-lining the reporting lines, responsibilities and accountability for each relevant agencies (Sindh Govt, and other institutions)

• Project actively supported the development of Draft Sindh Urban Transport Policy which is currently in the approval process.

Sub-Output 2.3: A strategic plan for the development of sustainable urban transport in Sindh Province Indicator 1: Number of strategic integrated urban transport plans

Target 1: 1 strategic plan for holistically planned integrated urban transport

• Any such strategic plan for planned integrated urban transport did not materialize.

Sub- Output 2.4: Approved and enforced Sindh provincial policy that enables development and operation of sustainable urban transport systems Indicator 1: Number of provincial policies for developing sustainable urban transport for Sindh province

Target 1: 1 approved Sindh provincial policy on sustainable urban transport with associated implementing rules and regulation (IRRs)

• Draft Sindh Urban Transport Policy has been developed and is currently in the approval process

• Developed Provincial Environmental Quality Standards for the transport sector in Sindh province.

Sub-Output 2.5: An operational demonstration BRT system

Indicator 1: Number of institutions with enhanced capacity to operate, maintain, and manage a BRT system Indicator 2: Cumulative GHG reductions from the BRT demos in 02 cities of Sindh- ktonnes CO2

Indicator 3: Cumulative energy savings generated by BRT pilot demonstration

Indicator 4: % increase in public transit ridership

Indicator 5: Methodology and M&E plan designed for the measurement of the specific energy and emission parameters

Target 1: One operational BRT system with no. of institutions having enhanced capacity to operate, maintain, and manage a BRT system Target 2: 10,140 tonnes of CO2 reduced by the BRT demonstration

Target 3: 490 toe of energy saved from BRT demonstration Target 4: 4% increase in public transit ridership Target 5: M&E plan (including methodology) development for calculating energy & emission savings

• Currently there is no operational BRT system in Karachi. However work on Government sponsored Green and Orange line Metro is in progress

• No reduction as no operational BRT system

is currently existing in Karachi. • Project completed a Study on carbon

emissions for the selected BRT line and potential for Clean Development Mechanism


29

Output 3: Improved energy efficiency in truck freight transport

Sub- Output 3.1: Approved and enforced policies on energy efficiency in truck freight transport Indicator 1: Number of background studies completed to support Trucking Policy implementation Indicator 2: Number of implementing rules and regulations (IRRs) and implementing actions formulated and recommended for approval Indicator 3: Number of IRRs approved and enforced

Target 1: 10 background studies completed on supporting implementation of Trucking Policy Target 2: 5 implementing rules and regulations formulated Target 3: 5 IRRs approved and enforced

• Project completed 10 background studies related to trucking sector in Pakistan (For detail list please see table of interventions under section 3.3.3 related to Efficiency )

• Project completed a number of studies

related to Axel Load management, weight management, vehicle examination system and establishment of Central Data Repository for the motor registration (For detail list please see table of interventions under section 3.3.3 related to Efficiency )

• Work any such IRRs enforcement did not materialize

Sub-Output 3.2: Completed pilots to implement strategy to modernize trucking fleet Indicator 1: Number of trucks involved with pilots to demonstrate energy efficiency objectives of Trucking Policy Indicator 2: Cumulative GHG reductions from a pilot scheme to modernize the trucking fleet- ktonnes CO2 Indicator 3: Cumulative energy savings generated from truck modernization pilots

Target 1: M&E plan (including methodology) development for calculating energy & emission savings based on 50 trucks involved in pilot Target 2: 5 ktonnes CO2 (direct reduction from truck modernization programme) Target 3: 150 toe of energy saved from truck modernization pilot

• Work on truck modernization pilot did not materialize.

• Same as above • Same as above

Sub-Output 3.3: Established public-private partnerships (PPPs) in the modernization of the trucking fleet. Indicator 1: Number of public-private partnerships (PPPs) for truck modernization Indicator 2: Number of trucks planned for involvement in replication of pilots

Target 1: 3 public-private partnerships (PPPs) established Target 2: 2000 trucks involved in plans for replication of truck modernization

• Work any such PPP establishment did not materialize.

• Work on truck modernization pilots did not

materialize.

Output 4: Increased public awareness and institutional capacity on sustainable transport concepts

Sub-Output 4.1: Completed awareness raising campaigns on sustainable transport concepts Indicator 1: Extent to which completed awareness raising campaigns have been effective Indicator 2: Extent to which cities benefiting from awareness raising campaigns

Target 1: 5 awareness raising campaigns conducted on concept of sustainability in transport sector, BRT as best model for SUT in Punjab, Sindh and Rawalpindi/Islamabad

• Series of activities under the caption of Awareness Raising Campaigns from 2014-2016 had been under taken involving all stakeholders in Lahore, Karachi, Rawalpindi and Islamabad. (For detail list please see table of interventions under section 3.3.3 related to Efficiency )


30

Target 2: 3 cities where awareness raising campaigns have been conducted including; Karachi, Lahore and Rawalpindi/Islamabad

• Awareness raising campaigns had been under taken in 3 cities i.e. Lahore, Karachi, Rawalpindi and Islamabad etc.

Sub-Output 4.2: Completed training program on strategic urban land use and transportation planning conducted at various training, academic and vocational institutes in Pakistan Indicator 1: Number of completed training courses on strategic urban land use and sustainable urban transport planning (SUTP) Indicator 2: Extent to which cities and provincial planners and students effectively trained on land use planning (LUP) and SUTP Indicator 3: Number of educational institutes where LUP and SUTP courses are offered

Target 1: 8 training courses related to strategic urban land use and sustainable urban transport area (3 courses in 2014; 3 courses in 2015; 2 courses in 2016) Target 2: 30 city and provincial planners and students trained (10 city and provincial planners in 2014; 10 city and provincial planners in 2015; and 10 city and provincial planners in 2016) Target 3: 4 educational institutes where LUP and SUTP courses offered

• 8 trainings and capacity building events organized by the project where range of stakeholders were involved (For further detail please see table of interventions under section 3.3.3 related to Efficiency);

• Training Workshop on Climate Financing Opportunities • Political Dialogue on “Sustainable Urban Transport in Pakistan” • Training workshops for volunteers Rawalpindi-Islamabad-Metro-Bus-Service • Training course on Carbon Finance with a specific focus on transport • Training workshop on integrated Bus Rapid Transit was executed at Karachi and Islamabad • National conference on sustainable transport at Islamabad. • 3 Seminar for media on transport sector and related issues in Pakistan at Karachi. • Policy dialogue on Urban Mobility and Transport • Guidelines for curriculum on sustainable

transport have been developed which are under process of development and adoption. The courses will be offered at Fatima Jinnah Women University, Rawalpindi and NED University, Karachi.

Sub- Output 4.3: Sharing experiences on integrated BRT development and implementation of the Trucking Policy Indicator 1: Extent to which the completed workshops on integrated BRT development have been effective Indicator 2: Extent to which the completed workshops on the implementation of the Trucking Policy have been effective

Target 1: 8 workshops on integrated BRT development (conducted by CIUs & reporting/dissemination by IUCN) Target 2: 8 workshops on Trucking Policy implementation completed (conducted by CIU-Trucking & reporting/dissemination by IUCN)

• More than12 workshops, seminars and training were conducted by CIUs (For detail list please see table of interventions under section 3.3.3 related to Efficiency)

• 6 workshops, consultative dialogues,

seminars were organized by CIU trucking (For detail list please see table of interventions under section 3.3.3 related to Efficiency)

3.3.2RelevanceofProjectinterventionsandresults(EvaluationRating:2-Relevant)Interviews with stakeholders, situation analysis and project documents suggest that overallproject objectives and interventions were found highly relevant and consistent withGovernment of Pakistan policies, UNDP and GEF priorities and needs of the beneficiaryinstitutionsandcommunities.


31

Pakistan's National Environmental Policy (NEP 2005) specifically promotes the principles ofsustainabletransportundertheirsectoralguidelinesforairqualityandnoise(i.e.regulationofvehicle emissions, enforcing fuel specifications, encouraging cost effectivemass rapid transitsystems etc). Pakistan has also undertaken a comprehensive inventory of Greenhouse gas(GHG)emissionsourcesandhasprioritizedfeasiblemitigationoptionsandformulatedaGHGabatementstrategyundertheGEF/UNDPAsianLeastCostGreenhouseGasAbatementStrategy(ALGAS).In response to urban congestion and increasing GHG emissions from transport sector inPakistan’s largest urban centers, like Karachi, Lahore and Rawalpindi-Islamabad, efforts havebeenunderwayforanumberofyearstoprovidemasstransitandespeciallyBRTsolutions.Duetogreater interestsofGovernmentofPakistantwoMetroBusSystemshavebeencompletedand operationalized in the cities of Lahore and Rawalpindi-Islamabad in 2013 and 2015respectively and are presently successfully functioning. Similar efforts are also underway toextendMetroBusServicesinthecitiesofKarachi,Multan,FaisalabadandPeshawaretc.Itcanbe easily concluded that this high level of interest and investments from governmentalinstitutions in BRTs makes the project fully relevant and complaint with governmentalpriorities.On the other hand, the road freight transport, especially old age trucks and practices, havebeenoneoftheprimarycausesofroadcongestion,fuelinefficiencyandGHGemissions.Withthe importance of road freight to Pakistan’s overall economy, the Government of Pakistandeclared the trucking sector as a formal industry in January 2008 and adopted a “TruckingPolicy” to improve the performance of the sector. However, the said policy lacked animplementationstrategyandmechanisms.Projectcomponent-3wasfoundveryrelevantinthecontext of streamlining and implementation of trucking policy to subsequently improve fuelefficiencyandreductioninGHGs.InadditiontoitsrelevancetoGovernmentofPakistanpoliciesandpriorities,projectagendaisalsohighlyrelevanttoUNDPandGEFglobalprioritiesofpromotingtheagendaofsustainabletransport. GEF catalyzes transformational change by supporting sustainable transport, whichreduces greenhouse gas emissions. GEF has built an investment portfolio that addressesemissions from the transport sector in developing and transition countries. GEF sustainabletransportrelatedprojectsareunderwayinseveralcitiesaroundtheworld,creatingasustainedimpact inthetransportsector.Needlesstoemphasizethatprojectagenda isalso in linewithGlobalSDGs,Goal11:SustainableCitiesandCommunities,Goal12:ResponsibleConsumptionandProductionand,Goal13:ClimateAction.Nevertheless, the project was also very relevant in addressing the needs of poor and lowincome classes living in major cities like Lahore, Rawalpindi, Islamabad, Karachi etc. byproviding sustainable, convenient and affordable public BRT services and on the other handprovidingthemwithcleanerandbetterenvironmentthroughreductioninGHGemissionsandlocalairpollutants,congestionandnoisepollution.


32

3.3.3EfficiencyofProjectInterventions(EvaluationRating:4-ModeratelySatisfactory)TheTORdefinesefficiencyastowhatextenttheinputs(resourcesandtime)wereusedinthebest possible way to achieve results. Generally, efficiency of project interventions can beassessedthroughthreebasicdimensionsi.e.1)economicutilizationoffinancialresourceswithreference to stipulatedbudgetaryallocations2) timelinessof interventionswith reference toallocated timeframeand,3)quantityandspecificationsofprojectactivitieswith reference toworkplans.Analysis of financial statements shows that the project has utilized around USD 5.4 Million(70%)ofitstotalcommittedresourcesofUSD7.8Million.ComponentwiseexpendituredetailsshowthatactivitiesunderComponent-1consumedaround33%ofthetotalresources,followedbyComponent-4 at 22%,Component-2 at 17%,Component-3 at 15%and13%was spentonoperationsandmanagement.Component2and3weremostunderspentandcouldonlyutilizeless thanhalfof theallocatedbudgetary resourcesdueto latestartof theproject.Yearwiseanalysissuggeststhatprojectexpendituresgrewsteadilysince2012andwereatthehighestin2015.Componentwiseallocationsweremadebasedonannualworkplansandbudgets,whichweredulyapprovedbytheProjectboard.Forfurtherdetails,pleaseseesection3.2.2.Overall,itisconcludedfromanalysisthatprojectfinancialresourcesweremanagedandspentinanefficient,transparentandaccountablemanner,usingUNDPstandard(PCOM)proceduresfor financialmanagement,procurementand recruitmentprocesses, keeping in view thebestvalueformoney.Overtheyears,PAKSTRANProjecthasimplementedawiderangeofinterventionsandactivitiestoachieveitsstipulatedoutputsandoutcomes.(Pleaseseetablebelow).Analysissuggeststhattheproject hasmade rigorous efforts and commendableprogress, however asmentioned inprevious sections, implementation of project activities was considerably delayed due to anumberofissuesincludingtimeconsumedbyselectionsofsuitablepartners,lackofclarityandunderstanding of execution and organizational arrangements and time consuming andcumbersome implementation processes and procedures. Implementation started with slowpaceandsteadilygatheredsteamandmostofprojectimplementationtookplacefrom2014to2016.Activitiesundercomponent-3couldonlybestartedaslateas2015.Despite the complexity of working environment and involvement of diverse range ofstakeholders,itisimportanttoobservethattheprojecthasmadestrenuouseffortstocatchupwiththetimelagandhasconsiderablyspeduptheimplementationofactivitiesinthesecondhalf of its life. Discussion with stakeholders and review of records suggest that the overallqualityofactivitiesperformedandtechnical,humanandmaterialinputsprovidedwerefoundcost effective, instrumental, up to themark and in linewith the project stipulated aims andaspirationofstakeholders.Followingisthesummaryofcomponentwiseactivitiesimplementedduring2012-2016:


33

Component/Output MainInterventionsandActivitiesimplemented(2012-2016)Component/Output-1:OperationalSustainableUrbanTransportSysteminPunjabProvince

StudiesandResearchworks;• StudyonMappingofEnvironmentalandSocio-EconomicProfileAlongBRTCorridor-1

FromGajjumattatoShahdara• Studyon'mappingoflanduseandinfrastructurealongBRTcorridorfromGajjumatta

toShahdarainLahore• StudyonIdentificationofNon-farerevenuegenerationopportunitiesforRWP-ISB

MetroBussystem• StudyonImplementationStrategyforIslamabadBusService(IBS)• StudyonAssessmentoftheVehicularEmissions’StatusinthePreandPostBRT

ProjectImplementationScenarioattheTwinCities,RawalpindiandIslamabad.• StudyonPeerReviewofFeederRouteNetworksStudiesinIslamabad.• M&EPlanforEnergy&GHGEmissionsReductionBenefitsCalculations

Events,TrainingandAwarenessraising;• Workshopon“TheProvisionofSustainablePublicTransportSystem”(Asteptowards

makingaGreencity),Lahore• Seminaron“GISApplicationsintheSustainableUrbanTransportSystem,Lahore• 3DaysCDMTraininginLahore(Attendedby27participants)• Oneweek'TrainingofPoliceWardens'onBusrapidTransitduringtheconstruction

phase"inRawalpindi(Attendedby30participants)• InaugurationofMetroBusProjectRawalpindi-IslamabadBusRapidTransit• One-weekCityTrafficPolicetrainingonBusRapidTransit(Batch-I)inMultan

(Attendedby31participants)• One-weekCityTrafficPolicetrainingonBusRapidTransit(Batch-II)inMultan

(Attendedby27participants)• 3-daystrainingcoursewascarriedoutatBhurbanon“NationallyAppropriate

MitigationMeasures”(Attendedby27participants)• ConsultativeSessionsforAcademicInstitutions/UniversitiesofRawalpindiand

IslamabadonSustainableUrbanTransportforAwarenessRaisingamongstStudents,Academicians&Stakeholders.

ResearchCenters;

• EstablishmentofResearchandDevelopmentUnitforSustainableUrbanTransportatDTEM,UET,Lahore

• EstablishmentofCentreforExcellenceforSustainableTransport(CESTAC)atFatimahJinnahWomenUniversity,Rawalpindi

Component/Output-2:OperationalsustainableurbantransportsysteminSindhProvince

StudiesandResearchworks;• StudyonInstitutionalanalysisoutliningtheimpactsofstakeholdersinterestsonBRT

projectdesignoptionsinordertoimprovelivingandworkingconditionsalongRedBRTcorridor

• Studyonsurveyof'mini-cabs(Quinquis)touseaspick&dropcarrier/feederroutesonGreenandRedBRTLinestobringthemundertheambitoflaw'

• Studyonsurveyof'mini-cabs(Quinquis)touseaspick&dropcarrier/feederroutesonOrangeandYellowBRTLinestobringthemundertheambitoflaw'

• DevelopedProvincialEnvironmentalQualityStandardsforthetransportsectorin


34

Component/Output MainInterventionsandActivitiesimplemented(2012-2016)Sindhprovince.

• DevelopmentofSindhUrbanTransportPolicyandsharingwithstakeholders• DevelopedPrincipleGuidelinesforUrbanTransportPolicy• StudyoncarbonemissionsfortheselectedBRTlineandpotentialforClean

DevelopmentMechanism.• Studyonreviewingthemotorvehicleslawsandproposingamendmentstosupport

theUrbanTransportPolicyofSindhandbringingitatparwithPunjab

Events,TrainingandAwarenessraising; • Oneweekarticulatedtrainingfortrafficpolice,transportofficials,driversofseveralof

publicservicetransport• AnumberofconsultativeSessionsheldfordevelopmentofSindhSustainableUrban

TransportPolicy• WorkshoponRolesandResponsibilitiesofInstitutions/DepartmentsinTransport

sector• ConsultativeworkshoponSafetyandSecurityofTransportation• SupportedtheSindhGovernmentinEstablishmentofSindhMassTransitAuthority,

whichisabigachievementwithregardtostrengtheninginstitutionalframework.• CIUSindhhasregularlyconductednumerousconsultativesessionsonweeklybasis

involvingurbantransportstakeholders

Component/Output-3:Improvedenergyefficiencyintruckfreighttransport

StudiesandResearchworks;• StudyonStocktakingonoptionsforenergyefficiencyfortruckfreighttransport• StudyonCO2emissionsfromtruckfreighttransportinalow-carbonscenario.• StudyonassessmentofCO2emissionsfromtruckfreighttransportinaBusiness-As-

Usual(BAU)scenario• Studyoninternationalbestpractices/trendsintruckfreightenergyuseanditslinkage

tothecontextofPakistan• Studyonenvironmentalimpactsofamajorfreightcorridor• Studyontheidentificationofgapsandstrategyfortheimplementationofexisting

truckingpolicy• StudyonAssessmentofAxelLoadmanagementsystemincontextofcrossborder

trade(CPEC,AfghanBorder,Iranetc.)• Studyonmodernweightstationstocomplywithinternationalstandards(crossborder

trade)• Studyformodernizingtheoverallmotorvehicleregistrationandestablishmentof

“CentralDataRepository”inthecontextoftheroadfreightsector• StudyfordevelopmentofmodernizedtruckfreightexaminationsysteminPakistan• PreparationofM&EPlanbycalculatingenergyandemissionssavinginTruckingsector

ofPakistanEvents,TrainingandAwarenessraising;

• ConsultativeSessionontruckfreighttransportinPakistanheldatIslamabad.(Attendedbylargenumberofparticipantsfrom25differentinstitutions)


35

Component/Output MainInterventionsandActivitiesimplemented(2012-2016)• SeminaronmodernizationoftruckfreighttransportinPakistanwasheldatKarachi.

(Attendedbyalargenumberofparticipantsfrom17differentinstitutions).• ExposurevisittoKarachiPortTrustwasheldonNovember6,2016.(morethan50

participantsfromrelevantstakeholderswereinvolved)• Two-daystrainingworkshoponchallengesofroadtruckfreighttransportinPakistan

heldatMultan.(Attendedbymorethan50participants)• ConsultativedialoguesessionontruckingpolicyandanexposurevisitheldatGwadar.

(50participantsfromrelevantstakeholders)• ConsultativeDialogueSessiononTruckingPolicyheldatNEDUET,Karachi.(Attended

bymorethan50participantsfromrelevantstakeholders)Component/Output-4:Increasedpublicawarenessandinstitutionalcapacityonsustainabletransportconcepts

Events,TrainingandAwarenessraising;• NationalconferenceonsustainabletransportatIslamabad.Attendedbyalarge

numberofrepresentativesfromtheGovernment,media,academia,nationalandinternationalNGOs.

• SensitizationseminarformediaontransportsectorandrelatedissuesinPakistanatKarachi.(Attendedbymorethan50participants)

• Twomediaseminarswereorganized.(51participantsfromcivilsocietyandprintandelectronicmedia)

• PolicydialogueonUrbanMobilityandTransportwasorganizedatNUST.(Attendedbymorethan150stakeholders)

• SeriesofactivitiesunderthecaptionofAwarenessRaisingCampaignsof2014and2015hadbeenundertakenatthehighereducationinstitutionsinLahore,Karachi,RawalpindiandIslamabad.Thecampaignincludedpostercompetition,debatecompetitionandvideoclipcompetition,particularlydesignedtoensuretheparticipationoflargenumberofstudents.

• PoliticalDialogueon“SustainableUrbanTransportinPakistan”wasorganizedatIslamabad.Thedialoguewasattendedbyrepresentativesofeightmajorpoliticalpartiesinthecountry.

• Three-dayawarenessraisingcampaignwasorganizedatPotoharMetroBusStation,I-9,Islamabad.

• TwotrainingworkshopswereconductedforvolunteersfortheInauguralCeremonyofRawalpindi-Islamabad-Metro-Bus-Service(RIMBS)

• ComprehensiveCapacityDevelopmentPlanforthetargetgroupsinPunjabandSindhdevelopedthroughaconsultativeprocess.

• TrainingcourseonCarbonFinancewithaspecificfocusontransportorganizedatKarachifor29relevantinstitutionsinwhich49peopleparticipated

• Nationalexposurevisitwasorganizedfromtofacilitatethecapacitybuildingofpolicymakersandconcernedfederalandprovincialdepartments

• TrainingworkshoponintegratedBusRapidTransitwasexecutedatKarachiandIslamabad

• TrainingWorkshoponClimateFinancingOpportunitiesatIslamabadforrelevantstakeholders

• StaffTrainingonfinancialmanagementorganizedbyMANGO.• InternationalexposurevisittostudythetransportsystemofSeoul,SouthKoreawas

organized.Adelegationof11staffmembersfromtheprojectparticipated• ParticipationofPMU,UNDPandIUCNpersonnelattheUNFCCCCOP21inFrance


36

Component/Output MainInterventionsandActivitiesimplemented(2012-2016)• AwarenessraisingmaterialincludingPAKSTRANnewsletter,calendars,Jareeda

magazine,posters,stickers,notebooks,mugs,foldersandprojectstandeesdeveloped,displayedanddisseminated.

• Developmentandairingofdocumentaryon"SustainableUrbanTransport"incollaborationwithPTV.

• Projectwebsitedevelopedandperiodicallyupdatedandmaintained.• ThePAKSTRANpagesonsocialnetworkingsitesdevelopedandmaintained• Guidelinesforcurriculumontransporthavebeendevelopedwhichareunderprocess

ofdevelopmentandadoption.ThecourseswillbeofferedatFatimaJinnahWomenUniversity,RawalpindiandNEDUniversity,Karachi.

ProjectManagement,Operations,CoordinationandM&E

• SelectionofIPsandRPsandestablishmentofPMUsandCIUsandrecruitmentofProjectstaffandpreparationandsigningofLoAswithpartners

• EstablishmentofProjectBoardandorganizationofPBmeetings• RevisionofProjectresultsframework• Preparation,facilitationandapprovalofAnnualWorkPlansandquarterlyandannual

Progressreports• PreparationandimplementationofM&EPlanforproject.Preparationofconsolidated

Logs(risklog;issueslog;communication&monitoringplan)• OverallCoordinationofactivitiesofPMUalongwithCIUsandotherstakeholders

(Meetings,consultations,presentations)• FacilitationofCIUsinPreparationofTORsandprocurementprocessforrelated

studies.PMUwasalsoactivelyinvolveddirectlyinexecutionofsomestudieslikeAssessmentofvehicularemissionsfromBRTinRawalpindi,ImplementationstrategyfordevisingactionplanonIslamabadBUSServiceandReviewoffeederrouteNetworksStudiesinIslamabadetc.

• PMUwasalsoveryactivelyinvolvedinestablishmentofCESTACattheFJWUinRawalpindi

• PMUwasalsorigorouslyinvolvedinfinancialmanagementandreportingofprojectbudgetandspending.ThisinvolvedmultiplerequisitionandreportingmechanismsinlinewithUNDPfinancialmanagementandreportingprocedures.

3.3.4CountryOwnershipPAKSTRANprojectwas implementedusingNational Implementation (NIM)Modalitywith theMinistry of Water and Power, Government of Pakistan as main implementing partner. Theproject board was chaired by Secretary MoW&P and project was led by National ProjectDirector, a high ranking official of the MoW&P. At the provincial level, project responsibleparties included Government of Punjab, Government of Sindh and for trucking componentproject partneredwith theMinistry of Communication, Government of Pakistan. These highlevels of participation of governmental institutions suggest that project country ownershiplevelswereoptimalandgovernmentalinstitutionswereinthedrivingseat.Government of Pakistan has been very keen and made strenuous efforts to address urbantransport issues in big cities. In this regard the Government has implemented andoperationalized Metro Bus Systems in the cities of Lahore and Rawalpindi-Islamabad and anumberofBRTsystemsinbigurbancenterslikeKarachi,Multan,FaisalabadandPeshawarareeither under progress or in the pipeline. These high levels of governmental interests also


37

suggestthatthereisagooddealofownershipatthecountrylevelespeciallyforthepromotionofsustainablepublictransportsystemsinurbancenters.3.3.5MainstreamingThe ToR outlines that UNDP-supported GEF-financed projects are key components in UNDPcountry programming, aswell as regional and global programmes. Therefore, the evaluationalsoassessedtheextenttowhichtheprojectwassuccessfullymainstreamedwithotherUNDPpriorities, including poverty alleviation, improved governance, the prevention and recoveryfromnaturaldisasters,andgender.ItisimportanttohighlightthatPAKSTRANprojectwasoneofitskindthereforeduetothedistinctivenatureandobjectivesitisfounddifficulttogaugetheextenttowhichprojectwasdirectlymainstreamedwithotherUNDPpriorityareas.However,havingsaidthis,theprojectinterventionsindirectlycorrespondedandcontributedtopriority areas like improved governance, especially the capacity building and provision oftechnicalsupporttogovernmental institutionsinthecontextofurbantransportmanagementandpromotionoftruckfreighttransport.Indirectly,projectalsocontributedtoimprovinglivingconditions inurbanareas through improving air quality through reducedGHGemissions andlocalairpollutantsandreducedtrafficcongestion.Nevertheless,theimprovedBRTservicesinLahore and Islamabad also provide safe, efficient, decent and cost efficient urban transitfacilitiestoallcitizens,especiallythepoorandwomenfolk.3.3.6EffectivenessofProjectResults(EvaluationRating:5-Satisfactory)PAKSTRANprojectaimedatachievingthestipulatedoutcomesofreducingGHGemissionsandincreasingenergyefficiencythroughachievingfourmainoutputs,theseincluded;OutcomeComponent1:AnoperationalsustainableurbantransportsysteminPunjabprovinceOutcomeComponent2:AnoperationalsustainableurbantransportsysteminSindhprovinceOutcomeComponent3:ImprovedfuelefficiencyintruckfreighttransportOutcomeComponent4:IncreasedpublicawarenessandinstitutionalcapacityonSustainable

TransportconceptsFollowingisthedetailsofachievementstatusoftheseoutputs;1)OperationalsustainableurbantransportsysteminPunjabprovinceUnderthisoutput,projectprovidednecessarysupportfortechnicalresearch,capacitybuilding,coordinationandawarenessforpromotionofsustainabletransportinPunjab.ProjectmainRPfor this outputwasUrbanUnit, P&DDPunjab and also involvedother relevant stakeholders.DiscussionssuggestthatcollaborationbetweenIPandRPremainedsomehowstrainedduetodifferences in understandings of project scope, priorities, roles of IP and RP andimplementationmodalities.OverallthesedifferencesattimeshamperedthetrustbetweenIPandRPandresultedinimplementationdelays.Project interventions under this output mainly concentrated to support BRT systems in thecitiesofLahoreandRawalpindi-Islamabad.SinceLahoreMetroBusprojectwascompleted, in


38

Feb2013,beforethepracticaloperationalizationoftheprojectthereforeprojectwasnotinapositiontoprovideassistanceinthedesign,feasibilityandimplementationstageoftheproject.HoweversoonaftertheprojectactivelycollaboratedwithRPandstakeholdersandexpeditedanumber of studies regardingmapping of environmental and socio-economic profile and landuse and infrastructure along BRT Lahore Corridor-1 and preparation of M&E Plan for GHGEmissionsReductionBenefitsCalculationsOn the other hand during the development and operationalization of Rawalpindi-IslamabadMetro Bus project, PAKSTRAN project was able to provide substantial support and hasconducted a number of research work like identification of non-fare revenue generationopportunitiesforRWP-ISBMetroBus,implementationstrategyforIslamabadBusService(IBS),assessmentofthevehicularemissionsstatusinthepreandpostBRTscenarioinRawalpindiandIslamabadandpeerreviewoffeederroutenetworksinIslamabadetc.Inadditiontoresearchworksprojectalsoprovidednecessarysupportforcapacitybuildingofand coordination among stakeholders through organizing series of trainings, workshops andconsultations on sustainable urban transport systems. Discussionswith stakeholders suggestthat these studies and capacity building interventions were found very instrumental inpromotionofsustainabletransportsysteminthecitiesofLahore,RawalpindiandIslamabad.OneofthemostpromisingachievementsoftheprojectwastheestablishmentoftwoResearchCenters in public sectors universities i.e. 1) Research and Development Unit (R&DU) forSustainableUrban Transport at theUniversity of Engineering and Technology, Lahore and 2)CentreforExcellenceforSustainableTransport(CESTAC)atFatimahJinnahWomenUniversity,Rawalpindi. Though the establishment process took considerable time and the centers havebeenmadeoperational very recently,however it is expected in the longer run these centerswill play a central role in promoting research and development on sustainable transport intimestocome.Discussionswiththesaiduniversitiesofficialsalsosuggestthattheyarealsointheprocessof introducingBachelorandMasterLevelcoursesonsustainable transport in thenearfuture.In nutshell, it can be concluded that despite a number of impediments and complex workenvironment the project has contributed through providing technical assistance in theoperationalization and promotion of sustainable transport systems especially in the cities ofLahore,IslamabadandRawalpindiofPunjabProvinceandFederalCapital.2)OperationalsustainableurbantransportsysteminSindhprovinceUnderthisoutput,projectprovidednecessarysupportfortechnicalresearch,capacitybuilding,coordinationandawareness forpromotionofsustainable transport inSindh.ProjectmainRPfor this output was Sindh Department of Transport and also involved other relevantstakeholders.Project interventions under this output mainly concentrated to support development BRTsystems mainly in Karachi city. In this regard project, in collaboration with stakeholders,


39

expedited a number of studies for promotion of sustainable transport in general and BRT inparticular.Theseincludedinstitutionalanalysisofstakeholder’sinterestsonBRTprojectdesignoptions,surveyof'mini-cabs(Quinquis)onfeederroutesonproposedBRTLines,developmentof Provincial Environmental Quality Standards for the transport sector and study on carbonemissions for the selected BRT line and review of the motor vehicles laws and proposedamendments(e.g.76MotorVehicleLawsreviewedandadopted).Inadditiontoresearchworkstheprojectalsoprovidednecessarysupportforcapacitybuildingofandcoordinationamongstakeholdersthroughorganizingseriesoftrainings,workshopsandconsultations on sustainable urban transport systems. Project established a technicalcommitteecomprisingwiderangeofstakeholders,whichmetonregularly,almostonweeklybasis, and consultations were held on range of sustainable transport related issues andinterventions.Presently physicalwork on twoBRT Lines, in Karachi, is in full progress i.e.Orange Line andGreen Lines, sponsored by GoS and GoP respectively. Discussions with stakeholders suggestthat project technical, capacity building, awareness and coordination related interventionswere found very instrumental in promotion of sustainable transport system in general andproviding support for ongoingBRTprojects in particular.Needless to emphasize that projectclose collaboration and interaction with stakeholders also greatly helped improvingcoordinationamongwiderangestakeholdersOneofthemostpromisingachievementsoftheprojectisthefacilitationofthedevelopmentofSindh Urban Transport Policy. The said policy was formulated in close and rigorousconsultationswithallstakeholdersandbynowitisinthefinalstagesofitsapproval.Themainobjectives of the proposed policy is tomeet the basic access and development needs of allbeneficiaries, to provide a transport system that is affordable and efficient, and to limitemissionsofGHGsandwaste.Onceapprovedandimplementedthepolicywillgreatlyhelpinimproving andpromoting sustainableurban transport systems in Sindh.NeverthelessprojectalsosupportedtheSindhGovernmentinEstablishmentofSindhMassTransitAuthority,whichisconsideredabigachievementregardingstrengtheninginstitutionalframework.In nutshell, despite the late start and complex work environment it can be concluded theprojecthashandsomelycontributedthroughprovidingtechnicalassistanceinthestreamliningand promotion of sustainable transport systems especially in the under completion BRTsystemsinKarachi.3)ImprovedfuelefficiencyintruckfreighttransportUnderthisoutput,projectprovidednecessarysupportfortechnicalresearch,capacitybuilding,coordination and awareness for promotion of improved fuel efficiency in truck freighttransport. Project main RP for this output was Ministry of Communication, Government ofPakistanandalsoinvolvedotherrelevantstakeholders.


40

Interventions under this output mainly concentrated to support implementation of truckingpolicy (2007)andto improveknowledgebase in fuelefficiencyof truck freight. In this regardproject, in collaboration with stakeholders, expedited a good number of studies related toenergyefficiencyfortrucktransport,CO2emissionsfromtruckfrighttransport,bestpracticesintruckenergyuse,environmental impactsoffreightcorridors,strategyforthe implementationof trucking policy, load management systems for truck freight, modernizing motor vehicleregistrationandmodernizedtruckfreightexaminationsystemetc.Inadditiontogooddealofresearchworks,projectalsoprovidednecessarysupportforcapacitybuilding of and coordination among stakeholders through organizing series of trainings,workshops and consultations on trucking policy and improving efficiency of truck freighttransport.Itisimportanttohighlightthatworkonthiscomponentstartedverylate,inMarch2015, therefore despite rigorous efforts work on some of the sub-outputs outlined in therevisedresultsframeworkcouldnotbeexpeditedtheseincludedpilotstoimplementstrategyto modernize trucking fleet and to established public-private partnerships (PPPs) in themodernization of the trucking fleet. Overall the project aim was to reduce GHG emissionsthrough these pilots for modernization of trucks therefore non implementation of theseinterventionshaditsownimplicationsforprojectoveralloutcomes.In nutshell, discussions with stakeholders and review of records suggest that the quality ofthesestudiesandcapacitybuildinginterventionswasverygood,astheyweremostlyexecutedthroughvery reputableacademic institutions likeNUSTandNEDetc.Stakeholders,especiallygovernmental institutions, found the technical assistance of project instrumental instreamliningmechanismsfortheimplementationoftruckingpolicyandenhancingknowledgebaseinfuelefficiencyoftruckfreightinPakistan.Projectrecordsalsosuggestthatduetotheeffortsoftheproject,theMinistryofPlanning,DevelopmentandReformshastakenownershipofthetruckingpolicyanditsimplementation,astheministryhastheleadroleinimplementingtheChinaPakistanEconomicCorridor(CPEC).4)IncreasedpublicawarenessandinstitutionalcapacityonsustainabletransportconceptsUnder this output, project provided necessary support for increased public awareness andinstitutionalcapacityonsustainabletransportconcepts.Thisoutputwascrosscuttinginnatureand supported all other three outputs andoverall project outcome. ProjectmainRP for thisoutputwasInternationalUnionforConservationofNature(IUCN)andinvolvedwiderangeofrelevantstakeholders.Interventions under this output were mainly related to awareness raising and building ofinstitutional capacities. A wide range of stakeholders were involved and engaged includinggovernmental institutions, universities, media, politicians, students, transports and localpopulations. A good number of interventions including trainings, workshops, nationalconferences, dialogues, seminars, campaigns, and competitions were implemented involvingforesaidstakeholders.Accordingtoestimatesfromprojectrecordsintotal3958personsamongvarious stakeholders directly participated in these capacity building and awarenessinterventionsof theproject. Inadditionan internationalexposurevisit tostudythetransport


41

system of Seoul, South Koreawas also organized formembers of project staff and relevantpartners.Furthermore,adiversearrayofawarenessmaterialincludingPAKSTRANnewsletter,calendars,Jareeda magazine, posters, stickers, notebooks, mugs, folders and project standees weredeveloped,displayedanddisseminatedamongstakeholdersfromtimetotime.Theprojectalsocreatedandairedadocumentaryon"SustainableUrbanTransport"incollaborationwithPTV.Project website and social media pages were developed and periodically updated andmaintained. Nevertheless project also helped in formulation of guidelines for curriculum ontransport related courses,whichafter finalizationwill beofferedat thebachelor andmasterlevelatFJWUandNEDUniversities.(Pleaseseedetaillistofinterventionsinprevioussection)Discussions with stakeholders and review of project documents suggest that sustainabletransportremainedamuchneglectedsubjectandthereislittleawarenessamongstakeholdersaboutitsconceptsandmodalities.Projectawarenessandcapacitybuildinginterventionswerefoundveryinstrumentalinbringingtheissueofsustainabletransporttolimelightandhelpedthe stakeholders inunderstanding various concepts, dimensions, applications andbenefitsofsustainabletransport.Interventionsunderthisoutputalsoplayedacatalystroleforachievingotherprojectoutputs.3.3.7SustainabilityofProjectInterventionsandResults(EvaluationRating:4-Likely)The ToR defines sustainability in terms of financial resources, socio-economic, institutionalframework and governance, environmental, and overall likelihood. Following is a briefdescriptionofeachofthesecriteria;a) Financial Resources: In recent years Government of Pakistan has been very keen and hasprovided sufficient and timely financial resources for the successful implementation andoperationalizationofLahoreandRawalpindi-IslamabadMetroBusSystems.SimilarlyprovincialgovernmentsofPunjabandSindhavealsoallocatedadequatefinancialresourcesofupcomingBRTsystemsinotherbigcities.FromthesehighlevelsofinterestandinvestmentsinBRT,itcanbededucedthatexistingandupcomingBRTsystemsarelikelytobefinanciallysustainedbythegovernmentinthefuture.On the other hand, project interventions related to research work, capacity building,coordination and awareness raising etc. were found very instrumental and there is still acontinuousneed for such interventions in times tocome.However, it isnotclear that in theabsenceofPAKSTRANsupporthowwill these interventionsbefinanced,asthegovernmentalfunding ismostly dedicated for the hardware. Furthermore, the project has established twoR&DCentersatUETandFJWU,howeverthesecentersareatveryearlystagesofdevelopmentandstillneedtobesupportedtechnicallyandespeciallyfinanciallyintheshorterruntomakethemproductiveandself-sustaining.b)Socio-economic:ProjectinterventionsespeciallyrelatedtoBRTinbigcitiesarefoundhighlysociallyacceptableandaredulyappreciatedbythemasses.Theseservicesprovidethecommon


42

citizena safe, efficient, cost effective, anddecentmodeofurban transport in the congestedurban environment. Therefore, it can be deduced that socially there is no barrier tosustainability.c) InstitutionalFrameworkandGovernance:GovernmentofPunjabhasrecentlyestablishedaMetroBusAuthoritytooperate,maintainandimproveBRTsystems.InKarachi,effortsarealsounderway to establish such an institution to operate and take care of the upcoming BRTsystems. In view of these institutional efforts, it can be deduced that adequate institutionalframeworkandgovernanceislikelytobefurtherstrengthenedandsustainedintimestocome.d)Environmental:Environmentally, theproject is foundmost feasible in improving livingandenvironmental conditions through reduction of GHG emissions, traffic congestion and noisepollutionetc.Thereisnoenvironmentalbarrierinsustainability.e) Overall: In view of the high level of relevance and demand and keen interest, will andsignificantinvestmentsoftheFederalandrespectiveProvincialGovernmentsinBRT,itcanbeeasilyconcludedthatthecompletedandforthcomingBRTsystemswillbeadequatelysustainedand the benefits will continue to flow in times to come. Similarly, the strategies forimplementationoftruckingpolicyalsoenjoyhighlevelofownershipfromtheGovernmentanditislikelythatbenefitswillcontinuetoflowinthefuture.3.3.8ImpactofProjectInterventionsandResultsPAKSTRAN project overall outcome was to reduce GHG emissions associated with urbantransportation.Twomainindicatorswereidentifiedtomeasurethestipulatedoutcomei.e.1)Cumulative, direct CO2 emission reductions in the urban transport sector compared to BAUscenario over a 20-year period (Target: 608 ktonnes CO2) and, 2) Cumulative, direct CO2

emission reductions from a pilot scheme tomodernize the trucking fleet compared to BAUscenario over a 10-year period (Target: 5 ktonnes CO2). Originally, these calculations werebased on direct emission reductions from two BRT demonstrations, one each in Punjab andSindhand,onepilottruckmodernizationinitiativeinvolvingafleetof50trucks.DiscussionandreviewofrecordssuggestthattheproposedBRTdemonstrationsandpilotsformodernizationoftruckscouldnotbeimplementedassuchasenvisagedbytheoriginalprojectdesign. However, it was expected that the required project target for reduction in GHGemissions could be achieved through the two operationalMetro Bus systems in Lahore andIslamabad-Rawalpindi.TheenvisagedGHGreduction target forKarachididnotmaterializeastheestablishmentofOrangeandGreenlineMetroSystemisstillunderprogress.In August 2016, the project completed a study on estimation of greenhouse gas emissionsstatusbeforeandaftermetrobusprojectimplementationalongtheMetroBusCorridorRWP-ISB.ThefindingofthestudyrevealedthatthenetreductioninCO2fromRWP-ISBMetroBusis4486.76tonsannually.Thisreductionisduetoavoidanceofusingthepersonalvehiclesby9%of the current commuters of Metro bus and reduction of another 668 number of public


43

transport vehicles along theMetro route. If theseCO2 figures areprojected for20 yearBAUscenario,thenthetotalreductioninCO2fromRWP-ISBMetrowillbearound89.7ktonnes.PresentlythereisnoestimationsavailableregardingreductionofCO2fromLahoreMetroBus.However,keepinginviewthesimilarnatureofthetwoBRTs(RWP-ISBMetrototallength22.5Km with 24 stations, Lahore Metro total length 27 Km with 27 stations), it can be easilyassumed that Lahore Metro will also reduce CO2 emissions at least by the same amount.Therefore,itisexpectedthattotalreductioninCO2fromthetwoBRTsinRWP-ISBandLahorewillbearoundtotal179.4ktonnesascomparedtoBAUscenarioovera20-yearperiod.ThesecalculationsprovethatBRThasveryeffectiverole inreductionofCO2emissions.However, inview of the results of the abovementioned study, it is also important to highlight that theoriginalprojecttargetofCO2reductionof608ktonnesfromthetwoBRTdemonstrationswascalculated too ambitiously. On the other hand, the second indicator for CO2 emissionreductionsfromapilotschemetomodernizethetruckingfleetcouldnotbeachievedduetonon-implementationofthepilotformodernizationoftruckingfleet.


44

4.MAINCONCLUSIONSANDRECOMMENDATIONSBasedonthedetailedanalysisandfindingsoftheevaluationexercisefollowingarethemainconclusionsandrecommendations:a)OverallRelevanceandContinuityofTechnicalSupportPAKSTRANprojectwashighlyrelevantandinstrumentalinbringingforthandaddressingissuesof sustainable transportandprovided technical support in implementationofBRTsystems inlargeurban centers.However, the issuesof the transport sector are very complex, timeandresourceconsuming.Ontheotherhand,itisalsoimportanttohighlightthatthetruckingsectoralsoremainsoneof theneglectedsectors.Theprojecthashandsomelycontributedto it,buttheworkhasjuststartedandthereisalongroadahead.Recommendation1: It isrecommendedthattechnicalandcapacitybuildingsupporttourbantransport (BRT)andtruckingsectorshouldcontinue. Inthis regard,soundconceptpapers forfutureprojectsshouldbepreparedandsharedwithrelevantdonorsandinstitutionslikeGreenClimate Fund (GCF) etc. to secure necessary funding. For the trucking sector, possibilities ofsecuringfundsfromChinaPakistanEconomicCorridorProgrammeandprivatesectoralsoneedtobeexplored.b)ProjectDesignandResultsFrameworkPAKSTRANProjectwasoriginallydesignedin2010,withtotalbudgetofaroundUSD78Million,including USD 70Million from theGovernment and other partners. However, the stipulatedcontributionsdidnotmaterializeandtheprojectwasimplementedwithbudgetaryresourcesofUSD 7.8 Million, provided by GEF and UNDP. In addition, during project implementation,circumstances of the urban transport sector also kept continuously changing. Overall, theoriginalprojectresultsframeworkwaswellformulatedandexhibitedclearlinkagesamongsub-outputs, outputs and outcomes. However, the changes in budgetary resources andcircumstanceshaditsownimplicationsforprojectdesignandachievementofprojectoutputsandoutcomes.Theproject’sResultsFrameworkwasaccordinglyrevisedandadjusted,buttheoutcomesandoutputsremainedthesame.Recommendation2:Itisrecommendedthatfuturesuchprojectsneedtobedesignedusingatheoryofchangeapproachthroughrigorouslyinvolvingallstakeholdersinallstagesofprojectformulation. Stakeholders’ financial commitments and roles and obligations also need to beclearlydefinedandagreeduponwell inadvance.Projectdesignshouldalsoduly incorporatetheelementsofflexibilitytoallowfordesiredchangesduringimplementation.c)OverallProjectEfficiency,ManagementandImplementationGiventhecomplexityofcircumstances,theprojecthasmaderigorouseffortsandconsiderableprogresstowardsachievingitsoutputsandoutcomes.Asatechnicalassistanceproject,projectinterventionsweremainlygearedtowardsprovidingdesiredsupport intheareasoftechnicalresearch,capacitybuilding,coordinationandawarenessraisingetc.However,theprojectalsofacedanumberofmanagementandimplementationissueswhichconsiderablydelayedprojectstartandimplementationofactivities.


45

Main issues included changes in theproject’s original ImplementingPartner andResponsibleParties, frequent turnoverof theNPDsandproject staff, lackof clarityandunderstandingofexecution and organizational arrangements, lack of coordination/understanding especiallybetween IP and RP in Punjab, late releases of funds and cumbersome work planning,managementandapprovalprocesses.Recommendation 3: It is recommended that in future such projects, partners should beselected carefully keeping in view their relevance, expertise, interest and commitment and,their roles and obligations should be very clearly defined and agreed upon. Similarly,agreement/contracts(LOAs)withallpartnersshouldalsobesignedbeforeprojectinception.Recommendation 4: It is recommended that in the timeframes of future such projects,adequate time should be designated for formation of governing bodies/structures,establishmentofoffices,recruitmentofstaff,procurementofgoodsandservices,formulationof necessary implementation processes and procedures. Needless to emphasize that projectschedules should also provide necessary allowance and flexibility for unforeseen delays androadblocks.Recommendation5: It isalsorecommendedthat insuchprojects,withmatrixorganizationalstructures, where multiple and independent partners are involved, coordination amongstakeholders should be further strengthened. Regular six-monthly and annual review andcoordination meeting/workshops can greatly help in improving coordination amongstakeholders.Ifneeded,technicalcommitteesshouldbeestablishedatthecomponentleveltofurtherfacilitatecoordinationamongstakeholders.Recommendation 6: It is recommended that UNDP project management and operationalprocesses likeworkplanning, recruitment,procurement,monitoringandevaluation,progressreporting and fund releases etc. need to be further simplified and made more swift andefficient.Moreover,UNDPshoulddulytrainandbuildthecapacitiesofstakeholdersinUNDP-PCOMproceduresatthetimeofprojectstartandprovideperiodicrefreshers.d)EffectivenessofProjectResultsThrough implementation of awide range of interventions, project contributed to achieve itsfour outcomes related to sustainable urban transport in Punjab and Sindh, fuel efficiency intruck freight transport and awareness raising. Support was provided in development and/oroperationalizationofBRTsystemsinthecitiesofLahoreandRawalpindi-IslamabadandKarachi.Researchandpolicyworks, institutionalcapacitybuildingandawarenessraisingremainedthehallmarkinterventions.Inaddition,sustainabletransportresearchcenterswereestablishedatFJWURawalpindiandUETLahore,tocaterforfutureneeds.Recommendation 7: It is recommended that project research works and all knowledgeproductsshouldbewidelysharedanddisseminatedtorelevantstakeholdersfortheirreferenceanduse.Presently,theseresourcesareavailableonlineonPAKSTRANwebsite,howeverafter


46

theterminationofprojecteitherthesiteneedstobemaintainedorthesevaluableresourcesshouldbemovedeithertoIUCNorUNDPwebsitetomakethemavailablefortimestocome.Recommendation 8: As mentioned, the project has contributed handsomely to transportrelatedpolicywork.However,thereisalotmoretobedone,thereforeit isrecommendedtorigorously pursue Sindh Transport Policy approval process and finalization of strategies fortruckingpolicy2007.Similarly,developmentoftransportpoliciesforotherprovincesalsoneedstobedulyconsideredandprioritized.e)SustainabilityofprojectinterventionsandbenefitsIn view of the high level of relevance and demand and keen interest, will and significantinvestmentsoftheFederalandrespectiveProvincialGovernmentsinBRTinLahore,Islamabad,Rawalpindi and Karachi, it can be easily concluded that the completed and forthcoming BRTsystemswillbeadequatelysustainedandthebenefitswillcontinuetoflow.Similarly,thepolicyworks also enjoy high level of ownership from the Government. However, in the wake ofupcoming BRT systems in other cities, there is still a good demand and need for projectinterventions related to research work, capacity building and awareness raising in times tocome. Though it is not very clear that in the absence of PAKSTRAN support howwill theseinterventionsbefinanced.Recommendation9:ProjecthasestablishedtwosustainabletransportresearchcentersatFJWUniversityRawalpindiandUETLahore,tocaterforfutureresearchneeds.Thesecentersarestillinveryearlystageofdevelopment.Therefore, it isstronglyrecommendedthatthesecentersshouldbesupportedtechnicallyandfinanciallyintheshorterruntomakethemproductiveandsustainable in the longer run. Furthermore, these centers need to be connected to otherdonors,institutionsandindustryforsustainabilitypurposes.f)OverallImpactofProjectPAKSTRAN project overall outcome was to reduce GHG emissions associated with urbantransportation.Estimatesfrom2016projectstudyongreenhousegasemissionsstatusbeforeandaftermetrobusproject implementationalongtheMetroBusCorridorRWP-ISB,suggeststhatthenetreductioninCO2fromRWP-ISBMetroBusisaround4486.76tonsannually.For20yearBAUscenarioitcomestoaround89.7ktonnes.IfthesameestimatesareappliedtoLahoreMetro, then the assumed cumulative total reduction in CO2will be around179.4 ktonnes ascomparedtoBAUscenarioovera20-yearperiod. It is importanttohighlightthattheoriginalprojecttargetofCO2reduction,fromthetwoprojectenvisionedBRTdemonstrations,was608ktonnes for 20 year BAU. Results of the mentioned study shows that project targets werecalculatedtooambitiously.Recommendation 10: It is recommended that in future such projects, targets related tooutcome and impact indicators need to be realistically determined and should be based onsoundcalculationsandrealisticestimation,keepinginviewitsachievabilitystatusandeaseofmeasurability.


47

Annex-1

Listofpersonsmet/interviewed

Place

1.Mr.AmanUllahKhan,ACD,UNDP2.Mr.UsmanManzoor,ProgramOfficerUNDP3.Dr.SaleemJanjuaNPM,PAKSTRAN4.Mr.HameedAkhtar,ComponentDirector,Trucking,MinistryofCommunication5.Mr.IrfanHaiderComponentManager,Trucking6.Mr.KhanGhulam,M&EOfficer,PAKSTRAN7.Mr.OmarQazi,DirectorBRT,CapitalDevelopmentAuthority8.Mr.AzamLodhi,Director,TrafficEngineering,CapitalDevelopmentAuthority9.Ms.FauziaMalik,ComponentManager,Awareness,IUCN,Pakistan10.Dr.UzraRafiq,Dean,Co-chairCESTAC,FJWU,Rawalpindi11.Dr.SofiaKhalid,EnvironmentalSciences,FJWU,Rawalpindi12.Ms.NaginTariq,ResearchOfficer,CESTAC,FJWU,Rawalpindi13.Ms.AlamasKhurshid,ResearchOfficer,CESTAC,FJWU,Rawalpindi

Islamabad/Rawalpindi

14. Mr. Mahmood Akhtar Cheema, Component Director, Awareness,CountyRepresentative,IUCN,Pakistan15.Mr.FazalKarimKhatri,ComponentDirector,SindhComponent16.Mr.YarMohammad,ComponentManager,SindhComponent17.Dr.Shaber,NEDUniversity18.HajiMohammadAfzal,SindhBusOwnerAssociation

Karachi

19.Mr.NasirJavaid,ComponentDirector,PunjabComponent20.Dr.AbdulSattar,Dean,UET,Lahore21.Dr.AmmadHussain,ChairmanDTE,UET,Lahore,ChairR&DUnit.22.Mr.MohammadAniqGul,Researchofficer,R&DUnit,UET,Lahore

Lahore

Inadditiontoabovemeetings theevaluatoralsohadtheopportunity toattendProjectworkshops/meetingon;• ClimateFinancingOpportunitiesinIslamabad,• PresentationofTruckingrelatedstudiesatNTRC,Islamabad• ConsultativemeetingoftransportsectorstakeholdersinSindh,

Karachi


48

ANNEX-2

TERMINALEVALUATIONTERMSOFREFERENCE

INTRODUCTION

In accordance with United Nations Development Programme (UNDP) and GEF M&E policies and

procedures,allfullandmedium-sizedUNDPsupportedGEFfinancedprojectsarerequiredtoundergoa

terminalevaluationuponcompletionof implementation.These termsof reference (TOR) setsout the

expectations for a Terminal Evaluation (TE) of thePakistan Sustainable Transport (PAKSTRAN)project

(ProjectID:PIMSNo.3953).

Theessentialsoftheprojecttobeevaluatedareasfollows:

PROJECTSUMMARYTABLE

ProjectTitle: “PakistanSustainableTransportPAKSTRANProject”

GEFProjectID: 3539 Projectfinancing atendorsement(MillionUS$)

atcompletion(MillionUS$)

UNDPProjectID:

PIMS3953 GEFfinancing: 4.8 $4.8

Country: Pakistan UNDPcontribution:

$3 $3

Region: AsiaandthePacific Government: $64.360

FocalArea: ClimateChangeMitigation

WorldBank

JICA

3

2

FAObjectives,(OP/SP):

SustainableTransportSystem

Totalco-financing: 69.360

ExecutingAgency:

MinistryofWater&Power,Governmentof

Pakistan

TotalProjectCost: $78.020 $7.8

OtherPartnersinvolved:

• GovernmentofPunjab

• GovernmentofSindh

• Ministryof

ProDocSignature(dateprojectbegan): June2011

(Operational)ClosingDate:

Proposed: Actual:


49

Communications• IUCN

30September2016 31December2016

OBJECTIVEANDSCOPE

PakistanSustainableTransport(PAKSTRAN)project(ProjectID:00072773;PIMSNo.3953)isaninitiative

ofUNDP-GEFandtheGovernmentofPakistanthataimstoprovidetechnicalassistancetoreducethe

growthof energy consumption and related greenhouse gas (GHG) emissions from transport sector in

Pakistan, while simultaneously improving urban environmental conditions and improving Pakistan’s

tradecompetitiveness.

TheglobalobjectiveofPAKSTRANistoreducetheGHGemissionsfromtransportsectorinPakistan.The

developmentalobjectivesofPAKSTRANare:toimproveurbanenvironmentalconditions(i.e. improved

airquality,urbanmobility,equity,cityaesthetics);andtoimproveenergysecurityforPakistan.

Moreover,theprojecthasfourcomponents(toachieveoutcomes),whicharegivenbelow:

Outcome1: An operational sustainable urban transport system in Punjab province (Punjab P&D

DepartmentistheResponsiblePartner-RPtoachievethisoutcome);

Outcome2: An operational sustainable urban transport system in Sindh province (Sindh Transport

DepartmentistheResponsiblePartner-RPtoachievethisoutcome);

Outcome3: Improved fuel efficiency in truck freight transport (Ministry of Communications is the

ResponsiblePartner-RPforthisoutcome;&

Outcome4: Increasedpublicawarenessandinstitutionalcapacityonsustainabletransportconcepts

(IUCN-PakistanistheResponsiblePartner-RPforthisoutcome).

TheTEwillbeconductedaccordingtotheguidance,rulesandproceduresestablishedbyUNDPandGEF

asreflectedintheUNDPEvaluationGuidanceforGEFFinancedProjects.

Theobjectivesoftheevaluationaretoassesstheachievementofprojectresults,andtodrawlessons

that can both improve the sustainability of benefits from this project, and aid in the overall

enhancementofUNDPprogramming.


50

EVALUATIONAPPROACHANDMETHOD

Anoverallapproachandmethod1forconductingprojectterminalevaluationsofUNDPsupportedGEF

projectshasdevelopedover time. Theevaluator is expected to frame theevaluationeffort using the

criteriaofrelevance,effectiveness,efficiency, sustainability,and impact,asdefinedandexplained in

theUNDPGuidanceforConductingTerminalEvaluationsofUNDP-supported,GEF-financedProjects.A

setofquestionscoveringeachofthesecriteriahavebeendraftedandareincludedwiththisTOR(Annex

C) The evaluator is expected to amend, complete and submit thismatrix as part of an TE inception

report,andshallincludeitasanannextothefinalreport.

The evaluation must provide evidence-based information that is credible, reliable and useful. The

evaluator is expected to follow a participatory and consultative approach ensuring close engagement

with government counterparts, in particular the GEF operational focal point, UNDP Country Office,

project team,UNDPGEFTechnicalAdviserbased in the regionandkeystakeholders.Theevaluator is

expected to conduct a fieldmission to LahoreandKarachi. Interviewswill beheldwith the following

organizations and individuals at a minimum: National Project Director (NPD) and National Project

Manager (NPM) of PAKSTRAN, ComponentManager and Component Director CIU-Sindh, Component

Manager and Component Director CIU-Punjab, Component Manager and Component Director CIU-

Trucking,andComponentManagerandComponentDirectorCIU-IUCN.

The evaluator will review all relevant sources of information, such as the project document, project

reports–includingAnnualPRs,projectbudgetrevisions,midtermreview,projectfiles,nationalstrategic

and legal documents, and any other materials that the evaluator considers useful for this evidence-

basedassessment.Alistofdocumentsthattheprojectteamwillprovidetotheevaluatorforreviewis

includedinAnnexBofthisTermsofReference.

EVALUATIONCRITERIA&RATINGSAn assessment of project performancewill be carried out, based against expectations set out in the

ProjectLogicalFramework/ResultsFramework(seeAnnexA),whichprovidesperformanceandimpact

indicators for project implementation along with their corresponding means of verification. The

evaluationwill at aminimum cover the criteria of: relevance, effectiveness, efficiency, sustainability

1 For additional information on methods, see the Handbook on Planning, Monitoring and Evaluating for Development Results, Chapter 7, pg. 163


51

andimpact.Ratingsmustbeprovidedonthefollowingperformancecriteria.Thecompletedtablemust

beincludedintheevaluationexecutivesummary.TheobligatoryratingscalesareincludedinAnnexD.

EvaluationRatings:1.MonitoringandEvaluation rating 2.IA&EAExecution ratingM&Edesignatentry QualityofUNDPImplementation–Implementing

Agency(IA)

M&EPlanImplementation QualityofExecution-ExecutingAgencies(EAs)-MinistryofWaterandPower(IP),P&DDPunjab(RP),TransportDept.Sindh(RP),MinistryofCommunications(RP),IUCN(RP)

OverallqualityofM&E OverallqualityofImplementation/Execution 3.AssessmentofOutcomes rating 4.Sustainability ratingRelevance Financialresources: Effectiveness Socio-political: Efficiency Institutionalframeworkandgovernance: OverallProjectOutcomeRating

Environmental:

Overalllikelihoodofsustainability:

PROJECTFINANCE/COFINANCE

TheEvaluationwillassess thekey financialaspectsof theproject, includingtheextentofco-financingplanned and realized. Project cost and funding data will be required, including annual expenditures.Variancesbetweenplannedandactual expenditureswill need tobe assessed andexplained. Resultsfrom recent financial audits, as available, should be taken into consideration. The evaluator(s) willreceiveassistance from theCountryOffice (CO)andProjectTeam toobtain financialdata inorder tocompletetheco-financingtablebelow,whichwillbeincludedintheterminalevaluationreport.

Co-financing(type/source)

UNDPownfinancing(mill.US$)

GEF(mill.US$)

PartnerAgency(mill.US$)

Total(mill.US$)

Planned Actual Planned Actual Planned Actual Planned ActualGrants Loans/Concessions

• In-kindsupport

• Government

• WorldBank

• JICA


52

MAINSTREAMING

UNDPsupportedGEFfinancedprojectsarekeycomponents inUNDPcountryprogramming,aswellas

regional and global programmes. The evaluation will assess the extent to which the project was

successfully mainstreamed with other UNDP priorities, including poverty alleviation, improved

governance,thepreventionandrecoveryfromnaturaldisasters,andgender.

IMPACTTheevaluatorswillassesstheextenttowhichtheproject isachieving impactsorprogressingtowards

theachievementofimpacts.Keyfindingsthatshouldbebroughtoutintheevaluationsincludewhether

theprojecthasdemonstrated:a)verifiableimprovementsinecologicalstatus,b)verifiablereductionsin

stressonecologicalsystems,and/orc)demonstratedprogresstowardstheseimpactachievements.2

CONCLUSIONS,RECOMMENDATIONS&LESSONS

Theevaluationreportmustincludeachapterprovidingasetofconclusions,recommendationsand

lessons.

IMPLEMENTATIONARRANGEMENTS

The principal responsibility for managing this evaluation resides with the UNDP CO in Pakistan. The

UNDP CO will contract the evaluators and ensure the timely provision of per diems and travel

arrangements within the country for the evaluation team. The Project Team will be responsible for

liaisingwiththeEvaluatorsteamtosetupstakeholder interviews,arrangefieldvisits,coordinatewith

theGovernmentetc.

EVALUATIONTIMEFRAMEThetotaldurationoftheevaluationwillbe30daysoveratimeperiodoftwomonthsduringDecember2016andJanuary2017.accordingtothefollowingplan:

2A useful tool for gauging progress to impact is the Review of Outcomes to Impacts (ROtI) method developed by the GEF Evaluation Office: ROTI Handbook 2009

• Provinces

Totals


53

EVALUATIONDELIVERABLES

Theevaluationteamisexpectedtodeliverthefollowing:

Deliverable Content Timing Responsibilities

InceptionReport

Evaluatorprovidesclarificationsontimingandmethod

Nolaterthan2weeksbeforetheevaluationmission

EvaluatorsubmitstoUNDPCO

Presentation InitialFindings Endofevaluationmission Toprojectmanagement,UNDPCO

DraftFinalReport

Fullreport,(perannexedtemplate)withannexes

Within3weeksoftheevaluationmission

SenttoCO,reviewedbyRTA,PCU,AFOFPs

FinalReport* Revisedreport Within1weekofreceivingUNDPcommentsondraft

SenttoCOforuploadingtoUNDPERC

*Whensubmittingthefinalevaluationreport,theevaluator isrequiredalsotoprovidean 'audittrail',detailinghowallreceivedcommentshave(andhavenot)beenaddressedinthefinalevaluationreport.AnaudittrailtemplateisavailableinAnnexH.

TEAMCOMPOSITION

The evaluation team will be composed of 1 National Evaluator. The consultant shall have prior

experience inevaluatingsimilarprojects. ExperiencewithGEF financedprojects isanadvantage.The

evaluatorsselectedshouldnothaveparticipatedintheprojectpreparationand/orimplementationand

shouldnothaveconflictofinterestwithprojectrelatedactivities.

TheTeammembersmustpresentthefollowingqualifications:

• Minimum10yearsofrelevantprofessionalexperience;

• KnowledgeofUNDPandGEF;

• Experienceworkinginevaluationofsimilarkindofprojectswillbeconsideredanasset.

• Previousexperiencewithresults-basedmonitoringandevaluationmethodologies;

• Technicalknowledgeinthetargetedfocalarea(s):Transport,Sustainabilityandrelatedfields;

• Projectevaluation/reviewexperienceswithinUnitedNationssystemwillbeconsideredanasset;

• AMaster’sdegreeinclimatechangerelateddiscipline,environmentalscience/management,

transport,urbanplanningorothercloselyrelatedfield.


54

EVALUATORETHICSEvaluationconsultantswillbeheldtothehighestethicalstandardsandarerequiredtosignaCodeof

Conduct(AnnexE)uponacceptanceoftheassignment.UNDPevaluationsareconductedinaccordance

withtheprinciplesoutlinedintheUNEG'EthicalGuidelinesforEvaluations'

PAYMENTMODALITIESANDSPECIFICATIONS

% Milestone10% FollowingsubmissionandapprovalofTEmissioninceptionreport40% Followingsubmissionandapprovalofthe1STdraftterminalevaluationreport50% Followingsubmissionandapproval(UNDP-COandUNDPRTA)ofthefinalterminal

evaluationreport

APPLICATIONPROCESS

Applicantsarerequestedtoapplyonlineonthefollowinglink:

http://jobs.undp.org/ by 1st November 2016. Individual consultants are invited to submit applications

togetherwiththeirCVforthesepositions.TheapplicationshouldcontainacurrentandcompleteC.V.in

English with indication of the e-mail and phone contact. Shortlisted candidates will be requested to

submitapriceofferindicatingthetotalcostoftheassignment(includingdailyfee,perdiemandtravel

costs).

UNDPappliesafairandtransparentselectionprocessthatwilltakeintoaccountthecompetencies/skills

oftheapplicantsaswellastheirfinancialproposals.Qualifiedwomenandmembersofsocialminorities

areencouragedtoapply.


55

Annex-3:SummaryofFieldVisits• 16-28December2016:Islamabad-Rawalpindi(Meetingswithstakeholdersandattendedproject

workshops)• 29-31December2016:Lahore(MeetingswithstakeholdersinPunjabProvince)• 03-06January2017:Karachi(MeetingswithstakeholdersinSindProvince)

FordetailsofpeoplemetpleaseseeAnnex-1

Annex4:ListofDocumentsReviewed

• ProjectDocumentsignedVersion• AnnualProgressReports• ConsolidatedProgressReports• AnnualWorkPlans• MinutesoftheProjectBoardMeetings• Mid-TermReviewReport• FinancialStatements• RevisedResultsandResourceFrameworks• Projecttechnicalstudies,TrainingReportsetc.


56

Annex-5:EvaluationMatrix


57

Annex-6:EvaluationQuestionsforTEofPASTRANDue to the qualitative nature of methodology employed no structured quantitativequestionnairewasused,howeverfollowingarethemainquestionsdiscussedwithrespondentsduringindividualandgroupdiscussions:

1. RelevanceandDesign• Whatisthepresentlevelofrelevanceoftheproject?• Aretheprojectoverallobjectivesconsistentwith,andsupportiveofPartner

Governmentpolicies?• Doestheprojectrespondtotheneedsofthekeypartners?• WastheprojectalignedwithgovernmentandUNDPpriorities?• Wastheprojectappropriatetothelocalcontext?• Aretheprojectobjectivesandresultsclearandlogical,anddotheyaddressclearly

identifiedneeds?• Aretheresuitableandinformativetargets,e.g.aretheySpecific,Measurable,

Achievable,RealisticandTime-bound(SMART)?• Aretheactivitiesandoutputsplannedappropriatelytoachievetheprojectoutcomes?• Havekeystakeholdersbeeninvolvedinthedesignprocess• Arecoordination,managementandfinancingarrangementsclearlydefined• Aretheobjectivesclearlyunderstoodbytheprojectpartners?• Howwelltheprojectdesignadaptedtochangingsituation?

2. Effectiveness:• Howwelltheprojectisachieveditsplannedpurposeandresults?• Arethetargetsfortheprojectappropriate?• Whatisthequalityoftheachievedresults?• Aretherisksandassumptionsholdingtrue?Areriskmanagementarrangementsin

place?

3. Efficiency:• Howwellareinputs/resourcesbeingmanagedandusedinthebestpossibleway?• Areinputsprovided/availableontimetoimplementactivities?• Towhatdegreeareinputsprovided/availableatplannedcost(orlowerthanplanned)?• Areprojectresourcesmanagedinatransparentandaccountablemanner?• Howwellistheimplementationofactivitiesmanaged?• Towhatextentareactivitiesimplementedasscheduled?Aretherearedelaysandwhy• Arefundscommittedandspentinlinewiththeimplementationtimescale?• Haveallpartnersbeenabletoprovidetheirfinancialand/orothercontributions?

4. Sustainability:• Iftheservices/resultshavetobesupportedinstitutionally,arefundslikelytobemade

available?Ifso,bywhom?


58

• Whatisthelevelofownershipoftheprojectbykeypartnersandwillitcontinueaftertheendofexternalsupport?

• Howfartheprojectisembeddedinlocalstructures?• Whatisthelikelihoodthatkeypartnerswillcontinuetomakeuseofrelevantresults?• Arethematerial,servicesandequipmentsupportlikelytocontinueaftertheprojecthas

finished?• Istheprojectsociallyandenvironmentallysustainable

5. Impact:• Whatarethedirectimpactprospectsoftheprojectatoverallobjectivelevel?• What,ifanyimpactsarealreadyapparent?• Arethetargetsrealisticandaretheylikelytobemet?


59

Annex-7:RatingScales


60

Annex-8


61


62

TerminalEvaluationReportof

PakistanSustainableTransportProject(PAKSTRAN)

UNDPProjectID:3953,GEFProjectID:3539

PreparedBy

NisarAhmadKhan

IndependentConsultant

[email protected]

January2017CommissionedbyUNDPPakistan