developing good practice for measuring pssm success, effectiveness and impact
TRANSCRIPT
-
8/15/2019 Developing Good Practice for Measuring PSSM Success, Effectiveness and Impact
1/16
Practical Disarmament Initiative
Developing good practice for
measuring the success, effectiveness
and impact of PSSM
-
8/15/2019 Developing Good Practice for Measuring PSSM Success, Effectiveness and Impact
2/16
This document summarises key themes of a workshop undertaken as part of an initiative supported by the UN Trust FacilitySupporting Cooperation on Arms Regulation (UNSCAR). Every effort has been made to capture and fairly represent input;however, comments and views have not been attributed to participants. Comments and views expressed in the report are thesole responsibility of the participants and do not necessarily reflect the official positions of MAG or UNSCAR. If this report orextracts from it are used, the date of the event, as well as MAG and support from UNSCAR, should be credited.
Written by: Chris Loughran, Director of Policy, MAG
Edited and designed by: Mike Fryer, Gayle Gabe, Jessica Riordan and Portia Stratton
Published by: MAG, Manchester (United Kingdom). May 2016
Contact: [email protected]
Photographs: © MAG/Sean Sutton
Introduction......................................................
Setting the scene: PSSM drivers and the
need to measure success, impact andeffectiveness.....................................................
Outputs and outcomes: framing andmeasuring PSSM’s contribution.................
Identifying and overcoming barriers tosuccess...............................................................
Two potential approaches for measuringthe effectiveness and impact of PSSM....
Conclusions and recommendations.........
Contents
3
5
8
11
12
14
-
8/15/2019 Developing Good Practice for Measuring PSSM Success, Effectiveness and Impact
3/16
MAG convened a two-day expert meeting inLondon (United Kingdom) in March 2016, aspart of the Practical Disarmament Initiativeproject. The meeting considered approaches
to measuring the success, effectiveness andimpact of international assistance in PhysicalSecurity and Stockpile Management (PSSM).This report summarises the key topics and themes identified, along with recommendations for policy and practice.
The meeting was the main component of aproject supported by the UN Trust FacilitySupporting Cooperation on Arms Regulation(UNSCAR).
Forty-five experts and stakeholders participatedin the meeting, representing states providingand receiving international cooperation andassistance, regional organisations, non-governmental organisations (NGOs), policy andresearch institutes, academia and different partsof the United Nations system.
The meeting considered PSSM relating to both
weapons and munitions. It had two principalobjectives:
• To identify potential good practicefor considering and measuring success,effectiveness and impact in internationalcooperation and assistance in PSSM, including
potential indicators.
• To promote and support dialogue andcooperation between PSSM stakeholders,as well as between regions and sub-regionsengaged in PSSM.
The meeting was divided into three mainworking sections. The first aimed to ‘set thescene’, looking at the rationale behind PSSMand need to establish good practice around
measurement of its impact and sustainability.It covered the growing prominence, scale andmaturity of PSSM, the increasing levels ofscrutiny over the impact of donor funding andopportunities provided by the recent adoption ofthe 2030 Agenda for Sustainable Development.
The second section considered the utility oftypical outputs used for PSSM assistanceprojects in the context of stakeholder needs at
the national, regional and international level.It identified a set of common output indicators
Introduction
3
-
8/15/2019 Developing Good Practice for Measuring PSSM Success, Effectiveness and Impact
4/16
for PSSM projects, but also highlighted theirshortcomings as indicators of sustainabilityand measurable contribution to broader effortsto address the illicit trade in arms. Discussionsdrew on and benefited from Life CycleManagement approaches.
The third session aimed to translate discussionsinto tangible actions to enhance good practicein PSSM assistance and cooperation. It wasbased around working themes identified byparticipants from earlier sessions. Themesincluded nation ownership and planning, linksto broader security and development agendas,partnerships and sustainability of changeachieved through by PSSM assistance.
This report follows the structure and workingsessions of the meeting. It then outlinestwo complementary approaches that couldbe used to measure PSSM’s impact andenhance effectiveness. It concludes withrecommendations for potential approaches andactions that would improve the collective abilityto frame and measure their impact, and alsoenhance the effectiveness of assistance.
Conclusions and recommendations have beencompiled by MAG, based on the discussions andpriority areas raised during the meeting.The meeting was informal in nature. It was heldin French and English and under the ChathamHouse Rule. This report therefore does notattribute any specific comment to individualparticipants.
4
The meeng aimed to promote the parcipaon of women in all aspects of its design and delivery. Overall, 37%
of parcipants were women, with women being underrepresented among delegates from states and regionalorganisaons. Fiy ve percent of working sessions were chaired and facilitated by women. Twenty seven percent of
speakers were women.
• Twenty-two members
of non-governmental
organisaons, policy
and research focused
organisaons, and
academia.
• Fieen representaves
from nine states involved
in giving or receiving
internaonal cooperaon
and assistance in PSSM.
States represented three
regions.
• Two representaves of
regional organisaons
acvely engaged in PSSM
acvies and broader
eorts to address illicit,
surplus and insecure
weapons and munions.
• Six representaves
from UN enes involved
in PSSM.
WORKSHOP COMPOSITION
-
8/15/2019 Developing Good Practice for Measuring PSSM Success, Effectiveness and Impact
5/16
Growth of PSSM
The last decade has seen a steady increase inthe profile and priority of PSSM as an area of
international cooperation and assistance. Thishas covered PSSM relating to munitions, aswell as small arms and light weapons (SALW).Assistance projects and programmes haveincreased in scale, particularly in West Africa,the Sahel, the CARICOM region and LatinAmerica.
There has been a corresponding increase inthe frequency of requests for internationalassistance with the security and management
of national stockpiles. This has been reflectedin national reporting under the Programmeof Action (PoA). The increase in willingnessby many states to seek external support withPSSM is a marked change from the 1990s andearly 2000s, when stockpile management wasnormally considered highly sensitive.
A ‘community of practice’ has becomeestablished and continues to develop. Assistance
projects supporting national militaries andsecurity authorities are now undertakenby a range of actors, including NGOs and
departments within the United Nations system.This activity sits alongside a long-standingprogramme of bilateral and multilateralmilitary assistance, and also reflects the
growing openness around PSSM needs andpotential benefits.
Rationale behind PSSM assistance
Participants agreed that PSSM cooperation andassistance is driven typically by the followingaims:
• To prevent the diversion of weapons andmunitions from state stocks or custody to the
illicit arms trade, given its role in fueling armedviolence and conflict and its detrimental impacton security, stability and prospects for economicgrowth.
• To reduce the likelihood and/or impact ofunplanned explosions at munitions sites, andthe resulting death and injury, destruction ofinfrastructure and socio-economic cost whichthey frequently cause.
In many instances, international cooperationand assistance in PSSM also aims to strengthen
Setting the scene: PSSM drivers and the need to
measure success, impact and effectiveness
5
-
8/15/2019 Developing Good Practice for Measuring PSSM Success, Effectiveness and Impact
6/16
face a requirement to justify national political,human and financial investment in PSSM. In thecontext of finite and frequently limited nationalresources, PSSM competes with other police andmilitary activities, as well as a broad spectrumof national priorities. Success in the engagement
of national stakeholders and budgetary resourcedepends on the need to demonstrate results andadded value.
Providers of international assistance – includingparts of the United Nations system andinternational NGOs – are also under increasingpressure from donors, governance and oversightbodies to justify PSSM assistance as a priorityarea. For NGOs in particular, results and impact
need to be articulated in terms of a positiveeffect on people, their human security andprospects for opportunity and development.
Demonstrating success against measurableaims and objectives
Understanding the impact of activities andtesting the assumptions which underpinprojects and partnerships is an established partof international cooperation and assistance.
It is a vital component of accountabilityto stakeholders, and also a foundation ofmonitoring and evaluation activities. Evaluationfundings can be used to identify and promotegood practice, but also identify areas whereapproaches can be improved to enhance theeffectiveness of partnerships.
PSSM is a comparatively new area of structuredinternational cooperation and assistance,
with some partnerships being only months ora few years old. Projects are often driven byassumptions relating to PSSM’s contributionto diversion prevention, supported by successmetrics articulated in terms of tangible projectoutputs.
Assumptions around diversion prevention andreduction in unplanned explosions have beenwell-founded, and there is general consensus
that output-based success measurement hasbeen commensurate with the scale and maturityof PSSM work to date. There is nevertheless
implementation of international and regionalpolitical and convention commitments,particularly relating to the PoA and related sub-regional instruments.
For several donors, PSSM assistance projects
are linked to national security and foreign policypriorities, particularly relating to stemmingillicit arms flows in and across regions andaccess to arms by non-state armed groups. Manystakeholders providing assistance, particularlyNGOs, are driven by the desire to have positiveimpact on people affected by conflict, insecurityand armed violence.
This combined set of aims is frequently shared
by national authorities seeking assistance, butoverlaps in priorities and aims are not alwaysexact.
Accountability and the need to demonstrateresults
All PSSM actors are under increasing pressureto demonstrate relevance and results tostakeholders. For many donors, politicaland financial commitments to international
cooperation and assistance are viewedincreasingly as investments. Continued donorfunding depends on proof of success and impactto parliamentarians, the legislature, taxpayersand the wider public.
Demonstrating the return on investment canlead to continued or increasing funds, while thefailure to do so can prevent further allocationof resources. Cooperation and coordination
between donors and other stakeholders isseen as multiplying the impact and return oninvestment of individual efforts.
The commitment to cooperation partnershipsby states receiving assistance is increasinglygauged by donors in terms of the commitment ofnational budgets to complement internationaldonor funding. This is also linked to assessmentsof the likelihood of change being sustainable
after the inevitable end of donor funding.
National authorities seeking assistance similarly
6
-
8/15/2019 Developing Good Practice for Measuring PSSM Success, Effectiveness and Impact
7/16
PSSM as part of a wider arms control effort
PSSM is part of a broad set of activities whichaim to address the illicit arms trade and itseffects. These are wide-ranging in nature, fromthe development of national legislation to
border control and community participation.This is reflected in the PoA and also a range ofregional and sub-regional instruments, many ofwhich also cover ammunition.
There are a number of current initiatives toavoid duplication of PSSM assistance andimprove the complementarity of differentassistance modalities and projects. Thishas been welcomed by many stakeholders,
particularly given the increase in scale ofassistance. There has not yet been, however, asignificant focus and effort to increase PSSM’sinterlinkage to other areas of arms controlassistance and strategy. One notable exceptionhas been increased coordination betweentechnical assistance in SALW marking andstockpile management and record-keeping onwhich marking’s success depends.
While PSSM activities can have short-term
tangible outcomes – particularly aroundpreventing unplanned explosions or creating anentry point for institutional change – its longer-term success depends on strong links to otherareas of arms control and institutional reform.
a general view that the further growth anddevelopment of international cooperationand assistance in PSSM requires an increasedability to demonstrate results in terms of impactagainst measurable aims and objectives.
The 2030 Agenda for Sustainable Development
The Agenda for Sustainable Development wasagreed by UN member states in 2015, replacingthe Millennium Development Goals (MDGs).The Agenda established 17 new goals – theSustainable Development Goals, or SDGs –along with a range of associated targets andindicators for each goal. The scope of SDGsis broader than that of their predecessors. Of
central relevance to PSSM and broader efforts toaddress the impact of the illicit arms trade, theSDGs make clear links between development,peace, security and arms control.
For many stakeholders, the absence of the clearand stated interlinkage between arms controland development was a significant weaknessin the MDGs. SDG 16 is to ‘Promote just,peaceful and inclusive societies for sustainabledevelopment … and build effective, accountable
and inclusive institutions at all levels.’ Target16.4, associated with this goal, requiresmeasurable and time bound results: ‘By 2030,[to] significantly reduce illicit financial and armsflows, strengthen the recovery and return ofstolen assets and combat all forms of organisedcrime.’
Target 16.4 reflects the main driver behind PSSMactivities – to reduce the diversion of weapons
and munitions to the illicit market. This offersa practical link between PSSM activity andmeasureable impact on illicit arms flows, basedon the principle that illicit arms are detrimentalto development, peace and security. Despitethe 2030 Agenda being global in nature, theframework is based on national implementation,including the development of national baselinesagainst which progress can be measured. Itsimplementation is potentially compatible
with the principle of national ownership thatunderpins all PSSM activity.
7
-
8/15/2019 Developing Good Practice for Measuring PSSM Success, Effectiveness and Impact
8/16
outputs have been used to demonstrate thattangible action is being taken to promote citizenconfidence.
Developing standard output indicators
Outputs for PSSM cooperation and assistanceprojects are typically quantitative and reflectthe operational and capacity building activitiesthat have been undertaken. For example, thenumber of weapons or munitions destroyed, orthe number of explosive store houses that havebeen assessed or rehabilitated.
There is a significant degree of commonality
between the output indicators that are usedby expert organisations providing PSSMcooperation and assistance. Meeting participantsconsidered the extent to which the outputslisted on page 10 could act as a general set ofindicators for assistance providers. There wasgeneral agreement that this set of indicatorsrepresented most of the practical activitiesundertaken as part of operational PSSMprojects.
Several participants suggested that additionaloutputs could add value in terms of representing
Duration of assistance projects and stakeholderneeds
PSSM partnerships have mainly taken the form
of short-term project interventions, typicallywith timeframes between six months to twoyears. In many cases, the timeframe has been areflection of pilot projects, or initial partnershipswhich need to prove the benefit of furtherinvestment. Donor budget cycles based onallocations of funding by individual fiscal yearhave also been a key factor in determining thelength of assistance projects.
Short-term project duration is linked closely
to the focus on outputs as the principalmetric of success in PSSM assistance. Whileacknowledging that PSSM projects fit intolonger-term strategies, donors, nationalauthorities and organisations implementingassistance need to demonstrate tangible resultsand return on investment of individual projects.Outputs have been vaulable justifying thepast and planned expenditure and to feed intostrategic planning.
In contexts where there are high levels of gunviolence and homicide involving small arms,
Outputs and outcomes: framing and measuring
PSSM’s contribution
8
-
8/15/2019 Developing Good Practice for Measuring PSSM Success, Effectiveness and Impact
9/16
assessments as a funded component withinassistance projects, with their results informingsubsequent operational and capacity buildingactivities. Good practice also involvesassessments being undertaken under a principleof national ownership.
A number of tools have been developed byexpert organisations to set technical baselinesduring assessments of individual armouries ormunitions sites. These have typically sought tomeasure the risk and likelihood or diversionor unplanned explosion, drawing on theInternational Small Arms Control Standards(ISCAS) and the International AmmunitionTechnical Guidelines (IATG). This has enabled
the outcomes to be measured at individualfacilities.
Drawing on ‘Life Cycle Management’approaches
PSSM is increasingly considered as part of widerLife Cycle Management of arms. This considerssustainable management of all aspects ofweapons and munitions, from needs andprocurement to management and ultimately use
or disposal.
A Life Cycle Management approach is based onseven interrelated ‘conditions’ being in placeto achieve sustainable change in systems andapproach:1. National normative frameworks
2. Organisational structures and procedures 3. Training and doctrine development 4. Equipment and maintenance
5. Personnel management 6. Finances 7. Infrastructure to implement
These seven conditions have been usedto set a capacity or performance baselineagainst which progress can be measured indifferent areas. Lower levels of capacity orperformance in some areas does not preventcooperation and initiatives from taking place.
Instead, information is used to identify areasof complementary programming which willovercome risks to sustainability. Effectiveness
the reduction in risk of unplanned explosionsat a munitions site. Suggestions included‘the number/tons of unserviceable munitionssegregated from serviceable munitions’, and ‘thenumber/tons of munitions segregated accordingto compatibility groups’.
Benefits and limitations of outputs as a measureof success, effectiveness and impact
There was broad agreement that the outputs onpage 10 add value and meet some stakeholderneeds, particularly in providing quantifiablemetrics to justify short-term expense orinvestment. They also provide data that is usefulfor work plans and for inclusion in national
reports. In some cases, they can contribute topublic confidence building and the perceptionthat action is being taken to address illicitweapons, insecurity and armed violence.
There was, however, also agreement thatactivity-based outputs are insufficient tomeasure the impact and effectiveness of PSSMinitiatives.
Major shortcomings and issues include the
following:• Outputs are typically framed in terms
of activity and do not reflect effects-basedlanguage.• PSSM success depends on a fully functioning
system for management which is not reflected inactivity outputs.• Outputs do not consider the quality of
activities, whether these relate to operationalactivity or capacity building.
• Technical and practical outputs do notidentify risks to the sustainability of knowledge,skills, practice and systems achieved throughPSSM assistance. • The impact on communities, their securityand their perception of security is not reflected.
Baselines and assessments
The introduction of armoury and stockpile
assessments has been a significant developmentin the design of assistance projects. Goodpractice now involves the inclusion of
9
-
8/15/2019 Developing Good Practice for Measuring PSSM Success, Effectiveness and Impact
10/16
Evaluations of projects and programmes inDRC have identified various lessons, includingthe need for closer synergy between markingprojects, national record keeping and PSSM.They have also highlighted the negative effectson the confidence of military stakeholders if
national and international resources are notidentified to follow-up on the outcomes oftechnical assessments.
Several donors are now incorporatingevaluations into more complex assistanceprogrammes. This is a welcomed trend andreflects good practice in broader overseasdevelopment assistance programming.Evaluation findings, lessons learned and
recommendations for good practice are likelyto be useful in a number of regions and sub-regions.
can be measured through sustained changein the seven conditions, based on follow-upevaluation assessments.
Applying good practice in monitoring andevaluation
Many programmes and partnerships arein their infancy, and there has not beensignificant opportunity to date to apply broadergood practice approaches in monitoring andevaluation. A notable exception, highlightedby participants, is the Democratic Republic ofCongo (DRC), where PSSM cooperation andassistance partnerships have been in place fora number of years and a series of evaluations
have taken place. Evaluations have benefitedfrom covering a number of shorter and relatedprojects.
Proposed PSSM project output indicators
Number of armoury assessments conducted
Number of explosive store house sing plans conducted
Armouries refurbished
Armouries constructed
Munions stores refurbished
Munions stores constructed
Small Arms Ammunion Destroyed (
-
8/15/2019 Developing Good Practice for Measuring PSSM Success, Effectiveness and Impact
11/16
take place. An absence or perceived absence
of national ownership would increasingly bea disincentive or barrier to allocation of donorfunding. The allocation of national budgets andsenior level political support for investmentin PSSM are viewed as indicators of nationalownership and ultimately sustainability.
Comprehensive national plans
National plans to address weapons andmunitions should be developed under a
principle of national ownership. Plans are likelyto have the greatest utility and buy-in whenthey are developed through consultation withstakeholders, including donors and providers ofcooperation and assistance. The developmentof national plans could identify synergiesbetween PSSM and other areas of assistance,while identifying gaps that would underminesustainability. National plans could also informrelevant national reporting.
End-states for international cooperation andassistance
If developed through consultation with donorsand other stakeholders, national plans couldbe used more in the design of internationalcooperation and assistance projects. Thiscould also support the identification of end-states for donor support in PSSM, along with
complementary resources and actions thatwould be required to ensure the sustainabilityof knowledge, skills and systems.
A range of cross-cutting themes emerged during
the course of the meeting that were relevant toidentifying and overcoming barriers to short-and longer-term success in PSSM cooperationand assistance.
Avoiding a PSSM ‘silo’
The scale of PSSM assistance has grown, butPSSM is frequently viewed as a primarilytechnical activity that is implementedindependently. Ensuring the quality of PSSM
activities will remain vital. However, the long-term success and impact of PSSM activities willdepend on closer synergies with other areas ofassistance planning.
Scope of stakeholder engagement
Despite increased partnership, coordination andjoint planning, dialogue at the national level isstill primarily between organisations directly
involved in PSSM initiatives. There is scope forengagement of a wider range of stakeholderswho are not directly involved in PSSM, butwhose support and involvement could be criticalto success or failure. National finance ministriesand departments or organisations involved ininstitutional and security sector reform werehighlighted as frequent omissions.
National ownership
National ownership was identified as theprinciple under which PSSM assistance should
Identifying and overcoming barriers to success
11
-
8/15/2019 Developing Good Practice for Measuring PSSM Success, Effectiveness and Impact
12/16
Its associated Target 16.4 seeks to achieve ameasureable reduction in illicit arms flows by2030. The primary aim of PSSM is to contributeto the reduction in illicit weapons and munitions
– primarily through the prevention of diversionto the illicit market – based on their negativeimpact on peace, human security and people’sopportunities for development.
There is a very close overlap between the aimsand rationale of SDG 16 and Target 16.4, andthe theory of change and assumptions whichunderpin PSSM.
The SDG framework and PSSM are also both
based on the principle of national ownershipand implementation, including through thedevelopment of baselines and measurementof progress. The implementation of SDG 16therefore provides a clear framework to considerthe impact of PSSM assistance.
PSSM activities involving state stockpiles do notaim to address illicit weapons and munitionsin isolation; they are part of a wider body of
complementary and related activities andapproaches whose collective success and impactwill be represented in a measurable reduction
The meeting identified two potentialapproaches that could consider and measurethe impact of PSSM, as well as its effectivenessand sustainability. These draw on the SDG
framework, particularly relating to Target16.4, and the incorporation of the Life CycleManagement approaches into the design,implementation and evaluation of internationalcooperation and assistance.
Both approaches are summarised below,with potential obstacles highlighted forfurther consideration. They are presentedas complementary, and have a foundation indemonstrable national ownership. Further, their
implementation would be enabled by longer-term national plans which have been developedthrough consultation, and which involveincreased cooperation with stakeholders in thewider arms control and development sectors.
Approach 1: Linking PSSM to SDG Target 16.4 at the national level
Rationale
SDG 16 aims to promote just, peaceful andinclusive societies with accountable institutions.
Two potential approaches for measuring the
effectiveness and impact of PSSM
12
-
8/15/2019 Developing Good Practice for Measuring PSSM Success, Effectiveness and Impact
13/16
assessment of need, principally relating to riskof diversion. Projects draw on assessments,with operational and capacity building activitiesdeveloped on the basis of their results and inpartnership with national authorities.
Potential implementation
There is scope to broaden existing assessmentsand partnerships with national authoritiesto include an assessment of capacity andperformance across the seven criteria withinLife Cycle Management approaches. This couldbe used to identify risks to sustainability at thedesign phase, as well as potential mitigationapproaches. It could also be used to identify
synergies with other areas of effort in armscontrol and institutional reform, as well as widercooperation and assistance needs.
Assessments against the seven ‘conditions’ forLife Cycle Management would provide a baselineagainst which progress could be measuredand success evaluated through subsequentassessments.
Potential obstacles to overcome
• Follow-up evaluation assessments wouldtypically fall outside of the timeframe ofindividual assistance projects.
• Assessment of the seven criteria mayinvolve successful engagement across multipledepartments or stakeholders.
• Projects may already have been designed and
prioritised by donors or national authorities.
in arms flows. PSSM’s impact should thereforereflect its contribution.
Potential implementation
Existing PSSM assessment tools enable the
measureable reduction in risk of diversionas a result of PSSM activities at specificsites. Progress at the national level could berepresented in terms of progress against anational register of armouries and munitionsstorage sites that comprise the nationalstockpile.
Potential obstacles to overcome
• Absent or incomplete national databases ofstate armouries and munitions storage sites.
• Potential inconsistency in armoury andexplosive store assessments undertaken bydifferent entities.
• Lack of coordination/cooperationbetween PSSM assistance and other areasof arms control, development assistance andinstitutional reform.
• Need for improved dialogue and coordinationbetween multiple authorities with responsibilityfor different elements of state stockpiles.
• Need for improved cooperation and dialoguebetween national arms control authorities andstakeholders involved in implementation of SDG16 at the national level.
Approach 2: Applying Life Cycle Managementmethodologies to the design, implementationand evaluation of international cooperation andassistance in PSSM
As summarised on page 9, Life CycleManagement approaches are based aroundthe presence of seven conditions for effectiveand sustainable management of weapons andmunitions.
At present, many PSSM initiatives assistanceprojects are implemented based on an
13
-
8/15/2019 Developing Good Practice for Measuring PSSM Success, Effectiveness and Impact
14/16
and activity-based, partly a result of projectdurations and annual donor funding cycles.There is also broad agreement that activity-based outputs need to be complemented by
other indicators of success that consider impact,effectiveness, sustainability and links to otherareas of effort and assistance.
4. There is scope to draw on good monitoringand evaluation practice in broader internationaldevelopment assistance. This is vital forthe continued development of PSSM andin identifying and sharing learning. Whereevaluations of PSSM assistance have takenplace, they have identified lessons learned and
potential synergies. Evaluations are increasinglyincluded in longer-term assistance projects.Where programmes comprise a number ofshorter projects, evaluations could covermultiple related initiatives within the samescope of work.
5. Good practice and lessons learned that areidentified through evaluation should be sharedas widely as possible. Every national context
is specific, with unique opportunities andchallenges. There are nevertheless similaritieswithin and between regions.
1. There is a common rationale and theory ofchange behind international cooperation andassistance in PSSM. PSSM aims to contributeto the reduction in illicit arms flows in support
of peace, stability and human security, and tocreate an environment that is conducive forsocio-economic development. It achieves thisprimarily through reducing the risk of diversionto the illicit market. PSSM involving munitionsalso aims to reduce risk of unplanned explosionsat munitions sites and their negative human andsocio-economic impact.
2. PSSM must establish ways to measureimpact and return on investment of donor and
national funds in order to develop further.Donor support to states in PSSM has grown inscale over the last decade, as have requests forassistance in this previously highly sensitivearea. PSSM is nevertheless still based heavilyon assumptions of its impact that need to bedemonstrated.
3. The PSSM community has developed outputs to gauge immediate success of assistance. There
is general agreement around the type of outputindicators that represent PSSM’s activities todate. Outputs have typically been short-term
Conclusions and recommendations
14
-
8/15/2019 Developing Good Practice for Measuring PSSM Success, Effectiveness and Impact
15/16
to sustainability and mitigation actions,and provide a framework for evaluation.Consideration of practical approaches to follow-up assessments and evaluation would be avaluable area for further exploration betweenPSSM stakeholders.
11. National ownership will remain central toachieving PSSM’s aims and ensuring continueddonor support. National budget allocation andvisible political support for PSSM are a cleardemonstration of national ownership, and willbe vital to retain and sustain donor confidence.
12. Longer-term national plans could bedeveloped in many locations, involving a
broader range of stakeholders. This couldincrease sustainability and impact at thenational and international level.
13. National plans will be most effective when they are developed in consultation with donorsand other stakeholders. Consultative planningshould consider end-states to donor assistance,incorporating measures that will enablesustainability when donor funding ceases.This will strengthen partnerships and enhance
the design of international cooperation andassistance initiatives.
6. Expert organisations involved in PSSM(munitions) have developed a range of tools to measure the reduction in risk of unplannedexplosions. Tools draw on the IATG, and can beused to quantify the damage and death or injurythat PSSM has helped to avoid. Maintaining
reduced levels of risk will depend on continuedand effective management of munitionstockpiles.
7. The national implementation of SDGGoal 16 and Target 16.4 gives a tangible andpractical opportunity to demonstrate PSSM’simpact in terms of its measurable contribution to reducing illicit arms flows. SDG 16 andits associated Target 16.4 aim to achieve a
measurable reduction in illicit arms flows, whichis consistent with the rationale underpinningPSSM partnerships.
8. The proportion of armouries and munitionssites where diversion prevention measureshave been effectively undertaken could be abroad indicator of PSSM’s contribution to Target16.4 at the national level. This would dependon increased effort to maintain data on statestockpiles and PSSM initiatives. It should also
be linked to efforts to ensure quality assuranceof PSSM and the sustainability of changes inknowledge, practice and systems. A tangiblenext step could be to pilot the approach andshare findings and lessons.
9. There is scope for increased cooperationbetween actors directly involved in PSSMand stakeholders in broader arms control anddevelopment sectors. Broadening partnership
and cooperation will be essential to measuringPSSM’s contribution to diversion prevention,institutional reform and peace, security anddevelopment. PSSM stakeholders should drawon current initiatives to develop cooperation andcoordination platforms.
10. There is a growing awareness of theneed to systematically identify broader risks to sustainability. PSSM approaches could
draw on Life Cycle Management methodology,particularly the seven ‘conditions’ for successand sustainability. These identify risks
15
-
8/15/2019 Developing Good Practice for Measuring PSSM Success, Effectiveness and Impact
16/16
Practical Disarmament Initiative
Developing good practice for measuring the
success, effectiveness and impact of PSSM