some people met and spoke to each other... so what?
TRANSCRIPT
-
8/9/2019 Some People Met and Spoke to Each Other... So What?
1/29
SOME PEOPLE MET AND SPOKE TO EACH OTHER...
... SO WHAT?
A PROPOSED APPROACH TO MONITORING AND
EVALUATING AN UNOFFICIAL PEACE INTERVENTION
Lucian Harriman
Post Graduate Certificate in Conflict Resolution Skills
University of Coventry December 2008
6,318 Words
-
8/9/2019 Some People Met and Spoke to Each Other... So What?
2/29
2
CONTENTS
Introduction .............................................................................................................. p3
Accountability formal and informal, external and internal .............................. p5
Learning .................................................................................................................... p9
Method and rationale ............................................................................................. p10
Constraints and challenges to implementation ................................................... p14
Investigating and understanding impact and effectiveness ................................ p21
Conclusion ............................................................................................................... p28
Bibliography ........................................................................................................... p29
-
8/9/2019 Some People Met and Spoke to Each Other... So What?
3/29
3
Introduction
Monitoring and evaluation relates to understanding, justifying, accounting for and
assessing an intervention and its relationship to the context that it engages. Adopting
a framework for monitoring and evaluation can provide a systematic means of
supporting learning and accountability in the implementation of a peacebuilding
initiative.1
It requires an agency to explain a projects methodology and its
underlying rationale. It also involves the agency planning and articulating how it will
set out to understand the impact of the intervention, and determine whether or not, or
to what extent, it has been effective or successful. This all relates to how the agency
will obtain, document, interpret and share information related to the project, and also
how information obtained during the course of the project will be used to make
decisions related to the conduct of the work.
This paper proposes an approach to the internal monitoring and evaluation of an
unofficial dialogue project which will be conducted in Sudan over two years, starting
in January 2009, by Concordis International (Concordis, the agency), a small
British NGO.2
The objective of the project will be to facilitate dialogue addressing
sticking points and challenges to the implementation of the Comprehensive Peace
Agreement of 2005, seeking to contribute to the prevention of a recurrence of large
scale armed violence, particularly in the areas of transition between Northern and
Southern Sudan. The primary component of the project will be the facilitation of a
series of low-profile consultations between influential individuals from a range of
1 OECD, 2008, p262
The author of this paper is the Research Manager of Concordis International.
-
8/9/2019 Some People Met and Spoke to Each Other... So What?
4/29
4
Sudanese constituencies. The consultations are intended to encourage cooperative
relationships across conflict divisions whilst developing consensus on potential means
of addressing contentious issues.
The paper suggests that Concordis uses the research process that will determine how
it applies its method during the course of the project as the basis for monitoring. This
means that the collection of information described as monitoring will not simply be
undertaken to facilitate a retrospective evaluation. Rather, its primary purpose will be
to determine and document the decisions the agency makes regarding who it will
bring into dialogue, and on which issues. Therefore the monitoring process will
itself be an inherent element of the projects implementation. It will involve planning
how information will be gathered, documented, interpreted and used to make
decisions.
Making explicit the projects method and rationale, along with any values or
principles which are supposed to guide the work, and documenting how these have
been applied during the course of the project, particularly how they have determined
the making of specific decisions, will provide a basis for internal accountability and a
means of making the work structured and systematic, as opposed to being simply ad
hoc or intuitive. This will be especially important working in a dynamic, largely
unpredictable context, where the agency is accountable formally and informally to
numerous external actors.
Implementing the project will be far from a straightforward, predictable, technical
exercise and planning and evaluation should take into consideration potential
constraints and challenges and possible limitations of the method. It should also be
-
8/9/2019 Some People Met and Spoke to Each Other... So What?
5/29
5
recognised that the project will likely face unanticipated challenges and that there may
be unanticipated effects of the work on the situation in Sudan. An overall objective of
the monitoring and evaluation process should be, therefore, to identify strengths and
limitations of the method, and lessons to inform future work. The extent to which the
project is successful will be identified by tracing outwards how interventions
have impacted upon the broader Sudanese context.
Accountability formal and informal, external and internal
Evaluation is usually seen primarily as a means of accountability, control or
documentation [to] find out whether an activity has been performed as intended
and/or whether the expected results have been achieved3. This is accountabilityfor
something; it relates to whether an organisation has done what it has said it would do
and, also, whether it has achieved what it has said it would achieve. Accountability,
and accordingly evaluation, can be understood in a short-term and functional sense
related to the use of resources and the immediate impact of a project but they can also
take a longer term and strategic approach, related to the broader impact of an
intervention.4
Accountability has been defined as the means by which individuals or organisations
report to a recognized authority (or authorities) and are held responsible for their
actions.5
This is accountability to somebody. It can be argued that an organisation is
more likely to perform well if it is held accountable and its activities are evaluated. In
3 OECD, 2008, p864 Conradi, 19985
Edwards and Hulme 1996 cited in Conradi 1998
-
8/9/2019 Some People Met and Spoke to Each Other... So What?
6/29
6
turn, the way in which an organisation is held accountable, can define standards of
performance.6
There is a difference between formal, official accountability, as defined by contracts
and laws, and informal accountability, which may be defined by the relationships that
an organisation needs to sustain in order to continue to operate.7
NGOs are said to
have multiple accountabilities and, therefore, [d]emonstrating accountability for
NGOs is a process of promoting, bargaining, and negotiating among the various
stakeholders in the NGO work.8
Concordis is accountable to many different entities in both the formal and the
informal sense. What the agency is accountableforis determined by the specific
relationship.
Formally, the agency is accountable to its donors, explicitly by means of contract. It
is also accountable to governmental authorities in Sudan, with whom it will have to
sign formalised agreements, and to whom it will have to provide regular reports on
project implementation. There are rules and guidelines which Concordis has or will
undertake to follow in order to be able to participate in forums for International NGOs
in Sudan. Formalised agreements are also necessary with UN agencies who provide
transport, security and other infrastructural support in Sudan. Naturally, Concordis is
accountable to the law in all of the countries where it operates.
6 Conradi, 19987 IBID8
IBID
-
8/9/2019 Some People Met and Spoke to Each Other... So What?
7/29
7
The agency is also dependent on relationships with many other groups and actors,
each of which needs to be managed slightly differently. This could be described as
informal accountability. Concordis is to some extent dependent on informal
relationships with sympathetic individuals in Sudanese organisations close to
government for indigenous political support and facilitation of its activities. This
includes individuals working in governmental and quasi-non governmental
organisations in Sudan. Concordis also relies on relationships with other international
NGOs working on similar issues, in order to access information essential for the
success of its work, and also for other forms of support. Ultimately, for Concordis to
operate at all, it relies on people agreeing to participate in its consultations. There is
therefore a degree of informal accountability to participants, on whom the agency is
dependent if it is to function.
Overall, Concordis aspires to be acting in the interest of all Sudanese, or at least those
who are negatively effected by violent conflict. Whether or not there is a mechanism
of accountability to the broader population is questionable, although Concordis is
formally accountable to governing authorities. However, one could argue that out of
security necessity Concordis is informally accountable to the people located in all of
the areas where it operates. Acceptance is recognised as one of the main
components of NGOs security strategies when operating in violent environments.9
Concordis relationship with its neighbours and the way it is perceived as it carries out
its activities will affect the agencys security and its overall ability to operate.
Therefore it is in Concordis interest to account publicly for what it is doing, in order
to allay suspicions, and maintain positive public relations. This involves being aware
9Van Brabant, 2000
-
8/9/2019 Some People Met and Spoke to Each Other... So What?
8/29
8
of how local communities or specific actors perceive Concordis, and not only to how
Concordis conveys itself to them. It involves interaction, listening and reciprocation.
In order to be able to manage so many different and complex strands of formal and
informal accountability to external actors, Concordis could benefit from ensuring that
it is first and foremost accountable to itself. Representations of the work may need to
vary in emphasis and detail when provided to external actors, but Concordis should
also have its own internal, consistent and frank understanding and narrative of the
principles which have guided its decisions and the factors which have constrained or
enabled its operations. This should be the basis from which accounts to external
parties, and external evaluators, are drawn. This paper, therefore, concerns itself with
how Concordis is accountable to itself- how it conceives and applies its method, how
it makes decisions and how it seeks itself to understand the impact of its work.10
To facilitate and substantiate this kind of internal accountability, information should
be recorded in a way that enables Concordis to account for its decisions. The agency
should record, for example, why it has chosen to approach a certain individual to
participate in a consultation. The record should note the principles that determined
the decision as well as the pieces and sources of information that informed it. This
will encourage Concordis to be systematic and principled in its decision-making and it
will enable the agency to trace and show how and why it has acted. This will form a
resource for retrospective evaluation as well as a means of disciplining decision-
making.
10
This is for ethical as well as practical or methodological purposes. There is a real and significantpressure to represent all of ones activities as successful, in order to assert credibility, attract funding
and continue to be able to operate; the income and reputation of an organisation and its staff depend
upon it.
-
8/9/2019 Some People Met and Spoke to Each Other... So What?
9/29
9
Learning
As well as being a source of accountability, evaluation can be a means for learning
and improvement [to] systematise knowledge of results and performance, which can
help improve [the specific project in question] or similar activities.11
In this latter
sense it may be used to develop knowledge and theory,12
to serve internal learning
and external sharing.13
Developing theory involves assessing the method applied in a given intervention. A
peacebuilding project may be deemed to have failed as a consequence of poor
implementation of a method, but failure may also be a result of a misconceived
methodology. Therefore, a distinction can be made between implementation failure
and theory failure.14
The evaluation of a peacebuilding initiative should therefore seek to determine to
what extent a method is being consistently applied, as well as whether the applied
method is appropriate.15
Articulating the methodology to be implemented during a
peacebuilding initiative is necessary to ensure accountability and consistency in
implementation, determining if the agency has done what it has said it would do.
Explaining the underlying rationale of the method is necessary to justify why it is
being used in a specific context. Once the rationale is articulated, one can observe
11OECD, 2008, p86
12
Lederach et al, 2007, p213 Church and Rogers, 2006, p17914 OECD, 2008, p4015
Conradi, 1998
-
8/9/2019 Some People Met and Spoke to Each Other... So What?
10/29
10
and compare whether in practice the project has had the effect that was anticipated
and if the logic behind the method was accurately conceived.
Method and rationale
Concordis method involves organising unofficial discussions between individuals
from the various constituencies involved in an ongoing, or potential conflict. These
discussions, called consultations can include up to approximately 30 participants
and last up to five days. They are intended to take place out of the public and media
eye and are supposed to be attended by senior individuals, who can influence policy,
invited in a personal capacity in order to relieve them from pressure to maintain a
particular party line. Social interaction is encouraged during the course of a
consultation and participants, as much as possible, are given equal status within the
consultations microcosm. Participants, well connected with their respective
leaders, and respected within their constituencies, are seen to have the potential of
building bridges across conflict lines on the basis of shared interest. The
atmosphere of the consultations is intended to encourage trust and develop personal
relationships across conflict or social divisions, both of which are hoped to contribute
to the success of negotiations and more general peaceful relations.
Concordis method and rationale correspond to its theoretical understanding of the
dynamics of conflict. Three overarching analytical perspectives from which conflict
and violence tend to be understood, as delineated by Tilly,16
help to inform the
agencys approach. One perspective understands social activity, and by extension
violence and conflict, as driven by self-interested responses to incentives. Another is
16Tilly, 2000
-
8/9/2019 Some People Met and Spoke to Each Other... So What?
11/29
11
concerned with the role of ideas and perceptions in causing violence, and the third
conceives violence and conflict as a consequence of relationships and social
structures. The latter, relational perspective emphasises that both incentive structures
and perceptions are shaped by and exist in the context of social relations. In this
view, restraining violence depends less on destroying bad ideas, eliminating
opportunities, or suppressing impulses than on transforming relations among persons
and groups.17
Concordis explicitly takes such a relational approach and it uses the definition of
peace, and peacebuilding, as the facilitation of non-exploitative, sustainable and
inclusive relationships free from direct and indirect violence and the threat of such
violence.18
The agency recognises nevertheless that a relational approach to
reducing violence depends to some extent on changing perceptions and on creating
incentives, or at least facilitating the perception of incentives, for parties in conflict to
change their individual relationships and for change in broader structures of social
relations to be possible.
Lederach et al, identify four dimensions of conflict related change: thepersonal
dimension, which refers to individuals attitudes and behaviours; the relational
dimension concerns patterns of interpersonal communication and perceptions between
individuals and communities; the structural dimension concerning how conflict
impacts systems and structureshow relationships are organized, and who has
access to power; and the cultural dimension, which relates to the norms that guide
17 Tilly, 200018
Mac Ginty, 2006, p10
-
8/9/2019 Some People Met and Spoke to Each Other... So What?
12/29
12
patterns of behaviour.19
Curle indicates one way in which these dimensions might be
connected, explaining violence as a response to a sense of social and personal
disconnection and an accompanying alienation from our common humanity.20
This
understands personal and relational dimensions of conflict to be largely a
consequence of social and cultural systems and structures.
Concordis approach responds explicitly to the personal, relational and structural
dimensions of conflict related change. The agency facilitates a process designed to
encourage personal reflection and changes in participants interpersonal relations. At
the same time, it engages participants in analytical discussions concerning how to
facilitate structural change in their society. Participants are selected on the basis of
their position within social structures, and it is intended that any potential changes in
relationships and attitudes developed during consultations will be transferred into
the working practices and into the explicit policies of their broader groups and
institutions.
The method corresponds very closely, if not identically, to Interactive Conflict
Resolution as described by Ronald Fisher.21
The method also draws on Lederachs
concept of levels of leadership adopting a middle-out approach seeking to engage
a set of leaders with a determinant location in the conflict who, if integrated
properly, might provide the key to creating an infrastructure for achieving and
sustaining peace.22
19
Lederach et al, 2007, p1720 Curle, 1995, p5421 Fisher, 2005, pp2-322
Lederach, 1997, ch4
-
8/9/2019 Some People Met and Spoke to Each Other... So What?
13/29
13
In its consultations, Concordis seeks to apply thinking about the dynamics of
interpersonal relationships as developed by the Relationships Foundation, which
argues that developing and experiencing relational proximity in five domains,
creates an enhanced quality of relational experience, which can contribute to
outcomes such as trust, understanding, support, accountability or belonging.23
To
elaborate, in the domain ofcommunication, proximity in directness reducing the
extent to which presence is mediated or filtered is said to produce a relational
experience characterised by connectedness. In the domain oftime, proximity is
achieved via continuity, which is said to produce a relational experience ofshared
story. In the domains ofinformation and knowledge, proximity is characterised by
multiplexity, a term used to describe the breadth and quality of information shared
between the parties, which allegedly leads to an experience ofbeing known and
having a mutual understanding. In the domain ofpower, proximity is achieved
throughparity, which is said to be the fair use of power, said to produce the
experience ofmutual respect. Finally in the domain ofpurpose, proximity is
characterised by commonality, the building of shared purpose, which can create the
experiences ofsynergy and unity.24
Concordis sets out to encourage relational proximity amongst the participants in its
consultations by framing discussions towards a common purpose, establishing
continuity of participation is a series of meetings over time, and providing a different
environment to usual for participants to interact, face to face, not only during
facilitated discussions but also during opportunities outside of the sessions to speak
and socialise. The agency seeks to produce parity during consultations by
23 Ashcroft et al, forthcoming24
IBID
-
8/9/2019 Some People Met and Spoke to Each Other... So What?
14/29
14
according all participants equal opportunities to contribute, regardless of their relative
social positions. This is partially achieved by inviting participants to attend in their
personal capacities and also by asking participants not to attribute statements made
during consultations to specific individuals or parties.
Constraints and challenges to implementation
Any realistic attempt to evaluate the Peace Building Initiative should recognise that
implementing the method described above will not be a straightforward, technical,
predictable process. The projects character and success, and whether it is even
possible for it to begin to be implemented, will all be contingent on the extent to
which Concordis is able to negotiate and manage a number of challenges, many of
which will be beyond the agencys control. As much as possible, these challenges
should be anticipated in plans to conduct and monitor the project, and considered in
any evaluation.
Authorization of the process by government
For the project to go ahead at all, it will need to be authorised, politically accepted and
not obstructed by the governments in North and South Sudan. Concordis has
succeeded in registering as an NGO with the Southern Sudan Relief and
Rehabilitation Commission (SSRRC) and with the Ministry of Humanitarian Affairs
in the North. However, a lengthy bureaucratic process remains to be completed with
the Government in the North, including signing a Technical Agreement and obtaining
work and travel permits. Political obstruction and bureaucratic delays could severely
slow down, disrupt or prevent project implementation. In the South, although
-
8/9/2019 Some People Met and Spoke to Each Other... So What?
15/29
15
Concordis has registered with the SSRRC, it is not clear whether further registration is
necessary and if so which ministry should be approached.
The challenge for Concordis in negotiating these issues will be to represent and
describe to government authorities its planned work in a manner that will not arouse
suspicion and will encourage them to allow it to happen. This is not to suggest that
the nature of the project be misrepresented, rather that elements of the project which
would be most appealing to the respective authorities be emphasised as it is described
to them. A further challenge will be to manage effectively and appropriately
Concordis relationships with the relevant individuals within the government
institutions in question. The design, documentation and evaluation of the project
should take these challenges into account and record the approaches that Concordis
takes and the responses of the respective authorities and how this has affected activity.
Recording this will enable the agency to account for how it has negotiated
government constraints.
Acceptance of the agency in its role as facilitator
Although it has nine years of experience in doing so, Concordis should not assume
that it is automatically entitled to mediate or facilitate in the Sudanese context. The
agency will need to establish, or re-establish, relations and credibility and earn the
acceptance and trust of the parties it wishes to engage. Influential individuals will
have many demands on their time and they live in a politically tense environment,
where there is a history of suspicion, animosity, violence and loss. The project will be
trying to addresses politically sensitive issues. Concordis will need to establish
interest in and acceptance of the process, and respond to potential participants
-
8/9/2019 Some People Met and Spoke to Each Other... So What?
16/29
16
questions and concerns, encouraging them to participate themselves. The agency has
existing contacts and relationships with a number of influential individuals from
relevant Sudanese civil society groups, political parties, armed movements and other
individuals, some of whom participated in past series of consultations. Former
participants and other contacts will be approached and included in the forthcoming
initiative. However, Concordis network will still need to develop new relationships.
The processes of identifying relevant actors, approaching them and establishing or
resuming bi-lateral relationships with them should be recorded and considered in the
documentation and evaluation of the project, not only so Concordis can account for
how this has been conducted but also because these meetings will be a source of
initial, baseline data regarding the various parties perspectives and positions.
Consent to the guiding principles of the process
The key characteristics of Concordis approach to peacebuilding are articulated in the
section on method and rationale above. It should be recognised, however, that the
agency has no real power to enforce the principles of the process either inside or
outside of the facilitated meetings. Participants consent is required for the process to
go ahead according to the terms that Concordis intends. The agency cannot guarantee
to participants that others will uphold the rules of the process. They will need to be
confident themselves that this will be the case, or, at least, be prepared to accept the
risk that the rules guiding the meetings might not be upheld.
Non-attribution of statements
-
8/9/2019 Some People Met and Spoke to Each Other... So What?
17/29
17
A good example of a rule exposed to such a risk is the principle of non-attribution of
statements made off the record during the course of consultations. Individuals are
invited to attend consultations in their personal capacity and discussions are supposed
to be conducted under the Chatham House Rule.25
This is supposed to provide
participants with flexibility to explore ideas and positions that they might not be able
to in a highly pressurised public discussion. However, Concordis is not able to
guarantee that participants in the meeting will not report publicly or attribute what
they have heard said in the meetings. The agency can express the intention that
statements are not attributed, and it can undertake to refrain from attribution itself, but
ultimately the broader implementation of this rule rests with participants themselves.
Participants will need to trust Concordis, and more importantly each other, to respect
the principles of the process. Concordis will need to ensure that all participants
understand what the principles are, but also that participants understand the limits of
Concordis ability to enforce them, beyond its own conduct as an organisation. If a
participant feels that they are at risk of being reported and facing negative
consequences for a particular statement, he or she may refrain from articulating it in
the consultation environment.26
This is in a context where arbitrary arrest is a
systematic practice of the national security services.27
How Concordis conveys the
principles of the process and its limited means of enforcement, as well as how
participants react to this, should be documented and reflected upon in the evaluation
of the project.
25
When a meeting, or part thereof, is held under the Chatham House Rule, participants are free to
use the information received, but neither the identity nor the affiliation of the speaker(s), nor that of
any other participant, may be revealed - http://www.chathamhouse.org.uk/about/chathamhouserule/26
This was raised to a member of Concordis staff by a participant of a past consultation, who said thatthe presence of members of state security meant that participants were cautious about what they said
for fear of potential difficulties in the future.27
UNHCHR, 2008
-
8/9/2019 Some People Met and Spoke to Each Other... So What?
18/29
18
Inclusion and exclusion
Concordis will need to restrict the number of participants in each consultation, not
least for practical reasons. However, the agency may encounter pressure to include
individuals or groups who it might not have determined to be relevant or suitable.
Furthermore, Concordis may be unaware of, or choose not to engage, certain groups
or constituencies who have become aware of the process and may wish to participate.
A challenge will be to limit participation without alienating some constituencies or
appearing biased. Those denied the opportunity to participate may become frustrated,
suspicious or even hostile to the process. This could pose a risk to the organisations
reputation, and pose potential political difficulties in conducting the work. At the
furthest end of the spectrum it could pose security risks both to Concordis and to
others participating in the initiative. Planning, documentation and evaluation of the
process should address how Concordis will deal and has dealt with this issue and what
the implications are for the projects implementation and impact.
Maintaining a low-profile
A related challenge is that of managing and minimising publicity associated with
consultations to maintain the low-profile nature of the initiative. Deliberately
minimising publicity or public awareness of the process may arouse suspicion and
compromise some peoples acceptance of the projects legitimacy. Again, how this
issue is managed and the implications of the approach taken, should be addressed in
project planning, documentation and evaluation.
-
8/9/2019 Some People Met and Spoke to Each Other... So What?
19/29
19
Material and political pressures on participants
Concordis consultations seek to provide a neutral, non-pressurised environment for
participants to explore rationally and analytically their common challenges and
consider potential non-violent solutions to their problems. However, even if such an
environment is achieved, it does not eliminate the real material, political, social and
cultural pressures experienced by participants outside of the meetings. It is worth
noting that participants social status, political influence and means of livelihood may
depend upon their position within systems of patronage or their conformity to the
political agendas of their benefactors or constituencies. Therefore one should not
underestimate the significance of such pressures in shaping participants decisions.
This may constrain their opportunities to promote, implement or otherwise transfer
what they may have agreed to be an appropriate policy within the confines of the
consultations.
Producing parity
A central component of Concordis relational approach is that it seeks to promote
parity between participants; that they are able to engage with each other on equal
terms. This is likely to be uncharacteristic of their relationships outside of the
consultations. Within consultations Concordis may be able to afford equal
opportunities to participants to express their views and engage in discussions.
However, there will be power dynamics between participants that Concordis
nevertheless will be unable to redress and some of these may be unknown to the
-
8/9/2019 Some People Met and Spoke to Each Other... So What?
20/29
20
agency. This may relate to potential threats that some participants could pose to
others. Concordis should seek to identify such power-dynamics within its
consultations, if it is to get a sense of the nature of the interaction that it is facilitating.
Such information will likely need to be conveyed by participants themselves, although
they may not feel secure enough to express it openly or even to Concordis on the
promise of confidentiality.
Safety and security of staff and participants
As an overall goal, the project seeks to contribute improving of the security situation
in Sudan, or, at least, preventing its deterioration. Ensuring staff security may on
some occasions limit where staff are able to travel and therefore reduce their access to
some groups or individuals whom the project should ideally be seeking to engage.
The documentation and evaluation of the project should convey if and how the
agencys security policy has affected the implementation of the project.
Ensuring the security of participants is a different challenge, arguably more difficult.
Concordis should try to ensure that it does not expose participants or others to security
threats that they would otherwise not incur. Partially this relates to how the initiative
is perceived and the implications for those involved. It also relates to the choice of
locations of consultations and the means of transport that participants use to get to
them. It may also relate to how participants understand the information they disclose
in consultations will be used. There is a risk that consultations be used as a
surveillance exercise by some parties or even as opportunities by some groups to
attack specific participants. Concordis should at the very least establish a system
whereby it seeks to prevent the disclosure of any information it has recorded that may
-
8/9/2019 Some People Met and Spoke to Each Other... So What?
21/29
21
make participants vulnerable. This should be documented and considered in
evaluation.
Potential lessons and overall limitations
The discussion of these challenges provides an insight into some of the complexities
of implementing the method. The monitoring, documentation and any evaluation of
the project should recognise these complexities, and identify how Concordis has
approached them, to what effect.
All of these challenges indicate the limitations and fragility of the process that
Concordis will be offering. They highlight a defining characteristic of the planned
intervention, that it seeks to provide an opportunity for something potentially
constructive, but it cannot guarantee how it will be used by those engaged.
An outcome of an evaluation of the project may be to draw lessons about how the
agency was able to negotiate challenges successfully and which elements of
Concordis approach were problematic or impossible to implement. Monitoring and
evaluation should also identify unanticipated challenges which arose during the
course of the project. This may produce some generally applicable lessons about
how to manage such challenges and it may also point to elements of the organisations
methodology which might need rethinking or further development.
Investigating and understanding impact and effectiveness
-
8/9/2019 Some People Met and Spoke to Each Other... So What?
22/29
22
Whereas, the section above has described some of the challenges to the
implementation of Concordis method, the following section addresses how the
agency should investigate and indentify the impact of its activities.
There a number of challenges to identifying the impact of a peacebuilding
intervention. Relationships and trust are difficult to measure objectively or
quantitatively,28
primarily because they are based in intangible in attitudes and
perceptions.29
Furthermore, [v]arious actors may have diverse or even contradictory
interpretations of an interventions impacts (positive/negative) or relevance, based
upon their own position within the conflict.30
Peacebuilding interventions take place in an open system with a broad range of
factors and actors effecting the context. Therefore, the context is dynamic. Change
in the conflict patterns occurs within a changing environment.31
It has been
suggested that in peacebuilding, long-term [s]trategic accountability is nearly
impossible in part because NGOs operate in contexts in which so many factors are
beyond their control.32
This relates to the challenge of determining causality between
an interventions inputs and a broader change in context. For work intended to
prevent violence there is the additional challenge of proving the counterfactual
situation that would have occurred if an intervention had not taken place.33
Such
challenges make it all the more necessary to be clear about how an agency will seek to
understand the impact of what it is doing.
28 Lederach et al, 2007, p229
Church and Shouldice, 2002, p230
OECD, 2008, p3831 Lederach, 1997, p13632 Conradi, 199833
Lederach et al, 2007, p2
-
8/9/2019 Some People Met and Spoke to Each Other... So What?
23/29
23
When seeking to determine how an intervention has affected a situation, one can look
at a number of different levels of analysis, each a further degree of separation away
from the implemented activities. Investigations at the micro level relate to how
individual participants positions, perceptions or actions have changed as a result of
the activity. Impact at the meso level refers to how participants reference groups
and constituencies have been affected. Identifying impact at the macro level relates
to seeing how the intervention has affected the broader society and context.34
A sensible approach to investigating the impact of Concordis consultations would be
to start at project implementation and the micro level and investigate outwards,
seeking to trace how the meso and macro levels have been affected. It should
also be recognised that changes in the broader context will affect the possibilities and
priorities for project implementation and the characteristics of change at the mico
and meso levels.35
Therefore, it is necessary also to monitor the macro level
situation and try to trace how that affects the situation inwards.
A convention in evaluation is to establish a baseline prior to an intervention in order
to determine how a situation has changed following the activity. This can be done at
the micro level by pre-testing the attitudes of participants and at the macro
level by analysing the characteristics of the context prior to an intervention.36
For Concordis Peace Building Initiative establishing a baseline and investigating
and monitoring the overall context, as well as the projects implementation and
outcomes, will all be integral elements of the implementation and development of the
34 USIP, 200435 Lederach et al, 2007, p5936
USIP, 2004
-
8/9/2019 Some People Met and Spoke to Each Other... So What?
24/29
24
work. They will be necessary to determine how the agency decides to apply its
method, and so, more than just being part of a retrospective assessment of the project,
these activities will actually determine the direction the work takes.
Concordis will carry out a research process to determine what should be the specific
issues addressed in the consultations. The agency will also need to investigate and
identify which individuals would be relevant and significant participants in the
dialogue process.37
Bilateral engagement with potential participants will be necessary
to establish, or re-establish, credibility with them and also to secure their interest and
commitment to participating in consultations. These initial phases of the projects
implementation will be useful in collecting baseline data, but they would be
necessary in any case.
The initial thematic development and participant identification process will involve a
review of relevant analytical literature and a series of consultations with individuals in
donor, diplomatic, peacebuilding, analytical and other relevant communities. These
will contribute to the projects baseline analysis of the overall macro context.
Identification of potential participants will also include a review of Concordis
archive and database in order to identify with whom the agency already has a
relationship.
The baseline at the micro and meso levels - concerning the positions,
perceptions and actions of specific participants and their constituencies - will be
developed during initial bi-lateral discussions prior to consultations, as well as from
the statements they make during the first consultation.37
This process is already underway.
-
8/9/2019 Some People Met and Spoke to Each Other... So What?
25/29
25
Decisions regarding who should participate in subsequent consultations and which
issues should be their focus, will be based on conclusions articulated by participants
during consultations, further bi-lateral discussions with participants and others, and on
analysis of the overall context and political priorities in Sudan.
Over the course of the project, data for micro and meso analysis will primarily
come from meetings and interviews with participants, before, during and after
consultations. These will also inform context analysis, but context analysis will also
be informed by consultations with other international actors and by reviewing relevant
media and publications.
As mentioned above, the primary reason for monitoring and analysing the context,
will be to inform decisions about the conduct of the project. The primary logic for
recording this information, will be to discipline decisions and ensure a record of how
decisions have been made. Records from specific meetings, and records of relevant
points of reviewed literature, will be stored in a way in which Concordis can retrieve
them in order to provide evidence of the process and sources which informed its
decisions. This information will at the same time provide a means for the agency to
attempt to determine the relationship between the project and the context, both
whether the project is affecting the context, and, how the context is affecting the
project.
Identifying success
-
8/9/2019 Some People Met and Spoke to Each Other... So What?
26/29
26
Rather than seeing success as something absolute, it would seem more useful and
appropriate to understand success in terms of degrees or increments. Success is an
arbitrary determination of progress and can be set at any point along the continuum in the
desired direction of change.38 Considering success in terms of the extentof a projects
impact enables an observer to identify its achievements and its limitations, possibly
tracing which factors constrained the work. This can help draw lessons about the scope
and overall utility of the method in question.
A degree of success could be attributed to the projects activities simply being conducted
as planned. If consultations happen and are felt to have conformed to recognised
principles of good practice, this could be identified as evidence of successful, or at least
correct implementation.39
However, this would not verify whether consultations had produced what was intended.
As mentioned above, Concordis broadly hopes to build relationships and develop
consensus. Observations of proceedings, any concluding statements, and participants
reflections and feedback on consultations could all shed light on whether, within the
meetings, either of these objectives were achieved.
This still, however, would not indicate whether the project had made a difference at the
broader level of society. As noted above there are different types of conflict related
change which Concordis seeks to induce. The agency should therefore investigate if any
of these types of changes have taken place, at which levels, and if they can be attributed
to the intervention. As mentioned above, this could be achieved by working outwards
from the project, starting with observations of proceedings and discussions with
38 Church and Rogers, 2006, p1239
Berry Associates, 2006
-
8/9/2019 Some People Met and Spoke to Each Other... So What?
27/29
27
participants, and investigating how a broader impact has followed, if at all. This could be
described as tracing transfer: the transmission of developments generated in the
unofficial sphere of the intervention into official negotiations and agreements, which can
includes effects of unofficial processes such as attitudinal changes and new
realizations as well as outcomes such as frameworks for negotiation or principles
for resolution.40
Transfer may be achieved via participants communication to their broader constituencies
of new insights and understandings developed during unofficial consultations; the
development of groups of individuals who, having participated in unofficial
consultations, become willing to engage constructively in official negotiations, and,
following on from this; the introduction into official negotiations of proposals and
frameworks for resolution which were developed in the more flexible context of informal
consultations. These achievements would on changes in attitudes and perceptions made
possible in unofficial contexts, which may facilitate more open and accurate
communication, more accurate and differentiated perceptions and images, increased trust
and a cooperative orientation.41
According to this model, transfer can occur across
decision and policy making interfaces between leaders, official and unofficial
diplomats, negotiators, governmental-bureaucratic constituencies and public-political
constituencies.42
It will be up to Concordis to trace such processes, and identify, if they
exist, the extent of their impact.
However, the notion of success should not be limited to the projects influence on
official negotiations or agreements. One could more broadly investigate the projects
peace impact: how it supports sustainable structures and processes which
40 Fisher, 2005, p341
IBID, p442 IBID, p6
-
8/9/2019 Some People Met and Spoke to Each Other... So What?
28/29
28
strengthen the prospects of peaceful coexistence and decrease the likelihood of the
outbreak, occurrence/recurrence or continuation of violent conflict. One could also
assess it in terms of its violence impact: how it effects the likelihood that conflict
will be dealt with violently.43
Conclusion
This paper has set out some parameters for how its author intends to conduct and
comprehend his and his colleagues work over the coming two years. It is a proposal,
the final approach will have to be agreed by his colleagues and be workable in
practice. Producing systematic and consistent documentation of how Concordis has
made decisions, including the principles and evidence that have informed them, it is
hoped will provide an internal resource that should be useful for the purposes of
internal, or external, evaluation of the Peace Building Initiative. Evaluation could
assess to what extent the agency was able to implement its methodology, how it dealt
with anticipated and unanticipated challenges and how its work impacted on the
situation in Sudan. Evaluation could also produce lessons about the limits of
Concordis methodology and the validity of its underlying rationale.
43
Fisher et al, 2000, p162
-
8/9/2019 Some People Met and Spoke to Each Other... So What?
29/29
BIBLIOGRAPHY
John Ashcroft and others, The Relational Lens: Focussing on Solutions, (forthcoming).
Geoff Berry Associates,Evaluation of WMTTS Mediation Project in Birmingham, May 2006.
Koenraad Van Brabant, Operational Security Management in Violent Environments,
Humanitarian Practice Network, Overseas Development Institute, June 2000.
Cheyanne Church and Mark M. Rogers, Designing for Results: Integrating Monitoring and
Evaluation in Conflict Transformation Programs, Search for common Ground, 2006.
Cheyanne Church and Julie Shouldice, The Evaluation of Conflict Resolution Interventions:
Framing the State of Play, INCORE International Conflict Research
(University of Ulster & The United Nations University), 2002.
Cheyanne Church and Julie Shouldice, The Evaluation of Conflict Resolution Interventions,
Part II: Emerging Practice and Theory, INCORE International Conflict Research (Universityof Ulster & The United Nations University), 2003.
Loramy C. Conradi,Accounting for Success?: Improving Performance Evaluation and
Accountability of Non-Governmental Organizations Engaged in Conflict Resolution,
International Studies Association March 18-21, 1998.
Adam Curle,Another Way: Positive Response to Contemporary Violence Oxford: Jon
Carpenter, 1995.
Ronald J Fisher, ed., Paving the Way: Contributions of Interactive Conflict Resolution to
Peacemaking. Lanham, Maryland: Lexington Books, 2005.
Simon Fisher et al, Working with Conflict. Skills and Strategies for Action. London, Zed
Books 2000.
John Paul Lederach, Building Peace: Sustainable Reconciliation in Divided Societies
Washington DC: US Institute of Peace (1997)
John Paul Lederach, Reina Neufeldt & Hal Culbertson, eds.Reflective Peacebuilding: A
Planning, Monitoring and Learning Toolkit, The Joan B. Kroc Institute for International
Peace Studies, University of Notre Dame, 2007.
Organisation for Economic Cooperation and Development (OECD), Guidance on Evaluating
Conflict Prevention and Peacebuilding Activities Working Draft for Application Period,
Development Assistance Committee, 2008.
R. Mac Ginty,No war, No Peace. The Rejuvenation of Stalled Peace Processes and Peace
Accords. Houndmills and New York: Palgrave Macmillan, 2006.
C. Tilly, Violence viewed and reviewed, Social Research, Vol. 67, No. 3, 2000.
UNHCHR, Tenth periodic report of the United Nations High Commissioner for Human
Rights on the situation of human rights in the Sudan. Arbitrary arrest and detention
committed by national security, military and police, 28 November 2008.
United States Institute for Peace, What Works? Evaluating Interfaith Dialogue Programs,
USIP Special Report 123, July 2004.