November 2018

Building a Stronger Britain Together:

Liverpool World Centre

Call 1 in-depth project evaluation report

[16-050786-01] | Version FINAL | Public | This work was carried out in accordance with the requirements of the international quality standard for Market Research, ISO 20252, and with the Ipsos MORI Terms and Conditions

which can be found at http://www.ipsos-mori.com/terms. © Home Office 2018

1 | 48

Summary

Project scope and aims

Liverpool World Centre (LWC) is a charitable organisation working with schools, young

people and adults across all ages in Merseyside. LWC’s work centres around making world

issues relevant to the lives of young people, using schools and teachers as facilitators to

raising young people’s understanding of global issues and promote social change. The

charity has been running for over 18 years.

LWC sought BSBT Call 1 grant funding of £12,250 to extend their Counter Narrative (CN)

programme. The pre-existing CN programme targets teaching students enrolled at

Liverpool John Moores University (LJMU), and aims to give them confidence to hold

difficult conversations around radicalisation and extremism by running training

workshops and hosting events for networking, dialogue and debate. The programme

delivered at LJMU has its own (non-BSBT) funding. With BSBT funding, LWC set out to (1)

extend their target audience, providing qualified teachers, community leaders, social and

charity workers with these same skills and techniques, and (2) enhance the programme

through hosting additional events for networking, dialogue and debate among this wider

audience. This report focuses solely on the impact of the BSBT-funded CN project.

The CN project was aligned with the BSBT macro-level outcome of “more resilient

communities”.

Project rationale and local need

Merseyside has a long history of migrationi.LWC believes that the area’s enhanced cultural

and ethnic diversity contributes to a continued increase in reported hate crimesii. The CN

project is LWC’s response to what they believe is a growing need for local frontline staff,

charity workers, teachers, social workers and community leaders to hold open dialogues

that better challenge extremist views. They aimed to offer training to these local target

audiences based on a ‘Community of Enquiry’ approach.

The project also explored digital means of creating and sharing stories and resources that

can undermine extreme narratives by bringing activists, community workers, academics

and creativesiii together in knowledge sharing activities.

Evaluation scope

The evaluation covered project activities delivered between March 2017 and April 2018.

Findings are based on monitoring information, analysis of 30 responses to pre/post

training questionnairesiv, and qualitative face-to-face or telephone consultations with five

project participants, two delivery staff and three wider stakeholders.

2 | 48

Project impact

There is evidence to suggest that the expected outcomes for the CN project – consistent

with the macro outcome “more resilient communities “– have been met:

▪ Quantitative feedback showed that participants increased their confidence, skill

and understanding of facilitating dialogues. Similarly, participants reported

increased confidence in dealing with extreme views in conversations.

▪ Based on qualitative feedback from the innovation lab participants, these sessions

led to an improved understanding of extremism and counter narratives.

▪ Evidence suggests that LWC and participants refined their knowledge of and

capacity to create powerful online counter narratives in future projects, and

made meaningful connections with other participants that indicate this increased

capacity will be taken forward in future projects.

▪ Whilst there is no evidence to support the expected outcome of increased digital

and social media skills, this outcome became less of a priority throughout the

course of the project.

What works

Success factors

▪ A good reputation and existing connections with LWC attracted further

partnerships, new and existing, and generated interest in events among local

partners

▪ Commitment and organisational skills of the project lead who demonstrated

the ability to develop and run interesting and thought-challenging events

▪ Ability to make use of LWC staff expertise in facilitating dialogue and using

enquiry methods by introducing participants to dialogue focused education

techniques such as “Open Spaces for Dialogue and Enquiry Methodology”

▪ Benefiting from the expertise of partnering organisations and ability to recruit

interesting organisations/ individuals to contribute to the innovation labs

▪ The project was designed to cascade learning and broaden the reach of the CN

project beyond LWC’s direct influence. This was through asking participants to

carry out further workshops on CN, reaching a different and wider audience.

▪ Bringing together likeminded people and giving them the opportunity to

network (through the innovation labs), people who might otherwise not have met

were able to come together and collaborate both inside and outside of the project

3 | 48

Challenges

▪ Participants recruitment/ attendance – getting enough participants to attend the

workshops and innovation labs was a challenge as events were held at short notice

and during working hours

▪ Engaging a hard-to-reach/vulnerable audience – getting buy-in not only from

people interested in cohesion but also from those who are less engaged with the

community and at risk of developing extreme views

▪ Ensuring uptake of participant-led workshops – putting formal measures in

place to monitor uptake of participant-led workshops proved difficult because

LWC feared this might discourage people from taking part in the CN programme

▪ Finding committed partners to support the facilitation and delivery of innovation

labs earlier in the process would have enabled LWC to schedule in events at an earlier

date, helping with recruitment

▪ Delay in receiving in-kind support – not receiving in-kind support in time to help

with the promoting of the CN events meant that LWC was unable to benefit from

M&C Saatchi’s marketing expertise

4 | 48

1. Project Summary

Aims and rationale

LWC is a local charity based in Liverpool. The organisation specialises in tackling global

issues through education, using different enquiry techniques such as “Communities of

Enquiry” and “OSDE”.

A Community of Enquiry approach consists of a group of people who engage in a

structured conversation around clear concepts identified for further enquiry. This is

facilitated by a trained professional who is in charge of setting clear boundaries for the

conversation, thereby encouraging young people to think critically whilst respecting

others and collaborating with the group. Constructive dialogue involves sharing views,

identifying prejudices, examining reasons, and suggesting an alternative narrative.

The OSDE – Open Spaces for Dialogue and Equity methodology is a complimentary

technique, that offers “a set of procedures and ground rules to structure safe spaces for

dialogue and enquiry about global issues and perspectives focusing on interdependence,

aiming to promote critical and independent thinking and responsible and accountable

reasoning.

The CN project is built on similar work previously run by LWC for trainee teachers enrolled

at Liverpool John Moores University (LJMU) and funded by the university, which aimed to

give them confidence to hold difficult conversations around radicalisation and extremism

by running training workshops. With the CN project, LWC set out to extend their target

audience, providing qualified teachers, community leaders, social and charity workers with

these same skills and techniques, and enhance the programme through hosting

additional events for networking, dialogue and debate among this wider audience. The

ultimate aim of the project is that the trained teachers/ community leaders take their

learning to young people and in turn give them the skills and confidence to counter

extreme views. Most of the training is run by staff members, who have a background in

adult education, drama and teaching.

The organisation works with local schools, universities and other charities. Although the

work of LWC is not limited to a particular group, their focus is ultimately on educating

young people on global issues.

Local issues the project is trying to address

Merseyside has a long history of migration and according to official 2016 ONS figures,

the number of non-UK born migrants living in the area has increased by two thirds in the

last 10 yearsv: from 52,000 of Merseyside’s, then 1.35m residents, to 88,000 out of 1.38m

5 | 48

residents. This rise in cultural and ethnic diversity, combined with 1,697 racial hate crimes

reported to the Merseyside police between June 2015 and January 2017vi, led the LWC to

conclude that there is an existing and growing need to address extreme views. At the

national level, 2016/17 saw – compared to 2015/16 – racially motivated hate crime

increased by 27% (+13,266 recorded offences), and religiously motivated hate crime

increase by 35% (+ 1,549 recorded offence)vii. In this local and national context, LWC’s

proposed solution is to offer frontline staff, charity workers, teachers and social workers,

practical guidance and training to hold open conversations and encourage cohesion

between different groups.

Local context also includes the presence of far-right movements. According to the 2018

Hope not Hate reportviii, although organisationally the far-right movement in the UK is at

its lowest membership numbers for 25 years (approx. 600-700 people), far right related

hate crime and terrorism is on the rise. Notably there seems to be a shift in ideology, with

far-right groups moving away from neo-Nazi ideas to anti-Islam propaganda. This can be

observed in the online content published by some of the established right-wing groups,

such as EDL and National Rebirth of Poland (NOP), which are active in Liverpool and

surrounding areasix. Equally, Liverpool was in the news in 2015 when it was discovered

that four Islamic extremists had links to the LJMUx.

The CN project is designed to address extremism in “all forms”, whether right-wing, jihadi

or other.

Project scope

The CN project focused on tackling extremism by training community leaders and

educators to facilitate open dialogues and give them the skills and confidence to address

extreme views in everyday conversations. Using counter narratives as a tool to tackle

extremism has been widely embraced by governments, think-tanks and NGOsxi.

The training workshops targeted different types of participants in order to reach a broad

spectrum of professionals working with young people. Between March 2017 and April

2018, LWC ran a total of four train-the-trainer workshops as part of the CN project (stage

1) and five innovation labs (stage 2). Upon completion of stages 1 and 2, LWC created a

17-page toolkit aimed to provide practical guidance and support to participants in the

future.

In stage 1, the first workshop took place in June 2017. Across all four workshops (schools

and communities), LWC reached a total of 43 out of 40 anticipated participants. Stage two

was formed of five innovation labs: interdisciplinary sessions aimed to bring together

experts, activists, social workers etc. to discuss and explore different ways of presenting

6 | 48

counter narratives using practical guidance mainly from but not limited to the Institute

for Strategic Dialogue (ISD). Whilst the objective was to bring together approximately 50

participantsxii, LWC engaged a total of 76 participants. Figure 1 shows the project

structure including content for each of the innovation labs, alongside the timeline and

venues.

Figure 1: Counter Narrative (CN) project structure

2. Logic model

A logic model is a diagrammatic representation of a project which depicts the various

stages required in a programme or intervention that are expected to lead to the desired

outcomes. Evaluations use logic models to establish the types of outcomes and impacts

a programme expects to see.

Figure 2 outlines LWC’s logic model based on an initial telephone conversation with LWC

and further refinements agreed during a face-to-face meeting with key staff and Ipsos

MORI.

Counter Narrative project

Stage 1: Training workshops

Community workshops

x2 (Jun & Dec 2017

Teacher workshops

x2 (Jul & Nov 2017)

Stage 2: Innovation labs

Digital Innovation

Lab

Dec 2017 | FACT

Liverpool

Content

- Diversity

- Migration

- Extremism

- Alternative film-making

Media Innovation

Lab

Mar 2018 | University of

Liverpool

Content

- Writing for digital media

- Creating digitial counter-narratives

Dialogue Innovation

Lab

Jan 2018 | Hope

University

Content

- Diversity

- Extremism

- Academic perspectives

DocumentaryLab

Mar 2018 | Unity Theatre

Content

From dialogue to action -documentary technique

Dialogue Lab

Mar 2018 | Unity Theatre

Content

Extremism and terrorism : A real life perspecive (how to challenge extreme views)

7 | 48

The CN project was aligned with the BSBT macro-level outcome of more resilient

communities.

Key desired outcomes included:

▪ Increased confidence & skills in facilitating dialogue

▪ Increased understanding of extremism and counter narratives

▪ Increased confidence in dealing with extreme views in conversations

▪ Increased capacity for LWC and partners to create powerful online counter

narratives in future projects

Figure 2: LWC Counter Narratives logic model

3. Overview of the evaluation approach

The evaluation approach was developed in cooperation with the project lead. The project

lead played an important part in supporting the evaluation by collecting monitoring and

evaluation data, acting as a gate keeper for qualitative consultations and providing

materials for the local context review.

8 | 48

3.1 Evaluation tools employed

The main forms of primary data collection for this evaluation were a quantitative pre-

/post survey, qualitative feedback forms and participant/ staff/ stakeholder interviewsxiii.

1. Quantitative

Participants monitoring data through LWC’s attendance system where participant sign

in to each training/ innovation lab session. This data was later anonymised by the project

lead and entered into an Excel spreadsheet that was shared with Ipsos MORIxiv.

Pre- and post- intervention participant questionnaire: LWC’s own pre and post

questionnaires that had been developed through a previous evaluation programme in

partnership with LJMU were used with training workshop participants (see appendix 1).

Questions were asked in relation to participants understanding of dialogue techniques

and their skills and confidence in dealing with extreme views. LWC used a 10-point Likert

scale and measured success when achieving a 1-point increase. Responses were received

from 30 workshop participants.

2. Qualitative

Feedback forms: LWC designed qualitative feedback forms for the innovation lab

sessions (see Appendix 3). Feedback was gathered around motivation for attending the

event, what perceived as useful and what could have been better.

Ipsos MORI also undertook a series of qualitative consultations.

• Project staff - two consultations (one face-to-face, one telephone) with delivery

staff (project lead and LWC director) to reflect on the project.

• Project participants – five x 60-minute telephone in-depth interviews with

workshop and innovation lab participants

• Wider stakeholders – three x 60-minute telephone in-depth interviews with an

academic involved in education, a local authority staff and a PSHE coordinator at a

local school.

3.2 Strengths and limitations of the evaluation research

Strengths

• Mix of qualitative and quantitative methodology techniques; a range of qualitative

and quantitative evaluation activities were undertaken with project participants,

9 | 48

including qualitative interviews, pre-post surveys and qualitative written feedback

following programme activities

• Existing evaluation materials were of adequate standard as they were developed

with the help of academics at LJMU

• High engagement from delivery staff who were competent and willing to collect

data

• Monitoring data for all workshop and innovation lab sessions was collected and

made available

• Good quality of quantitative and qualitative data due to educational level of

participants

Limitations

• Absence of counterfactual (control group) meaning the evaluation cannot compare

outcomes for participants with what would have happened without the CN

intervention

• Use of existing pre-post questionnaire due to the project starting before evaluation

activities began limited the insight for the quantitative research and does not allow

comparison to other BSBT funded projects

• Pre-/post- questionnaires were only used for the training sessions, whilst a more

general feedback form was used for the innovation labs

• Matching of individual answers was not possible as only aggregated and

anonymised data was shared with Ipsos MORI

• Small overall sample size due to small number of participants means that robust

statistical analysis is not possible; Pre-/post- questionnaires were not used at the

innovation lab sessions; it is therefore not possible to establish a baseline or any

quantitative findings herexv

• As workshops and innovation labs were catered to different participants and

audiences, it was not possible to include someone from all sessions in the

qualitative participant interviews

• Due to small size of the LWC charity and as not everyone was involved in the

running of the project, it was not possible to conduct six staff interviews

10 | 48

4. Key Findings: Outputs/Outcome Evaluation

This section looks at outputs and outcomes achieved. Wherever closely related, they are

grouped together. Outcomes are divided into training workshops and innovation labs.

4.1 Outputs

Desired project output 1: Usage of toolkits facilitating dialogue

As part of the project, LWC decided to collate information and practical guidance

regarding the use of counter narratives in addressing extreme views in an online toolkit.

LWC created a 17-page PDF document, entitled “Counter Narratives Toolkit: A resource

for community and youth workers”xvi that has been shared with Ipsos MORI (see appendix

5). The document provides an overview of different methods for exploring and countering

extremism as well as links to a wide range of existing resources ranging from ISD’s “A

Guide To Countering Far Right Extremism” to materials for teachers and youth workers

(guides by UNESCO/ British Council & Salto).

The document was produced after the final innovation lab concluded and was made

available to workshop and innovation lab participants alike via email. At the time of the

evaluation participants were aware that they would receive the document but had not

received it yet and have therefore not been able to comment on the usefulness of the

toolkit.

Desired project output 2: Number of workshops run by LWC

LWC conducted more events than originally aimed for; they set out to run two training

workshops, one for teachers and one for community leaders. The first workshop took

place in June 2017 and was aimed at community leaders, social workers and volunteers.

The second workshop was aimed at teachers and took place at the Liverpool Hope

University in July. Due to a lower than expected number of participants, LWC decided to

run a second teacher workshop in November 2017 and a second community leader

workshop in December 2017 with MRANG, a local charity for asylum seeker and refugee

women. Over the course of the BSBT funding, LWC ran in total four workshops,

overachieving on their aim of two workshops (see table 1).

Desired project output 3: Number of community leader and teachers trained

LWC aimed to train a total of 40 community leader and teachers across the training

workshops:

11 | 48

Table 1: Overview of BSBT funded train-the-trainer workshops

Workshop Date Type of

participants

Target number

of participants

Achieved

number of

participants

1 28th Jun 17 Community leaders 20 14

2 5th July 17 Teachers 20 11

3 6th Dec 17 Community leaders / 8

4 10th Nov

17

Teachers / 10

Total 40 43

Based on the monitoring data provided by the project and including the two additional

workshops, LWC overachieved by training 43 out of 40 anticipated community leaders

and teachers.

Desired project output 4: Number of community/ school workshops run by trainers

Desired project output 5: Number of people reached through participant led

workshops

These three outputs can be grouped as they relate to workshops run by community leader

and teacher participants of the stage one training. LWC asked participants to take the

training they had received and apply it themselves by running their own workshops. The

project lead followed up with participants after the training workshops and asked those

who had run events to complete a short feedback formxvii. Feedback shows that, at least

2 out of 43 participants used the training they had received and delivered community or

school based workshops, reaching a minimum of 40 additional participants:

Table 2: Overview of trainee/participant-led workshops

Type of workshop Number of

attendees

Type of

participants

LWC materials used

• Liverpool Polish

Supplementary School

• London-based

teachers from EAL and

SEN departments

30 (unknown

whether this is per

workshop or for

both combined)

Supplementary

School teachers /

EAL workers / SEN

teachers

Training PowerPoint

• Team meeting 10 Charity workers Discussion tools

12 | 48

Desired project output 6: Links with local communities/ charities and other

organisations

LWC aimed to engage and create strong links with multiple other local organisations

through the course of the CN project. The underlying motivation was to reach a breadth

of different local organisations and create the foundation for future partnerships. Overall,

LWC engaged with a total of 2 universities, 8 academics, 9 artists/ art organisations, 3

independent experts and the Liverpool City Council (see full list in Appendix 2).

When speaking to wider stakeholders, including local authority staff, it became evident

that LWC was known for being well-established within the area and having a strong

network:

“The World Centre is well-established and well-regarded within the local community” –

local authority staff

LWC not only created these links but encouraged their CN project network to take part in

the running of the CN events, either by hosting, or by making their specific expertise

available. Many of the links were new relationships, specifically sought out and developed

as part of the CN project; these relationships extended the LWC network, adding to their

existing relationships, especially with the local Universities.

The innovation labs in particular benefited from these links as all five were run in

cooperation with other organisations. According to participant feedback, this was a strong

appeal of the CN project, as it allowed participants to come into contact with different

groups and individuals they would otherwise not have met.

“It was good to be able to meet and work with people not from my neighbourhood.” –

Innovation lab participant (qualitative consultation)

At least three participants who took part in the qualitative interviews stressed that one of

the main benefits they took from the project was the ability to network and partner with

“likeminded” people. Based on anecdotal feedback, some participants had already started

working together on new counter extremism projects.

Desired project output 7: Number of schools engaged with LWC

In addition to creating links with local communities, charities and universities, LWC also

engaged with 13 schools. LWC specifically wanted to increase the number of schools

within their network and achieved to form new connections by running training

workshops that were specifically aimed at teachers. Unlike with organisations in output 7,

LWC did not further engage with the schools by asking teachers to take part or host other

13 | 48

CN events but rather primarily regarded these new connections as an opportunity for

teachers to apply their newly acquired skills to educating other teachers and/ or using it

in conversation with their students.

Additional project output 8: Number of participants attending innovation labs

As part of the second stage of the CN project, LWC aimed to run five innovation labs, with

the objective to reach 50 participants across the innovation lab sessions. LWC delivered

all five events and reached a total of 76 participants.

Table 3: Overview of level of participation in Innovation Labs

Innovation lab Subject

Date Target number Achieved

number

Digital Innovation Lab Diversity / Migration /

Extremism / Alternative

Film-making

1st Dec 17 10 14

Dialogue Innovation

Lab

Diversity / Extremism /

Academic perspectives

31st Jan 18 10 10

Media Innovation Lab Writing for Digital

Media / Creating

digital counter

narratives

2nd Mar 18 10 36

Dialogue Lab Extremism and

terrorism – A real life

perspective / how to

challenge extreme

views

21st Mar 18

am

10 8

Documentary Lab From dialogue to

action – documentary

techniques

21st Mar 18

pm

10 8

TOTAL 50 76

4.2 Outcomes

This section uses quantitative monitoring data, results from LWC’s pre- and post- surveys

and findings from the qualitative interviews to look at the outcomes of the project. The

14 | 48

section is divided into outcomes of the training workshops and those of the innovation

labs.

4.2.1 Training workshops

Desired project outcome 1: Increased confidence & skills in facilitating dialogues

➢ Participants are enabled to create strong counter narratives to tackle extremist ideas

➢ Participants are able to develop counter narratives to address negative stereotyping

LWC pre-post survey combined with qualitative feedback indicates that this outcome has

been achieved. Relevant questions, alongside pre-/post- results and percentage point

differences between pre and post surveys can be seen in table 4 below. LWC’s T4B

scoresxviii as presented here are broadly comparable to “agree/strongly agree” on the 5-

point scale used in the standard BSBT questionnairexix.

Please note that different questions were asked for those attending teacher training

versus those attending community leader training, leading to a very small sample sizexx.

Results should therefore be viewed as indicative only.

Table 4: Findings from questions relating to dialogue facilitation from the pre/post survey

PRE

(T4B –

7,8,9,10/10)

POST

(T4B –

7,8,9,10/10)

Difference

Q2 schools: How skilled are you at

using resources and techniques to

discuss extremism?

13% 75% +62 ppts

Q1 community: What are your

current skill levels for facilitating

dialogue?

32% 58% +36 ppts

Overall, 97% of workshop participants agreed that they felt their skills in facilitating

dialogues or using techniques to discuss extremism had improved as a result of the

training they had receivedxxi.

“The training gave useful ideas for how to go about challenging extreme views which are

not just defensive but productive and useful and more likely to have an impact rather than

just get into argument.” – training workshop participant (teacher)

Many of the training participants found the facts and figures they were given by LWC to

counter stereotypical claims about immigrants were helpful and complemented the

dialogue skills they had received to allow them to hold open conversations.

15 | 48

“The stats! So powerful” – training workshop participant (community leader)

Desired project outcome 2: Increased understanding of dialogue teaching in

community context

Qualitative and quantitative evidence show that this outcome was achieved. Findings

show that when asked about increased understanding of dialogue teaching, all

community leaders self-reported improvement in understanding by at least 1 point.

Similarly, when asked about increased understanding of counter narratives, all

teachers agreed:

Table 5: Findings of questions relating to understanding of dialogue teaching from the pre/post survey

PRE

(T4B –

7,8,9,10/10)

POST

(T4B –

7,8,9,10/10

Difference

Q1 schools: What is your level of

understanding of Counter

Narratives?

13% 88% +75 ppts

Q2 community: What is your level

of understanding of dialogue

teaching in a community context?

5% 58% +53 ppts

Most participants had previously not been engaged with dialogue or counter narrative

techniques, which explains the low baseline scores.

“I feel that the workshop provided me with additional tools to enable me to have open and

frank discussions with clients [refugees].” – training workshop participant (community

leader)

Desired project outcome 3: Increased confidence in using enquiry techniques with

young people

Only community workshop participants were asked about their confidence in using

enquiry techniques. The question was asked as general question encompassing any types

of enquiry or dialogue techniques. The workshops specifically covered the OSDE

methodology. Almost all (95%) scored more highly in their self-reported confidence rating

for using this technique after the training.

16 | 48

Table 6: Participant confidence in using enquiry techniques from the pre/post survey

PRE

(T4B – 7,8,9,10/10)

POST

(T4B – 7,8,9,10/10)

Difference

Q3 community: How confident are you

currently in using enquiry techniques?

18% 53% +35 ppts

“[One thing I learnt today…] OSDE and what it entails – feedback form training workshop

participant (community leader)

“I found the event useful as it was Interesting to discuss ideas and learn a few stats as well as

touching on OSDE – feedback form training workshop participant (community leader)

Desired project outcome 4: Increased confidence in dealing with extreme views in

conversations

Most of the training participants worked directly with young or vulnerable people. The

training aimed to equip these participants with the confidence to address extreme views

when they arose in everyday conversation, in order for educators to be able to challenge

these views without imposing their own.

Based on findings from LWC’s pre-/post- survey, confidence in dealing with extreme views

was low to begin with, especially for teachers, and increased as a result of the training.

Almost all (95%) community leaders and all eight teachers, for whom pre-and post-

responses were available, reported feeling much more confident after the training.

Table 7: Confidence in dealing with extreme views from the pre/post participants survey

PRE

(T4B – 7,8,9,10/

10)

POST

(T4B – 7,8,9,10/10)

Difference

Q3 schools: How confident are you in

dealing with extreme views?

13% 88% +75 ppts

Q4 community: How confident are you in

dealing with extreme views?

27% 63% +36 ppts

This was further illustrated in the written qualitative feedback:

“It has built my confidence in challenging extremism” – training workshop participant

This was also highlighted by anecdotal evidence of examples where participants used their

newly gained confidence to challenge views or went on to teach others about different

ways to address extreme views: one participant mentioned having the skill and confidence

17 | 48

to challenging comments online, another mentioned feeling empowered to counter

negative views about immigrants due to the training. At least three participants

mentioned using their new-found confidence to train others:

“The whole project has been very informative – It has strengthened my confidence in

tackling difficult issues with trainee teachers in turn giving them the confidence to do this

in school.” – training workshop and innovation lab participant (academic)

“I have used my new confidence and skill to provide training to other teachers at my school

and the CN project has become part of our cohesion curriculum.” – training workshop and

innovation lab participant (teacher)

4.2.2 Innovation labs

Unlike for the training workshops, the innovation labs were not evaluated using a pre-

/post- survey but rather by participants completing feedback forms (n=23; see Appendix

3), and qualitative feedback was gained through subsequent in-depth interviews (n=3).

This means that results are not quantifiable and for reference only.

Desired project outcome 1: Increased knowledge and understanding of using social

media to create counter narratives

There is evidence to suggest that this outcome had been achieved. As noted above, the

focus shifted from hard skills to how to use social media as a tool to create counter

narratives. Two of the innovation labs focused on this. The first innovation lab looked at

virtual reality as a mechanism to create a more human bond with refugees. The final

innovation lab considered how to use videos to explore people’s views and engage with

them without judgement.

“I have learned a lot about extremism and collected keywords for further research: Also,

how to use virtual reality to support research and design.” – Innovation lab participant

“It raised awareness as to how extremist groups use the internet and social media to distort

facts and create ‘fake news’, as well as given me the ability to use some of the tools that

[we] were introduced to in [our] own projects moving forward”– Innovation lab

participant

Desired project outcome 2: Increased capacity for LWC and partners to create

powerful online counter narratives in future projects

18 | 48

This outcome became a priority for LWC as the project advanced, and the project team

realised that they benefit most from treating the innovation labs mainly as information

gathering and networking sessions.

The innovation lab sessions also allowed for new partnerships between participants to

develop, with the aim to take learnings around counter narratives and social media and

create follow on projects and activities, both within LWC and outside the organisation.

Based on staff and participants’ verbal and written feedback, this outcome was achieved:

“It has taken me out of my comfort zone – but in doing so has made me think about how

to implement this with teacher trainees.” – Innovation lab participant

“I’ve received so much useful knowledge that I have started to use then in other platforms

and research that I am involved [in].” – Innovation lab participant

Desired project outcome 3: Increased digital and social media skills

The original aim of this outcome was to improve participants’ ability to use different

digital tools (videos, virtual reality, social media). However, this outcome became less of

a priority as the project progressed and LWC discovered that after completing stage 1, an

appetite remained for participants to further explore counter narratives. The focus of the

innovation labs therefore shifted to improving participants’ awareness on how digital

tools can be used as a platform for countering extremist narratives in future projects (see

outcome 2 above).

“The innovation lab sessions were less about concrete training (…) and more about the

exchange of ideas to encourage collaboration and the brainstorming of new ideas that

would ideally lead to new and innovative ways of tackling extreme views.” – Innovation

lab participant

Based on both participant and staff feedback, this outcome was not achieved.

Desired project outcome 4: Increased understanding of extremism and counter

narratives

LWC started each session by looking at statements from different organisations (including

ISIS, Britain First, a range of newspaper headlines) and asked participants to discuss

whether they thought these statements were ‘extreme’. Although discussions were

facilitated, it was left to individual participants to ultimately decide what they thought,

19 | 48

simply giving them an idea that there might be different interpretations of the word

“extremism”.

There is evidence to suggest that this outcome has been achieved. As the innovation lab

sessions were aimed at organisations and individuals who were active – or at least

displayed an interest in working in community projects focusing on cohesion –all

participants were presumed to have had some form of understanding of extremism before

attending the session. However, in the qualitative interviews, participants pointed out that

they often looked at extremism only from one angle and that the innovation labs helped

them to understand that, as one participant put it: “Jihad comes in many forms”.

Evidence from written feedback on the evaluation forms further suggested that

participants felt that their understanding of extremism and counter narratives had

increased:

“I was able to meet others exploring similar issues and learn about creative approaches to

engaging with identity/displacement/extremism.” – Innovation lab participant

5. Key Findings: Process Evaluation

The CN project was delivered to plan and within the original timescale. The design had a

number of factors that helped ensure this was the case, including a well-organised project

delivery plan, targeting the right mix of participants and strong local links. The main

challenge LWC faced was recruitment of participants and partnering with organisations

to run the innovation labs.

5.1 Project delivery

Written feedback as well as feedback from the qualitative consultations highlighted that

the CN project was well organised and delivered. In particular, participants liked the

environment created by the delivery staff; it felt safe and they encouraged discussion.

“The training was varied, well presented, enjoyable and non-threatening.”– training

workshop participant

There was enough flexibility in the project delivery, allowing the staff to adjust the

sessions to the needs of the participants. This flexibility also allowed them to re-visit their

original objectives for the more iterative innovation lab sessions later in the project, to

20 | 48

ensure they fully explored issues around the complexity and nuances of extremism before

moving on to how these could be tackled.

Participants of the innovation labs also mentioned that they enjoyed the smaller groups

as this enabled them to have more in-depth conversations.

Whilst there was no formal follow up procedure in place, all participants felt they were

able to access all the materials they needed and that, in case they needed any further

support, they would be able to reach out to the project lead and receive the support they

needed.

5.2 Targeting the right participants

Both the training workshops and innovation labs targeted adults working in education or

otherwise engaged with young or vulnerable people by leveraging their existing network

and local newsletters. Whilst there was some feedback from interviews with project

participants stating that the project “preached to the converted” (by only targeting

individuals who were open to learning about ways to counter extreme views and shared

similar values with regards to immigration, treatment of refugees etc.), taking the

approach of targeting “frontline staff” allowed LWC to reach a far greater number of

people and this contributed to the overall success of the project.

“I think LWC targeted the right participants. It is very difficult to find and reach out to

vulnerable individuals. Speaking to frontline staff, such as teachers, social workers and

charity workers, LWC ensured that people who do get in direct contact with vulnerable

people are able to address extreme views. Targeting these ‘frontliners’ also means that the

project has a much wider range” - local authority staff

“Children tend to be more open but adults don’t know how to talk about difficult issues, they

need training and confidence to be able to encourage children to hold open dialogue. I hope

that there will be a ‘mushroom type effect’ from these sessions, reaching far beyond the

initial participants”. – Innovation lab participant

5.3 Local links

LWC engaged with a wide range of local organisations (see full list in Appendix 2). The

teacher training in stage 1 was run in cooperation with Liverpool Hope University and was

held at the university, whilst one of the community leader workshops was held specifically

for volunteers and employees of the refugee charity MRANG, who offer support to local

refugee mothers.

21 | 48

The innovation labs were also run in collaboration with different national and local

stakeholders;

1. Digital Innovation Lab was held at FACT (Foundation for Art and Creative

Technology) Liverpool and in corporation with “Future of Aleppo” (new partner)

2. Dialogue Innovation Lab was held at the Liverpool Hope University in

cooperation with the University’s Department of Education (existing partner)

3. Media Innovation Lab was held at the University of Liverpool in cooperation with

a freelance trainer, educator and founder of Christian Muslim Encounters (existing

partner)

4. Dialogue Lab was held at the Unity Theatre in collaboration with an ex-UVF

terroristxxii (new partner)

5. The Documentary Lab was held on the same day as the dialogue lab, also at the

Unity Theatre and in collaboration with a videographer (new partner)

Feedback from the events indicated that the collaboration allowed local stakeholders to

explore new areas and ways of thinking that they would otherwise not have been able to

do.

The first innovation lab in corporation with FACT allowed participants to experience a

virtual reality (VR) film about the city of Aleppo as seen through the eyes of a Syrian

refugee, Mohammedxxiii, and subsequently discuss the social impact VR can have.

“I loved learning about the Future Aleppo project – very inspiring!” – Innovation lab

participant

Similarly, the fourth innovation lab was enhanced by a presentation and subsequent Q&A

session with an ex-terrorist, who was able to speak about his personal experience with

terrorism, first as a victim (his father was shot by the IRA when he was 12), then as a

perpetrator (he later shot a man when aged 17).

Participants mentioned that: “the event went beyond networking, it has allowed

cooperation between different projects and different people with different skills to come

together in new projects [outside of LWC and the CN project]” – Innovation lab participant

The example the participant gave during the qualitative consultation was that he started

collaborating with another participant who he had met at the event, and had since worked

with a primary school in Manchester and held a workshop discussing counter narratives

22 | 48

with 80 children. His hope was that he will reach at least double this number through

word of mouth, children influencing their parents, teachers, each other etc.

Based on combined feedback from the qualitative consultations with Innovation lab

participants and wider stakeholders, being able to attract interesting organisations and

individuals to take part in the CN project gave it much more depth, as the LWC could

benefit from experience and expertise outside of that of their own staff, and played a

considerable part in the success of the overall project.

5.4 Recruitment

Participant numbers

LWC had some issues with reaching the targeted number of training and innovation lab

participants.

In the workshop phase, LWC had to set up two additional events to ensure that the target

number of 40 participants was met. One of the reasons why participant levels were lower

than expected might be that the events were held weekdays during working hours and as

they were aimed at teachers, youth workers, charity workers etc.; some potential

participants might not have been able to attend due to clashes with other work

commitments. There were also delays in receiving the BSBT funding which resulted in the

project starting later than originally planned.

Equally, although LWC reached its target of 50 participants for the innovation labs, the

turnout per session was lower than expected. Feedback from participants suggests that

this might be due to the fact that invitations were sent out quite late with insufficient

notice. This was especially the case for the last two innovation labs. According to LWC

staff, securing and getting firm time commitments from some of the partners hosting the

innovation labs contributed to events being held last minute.

As the innovation labs were of a very iterative nature, it could also be the case that

potential participants were not sure what to expect and therefore were less likely to

attend.

LWC was hoping to utilise their BSBT in-kind support to help with the promotion of future

events. This, in combination with holding some of the events targeted at people in work

outside of working hours (evenings, weekend), as well as promoting events earlier in

advance to allow potential participants to plan around the events accordingly and being

clear about the content of the events. It was hoped this approach would boost response

rates.

23 | 48

Innovation lab partners

One of the strengths of the CN project was the ability of the LWC to draw in different

organisations and individuals to participate and support the running of the workshops

and innovation labs. Whilst there was a clear interest in the subject matter, the process of

engaging partners to host events was more difficult than originally anticipated by the

project. This resulted in some of the innovation labs being held at shorter notice than

planned for, which in turn made it more difficult to secure target numbers of participants.

Ensuring well-ahead of time that potential partners are fully bought in and committed to

going through with the hosting of events would make the organising of events easier. It

would allow more time to promote the event, increasing the chances of high participant

turnouts.

6. Lessons learnt and conclusions

The evidence examined indicates that LWC’s CN project achieved the main outputs and

outcomes, which are expected to contribute towards the longer-term macro BSBT

outcome of “Creating more resilient communities”. Despite some challenges in the

recruitment of participants, which the project overcame by putting on additional events

and securing innovation lab partners early in the project’s delivery, based on the collated

feedback from participants and stakeholders, LWC’s CN project was well-organised and

well-executed project, that could be replicated

Sustainability of the programme

The CN project is sustainable. It was not very resource intensive; once developed, training

resources and toolkits can be re-used and easily adapted to suit the needs of different

organisations/ participants. However, some wider stakeholders and participants

mentioned the risk that without further and ongoing events, the long-term impact of

creating more resilient communities might not be achieved. A more consistent funding

approach would give continuity and might help achieve the desired long-term impact.

Replicability of the programme

Based on feedback received and observation of LWC’s CN project, the project is replicable

if certain conditions are met:

24 | 48

• Existing links with other local institutions are key, as these provide both the

potential participants to engage in training/ networking sessions as well as

allowing for cooperation in running the events

• Expertise in educational techniques and facilitating dialogues (both in-house and

among partners)

• Digital skills or connections to individuals/ organisations who have specific digital

skills

• The timing of workshops should be mindful of existing commitments of intended

beneficiaries – many may work standard office hours (Monday-Friday, 9-5).

Timings should be flexible so as to attract a wider pool of participants.

To a certain degree, the CN project was already being replicated by participants of the

training element of the project, carrying out further training based on the skills learned

through and toolkit provided by LWC.

LWC’s CN project is also scalable through increasing staff capacity, the number of events,

and engaging with additional organisations. All of which are determined by funding.

Key success factors

▪ Good reputation of the LWC as a means to promote interest and create

connections with other organisations and within the local community

▪ Commitment and organisational skill of project lead – having the capacity to

develop and run interesting and thought challenging events

▪ Ability to make use of staff expert skills in education techniques

▪ Benefiting from expertise of partnering organisations and ability to recruit

expert organisations/ individuals to contribute to the innovative training

sessions – this includes sharing real life experiences that add meaning to the

training by placing it in real-life context that participants can relate to

▪ Working with individuals able to cascade learning and broaden reach of CN

project beyond LWC’s direct reach – asking participants to carry out further

workshops on CN, reaching a different and wider audience

▪ Bringing together of likeminded people and giving them the opportunity to

network – through the innovation labs people who might otherwise not have met

were able to come together and collaborate both within and outside of the project

25 | 48

▪ Ensuring programme flexibility by allowing staff to adjust sessions to the need

of participants but also re-visiting initial objectives around understanding of

extremism before moving on to how these can be addressed

▪ Working in smaller groups thereby enabling participants to have more in-depth

conversations

▪ Providing participants ease of access to material so they feel supported in their

own independent or collectively line of enquiry throughout the programme

Key challenges

▪ Participants recruitment/ attendance – getting enough participants to attend

the workshops and innovation labs was a challenge, with problems for some

regarding the timing of events

▪ Engaging harder to reach/ vulnerable audiences – getting buy-in not only from

people interested in cohesion but also from those who are at risk of developing

extreme views

▪ Ensuring uptake of participant-led workshops – putting formal measures in

place to monitor uptake of participant-led workshops

▪ Finding committed partners to support the facilitation and delivery of innovation

labs earlier in the project planning phase

▪ Delays receiving in-kind support – not receiving BSBT in-kind support in time to

help with the promoting of the CN events

Stakeholders and participants alike have spoken about the continued need for a project

to challenge current views on extremism and to act as an enabler to promote techniques

in countering extreme views. Should the CN project be repeated, the LWC should aim to

start communication and promotion of the project earlier in the planning phase and

consider putting formal measures in place to monitor the ultimate reach of participant-

led activities.

The findings from this IDPE will be integrated into the overall analysis and synthesis of the

BSBT programme in order to establish to what extent the programme as a whole has

contributed to an increased sense of belonging, more resilient communities and increased

support for shared values at a local level

26 | 48

Appendix 1

Baseline Survey [Community] LWC

Counter Narratives

Baseline Evaluation – Start of the Day Name: _____________________

Date: ______________________

Circle your answers – 1 being very low and 10 being very high

1. What are your current skill levels for facilitating dialogue?

1 2 3 4 5 6 7 8 9 10

2. What is your level of understanding of dialogue teaching in a community context?

1 2 3 4 5 6 7 8 9 10

3. How confident are you currently in using enquiry techniques?

1 2 3 4 5 6 7 8 9 10

4. How confident are you in dealing with extreme views?

1 2 3 4 5 6 7 8 9 10

27 | 48

Endline Survey [Community] LWC

Counter Narratives

Endline Evaluation – End of the Day Name: _____________________

Date: ______________________

Circle your answers – 1 being very low and 10 being very high

1. What are your current skill levels for facilitating dialogue?

1 2 3 4 5 6 7 8 9 10

2. What is your level of understanding of dialogue teaching in a community context?

1 2 3 4 5 6 7 8 9 10

3. How confident are you currently in using enquiry techniques?

1 2 3 4 5 6 7 8 9 10

4. How confident are you in dealing with extreme views?

1 2 3 4 5 6 7 8 9 10

28 | 48

Appendix 2: List of local organisations

Charities:

Young Peoples Advisory Service

YMCA

Sahir House

Spirit Level

MRANG

SOLA ARTS

Local Solutions

Claire House

South Liverpool Against Poverty

A Quiet Place

Inspire Trust

4 Wings

Independent Community Unit

Education Organisations:

Liverpool Supplementary Schools Network

Liverpool Polish School

National Resource Centre for Supplementary Education

University of Liverpool (Media Department)

Liverpool John Moores University (Department of Initial Teacher Education)

Liverpool Hope University

9 x local schools (6 x primary, 2 x secondary, 1 x SEN)

Academics:

Senior Lecturer in Education at Liverpool Hope University

Lecturer in Education Studies at Liverpool Hope University

29 | 48

Associate lecturer, PhD Candidate – Global Education at Liverpool Hope University

Senior Lecturer in Education Studies at Liverpool Hope University

PhD Candidate – Social Science at Liverpool Hope University

Lecturer in Industrial Design at University of Liverpool

PhD Candidate – Industrial Design at University of Liverpool

Lecturer in Professional & Media Writing at University of Liverpool

Statutory Bodies:

Liverpool City Council

Artists/Arts Organisations:

SOLA ARTS

Docu-Postcards

Independent videographer

Glitch Theatre

Collective Encounters

VR artist

FACT Liverpool

Musician

Thunder’s Mouth Theatre

Independent Experts:

Ex-UVF terrorist & counter terrorism advocate for Extreme Dialogue

Consultant on Terrorism Law & Policy, previously School of Advanced Policing at LJMU

30 | 48

Appendix 3: LWC Innovation lab feedback form

1. I would describe my role/interest in extreme views / counter extremism as..…

2. I would describe my role/interest in digital technology / online campaigns

as..…

3. I decided to come to today’s event because….

4. I have found this event useful because….

5. One thing I learnt today…

6. One thing I would have changed / I didn’t enjoy so much…

7. Any other comments?

31 | 48

Appendix 4: LWC Feedback form for participant led training workshops

Community Workshop Session Feedback (to be provided by CN for Communities

participants after delivery of a related session in the community)

Where and when did you deliver your session?

How big was the group?

Who was the target audience?

What did you learn in the Counter Narratives workshop that you brought into your

delivery?

What aspect of this session worked best?

What didn’t work so well?

Did you carry out evaluation with your participants? Please provide copies or an

overview of the results.

Any other comments / feedback / concerns / questions?

32 | 48

Appendix 5: Discussion guides used in qualitative consultations

Discussion guide: Service users training workshop

Thank participant for taking part.

Introduce self, Ipsos MORI.

Text in italics refer to interviewer notes.

Introduce research

- We are conducting an evaluation of the LWC’s Counter Narratives project as part

of the Building a Stronger Britain Together, to find out how well it is working and

how it can work better.

- The discussion is completely voluntary and they are free to decline to answer any

question or to stop the interview at any time. They will be at no advantage or

disadvantage as a result of their decision about taking part.

- Talk through participant information sheet (read this through if necessary) with

both the service user and the case worker (if applicable). Make sure that they

understand all of the details of this and if they’re prepared to go ahead.

- Reiterate voluntary nature of interview and they are at no advantage or

disadvantage if they decide to take part.

- Reiterate confidentiality and anonymity – we will protect their identity as far as

possible but it may be possible to identify them in outputs due to the small

number participating.

- Ask their permission to record the interview, ensuring that all recordings are

securely stored under the Data Protection Act and the research team are the only

people who will listen back to the recording.

- Turn on recorder.

Introduction

Be aware that participants come from a range of different backgrounds; it may not be

appropriate to ask service users who they live, or whether they work.

- Tell me a bit about yourself – what do you do day-to-day

- If participant part of an organisation (local charity, university etc.): What does

your organisation do?

- How long have you lived in the area?

Recruitment

33 | 48

- How did you hear about the Counter Narratives project?

- What made you want to attend the training workshop?

- How did you join the activities i.e. opt-in, email register etc.

- How easy was it to sign up?

- Have you taken part in or received any support from LWC activities before? If yes,

how regularly was this support?

- Has there been any support post training workshop to implement acquired skills?

- How does this support look like?

Participation

- What were your expectations of the LWC Counter Narratives workshop?

- Why did you choose to attend the training workshop?

- How was the workshop structured?

- What was their overall opinion of the training? Explore the opinion – why this

rate?

- Did anything surprise you about the project?

- How easy or difficult is it to access the activities in terms of

o Travel

o Associated costs

o Location of workshop

o Timing of the workshop / clashes with work/ other commitments

- What did you think of the environment of the event?

- Was the training engaging?

- Did they feel you could talk freely?

- What did you most enjoy?

- What did you least enjoy about the workshop?

Outcomes

We are aware that you might have attended similar workshops previously but please focus

your answers on your experience with the Counter Narrative training workshop only.

Key learnings

- What are the key learnings you have taken out of the training?

- What difference has the project made in the way you engage in a dialogue with

young people?

- What difference has it made to the way you think about dealing with extreme

views?

34 | 48

- Has the project challenges your perceptions of extremist views? If so, how?

- What tools have you been provided with? How useful are these? Can you give

examples of when and how you would use these in conversation?

- Did the workshop deliver what you expected it to deliver? If not, why not?

Objectives:

- For community leaders: Do you feel you have increased your confidence & skills

in facilitating dialogues? If so, how?

- For teachers: Do you feel you have increased your understanding of dialogue

teaching in community context? If so, how?

- Do you feel you have increased your confidence in using enquiry techniques with

young people? If so, how?

- Do you feel you have increased your confidence in dealing with extreme views in

conversations? If so, how?

- What do you think would have happened if you were not part of the project?

Would that have made any difference to you?

Attitudes/behaviours:

- Do you feel more enabled to create strong counter narrative to tackle extremist

ideas?

- Do you feel better able to develop counter narrative to address negative

stereotyping?

- Do you think this event has changed how you feel about yourself and skillsets?

How/ why?

- Do you think differently about extreme views after attending the project?

- Will you change your behaviour in dealing with extreme views after attending the

workshop?

Impact

- Do you think you will do anything differently as a result of this event? Probe on

ability to hold open dialogue, challenge religious, race and cultural stereotypes

and confidence in dealing with extremism.

- Do you think this project had an effect on your local community? If so, how?

Probe on resilient communities and understanding of global issues.

- Do you think the objectives the Counter Narratives set out to do have been

achieved?

35 | 48

Wrap-up

- What do you feel could be improved about this event?

- Would you recommend the project to others?

- Do you think the Counter Narrative project has had an effect on your local

community?

- Anything else you would like to add about taking part in the Counter Narratives

project?

Thanks and close. Ensure service user takes participant information leaflet with Ipsos

MORI contact details.

36 | 48

Discussion guide: Service users Innovation Lab

Thank participant for taking part.

Introduce self, Ipsos MORI.

Text in italics refer to interviewer notes.

Introduce research

- We are conducting an evaluation of the LWC’s Counter Narratives project as part

of the Building a Stronger Britain Together, to find out how well it is working and

how it can work better.

- The discussion is completely voluntary and you are free to decline to answer any

question or to stop the interview at any time. You will be at no advantage or

disadvantage as a result of your decision about taking part.

- Talk through participant information sheet (read this through if necessary) with

both the service user and the case worker (if applicable). Make sure that they

understand all of the details of this and if they’re prepared to go ahead.

- Reiterate voluntary nature of interview and they are at no advantage or

disadvantage if they decide to take part.

- Reiterate confidentiality and anonymity – we will protect their identity as far as

possible but it may be possible to identify them in outputs due to the small number

participating.

- Ask their permission to record the interview, ensuring that all recordings are

securely stored under the Data Protection Act and the research team are the only

people who will listen back to the recording.

- Turn on recorder.

Introduction

Be aware that participants come from a range of different backgrounds; it may not be

appropriate to ask service users who they live, or whether they work.

- Tell me a bit about yourself – what you do day-to-day

- If participant part of an organisation (local charity, university etc.): What does

your organisation do?

- How long have you lived in the area?

Recruitment

- How did you hear about the Counter Narratives project?

- What made you want to attend the Innovation labs?

37 | 48

- How did you join the activities i.e. opt-in, email register etc.

- How easy was it to sign up?

- Have you taken part in or received any support from LWC activities before? If yes,

how regularly was this support?

- How does this support look like?

- Have you worked with LWC before? If yes, in what capacity?

Participation

- What were your expectations of the Innovation lab sessions?

- Why did you choose to attend the session?

- How was it structured?

- What was their overall opinion of the Innovation lab session(s)?

- Did anything surprise you about the project?

- Were you involved in the running of the sessions? If yes, in what capacity?

- How easy or difficult is it to access the activities in terms of

o Travel

o Associated costs

o Location of sessions

o Timing

o Length of session

- What did you think of the environment of the event?

- Was the training engaging?

- Did you feel they could talk freely?

- What did you most enjoy?

- What did you least enjoy about the session?

Outcomes

- What are the key learnings you have taken out of the Innovation labs?

- What difference has it made to the way you think about dealing with extreme

views?

- Did the Innovation labs deliver what you expected it to deliver?

- Do you feel you have increased your knowledge and understanding of using

social media to create counter narratives? If so, how?

- If applicable: Do you feel you have increased your capacity to create powerful

online counter narratives in future projects? If so, how?

- Do you feel you have increased their digital and social media skills? If so, how?

- Do you feel you have increased your understanding of extremism and counter

narratives? If so, how?

38 | 48

- Do you think this event has changed how you feel about yourself and skillsets?

- What do you think would have happened if you were not part of the project?

Would that have made any difference to them?

- Do you think differently about extreme views after attending the project?

- Will you change your behaviour in dealing with extreme views after attending the

Innovation labs?

Impact

- Do you think you will do anything differently as a result of this event?

Wrap-up

- What do you feel could be improved about this event?

- Would you recommend the project to others?

- Do you think the Counter Narrative project has had an effect on your local

community?

- Anything else they would like to add about taking part in the Innovation labs?

Thanks and close. Ensure service user takes participant information leaflet with Ipsos

MORI contact details.

39 | 48

Discussion guide: Wider stakeholders

Thank participant for taking part.

Introduce self, Ipsos MORI.

Text in italics denotes interviewer instructions.

Introduce research

- We are conducting an evaluation of LWC’s Counter Narratives project as part of

the Building a Stronger Britain Together, to find out how well it is working and

how it can work better.

- The discussion is completely voluntary and you are free to decline to answer any

question or to stop the interview at any time. You will be at no advantage or

disadvantage as a result of your decision about taking part.

- Talk through participant information sheet (read this through if necessary). Make

sure that they understand all of the details of this and if they’re prepared to go

ahead.

- Reiterate voluntary nature of interview and they are at no advantage or

disadvantage if they decide to take part.

- Reiterate confidentiality and anonymity – we will protect their identity as far as

possible but it may be possible to identify them in outputs due to the small

numbers participating.

- Ask their permission to record the interview, ensuring that all recordings are

securely stored under the Data Protection Act and the research team are the only

people who will listen back to the recording.

- Turn on recorder.

Background and intro

- Background, role, organisation

- Role in relation to the Counter Narratives project and LWC

- Knowledge of the BSBT programme

- Involvement in counter-extremism work – if no involvement in CE work ask about

wider safeguarding/ vulnerability work

- Overview of local counter-extremism work in the area

Awareness and knowledge of project

- How and when did you first hear about the organisation and project?

40 | 48

- What is your understanding of the project? (Why? Objectives? What activities?

For whom?)

- What is your view on how the project was set-up? Probe on underlying

assumptions and context [to test replicability]

Perception of the project rationale and scope and relevance

- To what extent do you agree there is a need for the project? Why?

o What particular local problems is the project addressing?

o Is the project engaging the right participants/clients? Why?

o How well does the project fit with the local context? Why?

- How/to what extent does organisation / the project align with your own work?

- How does BSBT/counter-extremism work fit with your objectives?

Outcomes and impacts

- To what extent do you feel that the project addresses the issue or problem

identified?

- What do you think some of the benefits of the project been on:

o Service user/participants

o The organisation delivering the project

o Your organisation

o The local area

o Counter-extremist narrative

- Have you seen any changes in the local community? Do you see more resilience

within the local community? Could you give any examples? Probe on resilient

community, promotion of social changes within the community, increased

understanding of global issues.

- Has the project been spoken about within your network/ the local community?

- Has there been any (local) media coverage of the Counter Narratives project?

- Have you identified any unexpected outcomes as a result of the project?

- What do you think the longer term impact of the project will be on participants,

local area?

- What needs to happen to ensure the project has a legacy in the local area?

- Is there an ongoing need for the project or similar ones?

- How replicable and scalable do they think the project is?

- What do you think would have happened without the project and BSBT funding?

- Has anyone been referred to your organisation/service as a result of the Counter

Narratives events?

- Is there any learning you think could be applied to similar projects in future?

Project delivery

41 | 48

- What do you think were any particular strengths or weaknesses of the project?

- What are some of the delivery challenges the project has faced?

- How well did different organisations partner in order to deliver and/or

disseminate the project?

- What do you feel was achieved as a result of their involvement in the Counter

Narratives project?

o What was the impact on them?

- Anything else to add?

Thanks and close.

42 | 48

Discussion guide: Delivery staff

Thank participant for taking part.

Introduce self, Ipsos MORI.

Text in italics denotes interviewer instructions.

Introduce research

- We are conducting an evaluation of the Counter Narratives project as part of the

Building a Stronger Britain Together, to find out how well it is working and how it

can work better.

- The discussion is completely voluntary and they are free to decline to answer any

question or to stop the interview at any time. They will be at no advantage or

disadvantage as a result of their decision about taking part.

- Talk through participant information sheet (read this through if necessary). Make

sure that they understand all of the details of this and if they’re prepared to go

ahead.

- Reiterate voluntary nature of interview and they are at no advantage or

disadvantage if they decide to take part.

- Reiterate confidentiality and anonymity – we will protect their identity as far as

possible but it may be possible to identify them in outputs due to the small

numbers participating.

- Ask their permission to record the interview, ensuring that all recordings are

securely stored under the Data Protection Act and the research team are the only

people who will listen back to the recording.

- Turn on recorder.

Interviewers should note that not all delivery staff will have knowledge/be able to respond

to all sections of the guide.

Background and intro

- Background, role, how long they have worked at the organisation

- Role in relation to the Counter Narrative project

- Previous/ wider involvement in counter-extremism work or safeguarding/

vulnerability work

- Knowledge of the BSBT programme: what do they think BSBT is about?

- Overview of local counter-extremism work in the area

Developing the project

43 | 48

Probes into the issues identified in the logic model. Do the interviewee mention the same

issues, if not, what are they and why do they identify these? Important to ensure a shared

understanding so as to consolidate the logic model.

- What is the need for the project in their local area? How was this identified/

evidenced?

- Designing the project: who was involved, how was it developed (was the project

designed with the BSBT programme aims in mind – or was a pre-existing project

idea adapted to meet BSBT criteria), how was the target audience determined?

- What was the basis for deciding how best to meet the needs identified? Any

evidence of effectiveness of the planned approaches? What was the theory of

change?

- Has the project changed or evolved at all? If so, how (in what direction) and why?

- How did they think their project would contribute to delivering the government’s

CE strategy?

- Had the project run previously (with different funding)? Was this project

adapted/developed from one previously run – or was it a completely new venture

for your organisation?

- Contact and support from BSBT Community Coordinator: frequency, extent of

involvement and support

- Contact and support from the UK Community Foundation: frequency, extent of

involvement and support

- Contact and support from any other local experts or stakeholders in CE,

vulnerability and safeguarding activity e.g. Prevent coordinators?

- Anything that went particularly well/ badly; any learning they would share with

others about project scoping and applying for funding?

Marketing and advertising the project

- What marketing and advertising approaches were used to promote/attract

service users?

- Were specific marketing activities developed for project beneficiaries? If yes, what

were they?

- What additional marketing approaches have been put in place?

- How well does the project fit with the local context/issues?

- What improvements could be made to the marketing and advertising process?

Recruitment and referral process

44 | 48

- What approach to recruitment and referral was used on the project? What

proportion of their project participants were already engaging with the

organisation before the BSBT project started?

- How easy or difficult were any referral processes? Why [difficult or easy – informs

assumptions/process in identifying the right beneficiaries]?

- What improvements could be made to the recruitment and referral process?

- Were any assessment / qualifying criteria used on the project? If so, what and

why?

- Are they on course to achieve the anticipated participation numbers and

participant profile? If not – any key factors?

- To what extent did marketing activities targeted the right individuals for the

project?

Delivering the project

Might need probes on activities mentioned in logic model. Note that if more than one

type, need to know the below for each type identified i.e. type of challenges, replication

and capacity building on the back of each activity type.

- Setting project milestones and delivery dates: was this done? how easy/difficult?

What were the challenges?

- What activities were you involved in?

- To what extent have project activities been delivered and received as planned?

- Anticipated challenges faced in project delivery: attrition, project content,

language barriers etc. If yes, how have they overcome these challenges?

- Any unanticipated challenges to project delivery? How did you manage these?

- To what extent could the project be replicated? Explore specific

conditions/contexts rendering the projects replicable.

- To what extent could the project be scaled up? Explore specific

conditions/contexts rendering the project scalable.

- For Innovation labs: what partners have you worked with? How well has this

worked? Will there be future collaboration?

- [If responsible for project finances] Has funding received allowed you to

complete all activities outlined in the application form? Was the budget planning

accurate?

- What changes – if any – would you make to project delivery if you were running it

again?

Outputs

Focus on reasons for not achieving targets to inform logic model pathway

45 | 48

- Have you achieved the number of training workshops you wanted to run? How/

why not?

- Have you increased the number of trained community leaders and teachers?

How/ why not?

- Have you developed dialogue facilitating toolkits? How/ why not?

- Have you facilitated community/ school run workshops run by participants? How/

why not?

- Have you increased links with local communities/ schools/ charities etc.? How/

why not?

- Have you achieved the number of innovation labs you wanted to run? How/ why

not?

- Have outputs been as expected? How/ why not?

- How have you monitored output delivery?

- Do they think the targets / expectations set were realistic?

- Have you been required to provide monitoring data to UKCF/ HO? How easy has

this been? Any particular challenges.

Outcomes and impacts

- To what extent do they feel that the project addresses the issue or problem

identified? Can they give a specific example linking the outcome to preceding

activities and inputs?

- What do they think the benefits of the project have been on:

o Service user/participants

o The organisation delivering the project

o The local area

o Counter-extremist narrative

- To what extent do they think the project has met/realised their expected

outcomes?

o What difference do you think the programme made to the participants?

Training workshops:

The following questions require you to think specifically of the project and isolate the

project activities to identify impact on young people regardless of possible previous

involvement with the organisation. Thinking of the project activities:

o Do you think that participants’ confidence and skills in facilitating dialogues

has improved as a result?

o Do you think that participants’ confidence in using enquiry techniques with

young people/their peers has improved?

o Do you think that participants’ confidence in dealing with extreme views

has improved? How? Probe on ability to build a counter-argument.

46 | 48

o Do you think that participants’ understanding of dialogue teaching in the

community context has improved?

o Do you think that participants’ skills in facilitating dialogues has improved?

o Do you think that participants will be more able to pass these onto others?

o For participants of Innovation labs– did they gain social media skills? Any

other skills?

o Do you think that you provided them with effective tools to counter

narrative online?

o Have LWCs’ digital and social media skills increased?

o Has LWC’s capacity to create powerful online counter narratives in future

projects increased?

o What impact do you think taking part in the Counter Narratives project had

on participants’? Any examples?

- Have you identified any unexpected outcomes as a result of the project?

- What are some of the challenges the project has faced?

- What do you think the longer term impact of the project will be?

- What do you think needs to happen to ensure the project has a legacy in the

local area?

- What are the anticipated longer-term impacts of the project (if any)?

- What do they think would have happened without the project and BSBT funding?

- What were the particular strengths and any weaknesses of the project?

- What learning would they apply if they were running the project again?

Wrap up

- What is the biggest difference they think the project has had on them,

participants and their local community?

- Anything else they would like to add about delivering the Counter Narrative

project?

Thanks and close.

47 | 48

Appendix 6

Screenshot of front page of toolkit

ihttps://www.ons.gov.uk/peoplepopulationandcommunity/populationandmigration/internationalmigration/articles/whataremigrationlevelslikeinyourarea/2015-08-28 ii https://www.merseyside.police.uk/media/801488/racial-hate-crimes-dm2017-0181.pdf. iii Including virtual reality artists, musicians, videographers. For the full list see Appendix 2 iv Sample size for schools: n= 8; sample size for communities: n=22 for pre, n=19 for post questionnaires vhttps://www.ons.gov.uk/peoplepopulationandcommunity/populationandmigration/internationalmigration/articles/whataremigrationlevelslikeinyourarea/2015-08-28 vi https://www.merseyside.police.uk/media/801488/racial-hate-crimes-dm2017-0181.pdf. vii https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/652136/hate-crime-1617-hosb1717.pdf viii https://www.hopenothate.org.uk/wp-content/uploads/2018/03/State-of-Hate-2018.pdf ix https://www.hopenothate.org.uk/wp-content/uploads/2018/03/State-of-Hate-2018.pdf x https://www.liverpoolecho.co.uk/news/liverpool-news/revealed-liverpool-john-moores-university-8858428?_ga=2.73004560.221934918.1523361523-2005808517.1523360392) xi http://www.europarl.europa.eu/RegData/etudes/STUD/2017/596829/IPOL_STU(2017)596829_EN.pdf and http://www.isdglobal.org/wp-content/uploads/2018/02/Executive-summary.pdf xii The original aim was to reach 100 across the five events but was revised to 50 after struggling to recruit enough participants for the stage 1 workshops xiii Appendix 5 xiv The excel spreadsheet included an assigned number per participant (i.e. #01), score for base and endline and whether 1-point increase had been achieved. xv Sample size for schools: n= 8; sample size for communities: n=22 for pre, n=19 for post questionnaires xvi Full PDF available on request xvii See Appendix 4 for more details xviii LWC used a 10-point Likert scale to measure success.

48 | 48

xix http://www.trchome.com/component/content/article/66-market-research-knowledge/published xx Sample size for schools: n= 8; sample size for communities: n=22 for pre, n=19 for post questionnaires xxi Note that percentage calculations reflecting the level of agreement for survey questionnaire statements are based on the number of participants selecting at least 1-point increase divided by the base. xxii xxii http://extremedialogue.org xxiii https://futurealeppo.com/virtualreality-2

Ipsos MORI | [Report title] 1

For more information

Contact the Ipsos MORI BSBT Evaluation Team on telephone: 0808 101 6229 or email: BSBTevaluation@ipsos.com

3 Thomas More Square

London

E1W 1YW

t: +44 (0)20 3059 5000

www.ipsos-mori.com

http://twitter.com/IpsosMORI

About Ipsos MORI’s Social Research Institute

The Social Research Institute works closely with national governments, local public services and the not-for-profit sector.

Its c.200 research staff focus on public service and policy issues. Each has expertise in a particular part of the public sector,

ensuring we have a detailed understanding of specific sectors and policy challenges. This, combined with our methods

and communications expertise, helps ensure that our research makes a difference for decision makers and communities.