evaluation of the face to face service - nspcc | … · 3.3 changes within the family 29 ... 6...

Impact and Evidence series

June 2015

Evaluation of thE facE to facE sErvicE

using a solution-focusEd approach with childrEn and young pEoplE in carE or on thE EdgE of carE Prakash Fernandes NSPCC Evaluation department

Impact and Evidence series

This report is part of the NSPCC’s Impact and Evidence series, which presents the findings of the Society’s research into its services and interventions. Many of the reports are produced by the NSPCC’s Evaluation department, but some are written by other organisations commissioned by the Society to carry out research on its behalf. The aim of the series is to contribute to the evidence base of what works in preventing cruelty to children and in reducing the harm it causes when abuse does happen.

©2015 NSPCC. Photography by John Challicom. The people pictured are models.

3Impact and Evidence series

Contents

ACKNOWLEDGEMENTS 5

KEY FINDINGS: YOUNG PEOPLE’S VERSION 6

KEY FINDINGS 7

ExECUTIVE SUMMARY 8

MAIN REPORT 12

Chapter 1: Introduction 121.1 Rationale for the Face to Face service 121.2 Structure of the service 131.3 Key elements of a solution-focused approach 141.4 Methodology 15

Chapter 2: Changes in wellbeing 192.1 Overall changes in wellbeing 192.2 Movement of children’s scores on the wellbeing scale 202.3 Outcomes and demographic characteristics 222.4 Changes three months after the service 23

Chapter 3: Children and young people’s experiences of change 26

3.1 Achieving goals 263.2 Changes in self-perceptions 273.3 Changes within the family 293.4 Changes at school 303.5 Changes in social interactions and peer relationships 31

Chapter 4: How Face to Face helped 334.1 Key aspects of the approach 334.2 Barriers to change 384.3 Relationship with NSPCC practitioners 40

Chapter 5: Conclusion 42

Bibliography 43

Evaluation of the Face to Face service4

AppendicesAppendix 1: Guidance on use of the ORS and SRS 45Appendix 2: ORS and CORS 48Appendix 3: SRS and CSRS 50Appendix 4: Representativeness of the evaluation sample 52Appendix 5: Interview questions for children and young people 53Appendix 6: Interview questions with referrers 55Appendix 7: Interview schedule with foster carers 57Appendix 8: Ethical considerations of the study 59Appendix 9: Average change in ORS scores across age groups 61


acKnowlEdgEMEntsThe author would like to thank:

• the children and young people who consented to take part in the evaluation and to give their time to complete questionnaires after each session

• children, foster carers, parents and referrers and NSPCC practitioners who took part in the interviews and shared their perspectives about the service

• team managers, practitioners and administrative staff at the NSPCC Face to Face service centres in Birmingham, Bristol, Croydon, East London, Foyle, Gillingham, Grimsby, Hull, Liverpool, Middlesborough, Nottingham, Peterborough, Prestatyn, Sheffield, Southampton, Stoke, Swansea and Warrington, who supported the implementation of the evaluation by explaining its importance to children and ensuring that data was captured for analysis

• Tom Rahilly, Louise Bazalgette, Neil Anderson, Sharon Breslin, Alex Modder, Maria Emilsson, Gillian McElroy, Julia Mayes, Kate Tilley, Caroline Wilkinson, Robin James, Alex Stevenson and Helen Walters, for their ongoing support to the evaluation

• Dr. Sami Timmimi and Dianne Tetely from the OO-CAMHS (Outcome Orientated Child and Adolescent Mental Health Services), for sharing their insights on the use of the ORS (Outcome Rating Scale) and the SRS (Session Rating Scale) within the CAMHS service

• Dan Tallis, for the information graphics in the report

• Paul Whalley, for assisting in interviews with children and young people

• Matt Barnard and Richard Cotmore, for their support and guidance in reviewing the evaluation plans and reports.


KEy findings: young pEoplE’s vErsionFace to Face is a new NSPCC service that offers support to all children and young people aged 5–18 who are in care, or those who are very worried about how they get along with family members. The Face to Face workers talk to children and young people about the things they would like to change in their life. They help them work out what needs to happen to make things better, both now and in the future.

The NSPCC did some research to find out what these children and young people thought of the service. They asked children and young people to answer questionnaires. They also did interviews with children, foster carers and social workers.

These are the main things that they found:

• Children and young people said they were happier at the end of the service. They said that this was because they felt more confident, started to get on better with people close to them and their friends, and felt more positive about the future.

• At the end of the work, most children and young people said that they no longer worried about the problem they had. After three months, things had stayed the same or got better for most of these young people, but a few felt that things had got worse.

• Very few children and young people said they still needed more help at the end of the work.

• There were no differences between what younger or older children thought about the service, or between children with or without learning difficulties.

• Children and young people said that the Face to Face service had helped by listening to them and helping them to find their own solutions to their problems.

• Social workers and foster carers liked the service because young people did not have to wait long before they could begin the work.

• Some young people said they would have liked more activities during the work and would have liked to meet the workers in places such as a park or café.

The NSPCC will now share this research with other organisations so that more young people can use a service that is similar to Face to Face.


KEy findingsThe NSPCC Face to Face service seeks to support children and young people aged 5–18 who are in care or on the edge of care. It aims to identify and find solutions to issues that are affecting their life, impacting on their emotional wellbeing, or placing them at risk. The evaluation measured the change in wellbeing for children and young people at the end of the service as well as three months after finishing. It explored the role of Face to Face in helping children and young people resolve their immediate concern and some of the barriers to achieving change.

The key findings from this evaluation are:

• Before receiving the Face to Face service, overall 58 per cent of children and young people had wellbeing scores indicating clinical levels of distress, but by the end of their sessions this had reduced to only 15 per cent. In total just under 60 per cent of all children and young people showed a reliable improvement in their levels of well being at the end of the service.

• Of the children who were most in need at beginning the service - those with clinical levels of distress - 70 per cent showed reliable improvement and ended the service with normal levels of wellbeing.

• Seventy-three per cent of children and young people said that Face to Face had helped them to solve the immediate concern that was affecting their emotional wellbeing.

• Three months after finishing their last Face to Face session, the improvements in the well being scores of 84 percent of children and young people were sustained.

• Children and young people said that the Face to Face service had helped by listening to them; being confidential; encouraging them to reflect on issues; focusing on their strengths; giving them control over the process, and by rehearsing strategies to solve their problems.

• Referrers and foster carers found the referral process easy and were able to see the changes in the young person. They would like to have been more involved in joint sessions with the child at the end of the work, and would like to see the service expanded to all children in need.

• The findings indicate that there is a gap in provision and that a short, solution-focused intervention can help address the needs of children and young people.

Overall, the evidence indicates that Face to Face is a promising approach to helping children and young people in care or on the edge of care. However, the evaluation did not include a comparison group, so further research is required to be confident that the improvements in outcomes are a direct result of the Face to Face service.


ExEcutivE suMMaryBackgroundFace to Face is an NSPCC service for children and young people aged 5–18 who would like support to deal with an issue that is affecting their emotional wellbeing or placing them at risk. Children and young people eligible for the service include those who are in care as well as those on the edge of care. The service is intended to prevent escalation of difficulties that these children may be experiencing to a level that could require more intensive, but less readily available services or support. The service does not seek to define an alternative to long-term therapeutic treatment for children in care, but recognises that given the lack of capacity in the current system, it is important to examine the effectiveness of other forms of support.

Young people can access the service by referring themselves or through referral from an agency with their consent. The service uses a solution-focused approach and is designed to be led by the needs of the young person. This means that the young person identifies their own goals in an initial session, and then works towards achieving these with the support of a trained NSPCC practitioner. The approach focuses on the strengths of children and young people and life without the problem, rather than a detailed analysis of problem dimensions. They are offered up to eight sessions but may terminate the service as soon as they feel they have met their objectives. The service was delivered from 18 locations1 across the UK between September 2011 and March 2015: Birmingham, Bristol, Croydon, East London, Foyle, Gillingham, Grimsby, Hull, Liverpool, Middlesbrough, Nottingham, Peterborough, Prestatyn, Sheffield, Southampton, Stoke on Trent, Swansea and Warrington.

Methodology Face to Face was evaluated using a mixed-method design. At each session the child or young person completes the Outcome Rating Scale (ORS), a four-item measure designed to track wellbeing outcomes. This tool and its equivalent version for younger children aged 6–12, the Child Outcome Rating Scale (CORS), have been validated in a variety of academic studies. The ORS and CORS demonstrate reasonable consistency and test-retest reliability despite the ultra-brief nature of the measure. At the end of the work the young person is also asked to complete a questionnaire to review the work done and to describe the changes they have experienced.

1 Not all sites were delivering the service for this entire duration.


The evaluation sample consisted of 611 young people who had completed the Face to Face work and the ORS at more than one session till May 2015. The sample was representative of all users of the service in terms of age, gender and number of sessions attended. It was possible to follow up with 103 of these children and young people three months after they had finished the service to complete the ORS.

The qualitative data for the evaluation included interviews with children who had completed the programme, foster carers, referrers and NSPCC practitioners. A total of 18 interviews with children, ten interviews with foster carers/referrers, and three focus groups with NSPCC practitioners were held.

Key findings • Before receiving the Face to Face service, 58 per cent of children

and young people had wellbeing scores indicating clinical levels of distress. By the end of the work this had reduced to 15 per cent. This data suggests that 43 percent of children had moved out of the clinical range and therefore no longer required further therapeutic support by the end of the service.

• In total just under 60 per cent of all children and young people showed a reliable improvement in their levels of well being at the end of the service. These children and young people had improved their well-being scores by at least 5 points from the beginning to the end of the sessions. This increase of 5 or more points is considered by the authors of the scale as a reliable improvement and not just a variation in peoples scoring at different points in time.

• Of the children who were most in need at beginning the service - those with clinical levels of distress - 70 per cent showed reliable improvement and ended the service with normal levels of wellbeing.

• Seventy-three per cent of children and young people reported that Face to Face had helped a lot in addressing the immediate concern that had been impacting on their emotional wellbeing.

• Three months after finishing their last Face to Face session, 84 percent of children and young people were still reporting increased levels of wellbeing, though for around 16 per cent there had been a deterioration since their final Face to Face session.

• There were no differences in outcomes related to the age of the children and young people, or whether they had a learning difficulty or not. This suggests that NSPCC practitioners have been successful in adapting the model to suit the needs of the child or young person. However, children on the edge of care were more likely to have positive outcomes compared to children in care.


• Children and young people said that Face to Face had helped them achieve change by making them feel good about themselves and by giving them practical advice about things they could do differently. Referrers and foster carers valued the quick referral process to the service so that children and young people could receive timely support. Children and young people said that one of the barriers to achieving change was the ‘formal’ atmosphere of the venues. They also felt that at times there was too much talk about describing feelings.

• Referrers and foster carers found the referral process easy and were able to see the changes in the young person. They would like to have been more involved in joint sessions with the child at the end of the work, and would like to see the service expanded to all children in need.

The evaluation did not include a comparison group, so further research is required to be confident that the improvements in outcomes are a direct result of the Face to Face service.

ConclusionIn the absence of a comparison group, it is not possible to attribute the changes reported by children and young people directly to the Face to Face service.

However, these evaluation findings demonstrate the potential of the Face to Face service to increase the emotional wellbeing of children and young people who are at risk to a level where they no longer require any further therapeutic support. These findings are very promising, especially as it might be expected that the difficulties of children in care and those on the edge of care would be entrenched and therefore not show much change during the relatively brief intervention period.

Referrers and foster carers acknowledged that there is current gap in provision to meet the emotional needs of children in care or on the edge of care. The timely, easily accessible support that puts children and young people in control of the issues discussed as well as the duration of the work were highlighted in the evaluation as important features of an alternate form of service delivery.

The solution-focused approach was valued by children and young people, as it helped build rapport with the staff, make them feel better about themselves, and give them techniques for addressing future concerns. This strengthens the evidence in favour of using the brief solution-focused approach with children and young people who are considered to be at risk of family breakdown or are already in the care system.


The evaluation has also highlighted that the process of delivering the solution-focused approach challenged some practitioners to find creative ways of engaging younger children and children with learning difficulties. The data shows no difference in outcomes relating to age of the children, or whether or not they had any learning difficulties, which indicates that the tools and techniques developed for these groups of children are valuable resources for practitioners who work with them.

The evaluation data suggests that children and young people on the edge of care are more likely to have positive outcomes compared to children in care. Although neither the service nor the evaluation was designed to test whether the Face to Face helps reduce the number of children entering care, these very promising findings for children on the edge of care suggest that the service could help to prevent family relationship breakdown, which is one of the causes of children entering care.

Foster carers and referrers suggested that having joint sessions with the young person and the other adults in the child’s life may strengthen the approach, as it would help them sustain change for the young person. This may need to be further explored, as children and young people valued the independence and confidentiality of the service, which would need to be balanced with the involvement of other adults in the work.

In conclusion, the evaluation has identified that there is a gap in service provision to meet the emotional needs of children in care and those on the edge of care. It has also evidenced the potential for these children to benefit from a service model such as the Face to Face service, and that it should be easily accessible and available to all these children.


Main rEport

Chapter 1: Introduction

1.1 Rationale for the Face to Face serviceMany children in care have experienced years of abuse, neglect and severe family problems before they come into care. Care provides an opportunity for these young people to feel safe and to reshape their lives. However, research indicates that 45 per cent of children in care have a mental disorder (Meltzer et al, 2003): a rate that is four times that of children in the general population (Ford et al, 2007). It consistently also highlights inadequate levels of support for children in care (Hooper et al 2007; Rees et al 2010; Coy 2009).

The NSPCC commissioned the Who Cares? Trust to gather the views of children in care on what sort of service they would want. More than 270 children and young people in care were consulted as part of this study. Overall, the findings revealed that they had a strong preference for seeking support from a person they can trust; a face-to-face rather than a telephone service; one that they can access quickly and is not ‘too heavy’ (Hull, 2011). It was important to them that they could bring their own problems as they see them and not be referred for things that others think they need. They also wanted the counsellor to be someone who listens very well, does not judge them or their history and has the ability to offer them an intervention that results in a tangible change in their situation.

The NSPCC Face to Face service was developed to address these preferences by offering a short-term, time-limited intervention using a solution-focused approach. The aim was to provide a timely, confidential response to children and young people in care, or those at the risk of coming into care, who present with problems that are or may be affecting their emotional health, welfare and safety. It was hoped the service could prevent difficulties escalating to a level that required more intensive, but less readily available services or support. The service did not seek to provide an alternative to long-term therapeutic care for children in care, but to examine the effectiveness of a different form of support.

It was hoped the approach would be empowering, because the NSPCC regards children and young people as ‘the experts’ on their own lives, and that it would also provide them with an opportunity to explore their aspirations. In addition, rather than repeat the pattern of criticising young people, solution-focused conversation invites them to describe the times they have achieved positive actions and outcomes.


1.2 Structure of the serviceThe Face to Face service is aimed at young people aged 5–18 (in care or on the edge of care) and offers up to eight sessions. It seeks to provide a timely and responsive service, with the aim of contacting the young person within 24 hours from the referral being received. Access to the service is via referral from professionals or self-referral.

The Face to Face service was delivered by the NSPCC at 18 sites across the UK. A total of 1,354 children and young people accessed the service during the period under review. When the service started in 2011 it was aimed only at children in care, but in 2013 some sites extended it to children at risk of being taken into care, in response to the need identified in these areas. The eligibility criteria for the service are:

• Children in care: this includes those in a children’s home; foster care; kinship care (looked after by a family member because their parents are unable to); private foster care (cared for by someone who is not a parent or ‘close relative’); a young offenders institution; a secure immigration or removal centre.

• Children at risk of coming into care: this is based on practitioner’s judgment and input from a referral agency where relevant.

• Children who have recent experience of being in care: this includes those have recently returned home from care; have recently run away from home (or care); are adopted, or are care leavers (aged up to 19 at the time of referral).

The service exclusion criteria are:

• children and young people with an immediate safeguarding concern

• children and young people who require support or advice on a children’s rights issue that would be better dealt with by an advocate, for example, issues relating to accommodation, personal allowances, household regimes etc

• children and young people involved in some types of legal proceedings, for example, criminal proceedings where the young person is a witness, or private law proceedings where contact or residency is being disputed, as this may compromise confidentiality.

Children and young people can be re-referred to the service up to three times if there are other issues they need to work on. So far, 15 per cent of all children and young people have taken up this offer.


1.3 Key elements of a solution-focused approachA solution-focused approach was chosen, as there is a large body of clinical research, particularly from the United States (Kiser, 1988; Kiser and Nunnaly, 1990; de Jong and Berg, 1998; MacDonald, 1997) that shows solution-focused brief therapy (SFBT) is an effective approach for many service users, including children and young people. It has also been used in a variety of settings such as schools, youth groups and prisons. Some features of the approach are particularly child friendly (Lethem, 1994). For example, the language used is concrete and relatively easy for even young children to grasp (focusing on ‘how/what/when/where’ questions rather than ‘why’) and it utilises children’s imagination through the visualisation of a ‘preferred future’. The approach uses rating scales that have been adapted for use with children, for example, using pictorial images of facial expressions (eg smiley faces and frowning faces); stepping stones; rungs of a ladder, or the distance from the bottom to the top of a hill.

The approach is based on the theory that the same thinking that led to the problem will not lead to the solution. The key questions in the solution-focused approach are:

• What are the best hopes from this work?

• What would your day-to-day life look like if these best hopes were realised?

• What are you already doing and/or have done in the past that might contribute towards your best hopes being realised?

• What has got better since your last session?

The approach thus focuses on strengths and life without the problem rather than a detailed analysis of problem dimensions. The model fidelity of the Face to Face sessions with the service users was tested by looking at evidence of a solution-focused approach, explicitly stating the use of the following techniques:

• references to service user’s ‘preferred future’

• references to ‘exceptions’ or ‘instances’ by the service user

• the use of scaling questions with the service user.

Most of the practitioners delivering this approach had social work qualifications. The other qualifications held by practitioners included the following fields: psychology, social care and social policy, youth work, childhood studies, leadership and management, and practice teaching. All practitioners received two days of training in solution-focused brief therapy. This training was delivered by Guy Shennan (Guy Shennan and Associates), an independent consultant who uses solution-focused practice as a counsellor, coach, supervisor and group


worker. The teaching methods included a mixture of explanation, demonstration, practice and discussion. Practice exercises were given throughout the course, as well as demonstrations in videotape extracts and by the trainer. In addition, practitioners also attended an introductory course on the needs of children in care, and participated in an online course on legislation and guidance for children in care.

1.4 Methodology The evaluation of Face to Face aimed to measure the changes in outcomes for the children and young people who received the service; to explore their experiences of the service, and the barriers and facilitators to change. To do this, a mixed-method design was used.

Quantitative component

To measure the changes in wellbeing, at each session the child or young person completed the Outcome Rating Scale (ORS), a four-item measure designed to track wellbeing outcomes. The scale comprises four dimensions:

• individual: personal or symptomatic distress or wellbeing

• interpersonal: the quality of the person’s relationships within the home

• social: the person’s view of satisfaction with work/school and relationships outside of the home

• overall: a big picture or general sense of wellbeing.

The Child ORS (CORS) was developed for children aged 6–12 in the same format as the ORS but with more child-friendly language and ‘smiley’ and ‘frowning’ faces to facilitate the child’s understanding when completing the scales. Guidance on the use of the ORS and SRS is attached in Appendix 1, along with copies of the measures in Appendix 2.

The total scores on the ORS and CORS range from 0 to 40. The authors of the scale2 have identified scores that would constitute clinical distress and those that would be considered in the normal range. The score in between is termed the ‘cut-off point’. Scores below it indicate that the young person is experiencing a clinical level of distress, while those above it are considered to be in the normal range. These cut-off points vary according to the age of the child or young person. For children aged 6–12 the cut-off point is 32; for young people aged 13–17 it is 28 and for those over 18 it is 25 points. The authors of the scale have also developed a Reliable Change Index (RCI) to identify when change is clinically significant, which is a

2 The authors of the scale are Scott. D. Miller and Barry L. Duncan (2000)


five-point change. This means that if a score changes by five points or more, the change is substantial enough to be considered robust and not just a variation in people’s scoring at the different points in time.

The strengths of the ORS for the Face to Face service are that:

• children and young people are in control of defining how the issue is impacting on them

• it is not based on measuring reduction in symptoms but attempts to measure how the issue is impacting on their quality of life

• it fits in with scaling questions of the solution-focused approach.

As indicated earlier, during the period under review 1,354 children and young people accessed the service. Of these, 611 children and young people completed the ORS/CORS at more than one session. The way this is distributed between children of different ages and in different situations is shown in Table 1.

Table 1: Number of ORS completed based on age and situation of young person Situation of child/young person Age 6–12 Age 13–17 Age 18/19 Total

On a care order 169 214 18 401

In care of extended family 63 33 1 97

At risk of coming into care 37 35 0 72

In private foster care 8 8 0 16

Adopted 4 7 1 12

Care Leaver 0 2 4 6

In custody 3 1 4

Recently returned home from care 1 2 0 3

Total 282 304 25 611

This sample is representative of all children and young people who received the service in terms of the number of sessions attended; age; gender, and whether they had a learning disability or not. Further details on the sample are attached in Appendix 4.

Qualitative component

The aim of the qualitative component was to describe the range of outcomes for children; identify the facilitators and barriers to children and young people achieving positive outcomes, and to examine perceptions about the value of the service. The qualitative component of the evaluation included interviews with children and young people who completed the programme, foster carers, referrers and NSPCC practitioners.


Children were selected purposively. Purposive sampling is used to enable the evaluation to describe and understand the full range of views and experiences within the study population. It involves setting quotas using criteria that are based on dimensions that reflect key differences in the study population that are relevant to the study’s objectives, rather than trying to ensure that the sample is statistically representative.

The first primary criterion in choosing children to be interviewed was whether they perceived there had been an improvement in their lives after receiving the service or not. This was based on their responses to the ORS/CORS. The second primary criterion was whether children were already in care or at risk of coming into care.

While the aim was to get equal numbers across categories this was not possible, largely due to difficulties in contacting children for whom there had not been an improvement. Table 2 sets out the achieved sample in terms of the primary criteria.

Table 2: Numbers of children interviewed based on pre-post changeDid not complete measures

Moved 5 points or more (RCI)

Moved less than 5 points

Total

Young people in care 1 8 3 12

Young people on edge of care 1 4 1 6

Secondary criteria were the age and gender of the children and young people. Table 3 sets out the achieved sample in terms of the secondary criteria.

Table 3: Age and gender of children interviewed Age 5–12 Age 13–17 Total

Girls 2 8 10

Boys 2 6 8

Total 4 14 18

The interviews with children lasted about 30 minutes and were conducted in the child’s home, the NSPCC service centre or school.

Interviews with significant others in the child’s life

We conducted ten interviews with ‘significant others’ in the child’s life: four interviews with foster carers and six interviews with referrers. The referrers included social workers, learning mentors, specialist teachers at children’s services, and schools. These interviews drew from cases across different sites where the service was delivered, including Liverpool, Hull, Warrington, Croydon, Birmingham, Sheffield East London and Foyle (NI). The interview schedules with children, referrers and foster carers are attached in Appendices 5 to 7.


Focus groups with NSPCC practitioners

Three focus groups were attended by a total of 22 NSPCC practitioners, representing 12 of the sites delivering the Face to Face service.

Analysis

The quantitative data was analysed using SPSS. The statistical tests used were the t-test, Chi-square and McNemar’s test. All interviews and focus groups were recorded and transcribed verbatim and the qualitative data was analysed using a framework ‘case and theme’ approach (Ritchie, J. et al, 2013).

Ethics

Prior to the study commencing it was approved by the NSPCC’s Research Ethics Committee (REC). The REC includes external professional experts and senior NSPCC staff members. This ethics governance procedure is in line with the requirements of the Economic and Social Research Council (ESRC, 2010)3 and Government Social Research Unit (GSRU, 2005)4 Research Ethics Frameworks. More details on the ethical considerations of the study are included in Appendix 8.

Limitations of the research

Although the study provides promising evidence regarding changes in outcomes, it does not include a comparison group, which means it is not possible to directly attribute change in outcomes to the service. In addition, although children and young people were followed up three months after using the service, further research would be needed to understand longer-term changes. Finally, the CORS was validated only with children from age six upwards and hence excluded the five-year-old age group (comprising approximately 2 per cent of all users) who completed feedback forms at the end of the service.

3 http://www.esrc.ac.uk/_images/framework-for-research-ethics-09-12_tcm8-4586.pdf4 http://www.civilservice.gov.uk/networks/gsr/publications

http://www.esrc.ac.uk/_images/framework-for-research-ethics-09-12_tcm8-4586.pdf

http://www.civilservice.gov.uk/networks/gsr/publications


Chapter 2: Changes in wellbeing This chapter examines the evaluation data collected from children and young people who used the Face to Face service. It focused on the three questions:

• Had the levels of wellbeing changed for young people at the end of the Face to Face service?

• Were there differences in outcomes based on the age of the child; whether they were in care or on the edge of care, or whether they had any learning difficulties?

• Was the change sustained for young people, three months after using the service?

2.1 Overall changes in wellbeing The data indicated that after using the service, on average the wellbeing of children and young people had improved. The average change was 7.3 points. This change was statistically and clinically significant for all age groups, and further information on this change according to age group is outlined in Appendix 9.

It particular, the proportion of children and young people with clinical levels of distress reduced from 58 per per cent when they started the intervention to 15 per cent by the end (see Figure 1). This indicates a substantial reduction in the number of children who would otherwise have needed further intervention after the end of the service.

Figure 1. Change for young people on the Outcome Rating Scale (ORS) and Child Outcome Rating scale (CORS)

256

355

517

94

517(85%)childrenwere abovethe cut off

94 (15%)childrenwere belowthe cut off

256 (42%)childrenwere abovethe cut off355(58%)childrenwere belowthe cut off

Cut

off

611 children completed the ORS or CORS

First Session Last Session

The number of children with clinical levels of distress reduced from 58% at the first session to 15%, a fall of 43 percentage points by the end of the service.

The authors of the ORS/CORS scale have developed cut-off points based on the age of children and young people.

• Children above the cut off point report levels of well-being in the normal range.• Children below the cut off point report clinical levels of distress indicating the need for intervention.

20 Evaluation of the face to face service

2.2 Movement of children’s scores on the wellbeing scaleThe authors of the ORS found that a change of five points or more is ‘reliable’: in other words, that it reflects a real change in wellbeing and not just a variation in people’s scoring at the different points in time. The analysis below shows the reliable changes for groups of children starting above and below the cut-off points for clinical depression.

Children who began the service below the cut-off point

Fifty-eight per cent of children and young people began below the cut-off point, indicating a clinical level of distress suggesting that they were in most need of the service. The overwhelming majority of this group experienced a reliable improvement in wellbeing, as shown in Figure 2.

Figure 2. Journey for children below cut off

The key points are that:

• Seventy per cent of these children (250 out of 355) had moved more than five points, finishing above the cut-off point. The average change for this group of children and young people was 13.6 points.

• Five per cent (17 children) had moved above the cut-off point, but this was by less than five points, with an average change of 3.7 points and hence not considered a reliable improvement.

250 showedreliableimprovement

17 moved above cut off but by less than 5 points

355children(58%) were below the cut off

Cut

off

First Session Last Session 290 (48% of 611) moved 5 points or more indicating a reliable improvement.

70% (250 of355) of those starting below the cutoff point achievedreliableimprovementto end abovethe cut offpoint.

• The authors of the scale have defined a reliable improvement as an increase of five points or more.• Percentages calculated on all children and young people who completed the ORS and CORS.

48

267 of those children (44%) moved above the cut off


48 moved lessthan 5 points

88 of those children (14%) werebelow the cut off

40

250

17


• Eleven per cent (40 children) had moved more than five points, indicating a reliable change, but this improvement was not enough to push them above the cut-off point. The average change was 11.1 points.

• Fourteen per cent (48 children) had moved less than five points by the end of the sessions and continued to remain below the cut-off point. The average change for this group was 0.5 points.

Children who began the Face to Face sessions above the cut-off point

As the previous section implies, 42 per cent of children began their first session already above the cut-off point and virtually all of these children remained above it by the end of the intervention, as shown in Figure 3.

Figure 3. Journey for children above cut off

Key points to note are that:

• Ninety-eight per cent (250 of the 256) children who began above the cut-off point had remained above it by the end of the sessions. Within this group, 59 children (23 per cent) had moved five points or more, indicating a further improvement to their wellbeing. On average they moved 7.1 points.

• Two per cent (6 of the 256) children had moved below the cut-off point. Two of these children had moved down by more than 5 points: a reliable deterioration, while four had moved down by less than five points.

The reasons for 42 per cent of children and young people starting their sessions above the cut-off point were discussed with NSPCC practitioners, who outlined the possible explanations:

256


191 moved less than five points or remained the same

256 children(42%) were above the cut off

Cut

off

First Session Last Session

59 (10% of 611) had moved 5 or more points, indicating a reliable improvement.

• The authors of the scale have defined a reliable improvement as an increase of five points or more.• Percentages calculated on all children and young people who completed the ORS and CORS.

59 250 children(41%) remainedabove thecut off

6 childrenmoved below the cut off but by less than 5 points

6 children(1%) movedbelow thecut off

191


the data indicates that

just under 60 percent

of young people (58%)

using the service

experienced a reliable

improvement in their

wellbeing.

• Some children did not wish to say they had a problem, and so marking high on the scale would help them avoid explaining this.

• Some children felt comfortable to talk about the issue impacting on them only after the first session. The ORS data provides some evidence for this: for 48 young people in this group of 191 children, the scores did dip below the cut-off point subsequent to the first session. By the end of the work, all 48 were again above the cut-off point, and their scores had increased by 5 or more points.

• Some young people had been referred by a professional, but they themselves did not see the need for any intervention and so had scored themselves high on the scale.

• Some young people did not see the evaluation as related to the issue they were discussing, and some NSPCC practitioners also viewed the evaluation measures as being separate from the sessions. The mismatch between their marks on the scale and how the issue being discussed was affecting their life, may have resulted in a higher marking.

In summary, the data indicates that just under 60 percent of young people (58%) using the service experienced a reliable improvement in their wellbeing. This includes all the young people who moved five points or more on the ORS scale between their first and last session. It comprises the 48% of children and young people who began below the cut off point and the 10% who began above the cut off point- who experienced a reliable improvement in their well being. The data also highlights that the majority of children experiencing clinical levels of distress were no longer doing so after the intervention.

2.3 Outcomes and demographic characteristics The Face to Face service was accessed by children and young people from a wide age range (5–18); children who were already in the care system as well as children who were at risk of coming into care, and children with learning disabilities. The analysis found that children and young people of all age groups were equally likely to achieve positive outcomes, as were those either with or without learning difficulties. However, children coming into care were more likely to achieve reliable change. These results are summarised in Table 4.


Table 4: Relationship between outcomes and age, status of child and learning disabilities of children using the Face to Face serviceCharacteristic Categories % achieving

reliable change Result of analysis*

Age 6–8 (n=81) 57.8 No statistically significant difference9–12 (n=201) 57.2

13–18 (n=329) 56.8

Living circumstances

Child in care (n=539)

55.4 Statistically significant difference

Child at risk of coming into care (n=72)

69.9

Learning Difficulties

No recorded learning difficulties (n=578)

57.2 No statistically significant difference

Learning difficulties(n=33)

54.5

* Statistical significance was examined using Chi-square tests

These findings suggest that age and learning difficulties were not barriers to children achieving positive outcomes. It also highlights the potential of the service to intervene early with children who are on the edge of care.

2.4 Changes three months after the serviceTo understand whether change experienced by children and young people was sustained after the end of the service, we followed up a sample of 119 young people three months after completing the intervention. The reasons for not following up with all children and young people who completed the evaluation were: young person having moved placement; young person not having consented to being contacted; inability to contact the young person despite several attempts, and case load pressure that made it difficult to make contact with the young person for follow-up at three months.

During this follow-up, children and young people were asked to complete the ORS (103 of the 119 completed this) as well as answer a short survey. Figure 4 illustrates that for the majority of children and young people who responded, improvements in wellbeing had been sustained.


Figure 4. Three month follow up of 103 young people

Key points to note are that:

• Eighty-four per cent of all children and young people who were above the cut-off point at the end of the Face to Face sessions had continued to maintain that improvement, three months after the service.

• At the three-month follow-up, 20 per cent of young people were experiencing clinical levels of distress and this was still significantly lower that the proportion of young people who had been in clinical distress at their first session (53 per cent).

A note of caution is needed in interpreting this data. The profile of children and young people who completed the three-month follow-up is similar in terms of age and gender to the 611 who completed the evaluation; the scores of this sample at the first session were also similar. However, the children for whom there was follow-up data are not representative of all children and young people in terms of their scores at the last session. In the three-month follow-up sample (n=103), 95 per cent of children and young people ended the work in the normal range; this compared to 85 per cent of children and young people in the normal range for the entire evaluation sample (n=611). The three-month follow-up sample therefore has a higher proportion of children and young people who had positive outcomes by their last session.

Cut

off

First Session Last Session 3 months

• More than half of all young people (53%) were in clinical levels of distress at the start of the work but only a fifth (20%) reported clinical levels of distress three months after finishing the work.

• There was a 15 percentage points deterioration in children’s level of well being, three months after finishing compared to when they finished the work.

We were more likely to follow up with children and young people who ended the work positively and hence need to consider results with caution.

48 children (47%) were above the cut off



55 children (53%) were below the cut off




The follow-up questionnaire attempted to understand if things had changed or not for the young person; if they had access to any other support after the Face to Face service, and for them to identify any significant changes during this time. The data indicates that over two-thirds of children and young people in this sample reported that things had got better since their last Face to Face session. About a quarter of young people reported that things had been positively sustained, but 4 per cent of young people perceived that the problem still affected them. These findings are set out in Table 5.

Table 5: Perception of change, three months after using Face to FaceHave things changed for you? n=119

Things improved 70%

Things stayed better 26%

No change, problem still affects me 4%

Children and young people were also asked about the services they had accessed since finishing the Face to Face work, as shown in Table 6.

Table 6: Services accessed after the Face to Face serviceService N=119 (%)

CAMHS/counselling service 9%

Regular contact with social worker 41%

No contact with any service 50%

Half of the young people in the sample had not accessed any further support from services, though many of the children in care would have an allocated social worker. The other half identified support they were receiving from CAMHS, children’s services and other counselling services.


almost three-quarters

(73 per cent) felt the

service had helped

them a lot in solving their

immediate concern.

similar percentages

felt it had helped a

lot in increasing their

confidence, making

them more hopeful, and

helping them to feel

more in control

Chapter 3: Children and young people’s experiences of change The previous chapter indicated that a majority of children and young people’s wellbeing improved after using the Face to Face service. This chapter describes how these changes were experienced in their daily lives.

3.1 Achieving goalsThe underlying ethos of the Face to Face service is that children and young people identify their own goal or concern that they want to change. An analysis of case records5 showed that the range of issues the child or young person wanted to change included:

• developing safer strategies to cope with emotions such as anger, rejection, isolation, loss or anxiety, or managing associated behavioural issues

• increasing their self-confidence and being able to engage in healthy relationships

• improving communication with professionals in their life, foster carers, family members or siblings

• better learning in school.

Using the solution-focused approach, practitioners attempt to help children and young people focus on an achievable goal and enable them to identify and use their strengths to work towards this. According to the questionnaires that young people completed at the end of the programme, almost three-quarters (73 per cent) felt the service had helped them a lot in solving their immediate concern. Similar percentages felt it had helped a lot in increasing their confidence, making them more hopeful, and helping them to feel more in control (see Table 7). Overall, very few young people felt the service made no difference or did not help.

5 A random analysis of 100 case recordings across six locations, sampled on age and source of referral.


Table 7: What changed after the Face to Face service and how much (n=459)

Helped a lot (%)

Helped a little (%)

Made no difference (%)

Did not help(%)

Increasing my confidence to get help

81 14 5 0

Making me more hopeful

75 18 5 2

Solving my immediate concern

73 22 5 0

Helping me feel more in control of my life

70 26 4 0

3.2 Changes in self-perceptionsThe intervention was most successful at the core of the change experienced by young people, in that there was a fundamental shift in the way they thought about themselves and their situations. They felt they were becoming more confident and had a greater sense of self-esteem. This increased sense of confidence enabled them to:

• Learn to accept the past

Young people noticed a change in their attitude towards the past. They had learnt to accept several emotions related to the past by concentrating on making things better in the present. This change was also noticed by people around the young person, as one foster carer observed:

unfortunately, one of the girls had lost her mother to cancer

six weeks before she came to us and we were not allowed

to use the words ‘cancer’, ‘dying; she would just put her

hands over her ears. she’s been having these [face to

face] sessions with workers; she’s into her second lot of

eight now. when something comes on tv, like MacMillan

or whatever, and they use the word ‘cancer’ now, it won’t

upset her. she’d normally be, “turn the tv off. i don’t like it.

i don’t like that word”. she’s much more comfortable. and

also we’ve just had Mother’s day quite recently. she found

that very difficult last year. this year it was more about she

wanted to lay flowers and she wanted to think about the

nice things that she did with mum, which was definitely

something that we found was really a big step forward

for her.

Foster carer of 11-year-old girl


the increased

confidence of children

and young people to

take a lead to think

and act differently

suggests that this

could not only help their

present concern but

also influence future

problems they may

encounter.

• Understand triggers of emotions and take steps to resolve concerns

Young people noticed that they were now more aware of where their anger, frustration or disappointment was coming from. This awareness had enabled them to think about how they could approach situations differently, and help them to apply a forward-looking ‘lens’ to situations, as this young person described:

i don’t know if ‘preventing technique’ is the sort of word.

i think that’s changed. i think i talk more about it whereas

before i used to just bottle it up…i’ve got something wrong

with my eyes and i can’t see as well as other people, so to

talk to the teacher about it instead of getting frustrated and

just wanting to break or hurt someone. …it’s really been like

trying to prevent the inevitable, really.

Child in care, age 14

• Be more willing to ask for and accept help

Prior to receiving the Face to Face service, some young people felt they would push other people away as they did not believe that talking would really make a difference. However, after the Face to Face work they felt more confident and had a greater motivation to talk about their concerns and feelings:

i always knew i should talk to someone when i was worried,

but i was scared to try. But those sessions just made me

try. i feel a better person, as now i can talk more to people.

if i had something bothering me, i can go and talk to them

because before i couldn’t have done that because i would

have felt ‘nobody’s listening, who cares’? But now, i know

there’s people out there that do care and do want to help

me. so i feel a better person for managing to talk.


The increased confidence of children and young people to take a lead to think and act differently suggests that this could not only help their present concern but also influence future problems they may encounter. This supports the three-month follow-up analysis, which found that children and young people felt that things had improved further after they finished using the Face to the Face service.


3.3 Changes within the familySome young people noticed that the quality of the relationships with people they lived with had improved. These changes, especially with children at risk of being taken into care, seemed to have the effect of strengthening strained relationships, as one young person observed:

when i woke up in the morning he [dad] usually would be

grumpy. i’d be grumpy as well, so it wasn’t just his fault. we

would start arguing from the morning and then i’d go to my

friends and when i got back we’d start arguing. [nspcc worker]

just told me when i wake up in the morning to be more happy….

he said: ‘what would your miracle day be,’ and he told me to

think of my miracle day. now me and my dad don’t argue when

i wake up. he helps me do the housework, because he never

usually lets someone help him with the housework. we just get

along, watch the same shows and joke about stuff, just joke

around and play football together.

Young person on edge of care, age 10

Some social workers commented in interviews that the young people they had referred seemed more settled in their placements after finishing the Face to Face sessions, especially those who had recently come into care.

the issue we had was about a fostered young person who

considered himself on the outside of the family. he didn’t want to

heavily engage within the family and… what came out of those

[face to face] sessions and talking, he is now inclusive within the

family. he wants to be part of it and he does engage better with

everyone in the household, less conflict, which is what we had

prior to those sessions. it’s partly his coming to a new family and

that’s taken some time... But i do think the nspcc… just got him

to think about things and got him focused in sort of the areas that

we were concerned about.

Children’s services social worker

The key changes that children and young people identified in their family/close relationships after the Face to Face work included:

• better verbalisation of feelings to their parent or foster carer instead of acting them out

• greater motivation to participate in family activities


• more willingness to share and contribute to family chores

• greater empathy for family members

• greater acceptance of loss of key relationships

• less arguing with siblings

• better communication with family members

• better relationships with family members.

Foster carers noted that though the work had been short term, it had often resulted in significant behavioural changes for young people:

[…] but we shifted a lot actually, since those eight sessions,

because whereas before she didn’t always introduce me as mum

– i was a carer – she’s doing that a lot more now, and calling me

her mum in front of other people rather than just within the family.

and also to her siblings, which i think is a big step forward: she

refers to me as mum with her siblings, and i think that’s a big

step. again, i think that’s coming to terms with the loss, and also i

know the worker is doing work with her on [that] it’s oK to call me

mum and have me as mum and that doesn’t mean there’s any

disrespect to her birth mum, obviously.

Foster carer of 11-year-old girl

Family relationships evoked many emotions for some young people in care. NSPCC practitioners observed that in some instances, young people chose unachievable goals in terms of who they would like to live with. These and other challenges, including joint work with family members, are discussed in the next chapter.

3.4 Changes at schoolPositive changes in their school life after the Face to Face work were described by young people, and these changes were also observed by referrers and foster carers. As one young person highlighted:

… i had … issues in school in that i was kicking off and used

to run out. and one time i got suspended, but after meeting

[nspcc worker] my attitude has changed now. My head teacher

has noticed that, because before i used to go to his office

every single day because i was naughty. i kept running out the

classroom and being bad, but now i only go to the office to get...

prizes, certificates and that.



referrers observed that

after the face to face

service, some young

people were using

strategies they had

learnt during the session

and this had been

successful in changing

their behaviour in the

classroom.

The range of changes in their school life that children and young people identified included:

• less disruptive behaviour in class, and as a result not getting detentions or into trouble

• greater motivation to attend school

• better learning

• improved friendships within the class.

Referrers observed that after the Face to Face service, some young people were using strategies they had learnt during the session and this had been successful in changing their behaviour in the classroom.

we’ve got a purple room in school, which is just like a padded

open room for the children to use as one-to-one – as a time-out,

really, and he was using that a lot of the time and he was getting

really angry and walking out of class a lot. now he was able to

put things into place that [nspcc worker] had used with him.

and it could be little interventions such as massaging his palm,

or counting, or thinking of a special place, and recognising why

he was feeling like that and speaking to staff about it. so he was

definitely able to use a number of tools that were taught to him at

the sessions.

Learning mentor at a school

NSPCC practitioners said that when young people presented with concerns related to school, in some cases it was important to work closely with staff within school and have joint sessions with the young person to ensure changes identified by the young person would be sustained.

3.5 Changes in social interactions and peer relationships A key benefit of the service that some children and young people highlighted was that they became more confident in their social interactions. They felt less afraid of how other people might perceive them and more confident in making new friends.

well, in the past, whenever i was out with my friends, i’d be the

quiet one sitting in the background and not knowing what to say,

and now i’m starting the conversations! i now feel i have a life. i’m

not scared anymore to go out with them.

Young person in care, age 15


Foster carers and referrers emphasised the level of isolation that children and young people in care sometimes experience, which could contribute to them engaging in risky behaviour, such as using alcohol or substances, absenteeism from school or acting out bullying behaviours. They noticed that after the Face to Face service, some young people were more able to develop healthy relationships with their peers.

she was struggling a great deal at school with her socialising.

she doesn’t always perceive situations as they are. i know that

her worker is also doing some work on that, on the social skills.

in actual fact she was having friends and dropping them like hot

potatoes and falling out with them. she’s now actually managing,

starting to build a little core group of friends at school. i think that

is in some respects due to the work that she’s been doing with

the worker around the peer relationships.

Foster carer of 13-year-old

Referrers also said that they had seen a marked change in the social interactions of some young people, which was helping them to ‘come out of their shell’ and develop their talents. As a social worker who had referred a young person in care commented:

oh great, yes, we do appreciate what face to face has done

to my young person because previously she used to refuse to

get engaged into any activities at all and she was so adamant

that she didn’t want to do anything at all, but after working with

face to face she’s now engaged in different activities, she’s now

enrolled in a stage course. yes, she’s doing different activities,

music, dancing and drumming, via face to face.

Children’s services social worker

The interviews with children and young people, foster carers and referrers have demonstrated a wide range of positive outcomes for the Face to Face service. The next chapter analyses how the Face to Face service helped children and young people achieve these outcomes and identifies some of the barriers to achieving change.


the face to face work

enabled them to think

about one key issue

that they would like

to change and help

them think why it was

so important for them

to change.

Chapter 4: How Face to Face helped This chapter explores how the Face to Face service helped children and young people achieve positive change, and the barriers within the service to them achieving change. The analysis draws on the interviews with children and young people, foster carers, referrers and focus groups with practitioners.

4.1 Key aspects of the approach Young people identified three key areas that helped make the approach effective: the techniques used by practitioners; the relational framework within which the approach was delivered, and the nature and operation of the service. Each of these is discussed below.

Techniques

Four techniques used by practitioners were felt to be particularly important in helping young people achieve their goals, and these mirrored the principles underlying the solution-focused approach. They were:

Focusing on one issue

While some young people had specific goals they wanted to achieve, others had more general ones in terms of increasing confidence or managing anger. The Face to Face work enabled them to think about one key issue that they would like to change and help them think why it was so important for them to change.

i did not know where to begin as i felt i just too much stuff to

sort out. But [nspcc worker] helped me break this down,

because if i worked on my confidence in going out, i could then

make friends.


Practitioners observed that sometimes young people had ‘unachievable’ goals in terms of their care arrangements. This was a barrier when the young person refused to work on any other goal and did not want to focus on thinking about managing the loss or rejection of that relationship. The solution-focused approach, however, facilitated having an honest conversation about this at the start of the work.


young people felt the

sessions had helped

them think of practical

things they could do to

change things.

Increasing awareness of their strengths

Young people felt that the Face to Face service had helped them discover the good things in themselves and that this increased their confidence. They were able to appreciate the small changes in their behaviour each week, changing their outlook from seeing themselves as the problem to having the resources to overcome their problems.

i think it’s like there was a wall in front of you, and you just

grabbed a hammer and smashed it down. the hammer would be

like the good things about yourself, and you wouldn’t put yourself

down and you would say something good about yourself.


Thinking of their preferred future

Many children and young people found the process of thinking of their preferred future a bit ‘weird’ as they had never done anything like it before. Despite this, it was described as being very helpful, and visualising what it would look and feel like made them more hopeful for the future.

My social worker always looked at what i was doing. he said,

“you shouldn’t be doing this, you shouldn’t be doing that.” and,

“why are you doing this, and why are you doing that?” and i

always felt it was always jumping on the bad things that i’d done,

but in the face to face, [nspcc worker] was always looking

for the good things i can do…i never thought that way, i’d never

done anything like it before; i just thought to myself ‘i’m going to

go to jail anyway, so i might as well just do what i want.’ But then

he changed all this and he says what you can do in the future

and i started thinking more…about the future. he made me think

that i can do stuff, that i can do better things and i will do better

things. so it was very helpful.


Rehearsing strategies

Young people felt the sessions had helped them think of practical things they could do to change things. They liked the fact that they could rehearse strategies through role plays during sessions to test them out and then report back in the following session.


sometimes he’ll become the social worker and i’ll try to talk to

him and we do it like that. My feelings are never really changed

but i think slowly – he gives me ways to talk to them, which

makes me kind of get somewhere without getting angry. Before,

when i got angry, i would swear and just lose my temper but he’s

given me… techniques to talk to them [social workers] in…the

appropriate way that makes them listen.


Young people also valued the practical scaling techniques that they had learnt through the session. These scaling techniques required the young person to rate their situation on a number from 1 to 10 and describe what a higher number on the scale would feel like for them. This motivated them to keep on looking for and talking about the positive change they had experienced.

i could see them myself, so that’s why i was noticing every time

the scale went up i was thinking, ‘right keep climbing, don’t go

down, keep climbing.’

Child on edge of care, age 12

Relational framework

The second aspect of the approach that young people highlighted as particularly important was how the relationship between the practitioner, the young person and their family was framed, and the balance that was achieved. There were two main elements to this:

Children and young people leading the work

The approach helped young people feel more in control than they had experienced with previous services. The impact of this was that it gave them an increased awareness that there were actions they could take themselves to resolve situations rather than relying on others, and this increased their motivation to change.

Because if someone else has said… “right go and do this. go

and talk to this person,” then you’ll feel pressurised and you don’t

want to, you’re scared, whereas if you’re thinking… ‘i could do it

myself,’ then you find you make it easier for yourself.



The control young people were given included the flexibility offered to them in terms of the choice of venue as well as when they wanted to end the work, which reinforced their engagement through the sessions:

it was good because you were in control, some young people

sit there and be worried about not wanting to go because they

were letting someone down. you could make an appointment

whenever you were feeling alright to go. Knowing that helped me

through the work.


However, practitioners pointed out that it was not always possible to offer many choices of venue to the young person. This was either because they needed somewhere quiet, or due to logistical issues such as the availability of transport, or the fact that young people using the service came from a wide area, which meant it was not practical to meet somewhere other than at the NSPCC service centre or school. Some young people said that they would have liked to meet in more informal venues, such as the park or a café, especially during the earlier sessions, as that may have alleviated some of their anxiety about the initial meeting.

Involving family members in work

Children at risk of going into care found it especially helpful to have family members involved in some of the sessions, as it enabled them to communicate feelings in a safe environment:

i think she [mum] came in for a couple of sessions…just talking

about …the way she would make me feel sometimes if i was

feeling sad. and i found it difficult to talk to her. and i brought

her in [to the sessions] so that i could talk to her there without…

at the house maybe i would say something wrong, and maybe

it would just get into an argument. and with the [nspcc

practitioner] there it felt like that wouldn’t happen. so it was a

safer place to actually tell her how i felt.


Foster carers who had not been involved in the sessions felt that more of their involvement at the end of the work would have been helpful to take things forward with the young person.


Nature and operation of the service

The final aspects of the service highlighted by young people and referrers as being important to its effectiveness were related to the nature of the service and the way it operated:

Independence of the service

The fact that the service was seen as independent was important, because it enabled the sessions to be perceived as a confidential space, separate from young people’s foster carer, family or their social worker. This helped young people to talk about issues that they would not have been comfortable discussing otherwise.

it was different, definitely. Because i wasn’t that comfortable to

talk with my social worker because that goes to my carer, but this

session is private; i believe that because she [nspcc worker]

kept saying that she’s not going to tell anybody about my story

and ‘it’s going to be just you and i, so don’t worry.’ so i could say

anything to her.


Although referrers appreciated the importance of the work being confidential, they still said they would have liked to be involved in a joint session with the child, especially for younger children. They felt that this was important to maintain continuity in the support the child was receiving. While they valued the written information they received at the end of the work, they felt that a joint session with the NSPCC practitioners could have helped break down barriers that young people had about engagement with professionals in their life.

Easy referral process

The timeliness of the support was an important factor that helped young people achieve positive outcomes. As a foster carer commented, it was important that they could access a service as soon as possible after the young person had agreed to engage, to avoid missing an opportunity for change.

it was not a crisis situation, but we had just got the young person

to a point he was ready to talk to someone, and then the social

worker told us it would be take at least four months to see

someone at [a therapeutic service]. they then told us about the


nspcc service. i had not realised that i could make the referral

myself. i called and the next week we had it all arranged, as they

did not have any waiting list. that was amazing.

Foster carer of 8-year-old

Some referrers, however, felt that the referral criteria – that children and young people should be at risk of going into care – was ambiguous, and felt that the service should be available to all children in schools who may have issues affecting their emotional wellbeing.

4.2 Barriers to change In general, NSPCC practitioners were positive about the service and felt that it had the potential to improve the wellbeing of young people. However, they identified three issues that undermined its effectiveness. These were:

Confidence and consistency

Practitioners felt that the two-day training on the solution-focused approach had not sufficiently prepared them for the work, especially in adapting it to the specific needs of children. Initially they had lacked a sense of clarity on the extent to which other adults/professionals in the child’s life could be involved in the work, and how important it was to have the work finished in only eight sessions. However, once there was an understanding that teams could be creative in adapting the solution-focused techniques, and that the guiding principle of the work was that it was child-led, experiences were shared within and between teams, leading to a greater confidence among practitioners.

when we first started off i was following the model quite strictly,

and the young people found it really sort of repetitive and boring

because you’re asking the same questions like, “what else?” and

stuff like this, writing lots of lists, doing lots of ratings and stuff like

this. now i use a football pitch to talk about goals and scaling,

body maps, crystal balls to think about the preferred future and

things like that, and just got more and more creative as i’ve been

going along.

Practitioner in focus group discussion

However, this freedom also meant that some practitioners said there was a lack of consistency in how the approach was used within and across teams. It was felt that some teams used solution-focused techniques as a general approach, while others followed a solution-focused model in a purer sense.


practitioners felt that at

times they needed to do

some sessions that were

not solution-focused,

and then return to the

approach.

Addressing safeguarding concerns

Practitioners believed that as the approach does not require a formal assessment process to be undertaken before starting the intervention, within the context of a safeguarding agency such as the NSPCC, a lack of a formal assessment might mean that safeguarding issues are not picked up at the start of the work.

Practitioners felt that at times they needed to do some sessions that were not solution-focused, and then return to the approach. The activities that practitioners considered to be outside of the approach included:

• attending child protection conferences to advocate for the young person

• preparing court reports to capture the wishes of the young person

• doing short pieces of work with grandparents, parents or foster carers on topics such as attachment

• addressing disclosures made during the work, such as bullying or neglect.

Practitioners felt that to meet some of these challenges it would be necessary to have greater sharing of resources among teams about techniques that had worked; clearer guidance on involvement of other adults in the child’s life, and trained supervision to support the delivery of the intervention.

Applicability across diverse client groups

Practitioners reflected that although the approach was flexible, they struggled with engaging some children and young people. This was particularly an issue when using the technique of getting young people to visualise their preferred future, which required a minimum level of cognitive ability. They found it difficult to use this approach with:

• younger children aged 5–8

• children and young people with severe learning difficulties

• children in very unstable placements who were more concerned about the present crisis.


for some children the

approach involved

too much talking, and

at times they found

it difficult to verbalise

feelings.

For some children the approach involved too much talking, and at times they found it difficult to verbalise feelings.

getting there was sometimes difficult. [the nspcc practitioner]

kept asking me to name feelings and she said “give me a

situation when you feel like that.” that was hard.

i needed to always be thinking of words. when [the nspcc

practitioner] says “describe what does your six on your happy

scale look like,” i could not think of big words.


These children would have liked alternatives to talking, such as drawing, in order to get away from the reliance on words.

4.3 Relationship with NSPCC practitioners Children and young people, foster carers and referrers all felt that the relationship with the NSPCC practitioner was important in achieving change. The key aspects of the relationship that children and young people valued were:

• the persistence of the NSPCC practitioner to help them, which made them feel less alone

• the belief in their ability to change

• the non-judgmental attitude, which made the children and young people trust the practitioner

• the humor of the practitioner, which made them feel at ease and the work did not seem too heavy

• the ability not to rush sessions and go at children and young people’s pace, which made them feel less pressured to have to do things

• the active listening skills of the practitioners, which made them feel valued.

Practitioners felt that at times, the brief nature of the work made them feel they had to rush the process of building rapport in the initial sessions, because they were aware they needed to finish the piece of work within eight sessions. To overcome this, some practitioners had spent more time on getting to know the child’s interests and life outside before talking about the precise goal that the young person wanted to achieve from the service. They felt that this problem-free talk was a very important component of the approach as well, helping to build trust with the young person, who might then feel more comfortable about disclosing concerns about abuse or neglect.


However, children and young people generally thought that the work took the correct length of time. At the start of the work, some young people had been concerned that eight sessions might be too short a time for them to achieve change. However, once the sessions had started they felt a natural conclusion had been reached when they had achieved their goals, either at or before the end of eight sessions. Others had ended their work before completing all eight sessions, even though they had not completely achieved their goals. They felt they had learnt sufficient skills to manage their situation, and that they did not need more input from the sessions. They needed to see more change before they knew they had reached their goal.

Practitioners reflected that while most of their previous work was child-led, the solution-focused approach had given them a new perspective in helping children and young people to define their own problems, and then search for their own solutions.


Chapter 5: Conclusion Children and young people who used the Face to Face service reported an increase in wellbeing, and there is a particularly striking level of improvement for those who started with a clinical level of distress. There is evidence that this change was sustained three months after the end of the service. The data also suggests that after using the service there were substantially fewer children who reported the need for additional services to address their concern. Children and young people said that an important factor in the effectiveness of the service was that they felt able to trust the NSPCC practitioners throughout the work. Also important was the fact that the approach had given them an increased sense of ownership in terms of finding and implementing solutions to their concerns. The quick and easy referral process helped children and young people receive timely support. These findings strengthen the argument for having services for this group of children and young people that are accessible, child-centred and focused on identifying their strengths.

Foster carers and referrers suggested that having joint sessions with the young person and the other adults in the child’s life may reinforce the approach, as it would help them sustain change for the young person. This involvement was also valued by young people, where it had been possible to do so with their consent. However, this may need to be further explored as children and young people also valued the independence and confidentiality of the service, and this would need to be balanced with greater involvement of other adults in the work.

The service was not explicitly designed to reduce the number of children entering care and the evaluation did not specifically address this issue. However, the very promising findings for children on edge of care suggest that the service may contribute towards reducing family relationship breakdown by helping young people develop strategies to manage family conflict. More research would be required to understand whether this is enough to reduce family breakdowns.

In conclusion, the evaluation has demonstrated that children both in care and on the edge of care have reported better outcomes at the end of the service. Referrers and foster carers have highlighted the gap in service provision for therapeutic interventions that are easily accessible for children experiencing emotional difficulties. This indicates that the Face to Face service has the potential to address an unmet need in a variety of settings, including schools, to address their emotional concerns. In the absence of a comparison group it is not possible to conclusively attribute change directly to the Face to Face service. However, the findings are very promising, especially as it might be expected that the difficulties of children in care and those on the edge of care are entrenched and would therefore not show much change during the relatively brief intervention period.


BibliographyCoy, M. (2009) “Moved around like bags of rubbish nobody wants”:

how multiple placement moves make young women vulnerable to sexual exploitation. Child Abuse Review 18 (4), 254–266.

De Jong, P. and Berg, I. K. (1998). Interviewing for solutions. Pacific Grove, CA, USA: Brooks/Cole

Dixon, J. (2008) Young people leaving care: health, well-being and outcomes. Child and family social work, Vol.13 (2), 207-217.

Duncan, B. L., Sparks. J., Miller, S.D., Bohanske, R. and Claud, D. (2006) Giving youth a voice: A preliminary study of the reliability and validity of a brief outcome measure for children, adolescents, and caretakers. Journal of Brief Therapy, 5, 66-82.

ESRC (2010): Research Ethics Framework

Ford, T., Vostanis, P., Meltzer, H., and Goodman, R. (2007) Psychiatric disorder among British children looked after by local authorities: Comparison with children living in private households. The British Journal of Psychiatry, 190(4), 319–325.

Hooper, C.A. et al (2007) Living with Hardship 24/7: the diverse experiences of families in poverty in England. York: The Frank Buttle Trust.

Hull. V. (2011) Trust is everything: Looked after children and young people’s views on counselling and support services. The Who Cares? Trust

Kiser, D. (1988). A follow-up study conducted at the Brief Family Therapy Center. Unpublished manuscript.

Kiser, D., and Nunnally, E. (1990). The relationship between treatment length and goal achievement in solution-focused therapy. Unpublished manuscript.

Macdonald, A. J. (1997) Brief therapy in adult psychiatry: Further outcomes. Journal of Family Therapy, 19, 213-222.

Meltzer, H., Corbin, T., Gatward, R., Goodman, R., and Ford, T. (2003) The mental health of young people looked after by local authorities in England. Office for National Statistics, London: HMSO.

Rees, G., Bradshaw, J., Goswami, H. and Keung, A. (2010) Understanding Children’s Well-being: A National Survey of Young People’s Well-being. London: The Children’s Society.


Ritchie, J. et al (2013) Qualitative Research Practice. London: SAGE

Timimi S., Tetley D., Burgoine W., Walker G. (2013) Outcome Orientated Child and Adolescent Mental Health Services (OO-CAMHS): a whole service model. Clinical Child Psychology and Psychiatry 2013; 18: 169-84.


Appendix 1: Guidance on use of the ORS and SRS

What do we want to learn from the evaluation of this commission?

The NSPCC strategy puts learning at its heart and in order to help more children than those we can support directly, we need to have a robust evaluation strategy in place for every service. Through the Face to Face evaluation we aim to gather evidence of the changes that children experience in their level of distress with the particular issue that they seek to address. We will try to understand the role of the solution-focused work either in helping to address issues before they escalate, or in supporting children while they wait for more intensive services.

The evaluation measure that we are using to evidence change is the Outcome Rating Scale (ORS). This measure, along with questionnaires completed at the first and last session, the three-month follow-up and a sample of interviews, will give us evidence from the perspective of children, professionals and practitioners.

What are the different evaluation tools I need to use?

The evaluation tools offered to children and young people are: to complete a questionnaire at the end of their first and last session; to complete the ORS and the Session Rating Scales (SRS) at each session; to complete the ORS and a short questionnaire three months after they have completed work.

What is the difference between the Outcome Rating Scale (ORS) and the Session Rating Scale (SRS)?

The ORS is completed at the beginning or close to the beginning of every session to enable the young person to reflect on changes for them. The SRS is completed at the end of every session to give the young person an opportunity to talk about their relationship with the practitioner. The SRS is not reported on in the evaluation.

When should I introduce the ORS at the first session?

At the first session, after rapport has been built with the young person and there has been problem-free talk, the ORS could be introduced along the lines of:


Before we get started, I would be grateful if you could help me out by taking a bit of time to fill out a very brief questionnaire to help me understand how things are going for you. Every time we meet I would like to ask you to fill out this form again, as it will help us track progress and see whether what we are doing here is making a difference to you or not. Are you OK with that?

If the young person asks for clarification about one of the four subscales, these could be explained in the following ways (depending on the age of the child):

• Individually: This asks you to think about how you feel in yourself. So thinking about the last few weeks, where this side (left) is you have been feeling unhappy or angry or frustrated and so on, and (pointing to the right) you have been feeling really happy and good and relaxed and so on. Where would you rate yourself?

• Interpersonally: This is about how you are getting on with your family, carers or other people you are close to – generally speaking, the people you are living with.

• Socially: This is about your life socially, outside the home, which is mainly to do with how you are getting on at school and with friendships

• Overall: One way to think about this is to think about where you are in your life right now, compared to where you expected or wanted to be.

Should I feed back the ORS results to the young person?

Yes, to make the ORS scoring meaningful for the young person it is advisable to let them know the total scores to enable them to see change. This could be done through plotting the scores on the Excel or paper template. The scores provide an opportunity to talk about things the young person has done to bring about change or the resources they have had to draw upon to get them through difficult times. Please refer to the T-drive for further information about the measure and the cut-off points for age groups.

What should I be mindful of while using the ORS?

The ORS is not intended as a tool to structure the session or as an assessment tool to probe areas that the young person may not wish to discuss with us. During the session, the more linkages are made between what the young person says and the marking on the scale, the more relevant the ORS will be for the young person. If it is felt that the ORS score bears no relation to what the young person is talking about during the session, they could be invited to do it again if they want to.


How do I use the parent/carer version of the ORS?

Some teams have reported that younger children (age 5–8) find it difficult to complete the ORS. In order to capture change for these children it is possible to ask the parent/carer who accompanies the child to fill out the ORS as well, based on their perception of how the child is doing. This may not be possible in all situations and child service practitioners can make their own judgements when this would be appropriate. It may be most appropriate when the parent/carer is involved in the sessions with the young child.

How could I introduce the Session Rating Scale to the young person?

The SRS could be introduced to the young person along the lines of:

We need to end soon, but before we do I would be grateful if you could fill out this brief questionnaire which asks your opinion about today’s session. The feedback you give me helps me understand your opinion about today’s meeting and in particular alerts us to anything that was maybe missing or did not go quite right. The more feedback you can give the better, so please try and be as honest as you can when filling this out. Is that OK?

How is the evaluation done after three months (T3)?

The aim of the three-month follow-up measure is to help us evaluate if:

• the change achieved by the young person at the end of the intervention has been sustained or not

• whether the young person has accessed any other services and what they were

• the reasons that the young person may have for things having improved or not.

Ideally, the same practitioner who worked with the young person will contact them to complete these follow-up measures. We are able to offer a £5 voucher to young people who complete the T3 evaluation. The preference is for the T3 measures to be completed face to face, but if this is not possible, then sent through the post in a stamped return envelope to the service centre. Unfortunately we do not have the copyright permissions to send the questionnaire electronically.


Appendix 2: ORS and CORS

Outcome Rating Scale (ORS)

Name ________________________

Date: _________________________

Looking back over the last week, including today, help me understand how you have been feeling by rating how well you have been doing in the following areas of your life, where marks to the left represent low levels and marks to the right indicate high levels. If you are filling out this form for another person, please fill out according to how you think he or she is doing.

Individually

(Personal wellbeing)

❘––––––––––––––––––––––––––––––––––––––––––––––––––––❘

Interpersonally

(Family, close relationships)

❘––––––––––––––––––––––––––––––––––––––––––––––––––––❘

Socially

(Work, school, friendships)

❘––––––––––––––––––––––––––––––––––––––––––––––––––––❘

Overall

(General sense of well-being)

❘––––––––––––––––––––––––––––––––––––––––––––––––––––❘

Institute for the Study of Therapeutic Change

www.talkingcure.com

© 2000, Scott D. Miller and Barry L. Duncan


Child Outcome Rating Scale (ORS)

Name ________________________Age (Yrs):____ Sex: M / F

Session # ____ Date: ________________________

Who is filling out this form? Please check one:

Self ____ Other ____

If other, what is your relationship to this person? _____________________

Looking back over the last week, including today, help us under-stand how you have been feeling by rating how well you have been doing in the following areas of your life, where marks to the left represent low levels and marks to the right indicate high levels.

Me

❘––––––––––––––––––––––––––––––––––––––––––❘

Family

(Family, close relationships)

❘––––––––––––––––––––––––––––––––––––––––––❘

School

(Work, school, friendships)

❘––––––––––––––––––––––––––––––––––––––––––❘

Everything

(General sense of well-being)

❘––––––––––––––––––––––––––––––––––––––––––❘


www.talkingcure.com

© 2000, Scott D. Miller and Barry L. Duncan


Appendix 3: SRS and CSRS

Child Session Rating Scale (CSRS)

Name ________________________ Age (Yrs): ____

Sex: M / F

Session # ____ Date: ________________________

How was our time together today? Please put a mark on the lines below to let us know how you feel.

Listening

Did not always

listen to me.

❘–––––––––––––––––––––––––❘

Listened to me.

How Important

What we did and talked about

was not really that important

to me.

❘–––––––––––––––––––––––––❘

What we did and talked

about was important

to me.

What We Did

I did not like what we did today.

❘–––––––––––––––––––––––––❘

I liked what we did today.

Overall

I wish we could do

something different.

❘–––––––––––––––––––––––––❘

I hope we do the same kind of things next time.


www.talkingcure.com

© 2003, Barry L. Duncan, Scott D. Miller, Jacqueline A. Sparks

http://www.talkingcure.com


Session Rating Scale (SRS V.3.0)

Name ________________________ Age (Yrs): ____

ID# _________________________ Sex: M / F

Session # ____ Date: _________________________

Please rate today’s session by placing a mark on the line nearest to the description that best fits your experience.

Relationship

I did not feel heard,

understood, and

respected.

❘–––––––––––––––––––––––––❘

I felt heard, understood,

and respected.

Goals and Topics

We did not work on or talk

about what I wanted to work

on and talk about.

❘–––––––––––––––––––––––––❘

We worked on and

talked about what I

wanted to work on and talk about.

Approach or Method

The approach is not a good fit for me.

❘–––––––––––––––––––––––––❘

The approach is a good fit for me.

Overall

There was something missing in the session

today.

❘–––––––––––––––––––––––––❘

Overall, today’s

session was right for

me.


www.talkingcure.com

© 2002, Scott D. Miller, Barry L. Duncan, & Lynn Johnson


Appendix 4: Representativeness of the evaluation sampleCriteria All users

(n=1354)Evaluation sample (n=611)

Evaluation sample(n=103)

Representative

Gender (%) Yes

Male 49 48 51

Female 51 52 49

Age (%) Yes

6–8 13.6 13.5 18.4

9–12 30.7 32.8 30.1

13–17 54.0 49.6 46.6

18–19 1.7 4.1 4.9

Percentage recorded as having a learning difficulty

6.8 5.4 6.7 Yes

Average number of sessions

8 6 6 Yes

Percentage of users who re-referred to the service

16 14 12 Yes

Note: The three-month evaluation sample (n=103) had more children and young people who had positive outcomes at the last session compared to all children and young people who took part in the evaluation.


Appendix 5: Interview questions for children and young people

Accessing the service

• How did you hear about the Face to Face service?

• What did you know about it before you had your sessions?

• Was it easy or hard to gain access to the sessions? What were the difficulties?

• Did you know what you wanted to get from the service?

About the work

• Could you tell me how many sessions you had?

• What did you choose to work on in your sessions? (eg, your best hopes for the work/ what you wanted to change in your life)

• Did you involve anybody else in your sessions (eg, foster carer or parent)?

• What was the type of things you did during the sessions?

• Was talking about things easy? (Using a scaling question of 1 to 10), describe the rating and why not above or below. What helped you to talk? Or what made it a bit difficult to talk?

• Did the work feel too long, short, or about right?

• How did you know when you were ready to stop receiving the service? (If young person had fewer than eight sessions)

• (If applicable) Was this service different in any way from others you have gone through previously? How?

• Did you do things in between your sessions to help make the changes you wanted to see?

• How did you feel about completing the ORS and SRS measures?

About changes

• Was the work you did with the Face to face worker helpful?

• If yes, what has changed and how did the sessions help?

• If not, what could have been done differently; what change would you have liked to see?

• Anything new you think you learnt about yourself?

• Do you think that the sessions will continue to help you in the future? How?


• Did you develop any new skills through your sessions?

• Did any of your relationships (with friends or family) change as a result of the work you were doing?

Suggestions for improvement

• What did you like best about the work?

• What did you not like or could have been better?


Appendix 6: Interview questions with referrers

Background to extent of use of the service:

• Introduction of self – how long in role and organisation.

• Roughly how many referrals have they made to the service?

• How did they hear about the service?

• In what ways did they feel that it would be a useful service for the young people they referred?

• What criteria do they use/ process do they go through in order to make referrals to Face to Face?

• What attracts them to refer certain young people? In what situations would they feel young people are unsuitable for the programme? Do they refer these young people on to any other services?

Experience of families referred to the service:

• Can they tell us about one particular case that they referred to Face to face where they felt the young person benefited from the service?

• What were the key issues facing the young person, which they felt the service could help with?

• Can they describe any outcomes from the programme for the child?

• Are they aware of longer-term outcomes from the programme?

• Was the service able to meet the needs of the young person in the ways they had anticipated they would?

• Were there any outcomes they hoped the service would achieve for the young person that were not achieved?

• Any cases they have referred to the service where Face to Face has not been that successful? In what ways was it unsuccessful? Were there any negative outcomes they feel are attributed to the programme? Why do they think that it did not help that particular young person ?


Views on the practice mode

• Thoughts on the structure and content of the solution-focused programme – time limited; no assessment; young person may choose to work on different goals?

• Whether the programme filled any gaps in their own work?

• Whether there is anything missing from the programme that would support their work with the young person even more?

Contact with the NSPCC:

• Level of contact with the NSPCC over the course of the young person’s involvement with Face to Face:

• Updates on changes for the young person

• Level of communication – not enough/ too much/ just right?

Existing similar services:

• Are they aware of any existing services in the local area – how does Face to Face compare?

• Are there any improvements that could be made to the programme?

• Any other comments or suggestions?


Appendix 7: Interview schedule with foster carers1) How did you hear about the Face to Face service?

a. Who told you about it?

b. What did they tell you?

c. Your initial thoughts about it?

d. Was the young person keen to go to this or did they require some coaxing?

2) What happened during the work

a. How long?

b. Where?

c. Motivation of young person through the programme?

d. Were you involved in the work? Did you feel threatened?

3). Do you think the work has helped your child in any way, or not?

(Prompts used if necessary) For example, coping with their past experiences, affecting their behaviour in any way, dealing with their emotions differently.

4a) (If yes) Please give details.

4b) (If no) Are there any reasons you think the sessions have not helped?

5) Do you feel that your relationship with your child has been affected in any way after the Face to Face service?

5a) If yes, how has it been affected? Eg, positively or negatively?

6) Were you aware of the solution-focused approach of the model?

• Ending of the work – how was that managed?

• Were you involved in discussing the issue that the young person chose to work on?

• What were the other agencies involved with the young person?

• Do you think the change will be sustained?


9) Do you feel you would know where to go for further support if you needed it?

Where do you think you would go to get this support?

10) What did you think of the worker on the programme?

Eg, in terms of how well you related to them, how easy it was to talk to them, how they dealt with the group, how they dealt with sensitive issues?

11) Do you have any suggestions for how this service could be improved?


Appendix 8: Ethical considerations of the studyAll NSPCC practitioners delivering the Face to Face service attended evaluation training in order to gain an understanding of how the evaluation worked and the key ethical considerations in carrying out the evaluation. The ethical issues central to this evaluation are listed below:

• Gaining informed consent from service users – Practitioners explained the purpose of the evaluation to service users, their role within the evaluation, and the way in which information they shared with the evaluation team would be used. The consent forms and discussion about consent let all children and young people know that they did not have to take part in the evaluation. Practitioners explained the consent form to children and young people to ensure that they understood it, and then asked them to sign if they agreed with what was being outlined. For those young people who were not assessed competent by the practitioner, parents/carers were asked to sign that they agreed that their child could take part. If parents/carers agreed to their child taking part in the evaluation, the children/young people were asked verbally if they were willing to take part in the evaluation. If a young person did not wish to take part in the evaluation, their wishes were respected, even if parents/carers had agreed for them to take part.

• Confidentiality – It was explained to children and young people that all information they shared with the evaluation team would be treated in the strictest confidence, unless they shared something with the team that raised concerns about a child’s safety.

• Option to withdraw from study – Children and young people understood at the time that they consented to be part of the evaluation, that they could change their mind at any time and withdraw their involvement or contribution to the evaluation before the final report was produced.

• Protection of participants –Practitioners were able to use their judgement to ensure that children and young people who may have felt vulnerable in an interview situation were not included in the sample for qualitative interviewing. In this way, participants were protected from any potential harm. All interviews were gently wound down at the end to ensure that participants were left feeling as upbeat as possible at the end of the interview.


• Advice/support for participants – Should any evaluation participant be left feeling in need of extra support following an interview, researchers were able to suggest services that might be able to help them. While practitioners administered measures with service users, they were able to discuss any feelings prompted by completing the measures, and to obtain advice and support directly from their NSPCC worker.

• De-briefing – All interviews and completion of measures ended with a discussion about how the child or young person had found the experience of taking part in the evaluation, to ensure that they felt comfortable with what they had just done and to provide them with the opportunity to ask any questions.


Appendix 9: Average change in ORS scores across age groupsAge groups

Mean pre- programme score

Standard deviation at pre- programme

Mean post- programme score

Standard deviation at post- programme

P value

6–12 (n=282)

29.7 8.1 36.7 5 p<0.001

13–17 (n=304)

26.2 8.6 33.4 6.9 p<0.001

18–19 (n=25)

21.5 8.4 32.1 6 p<0.001

P<0.001 indicates statistically significant change between pre-programme and post- programme scores. This data is graphically represented below along with the cut-off points.

40

35

30

25

20

15

10

5

0

6 to 12 years 13 to 17 years 18 years (n=282) (n=304) (n=25)

First session Last session Cut-off point

www.nspcc.org.ukRegistered charity numbers 216401 and SC037717. Photography by Jon Challicom, posed by models.

evaluation of the face to face service - nspcc | … · 3.3 changes within the family 29 ... 6...

Documents