evaluating the involvement of experts by experience in the

Service Evaluation Project Experts by Experience Involvement

Prepared on the Leeds D.Clin.Psychol. Programme, 2018 1

Evaluating the involvement ofExperts by Experience in the

Doctorate of Clinical PsychologyProgrammes

Emma Howarth

Commissioned by Dr. Tracey Smith

Clinical Psychologist and University of Leeds DClinPsy Clinical Tutor



Abbreviations

BPS = British Psychological Society

DoH = Department of Health

DClinPsy = Doctorate in Clinical Psychology

DCP = Division of Clinical Psychology

EbE = Expert by Experience

GTiCP = Group of Trainers in Clinical Psychology

HCPC = Health Care Professions Council

NHS = National Health Service

PICT = Psychologist in Clinical Training

SUAC/s = Service user/s and carer/s



List of Tables

Table Title Page

1 Involvement of EbE in the DClinPsy Programmes: MainCategories, Subcategories, and Examples

12

2 Barriers to EbE involvement in the DClinPsy Programmes:Category, Subcategories, and Examples

16

List of Figures

Figure Title Page

1 Programme EbE involvement response by

question area

10



Table of contents

1. Introduction ............................................................................................................... 5

EbE involvement ............................................................................................................. 5

Involvement of EbE in health training ............................................................................ 5

Commissioning and Project Aims................................................................................... 7

Research Questions ......................................................................................................... 7

2. Method ........................................................................................................................ 7

Participants ...................................................................................................................... 7

Measure ........................................................................................................................... 8

Ethical Considerations..................................................................................................... 8

Procedure......................................................................................................................... 8

Data Analysis .................................................................................................................. 8

3. Results ......................................................................................................................... 9

Question 1. ...................................................................................................................... 9

Question 2. .................................................................................................................... 16

4. Discussion ................................................................................................................. 17

Overview ....................................................................................................................... 17

DClinPsy programme EbE involvement ....................................................................... 17

Barriers of involvement................................................................................................. 18

Is there a minimum standard of involvement DClinPsy programmes could achieve? . 18

A flexible model of involvement for DClinPsy programmes ....................................... 19

Limitations and Future Research................................................................................... 20

Dissemination of Findings ............................................................................................ 20

5. References................................................................................................................. 21

6. Appendices ............................................................................................................... 24

Appendix A: Questionnaire to DClinPsy Programmes................................................. 25

Appendix B: Participant Information Sheet (PIS)......................................................... 30

Appendix C: Covering email to DClinPsy Programmes............................................... 33

Appendix E: SEP Self-Appraisal ...................................Error! Bookmark not defined.

Appendix F: SEP Commissioner Appraisal ...................Error! Bookmark not defined.



1. Introduction

EbE involvement

The involvement of first‐hand experiential knowledge in health care is widely

thought to have intrinsic value and acknowledges the importance of EbE lived experiences

and expertise (Ahuja & Williams, 2005). This value is reflected in the statements and

strategies of professional bodies (DCP, 2015, 2018; HCPC, 2017), health care providers

(NHS England, 2006), and charities for people experiencing mental health issues (Together

& NSUN, 2014). The importance of involvement is also recognised at a national level in

government strategy (DoH, 2011, 2014), which makes explicit references to the value of

EbE involvement and focus on partnership working to improve services and outcomes.

These standards help to strengthen the influence of EbE and give greater control over the

services received by all. However, the move towards bottom-up approaches to decision-

making and change does not come without potential shortcomings. EbE involvement in

mental health services can often be limited to tokenistic activity, making meaningful

change challenging (Paul & Holt, 2017). With large and powerful organisations like the

NHS, some may argue that EbE enter into a co-opting relationship rather than genuine

partnership working (Eriksson, 2018).

Involvement of EbE in health training

The involvement of EbE is also valued in health training. The HCPC regulate

sixteen different professions and their education provides, including practitioner

psychologists and issue standards of education and training guidance. The document states

that “Service users and carers must be involved in the programme”, and further clarifies

that involvement could include involving individuals or existing groups and networks via

voluntary organisations (HCPC, 2017). It also gives examples of areas EbE involvement

could happen, including admissions and selection, planning and developing the

programme, and teaching and learning activities. In addition, the BPS accreditation

standards of DClinPsy programmes highlight the importance of involving EbE through all

strands of training, stating that “service users and carers should inform and participate in

the delivery of the curriculum.” (BPS, 2014).

Research has found many benefits of EbE involvement for health professionals in

training, such as personal and professional development of trainee learning (Clarke &

Holttum, 2013; Khoo, McVicar & Brandon, 2004), improved communication skills and

increase empathy (Repper & Brezze, 2007), increase self-efficacy (McCusker et al, 2012),

increased enthusiasm and the provision of different perspectives on trainee thinking

(Harper et al, 2003), positive impact on clinical practice (Khoo, McVicar & Brandon,

2004), influence of trainee attitudes (Simpson & House, 2003).

In relation to clinical psychology training, a study by Dunning (2015) found

DClinPsy trainees reported EbE involvement increased engagement and skill development,



and provided a valuable source of advice/feedback and diverse perspectives. Clarke &

Holttum (2013) also found positive aspects of EbE involvement perceived by DClinPsy

programme staff, such as opportunities to break down barriers and develop trainees

learning. However, they also found less positive aspects of involvement, as well as a

number of barriers, such as potential tokenistic involvement, and different agendas or

opinion between EbE and staff. Similar findings are seen in a study by Berry (2016) who

investigated EbE experience of DClinPsy involvement, finding positive experiences

including a sense of ‘having a voice’, as well as less positive experiences (e.g. awareness

of academic agenda) and personal and practical barriers to involvement.

These less positive factors and barriers to involvement highlight the need for regular

evaluation to monitor progress and improve standards. The DCP’s Good Practice

Guidelines (Sheldon & Harding, 2010) emphasise the importance of evaluating

involvement so that “work is not repeated and that projects are well planned, promote

inclusive practice, and achieve the aims determined by service users and carers at the

outset”. Programmes like INVOLVE (2012) support active public involvement in health

research with the aim of advancing current research process. Paul & Holt (2017) found

NHS researchers valued the perspective EbE brought to research, but identified frustration

with the tokenistic approaches rather than meaningful involvement work. While

researchers expressed motivation and enthusiasm for EbE involvement, the study suggests

EbE involvement guidelines may not be sufficient to change research practice. One model

used to measure and evaluate involvement is Arnstein’s (1969) Ladder of Participation.

Arnstein’s model illustrates eight levels of involvement which correspond to the extent of

power participants’ are seen to have in determining outcome - the higher up the ladder

equates to higher participant power and co-production. Arnstein highlights the importance

of the redistribution of power for authentic involvement, while also acknowledging that

disingenuous motivations for involvement can still reach the top levels of the ladder.

Although Arnstein’s research is almost fifty years old, this appears to remain a current issue

illustrated by Pollard (2018) when considering “whether [her own] lived experience is

being valued intrinsically (because of the substance of these contributions) or extrinsically

(because…it's simply just seen as 'good' to include these voices)”. The differing aims and

motivations of EbE, staff and trainees for involvement in training has been suggested as a

potential challenge or barrier that needs to be recognised to aid collaborative working (Lea

et al, 2016; Holttum et al, 2011).

This highlights the need for better understanding in the power of EbE involvement

for genuine and constructive change. In their 2009 paper, Boyle & Harris (2009) discuss

the need for co-production – an “equal and reciprocal relationship between professionals,

people using services, their families and their neighbours”. This relatively new approach

to thinking about public services, including DClinPsy training, encourages service design,

delivery and evaluation in equal partnership in order to become more effective, efficient,

and sustainable. One such example of co-production is the COMMUNE project. EbE and



nursing academics developed and evaluated a co‐produced ‘mental health recovery’

module for undergraduate nursing students (Horgan et al, 2018). They found the module

enhanced understanding and emphasised the importance of self-reflection and

communication.

DClinPsy programmes have therefore begun to measure and evaluate their own

EbE involvement. Using Arnstein’s Ladder of Participation, Eames & Phillips (2017)

measured EbE involvement across the DClinPsy programmes to investigate involvement

and co-production over two time periods. They found that the levels of involvement varied

across the programmes, from low levels of power such as providing information (i.e.

sharing personal experiences in teaching sessions), to higher levels of power such as

influence in decision-making (i.e. separate EbE selection panel). While involvement levels

had increased in some strands of training, the lack of detail and depth in the data means it

was not always possible to determine how involvement was happening in each of the

programmes included.

Commissioning and Project Aims

The current project was commissioned by Dr Tracey Smith: clinical psychologist,

clinical tutor, lead for EbE involvement on the Leeds DClinPsy programme, and co-chair

of the GTiCP Involvement Sub-group. Following on from the GTiCP involvement survey

by Eames & Phillips (2017), the aim of the project was to evaluate the involvement of EbE

in the DClinPsy programmes in the UK. It was anticipated that the evaluation would update

and add further detail and depth to what and how programmes are co-ordinating EbE

involvement in addition to the barriers that prevent involvement. It was hoped the findings

would contribute toward a flexible model of involvement in Clinical Psychology

programmes in the UK.

Research Questions

1. What EbE involvement is there across the DClinPsy programmes? More

specifically how does EbE involvement happen?

2. What are the barriers to EbE involvement across the DClinPsy programmes?

2. Method

Participants

All thirty-three DClinPsy programmes across the UK were invited to take part in

the project (i.e. to complete a questionnaire). Programme staff, EbE and individuals who

identify as both were invited to complete the questionnaire. Fifteen programmes replied

giving a response rate of 45.5%. Individuals who completed the questionnaire included

twelve programme staff, eleven EbE and two individuals who identified as both.



Measure

A questionnaire was developed based on the findings of the GTiCP Academic sub-

group Involvement Survey 2015, and refined in collaboration with the project’s

commissioner and Everybody’s Voice (see Appendix A). Areas of DClinPsy programmes

covered by the questionnaire include: overall programme, selection process, clinical,

research and academic strands, other involvement, and barriers to involvement. The

questionnaire consisted of forced-choice questions and free text sections. The force-choice

questions asked for a ‘yes/no’ response about involvement in the different areas of the

DClinPsy programmes. The free text sections allowed participants to detail what and how

EbE involvement happens, or does not happen, on their programme. The questionnaire was

designed to be clear and concise in the hope of maximising the response rate.

Ethical Considerations

This project was approved by the University of Leeds School of Medicine Research

Ethics Committee DClinPsy sub-REC (DClinREC project number 17-010). Participants

were provided with a Participant Information Sheet (PIS) outlining the purpose of the

project, what would happen if they chose to take part, possible risks and benefits, issues of

confidentiality and right to withdraw (see Appendix B). All participants were given the

opportunity to ask questions prior to taking part.

Procedure

All DClinPsy programmes across the UK were invited to take part in the project

via email by Leeds programme administration team (see Appendix C). The PIS and

questionnaire were attached to the email. Informed consent was assumed when participants

returned the completed questionnaire via email. All programmes were prompted to return

questionnaires one week before the deadline via email (see Appendix D). The project was

also discussed on the GTiCP Involvement group email list.

Data Analysis

The quantitative data collected was analysed using descriptive statistics, while the

qualitative data was analysed using qualitative content analysis. Qualitative content

analysis is “a method for systematically describing the meaning of qualitative data”

(Schreier, 2014, pp. 170). The questionnaire responses were analysed using a combination

of concept-driven and data-driven approaches of qualitative content analysis (Schreier,

2014, pp. 176). The aim of this project was to identify EbE involvement and this focus was

used to generate categories and subcategories in a concept-driven way. Further categories

and subcategories were then generated in a data-driven way using a ‘subsumption’

approach (Schreier, 2014, pp. 174-180) so that all data was accounted for.

In terms of the current SEP, this approach was felt to be the most suitable for the

volume of data as “qualitative content analysis reduces data, it is systematic, and it is



flexible.” (Schreier, 2014). While the detail and depth of the responses could potentially be

lost when the data is reduced, the approach allows a “sense of how different [responses]

compare and relate to each other.” (Schreier, 2014). The use of subcategory examples

from the raw data will illustrate some of the lost detail and depth. Alternative approaches

were considered, such as thematic analysis, however it did not seem appropriate as we

wanted to use prior research to guide our coding frame.

The coding frame was initially structured in a concept-driven way, using the

different stands of training covered by the questionnaire to create categories and the

question topics to generate subcategories. The responses for each question were read

through fully to gain a sense of the data as a whole. The responses were then read again

and any words, phrases, sentences or whole response that captured involvement were

highlighted and a note of the type of involvement was made in the margin. A further data-

driven category and subcategories began to emerge from the responses. Once coding for

all responses had reached saturation, the concept- and data-driven concepts were

amalgamated to develop the final coding frame. The final subcategories were then

generated, with very similar subcategories being collapsed. Names were then developed

for each of the final subcategories and the responses for each question were read though

for a final time in order to check that all coding was consistent with the subcategory names.

Creditability Checks

Credibility checks were used in an attempt to improve the quality of the findings

and reduce potential bias, following guidelines for publication of qualitative research

(Elliott, Fischer & Rennie, 1999). The development of the coding frame was discussed with

my academic tutor and with the project’s commissioner to enhance the reliability of

findings. Further, data examples are included in the results section to support the generated

subcategories.

3. Results

Q1. What EbE involvement is there across the DClinPsy programmes? Morespecifically how does EbE involvement happen?

The questionnaire asked participants to respond to a forced choice ‘yes/no’ question

regarding involvement in a specific aspect or area of the programme. The results identified

a variation in involvement over the different questions from the programmes (n=15) that

responded, ranging from one to fifteen ‘yes’ responses.



Figure 1: Programme EbE involvement response by question area

Six categories were identified from the questionnaires in regards to what EbE

involvement is there across the DClinPsy programmes, these included: Involvement in

overall programme, Involvement in selection process, Involvement in clinical strand,

0 2 4 6 8 10 12 14 16

Informal assessment

Formal assessment

Academic developments

Formal assessment design

Programme guidelines

Developing teaching resources

Co-/running or contributing to teaching

Other teaching

Neurological teaching

Health teaching

Psychosis teaching

Forensic teaching

Learning/Intellectual Disabilities…

Older Adult teaching

Child teaching

Adult teaching

Research co-supervision

Research development & feedback

Research consultation

Research presentation feedback

Placement involvement

Final cohort decisions

Selection process

Shortlisting

Other sub-committee

Academtic sub-committee

Ethics sub-committee

Research sub-committee

Clinical sub-committee

Selection sub-committee

Involvement lead/co-ordinator

Expert by Experience Reference group

Number of "Yes" responses

Invo

lve

me

nt

qu

est

ion

are

a

Involvement of Experts by Experience (EbE) in theDClin Programmes

Programmes were asked if they have EbE involved in thefollowing areas:



Involvement in research strand, Involvement in academic strand, and Other involvement.

A total of twenty-six subcategories emerged from these six categories in regards to how

EbE involvement is happening. A category outside of the research question was also

identified: Future involvement. The categories and subcategories are presented in Table 1

with examples from the raw data.



Table 1: Involvement of EbE in the DClinPsy Programmes: Main Categories, Subcategories, and Examples



Q2. What are the barriers to EbE involvement across the DClinPsy programmes?Eight subcategories were identified in regards to the barriers to involvement across the DClinPsy programmes. The subcategories

are presented in Table 2 with examples from the raw data.

Table 2: Barriers to EbE involvement in the DClinPsy Programmes: Category, Subcategories, and Examples



4. Discussion

OverviewThe current SEP aimed to (1) obtain an overview of what EbE involvement there is

across the DClinPsy programmes, (2) identify how involvement happens, and (3) what

barriers prevent involvement. It was hoped that the evaluation would update and add further

detail and depth to the previous research within the area, contributing toward a flexible

model of involvement in DClinPsy programmes in the UK.

DClinPsy programme EbE involvementThe findings suggest there is a lot of meaningful involvement that has been

integrated into training in various ways, potentially indicating an increase from last year

(Eames & Phillips, 2017), with future ideas and plans for involvement in the pipeline for

many programmes. However, there is notable variation in the level of power and

responsibility given to EbE within the involvement reported (e.g. providing feedback and

advice to influencing decision making), and discrepancies in the amount of involvement

between the different areas of training (e.g. more involvement in academic strand than

clinical strand). All programmes reported barriers to involvement which can prevent

involvement or mean it is not as widespread as they would like, which varied from barriers

faced by EbE, by programmes, and bigger systemic barriers.

While the overall trend of the results are consistent with previous research (Eames

& Phillips, 2017; Briggs et al, 2017; Berry, 2016; Dunning, 2015; Holttum et al, 2011), the

many subcategories of involvement identified offer a more in-depth insight into how

involvement is happening within DClinPsy programmes, braking down the different areas

of training to gain further understanding. For example, there is a large amount of EbE

involvement in teaching consisting of sharing experiential knowledge in all teaching block

areas apart from forensic teaching, and in co-/running or contributing to sessions. While

previous research found EbE involvement within the research strand of training (Eames &

Phillips, 2017), the current findings show the majority of programmes have involvement

in research consultation, development and feedback, and around half have involvement in

presentation feedback and co-supervising research projects.

Interesting, the findings also identify involvement outside of the usual training

strands focused on (selection, clinical, research, academic), such as programme staff

interviews, one-to-one EbE trainee mentoring or advising schemes, involvement in various

programme stakeholder events. This involvement could be viewed as a move towards

genuine partnership working and a more co-production orientated culture (Boyle & Harris,

2008) within the profession, going beyond what is required from HCPC (2017) and BPS

(2014) standards, and beginning to move the balance of power and influence from

professionals.



Barriers of involvementThe findings of the SEP highlighted a number of barriers faced by EbE and

DClinPsy programmes that prevent involvement or make involvement challenging. A

common barrier that was mentioned by the majority of the programmes was a sense of

limited resources from both EbE (e.g. availability, financial strains) and programmes (e.g.

funding, time). It could be argued these challenges are a reflection of a larger, systemic

difficulty experienced by the NHS and other publicly funded services (Boyle & Harris,

2009). However involvement on some programmes appear to be less impacted by this

barrier, and there is evidence that innovative, successful examples of co-production “have

arisen in places where problems seem most intractable” (Boyle & Harris, 2009). Therefore

better understanding what these programmes are doing may allow others to learn and

replicate.

Other barriers such as lack of representation, not having enough EbE to facilitate

involvement and difficulties recruiting individuals into programme reference groups are

similar findings in previous SEPs (Berry, 2016; Dunning, 2015), with issues of diversity

also being raised as a wider issue for the profession as a whole (Turpin & Coleman, 2010).

Is there a minimum standard of involvement DClinPsy programmes could achieve?Given HCPC guidance and BPS standards specify DClinPsy programmes have to

have EbE involvement, could a minimum standard of involvement be developed, guided

by the findings of this SEP and previous research, which could be achieved by all

programmes?

Findings indicate a potential minimum standard of involvement could include:

Links made with a EbE reference group or network (affiliated or external to the

programme)

An allocated involvement lead or co-ordinator for the programme (EbE or staff)

Standing invitation to programme meetings/sub-/committee meetings

Selection process – sat on interview panels/assessor on assessments days, and

involved in final cohort discussions/meetings

Clinical – feedback sought from clients by trainees on placement with

contributions to placement assessment/evaluation

Recommendation:

Programmes to consider how involvement and co-production could beintroduced more creatively and flexibly.

Programmes to further explore how to recruit individuals who appear to beunderrepresented.

Programmes to consider how to make involvement as convenient, inclusiveand accessible for EbE



Research – programmes actively encourage consultation/development

opportunities for EbE, trainees, and supervisors/tutors to discuss projects

Teaching – combining experiential knowledge with direct contributions to

teaching sessions

Academic – programmes actively consult with EbE around academic

developments, such as changes made to teaching modules, and informal

assessments, such as clinical skill observations and presentations.

The DCP (2018) EbE position statement and strategy includes a summary of ‘key

principles for practical involvement’ to support involvement.

A flexible model of involvement for DClinPsy programmesIf a minimum standard of involvement for all DClinPsy programmes was

established, programmes could begin to work towards a flexible model of involvement.

Arnstein’s (1969) ladder of participation is a good model to measure and evaluate

involvement, however as its approach to involvement assumes all activity should be at top

of ladder it is not very flexible for programmes given the systems they work within and

takes away EbE choice for level of involvement they would like or feel able to commit to.

A flexible model of involvement could actively encourage meaningful involvement

as high up the ladder of participation as possible while acknowledging the barriers to

involvement experienced by EbE, programmes and systemically. The flexible model would

consider (a) where and when can involvement take place, (b) what the aim of the task is,

and (c) what level of involvement does the task or activity need to be at. In a similar way

that the BPS recognises DClinPsy programmes will meet accreditation standards in

different ways and encourages a flexible approach in the methods used to meet them (BPS,

2014), a flexible model of involvement could allow EbE and programmes to consider

where and when involvement could take place.

Recommendation:

A future SEP or research project may wish to develop the suggested minimum

standard of involvement, working with EbE reference groups and DClinPsy

programmes to co-produce a minimum achievable standard.

Recommendation:

A future SEP or research project may wish to develop a flexible model of

involvement, co-produced with EbE reference groups and DClinPsy

programmes.



Limitations and Future ResearchThere are limitations in relation to the project which should be considered when

evaluating the findings. For example, methodological constraints may have had an impacton the findings as the questionnaire (a) did not include the EbE definition used by the SEPcommissioner, and (b) was completed independently. As such, varied definitions of who isperceived as EbE (e.g. individuals affiliated with DClinPsy programmes and clients seenby trainees on placement) and different interpretations of the questions may haveinfluenced how participants responded and captured the involvement on their programme.These limitations may have been why there was not as much detail and depth to the findingsas expected.

Furthermore, the representativeness of the current sample should also beconsidered. The programmes that took part represent less than half of all DClinPsyprogrammes (45.5%) and may capture those most proactive and motivated aboutinvolvement, therefore the findings may not be representative of DClinPsy programmes asa whole.

Dissemination of FindingsThe above findings have been shared with the project’s commissioners. The findings will alsobe shared with the GTiCP Involvement sub-group email list and at the GTiCP conference.

Recommendation:

A future SEP may wish to develop the questionnaire to include an EbEdefinition and the use of examples within questions to aid clarity and mutualunderstanding when responding.



5. References

Ahuja, A. S., & Williams, R. (2005). Involving patients and their carers in educating and

training practitioners. Current Opinion in Psychiatry, 18, 374-380.

Arnstein, S. R. (1969). 'A Ladder Of Citizen Participation'. Journal of the American

Planning Association, 35: 4, 216-224.

Berry, C. (2016). Service Evaluation Project: Evaluating Service User and Carer

Experience of Involvement in the Doctorate in Clinical Psychology Programme at

the University of Leeds. Leeds, UK: University of Leeds.

Boyle, D. & Harris, M. (2009). The Challenge of Co-production. NESTA, London.

Briggs, J., O'Key, V., Tickle, A. & Rennoldson, M. (2017). Involvement in UK DClinPsy

programmes: Attitudes and activity. Clinical Psychology Forum, 291, 25-31.

British Psychological Society. (2014). Standards for doctoral programmes in clinical

psychology. Leicester, UK: British Psychological Society.

Clarke, S. P. & Holttum, S. (2013). Staff perspectives of service user involvement on twoclinical psychology training courses. Psychology Learning & Teaching, 12(1), 32-43. doi:10.2304/plat.2013.12.1.32

Department of Health. (2011). No health without mental health; A cross- governmentmental health outcomes strategy for people of all ages. London, UK: Departmentof Health.

Department of Health. (2014). Five Year Forward View. London, UK: Department of

Health.

Division of Clinical Psychology. (2018). Experts by Experience Position Statement and

Strategy 2018-2019. Leicester, UK: The British Psychological Society.

Division of Clinical Psychology. (2015). Inclusivity Strategy 2016-2018. Leicester, UK:

The British Psychological Society.

Dunning, G. (2015). Service evaluation project: Evaluating Service User and CarerInvolvement in the Doctorate of Clinical Psychology Programme at the Universityof Leeds. Leeds, UK: University of Leeds.



Eames, C. & Phillips, P. (2017). Involvement Survey. GTiCP Conference Presentation.

Elliott, R., Fischer, C. T., & Rennie, D. L. (1999). Evolving guidelines for the publicationof qualitative research studies in psychology and related fields. British Journal ofClinical Psychology, 38, 215-229.

Eriksson, E. (2018). Incorporation and Individualization of Collective Voices: PublicService User Involvement and the User Movement’s Mobilization for Change.Voluntas, 29, 832–843.

Group of Trainers in Clinical Psychology (GTiCP) Academic sub-group. (2015).Involvement Survey. British Psychological Society, UK.

Harper, D., Goodbody, L., & Steen, L. (2003). Involving users of services in clinicalpsychology training. Clinical Psychology Forum, 21, 14-19.

Health and Care Professions Council. (2017). Your duties as education provider: Standardsof education and training guidance. London, UK: Health and Care ProfessionsCouncil.

Holttum, S., Lea, L., Morris, D., Riley, L., & Byrne, D. (2011). Now I have a voice: serviceuser and carer involvement in clinical psychology training. Mental Health andSocial Inclusion, 15(4), 190-197.

Horgan, A., Manning , F., Bocking, J., Happell, B., Lahti, M., Doody, R., Griffin,M., Bradley, S., Russell, S., Bjornsson, E., O'Donovan, M., MacGabhann, L.,Savage, E., Pulli, J., Goodwin, J., Jan van der Vaart, K., O'Sullivan, H., Dorrity, C.,Ellila, H., Allon, J., Hals, E., Sitvast, J., Granerud, A. & Biering, P. (2018). ‘To betreated as a human': Using co‐production to explore experts by experience involvement in mental health nursing education ‐ The COMMUNE project. International Journal of Mental Health Nursing, 27, 1282–1291.

Involve (2012). INVOLVE Strategy 2012–2015. [online] Accessed on 08.11.18. Retrievedfrom http://www.invo.org.uk/about-involve/involve-organisation-publications/

Khoo, R., McVicar, A., & Brandon, D. (2004). Service user involvement in postgraduatemental health education. Does it benefit practice? Journal of Mental Health, 13,481-492.

Lea, L., Holttum, S., Cooke, A., & Riley, L. (2016). Aims for service user involvement inmental health training: Staying human. The Journal of Mental Health Training,11(4), 208-219. doi:http://dx.doi.org/10.1108/JMHTEP-01-2016-0008

McCusker, P., MacIntyre, G., Stewart, A., & Jackson, J. (2012). Evaluating theeffectiveness of service user and carer involvement in post qualifying mental health



social work education in Scotland: Challenges and opportunities. The Journal ofMental Health Training, Education and Practice, 7(3), 143-153.

NHS England. (2006). Liaison and Diversion Manager and Practitioner Resources:

Service User Involvement. https://www.england.nhs.uk/commissioning/wp-

content/uploads/sites/12/2015/10/ohc-paper-06.pdf

NSUN. (2018). The National Survivor User Network. https://www.nsun.org.uk/

Paul, C. & Holt, J. (2017). Involving the public in mental health and learning disability

research: Can we, should we, do we? Journal of Psychiatric and Mental Health

Nursing, 24, 570–579.

Pollard, A. (2018). Experts by experience are invaluable in mental health: but how exactly?

Mental Health Foundation. https://www.mentalhealth.org.uk/blog/experts-

experience-are-invaluable-mental-health-how-exactly-0

Repper, J., & Breeze, J. (2007). User and carer involvement in the training and educationof health professionals: A review of the literature. International Journal of NursingStudies, 44(3), 511-519. doi:http://dx.doi.org/10.1016/j.ijnurstu.2006.05.013

Schreier, M. (2014) Qualitative Content Analysis. In U. Flick (Eds.), The SAGE handbookof qualitative data analysis (pp. 170-183). Los Angeles: SAGE.

Sheldon, K., & Harding, E. (2010). Good practice guidelines to support the involvement ofservice users and carers in clinical psychology services. Leicester, UK: Division ofClinical Psychology.

Simpson, E. L., & House, A. O. (2003). User and carer involvement in mental healthservices: From rhetoric to science. British Journal of Psychiatry, 183, 89-91.

Together & NSUN. Service User Involvement in the delivery of mental health services.May 2014. http://www.together-uk.org/wp-content/uploads/downloads/2014/06/Service-User-Involvement-briefing.pdf



6. Appendices

Appendix A: Questionnaire to DClinPsy Programmes

Appendix B: Participant Information Sheet (PIS)

Appendix C: Covering email to DClinPsy Programmes

Appendix D: Reminder email to DClinPsy Programmes



Appendix A: Questionnaire to DClinPsy Programmes

Evaluating the involvement of Experts by Experience in the

Doctorate of Clinical Psychology Programmes

Please indicate the programme you represent:

…………………………………………

The following completed the questionnaire:

Person 1: Programme staff Expert by Experience Identify as both

Person 2: Programme staff Expert by Experience Identify as both

Part 1: Overall programmeQuestions Yes / No If yes, please describe what and

how involvement happens.Q1. Does your programmeconsult with an Experts byExperience referencegroup?

Q2. Does your programmehave a member of staffwho leads/co-ordinatesExperts by Experienceinvolvement?Q3. Do Experts byExperience attend yourprogramme sub-committees?

SelectionClinicalResearchEthicsAcademicOther(pleasestate)

Part 2: Selection processQuestions Yes /

NoIf yes, please describe what and how

involvement happens.

Q4. Does your programmehave Experts by Experienceinvolved in shortlisting?



Q5. Does your programmehave Experts by Experienceinvolved in your selectionassessment/interview day?

Q6. Does your programmehave Experts by Experienceinvolved in the final cohortdecisions? E.g. meetings

Part 3: Clinical strandQuestions Yes /


involvement happens.Q7. Does your programmehave Experts by Experienceinvolved in providing traineeperformance feedback orevaluation on placement?

Part 4: Research strandQuestions Yes /


involvement happens.

PanelsQ8. Does your programmehave Experts by Experienceinvolved in providingresearch presentationfeedback?ProjectsQ9. Does your programmehave Experts by Experienceconsultation available totrainees regarding researchprojects?Q10. Does your programmehave Experts by Experienceinvolved in research projectdevelopment and/orfeedback? E.g. meetings,panelsSupervision



Q11. Does your programmehave or offer Experts byExperience in research co-supervisory roles?

Part 5: Academic strandQuestions Yes / No /

Don’t knowIf yes, please describe what and

how involvement happens.TeachingQ12. Does yourprogramme have Expertsby Experience involved inteaching sessions aroundsharing/discussingexperience andperspectives of, forexample: psychologicalproblems, mental healthdiagnosis, physical healthdiagnosis, services,therapy?

Adult

Child

Older adult

Learningdisability

Forensic

Psychosis

Health

Neurological

Other(pleasestate)

Q13. Does yourprogramme have Expertsby Experience in co-/running or contributing toteaching sessions? E.g.research, clinical skillsbuilding, reflective personaland professionaldevelopmentQ14. Does yourprogramme have Expertsby Experience involved indeveloping teachingresources available totrainees? E.g. DVDs,online videos



Development

Q15. Does yourprogramme have Expertsby Experience involvementin programme guidelines?E.g. involvement of Expertsby Experience in teachingQ16. Does yourprogramme of Experts byExperience involved informal assessment design?E.g. essay titles, examquestionsQ17. Does yourprogramme have Expertsby Experience reviewingand/or advising onacademic developments?E.g. change in teaching,new assessmentsAssessmentQ18. Does yourprogramme have Expertsby Experience involvementin trainee formalassessment and feedback?E.g. exams, essaysQ19. Does yourprogramme have Expertsby Experience in providinginformal assessment andfeedback? E.g. clinicalskills observations,presentations

Part 6. OtherPlease detail any other Experts by Experience involvement in your programme that isnot captured in the questions.E.g. training days, conferences, events, interview panels for new course staff, programmeEbE co-ordinator post



Are there reasons why Experts by Experience involvement is difficult or not aswidespread you would like?

Thank you for your time.



Appendix B: Participant Information Sheet (PIS)

Evaluating the involvement of Experts by Experience in the Doctorate

of Clinical Psychology Programmes: Participation Information Sheet

We are inviting you to take part in a survey of Doctorate of Clinical Psychology

programmes in order to evaluate the current Experts by Experience (EbE)

involvement which will hopefully contribute towards a flexible model of

involvement in Clinical Psychology programmes in the UK. We are hoping to collect

information regarding EbE involvement across the different stands of training (i.e.

academic, clinical, research, and selection) from those involved in co-ordinating

involvement work on your programme. The ethical approval for this service

evaluation project has been sought from the School of Medicine Research Ethics

Committee DClin Psych sub-REC (DClinREC project number 17-010).

What will I be asked to do?

Complete one questionnaire per programme, ideally completed jointly by a

member of academic staff and EbE involved in co-ordinating involvement.

The questionnaire looks at EbE involvement across four strands of the

Doctorate of Clinical Psychology programme: academic, clinical, research,

and selection.

We would be grateful if you could include as much detailed information as

possible.

Returning Questionnaires

By returning the questionnaire you will be consenting to take part in the

above project.

Questionnaires can be returned by emailing them to

[email protected]. If you would prefer a paper version of the

questionnaire please request one from [email protected].

Please return any questionnaires as soon as possible and by 21st June

2018 at the latest.

Do I have to take part?

No. It is your choice as to whether you take part and you do not have to

do so.

Please do not hesitate to contact me using the details below if you have

any questions before taking part.



What will happen with the information?

Returned questionnaires will stored securely on University of Leeds

servers.

The researcher and commissioner of the project (see below) will have

access to the questionnaires to check the quality of information gathered.

A summary of information gathered which will be included in a written

report and the findings from the project will be disseminated (e.g. GTiCP

conferences, poster presentations, and academic papers).

No individual programme will be readily identifiable in any write up or

dissemination of the findings.

Questionnaires will be archived for a period of 3 years following the

completion of the project.

What are the benefits of taking part?

We hope that the valuable findings of this survey will contribute towards a

flexible model of involvement in clinical psychology programmes in the UK.

However, the data you provide will be useful in helping us identify the

current level of EbE involvement across Doctorate of Clinical Psychology

programmes.

What risks are involved in taking part?

It is anticipated that there are no risks from taking part.

However, taking part will require a brief time commitment on your part.

If anything difficult does come up for you while completing the

questionnaire, you can contact me using the details below.

Can I withdraw from the project?

Withdrawal from the project is possible within the first week of returning

the questionnaire, and the questionnaire will be deleted. Withdrawal will

not be possible after this point.

Comments and concerns

If you have any comments or concerns about this project you can contact

the commissioner in the first instance using the details below.

Thank you for taking the time to read this.

Researcher: 201084884 (Psychologist in Clinical Training, [email protected])



The University of Leeds

Leeds Institute of Health Sciences

Doctorate in Clinical Psychology

Level 10 Worsley Building

Clarendon Way

Leeds

LS2 9NL

Commissioner: Dr Tracey Smith (Clinical Tutor, [email protected])

The University of Leeds

Leeds Institute of Health Sciences

Doctorate in Clinical Psychology

Level 10 Worsley Building

Clarendon Way

Leeds

LS2 9NL



Appendix C: Covering email to DClinPsy Programmes


of Clinical Psychology Programmes:

Initial Email to Doctorate of Clinical Psychology Programmes

Dear colleagues,

The Involvement of Experts by Experience (EbE) in Clinical Psychology Training is

widely thought to have intrinsic value and acknowledges the importance of EbE

lived experiences and expertise. We would like to capture the current EbE

involvement in the Doctorate of Clinical Psychology programmes. The project is

commissioned by Tracey Smith, Co-Chair of the Group of Trainers in Clinical

Psychology (GTiCP) Involvement Sub-group with Simon Mudie. The Involvement

Sub-group feel this is an important piece of work which we hope will contribute

towards a flexible model of involvement in Clinical Psychology programmes in the

UK. A previous brief survey carried out by the GTiCP Academic sub-group in 2015

collected important involvement data. This survey hopes to update and add further

detail and depth to what specific involvement is happening now and how

programmes are co-ordinating this.

If this sounds like something that your programme would like to be involved in,

please read through the participant information sheet attached. If your programme

would like to take part in this project, we would be grateful if those involved in co-

ordinating involvement, ideally both academic staff and EbE, would please jointly

complete the attached questionnaire for your programme. Please return the

questionnaire to [email protected] as soon as possible and by 18th June

2018 at the latest.

The ethical approval for this service evaluation project has been sought from the

School of Medicine Research Ethics Committee DClin Psych sub-REC (DClinREC

project number 17-010).


Emma HowarthPsychologist in Clinical TrainingLeeds Teaching Hospitals NHS Trust and University of Leeds



Appendix D: Reminder email to DClinPsy Programmes


of Clinical Psychology Programmes:

Reminder Email to Doctorate of Clinical Psychology Programmes

Dear colleagues,

We previously emailed you about a survey of Doctorate of Clinical Psychology

programmes regarding current Expert by Experience (EbE) involvement. The

project is commissioned by Tracey Smith, Co-Chair of the Group of Trainers in

Clinical Psychology (GTiCP) Involvement Sub-group with Simon Mudie.

If your programme would like to be involved in the survey, please read through

the participant information sheet attached. If your programme would like to take

part in this project, we would be grateful if those involved in co-ordinating

involvement, ideally both academic staff and EbE, would please jointly complete

the attached questionnaire for your programme. Please return the questionnaire

to [email protected] as soon as possible and by 21st June 2018 at the

latest.

The ethical approval for this service evaluation project has been sought from the

School of Medicine Research Ethics Committee DClin Psych sub-REC (DClinREC

project number 17-010).


Emma HowarthPsychologist in Clinical TrainingLeeds Teaching Hospitals NHS Trust and University of Leeds

evaluating the involvement of experts by experience in the

Documents