Evaluating a rating scale for assessing trainee clinical psychologists’ clinical skills in-vivo.Dr. Alison Tweed* and Ms. Rebecca Graber

Introduction and Objectives

Developing clinical skills is seen as a core competency within

clinical psychology training. Yet, there are no published, formalised

or standard means of assessing trainees’ acquisition of clinical skill

in-vivo, despite a call for this from the training community (Gallichan

& Mitchell, 2008).

This poster reports on reliability and validity evaluation research of

the Clinical Skills Assessment Rating Form (CSA-RF), a seven-

domain rating scale developed at the University of Leicester to

assess trainees’ clinical skills in-vivo.

Design and methodology

The evaluation incorporated a mixed-methods design, undertaken in

two phases: Phase 1 (Quantitative) used correlations and factor

analysis to establish validity and reliability estimates and determine

the factor structure of the scale. Phase 2 (Qualitative) investigated

how the scale was used by paired raters as intended for Programme

assessments. Tape-recorded rating sessions were analysed using

Thematic Analysis (Braun & Clarke, 2006).

Results

Phase 1 (Quantitative)

Statistical analysis revealed acceptable levels of internal

consistency (construct validity) of the scale. Chronbach’s alpha =

0.78. Inter-rater reliability analysis is still ongoing.

Factor analysis revealed 5 factors accounting for 50.7% of the

variance: Demonstrating Professional Therapeutic Engagement,

Creating a Secure Base, Formulation, Facilitating Mutual

Understanding and Session Structure.

Phase 2 (Qualitative)

Thematic analysis is still underway. A preliminary model has been

developed (see below). Core themes relate to the negotiation of

raters’ expectations according to principles or guidelines of “good

enough” performance. Negotiation between raters is undertaken

within personal and inter-personal frameworks, using what

happens on the video as a starting point for discussion.

Discussion and Conclusions

The CSA-RF appears to be a valid in-vivo measure of

the generic clinical skills of trainee clinical

psychologists. Whilst the research has been

undertaken using simulated patients, it is expected

that the findings can be generalised to genuine clinical

populations. Findings from Phase 2 will inform the

development of training materials for new raters. The

study has gone some way to answering a need for a

robust measure of clinical skill that can be applied to

diverse therapeutic orientations. The CSA-RF has

applicability for Clinical Psychology Programme staff

on a national level who are interested in providing

more formalised means of assessing trainees’ generic

clinical skills .

Participants

Eighteen qualified clinical psychologists took part in Phase 1, each

rating eight different 15-minute interactions of trainees undertaking a

clinical session with a simulated patient on video. In Phase 2, ten

clinical psychologists (five pairs) rated one 30-minute interaction,

first individually and then jointly to reach a consensus.

Expectations NEGOTIATIONOutcome:

Is the trainee good enough?

Personal

Inter-personal

What happens

References and Contact InformationBraun, V. & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3 (2), 77-101.Gallichan, D. & Mitchell, A. (2008). Making the case for in-vivo assessment s of clinical competence. Clinical Psychology Forum, 184, 18-21.

*University of Leicester, School of Psychology, Clinical Section, 104 Regent Road, Leicester, LE1 7LT. E-mail: aet2@le.ac.uk