www.monash.edu.au

Prepared by:

Melissa Noonan- PsychologistDepartment of General Practice

Using outcome directed rating scales as a demand management tool for counselling services at Inner South Community Health Service (ISCHS)

www.monash.edu.au

2

DHS directions for Counselling

• Accountability and Effectiveness

- Demand management

- Evidence-based Practice

- “Review of Counselling Services in Community Health Discussion Paper”

- “Counselling in Community Health: Future Directions and Guidelines for Quality Counselling”

www.monash.edu.au

3

Outcome informs practice … client by client• Rate of change is monitored and discussed

in sessions• Frequency of sessions is informed by rate of

change• Rating of therapeutic alliance is monitored and

discussed in sessions• Clients for whom outcomes are static or

worsening: – May be offered other options - NOT just ‘more of

the same’! – Are identified and reviewed

www.monash.edu.au

4

The Primacy of the Client in Achieving Positive Outcomes

• The client’s subjective experience of change in the early stages of treatment the is one of the best predictors of positive results (Brown, Dreis, & Nace, 1999; Garfield, 1994; Howard, Kopte, Krause, and Orlinsky, 1986; Howard, Moras, Brill, Martinovich, and Lutz, 1996, Lebow, 1997; Smith, Glass, and Miller, 1980; Steenbarger, 1992, 1994; Talmon, 1990)

• The client’s rating of the alliance is the best predictor of engagement and outcome (Duncan, Sparks, & Miller, 2000)

• Treatment models – including psychodynamic, client centred, alcohol & drug, marriage and family therapy, psychopharmacologic, to psychological interventions have been shown to be equivalent in effectiveness when compared to each other (Duncan, Sparks, & Miller, 2000)

www.monash.edu.au

5

Beginnings of the Project

• A working group of interested counsellors at ISCHS started using the outcome tools with their new clients from late September 2004.

• The Primary Health Care Research, Evaluation & Development (PHC RED) Fellowship was successfully applied for and commenced in June 2005

www.monash.edu.au

6

The Tools:Client outcome and process feedback

• Outcome Rating Scale (outcome)- ORS– Assesses individual, relational and social

functioning

– Takes less than a minute to complete and score

www.monash.edu.au

7

Outcome Rating Scale (ORS)

Name: UR#: Session # : Date

Looking back over the last week, including today, help us understand how you have been feeling by rating how well you have been doing in the following areas of your life, where marks to the left represent low levels and marks to the right indicate high levels.

Individually: (Personal well-being)

I------------------------------------------------------------------------I

Interpersonally:

(Family, close relationships)

I-------------------------------------------------------------------------I

Socially: (Work, School, Friendships)

I-------------------------------------------------------------------------I

Overall:

(General sense of well-being)

I------------------------------------------------------------------------I

Institute for the Study of Therapeutic Change _______________________________________

www.talkingcure.com

© 2000, Scott D. Miller and Barry L. Duncan

www.monash.edu.au

8

The Tools:Client outcome and process feedback

• Session Rating Scale (process) SRS

– Assesses the four components of the therapeutic alliance

– Takes less than a minute to complete and score

www.monash.edu.au

9

Session Rating Scale (SRS V.3.0)

Name ________________________Age (Yrs):____ ID# _________________________ Sex: M / F Session # ____ Date: ________________________

Please rate today’s session by placing a hash mark on the line nearest to the description that best fits your experience.

Relationship

I---------------------------------------------------------------------- I

Goals and Topics

I---------------------------------------------------------------------- I

Approach or Method

I----------------------------------------------------------------------I

Overall

I---------------------------------------------------------------------- I

Institute for the Study of Therapeutic Change _______________________________________

www.talkingcure.com

© 2000, Lynn D. Johnson, Scott D. Miller and Barry L. Duncan

I felt heard, understood, and

respected

I did not feel heard, understood, and

respected

We worked on and talked about what I

wanted to work on and talk about

We did not work on or talk about what I

wanted to work on and talk about

Overall, today’s session was right for

me

There was something missing in the session

today

The therapist’s approach is a good fit

for me.

The therapist’s approach is a not a

good fit for me.

www.monash.edu.au

10

Results Graph

Name: UR#: Date commenced:

40

35

30

25

20

15

10

5

0

Session Number

1 2 3 4 5 6 7 8 9 10

ORS – blue/black line SRS – red line

SRS Cutoff

Discuss

ORS Cutoff

www.monash.edu.au

11

Sample of Results Chart for ORS scores

Sample Client

10121416182022242628303234

1 2 3 4 5 6

No of Sessions

OR

S S

co

re

www.monash.edu.au

12

Interface between to ODW tools and Single Session Therapy (SST

• Single Session Therapy • 30-50% of clients across all treatment models only attend one

session• Assumptions as to why:

– Not pleased with therapy– Resistant

• But when followed up:– Most didn’t return because they had got what they wanted

• SST seeks to maximise the effect of the first session… ‘as if’ this is the only session

www.monash.edu.au

13

The Aims of the Research

To explore the use of ORS and SRS as demand management tools for ISCHS

• Identify the extent to which clients are achieving positive change in their functioning

• Identify the extent to which a positive therapeutic relationship is evident in the sample

• Assess the effectiveness of SST as a demand management tool at ISCHS

www.monash.edu.au

14

Method

• Participants: Recipients of counselling at ISCHS• Sample Size: 5 Counsellors. 48 Clients• Procedure: Retrospective audit of client files for

whom counsellors used outcome tools. Excel spreadsheet designed for this process.

• Measures: The ORS and SRS and additional variables collated from client files

• Data analysis: Data analysed using SPSS package

www.monash.edu.au

15

Client Demographic Data

• 34 female 14 male • Age range 17- 74 years. Mean age 41yrs• Site Consulted - Southport: 35, Prahran

12, St Kilda 1.• 31 had completed counselling• 17 still in ongoing counselling at time of

data collection

www.monash.edu.au

16

Results of study

• Positive change in functioning was measured by a 5 point or more increase in the ORS score from the original score taken to the final score at the end of counselling.

www.monash.edu.au

17

Results: Identify the extent to which clients are achieving positive change

Change for cases who have terminated counselling

0

1

2

3

4

5

6

15+ 10-14 5-10 4-0 0 to -5 -6 to -10 -11 to -14 -15 on

ORS Score Range

No

of c

ases

Achieved Positive Change

Did Not Achieve Positive Change

www.monash.edu.au

18

Results: Identify the extent to which clients are achieving positive change

Change for cases still in ongoing counselling

0

1

2

3

4

5

6

15+ 10-14 5-10 4-0 0 to -5 -6 to -10 -11 to -14 -15 on

ORS Score Range

No

of C

ases

www.monash.edu.au

19

Results: The extent to which a therapeutic relationship is evident

Therapeutic Relaltionship by SRS Score

0

5

10

15

20

25

30

35

Therapeutic Relationship

No

of

case

s

35 plus

Under 35

SRS Score

www.monash.edu.au

20

Results: Analysis of change achieved in Single Session Therapy

Change in ORS scores for SST clients

0

1

2

3

4

5

6

7

15+ 10-14 5-10 4-0 0 to -5 -6 to -10 -11 to -14

ORS Score Range

No

of

cases

www.monash.edu.au

21

Where does SST fit?

• Early results suggest that SST may be considered a useful demand management tool as it demonstrates that positive therapeutic change is occurring very early in the counselling process and delivering a successful result within a very short service period. However to evaluate this in a more rigorous way further research would need to be undertaken.

www.monash.edu.au

22

Discussion of Results: Limitations of study

• Sample size too small to be representative counselling clients at ISCHS

• Most of the data was gathered by one counsellor at one site

• Only clients who had completed counselling had scores that could be compared from start to finish

www.monash.edu.au

23

Discussion of Results: Interface between SST and ODW

• Frequently no ORS/SRS administered at SST or FUPC and therefore vital missing data in relation to change.

• Insufficient support, training, and encouragement for staff to use the measures consistently

• Lack of adequate information and supervision in relation to scoring and interpretation of the scales

www.monash.edu.au

24

What have we learned about the use of the ORS and SRS as demand management tools?

• ODW offers ways of measuring outcomes and monitoring and reviewing the therapy and the therapeutic relationship.

• Important for ensuring quality counselling using practice-based evidence

• Effective in terms of demand management by using client feedback to make timely and prudent decisions about the provision of treatment and when termination of counselling is indicated.

www.monash.edu.au

25

Considerations

• There may be individuals or client groups for whom it is inappropriate to utilise these tools.

• Awareness of the risk that the tools may be unhelpful in some contexts.

• Alternative ways of tracking clients functioning where the tools are not utilised need to be examined.

www.monash.edu.au

26

Recommendations: Future Research

• Future research is indicated to be able to more fully evaluate these outcome tools, however preliminary findings are positive in demonstrating their efficacy as part of a demand management strategy.

• If such a project were undertaken adequate time allocation and funding for the research is vital to it’s success.

www.monash.edu.au

27

Practice Recommendations:

• Sufficient support, training, and encouragement for staff to use the measures consistently is vital to the effective implementation of these tools at any community based setting

• Provision of adequate information and supervision in relation to scoring and interpretation of the scales to staff

• Streamlining processes with the SST model and ODW to reduce complexity and increase it’s practical application

www.monash.edu.au

28

Reflections on the Fellowship: Benefits

• Extremely beneficial as a clinician to undertake a research project relevant to my work context.

• Time from usual work allocation to undertake the project

• Expediential increase in knowledge relating to the theory & use of the measures.

• Funding for 4 days of training with Scott Miller when he came to Australia

• Developing research skills and experience

www.monash.edu.au

29

Reflections on the Fellowship:Challenges

• One day a week for 10 months (40 days) was a very ambitious exercise to undertake this type of project

• Difficult to absorb and process all the aspects involved in undertaking academic level research in a community context

• Up-skilling using IT: Online Databases, Excel, SPSS, PowerPoint, Endnote.

• Supervision – inadequate time allocation

www.monash.edu.au

30

Acknowledgments

• The Department of Human Services for project funding

• Monash University Department of General Practice• My Supervisor- Jo Mitchell• Pauline Middleton – Hargrave Andrew Library• Inner South Community Health• The clients• Scott Miller and his colleagues at The Institute of

Therapeutic Change – www.talkingcure.com