assessments&cpd ot learn2016 shelley hughes · 2016-09-23 · occupational therapist 22 sept...

9/23/2016

1

Learning to Administer

and Interpret Assessments

to Support your

Continuing Professional

Development (CPD)

Shelley HughesOccupational Therapist

22 Sept 20161CPD and Standardised Assessments

Shelley Hughes

Occupational TherapistSenior Product ManagerPearson Clinical Assessment

Twitter: #OTLearn

CPD and Standardised Assessments | 2

9/23/2016

2

By the end of this session, attendees will be able to…

… explain the CPD requirements for Occupational Therapists in the UK

… describe the steps required to begin learning to use a standardised assessment

… share ideas for working with colleagues to learn to use standardised assessments as a CPD activity

… explain some of the concepts of standardised assessments to evidence this as a CPD activity

3CPD and Standardised Assessments

Agenda

I CPD requirements UK Occupational Therapists

II The assessment process

III The essentials of getting started

IV Learning with colleagues

V Graduating from basic to advanced administration and interpretation


9/23/2016

3

CPD Requirements

CPD can be defined as:“a range of learning activities

through which health and care professionals maintain and

develop throughout their career to ensure that they retain their capacity to practice safely,

effectively and legally within their evolving scope of practice”

Health and Care Professions Council (HCPC)


9/23/2016

4

CPD Requirements in the UK

1. Standards of proficiency‒ Meeting the standards‒ Scope of practice

2. CPD and Audit Process ‒ Need to document CPD‒ Demonstrate a variety of learning activities‒ Seek to ensure that CPD contributes to quality of practice‒ Must benefit the service user[s]‒ May be audited

3. Examples of CPD activities‒ Work based learning‒ Professional activities‒ Formal education‒ Self-directed learning‒ Other


COT Guidelines

1.College of Occupational Therapists Code of Continuing Professional Development – 2014

2. Aligned with and complimentary to the HCPC guidelines (does not replace these)

3.Underpinning concept = critical reflection


9/23/2016

5

The Assessment Process


1.Hypothesis testing‒ A problem solving process‒ Identifying strengths

2.Types of Assessment‒ Screeners‒ Non-standardised‒ Criterion-referenced‒ Standardised‒ Full battery‒ More specific

‒ Predictors for problems in everyday life


9/23/2016

6

background informationdevelop hypothesis

screening assessment

revise hypothesis

full battery assessment

specific assessment

prove / disprove hypothesis

Intervention planning / outcome measure


CPD and Standardised Assessments

Standardised Assessments: Key Features

1. Allows you to compare your patients score to a criterion

‒ the patient’s score is compared to the norms - the scores of the reference group

2. Is always administered and scored in exactly the same way

3. Comparisons made using standardised scores

‒ Raw Scores (non-standardised)‒ Percentile Ranks‒ Standard (or Scaled) Scores

4. The aim is to decide whether:

‒ patient’s performance is evidently different to most or all of the typical sample

‒ the patients performance is significantly different to what you would expect in view of age, ability etc


9/23/2016

7

The Essentials of Getting Started

Where to Start…

1. Understand the population for which the assessment is intended

2.What service needs will the assessment meet

3.How was the assessment developed

4.How are assessment findings reported

5. Practice administration


9/23/2016

8

How was the Assessment Developed…

1. Theoretical Underpinnings

2. Data Collection


Evidence of Consistency & Usefulness…Reliability

Is the test a reliable and consistent measure?

Inter-rater reliability‒ are the scores likely to be the same no matter who is administering

the test?

Test-retest reliability‒ are the scores likely to be the same if the test is given on two

occasions separated by a short interval?

Parallel form reliability‒ where the test has parallel forms, are these equivalent in difficulty?


9/23/2016

9

Evidence of Consistency & Usefulness…Validity

Does the test measure what it claims to measure?

Construct validity‒ Is the test measuring the theoretical concept it says it is measuring?

Content validity‒ Does the test assess a reasonable range of behaviour related to the concept

being measured?

Criterion validity‒ Are the test results consistent with results from other measures of the

concept being measured?

Face validity‒ Does it appear as if it is measuring what it is supposed to be measuring?

Ecological validity‒ Do test results predict behaviour in everyday situations?


How we Measure Reliability and Validity: Correlation Coefficients

• Measure of the degree of relationship between 2 variables e.g. age and scores on a test

• Range +/- 1 to 0

• +1 and –1 are perfect correlations in different directions

• 0 = no relationship between the variables

• Correlation of .6 = acceptable, .8 or higher = very good


9/23/2016

10

How are assessment findings reported

‒ The patient’s score is compared to the norms - the scores of the reference group

The aim is to decide whether:‒ the patient’s performance is evidently impaired i.e.

worse than most or all of the typical sample

‒ the patients performance is significantly different to what you would expect in view of age, ability etc


Normal Distribution:

‒ Distribution of test scores for large groups of heterogeneous test takers

‒ 68.26% of population should fall within 1 Standard Deviation (S.D.) of the mean‒ 2 S.D.’s = 95.44%‒ 3 S.D.’s = 99.73%

‒ Few extremes at both ends

‒ Scores must be normally distributed for these to be useful


9/23/2016

11

Normal Distribution:


Interpretation of Scores

‒ A score on a test does not prove or disprove anything

‒ It just gives an estimate of performance in a particular area of behaviour

‒ Must be interpreted in the wider context of results from a range of tests

‒ Take into account previous abilities and achievements


9/23/2016

12

PracticeAdministration


The Learning Process Questions to ask yourself

1.Understand the basics

2.How would the assessment support your

decision making and practice

3.What is the evidence base for use of the

assessment


9/23/2016

13

The Learning ProcessDocumenting

1. Reflection‒ How important is this activity to

your professional practice?‒ How will this influence your

practice?‒ Your next steps for learning‒ Review your learning after several

months

2. Sharing with others‒ Teaching students‒ Share a case study with

colleagues‒ Share with other professions


Learning with Colleagues

9/23/2016

14

Teamwork

1. Practice administration

2. Inter-disciplinary

3. Application to different clinical populations

4. Establish a community of practice


Graduating from Basic to Advanced

Administration and Interpretation

9/23/2016

15

The Next Steps

1. Plan for your next steps

2. Review the research

3. Become more advanced in administration and interpretation


Advanced Administration

1. Understanding the assessment in context

2. Interpreting the assessment in detail


9/23/2016

16

Confidence Bands

Reflect the uncertainty in scores arising from measurement error i.e. –you would not expect the same score on every single occasion.


Confidence Bands

1. Reported as 68% or 95% confidence band

2. 95% confidence band = you are 95% confident that the true score will lie within this band

3. Based on the reliability measurements of the test

4. The higher the confidence interval is the wider the band will be

5. Which band is better for your situation?


9/23/2016

17

Confidence Bands

Overlapping Confidence Bands

86 (Standard Score)

95 (Standard Score)

33

79 Verbal Memory 93

91 Visual Memory 99

CPD and Standardised Assessments

Composite vs Subtest Level Interpretation

1. Composite more robust

2. Some subtests are sufficiently robust to use in isolation

3. Comparing subtest level data helps build understanding about an individual’s profile

4. Statistical difference vs clinical importance


9/23/2016

18

Pairwise comparisonsBOT-2 Example


Pairwise comparisons


9/23/2016

19

Consolidating and Documenting your Learning

1. Case studies‒ Work through examples in the

Manual‒ Listen to webinars

2. Talk through with colleagues‒ Explain your findings‒ Check your understanding

3. Reflect on whether a more in-depth interpretation changes your practice‒ Certainty behind recommendations‒ Communication with colleagues‒ More targeted intervention planning


Now have a goDocument what you have learned from

this session


9/23/2016

20

There’s so much more to learn

Find out more about us at

Pearsonclinical.co.uk

assessments&cpd ot learn2016 shelley hughes · 2016-09-23 · occupational therapist 22 sept...

Documents