Assessing clinical decision making skills using an

interactive online tool.

Verina Waights & Ali Wyllie Faculty of Health and Social Care COLMSCT CETL

Fellows

OU Conference ‘Making Connections’ (30th April 2008) Quality eLearning strand

2

Outline

Context of Project

Overview principles of quality eLearning experience

Outline characteristics of CDM tool

Demo CDM

Findings to date

Looking to the future

3

Background

Nurses make clinical decisions in response to changes in each patient's condition, which may occur within very small time-frames

The ability to make clinical judgements depends on both a sound theoretical background and good decision-making skills

Clinical decisions are strongly influenced by the context in which they are made (Bucknall 2000)

As the complexity of decision tasks increases, so context-specific knowledge determines the effectiveness of decision-making (Botti & Reeve 2003)

4

Aims

• to develop a web-based tool to provide decision-making opportunities, building on Laurillard (2002): who suggests that:

‘Traditional modes of assessment of knowledge are seen as inadequate because they fail to assess students‘ capability in the authentic activities of their discipline’.

• to assess nursing students’ decision-making skills in a context-driven virtual environment.

5

What makes quality eLearningBoud and Prosser’s (2002) principles for designing a high

quality learning environment suggests learning activities should:

• support learner engagement - build on prior knowledge

• acknowledge the learning context – integral with study

• seek to challenge learners – active participation

• provide practice – through individual and/or group activities

6

Principles of good feedback practice

Nicol and Milligan (2007:72) seven principles of good feedback practice. Principle 6: Good feedback practice ‘provides opportunities to close the gap between current and desired performance’.

Boud’s (2000:158) arguments about closing the gap can be viewed in 2 ways:

• It is about supporting students in the act of production of a piece of work

• It is about providing opportunities to repeat the ‘task –performance – feedback cycle’.

7

Pedagogical approach Based on principles of case-based learning design and

experiential learning (Kolb 1984).

Students’ options are scaffolded

Learning is student - led and asynchronous

Feedback is contexual, individualised to student’s learning journey and timely

Intention is to assist students to practice in risk-free environment until ready to practice in work setting.

8

Research methods Students’ perceptions of their decision-making skills and

efficacy of tool will be evaluated through:

• Individual completion of an online questionnaire to enable quantitative and qualitative evaluation

• Qualitative output from video observation and ‘Think aloud’ techniques with students’ interacting in the IET Userlab

9

Case study: patient with a leg ulcer Realistic: case prepared in collaboration with a practice nurse

to mimic reality

Relevant: Wound healing and clinical decision-making are integral components of Nursing Programme of study

Ethical: consent obtained from the patient and anonymity assured

Context: Miss Alice Phelps, an independent 80 year old lady

with an ulcer on her left lower leg.

10

Case study – patient with leg ulcer

11

Narrative• The story evolves over a series of consultations depending

on the treatment the patient receives.

• At each stage students consult resources to find out about their patient’s condition then determine the best treatment from 3 available options. Their decisions result in the patient progressing through successive consultations until an end point is reached.

• They are encouraged to have a second attempt at the maze if their first attempt is unsuccessful.

• Images of the consultations and patient’s final outcome create a visual narrative within the maze

12

Practice ResourcesThe students have a range of media resources to consult:

Audios of consultations between the nurse and patient

Photos of leg ulcers

Instruction sheets on possible treatments

Patient charts and records

Patient records

Laboratory reports

Links to the BNF (British National Formulary)

13

Reflective Log

The Reflective Log captures each decision point and the student’s reasons for their choice.

At the end of each attempt the student can review their pathway and reflect on their decisions and reasoning.

The final attempt additionally provides author feedback on decision-making at each stage.

14

Example Reflective log

15

ScoringScoring at each decision point depends on the type of decision

as outlined below and whether it is the first or second attempt:

For example:

1st attempt: a 5 b 3 c 0 2nd attempt: a 3 b 2 c 0

The final score is an average of the two attempts at the maze.

16

Findings to date - score7 students completed tool/8 students partly completed tool

Average score (%)

Number of students

Score decreased over 2 attempts

Score stayed the same over 2 attempts

Score increased over 2 attempts

45 – 49 50 – 59 60 – 69 70 – 79 80 – 89 90 -99

1 1 2 2 1

X

X

X X X X X

17

Findings to date - questionnaire

Strongly agree

Agree Neutral Disagree Strongly disagree

Tool

1)Easy to use 3 2

2)Realistic 3 1 1

Resources

3)Explored all thoroughly

3 2

4)Cursory glance only

2 1 1 1

5)Variety was valuable

2 3

6)They made learning memorable

2 2 1

18

Strongly agree

Agree Neutral Disagree Strongly disagree

Reflective Log

1) Easy to use 2 3

2)Useful to reflect during tasks

2 3

3)Useful to discuss with other students

2 3

4)Useful learning tool

2 2 1

19

Strongly agree

Agree Neutral Disagree Strongly disagree

Learning experience

1) New information re decision-making

3 2

2) New information re leg ulcers

2 2 1

3) Better prepared for practice

2 2 1

4) Useful for self-assessment

4 1

5) Useful for group discussion

3 2

6) Could feed into TMA

3 2

20

Very Quite Neutral Not very Not at all

Motivating

1) Context 3 1 1

2) Log 4 1

3)Audios 2 3

4)Images 3 1 1

5) Texts 3 1 1

6) Linked questions

2 3

7) Case study 2 2 Missing data

21

Strongly agree

Agree Neutral Disagree Strongly disagree

Decision-making

1) Range of resources reflects practice

4 1

2) Useful in other practice areas

3 1 1

3) Useful to have another attempt after ‘real’ practice experience

2 2 1

22

Summary to date Students felt using tool:-

• Increased their confidence in decision-making and caring for leg ulcers – support learner engagement

• would assist in their practice – acknowledge the learning context

• Positive, motivating experience – challenge learners

• Developed their knowledge and skills - provide practice

23

Looking toward the futureThe approach piloted in this project can be applied in part or

whole in other courses, programmes or disciplines.

The tool can be combined with other VLE tools, such as online discussion forums and voting tools, to create flexible and collaborative learning and assessment sequences.

The Reflective Log can become part of a TMA.

Our findings are being used to inform future assessment strategies within HSC. We are particularly interested in: Contextualised CMA questions; Narratives; Embedded multiple media; Case-based CMAs; Decision-Making; Confidence-indicator tool.