amee2013 eportfolio workshop - ucl, brighton and sussex, bristol

Use of an electronic ePortfolio for undergraduate medical students: lessons learnt from three UK medical schools

The UMeP Collaboration

ePortfolios

“a purposeful assembly of a collection of evidence of student learning and achievement in selected domains, with an element of intellectual engagement by the student with the portfolio content and associated learning.”

Purposes of portfolios

Helen C Barrett PhD http://electronicportfolios.org/balance/

OVER TO YOUTime for some group discussions

AMEEWorkshop on undergraduate e-portfolios Prague 2013

Pascale Avery, Reeanne Jones, Kathleen Meyer, Jennifer Schamotta

Year 5 medical students

At BSMS• Introduced into first clinical year (Year 3)

– Voluntary

– Used by medical students, and Clinical Academic Tutors (CATs)

• User guides, tutorials, student advocates, junior doctor

session

• Training for Clinical Academic Tutors (30)

• Formative use– Use of self appraisal form

– Use of personal development plan

– Use of personal library

– Meetings with Clinical Academic Tutor (CAT)

– Use of the reflective/educational logs

Research• Objective:• To explore its acceptability and impact of the

e-portfolio on students at BSMS, and to identify the drivers and barriers to its use

• Methodology - Mixed Methods approach :• Paper based questionnaire• Focus groups with advocates• Thematic analysis of qualitative data

BSMS data: Year 3 (2011-13)

• Participants n: 209 • (76% response rate)

• Students using the e-portfolio (%)

60

40YesNo

• Sections used by students

BSMS data: Year 3 (2012/13)

CAT PDP Reflective logs Personal library0

102030405060708090

100

Sections of the e-portfolio

Stu

dent

use

(%

)

Thematic analysis of data

• Acceptability of use• Practical issues - “I can't remember my username or

password• Time pressures - “we already have a huge workload”• “It is difficult to find time to use it” • “It’s overshadowed by the multitude of marked

assessment we have in Year 3”• Confidentiality “I don’t know who can access this

portfolio”• Non – Compulsory “maybe a compulsory element

would encourage and get the most out of the portfolio”

Thematic analysis of data

• Role modeling important• “having junior doctors describing what they put in their

eportfolio and why it has been helpful”• “make CATs encourage us to use it”

• Understanding the purpose of a portfolio• “provide more examples of what goes on the

eportfolio” • “could be checked by CAT”• Tell us what it is exactly we need to be writing about”• “I don’t know what I should be using it for” • “people have become doctors for years without it, why

the big fuss now?”

Reflective practice

• Unsure, not confident or motivated to ‘reflect’ • examples requested• concerns of privacy or negative consequences• “I am not particularly certain… what we are supposed to

reflect on”

• The format: not at ease with written reflection• “reflection should take any form, not necessarily written”

• Students may feel reflective practice is not relevant to them

Personal Opinions

• “Perhaps a true limitation is that at our stage, we may not understand the benefit of using the eportfolio early in our careers. I believe this will become more apparent when we are Foundation doctors.”

• “Using the eportfolio may be perceived as another thing to do – this makes many students reluctant to do in their free time. To overcome this, the portfolio should be made compulsory as many believe it is useful but lack the direction/understanding to complete it on their own”

Personal Opinions

• “I can relate to the comments given about the portfolio being at the bottom of the one’s ‘to do list’, especially in 3rd year. I think the portfolio would become a higher priority for us if it was used to record completion of skills and other assessments, as well as providing us with a lasting and visual record.”

Personal Opinions

• “Medicine seems to have a confusing conflict: teaching us values of being professional, competent, accountable and self directive, However, this manifests in a culture where we feel unable to express any weakness. Yet ability to do this is vital for effective reflection and personal development (a task required of us). For portfolios to be a success and at their most useful I feel an entire cultural change would be required.”

Personal Opinions

• “In order to use the undergraduate e-portfolio as a helpful transition mechanism into foundation years, compulsory use for clinical skills assessments and meetings would be highly beneficial”

• “The ability to demonstrate written reflective practice is of growing importance to professional development. However concerns over internet privacy may prevent me recording personal thoughts – I prefer to naturally reflect on events and talk to others (without putting ‘pen to paper’)”

UMeP as the Foundation for Academic Mentoring

Megan Lloyd (Year 4)Anna Taylor (Intercalating)University of Bristol Medical School

What is an Academic Mentor?

• Practicing clinician • Concerned with the student’s academic life, not

their personal life• Available to meet twice a year, preferably on a one

to one basis

During the meetings …

• Academic progress– Exam success– Possibility of intercalation – Other queries

• E-portfolio progress – Discuss any updates – Ensure it meets the targets

• Photo• Exam upload• CAPS logbook upload

• Career thinking• Professionalism

How do we feel about our Academic Mentor as students?

• Personal• Contact links • Knowledge for the future • Confidence booster

Our experiences

Thank you

CAPS LogbookDuncan Brown (Year 4)

Tom Lloyd (Year 5)

University of Bristol Medical School

What is CAPS?

• Consultation And Procedural Skills• Introduced to Bristol in 2012• Written record & proof of competence in procedural

skills; GMC requirement by the time of graduation• 32 procedural skills grouped in the following

categories:1. Diagnostic

2. Therapeutic

3. General Procedures

Why use CAPS Logbook?

• An aid to learning – provides learning objectives• Help with procedures• Guide & encouragement for practical examination

skills• Compulsory progression through medical school• Skills learning are a GMC requirement

CAPS Feedback

• Survey of the student body regarding the introduction of CAPS Logbook

• Recurring themes identified-aids practical skill learning

-re-enforces importance of practical skill learning

-inter-professional interactions & learning from MDT

-personal record of progress

-documentation and reference for practical skills

• Overall, scheme welcomed

Thank you

Any questions?

The UCL experience

Focus groups with students from pilot year (2011-12)

Students’ perception of intended purpose

Feedback

Minimum standards

Monitoring progress

Logbook/record

Students’ perception of intended purpose

I think the idea is that it helps you get feedback on your progress ... I

really don’t think it does.Feedback

Minimum standards

Monitoring progress

Logbook/record

Students’ perception of intended purpose

I think most of the time most people are doing fine, so they don’t really help ... but they

might flag up if you weren’t doing fine.

Feedback

Minimum standards

Monitoring progress

Logbook/record

Students’ perception of intended purpose

Feedback

Minimum standards

Monitoring progress

Logbook/record We go round so many

departments, it’s hard to have a

central [overview]... It’s a

good way of keeping track of

what we’re doing.

Students’ perception of intended purpose

Feedback

Minimum standards

Monitoring progress

Logbook/record

To sort of correlate everything

together in one place for easy

access later on... records of

practicals that you do.

Students’ perception of actual purpose

Tick-boxSelf-

monitoring

Way of lifePractice clinical skills

Students’ perception of actual purpose

Tick-boxSelf-

monitoring

Way of lifePractice clinical skills

Because this one is I guess more clinics-focused

and you have to get it done by

clinicians, I do feel it keeps you on your toes a bit

more...

Students’ perception of actual purpose

Tick-boxSelf-

monitoring

Way of lifePractice clinical skills

Because this one is I guess more clinics focussed and you have to

get it done by clinicians, I do feel

it keeps you on your toes a bit

more...

And if they’ve had two supervised

examinations and two history presentations .. it’s good, because in the process they’ve

done that and had the practice.

Students’ perception of actual purpose

Tick-boxSelf-

monitoring

Way of lifePractice clinical skills

I think at first a lot of students just thought this

is something that the med school is doing .. once you actually start

speaking to junior doctors and you hear that this is what they’re doing, [it]… it does make you feel ...

it’s not kind of a lost cause.

Students’ perception of actual purpose

Tick-boxSelf-

monitoring

Way of lifePractice clinical skills

I think the good thing to be said about us having the ePortfolio, talking to other people from other

med schools ... you have it when you’re a junior doctor and probably

forever, so at least we get used to it now... when we actually qualify it’s not this other new thing we have to get our heads

around.

Students’ perception of actual purpose

Tick-boxSelf-

monitoring

Way of lifePractice clinical skills

Although some might argue that it’s just... a

paper pushing exercise... to show that

you have been attending things.

Students’ perception of actual purpose

Tick-boxSelf-

monitoring

Way of lifePractice clinical skills

...they can’t be bothered typing you know more than a few words really in their

evaluation there. So I think the idea that it helps you get

feedback on your progress... I don’t really

think it does.

Actions

Students

• Information for students

Staff

•Training for assessors

Process

•Number of assessments

Ethos

•Hearts and minds mission

Thank you