Assessing Teaching and Learning of Medical Statistics: Teachers and Learners

Perspectives

Dr. Salma Ayis and UMS Stat teamKing’s College LondonDivision of Health and Social Care ResearchDepartment of Primary Care and Public Health Sciences

Burwalls MeetingIst July 2015King’s College London

Outlines

• Teaching and learning (statistics)• Challenges in medical schools• Assessing the process as perceived by learners and

teachers: reflection• Taking actions to improve• Assessing the change?

Why Do we Teach Medical Statistics?What are the intended learning outcomes?

o Statistics is arguably an integral component of scientific methods, particularly biologic and health sciences.

o Medical students are taught statistics to allow them understand the conceptual basis and usage of common statistical methods, and their applications in clinical medicine and public health.

o These are important for the critical reading, understanding and assessment of research findings, hence for the progress of evidence based medicine (EBM).

Teaching and Learning

o Teaching is as much about setting the context or climate for learning.

o The process of teaching and learning is affected by a range of factors:

• Student specific, such as motivation• External(extrinsic), such as the environment.

Most widely acknowledged were:o Differences in cognitive and learning styleso Approaches to learningo Learning environments • Teaching statistics in medical schools has some additional factors

that are often challenging.

Challenges ??

Aim: to meet the intended learning outcomes from a statistics course to medical studentso A subject that seem peripheral to the main thrust of the

medicine courseo Considered dryo Students tend to give a much lower approval to parallel

courses than to the main teachingo Time constraints and the heavy programmeo The proportion of time and resources allocated to teachingo The proportion of the subject in the final assessment, among

other reasons

6

Alignment of teaching and intended learning outcomes

o Clearly specified outcomes of teaching in terms of topic and content, in the

level of understanding that students were expected to achieve.

o Set up an environment that maximises the likelihood that students will engage

in the activities designed to achieve the intended outcomes(ILOs); and

o Choose an assessment tasks that will tell how well students have attained

these

[Biggs J. Constructive Alignment- and why it is important to the learning process. Higher Education

2010]

Teachers and Learners Perspectives

Assessing Teaching and Learning

o Students: 276 male and female, MBBS1 students took part in a pilot survey study.

• Questions 1 and 2, grades background knowledge and knowledge following tutorials.

• Question 3, grades the link between lectures and tutorials.• A space was provided for a free text / comment.• Data were analysed using quantitative and qualitative methods. o Teachers: two focus groups including all available teachers (10-20)

convene twice a year to share experiences and to reflect on their practice of teaching medical statistics

King’s College London, School of Medicine, MBBS1

Teaching Statistics Evaluation form (A Pilot)

 

Date: 27 September, 2012

 

We would like to know your views on this practical / tutorial and its link to the preceding lectures. These will help us improve the teaching and learning of statistics for MBBS1 students at King’s.

 

(1) Please rate the extent to which today’s practical / tutorial has improved on your knowledge and understanding of Statistics applied to medicine by circling one option for each of the items below, where:

1 = poor and 5 = excellent

(2) To what extent do you see this practical linked (aligned with / matched) with the preceding lecture;

1 = poor alignment and 5= excellent alignment

(3) Please add any comment or suggestion/s

Thank you very much for your response!

 

This is anonymous

Extent of alignment

1 2 3 4 5

Pre tutorial knowledge 1 2 3 4 5

Post tutorial knowledge 1 2 3 4 5

Example: Teachers Meeting End of Term Agenda

Aim: Share our experiences this year and plan further improvements focusing on: How the teaching went  How do we think it could be further improved Tips for new people who will be joining the team Feedback on tutorials you taught

Feedback structured as: o Environmental Factorso Studentso Materials

Findings

Knowledge gained via tutorials

Pre tutorial knowledge * post tutorial knowledge Cross tabulations

Post tutorial knowledge 1 2 3 4 5 TotalPre tutorial knowledge

1 0 7 18 7 1 33

.0% 21.2% 54.5% 21.2% 3.0%

2 1 3 37 48 1 90

1.1% 3.3% 41.1% 53.3% 1.1%

3 0 3 19 81 11 114

.0% 2.6% 16.7% 71.1% 9.6%

4 0 0 1 13 19 33

.0% .0% 3.0% 39.4% 57.6%

5 0 0 1 0 5 6

.0% .0% 16.7% .0% 83.3%

1 13 76 149 37 276

.4% 4.7% 27.5% 54.0% 13.4%

Extent of link between lectures and tutorials

1 = Poor 2 3 4 5 = Excellent Total

Frequency 8 10 88 107 49 262

Percent 2.9% 3.6% 32.0% 38.9% 17.8%

What Students Think

Too much / too fast

“I found taking to fellow students more useful. Sometimes tutor gives so much help. That it is a bit overwhelming”

“the MCQs were ambiguous. Quote, use the word certain, suggest stats is an absolute, a 95% CI is not certain it is 95%”

“Do work before hand and go through in more detail in tutorials. Maybe more difficult questions and problems “

“Please talk slower, for international student like me, it could be hard to get the meaning at instant”

“Classes should be optional for students with previous degrees etc with a statistical component”

Is stat relevant to being a doctor?

“Don't really see the practical side to this in being a doctor”

“it would be nice to highlight the relevance of these more with how it would help us in the future , because it would make it easier to put in context”

“Classes should be optional for students with previous degrees etc with a statistical component.”

“lecturer was enthusiastic & tried very hard to make this type of statistics interesting to a group of medical students. This is not easy!. Effort appreciated”...

Everything seems fine

“the tutorial was helpful, the facilitator was nice and knowledgeable”

“well explained. Answered questions”

What Teachers Think

Environmental Factors

o Availability of pens, marker boards, flip chart: Back up:• take yours• check room the day before• contact AV 24 hours before your session

o Difficulty using overhead systems• Training is proposed and in progress

o Virtual Campus – difficult to navigate VC, and some abbreviations not clear- • Provide direct links in circulations

Students’ Factors

o Attendance: varies greatly, much fewer students around exam times.• Head of year 1 & 2 re-inforce the importance of attendance• We take attendance (signed sheet) for each session

o Preparations: varies greatly.• A short standardised overview of core information is given • Head of year 1 & 2 reinforces the importance of preparation pre-all

tutorials including stat tutorials in a circulation to all studentso Students that ask for further assistance are not necessarily the ones that are

struggling with the core material.• We provide further assistance on demand for all who are interested:• A revision lecture• FAQ online service 

Material related

• Varied preparation and different knowledge backgroundo May need to go over key points of topic.o Streamlining of core knowledge - suggestion for a front page (overview

of material) for tutors, to address main points.o This could be given to student but may lead to them not reading wider

notes. (??)o It could highlight the essential knowledge – similar to ‘key messages’?o Put students into groups based on how they have prepared (or not)?

• Specific tutorials may benefit from shortening

Points that have arisen/tips for other

o Prepare 10-15 minutes of material beforehand in case students are not prepared.

o There are different styles of teaching – and don't need to stick to a particular format.

o Helpful to see others teach – to understand different teaching styles.

Further Actions

• We launched a revision to lectures and tutorials, including simplifying /shortening.

• Re-inforce the use of statistics and why students learn statistics formally in the first lecture (JP) and informally afterwards

Evidence-based medicine

o Why is it important for medicine?

o Do we need to do research to practice evidence-based medicine?

o What research skills are needed?

Tomorrow's Doctors

Document that describes medical curriculum in

broad terms

...quotes from section on the Doctor as a scholar and scientist

Conclusions

o Reflection on practise is the way to improveo Challenges include, time, background knowledge, the

perception of the subject among others: can be addressedo Assess changes as you go along in a formative wayo Beyond our reach is the percentage of the subject in the final

assessment