assessment fhk cmg

Assessment- Making Thinking Visible

Cormac McGrath

Director Unit for Medical Education (CLK, LIME)

04/15/23Centre for Medical Education, Karolinska Institutet

Summary

Students’ learning is guided by examinations Formative assessment can facilitate learning Examination at PhD level is oftentimes too basic As course leader you can influence the design of your course

and the ratio of teaching and examination.


Do you remember

All Boys Should Come Home Please

04/15/23Unit for Medical Education, Karolinska Institutet


Aim

Participants will analyse their courses in terms of constructive alignment, specifically emphasising assessment forms and share experiences and ideas for development.


Learning outcomes:

After the session you will have an understanding of:

• The function of assessment

• The effect of assessment on student learning

• Formative and summative assessment

• ”Backwash effect”

• SOLO taxonomy

• Criteria and norm-referenced assessment

And you will be able to:

Distinguish between formative and summative assessment

Set up a grading scheme for a number of objectives

Distinguish between criteria referenced and norm-referenced assessment


Discuss:(2 mins)

1. Positive experiences of assessing students’ work

2. What concerns/ queries do you have about assessment?

Share your discussion with the group


Two kinds of assessment

Formative assessment- Continuously runs during the teaching/ learning process - Diagnostic: gives feedback to students and teachers on

* strengths and weaknesses* difficulties* misunderstandings

- Gives an opportunity to modify/ improve

Summative assessment- Final. At the end of a course- Descriptive. How well did the students learn the material/ knowledge/ skill - For ranking and selection. Usually no possibility to modify/ improve

04/15/23Centre for Medical Education, Karolinska Institutet 15 april 2023 8

When do we assess?

Formativeassessment Summative

assessment

Course/Education


Use 2 minutes to write down

- An example of formative/summative assessment that you thought was good.

Good: one or more of the following:Educational, Valid, Reliable, Cost effective, acceptable to faculty & feasible


Why do we assess our students’ work?

Write down three of the most important reasons for assessing students’ work.

Turn to the person next to you. Compare and pick three of your reasons that you think are most important.

Turn to another pair and pick the three most important reasons(snowballing technique)

The function of assessment

To facilitate learning:-Find out whether students achieved the aims and goals-Motivate students to learn (stick or carrot?)-Make visible, assist and improve learning by giving useful feedback.

To facilitate teaching:-Give feedback, to give an opportunity to improve teaching

For institutional/ professional requirements:-Grading, ranking or select students-Maintain standards


Assessment is the main factor that influences student learning

It is the students’ understanding of the requirements for assessment that makes the ”hidden curriculum” and that has an impact on how students learn.

April 15, 2023

13

Backwash effect

ILOsTeaching methods

Examination

Examination Study methods Student learning

Implications for Teaching & Learning

What is the ratio between teaching and examination? When can examination act as a catalyst for learning and

teaching?



Structure of the Observed Learning Objectives (SOLO)

A way of describing how learners’ performance (level) grow in complexity when mastering academic tasks

Name the 6 wives of Henry VIII

Critique your examination Look at your exam questions What type of knowledge are you asking for Promoting Higher order thinking How do the questions stand up in terms of SOLO?



Progression in the choice of verbsSOLO taxonomy

Identify

Make simple procedures

Number

Describe

List

Combine

Compare

Contrast

Explain relationships

Analyse

Relate

Apply

Misses the point

Theorise

Generalise

Make a hypothesis

Reflect

(Biggs & Tang 2007)

Powerful examinations?

The design is supposed to create the possibility to assess: specific cognitive competences such as problem-solving,

including formulating questions and critical thinking, information competences, such as searching for relevant

information, making informed judgements, efficient use of information, analysing data,

communication competences, such as presenting data communicatively, both oral and written;

meta-cognitive competences such as self-reflection and self-evaluation.


Alignment

Criterion-referenced assessment:How well students have learnt what we intended them to

learn.

Norm-referenced assessmentComparing students performances with each other, by

ranking.


Constructive alignment (Biggs, 1999)

Curriculum that we teachTeaching methods that we useAssessment procedures, and methods of reporting

resultsClimate that we create in interactions with studentsInstitutional climate, rules and procedures we have

to follow.

Deep learning

Peer Learning 15 april 2023Cormac McGrath

21

Authentic learning and examination

Authentic learning and examination gives the students the opportunity to explore, discuss and create/construct meaningful concepts in a contexts that reflect knowledge that is valued.

Integrated Representational as opposed to

performance of understanding

http://www.youtube.com/watch?v=NUd1BhokZq4

Traditional contra Authentic

Traditional Authentic Selecting a Response ----------------------------------Performing a Task Contrived -----------------------------------------------Real-life Recall/Recognition ------------------------------------Construction/Application Teacher-structured -----------------------------------Student-structured Indirect Evidence ------------------------------------ Direct Evidence

23-04-15Cormac McGrath


Traditional assessment

Emphasis on product Is fair (?) Is economical Encourages repetition of large amounts of literature, which can provide

opportunity for overview of the subject

Criticism: Encourages quick answers rather than slow investigation Correctness rather than risk taking Memory rather than conceptual development Reproduction rather than creative application Little space for feedback Difficult to measure deep understanding of concepts and context

(Lindström, 1997)

Towards progressive Performances of UnderstandingScenario/caseIndividual Practical ExamMaster Programme in Medical Biology,5th semester spring term -04

Jan and Birgitta have, during some time, tried to conceive a sibling to Lisa, 4 years, without succeeding. The couple had to try quite some time before Birgitta became pregnant with Lisa but this time it has taken more than a year without success. Birgitta’s situation has, since she wet back to work three years ago, changed radically. She has been working really hard and been promoted to an intermediate executive position. The work has meant a lot of meetings, representations, and increased responsibility at the same time as it has been very stimulating. This has lead to a substantial increase in weight, irregularities in her menstrual cycle, and drastic drop in time spent with the family. After more than one year of trying to conceive Birgitta decides to seek help. Physical testing confirms her weight gain, mainly as abdominal fat, and drop in cardio vascular fitness. Blood test also shows high blood fat values. She receives the recommendation to change diet and starting to take care of herself.

Performance of understanding

Part I. Day I Do a general problem definition based on the included scenario Formulate as many sub problems/questions as you consider motivated.

Motivate shortly way you believe your sub problems are relevant and how you relate them to general problem.

Decide up on and point out one sub problem that you will penetrate in detailed during day II.

Do a work plan for how you intend to handle your sub problem. It should be clear on what you need to find out, how you intend to find it out and it should also indicate how you intend to present it.

Hand in your answers.

Day two…

Powerful examinations?

The design is supposed to create the possibility to assess: specific cognitive competences such as problem-solving,

including formulating questions and critical thinking, information competences, such as searching for relevant

information, making informed judgements, efficient use of information, analysing data,

communication competences, such as presenting data communicatively, both oral and written;

meta-cognitive competences such as self-reflection and self-evaluation.

Just a thought!

How do we embed generic skills in the context of performances of understanding allows students to see why such skills are important

How do we contextualize examination questions? If we re-conceptualise examinations as performances, would

this allow an opportunity for re-drafting course design? Do we want to?

Performance of understanding

Presenting a problem that requires Problem identification Decision making Inferences inductive-deductive Divergent thinking skills Evaluating thinking skills Is Genuine and authentic Requires reasoning

Discussion I

Look at a colleagues examination, Where is the focus, content, skills, performance? Give examples, elaborate and justify 5 min preparation 15 ventilation

Closing Thoughts on Alignment

Is your teaching aligned? Goals-Teaching and learning activities-

Examination

How do we achieve inter-assessor reliability? How do we get the students to understand the assessment

criteria?

The muddiest point

Take a post it note and write down one or a few things that still seem unclear to you, post on the reverse side of the door as you leave for the reading log task.


Examination and assessment and meaningful learning

To which extent does your examination support meaningful learning?

To which extent does your examination allow your students the opportunity to demonstrate that they have achieved the ILOs?

Centre for Medical Education, Karolinska Institutet

Discussion

To which extent do the assessment criteria express distinct qualitative differences?

Can you give examples? How do you share this with the students? How well do you discuss examination and

assessment/assessment criteria? Individual preparation 5 min Group discussions 10 min

Summary

Students learning is guided by examinations Formative assessment can facilitate learning Examination at PhD level is oftentimes too basic As course leader you can influence the design of your course

and the ratio of teaching and examination.



References

Biggs, J. &C. Tang (2007). Teaching for Quality Learning at University. SRHE & Open University Press, Buckingham.

Carroll, J. &C.-M. Zetterling (2009). Guiding students away from plagiarism. KTH Learning Lab, Stockholm.

McConnell, DA., Steer, DN., & Owens, KD. (2003) Assessment and active learning strategies for introductory geology courses. Journal of Geoscience Education, v. 51 n:o 2, p. 205-216


Links

Concept maps: http://users.edte.utwente.nl/lanzing/cm_home.htm

Venn diagrams: http://www.enchantedlearning.com/graphicorganizers/venn/

http://users.edte.utwente.nl/lanzing/cm_home.htm

http://www.enchantedlearning.com/graphicorganizers/venn/