Assessment and Problem Based LearningPBL 2004

Glen O’Grady

Director, Center for Educational Development

Overview of the Workshop

More about you Assessment Assessment in PBL Grading

How do you Assess in your PBL Programmes?

Draw your response– How it is done?– Why?– The effect (intended or

unintended)

Why do we assess?

To measure learning To foster learning

What are we assessing?

Learning…. but what is learning?

Conceptions of LearningConceptions of Learning

A. Increasing one’s knowledgeB. Learning as memorizing and

reproducingC. Learning as applyingD. Learning as understandingE. Learning as an interpretive process

aimed at understanding realityF. Learning as changing as a person

Willis 1993

Subject X

Subject X

Subject X

Subject Z

Syllabus

Instruction-based

Teacher

Students

Instruct, discipline, assess

Dissemination of Knowledge

Assessment is testing how well students have retained what the teacher has disseminated.

Prior Knowledge

Problem-based

Student

Team members

Facilitator

The Problem

(Curriculum)

Construction of Knowledge

Assessment is about coming to know how students construct their understanding of knowledge by giving tasks that also helps to foster this construction of knowledge

How can assessment be organized in PBL so that it both fosters & measures learning for understanding?

Assessment needs to be… considered as an integral

element in facilitation based on multiple methods. focused on both product and

process. holistic

Case Study on Assessment in PBL

PBL at The Republic Polytechnic (RP-PBL)

Each semester, 5 modules (subjects) each day students attend and work exclusively on one module

Students follow a regular schedule each day

RP-PBL: 1st meeting

Class of 25, 5 teams of 5 students

Presented a problem Students under the

guidance of the facilitator work on defining the problem and identify issues they will do research on.

Approximately 1 hour

RP-PBL: 1st Breakout

Student work individually and in their teams to:

– Find and review resources– Begin to develop tentative

solutions for the problem– Refine their definition of

the problem

RP-PBL: 2nd Meeting

Meet with the facilitator who checks on their progress

Focus on any difficulties students may be having

Helps students to develop learning strategies

RP-PBL: 2nd Breakout

Student continue to work in their teams

Review resources Develop a solution/

explanation based upon their shared understanding

Produce a presentation 2-3 hours

RP-PBL: 3rd Meeting

Meet with the facilitator Students present their

solutions/explanations Students observe how others

have solved the problem Facilitators probes and critique

these solutions giving additional information where necessary

Students further check their understanding by doing a quiz focussed on the key issues

RP-PBL: Assessment

Performance & Presentations Self & Peer Evaluation Reflection journal Students get feedback everyday

– Verbal feedback in class (interactions)

– Daily grade derived holistically– Written feedback

Students every month sit an understanding test

Module grade is determined by a combination of daily grades and understanding tests performance.

Exercise

Method of Assessment How does it foster learning? How can it assist facilitators

in their summative judgment (measure learning)

Key challenges

I. Team Oral/Written Presentations

How can it foster learning?– Develops communication skills (Hay 1994)– Verbal articulation (Rowntree 1987)

How can it measure learning?– It manifests the application of knowledge in the context of a

problem

Key challenges– Criteria – Collusion– Tip of the iceberg (Lakomski 2002)

II. Peer and self assessment

How can it foster learning?– Develops the capacity and ability to judge (Warren Piper et

al. 1996, Boud 2001)– Engenders normative values in the way students relate to

one another (Shoop 2000) How can it measure learning?

– Manifests the way students think about themselves and others

Key challenges– Reliability of these judgments

III. Learning Journal

How can it foster learning?– Writing as a tool for organizing thoughts– Reflection the integration of theory and experience (Schon 1983,

Butler 1994)– Focuses on the process of attributing meaning

How can it measure learning?– It allows the facilitators to have an insight into the experience of

the learner (process of reasoning, their interaction with others, their emotive connection)

Key challenges– Students not knowing how to reflect

IV. Tests and Quizzes

How can it foster learning?– Emphasizes specific skill sets and the need to be familiar with

particular information. How can it measure learning?

– Demonstrates understanding that can be compared– Gives insight into reasoning processes (Gibbs 1992b)

Key challenges– Artificial and ritualized could encourage memorization & recall if

questions are not carefully crafted– Difficult to infer understanding (and its gradations)– How to give feedback

V. Observations of students’ discussions

How can it foster learning?– Allows the facilitator to interact with students in

meaningful ways (Barrows 1980,1988, Huba 2000)

How can it measure learning?– Observe first hand elements of the construction of

meaning

Key challenges– Facilitators perspective (Pratt 1998)

Other methods?

How can it foster learning?

How can it measure learning?

Key challenges

Marking & Giving Feedback

Managing the task of marking– Criteria– Mark broadly, give specific

feedback

Making a difference with feedback– Personal– Regular– Cumulative – Holistic

How do we derive a grade?

How do we derive a grade?

Deriving the daily grade in a manner the fosters learning for understanding.

– Collect as much information that is possible and practical.– Consider information collected in assessment as evidence

that should be weighted in relation to the context.– Have explicit criteria but be comfortable with the

“subjectivism” in applying that criteria.– Make it explicit (in feedback) the rationale for the judgment

link it to the evidence.– Be prepared to explain this judgement to peers (and others)

Holistic Judgements

Marks for student work should not be seen simply as individual bricks in a wall that represents a student’s final grade. Rather students’ work in its various forms are pieces of information or (evidence) that should be used collectively to form a picture of the quality of a students’ learning.

Holistic Judgements

The final grade of a student should represent our best professional judgment about the learning of a student both in terms of process and product.

The learning and the judgement should enduring.