Business School

ww

w.b

usin

ess.

broo

kes.

ac.u

k

Seminar Series 2011Assessment and Feedback

Using Dialogic Feedback to Engage Students Dr Jill Millar

Ms Carole Thompson

7 June 2011

Business School

Overview

Outcomes

Help participants to examine the role of feedback in the context of their own discipline and institution;

Help participants to develop plans and strategies to use dialogic feedback at module and programme level.

Programme

Business School

Dialogic Feedback

What is dialogic feedback?

“a dialogical and two way process that involves co-ordinated teacher student and peer- to- peer interaction as well as active learner engagement” (Nicol, 2010, p. 503)

Business School

Problems and possibilities

Please spend the next few minutes identifying up to

3 problems and 3 possibilities

that you associate with assessment feedback, and writing them down on the post-it notes supplied.

For example you may feel that feedback is a bit of a waste of time (a problem) or that some students love it (a possibility).

Please write one problem/possibility per post -it note.

Business School

Problems and possibilities

Student engagement

Resources

“wave goodbye to really serious chunks of time”.

Feedback as a learning toolFeedback has extraordinarily high and consistently positive effects on learning compared with other aspects of teaching or other interventions designed to improve learning

Black and Wiliam (1998) - in a comprehensive review of formative assessment

Feedback and the NSS surveyQ.9 Feedback on my work has helped me clarify things I did not understand

Business School

Feedback song!

http://www.brookes.ac.uk/aske/MultimediaResources.html

Business School

Dialogic pedagogy?

Laurillard- ‘conversational framework’ in which student conceptions, understandings are revealed, debated and re-worked. “There is no escape from dialogue” (2002, p. 71)

Dialectic?

Bakhtin –distinction between

monologized pedagogical dialogue- ideas or concepts affirmed or repudiated by the authority of the teacher (cited in Matusov, 2004, p. 7)

and

dialogized pedagogical dialogue confronting and testing diverse ideas, concepts, understandings ‘born between people’ (cited in Matusov, 2004 p. 7)

Business School

Feedback within a dialogic pedagogy

Develop conceptual understandings

Develop an aligned understanding of standards and quality

Feedback works because it helps students close the gap between their actual and desired performance in a piece of work (Sadler 1998).

Business School

Rust, O’Donovan and Price 2005

Feedback within dialogic pedagogy 2

Business School

Dialogic feedback

“An impoverished and fractured dialogue” (Nicol 2010, p. 503).

Nicol focus on written feedback: a dialogue with feedback

F2F feedback: a dialogue about feedback

dialectic and dialogical

Business School

Dialogic feedback: purpose

Needs of learner and needs of assessment

Contextual

Justify the grade; clarify expectations; transform understandings

Business School

Dialogic feedback: participation and engagement

And engage-

HEA funded FDTL Project “Engaging students with assessment feedback” (2005-2009).

Engagement: staff- student openness to F2F dialogue supported engagement AND helped student understanding of feedback, of feedback utility and self efficacy in relation to feedback (reinforcing engagement)

Oxford Brookes Business School Face to face feedback initiative (2009-2010) created opportunities for dialogue

Facilitation rather than initiation

Focus of dialogue: confirmation of understandings; feedback purpose

Improved satisfaction

Plus: building relationships

Business School

Dialogic feedback: participation and engagement 2

A lack of opportunities for dialogue reduces participation and engagement?

Tariq

http://www.brookes.ac.uk/aske/MultimediaResources.html

“Lecturers”

http://www.brookes.ac.uk/aske/MultimediaResources.html

How create opportunities?

Business School

Opportunities for dialogue

Some factors to be considered:

Student engagement

Staff interaction

Clarity over aim of feedback session

Other factors…

Business SchoolFeedback type

“Spotting feedback”

Business School

Dialogic feedback- your experiences?

“a dialogical and two way process that involves co-ordinated teacher student and peer- to- peer interaction as well as active learner engagement” (Nicol, 2010, p. 503)

Opportunities

Purpose

Characteristics

Engagement

Business School

Dialogic feedback-our experiences

Staff buy-in

Scheduling within the module

Making it personal

Relevance and focus of feedback

Organisation

Managing expectations

Business School

Application to a year-long module: ‘Intro. to Business & Management’

Context

• Combined Honours module:'Intro' to Business & Management'

• Multidisciplinary - some economics, marketing, strategy, operations and so on

• Large classes - 100 (lecture + seminar)

• An orientation module (first year undergraduates)

• Wheatley is a satellite campus, so engagement is tough!

• For UGs, this is a discipline that's not always positively selected

Business SchoolL

earn

ing

deve

lopm

ent

Points in a module where students may require dialogic,

transformative feedback

Semester 1 Semester 2

Orientation

Epistemological ChangeE.g. Dualism to Relativism (Perry, 1970)

Business School

Targeting opportunities for dialogue: feedback purpose and resources

Low High

Resources

1

Low High

Resources

2

Low High

Resources

3

Low High

Resources

4

Semester 1 Semester 2

Learn

ing

& D

evelo

pm

en

t

Week 4 Week 10 Week 5 Week 12

Business School

• Complex problem; selection and application of theory

• 45% of module mark

• Context: Royal Mail; Vince efforts to calm disquiet about privatisation by offering a share ownership scheme to workers

• A ‘messy’ issue; no obviously 'right' motivational theory to apply to it

• The assignment went beyond analysis and was about applying appropriate theory/ies, building arguments, and making recommendations

• These were 1st year students; we knew they'd find it tough

Assignment 3

Low High

Resources

3

Business School

Assignment 3 “Deconstructing a complex problem, and applying relevant theory/ies ...”

Purpose:

•Transformational development; epistemological change; 45% of module marks

Preparation:

•Discussed brief and criteria (repetition and dialogue => familiarity with terminology and process)

•Peer review

Feedback (week 7, semester 2):

•Individual written feedback (including a 'review of reviews'); •... with some points chosen for 15-min feedback tutorial•Continuing with A-grade examples, to illustrate quality•2-week turn-around; 2-4 weeks for F2F feedback tutorial with feed-forward to final assignment which is an exam

Engagement with feedback (hand-back in week 9):

•F2F tutorial focuses on individual points for improvement

Low High

Resources

3

Business School

Targetting opportunities for dialogue: feedback purpose and resources

Low High

Resources

1

Low High

Resources

2

Low High

Resources

3

Low High

Resources

4

Semester 1 Semester 2

Learn

ing

& D

evelo

pm

en

t

Week 4 Week 10 Week 5 Week 12

Business School

Orientation (Yorke, 2007)

Radical movement involving zones of discomfort, ‘threshold concepts and troublesome knowledge’ (Meyer and Land, 2005)

Year 1 Year 2 Year 3

Lea

rnin

g de

velo

pmen

t

Points in a programme of study where students require dialogic,

transformative feedback

Business School

Sharing ideas and practices ...

1. Could (something like) this structure be adapted for your own modules, and for your discipline?

how could it be improved?

what might work instead?

2. How do you use props/resources/activities to engage students in conversations about feedback?

Business School

BibliographyBlack, P. and Wiliam, D. 1998 Assessment and classroom learning, Assessment in Education, Vol. 5, No.1, pp 7-74

Brown, E., and Glover, C. (2006) ‘Refocusing written feedback”, in Rust, C. (ed), Improving Student Learning by Assessment, Proceedings of the 2005 13 th International Symposium. Oxford: OCSLD

Gibbs, G., and Simpson, C., 2002. Does your assessment support your students’ learning? [online] Centre for Higher Education Practice, Open University. Available from http://www.open.ac.uk/science/fdtl/ documents/lit-review.pdf [Accessed 4 may 2005].

Handley, K., Price, M., and Millar., J. (2008) ‘Engaging students with Assessment Feedback’. Final Report for FDTL Project 144/03, Available online at: https://mw.brookes.ac.uk/display/eswaf/Home

Laurillard, D. (2002), Rethinking University Teaching: a conversational framework for the effective use of learning technologies , London: Routledge

Matusov, E., ( 2004), Bakhtin’s Dialogic Pedagogy, Journal of Russian and East European Psychology, Vol 42, No. 6. pp.3-11.

Meyer, J, & Land, R., (2005), 'Threshold concepts and troublesome knowledge (2): Epistemological considerations and a conceptual framework for teaching and learning',  Higher Education, 49, 3, pp. 373-388.

Moore, R., Arnot, M., Beck, J., Daniels. H. (eds), (2006) Knowledge, Power and Educational Reform: applying the sociology of Basil Bernstein , RouledgeFalmer.

Nicol, D.(2010), 'From monologue to dialogue: improving written feedback processes in mass higher education',  Assessment & Evaluation in Higher Education, 35, 5, pp. 501-517

Nicol, D, & Macfarlane-Dick, D 2006, 'Formative assessment and self‐regulated learning: a model and seven principles of good feedback practice',  Studies in Higher Education, 31, 2, pp. 199-218

Rust,C., O’Donovan, B., Price ,M. (2005), A social constructivist assessment process model: how the research literature shows us this could be best practice. Assessment and Evaluation in Higher Education, 30 (3), pp. 231-240.

Sadler, D.R.(1998), Formative Assessment: Revisiting the Territory. Assessment in Education: Principles, Policy and Practice, 5 (1), pp 77-85.

Skidmore, D 2006, 'Pedagogy and dialogue', Cambridge Journal of Education, 36, 4, pp. 503-514.

Wankat, P., and Oreovicz, F. 1993, Teaching Engineering, New York; London: McGraw-Hill Wegerif, R. (2008). Dialogic or dialectic? The significance of ontological assumptions in research on educational dialogue. British Educational Research Journal, 34(3), 347-361

Yorke, M.(2007), The fi rst-year experience: successes and challenges. Paper prepared for the third seminar in the series on Mass higher education in UK and international contexts, organised by the Centre for Research in Lifelong Learning, Glasgow Caledonian University and others, and held on 29-30 May. Available from: http://crll.gcal.ac.uk/docs/masshe/mantz%20yorke.

pdf [22 November 2007].