Business School

Lessons learned from our FDTL5 project:Engaging Students with Assessment Feedback

FDTL Final ConferenceNovember 2009

Dr Jill Millar & Dr Karen Handley jmillar@brookes.ac.uk & khandley@brookes.ac.uk

Our research project & aims for this workshop

Research project

• Investigate and encourage the adoption of feedback practices which support student engagement

• Share understandings of the student experience in HE

The 4 stages of our research

• 35 student and staff interviews; 760 questionnaires on student views on different types of feedback (2006-7)

• 7 case studies with 3 partner HE institutions (2006-7)

• 5 cascade partner initiatives in 5 HE institutions (2007-8)

• 12 transferability partner micro case studies in 5 HEIs (2008-9)

Aims for this workshop

• Lessons learned about our 'cascade' partner approach

• Lessons learned - and questions still remaining - about how to research students' experiences of feedback [access & methodology]

2Business School

Our cascade approach - structure

3Business School

Core project team

Oxford Brookes

Bedfordshire Bradford

Bournemouth Sunderland NorthumbriaUniversity of

West of England

London Met

University of West of England

Sunderland NorthumbriaBrookes London Met

7 case studies in 3 project partners (2006-7)

5 initiatives in 5 cascade partners (2007-8)

12 mini case studies in 5 transferability partners (2008-9)

Our cascade approach – benefits and tensions

4Business School

Benefits

Broadening and deepening of ideas

Broadening and deepening of involvement

Testing and re-testing of methods

Tensions

Heterogeneity of results: less robust?

Communication?

Control!

Lessons and questions about our methodology:

(1) talking to students …Ethical considerations:

• Dependency• Power relationships• Our ethics committee regulations

Attracting interest:• Emails?• Talking to large groups• PC 'message of the day'; or links from VLE• Adverts• 'Willing to listen' lists• Recruitment by 'friendly' students?

Retaining interest:• Incentives? (lunch; digital recorders; vouchers?) - what else?• Logistics and timetables

5Business School

Lessons and questions about our methodology:

(1) talking to students …Ethical considerations:

• Dependency• Power relationships• Our ethics committee regulations

Attracting interest:• Emails?• Talking to large groups• PC 'message of the day'; or links from VLE• Adverts• 'Willing to listen' lists• Recruitment by 'friendly' students?

Retaining interest:• Incentives? (lunch; digital recorders; vouchers?) - what else?• Logistics and timetables

9Business School

Lessons and questions about our methodology:

(2) researching student engagement

10Business School

Tutor’s assignment brief

Tutor’s feedback with the assessed assignment

Student’s submitted

assignment

1. Assessor writes the assignment brief

2. Student creates and submits the assignment

3. Tutor assesses the assignment, and gives

formative feedback

4. Student engages with the feedback

Tutor involvement

Student involvement

Lessons and questions about our methodology:

(2) researching student engagement

11Business School

Tutor’s assignment

brief

Tutor’s feedback with the assessed assignment

Student’s submitted

assignment

1. Assessor writes the assignment

brief

2. Student creates and submits the

assignment

3. Tutor assesses the assignment, and gives

formative feedback

4. Student engages with the feedback

SO

CIO

CU

LTU

RA

LC

ON

TE

XT

STUDENT OUTCOME

Lessons and questions about our methodology:

(2) researching student engagement

12Business School

Tutor’s assignment

brief

Tutor’s feedback with the assessed assignment

Student’s submitted

assignment

1. Assessor writes the assignment

brief

2. Student creates and submits the

assignment

3. Tutor assesses the assignment, and gives

formative feedback

4. Student engages with the feedback

SO

CIO

CU

LTU

RA

LC

ON

TE

XT

STUDENT OUTCOME

Outcomes influence student’s engagement with

future feedback events

Lessons and questions about our methodology:

(2) researching student engagement

• The need to consider the temporal and relational (sociocultural) dimensions of engagement (Price et al., 2009; Handley et al., 2009)

• The need to (re)consider the appropriate unit-of-analysis and appropriate methods:

Individual properties vs processes of sociocultural activity (Matusov, 2009, p320)

Holism as ‘an impossible methodological task’ (Matusov, 2009, p323)

Impossibility of seeing context; but can we see the ‘seeds of time’ (Mercer, 2009)

‘Planes of analysis’ (Rogoff, 1995) [UoA is never self-contained and is always part of a bigger system which has to be considered]

13Business School

Lessons and questions about our methodology:

(2) researching student engagement

• The need to consider the temporal and relational (and sociocultural) dimensions of engagement (Price et al., 2009; Handley et al., 2009)

• The need to (re)consider the appropriate unit-of-analysis and appropriate methods

• Choices we’re still thinking about:

14Business School

Snapshots-in-time

(interviews, observations, looking for the 'seeds of time')

Longitudinal research

(diaries, sequences of feedback 'events', observation)

Individual-in-context

(Discursive repertoires ...)

Sociocultural context

(Activity theory; Critical discourse analysis ...)

Lessons and questions about our FDTL project: Engaging students with assessment feedback

Lessons learned:

• Benefits and tensions in using a cascade approach

• The need to reconsider our unit-of-analysis

Questions ...

• How can we attract student involvement in our research?

• What methods give us a window onto the relational and temporal dimensions of student engagement with feedback?

15Business School

References

Handley, K., Price, M. & Millar, J. (2009 in submission) ‘Beyond 'doing time': investigating the concept of student engagement with feedback’

Matusov, E. (2009) ‘In search of ‘the appropriate’ unit of analysis for socio-cultural research’, Culture and Psychology, 13, 3, 307-333

Mercer, N. (2008) ‘The seeds of time: Why classroom dialogue needs a temporal analysis’, Journal of the Learning Sciences, 17, 33-59

Price, M., Handley, K. & Millar, J. (2009 in submission) ‘Feedback - focussing attention on engagement’

Rogoff, B. (1995) ‘Observing sociocultural activity on three planes’. In J V Wertsch et al., Sociocultural studies of mind. New York: Cambridge University Press

16Business School