Nicola Billam, Guinevere Glasfurd-Brown & Martin Sellens

CAA Conference 2006

Engaging Students in Formative Assessment: Strategies and Outcomes

Overview of the Project

• Student Portal Resources for Innovative Targeted Assessment

• Funded through the Higher Education Funding Council for England (HEFCE)

• FDTL-5• November 2004 – October 2006

• Dr Martin Sellens (Director)• Nicola Billam (Project Officer)

What is SPRInTA?

• Unfavourable staff/student ratios-affecting support on assessment and feedback given

Sports Science 8 staff; 75 undergraduates per year-9.4 students per member of staff

• Large classes• Lots of “marking”• National Student Satisfaction Survey: Essex-low

score on quality and time for turnaround of feedback

What is SPRInTA?

• Support student assessment by providing formative, targeted and personalised online support for assessment

• Facilitate independent study and self assessment

• On completion-15 Sports Science/Biological Science courses will be able to access a suite of online self assessment exercises and tutorial guidance

• All of these resources will be linked through the myEssex student portal

How?

• SPRInTA will design and develop online self-assessment and tutorial guidance for five types of assessment common to Sports Science students:

1.Multiple Choice Questions (MCQs)2.Short answer and essay style practice

examinations3.Data Analysis & Interpretation4.Practical Coursework (worksheets)5.Reporting in Scientific Paper Format

Progress to Date• The MCQ section is now complete• Currently working on the other sections (tutorial

guidance)

• MCQs• Provided online MCQ formative assessments for 8

Level 1 modules• 300-600 formative Multiple Choice Questions (MCQs)-

using QuestionMark Perception, linked into the portal• Split course material into week blocks and release after

final lecture of the week• Tiered difficulty of questions• Summative Exam Online (end of module MCQ exam)

Progress cont….

• 94% of students recommend online self-assessments for all level 1 modules

• 76% of students agree that SPRInTA assessments have aided their learning

• Significant improvement in summative MCQ exam results from previous year (p<0.01, t-test)

• Significant positive correlation between the number of formative assessments a student completes and their summative MCQ exam score (p<0.05, t-test)

• Significant improvement seen in the summer essay based exam (p<0.05, t-test)

Student Engagement-Pilot

47 31 76 73 140 166

2879

1279

0

500

1000

1500

2000

2500

3000

3500

1 2 3 4 5 6 7 8

Week of Formative Assessment

Nu

mb

er

of

Co

mp

lete

d

Fo

rma

tiv

e A

ss

es

sm

en

ts

Fig 1: Number of formative assessments completed per week in the BS155 pilot

Strategies for Engagement

• Publicity: Key rings, alerts

• Feedback: Removal of correct answer

• Release dates: Weekly

• Release pattern: o Module A-Questions closed after two weekso Module B-Open for the duration of the term

Results

• Engagement still high 81.4%

• Significant improvement in exam results when compared to previous years (p<0.01)

• Survey demonstrated 97% of thought the questions were well publicised (previously 73%)

• 76% of students found the PowerPoint slide useful

Results cont……

• No change in engagement

• Completed assessments in week leading up to exam:

o Pilot- 88.6%

o Module A- 68.2% (2 week release pattern)

o Module B- 56%

• The change in scheduling was not popular with the students

Conclusion

• Students like freedom of choice with regards to study time

• Students appear to take a strategic approach to learning/revision

• Further work is needed with regard to engaging students throughout the term:

o Summative componento Rankingo Just in time Teaching

QUESTIONS?

Contact: E-mail: sprinta@essex.ac.uk

Tel: (87)4369

Web: http://www.essex.ac.uk/sprinta/

Presentation can be found at: http://www.essex.ac.uk/sprinta/publications.htm