the use of caa for formative and summative assessment: student views and outcomes. sandra pattinson...
TRANSCRIPT
The use of CAA for formative and summative assessment: student views and outcomes.
Sandra PattinsonAnimal Production and Science Group
Harper Adams University CollegeNewportShropshire
TF10 8NB.e-mail: [email protected]
1.0 Introduction
1.1 Farm Animal Production
Compulsory core module (Level 1).
Non-agricultural students - high failure rate.
E.g. HND Agrifood Marketing and Business Studies + HND Countryside Management.Initial failure rate
2001 - 02 25 %2002 - 03 28 %
1.0 Introduction
1.2 Aim of study
To examine the effects of the introduction of formative CAA on the results for the module.
The module is taught over two semesters and results are presented for the first semester only.
2.0 Methods
26 students on Farm Animal Production module.
2.1 Odd semester:
Subjects taught: beef, sheep, poultry.
Methods: lectures, tutorials, farm classes.
Optional: self-assessment tutorials usingQuestionMark Perception.
2.0 Methods
2.2 Self-assessment tests
Each test consisted of 10 multiple-choice questions selected at random from database.
Size of databaseBeef production 67 questionsSheep production 58 questions
Feedback provided for each question.
2.0 Methods
2.3 Summative assessment
29 sheep production questions29 beef production questions40 poultry production questions2 general questions
Questions not randomly selected.
N.B. Negative marking used in tutorials andin summative examination
2.0 Methods
2.4 Questionnaire
Presented to students at the start ofthe subsequent semester(prior to publication of exam marks).
Aim - to assess the students’ views onthe use of the self-assessment tutorials.
3.0 Results
3.1 Examination results
2003 - 04 (tutorials available)mean mark 41.5 %pass rate 46 %
2002 - 03 (tutorials not available)mean mark 34.8 %pass rate 25 %
3.0 Results
3.1 Examination results
Use of tutorials Number of Mean students mark
0 times 10 (38%) 32.31 - 9 times 9 (35%) 37.610+ times 7 (27%) 59.6
3.0 Results
3.1 Examination results
Correlations
Use of tutorials v. exam markSheep rs = 0.59 P < 0.01Beef rs = 0.69 P < 0.01
Mark v. poultry markBeef r = 0.52 P < 0.01Sheep r = 0.36 P < 0.1
3.0 Results
3.2 Questionnaire
3.2.1 How often did you use the self-assessment tests?
Very often, often, occasionally, rarely,never.
3.2.2 As a method of learning I enjoyed the tests
7
64
29
0 00
10
20
30
40
50
60
70
Strongly
agree
Agree Neither
agree nor
disagree
Disagree Strongly
disagree
% o
f st
uden
ts
3.2.3 The wording of the questions was unclear
0
14
21
64
00
10
20
30
40
50
60
70
Strongly
agree
Agree Neither
agree nor
disagree
Disagree Strongly
disagree
% o
f st
uden
ts
3.2.4 The assessment questions were closely related to the content of the module
29
57
14
0 00
10
20
30
40
50
60
70
Strongly
agree
Agree Neither
agree nor
disagree
Disagree Strongly
disagree
% o
f st
uden
ts
3.2.5 I used my lecture notes whilst completing the self-assessment tests
7
0
21
29
43
0
10
20
30
40
50
60
70
Very of ten Of ten Occasionally Rarely Never
% o
f st
uden
ts
3.2.6 The self-assessment tests helped me to understand the lecture notes
0
50
21
29
00
10
20
30
40
50
60
70
Strongly
agree
Agree Neither
agree nor
disagree
Disagree Strongly
disagree
% o
f st
uden
ts
3.2.7 The marks helped me to assess how I was doing on the module
14
57
21
7
00
10
20
30
40
50
60
70
Strongly
agree
Agree Neither
agree nor
disagree
Disagree Strongly
disagree
% o
f st
uden
ts
3.2.8 The feedback given if you get the question wrong is useful
7
29 29 29
00
10
20
30
40
50
60
70
Strongly
agree
Agree Neither
agree nor
disagree
Disagree Strongly
disagree
% o
f st
uden
ts
3.2.9 Doing the tests made it easier to study for the exams
7
57
21
14
00
10
20
30
40
50
60
70
Strongly
agree
Agree Neither
agree nor
disagree
Disagree Strongly
disagree
% o
f st
uden
ts
3.0 Results
3.2 Questionnaire
3.2.10 Free response
A) Why did you use/not use the self-assessment tests
Not used Forgot
Used To see how much they knewTo practice for exams
3.2.10 Free response
B) What were the three best things about theself-assessment tests?
Easy to useDetailed answers (feedback)Similar format to exam
C) Give three suggested improvements.
Remove negative markingRemote accessProvide the correct answer in the feedbackMore questions and more feedback
3.2.10 Free response
D) Any other comments
They were a good way of learning and veryuseful.
It was very helpful. Thank you.
4.0 Conclusions
Students who sat optional practice tests had higher marks in subsequent summative tests.This is in agreement with other authors e.g. Sly and Rennie (1999), Charman (1999)Students who scored highest on the beef and sheep sections of the summative assessment also had the highest marks in the poultry section.
4.0 Conclusions
Questionnaire responses indicate that formost students the tutorials provided a positive learning experience.
Some students requested that the correct answer should be provided if the question was answered incorrectly
Deep v. surface approach to learning
5.0 Future direction
5.1 Current semesterPig and dairy production self-
assessmenttutorialsRevision test at start of lecture
5.2 Questionnaire
Questions relating toFeedbackTest structure
13
19
13
38
19
0
10
20
30
40
Strongly
agree
Agree Neither
agree nor
disagree
Disagree Strongly
disagree
% o
f st
uden
tsQuestion: The feedback should state which was
the correct answer.
5.0 Future direction
5.3 Question analysis - to identify problem areas of the moduleor poorly written questions
5.4 Remote access