on e-assessment
DESCRIPTION
Assessment is an integral part of learning, and yet very little attention is paid to e-assessment when it comes to e-learning. A significant effort has been directed towards developing high quality question items that take advantage of the new medium, and yet there appears to be some resistance to embrace this new medium. There have been interesting developments in the field of e-assessment. Symbolic systems such as Maple T.A. have matured sufficiently and provide a range of question items. Another significant development is in the area of short answer question items that are assessed that are tolerant to spelling and grammatical mistakes. The advent of digital pens that capture audio and written content opens new possibilities. We can now begin to address diagrammatic reasoning more elegantly. In this paper, we make an attempt to review the state-of-play in e-assessment discussing its pros and cons. Starting with a review of technology landscape, we move on to a discussion of some of the perceived concerns for embracing e-assessment both as a diagnostic, formative and summative assessment tool. The paper concludes with a discussion on timely intelligent feedback which still remains a big challenge, and the opportunities that exists to mediate this issueTRANSCRIPT
Contents
• Background, context• Why symbolic systems?• Examples • Analysis of responses• Some guidelines on design of question
items• Opportunities
Why symbolic systems?
• Offer a variety of question items
• Students input can be assessed by checking for symbolic equivalence(e.g., x^2+2xy is same as 2yx+x^2)
• Challenges – mathematical input
22 yx
yx
22 yx
yx
Maths input
A diagnostic test (6 Jan 2009)
• A 45 minutes test consisting of 15 questions
Analysisd-value = (F1/n1) – (F2/n2)
Measures discriminating power of an item
Analysis29%
Responses for Q14v = u + a t
(v-u)/a
36%
v-u/a
4%
v/(u+a)
2%
(u+a)/v
3%
u-v/a
1%
((v)-(u))/a
1%
(v-u)/a=t
1%
User Input for Q15
R1+R2 orr1 + r2
13%
(R1R2)/(R1+R2) or
(R1R2)/(R2+R1)
5%
3
2%
1/((1/R1)+(1/R2))
5%(r1r2)/r2+r1
1% (r1xr2)(r2+r1)1%
Feedback
q11 q15
Correct 29 13
Wrong 71 87
cq11_cq15 9%
wq11_wq15 67%
cq11_wq15 19%
wq11_cq15 4%
• for the cohort
• individual
Skills/Concepts Map
arithmeti
c algebraworded
problem linear eqfractiona
l arith indicesalgebraic fractions
formula manipula
tion
1 1 0 0 0 0 0 0 0
2 1 0 0 0 0 0 0 0
3 1 1 0 0 0 1 0 1
4 1 1 0 0 0 1 0 1
5 1 1 0 0 1 1 1 0
6 1 1 0 0 0 1 1 0
7 1 1 0 0 1 1 1 0
8 1 1 0 0 1 1 1 0
9 1 1 0 1 0 0 0 1
10 1 0 1 0 1 1 0 0
11 1 1 0 0 1 0 1 1
12 0 0 1 0 0 0 0 1
13 1 0 1 0 0 0 0 1
14 0 1 0 0 0 0 0 1
15 0 1 0 0 0 0 1 1
Performance of a student …
1 Arithmetic
2 Algebra
3 Worded problems
4 Linear equations
5 Fractional arithmetic
6 Indices
7 Algebraic fractions
8 Formula manipulation
Opportunities
• Digital pens• Intelligent Feedback• Mathematical input• Diagramatic input
Summary
• Free text input, symbolic input offer a rich variety of information that can be utilised to identify generic issues
• Mathematical input still remains a challenge
• Symbolic systems such as Maple T.A are best used for drill exercises. Students need to be trained adequately when used for high stakes exams