computer aided assessment (caa) for mathematics
DESCRIPTION
Chris Sangwin & Simon HammondTRANSCRIPT
Computer Aided Assessment(CAA) for mathematics
Chris Sangwin & Simon Hammond
Copyright c©Last Revision Date: June 1, 2009
2
Introduction.
... NOT multiple choice questions ...
• Computer aided assessment (CAA)
• CAA with computer algebra
• Practical issuesImplementations
• Pedagogical issues
• Future directions
3
JEM - Joining Educational Mathematics
eContentPlus Thematic Network
http://jem-thematic.net/
Founder members (15):
Universitat Politecnica de Catalunya, Helsingin Yliopisto, Tech-nical University, Jacobs University, Universiteit van Amster-dam, University of Birmingham, FernUniversitt Hagen, Mathsfor More, NAG Ltd, Liguori Editore, ISN Oldenburg GmbH,RWTH Aachen University, Univ. Nacional de Educacin a Dis-tancia, Universitat Oberta de Catalunya, Universidade de Lis-boa.
4
Use of objective tests
Consider the following question:
Example question 1Determine the following integral:∫
cos(x) sin(2x)dx.
As a multiple choice question:
◦ (2/3) cos3(x) + C◦ −(2/3) cos(x) + (2/3) sin3(x) + C◦ −(2/3) cos(x) + (1/3) sin(x) sin(2x) + C◦ Don’t know.
How do we know the students don’t differentiate thecandidate solutions to check?
5
Computer algebra marking
Computer algebra systems can be used to mark work.
This checks for algebraic equivalence.
(x + 1)2 ≡ x2 + 2x + 1
Useful for marking many routine problems.
6
Fundamental idea
if simplify(sa-ta) = 0 thenmark := 1 else mark := 0
7
STACK
System overview
The STACK system:
• internet based CAA system,
• uses very simpleMaxima (computer algebra), andLATEX (type setting)
• All components open source (e.g. GPL).
9
In learning and teaching
We are assessing a student provided answer.
This is an objective test.
This is
• not Multiple Choice Question;
• not string/regex match.
Other tests for the form of an answer.
10
Input of mathematics
This is a fundamental but unsolved problem.
There are a number of options
1. Strict CAS syntax. eg. 2*(x-1)*(x+1)
2. “informal” linear text syntax. eg. 2(x-1)(x+1)
x(t-1) ?
3. Graphical input tool. eg. equation editor.
4. (Pen-based input ?)
5. (Geometry applet ?)
Not all groups of students are equal.
11
Syntax innovations
Difficult to achieve!
Babbage 1830’s
“a profusion of notations [...] which threaten, if not duly cor-rected, to multiply our difficulties instead of promoting our progress”Babbage, C. (1827)
sin2(x) sin−1(x)
sin sin x = sin2 x
(composition)
12
Structure in random problem sets
In practice, the numbers often do not matter.
Tuckey, C. O., Examples in Algebra, Bell & Sons, London, (1904)
Too much randomization destroys structure.
An underlying question space.
13
Workshop task
Option A:Context: end of first calculus course. (Age 18)
Write 6 questions which test whether a student can differentiateelementary functions.
E.g. Differentiate cos(3x) with respect to x.
Option B:Context: age 11.
Write 6 questions which test whether a student can add frac-tions.
14
Randomization
1. What could you randomize?
2. What would you randomize?
3. What are some likely incorrect answers?
4. What feedback would you like to provide?
... with a view to implementing these questions live.
15
Issues
• Well-posed questions.
• Fair questions.
• Structure in question sets.Schemes of work, vs isolated questions.
• Algebraic form of answers as a goal.
16
Feedback
One third of feedback interventions decreased performance.
Kluger, A. N. and DeNisi, A., Psychological Bulletin (1996).
The nature of feedback determines its effectiveness.
17
Processing answers
Test for algebraic equivalence if simplify(sa-ta) = 0 thenmark := 1 else mark := 0
Using mainstream CAS
• Get a lot very quickly,Great for calculus and beyond.
• Elementary algebra can be a problem.
Maxima seems to be more suitable than most.
18
Every CAS is different!
Input Maple Maxima Axiom
(numbers)0.5-1/2 0.0 0.0 0.04^(1/2)
√4 2 2
4^(-1/2) 14
√4 1
212
-4^(1/2)√−4 2i 2
√−1
sqrt(-4) 2i 2i 2√−1
(indices)a^n*b*a^m anbam an+mb baman
(a^(1/2))^2 a a a(a^2)^(1/2)
√a2 |a|
√a2
(collecting terms)1+x^2-2*x x2 − 2x + 1 x2 − 2x + 1 x2 − 2x + 1
x/3+1.5*x+1/3 1.833x + .333 · · · 1.833x + 13
1.833x + 0.333 · · ·3*x/4+x/12 5
6x 5x
656x
3/(4*x)+1/(12*x) 56
1x
56x
56x
19
Input Maple Maxima Axiom
(brackets)
-1*(x+3) −x− 3 −x− 3 −x− 3
2*(x+3) 2x + 6 2(x + 3) 2x + 6
(2*x-1)/5+(x+3)/2 910
x + 1310
2x−15
+ x+32
910
x + 1310
(x-1)^3/(x-1) (x− 1)2 (x− 1)2 x2 − 2x + 1
(x^2-2*x+1)/(x-1) x2−2x+1x−1
x2−2x+1x−1
x− 1
(9*x^2+3*x)/(3*x) 13
9x2+3xx
9x2+3x3x
3x + 1
(other)
log(x^2) ln(x2
)2 log(x) log
(x2
)log(x^y) ln (xy) y log(x) log (xy)
log(exp(x)) ln (ex) x x
exp(log(x)) x x x
cos(-x) cos(x) cos(x) cos(x)
20
Issue: technical problems
• Mixed data types in polynomialsx/3 + 0.5?
• Unary minus (no simplification).
− 11− x
,−1
1− x, or
1x− 1
.
• Display,1. Implicit multiplication, (xy, x · y, x× y)2. i vs j,3.√
x vs x12 .
21
Language
Do we have a way to talk about these fine details?
Unhelpful phrases:
• simplify,e.g.221
= 4 or 221000= · · ·?
• “move over”
22
Checking for properties
To mark
Example question 2Give an odd function.
1. calculate f(x) + f(−x),2. simplify,3. check equality to zero.
23
Creating examples/instances
Some questions ask for examples of objects.
They require higher level thinking.
Such questions are rare. (11.5 questions from 486 ≈ 2.4%)Pointon and Sangwin, 2003
Perhaps because they are time consuming to mark.
STACK may mark some questions of this style.
Exemplar questions
24
Students’ answers
Students show great variety in their answer, and method.
For example, 190 students were asked for two functions thatsatisfy f ′(1) = 0.
Their answers were marked automatically.
25
The students (N = 190) gave 93 ‘different’ answers.
1st Answer Freq
uenc
y
2nd Answer Freq
uenc
y
x2 − 2x 45 x3 − 3x 29x2
2 − x 31 x2 − 2x 10x3
3 − x 11 x3
3 − x 9x2 − 2x + 1 7 (x− 1)2 8x2 − 2x + 3 7 x4
4 − x 8(x− 1)2 5 x4 − 4x 52x2 − 4x 5 ex−1 − x 1x3
3 −x2
2 5 ex−1 + e−x+1 10 4 ln(x)− x 1
26
Two strategies emerged:
JL: Ok, just take the parabola and shift it one.· · ·B: I said, x− 1 = 0, then integrated it.
27
These problems can be used to generate (short) discussions.
• sorting the data,
• methods used,
• ‘exotic’ examples.
f1(x) = 0, f2(x) = |x|(x− 2), f3(x) = e−1
(x−1)2 .
28
Automatic feedback
Sophisticated automatic feedback may be provided bycomputer algebra systems.
This
• is immediate,
• is based on properties of students’ answers,
• could be positive and encouraging,
• may be based on common mistakes,
• may be based on common misconceptions.
29
Common misconceptions
Computer algebra can also test for a type of incorrect answer.
Misconceptions may be identified by
• educational research,
• previous teaching experience,
• examining answers from previous students
30
Odd functions
On examining the odd functions given by students,
the majority of coefficients ( 6= 1) are odd,eg
3x5, 5x7, 7x5 − 3x.
Students’ concept image of an odd function requires odd coeffi-cients.
Furthermore, f(x) = 0 is odd, but was absent.
31
Functions that are odd and even.
When asked for a function that was both odd and even
35% gave the correct answer (eventually),35% failed to answer the question.
Incorrect answers revealed that 24% of the students added anodd and even function.
Examples include
x + x2, x2 + x3, x5 − x6.
The computer algebra system can test for these misconceptions.
32
Student feedback
What do you like about the system? Did you have any difficul-ties? If so please describe them.
Feedback & partial credit
i like the way that you are given credit if your an-swer is partially correct and also given guidance onachieving the full mark for that question.
I like the fact that feedback is immediate, but I donot like the fact that if I get an answer wrong I donot know where in my working I have made the error
33
Random questions
The questions are of the same style and want thesame things but they are subtly different which meansyou can talk to a friend about a certain question butthey cannot do it for you. You have to work it allout for yourself which is good.
Syntax problems
I feel the aim system is reasonably fair, however ihave lost a lot of marks in quiz 3 for simple syntaxerrors
34
Give me an example...
Recognising the turning points of the functions pro-duced in question 2 was impressive, as there are alot of functions with stationary points at x=1 andit would be difficult to simply input all possibilitiesto be recognised as answers.
35
Authoring questions
In authoring, there is tension:
1. Ability to use all features of CAS.
2. Ease of writing questions.Not making question authors into programmers.
36
Conclusion
Some important questions
• For what purposes is this tool useful?
• What properties do we want?– Not “looks correct”.– Not “select the correct answer”.
• What feedback should we give?