effective strategies for online testing in large classes · pdf filein math2019 online tests...
TRANSCRIPT
E�ective strategies for online testing in large classes
Jonathan Kress
School of Mathematics and StatisticsUniversity of New South Wales, Australia
Maple TA User Summit 2014
Jonathan Kress E�ective strategies for online testing in large classes
UNSW Australia and me
University of New South Wales
Sydney Australia
54000 students
Large Science and Engineering Faculties
25% international students
UNSW in top 8 of 35 Australian Universities
School of Mathematics and Statistics teaches approximately 3000�rst year students
Jonathan Kress
Senior lecturer in the School of Mathematics and Statistics
At UNSW since 2001
Creating and administering online tests since 2004
Jonathan Kress E�ective strategies for online testing in large classes
Courses using Maple TA at UNSWSchool of Mathematics and Statistics
MATH1131/1141 Science and Engineering (1700)assumed knowledge / class test preparation / online Maple /invigilated Maple lab
MATH1151 Actuarial Stuides (200)class test preparation / online Matlab
MATH1011 Lower level �rst year course (100)weekly tutorial preparation, online Maple
MATH1031 Maths for Life Sciences (400)class test preparation, Maple
MATH1041 Stats for Life Sciences (400)online quizzes, invigilated R or Excel lab
Higher Year Stats/Numerical Methods/Engineering Maths (2000)intro to matlab, various online tests
Faculty of Engineering
Higher year undergraduate and Masters Engineering (1000)mostly longer assignments
Jonathan Kress E�ective strategies for online testing in large classes
Why use online testing and Maple TA?
Regular testing encourages students to keep up to date
Provides immediate feedback, anytime, anywhere
Reduces labour on routine marking(What should be done with that saving?)
Allows repeated attempts according to student need
Personalisation of learning
encourages collaboration that isn't just copying
Analysis of student responses can be used to improve teaching
Very �exible algorithmic question generation and grading
It lacks features we don't want during invigilated testseg, messaging or access to course materials
Plan to use adaptive questions in the future
Jonathan Kress E�ective strategies for online testing in large classes
Di�erent scenarios for use of Maple TA
Formative � assessment for learning
Assumed knowledge quiz (placement test)for students in MATH1131 who should be in MATH1011
Class test preparationin the week preceding a written class test
Weekly tutorial preparationa short online test preceding each week's tutorial
Progressive (mastery) testsmust pass each test to access the next test
Practice followed by actual testunlimited practice but limited number of attempts count
Practice testsno marks awarded, feedback only
Summative � assessment of learning
Invigilated lab teststraditional exam conditions but Maple TA graded
Jonathan Kress E�ective strategies for online testing in large classes
How students view Maple TA
Students expect to be rewarded with marks.
Students expect things to work without having to read instructions.
Student confusion about the naming of �Maple TA� and �Maple�lead to an increase in student use of Maple!
From a MATH1131 survey in 2011:
Jonathan Kress E�ective strategies for online testing in large classes
Student comments (2013 and 2014)Feedback from quizzes was very helpful.
Constant tests are good.
Online tests are mostly just annoying and take up valuable time.
(Course could be improved by) more online questions. . .
The online tests and Maple tests instead of assignments were the bestfeatures for this course.
The online tests helps in getting a better grip over the subject.
(Best features of the course) on-line tests
I found the online tests provided a good revision tool as they providedimmediate feedback which I could use to improve.
Should have more smaller online tests on maple ta so that it forcesstudents to learn and be up to date with the course as it moves quickly.
Regular online and class tests prompt us to be continually up to date. . .
The online quizzes kind of force you to learn how to do each question, andsince you have 5 tries you can work on a question until you get it right.
More weighting on online tests. The time used to complete the onlinetasks was not appropriate in comparison to the percentage it makes up.
Online quizzes were well set out. It was useful to have several attempts.
Jonathan Kress E�ective strategies for online testing in large classes
Engagement
Do students engage with tests?
In MATH1131 online tests are worth approximately 1% each.Approximately 90% of students attempt each test.
In MATH2019 online tests do not count towards �nal grade.Between 1% and 10% of students attempt the tests.
Incentives
Marks count towards �nal grade0.5% per 30 minute online test seems su�cient!
Immediate feedback
Flexibility
allow some tests to be missed or failedeg only count best 8 of 12
allow repeated attemptsonly count best markallow plenty of time
More and smaller tests to avoid unnecessary repetition
Jonathan Kress E�ective strategies for online testing in large classes
Set and forget
Our main current use of Maple TA is to provide practice questions andfeedback to students with minimal intervention.
Provide written and screen capture video instructions.
Link to Maple TA provided next to link to instructions.
Maple TA class homepage tells students to read the instructions.
Compulsory �Using Maple TA� test must be completed �rst.
Jonathan Kress E�ective strategies for online testing in large classes
�Using Maple TA�
Jonathan Kress E�ective strategies for online testing in large classes
�Using Maple TA�
Jonathan Kress E�ective strategies for online testing in large classes
Set and forgetAllow multiple attempts.�Revision only� versions of tests available after deadline.Course outline states no extenstions or additional attempts allowed.Only best attempt counts.Allow some tests to be missed without penalty.Questions contain embedded instructions.
Jonathan Kress E�ective strategies for online testing in large classes
Tutorial preparation tests in MATH1011
Encourage regular preparationfor tutorials.
MATH1011 has separatealgebra and calculuscomponents.
Weekly tutorial preparationtests for the calculus introducedin 2004.
Amnesty for missed tests inweek 5 of semester.
Course survey commentsrequested similar tests foralgebra.
Evidence of increase in averagescore for �nal exam questionsrelated to online tutorial prepquestions.
Jonathan Kress E�ective strategies for online testing in large classes
Tutorial preparation tests in MATH1011
Jonathan Kress E�ective strategies for online testing in large classes
Class test preparation
Provide feedback to help students prepare for class tests.
Initially allowed 3 attempts but spent too much time dealing withspecial pleadings for extra attempts so increased to 5 attempts.
Jonathan Kress E�ective strategies for online testing in large classes
Invigilated tests
Maple component of MATH1131 is assessed with invigilated lab test
students book time for test on Moodlestudent ID cards are checkedcomputers (112) have exam mode with no internet access
Before 2010
paper based teststudent Maple worksheets collected, printed then marked by handand returned in tutorial classvery labour intensivestudents had no feedback prior to test
Since 2010
Maple TA testtest is available for unlimited practice on Maple TA before lab testtest restricted to lab IP rangeeasy immediate grading once all tests have �nished
Since 2014
MATH1041 Stats for Life Sciences has an invigilated lab test partlygraded by Maple TA.
Jonathan Kress E�ective strategies for online testing in large classes
Maple Lab Test
Jonathan Kress E�ective strategies for online testing in large classes
Allowing for special cases
Although we say no extenstions or additional attempts will begranted, in reality there will be special cases.
We use �revision only� versions for special cases.
�Revision only� tests also depend on �Declaration� and �Using MapleTA� tests.
�Revision only� test become available as soon as `for marks' deadlinehas passed.
Jonathan Kress E�ective strategies for online testing in large classes
Summary
Expect bugs in your questions.
Expect students to do uncontrolled tests collaboratively.
Students won't attempt �optional� tests.
Assume students won't read the instructions.
Assume students will start the test when they are not prepared.
But if you just want students to learn, the solution to all of this is. . .
don't worry!
O�er enough marks that the students care but you don't.
Provide lots of ways for lost marks to be recovered.
Have a strict non-intervention policy.
Jonathan Kress E�ective strategies for online testing in large classes
Summary
Expect bugs in your questions.
Expect students to do uncontrolled tests collaboratively.
Students won't attempt �optional� tests.
Assume students won't read the instructions.
Assume students will start the test when they are not prepared.
But if you just want students to learn, the solution to all of this is. . .
don't worry!
O�er enough marks that the students care but you don't.
Provide lots of ways for lost marks to be recovered.
Have a strict non-intervention policy.
Jonathan Kress E�ective strategies for online testing in large classes