Exploring the possibilities and constraints of online assessment to advance the DHET’s Open Learning agenda from a social justice perspective

(Case 13)

DHET Online Webinar16 September 2021

Cheng-Wen Huang

Res

earc

her b

ackg

roun

d

Position:Senior LecturerAcademic Staff DevelopmentCentre for Innovation in Teaching and Learning (CILT)University of Cape Town

Research Background:Multimodal social semiotics, argumentation, academic literacies, digital literacies, open learning, assessment, social justice

Cheng-Wen Huang

Current work:1) University Capacity Development Grant (UCDG)

Project: Enabling Student Success Through Transformed Programmes of Assessment

2) Learning Platforms Update Project – Course Evaluations

More space on slide

DHET’s Open Learning Agenda

The Open Learning Unit Tshwane North TVET College

increasing access improving success

Trig

ger f

or s

tudy

social justice → ‘parity of participation’ (Nancy Fraser)

Res

earc

h Q

uest

ion

Open learning “is driven by a concern for social justice” (DHET, 2017, p. 412).

What are the possibilities and constraints of online assessment to advance

the DHET’s Open Learning Agenda from a social justice perspective?

What are the possibilities and constraints of online assessment to enable

‘parity of participation’ from an economic, cultural and political

perspective?

Polic

ies

& P

rior R

esea

rch

increasing access

improving success

removing barriers to access

providing flexibility of provision

improving quality

(translates into increased enrolment)

(translates into improved throughput, success rates and

employability)

DHET’s Draft Open Learning Policy Framework (2017)

• Open learning as an education model that will complement the traditional (campus-based) model of education through diverse modes of provision

• Encourages the use of cost-effective modalities

Online AssessmentPo

licie

s &

Prio

r Res

earc

h

“the end-to-end electronic assessment processes where ICT is used for the presentation of assessment activity, and the recording of responses”

(JISC, 2007, p. 6)

Design Delivery Marking Record-KeepingReporting (Feedback)

Scully, D. (2017) Constructing multiple-

choice items to measure higher-order thinking

Ellis, C. (2013)Broadening the scope and

increasing the usefulness of learning analytics: The case for

assessment analytics

Bahar, M., & Asil, M. (2018) Attitude towards e-

assessment: influence of gender, computer usage

and level of education

Hepplestone et al. (2011)Using technology to encourage student

engagement with feedback: a literature review

Whitelock, D. (2006) Electronic assessment:

marking, monitoring and mediating

Polic

ies

& P

rior R

esea

rch Online Assessment

Jordan (2013)

Nicol (2006)

JISC (2007)

Herd & Mead Richardson (2016)

Automated marking• Saves time and human labor needed for marking• Immediate feedback• Reliability in producing consistent results Possibilities

Costs• Maintaining a Learning Management System• Technical and pedagogic support for staff and

students Constraints

Automated feedback• Allows for self-assessment• Allows for revision Possibilities

Country’s ICT infrastructure conditions• Connectivity challenges• Bandwidth costs Constraints

Met

hodo

logy

Site of study:

The Open Learning UnitTshwane North TVET College

Established in 20192020 enrolment: 547 students

• Qualitative case study

• 4 staff (in managerial or lecturer roles) - online interviews / email exchanges

• Data analysis

Met

hodo

logy

� Conducted a thematic analysis of :- concepts related to open learning- issues raised in relation to economics, culture and politics

• 4 students - WhatsApp exchanges

� Developed a coding framework

Find

ings

Economic DimensionFlexibility in time and place

“I’m working and I’m doing night shifts. I can do my studies at night at work”

Student:

Save costs

Enable non-traditional students to obtain a

qualification

“I’m going to give you a classic example of a student who works for the SABC. That student was supposed to go and shoot in Durban on the same day we were writing a mini exam. The student was able to write the mini exam, online, for that period […] imagine if the student was to fly from Durban to come here, just to write a paper that was only going to take one hour, and then fly back. So, it helps, especially for those who are working. That they can be able to do the assessment wherever they are, at whatever time”

Staff:

increasing access

Find

ings

3.

Staff:“You also find that sometimes there is no electricity. The student wants to log on – there is load shedding. Then you find yourself having to do the same task over and over again. So, it gives lecturers added responsibility, setting three or five tasks, because the student has got evidence, that I wanted to do the assessment, but there was load shedding”

“For instance, if they take it on their phones, and there is a call coming in, then it will kick them out of the assessment. And, it will show that they have done the assessment already”

Staff:

Accessibility issues

Data

Connectivity

Electricity

Device

Economic Dimension

Find

ings

Cultural Dimension

“We can set up the learning management system in such a way, that if a student doesn’t get maybe 80 percent with the self-assessment, then he/she can’t continue with the next lesson. So, you set it up, you must go back again, and go through the lesson again, and take the self-assessment. And, when it gets maybe 85 percent, then it opens the next lesson”

Staff:

‘assessment for learning’

Multiple attempts

Self-assessment

Cost-effective

(automation)

improving success

Find

ings

Digital literacy

Academic integrity / cheating

Staff:

“At the moment, we don’t have those facilities that are able to monitor if the learner is the learner who is actually writing or if somebody is writing on their behalf. That could be a challenge. That could you give you false results or false evidence. At the end of the day, if a learner gets 80%, is it really the learner, or is it somebody else”

Cultural Dimension

Find

ings

Political Dimension

“I think policy needs to be developed, not just do a copy and paste of full-time. You need to look at online learning as something different, completely different from face-to-face, then I think we can get it right. Because, what’s happening right now is that, you would do an online assessment, but the policy says, they want hard copies, so you still have to go back and print, and have a hard copy of the online”

Staff:

Lack of policy

Moderation

Number of assessment questions

Rec

omm

enda

tions

At the national level

• Need for a policy for online assessment

• Expansion of network infrastructure

e.g. connecting TVET colleges to the South African National Research Network (SANReN)

At the institutional level

• Assess institution needs (consider the cost of installing and maintaining a LMS etc )

• Experiment

• Would require a close cooperation between course lecturers and ICT units

At the individual (lecturer) level

• Share knowledge by making examples available

Ref

eren

ces

Bahar, M., & Asil, M. (2018). Attitude towards e-assessment: influence of gender, computer usage and level of education.

Open Learning: The Journal of Open, Distance and e-Learning, 33(3), 221-237.

Department of Higher Education and Training (DHET), (2017). Open Learning Policy Framework Post-School Education and

Training. Government Gazette.

Ellis, C. (2013). Broadening the scope and increasing the usefulness of learning analytics: The case for assessment

analytics. British Journal of Educational Technology, 44(4), 662-664.

Fraser, N. (1995). From redistribution to recognition? Dilemmas of justice in a “postsocialist” age. New left review, 68-68.

Hepplestone, S., Holden, G., Irwin, B., Parkin, H. J., & Thorpe, L. (2011). Using technology to encourage student engagement

with feedback: a literature review. Research in Learning Technology, 19(2), 117-127.

Herd, G., & Mead Richardson, A. (2016). World report on TVET—The promise and potential of ICT in TVET. Commonwealth of

Learning.

JISC (Joint Information Systems Committee). (2007). Effective Practice with e-Assessment: An overview of technologies,

policies and practice in further and higher education.

Jordan, S. (2013). E-assessment: Past, present and future. New Directions in the Teaching of Physical Sciences, 9(1), 87-106.

Nicol, D. (2006, December 3). Increasing success in first year courses: Assessment re-design, self-regulation and learning

technologies. In Proceedings of the 23rd annual ascilite conference.

Scully, D. (2017). Constructing multiple-choice items to measure higher-order thinking. Practical Assessment, Research, and

Evaluation, 22(4), 1-13.

Whitelock, D. (2006). Electronic assessment: marking, monitoring and mediating learning. International Journal of Learning

Technology, 2(2-3), 264-276.

● The DHET for the initiation and funding support.

● The mentors of the COOL project, especially, Emeritus Professor Cheryl Hodgkinson-

Williams and Dr Tabisa Mayisela for supervising and leading the project.

● The staff, particularly Dr Pieter Geel and Ms Henrietta Mongalo, and students

from Tshwane North TVET College’s Open Learning Unit whose inputs made the case study possible.

Ack

now

ledg

emen

tsWith thanks to:

Creative Commons 4.0 international license: Attribution

Huang, C.W. (2021). Case Study 13: Exploring the possibilities and limitations of online assessment to advance the DHET’s Open Learning Agenda from a social justice perspective. DHET Online Webinar, 16 September 2021.