using an adaptive e-learning curriculum to enhance … · using an adaptive e-learning curriculum...

Using an adaptive e-Learning curriculum to enhance Rhian Morgan; Sharon Bryan digital literacy: Challenges and opportunities [email protected]; [email protected]

AARE Converence 2016 – Melbourne, Victoria of 23

USING AN ADAPTIVE E-LEARNING CURRICULUM TO ENHANCE DIGITAL LITERACY:

CHALLENGES AND OPPORTUNITIES

Rhian Morgan*, Sharon Bryan*, Kathryn Meldrum, Bronwyn Mathiesen, Angela Hill.

James Cook University, Australia

Abstract:

Universities have established the need for digital literacies as key graduate

outcomes and must support students’ development in these areas (Lea, 2013;

Littlejohn, Beetham & McGill, 2012). In an era of widening participation, it

is erroneous to assume all students will enter the sector with the

technological skills required to support their studies. While some may be

“skilled technology users”, many will lack the digital competencies required

for academic success (Jisc, 2014). Students need support for developing

digital literacies, but explicit teaching may lead to frustration, as some are

forced to “learn” skills they possess, while others are lost in the crowd.

Adaptive learning technologies are particularly suited to addressing student

diversity, as they automatically adjust to individual student’s needs. This

paper documents the development of a digital literacies curriculum that uses

an adaptive e-Learning platform to prepare students for technology-rich

academic environments. The curriculum was co-developed by academics

and librarians and consists of lessons on information, data and media

literacies, and online identity management. The paper outlines the

pedagogical underpinnings of the lessons, alongside the challenges and

opportunities observed during development process, and reflects on the

impacts of adaptive technologies on teaching in blending learning

environments.

Keywords: Digital literacy, adaptive e-Learning, information literacy,

smart-sparrow, e-Learning, digital readiness

mailto:[email protected]




Introduction

Web 2.0 technologies have become so widespread in modern society that progression through

university studies is now increasingly dependent on students developing a range of information and

communications technology (ICT) skills alongside the specialist skills associated with their

disciplines. Students are expected to have a core level of digital literacy in order to be successful in

twenty-first century academic and work environments. However, with widening participation in

universities and colleges, it is important to take note of the cautions expressed by the Educating the

Net Generation report (Kennedy et al., 2009), which warned against an assumption that ‘digital

natives’ are entering the higher education sector with extensive existing digital literacies and the

capacity to use those literacies to support their academic endeavours. Even students who are highly

knowledgeable of ICTs “do not necessarily expect to use these technologies to support some

activities, including learning” (Kennedy et al., 2009, p.4). This paper describes the design of a digital

literacies curriculum, authored using proprietary adaptive e-Learning software, and outlines the

pedagogical, contextual and theoretical background behind the design. The aim of the paper is to

discuss the rationale of the project and position it within the context of the relevant literature, before

going on to outline an adaptive digital literacies curriculum that was undertaken as a collaborative

endeavour between teachers, academic support and library professionals. The project provided the

collaborators with several challenges and opportunities for curriculum and pedagogy development,

and the lessons learnt will inform future use of adaptive e-Learning.

Digital Literacy and Higher Education

Widening participation in higher education calls for differentiated instruction that responds to the

varying levels of technical experience, ability and knowledge our students exhibit – and this is

especially relevant to teaching digital literacies. Pedagogical research has ascertained that engagement

is a key determinant of success, as well as a means of reducing inequities among diverse cohorts





(Gabb, Milne & Cao, 2006; Kuh, Cruce, Shoup, Kinzie & Gonyea, 2008). Clearly, the provision of

equitable digital literacy instruction calls for deliberate curriculum design that acknowledges

engagement and differentiated instruction as key elements of academic success.

The US based organisation, EDUCAUSE, offers significant insight into the digital literacies of

students in their universities and colleges. In 2015, their annual Study of Students and IT surveyed the

digital literacy practices of over 50,000 students across 43 states and 11 counties (Dahlstrom, Brooks,

Grajek & Reeves, 2015). Key findings of the 2015 student survey included: students’ academic use of

technology is widespread, but not deep; although omnipresent in students’ lives, the leveraging of

technology as an engagement tool is still evolving; and students have a complex relationship with

technology – they recognise its value but still need guidance in order to use it in meaningful ways

(Dahlstrom, et al., 2015).

Universities are increasingly offering multiple modes of education to prospective students. However,

to enable students to fully take advantage of flexible learning and distance education, it is important to

support the development of digital literacies and online learning skills (Yang, Catterall & Davis,

2013). Benchmarks set by the New Media Consortium and EDUCAUSE Horizon reports (Johnson,

Becker & Hall, 2015) acknowledge the importance of training and institutional support for students

regarding the effective use of learning technologies. The notion that students require a core level of

digital literacy in order to succeed in university is becoming widely accepted, but a definition of what

actually constitutes digital literacy – or, rather, digital literacies – continues to develop (Kiili,

Mäkinen & Coiro, 2013).

The UK based Joint Information Systems Committee ([Jisc], 2011) have adopted a definition of

digital literacies as capabilities that support individuals to live, work and learn in a digital society.

This definition recognises that digital literacies are multidimensional – incorporating multiple

literacies and multiple aspects of literacy. Belshaw (2014), for example, put forward eight elements of





digital literacies: cultural, civic, cognitive, constructive, critical, communicative, confident, and

creative. Lankshear and Knobel (2003) had previously identified three dimensions of literacy:

operational, cultural and critical. Operational literacies included competency with tools and

procedures, while the cultural dimension concerned an individual’s ability to understand a text within

its cultural context. The critical dimension of this model regarded an individual’s awareness that

literacies are socially constructed; they include some values while excluding others. It has also been

remarked that digital literacies encompass a “plurality of understanding and skills” (Brown, 2014, p.

284), such as information, visual, technological and media literacies (Martin, 2006), hypermedia and

photovisual literacies (Aviran & Eshet-Alkalai, 2006) and games literacy (Schott & Selwyn, 2011).

Different frameworks and models for supporting students have emerged as investigations into the

nature of digital literacies have advanced (Buckingham, 2006; Eshet, 2012; Jisc, 2014; Jisc, 2015a;

The Open University, 2015).

The DigEuLit project (Martin, 2006) described a three stage model of digital literacies that include

digital competence, digital usage, and digital transformation. These stages describe how students

might use their competence, and then transform their use into creating and innovating using digital

tools. Belshaw (2014), however, warns that the development of digital literacies is not a linear

‘process’, but rather subject to contextual variation. Within the UK’s higher education context, Jisc

have been instrumental in developing a conceptual framework (Jisc, 2015a) that describes the digital

literacies necessary for student success. Jisc’s six capabilities model of digital literacies (illustrated in

figure one) identifies the range of digital literacies that university and college students need to master.

These include: ICT literacy; information literacy; data literacy; media literacy; digital learning and

self-development; and identity and well-being. These models – particulary the Jisc six capabilities

model – were used to form a guiding framework for the development of a suite of adaptive e Learning

lessons, which were designed to facilitate the embedding of digital literacies instruction into existing

curricula at a university, as outlined below.

Adaptive e-Learning Platforms (AeLPs)





Adaptive e-Learning Platforms (AeLPs) use interactive activities with multiple pathways in order to

redirect students to lesson material based on their demonstrated understanding of the content. This

enables instructors to provide appropriate remediation as and where necessary, rather than making all

of the students progress through lesson content at the same rate. The multiple pathways should also

allow for students who have a good grasp of the content to move through the lessons at a more

agreeable pace, lessening the potential for frustration at both ends of the spectrum. The interactivity of

AeLPs enables a shift away from static (or linear) presentations of information by adapting

instruction, learning activities and feedback to the individual needs of the learners. This, ideally,

facilitates a holistic learning experience in which active learning, explicit instruction and evaluation

combine to enable success among diverse learners (Mampadi, Chen, Ghinea & Chen, 2011; Murray &

Pérez, 2015; Wauters, Desmet & Van den Noortgate, 2010). Research indicates that personalised

content in online education can improve student performance (Gangadhara Prusty, Vrcelj, McCarthy,

Ojeda & Gardener, 2013). With widening participation, institutions need to cater not only to students

who are comfortable with using technology in a variety of contexts, but also to younger students who

are not necessarily “digital natives”, students from rural/remote areas who might not have had access

to a wide range of technologies, or mature age students who may be uncertain about the technologies

used for learning (Dawson et. al., 2013). AeLPs enable us to cater to this diverse range of students.

The capacity of AeLPs to respond to the individual needs of students means that such technologies are

well-suited to areas such as digital literacies, where students exhibit varying levels of pre-existing

ability, and online or blended learning environments where personal engagement with learners is not

always possible. This suitability derives from the platforms’ provision of formative feedback in

response to individual mistakes and personalised remediation during the learning process, rather than

after summative assessment (Edathil, Chin, Zank, Ranmuthugala & Salter, 2014). Three levels of

adaptivity were built into the platform used for our digital literacy curriculum:

1. Adaptive feedback, which allows instructors to provide help and address specific

misconceptions,





2. Adaptive learning pathways, which provide varying sequences of content to facilitate fast

movement through the package where information is known, or remediation where necessary,

and

3. Adaptive content authoring, which enable instructors to use analytics to track lessons’

outcomes, assess their efficacy, and adjust content and learning pathways accordingly (Smart

Sparrow, 2014).

These elements introduce students to processes of independent learning in a structured, but

personalised manner, therefore, “helping learners to become self-reliant, confident and able to make

judgements about the quality of their own learning” (Nicol, as cited in Jisc, 2010, p.10). As a result,

AeLPs are particularly well suited to the provision of instruction for diverse cohorts like the one used

in this study.

Institutional and Pedagogical Context

James Cook University (JCU) is a regional university located in Townsville and Cairns in North

Queensland, with a third campus located in Singapore. It is a significant provider of higher education

in the north east of Australia. A relatively small institution, JCU takes pride in responding actively to

the contextual factors associated with its regional location. Regional universities need to be

responsive to the access, engagement and outcome needs of their populations – this has been

identified as critical to regional development and prosperity (Universities Australia, 2013).

The digital literacies curriculum project documented below was intended to use the capacity of

current technologies to enable new types of learning experiences (Laurillard, Oliver, Wasson &

Hoppe, 2009), while also effectively responding to the incoming digital literacies of students and

developing their preparedness for academic and professional environments. The curriculum is

currently being trialled in JCU’s Diploma of Higher Education (DHE). The DHE is an open access





course that has attracted a diverse array of students. Students are encouraged to see the course as a

‘stepping stone’ to Bachelor qualifications. Within the 2016 cohort:

• 12% of students identify as Australian Aboriginal or Torres Strait Islander peoples;

• 63% are the first in their families to attend university;

• 32% are working with English as an additional language;

• 18% are from low socio-economic backgrounds;

• 10% are from regional or remote areas.

At the beginning of semester, students enrolled in the DHE were required to undertake a survey

regarding their readiness for using technologies in a university context. Responses to this readiness

assessment indicated that 38% of these students were highly anxious regarding online learning and

that approximately 42% had no Internet access at home. Anecdotal evidence gleaned from

conversations with students, tutors, academic support staff and lecturers suggests that there is also a

wide range of engagement and capacity in the digital readiness of students entering other

undergraduate degrees offered by JCU.

The DHE is designed to assist students with transitioning into university, and currently uses transition

pedagogy to carefully scaffold, mediate and support the student experience (Kift, Nelson & Clark,

2010). Transition pedagogy entails a whole of student/whole of course approach, with curriculum

serving as an organising device (Kift, 2009). This approach negotiates the gaps caused by diverse

levels of preparedness and cultural capital among incoming students (Kift, 2009). However, its

success is dependent on the “intentional design of learning, teaching and assessment approaches that

acknowledge the reality of the contemporary student context” (Sheppard & Hill, 2015, p.6). With

regard to digital literacy instruction, this acknowledgement involves recognising the potential impacts

of socio-economic, cultural, and demographic factors on students’ dispositions towards and

experiences of information technologies.





Online engagement data from DHE cohorts suggests a preference for content that utilises user-centred

design to respond to varying learning styles. Furthermore, student feedback from a core IT subject

indicates that low engagement rates could be the result of students ignoring content that seems overly

simplistic or, conversely, feeling overwhelmed by content that was overly challenging. While many

supporting resources (such as videos and text-based guides) are available to students to assist them

with building their digital literacies, these resources are linear in nature. Regarding interactivity, these

resources may sometimes test a student’s knowledge about a concept, but do not have the capacity to

remediate misconceptions. In 2016, the roll-out of the adaptive digital literacies curriculum extended

the DHE’s model of personalised support by using an adaptive e-Learning platform (AeLP) to provide

opportunities for skills development within an online environment that responds to students’ diverse

levels of preparedness.

An initial roll-out of the adaptive curriculum within the DHE has been trialled over two semesters, so

far. The first semester was used as a shakedown for the AeLP, and the lessons learnt from that roll-out

of the e-Learning modules were used to refine the product for the second semester. At the time of

writing, this semester is still in progress. It is anticipated that what is learnt from this project will

provide a contextually specific evidence base for the further development of a whole of institution

approach to digital literacy instruction for diverse learners – not only in this university, but in a

regional Australian context.

The Australian context is largely missing from the literature regarding digital literacies and

curriculum design at present. Projects undertaken in the United Kingdom with Jisc funding have

produced a strong evidence-base for the use of online learning platforms for implementing whole of

institution approaches to supporting digital literacies. Such approaches offer considerable support to

international institutions – especially with regard to their sustainability. The rational for our project

draws heavily from the work undertaken in other countries, acknowledging that the context of

Australasian institutions may not be completely analogous to the United Kingdom and Europe.





Throughout the project, the Educating the Net Generation report and toolkit (Kennedy et al., 2009)

has contributed significantly to our understanding of the digital literacies of students and staff in the

Australian context. Kennedy et al. (2009) administered an “experiences with technology”

questionnaire to explore student and academic experiences with technology, followed by qualitative

data collection. Key findings from the questionnaire indicate that:

• little empirical support exists for the popular conception that university students are

digital natives and university staff are digital immigrants;

• student and staff experiences with technology (and their preferences for the use of

technology in higher education) varies greatly; and

• the data paints a complex picture of the technological experiences first-year university

students bring to higher education (Kennedy et al., 2009, p. 3).

These findings, in conjunction with the institutional context described above, indicate a need for

differentiated and personalised digital literacies instruction. This need informed the selection of an

AeLP for the digital literacies curriculum described below. The design of the lessons was also

informed by principles of authentic and transformative learning.

Authentic learning occurs when task design and the context in which learning occurs reflect the way

knowledge is used in the ‘real-world’ (Herrington, Reeves & Oliver, 2010). Didactic (or linear) online

instruction can detach information from the context in which it is applied. Here, “knowledge itself is

seen by learners as the final product of education rather than a tool to be used dynamically to solve

problems” (Herrington & Oliver, 2000, p. 23). In contrast, authentic design integrates context and

learning by allowing for practice in knowledge application. Herrington, Reeves and Oliver (2010)

identify several characteristics of authentic e-Learning design, including real-world relevance, ill-

defined tasks that require problem-solving, and opportunities for engagement from a range of

theoretical and practical perspectives. When considering digital literacies, educational designers must

attend to the manner in which authentic learning and transformative processes can be incorporated





into procedural instruction. Such integration works best when learning includes reflection on the

“procedural assumptions guiding the problem-solving process … [to] reassess the efficacy of the

strategies and tactics used” (Mezirow, 2003, p. 204). AeLPs can facilitate such reflection through the

integration of ‘trap states’ that reflect common misconceptions. ‘Trap states’ are points in the

interactive activities in which students are redirected back to remediated content if they reveal that

they have not gained sufficient understanding. When students are faced with repeating information

and exercises, they are given the opportunity to reflect on their previous understanding and choices.

With these considerations in mind, a digital literacies curriculum was developed that could provide

personalised instruction on the use of technologies for research, assignment preparation, and online

identity management among diverse cohorts – particularly for students in non-computer science based

disciplines.

An Adaptive Digital literacies Curriculum

The curriculum designed for this project builds on findings from the 2007-2012 Adaptive Mechanics

project, which demonstrated that adaptive tutorials (ATs) had the capacity to improve student learning

outcomes and comprehension of threshold concepts (Gangadhara Prusty, et. al., 2013). The Adaptive

Mechanics project found a positive correlation between the use of ATs, student outcomes and course

satisfaction, and also showed reductions in failure rates and significant improvements among low

performing students (Gangadhara Prusty, et. al., 2013). Our curriculum builds on these findings and

responds to Jisc (2015b) recommendations that institutions provide a “progressive induction process

with targeted support for students’ digital needs [in conjunction with] … diagnostic tests on their

digital practices” (Jisc, 2015b, p.2). As the AeLP generates usage analytics, it can also be used “to

assess patterns of engagement and likely needs” (Jisc, 2015b, p.3) in an iterative processes of

development, roll-out, and refinement.

The curriculum designed for this project consists of three adaptive eLessons, two created in JCU by a

team consisting of a lecturer and two librarians, with the third developed by a team based in La Trobe





University, in Melbourne. These lessons were developed in keeping with the Jisc (2015a) six

capabilities model of digital literacy (see Figure 1 below). The two lessons developed by JCU follow

the narrative of a simulated group project. Creating a ‘story’ for the lessons was intended to provide

more interest and coherency and give the students a reason for completing the exercises, much the

way the narrative of games gives players a reason for progressing through the game’s challenges.

Over the course of the first two lessons, the student user is given the task of directing virtual group

members (henceforth referred to non-player characters, or NPCs) through a simulated group

assignment. Within the narrative, the student and NPCs are supposedly creating a poster, and must go

through a number of activities to prepare for this assignment. Throughout the lessons, two supporting

NPCs – a librarian and a lecturer – guide the user through the lesson activities as well as providing

feedback and remediation where necessary. Each of the student NPCs within the narrative embodies

specific strengths and common misconceptions, offering an opportunity to reflect on different

approaches to information and digital literacies. To complete the simulation, the student must

negotiate with the NPCs to solve a series of challenges orientated around core digital literacies, such

as information, data, and media literacies.

These activities allowed the curriculum to offer opportunities to integrate transformative process into

procedural instruction. For example, the information literacy lesson draws attention to the tendency of

Figure 1: Six capabilities model of digital literacies (Jisc, 2015a).





new students to only use Google, as opposed to specialist databases and library collections, to

conducting academic research. By allowing students to play out this misconception and experience its

consequences in a simulated environment, the lesson directs attention to, and corrects, a habitual

information seeking practice of novice researchers. In doing so, the lessons provide students with an

opportunity to reflect on and adjust their research practices before engaging with actual assignments.

By presenting students with scenarios they will be subsequently confronted with throughout the

duration of their studies, the transformative processes prompted by the e-Lessons should help them

apply their knowledge in new academic contexts.

The learning environment, tasks and interactions with fellow “students”, may have been simulated,

but they nevertheless provided an opportunity to present the learning material in a way that was far

more authentic than static and linear training material would have been. In the context of this adaptive

digital literacies curriculum, authenticity is engaged by using a narrative that reflects the actual

assessment processes that students are confronted with during studies at university. During these

lessons students are presented with multiple potential solutions to tasks through interactions with the

NPCs. Selecting from a range of solutions requires problem solving and engagement from a variety of

perspectives, while adaptive pathways provide opportunities for reflection and diverse outcomes. The

faux poster assignment also enables students to gain experience of group work and assignment

completion in advance of encountering those situations in reality, and there may be a flow on effect of

helping the students think about strategies for dealing with group assignments in the future.

The use of a narrative and adaptive activities with the eLessons was intended to combine authentic

learning theory and principles of game design in order to create an immersive (yet pedagogically

sound) learning experience. The narrative is presented by the means of branching, context-dependent

dialogue; the dialogue, interactions and activities that the student encounters in the course of the

lesson are dependent on the activities they have previously completed and the outcomes of those

activities. Interactions with NPCs trigger a selection of responses and choices. The students’ choices





then lead to particular branches of the lesson, which guide them through remediation pathways or

advanced content, as required. Possible choices are delimited (and limited) by the lesson design, but

the range of possible actions is broad enough to allow students to be held accountable for their choices

in terms of future actions and feedback. This design consequently reflects Olli Leino's (2012)

articulation of Don Idhe's idea that “the computer game [is] a technological artefact which makes

players responsible ... for the freedom it endows them with” (p. 59). Fittingly, the learning experience

“is not a feature designed into the game alone" (Jørgensen, 2008), but rather emerges from the process

of interaction and negotiation between the student and the AeLP, just as it might in gameplay. The

way the narrative provides context for the pathways enables the seamless integration of targeted

scaffolding and formative feedback within authentic activities that reflect the genuine day-to-day

digital literacy requirements of tertiary study.

Each lesson in the suite focusses on a different element of digital literacy. The first lesson introduces

the fundamentals of information literacy, which is defined in the Jisc (2015c) model as the ability to

find, evaluate, manage, and reference academic content in conjunction with a critical awareness of

credibility and provenance. The student is introduced to the NPCs and the faux poster assignment in

this lesson, and given the role of project manager. During the course of this lesson, the user guides

NPCs through the process of searching for, evaluating, and referencing a range of sources including

books, websites and journal articles. This process involves a series of simulations and activities that

introduce the student to digital research practices, such as identifying search terms, selecting

appropriate search tools and databases, using Boolean operators, and evaluating online resources. In

the second lesson, the group members begin to construct their poster. This lesson explores visual,

data, and media literacies through activities focussed on interpreting and presenting data,

understanding copyright law, recognising file formats, critically evaluating visual media, and creating

visual representations of academic content. The third lesson (developed by La Trobe University) sits

outside of this narrative, and provides an introduction to online identity management - including the





processes of creating and maintaining digital profiles for career development. This last lesson

addresses the aspect of digital literacy Jisc (2015c) defines as digital identity and well-being.

The activities in the lessons use common, simple input mechanisms such as multiple choice questions,

drag and drop interfaces, text input tools, hotspot activities, and dropdown menus – primarily because

the proprietary software used for this project tended to utilise these types of activities. It should be

noted that the software chosen for the design of these lessons influenced the design both by virtue of

its capabilities and because of the advice and support given by the educational designers working for

that company. Had we used different software, the activities may have taken a different form. From a

user perspective, the narrative should have made the activities seem less like a series of disjointed

tasks, but rather meaningful activities within a context. On one level, the student was asked to simply

select from a range of answers presented in a quiz format, but the narrative elevated the activity to one

where the student responded to scenarios that reflected the realities of tertiary study in digital

environments. Multiple choice activities are camouflaged within branching dialogue, and the adaptive

platform means that users are able to experience the consequences of choices – for example, selecting

to use Google rather than a university database for conducting research. This offers a process of

learning by (simulated) doing, and draws on a multimedia design philosophy that engages:

• Simulated interactions, in which the learner is integrated into the narrative;

• Construct interactions, in which the learner produces content through on screen actions;

• Immediacy, so that the learner experiences the events triggered by their own

actions/interactions;

• Feedback, in which the learner sees instantaneous responses from the platform;

• Goals, so that the learner is provided with a clear understanding of lesson objectives.

(Tan, Ling & Ting, 2007)

This learning by doing model has been shown to boost engagement and motivation by facilitating

practice in problem solving and self-evaluation (Tan, Ling & Ting, 2007). However, when regarding





the context of role-playing lessons, designers also need to attend to the believability the narrative in

order to promote immersion. In the context of our AeLP, narrative believability is closely related to

principles of authentic design and processes of transformative learning. However, we are under no

illusions that our delivery of that narrative enabled complete immersion by students. We were

constrained by the technology available to us, as well as our own skills in the design and development

of AeLPs and game-like environments.

Challenges, Opportunities and Reflections

This project, in its entirety, involved collaboration between academics, librarians and educational

designers across two universities and an external company. This provided us with the opportunity to

share knowledge and skills that enriched not only the final product, but also our working

relationships. Each of the three lessons was designed by the group of professionals with the most

teaching experience in that area. This resulted in a product that was significantly better than anything

that would have been produced by any one group working alone. In working collaboratively, we also

had the opportunity to learn from each other’s expertise and gain different perspectives on the content

of the lessons we were creating. The process of co-development enabled lecturers and librarians to

pool our collective experiences with regards to the misconceptions and digital learning challenges that

students face when working with learning and information technologies. The development of

authentic trap states and remediation pathways also required conscious reflection on the implicit

expectations that are placed on incoming students in technology-rich university environments. The

array of experiences that people brought to the project also meant that team members were not only

co-writers, but also a ready-made audience for testing activities and lesson content.

This method of collaboration was not, however, without its share of challenges. Possibly the greatest

challenge we faced involved our collaboration with the educational designers working for the third

party software providers. Working at a distance, and with large periods of time between meetings, we

found it difficult to fully convey what we wanted from the product and we frequently felt that we





were not getting as much support, direction and understanding from the software providers as we

would have liked. This was compounded by the fact that most of what we wanted to do with the

software was not within our control; we had to ask the designers to create something for us, which,

under different circumstances and using a different product, we might have been able to create for

ourselves. On reflection, and in terms of creative freedom and self-direction, it may have been better

to have used a more open source product and have been given training in how to create reusable

learning objects using that platform. However, with regards the scalability of the project and

expansion of the use of AeLPs within the university, an open source product would potentially shift

the onus of providing technical and design support back towards institutional IT staff.

Tensions between the technical/design constraints imposed by proprietary platforms and concerns

regarding resourcing for technical support and training in relation open source solutions are well

documented (van Rooij, 2007). In relation to AeLPs, the simplified user interface provided by

proprietary software lowers entry barriers for academics, but only when instructors are provided with

sufficient training in independent lesson development. The incorporation of technical support into

vendor licences also enables universities to outsource some of the risks associated with the early

adoption of new technologies. However, costs and the inflexibility of commercial software licences

impose usage constraints and raises concerns regarding the sustainability of third-party licencing

agreements. With regard to open source educational software, the presence of established

communities of users and specialised in-house IT support have been shown to mitigate institutional

concerns regarding the viability of open sources educational platforms (van Rooij, 2007). However,

the development of custom designed open source adaptive learning environments and the

establishment of specialised user communities is still in its infancy. As such, and with regard to the

use of open source adaptive learning solutions at an institutional level, “[t]he threshold that enables

perceived benefits to outweigh perceived risks” (van Rooij, 2007, p.442) has not yet been reached.





In terms of curriculum evaluation the analytics built into the AeLP have enabled us to monitor student

engagement, while initial anecdotal feedback from students and results monitoring has informed

preliminary appraisals of the curriculum. The lessons were integrated into an existing subject,

replacing a series of multiple choice quizzes that had previously been used for assessment purposes.

Completion of the adaptive curriculum constituted 15% of students’ overall subject grade and the

lessons were integrated into the subject’s learning management system. The lesson completion rate

(90 - 95%) was on par with – and, in some instances, marginally higher than – the completion rates

for the previous quizzes (90%). Following the initial roll-out, the lessons appear to have contributed to

improved results and submission rates on later student assignments. For example, the number of

distinctions (grades of 75% or higher) awarded for the research component of a later presentation

assignment increased by 13.7%, among students from the participating cohort located on the

Townsville campus. Submission rates for the same assignment rose by 6.8% within the same cohort.

However, these improvements in assignment results and engagement were not consistent across the

four campuses on which the lessons were trialled, and it is not yet possible to establish a direct causal

link between assessment results and lesson completion. Further analyses of cross-campus subject

results and lesson engagement data from the second semester roll-out (currently underway), in

conjunction with follow-up surveys among participating students, will inform fuller assessments of

the efficacy of AeLPs in relation to digital literacy instruction.

Overall, the project provided team members with the opportunity to explore and document the process

of moving from pedagogical theory and a conceptual framework to a workable curriculum design.

The adaptive nature of the platform and authentic design considerations required team members to

map out possible misconceptions and the implicit technological requirements of tertiary study. The

collaborative design processes provided team members with valuable insight into each other’s

working environments, enabling us to pool expertise from the fields of both library and information

science and game design. In addition, the project illuminated tensions associated with the selection

and development of sustainable e-Learning solutions that will support student engagement and





instructional design needs, while concurrently meeting institutional costing and IT support

requirements.

Conclusions

The move towards blended and online learning environments necessitates a base level of digital

literacy among students. Yet, within the widening participation agenda, we must remain mindful of

the different needs of diverse learners and the pressures these needs place on educators to provide

differentiated instruction to students with varying levels of expertise. Adaptive learning technologies

enable educators to respond to these challenges and provide personalised instruction to large cohorts

of diverse learners, particularly in areas such as digital literacy where students may have vastly

different pre-existing skillsets. In terms of teaching and learning, AeLPs enable a shift away from

didactic or linear online instruction. However, in doing so, the development of adaptive lessons

necessitates a design process that moves past the development of learning objectives and presentation

of content, and towards a consideration of potential misconceptions and implicit assumptions, in

conjunction with the development of variety of lesson pathways that enable fast movement through

known content and remediation where necessary. The simulated environments that AeLPs provide

also enable the development of authentic learning opportunities – but only when lesson design is

informed by considerations of the contexts in which knowledge is to be applied. The digital literacies

curriculum described above demonstrates how educational theories such as transition pedagogy,

authentic design, and transformative learning can work in conjunction with aspects of game design

(such as branching dialogue and multimedia design) to produce immersive personalised learning

experiences.

An effective transition pedagogy requires instructional design that carefully scaffolds, mediates, and

supports the student experience (Kift, Nelson & Clark, 2010). The digital literacies curriculum

described in this paper exemplifies this pedagogy through the provision of an orientation tool that

scaffolds and supports students’ entry into the online university environment. Initial evaluations of the





lessons points towards heightened engagement and improved student results, but further research is

necessary in order to determine the efficacy of the lessons in regard to enhancing students’ digital

literacies. Considerations of sustainability, technical training requirements for both IT and teaching

staff and a desire for control and flexibility with regards to lesson design will inform future decisions

about the use of either proprietary or open source software. However, as communities of practice

develop around open source adaptive learning solutions it is expected that perceived barriers to entry

at an institutional level may be reduced. The curriculum development process described in this paper

demonstrates not only the suitability of adaptive learning technologies for digital literacies instruction,

but also the manner in which collaborative design can inform the development of authentic online

learning experiences that respond to the needs of diverse learners.

References

Aviram, A. & Eshet-Alkalai, Y. (2006). Towards a theory of digital literacy: Three scenarios for the

next steps. European Journal of Open, Distance and E-Learning, 2006(1), 1-11.

Belshaw, D. (2014). The essential elements of digital literacies. Retrieved from

http://dougbelshaw.com/ebooks/digilit/

Brown, S. A. (2014). Conceptualizing digital literacies and digital ethics for sustainability education.

International Journal of Sustainability in Higher Education, 15(3), 280-290.

doi:10.1108/ijshe-08-2012-0078

Buckingham, D. (2006). Defining digital literacy. What do young people need to know about digital

media? Nordic Journal of Digital Literacy, 1(4), 263 - 276.

Dahlstrom, E., Brooks, D. C., Grajek, S., & Reeves, J. (2015) ECAR study of students and information

technology, 2015. Louisville, KY: ECAR.

Dawson, P. Charman, K., & Kilpatrick, S. (2013). The new higher education reality: What is an

appropriate model to address the widening participation agenda? Higher Education Research

& Development, 32(5), 706-721. doi: 10.1080/07294360.2013.776520





Eshet, Y. (2012). Thinking in the digital era: A revised model for digital literacy. Issues in Informing

Science & Information Technology, 9(2), 267- 276.

Edathil, S., Chin, C. K. H., Zank, S., Ranmuthugala, D., & Salter, S. (2014). Development of an

application with process feedback to enhance student-centred learning. In B. Hegarty, J.

McDonald & S.K. Loke (Eds.), Rhetoric and reality: Critical perspectives on educational

technology (pp. 394-398). Dunedin, Ireland: Ascilite.

Gabb, R., Milne, L., & Cao, L. (2006). Understanding attrition and improving transition. Retrieved

from http://learningandteaching.vu.edu.au/

Gangadhara Prusty, B., Vrcelj, Z., McCarthy, T., Ojeda, R., & Gardner, A. (2013). An adaptive e-

Learning community of practice for mechanics courses in engineering: Final report.

Retrieved from http://adaptive-mechanics.eng.unsw.edu.au/wp-

content/uploads/2013/08/CG10-1586-final-report-published.pdf

Herrington, J. & Oliver, R. (2000). An instructional design framework for authentic learning

environments. Educational Technology Research and Development, 48(3), 23-48.

doi:10.1007/BF02319856

Herrington, J. Reeves, T.C., & Oliver, R. (2010). A guide to authentic e-Learning. New York, NY:

Routledge.

Jisc. (2010). Effective assessment in a digital age: A guide to technology-enhanced assessment and

feedback. Retrieved from

http://www.webarchive.org.uk/wayback/archive/20140614115719/http://www.jisc.ac.uk/medi

a/documents/programmes/elearning/digiassass_eada.pdf

Jisc. (2011). ALT: Developing digital literacies. Retrieved from

http://www.webarchive.org.uk/wayback/archive/2014061400244/http://www.jisc.ac.uk/what

wedo/programmes/elearning/developingdigitalliteracies/alt.aspx

Jisc. (2014). Quick guide - developing students; digital literacy. Retrieved from

http://digitalcapability.jiscinvolve.org/wp/files/2014/09/JISC_REPORT_Digital_Literacies_2

80714_PRINT.pdf





Jisc. (2015a). Building digital capability: Building capability for new digital leadership, pedagogy and

efficiency. Retrieved from https://www.jisc.ac.uk/rd/projects/building-digital-capability

Jisc. (2015b). JISC-NUS benchmarking tool - the student digital experience. Retrieved from

http://repository.jisc.ac.uk/6140/1/Jisc_NUS_student_experience_benchmarking_tool.pdf

Jisc. (2015c). Digital capabilities: The 6 elements defined. Retrieved from

http://digitalcapability.jiscinvolve.org/wp/files/2015/06/1.-Digital-capabilities-6-elements.pdf

Johnson, L., Becker, S. A., & Hall, C. (2015). 2015 NMC technology outlook for Australian tertiary

education: A horizon project regional report. Retrieved from http://cdn.nmc.org/media/2015-

nmc-technology-outlook-australian-tertiary-education.pdf

Jørgensen, K. (2008). Audio and gameplay: An analysis of PvP battlegrounds in World of Warcraft.

Game Studies, 8(2). Retrieved from http://gamestudies.org/0802/articles/jorgensen

Kennedy, G., Krause, K.-l., Gray, K., Judd, T., Bennett, S., Dalgarno, B., … Churchward, A. (2009).

Educating the Net generation: A handbook of findings for practice and policy. Retrieved from

http://www.netgen.unimelb.edu.au

Kift, S. (2009). Articulating a transition pedagogy to scaffold and to enhance the first year learning

experience in Australian higher education. Retrieved from http://www.olt.gov.au/resource-

first-year-learning-experience-kift-2009

Kift, S. Nelson, K. J., & Clarke, J. A. (2010). Transition pedagogy: A third generation approach to

FYE: a case study of policy and practice for the higher education sector. The International

Journal of the First Year in Higher Education, 1(1) 1-20.

Kiili, C., Mäkinen, M., & Coiro, J. (2013). Rethinking academic literacies. Journal of Adolescent and

Adult Literacy, 57(3), 223-232. doi:10.1002/JAAL.223

Kuh, G. D., Cruce, T. M., Shoup, R., Kinzie, J., & Gonyea, R. M. (2008). Unmasking the effects of

student engagement on first-year college grades and persistence. Journal of Higher

Education, 79(5), 540–563.

Lankshear, C. & Knobel, M. (2003). New literacies: Changing knowledge and classroom learning.

Buckingham, England: Open University Press.





Laurillard, D., Oliver, M., Wasson, B., & Hoppe, U. (2009). Implementing technology-enhanced

learning. In N. Balacheff, S. Ludvigsen, T. de Jong, A. Lazonder & S. Barnes (Eds.),

Technology-enhanced learning: Principles and products (pp 289-306). Dordrecht, Germany:

Springer.

Lea, M. R. (2013). Reclaiming literacies: Competing textual practices in a digital higher education.

Teaching in Higher Education, 18(1), 106-118. doi:10.1080/13562517.2012.756465

Leino, O. T. (2012). Untangling gameplay: An account of experience, activity and materiality within

computer game play. In J. R. Sageng, H. Fossheim & T. M. Larsen (Eds.), The philosophy of

computer games, (pp 57-76). Dordrecht, Germany: Springer.

Littlejohn, A., Beetham, H., & McGill, L. (2012). Learning at the digital frontier: A review of digital

literacies in theory and practice. Journal of Computer Assisted Learning, 28(6), 547-556.

doi:10.1111/j.1365-2729.2011.00474.x

Mampadi, F. Chen, S. Y. Ghinea, G., & Chen, M. P. (2011). Design of adaptive hypermedia learning

systems: A cognitive style approach. Computers & Education, 56(4), 1003-1011.

doi:10.1016/j.compedu.2010.11.018

Martin, A. (2006). Literacies for the digital age. In A. Martin & D. Madigan (Eds.), Digital literacies

for learning (pp. 3-25). London, England: Facet.

Mezirow, J. (2003). How critical reflection triggers transformative learnin. In P. Jarvis (Ed.), Adult

and continuing education: Major themes in education (pp. 199-214). New York, NY:

Routledge.

Murray, M. C., & Pérez, J. (2015). Informing and performing: A study comparing adaptive learning to

traditional learning. Informing Science, 18(1), 111-125.

Schott, F., & Selwyn, N. (2011). Game literacy: Assessing its value for both classification and public

perceptions of games in a New Zealand context. In P. Felicia (Ed.), Handbook of research on

improving learning and motivation through educational games: Multidisciplinary approaches

(Vol. 1, pp. 176-191). Hershey, PA: Information Science Reference.





Sheppard, K. & Hill, A. (2015, October). Pathways to university: Intentional curriculum and student

support mechanisms and some tentative tales of success. Paper presented at Global Access to

Post-Secondary Education Conference, Kuala Lumpur, Malaysia.

Smart Sparrow. (2014). Our learning design platform. Retrieved from

https://www.smartsparrow.com/platform/

Tan, P. Ling, S., & Ting, C. (2007, September). Adaptive digital game-based learning framework.

Paper presented at the 2nd International Conference on Digital Interactive Media in

Entertainment and Arts, Perth, Australia.

The Open University. (2015). Digital and information literacy framework. Retrieved

from http://www.open.ac.uk/libraryservices/pages/dilframework/index

Universities Australia, (2013). An agenda for Australian higher education 2013-2016. Retrieved from

http://www.universitiesaustralia.edu.au

van Rooij, S. W. (2007). Perceptions of open source versus commercial software: Is higher education

still on the fence? Journal of Research on Technology in Education, 39(4), 433-453.

doi:10.1080/15391523.2007.10782491

Wauters, K. Desmet, P., & Van den Noortgate, W. (2010). Adaptive item-based learning

environments based on the item response theory: Possibilities and challenges. Journal of

Computer Assisted Learning, 26(6), 549-562. doi:10.1111/j.1365-2729.2010.00368

Yang, D. F. Catterall, J., & Davis, J. (2013). Supporting new students from vocational education and

training: Finding a reusable solution to address recurring difficulties in eLearning.

Australasian Journal of Educational Technology, 29(5), 640 - 650.



http://www.open.ac.uk/libraryservices/pages/dilframework/index

using an adaptive e-learning curriculum to enhance … · using an adaptive e-learning curriculum...

Documents