ensuring quality in online learning

1

Ferguson, RJ. (2013), SMT Plus, Penn Valley California

Quality in Online Teaching, Training Materials and Systems

Introduction

The technological advances during the past forty years in information processing and

social communication, has significantly increased the potential for online learning and teaching

at a distance (Erhmann, nd.). Technology is changing the landscape of society and in general is

also a catalyst for shifting delivery of training and education (Hawkins, 1999 as cited in Berge,

2001, p. 3). Technology’s “impact on education and training has been as great as on any other

area. Web sites provide instant access to data from active volcanoes, web cameras record

events as they happen, and web databases provide access to a multitude of academic

resources (Bates & Poole, 2003, p. 8). The plethora of learning strategies and activities that

can be implemented by learning designers has increased considerably “as hardware and

software have become more capable of replicating real experiences, or providing realistic virtual

ones” (Benenson, 2001; Amato, 2001; Wortman, 2001; “Playing to win,” 2004; Dollarhide, 2005;

Groopman, 2005, 2009 as cited in Fahy, 2012, p. 43). In short and for the most part, gone are

the days of using snail mail to send course materials and submit assignments and gone are the

days of paper-based learning with static imagery and no interactive components. Today,

learners are able to seek, organize and analyze information, communicate with each other and

their instructors using various web technologies and modalities (Bates A. , 2005, p. 9).

Learning solutions developed and deployed for online and distance learning purposes

can encompass the full spectrum of technologies. These comprise synchronous and

asynchronous strategies such as web-based conferencing; self-paced courseware, autonomous

digital assets and online communities of practice. Today, learning solutions also comprise

sophisticated learning infrastructures and platforms such as learning management systems

(LMSs), learning content management systems (LCMSs), talent management systems (TMSs),

human resource systems (HRMSs) and enterprise resource planning systems ERPSs).

Bringing these all together are interoperability standards which are used to integrate these

2


different types of systems into one cohesive system such that they’re able to exchange data and

perform the function(s) they’re intended to.

Regardless of how good or advanced the technology is or sophisticated the learning

infrastructure and/or systems, if learning strategies and activities are not properly designed,

deployed, evaluated and maintained, the learning effectiveness can be less than mediocre.

Bates states “good teaching may overcome a poor choice of technology, but technology will

never save bad teaching” (2005, p. 211)

Assuring Effective Learning

Poor learning results arise from issues in the design of learning strategies and activities;

problems for learners accessing the learning material and/or sessions, and the overall learning

experience. These problems generally stem from a learning development team’s inadequate

skills and inability to effectively collaborate. Also contributing to poor learning results is the

conflicting information pertaining to the effectiveness of various learning modalities. For

example, research has shown that self-paced instruction has the power to improve and

accelerate learning by about one third, compared with lectures and conventional assignments

on the same topic (Erhmann, nd.). Smith & Ragan (2005, p. 367) assert that the most powerful

uses of technology in both education and training appear to be those that fit best with integrated

designs. Yet there are those who argue there is no significant difference when technology and

media are used (Clark, 1983 as cited in Fahy, 2012, p. 32). In all fairness, it is worth mentioning

there are an equal number of proponents for the use of technology to enhance learning (Fahy,

2012, p. 32).

Mayer who struggles with the approaches used in the research of the effectiveness of

technology and media, shares the same assertion as Kozma (1994, p.13 as cited in Mayer,

2009, p. 232) which is that there needs to be a “shift in the focus of research from media as

3


conveyors of methods to media and methods as facilitators of knowledge construction and

meaning making on the part of learners”.

And so it is left to the subjective decision of learning designers. One can choose to go

with a single standalone traditional approach or alternatively, one can choose multiple

modalities utilizing various technologies thus applying an integrated holistic solution. But which

approach is best? Cleveland-Innes & Ally (2006, pp. 55-56) assert that fundamental to the

overall success of any learning initiative and/or project is:

1) The commitment from all stakeholders must be high and enduring.

2) All stakeholders must be involved in the learning development project

3) Resource allocation must be adequate to carry the project to completion.

4) All project staff should hold positions for which they are most competent.

5) There should be a high standard of quality in terms of the instructional

design, supporting materials and the infrastructure through which the learning

is deployed.

Of these five fundamental requirements, the last two requirements relating to the

competency of the staff of the learning development team and its adherence to high standards

of quality are of paramount importance to the collective capabilities of the learning development

team and ultimately, the quality and effectiveness of the learning design. But learning design is

about applying principles of ISD (Instructional Systems Design).

Although the ADDIE (Analyze, Design, Develop, Implement and Evaluate) approach

represents the fundamental concepts of instructional design, according to Reiser and Dempsey

(2012), the following are six key characteristics that should be present in all instructional design:

1. Instructional design is student centered.

2. Instructional Design is goal oriented.

3. Instructional Design focuses on meaningful performance.

4


4. Instructional Design assumes outcomes can be measured in a reliable and valid

way.

5. Instructional design is empirical, iterative, and self-correcting.

6. Instructional Design is typically a team effort.

Equally important to designing learning solutions is knowing how to develop strategies

that meet the needs of each learner without compromising cost-effectiveness, quality and

expediency of the learning. Grow (1991)identifies four stages that learners progress through:

(1) S1 – Dependent Learner

(2) S2 – Interested Learner

(3) S3 – Involved Learner

(4) S4 – Self-Directed Learner

Although Grow’s stages of learner development seem reasonable, designing for each of

these stages is a formidable challenge. The question needs to be asked, what are the skills a

learning development team requires in order to design units of learning that address the needs

of learners functioning at each stage and additionally, effectively promote individual learners

through to the next stage of learning? Ultimately the goal is to have each learner reach stage 4

where they’re capable of independently participating in self-directed and self-paced units of

learning. The point being made is that a highly skilled and knowledgeable learning development

team is required to assure the design and development of effective learning.

In order to ensure quality in online instruction, every aspect of the learning experience

needs to be part of the design, i.e. access, scheduling, learning strategies, learning activities,

assessment strategies and assessment activities, competencies of instructors and proctors,

learner ethnic and cultural backgrounds and entry level knowledge. With this in mind, the

importance of a learning development team’s skills, creativity, experience, and wisdom should

not be taken lightly. Collectively, the development team’s abilities are paramount to developing

learning experiences in a timely manner and with a level of accuracy and quality that yields

5


positive results such as ease of access, learning effectiveness, and cost effectiveness and

learner satisfaction.

Last but not least and equally important to ensuring effective learning results is the on-

going task of evaluating “how work gets done”. Focussing on the skills of the development team

and their collective ability to pay attention to the minutiae is principal to establishing streamlined

processes, minimizing effort and reducing rework. The successful collaboration of the learning

development team and their collective ability to work smart to expedite the completion of tasks

and projects should be a primary directive of any learning development team.

Part of working smart includes developing proficiency in everything a learning

development team is required to do. This speaks directly to the management of learning

development projects. Bates et al (2003) identifies the following areas that comprise a learning

design and development project:

Identification of the main target group or type of students to be taught.

Strategic aims in using technology.

Choice of appropriate mode of teaching: technology-enhanced classroom

teaching, mixed-mode, fully distance.

Desired learning outcomes: choice of content, skills, and values to be covered in

the unit of learning.

Method of working: Lone Ranger, boutique, collegial material development, or

project management .

A key performance indicator (KPI) for learning development teams is the efficiency with

which they can design and develop learning solutions from concept to completion. The

efficiencies in which tasks are completed are measured by the amount of overall time required

to produce a certain result and/or complete a deliverable (i.e. achieve a milestone).

6


To optimize efficiencies, organizations should be continuously creating opportunities for

their learning development teams to excel. This means cultivating a culture which encourages

new learning and constant innovation by the development team. It also means encouraging

cross-pollination of skills and assigning tasks that require team members to be multi-skilled.

This approach of cross-pollination of skills reduces development time; contributes to overall

quality of the finished learning solution, increases efficiency and reduces development costs.

Additionally, this type of thinking and approach enables organizations to significantly simplify the

logistics of managing and completing the development of online learning projects. This is a good

example of where “less is more”, i.e. fewer people producing more in less time. It also provides

high degrees of satisfaction for members of the learning development team. Additionally, it

means that each member of the learning development team becomes proficient in a broad

range of tools and skills so that they can help each other out during times of peak demand on a

project.

Areas of Focus During Design and Development

According to Moore (2002), the Sloan Consortium has developed a framework for ensuring

quality in online teaching. The framework is centered on what the consortium refers to as the

five pillars of quality:

1. Learning Effectiveness

2. Cost Effectiveness

3. Access

4. Faculty Satisfaction, and

5. Student Satisfaction.

If the learning design is effective, enjoyable and easy to access, it will result in both

student and teacher satisfaction. Therefore the key areas of interest are:

1. Cost Effectiveness

7


2. Learning Effectiveness.

3. Access

Cost Effectiveness

One way to ensure high-quality cost-effective technology-based teaching and learning is

through project management (Bates A. W., 2000, p. 66). In highly productive project teams, the

continuous monitoring of tasks in terms of time on task; progress, accuracy and quality of the

work completed each day is critical to the timeliness, accuracy and quality of the completed

project. Essential to ensuring the timeliness, accuracy, quality of a learning development

project is that each team member evidences their progress on a daily basis. Evidencing

progress contributes greatly to heading off problems at the pass. By continuously allowing all

stakeholders involved in a learning project to monitor the daily progress, accuracy and quality of

tasks being worked on, highly productive learning development teams are able to ensure that

finished projects exceed the expectations of all stakeholders including most importantly, the

target learners.

Learning Effectiveness

Although there are numerous lists of adult learning principles, eight principles of adult

learning that contribute to ensuring the effectiveness of online learning are:

1. Make learning goals and objectives clear to learners

2. Combine cognitive, equivalency, constructivism and behaviorism learning theories to

create learning strategies and activities that include prompt constructive feedback.

3. Provide an optimal balance of challenge and support that is tailored to the learner’s

knowledge and exposure of a subject matter.

8


4. Broaden a learner’s experience of a subject matter by creating learning strategies

that incrementally moves a learner up each of the levels within the three types of

learning domains:

a. Cognitive

b. Affective, and

c. Psychomotor

5. Link inquiries or issues of high interest to the learners to genuine problems to

enhance motivation and accelerate their learning

6. Elicit active and critical reflection by learners on their growing base of knowledge and

experience

7. Develop a participant’s effectiveness as a learner early in the learning session as this

has been proven to significantly enhance their confidence and motivation and

accelerate the learning experience

8. Create an environment that supports easy access to previously acquired knowledge

The abovementioned eight principles should be incorporated into the design of online

learning such that a unit of learning accommodates:

Anytime/anywhere access to synchronous and asynchronous learning sessions

Synchronous conferencing and collaboration

Feature-rich content management and development

Feature rich summative assessments with adaptive learning functionality

Communities of practice and knowledge management

Flexible scheduling.

9


For learning development teams, adhering to these key principles and having advanced

understanding of how learning occurs will in turn have a positive impact on the effectiveness of

the learning.

Access

The ease or difficulty learners experience accessing online units of learning is a decisive

component of determining student and faculty satisfaction. If access is difficult, it results in

numerous support calls detracting from the time allocated for the learning. This will cause

frustration for the learners as well as faculty. If the access is easy, then learners and faculty will

be able to focus energies on the learning material.

Designing For Maximum Learning Effectiveness

Similar to Grow’s 4 stages of learning, Schunk (2012, p. 19) asserts that Learners progress

through stages or phases of learning. An example is one of Gagne’s categories, Preparation for

learning. This category contains three of Gagne’s nine phases of learning: attending,

expectancy and retrieval. Attending is the provision of stimuli relevant to the material to be

learned. Expectancy is the presentation of the objectives to be achieved. Retrieval is the

retrieval of relevant information from LTM (Long-Term Memory) (Schunk, 2012, p. 222)

A learning development team’s ability to appropriately address these types of

requirements is important to the effectiveness of the learning. When an insufficient amount of

preparation for learning information is provided, a learner’s cognitive load is increased during

actual learning sessions. This is due to the learner attempting to understand the relevance of

small chunks of knowledge in the big picture of learning. Related to preparation for learning are

“three instructional applications that reflect information processing principles:

1. advanced organizers,

2. conditions of learning, and

10


3. cognitive load” (Schunk, 2012, p. 217)

Advanced organizers can be considered introductions, transitions and summaries that

help to structure and connect the learning material. “Organizers direct learners’ attention to

important concepts to be learned, highlight relationships among ideas and link new material to

what students know” (Faw and Waller, 1976 as cited in Schunk, 2012, 217).

Providing well-detailed organizers (i.e. unit introductions) assists the learning of

“unfamiliar and potentially confusable information” (Driscoll, p. 144). The following module

introduction (Figure A) was created for an online self-paced course and is an example of an

advanced organizer.

Figure A – Example of an Advanced Organizer

It states the purpose of the unit of learning as well as the objectives and the time required to

complete the learning

Conditions of learning specify the learning outcomes and the learning events (Schunk, 2012,

p. 219). In addition to stating the objectives using the advanced organizer approach previously

mentioned, it is equally important to ensure that as learners are progressing through a unit of

learning (regardless of how small or large it may be) that they are aware of what the current

11


objective is at all times and the conditions of the learning. All too often, within a larger unit of

learning, the preparation for the learning component will include an advanced organizer,

however for the smaller units of learning that comprise the larger unit will fall short on explaining

what the learning outcome is or the conditions of the learning. Figure B is an example from a

Training Guide for operators working in a hydrocarbon processing facility. It shows the objective

for the unit of learning (in this case the smaller unit of learning is a Learning Object) and states

the events (activities) the learner will participate in and complete including a summative

assessment component, i.e. the conditions of the learning.

Figure B – Conditions of Learning

Cognitive load refers to the strain that is put on working memory, also known as STM

(Short-Term Memory) by the processing requirements of a learning task (Driscoll, 2005, p. 136).

Although the instruction may help learners to learn how to; solve problems, acquire critical

thinking skills and adjust attitudes, the general opinion is that the more a person has to learn in

a shorter amount of time, the more difficult it is to process that information in working memory.

Take for example the mean-ends analysis approach to problem solving, “the cognitive load

12


imposed on a person using a complex problem solving strategy such as means-ends analysis

may be an even more important factor in interfering with learning during problem solving”

(Sweller, 1988, p. 268).

Reiterating, the quality of online instruction is dependent on the skills of the learning

development team and this is equally the case when designing for maximum learning

effectiveness and taking into consideration components of the instruction such as preparation

for learning.

Levels of Sophistication

The effectiveness of online learning is also a product of the level of sophistication which

is determined by the complexity of the learning strategies and the corresponding learning

activities. The amount of development effort is determined by the delivery medium combined

with the complexity of learning activities. Take for example the sophistication of self-paced

online learning sessions. The degree of sophistication will vary depending on the types of

learning activities and the various types of visual media used such as 3D models, computer-

generated animations, video, digitized photographs, and 2D/3D static imagery (see Figures C to

H). Each of the visuals shown below required different efforts to create and each varies

considerably in sophistication.

Figure C - Anthropomorhising Inanimate Objects Figure D – Comic Strip Animations

13


Figure E - Video Figure F – Animated Processes

Figure G – Interactive 3D Model Figure H- Digitized Enhanced Photograph

Additionally, learning activities such as discovery-through-exploration and critical

thinking activities, as well as short games, to transfer knowledge, skills, and attitudes also

determine the degree of sophistication. To view an example of a discovery-through-exploration

learning activity, point a web browser to the web address http://screencast.com/t/1Jjdam32eLj .

The learning development team was tasked with developing a learning activity that minimized

cognitive load while expediting the learning process. This seems a much more worthy endeavor

than accepting the general opinion that the more there is to learn, the higher the cognitive load

and therefore the more difficult it is to process the information in working memory. Cognitive

load can be minimized when persons are thoroughly enjoying the learning experience and are

so engrossed in the learning that they consciously don’t realize how much new knowledge

they’re learning at the time. Achieving this in an online distance education model is conducive to

http://screencast.com/t/1Jjdam32eLj

14


ensuring quality and effective learning. Additional samples can be seen at

www.youtube.com/smtlearningchannel .

There are many aspects to ensuring quality online learning that go beyond the scope of

this paper. Suffice to say a highly qualified learning development team; ease of access,

professional delivery, experiential learning activities and realistic learner expectations are

central to the effectiveness of online learning. However, online learning is distance learning

which requires independent learning and this paper would not be complete without mention of

Wedemeyer’s ten characteristics of independent learning.

Wedemeyer identified ten characteristics emphasizing learner independence and

acceptance of technology as a means to implement that independence (Simonson, Schlosser, &

Hanson, 1999). Wedemeyer’s ten characteristics are:

1. Be capable of operation any place where there are students—or even only one

student—whether or not there are teachers at the same place at the same time

2. Place greater responsibility for learning on the student

3. Free faculty members from custodial-type duties so that more time can be given to truly

educational tasks

4. Offer students and adults wider choices (more opportunities) in courses, formats, and

methodologies

5. Use, as appropriate, all the teaching media and methods that have been proved

effective

6. Mix media and methods so that each subject or unit within a subject is taught in the best

way known

7. Cause the redesign and development of courses to fit into an “articulated media

program”

8. Preserve and enhance opportunities for adaptation to individual differences

http://www.youtube.com/smtlearningchannel

15


9. Evaluate student achievement simply, not by raising barriers concerned with the place,

rate, method, or sequence of student study

10. Permit students to start, stop, and learn at their own pace

According to Simonson et al (1999) “Wedemeyer proposed the separation of teaching

from learning as a way to break education's "space-time barriers." He suggested six

characteristics of independent study systems:

1. The student and teacher are separated.

2. The normal processes of teaching and learning are carried out in writing or

through some other medium.

3. Teaching is individualized.

4. Learning takes place through the student's activity.

5. Learning is made convenient for the student in the student's own

environment.

6. The learner takes responsibility for the pace of learning, with freedom to start

and stop at any time.”

Wedemeyer’s ideas represent the utopia of learning and although institutions of higher

education have made significant advancement towards implementing Wedemeyer’s ideas, there

remains much more to be changed from the entrenched traditional models. For example,

consider a situation where the course work is overly heavy and a learner is struggling to juggle

work, family and studies, now consider the stress created on the learner. Is this a good thing?

According to the Mayo Clinic (Mackusick, 2013) a little bit of stress can be a good thing

however too much stress is unhealthy and detrimental to ones well-being. Nevertheless, it is not

uncommon for some higher education instructors to actively foster stress “as a matter of pride,

tradition, or rite of passage (“I had to deal with this stuff when I was a student, and so will

you”)” (Putz, nd.).

16


Overtime, these current meaning schemes and perspectives will change and learners will

be able to learn according an emerging paradigm. According to Barr and Tagg, there is a shift in

institutions of higher education from institutes of instruction to institutes that produce learning.

The new "Learning Paradigm" will liberate institutions from the current constraints that

prevent them from effectively addressing the challenges of present day including a shifting away

from the banking concept of education which flies in the face every principle for ensuring

effective quality learning. The Learning Paradigm ends the lecture's privileged position,

honoring in its place whatever approaches serve best to prompt learning of particular knowledge

by particular students (Barr & Tagg, 1995).

Conclusion

This paper has focused primarily on the knowledge and skills of the learning

development team and the areas of development necessary to ensure effective quality online

learning. Fahy (2012) refers to these as critical criteria for program success. He states that “it

should be clear that successful teaching in an online program requires that personnel have philosophies,

attitudes, behaviours, and interpersonal practices consistent with the program’s goals”

Bibliography

Barr, R. B., & Tagg, J. (1995, November/December). From Teaching to Learning - A New

Paradigm for Undegraduate Education. Change Magazine.

Bates, A. (2005). Technology, e-learning and Distance Education. New York, NY: Routledge.

Bates, A. W. (2000). Managing Technological Change. San Francisco, CA: Jossey-Bass Inc.

Bates, T., & Poole, G. (2003). Effective Teaching with Technology in Higher Education 1st ed.

San Francisco, CA: Jossey-Bass.

17


Berge, Z. L. (2001). Sustaining Distance Training. San Francisco, CA: Jossey-Bass Inc.

Cleveland-Innes, M., & Ally, M. (2006). Managing Online Learning Projects at a Distance: a

case for workplace training. In B. Pasian, & G. Woodill, Elearning Project Management:

Canadian Perspectives (pp. 53-57). Canadian e-Learning Enterprise Alliance .

Driscoll, M. P. (2005). Psychology of Learning for Instruction 3rd Edition. Boston: Pearson

Education, INc.

Erhmann, S. C. (nd.). Technology Changes Quickly: A Counter-Intuitive Strategy for improving

the Outcome of Higher Education. Retrieved June 7, 2013, from the TLT Group Web

site: http://www.tltgroup.org/resources/visions/Outcomes.html.

Fahy, P. (2012). MDDE 621: Online Teaching in Distance Education. Athabasca, AB:

Athabasca University CDE.

Grow, G. O. (1991). Teaching Learners to be Self-Directed. Adult Education Quarterly, Volume

41, Number 3, Spring, 125 - 149.

Mackusick, D. (2013). Top Stress Relievers for Online Learners. Retrived from the World Wide

Web: http://forwardthinking.ashford.edu/top-stress-relievers-for-online-learners/.

Mayer, R. E. (2009). Multimedia Learning Second Edition. New York, NY: Cambridge University

Press.

Moore, J. (2002). Elements of Quality: The Sloan-C Framework. Needham, MA: Sloan Center

for Online Education.

Putz, M. (nd.). Limiting Structural Stress. Retrieved from the world wide web:

http://www.bethel.edu/media/university/faculty/engaged-

teaching/media/LimitingStructuralStressforOnlineLearners.pdf: Bethel University.

Reiser, R. A., & Dempsey, J. V. (2012). Trends and Issues in Instructional Design and

Technology. Boston, MA: Pearson Education Inc.

Schunk, D. H. (2012). Learning Theories: An Educational Perspective, (6th ed). Upper Saddle

Hill, NJ: Pearson Education Inc.

18


Simonson, M., Schlosser, C., & Hanson, D. (1999). Theory and Distance Education: A New

Discussion. The American Journal of Distance Education Vol. 13 No.1.

Smith, P. L., & Ragan, T. J. (2005). Instructional Design, (3rd Ed). Hoboken, NJ: John Wiley and

Sons.

Sweller, J. (1988). Cognitive Load During Problem Solving: Effects on Learning. Cognitive

Science 12, 257 - 285.

ensuring quality in online learning

Business