ducation - mrsite.comface to face teachi…  · web viewrumble (1992) pointed out the...

33
Face to Face Teaching in Distance Education - a literature and case study review Ormond Simpson Visiting Fellow, Centre for Distance Education, University of London International Programmes Previously Visiting Professor, Open Polytechnic of New Zealand Previously Senior Lecturer in Institutional Research, UK Open University Contents Executive Summary………………………………………………………………………………………………………………………….…..1 Introduction…………………………………………………………………………………………………………………………………….…...2 Groupings of distance institutions…………………………………………………………………………………………. …2 Student retention in distance education………………………………………………………………………………..…3 Literature review……………………………………………………………………………………………………………………………..……5 OU research into f2f teaching……………………………………………………………………………………………..……5 Blended, traditional and online teaching…………………………………………………………………………..……..5 1. Comparison of f2f and exclusive online teaching modes…………………………………………6 2. Comparisons of blended and exclusive online modes……………………………………….….…6 Timing of f2f and blended teaching…………………………………………………………………………………….. …..7 Student skills for online learning………………………………………………………………………………………...……7 Student preferences……………………………………………………………………………………………………………..….7

Upload: others

Post on 16-Jul-2020

2 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: ducation - mrsite.comFace to face teachi…  · Web viewRumble (1992) pointed out the vulnerability of dedicated distance teaching universities to competition from dual mode universities

Face to Face Teaching in Distance Education- a literature and case study review

Ormond Simpson

Visiting Fellow, Centre for Distance Education, University of London International ProgrammesPreviously Visiting Professor, Open Polytechnic of New Zealand

Previously Senior Lecturer in Institutional Research, UK Open University

Contents

Executive Summary………………………………………………………………………………………………………………………….…..1Introduction…………………………………………………………………………………………………………………………………….…...2

Groupings of distance institutions………………………………………………………………………………………….…2 Student retention in distance education………………………………………………………………………………..…3

Literature review……………………………………………………………………………………………………………………………..……5OU research into f2f teaching……………………………………………………………………………………………..……5Blended, traditional and online teaching…………………………………………………………………………..……..5

1. Comparison of f2f and exclusive online teaching modes…………………………………………6 2. Comparisons of blended and exclusive online modes……………………………………….….…6

Timing of f2f and blended teaching……………………………………………………………………………………..…..7 Student skills for online learning………………………………………………………………………………………...……7 Student preferences……………………………………………………………………………………………………………..….7 Full-time faculty preferences………………………………………………………………………………………….……..…8 Adjunct faculty/part-time tutors/OU AL preferences.………………………………………………………………8 Time issues for faculty…………………………………………………………………………………………………………..…8 Time issues for students……………………………………………………………………………………………………….….8 Trends…………………………………………………………………………………………………………………………………..…8 Other media for blended teaching………………………………………………………………………………………..…8

Costs and benefits of f2f and online teaching……………………………………………………………………………………...9 International case studies…………………………………………………………………………………………………………………..11Conclusions………………………………………………………………………………………………………………………………….…..…16Appendices………………………………………………………………………………………………………………………………………...18References…………………………………………………………………………………………………………………………………..……..20

28 September 2014

Page 2: ducation - mrsite.comFace to face teachi…  · Web viewRumble (1992) pointed out the vulnerability of dedicated distance teaching universities to competition from dual mode universities

Face to face teaching in Distance Education 2014

Executive Summary

This review is of the role of face to face (f2f) study in distance education and comprises an international review with a set of case studies. The basic findings are as below.

1. F2f teaching survives in many distance institutions. It is actually a compulsory element in some well-known institutions.

2. F2f is usually part of a blended teaching approach. F2f teaching is often part of a ‘blended teaching’ approach in which exclusively online teaching is combined with email, phone, f2f and other forms of teaching.

3. Blended teaching has a very positive student retention effect. There is good evidence that blended teaching has a substantial retention effect over exclusively online teaching.

4. It is not clear what part of blended teaching is most effective. There is very little evidence as to which particular forms of blended teaching are the most effective either in retention or cost benefit terms. The report notes that there is evidence that certain blended teaching activities using the phone have been shown to not only have a clear retention effect, but also to have a positive cost-benefit effect with a return on investment of several hundred percent. But there is little research into the cost-benefits of f2f teaching specifically.

5. Blended teaching is particularly important at the beginning of a course. Whatever forms of blended teaching are used the evidence clearly suggests that it should be focused very specifically at the earliest possible stage of a course, and that it is particularly successful at enhancing social integration and overcoming initial anxiety.

6. It is important to maintain different forms of blended teaching including f2f teaching. Different students may find different types of teaching helpful at different stages. So it is probably important to use the different types of blended teaching to reach all types of student.

7. Younger students in the so-called ‘Google generation’ do not necessarily have a high level of skills in online learning. It is important not to make overly optimistic assumptions about the ability of especially younger students to study entirely online. They are likely to need some form of blended teaching to survive.

Page 1

Page 3: ducation - mrsite.comFace to face teachi…  · Web viewRumble (1992) pointed out the vulnerability of dedicated distance teaching universities to competition from dual mode universities

Face to face teaching in Distance Education 2014

Face to face teaching in Distance Education - a literature and case study review

Ormond Simpson

Introduction

A feature of of distance education is the enduring survival of face-to-face (f2f) teaching despite the increasing use of e-teaching systems1. This appears to be true, not only of dual-mode universities (where distance education is an addition to on-campus face-to-face provision) but also of single mode universities (which operate entirely by distance methods). What explains this survival and is it likely to continue? This review combines a literature survey of f2f teaching in distance education, with case studies of a number of individual distance institutions.

It is helpful to start by examining distance institutions in terms of their incorporation of f2f teaching, and by scrutinising some concerns about student success in distance education with particular respect to the OU.

Groupings of distance institutions. Rumble (1992) divided distance institutions into two basic groups - ‘Single Mode Universities’ (SMUs) who operate entirely at a distance, and Dual Mode Universities (DMU’s) who run both distance and face to face programmes. DMU’s often support their students through ‘hybrid’ or ‘blended teaching’ where students are offered both online and face to face teaching.

Rumble (1992) pointed out the vulnerability of dedicated distance teaching universities to competition from dual mode universities. His argument was that the latter, usually conventional campus-based universities offering opportunities for distance learners to take the same courses as on-campus learners and earn the same qualification, had significant competitive advantages in terms of cost, choice of courses and learner convenience over the dedicated distance learning provider.

Rumble may have been proved right: there are relatively few SMU’s extant and indeed in the last few years a number of start-up SMU’s have closed such as NYU Online, Cardean University, Columbia’s Fathom, the Oxford/Yale/Stanford ‘Allearn’ and, notoriously, the UKeU, many with losses in the million dollar range. However there are still examples of SMU’s such as Athabasca University in Canada and Indira Ghandi National Open University in India (which is apparently moving from a DMU to SMU - see the case studies).

The OU of course tends to fall into an intermediate group of its own, having no on-campus students2 whilst maintaining some f2f teaching, and at the same time as moving substantially into e-teaching using online tutorials. Nevertheless for the moment it is still a ‘blended’ institution.

1 The terms ‘blended teaching’ and ‘e-teaching’ are preferred throughout this report, as they are a more accurate description of what institutions do. ‘Blended learning’ and ‘e-learning’ are what students do. It is useful to avoid the category error of confusing the two activities. 2 Woodley (2013) argues that this means that OU Central Faculty can be somewhat disassociated from the day-today problems of students and that this may contribute to in some way to a lack of engagement with the issue of student dropout at the Centre generally.

Page 2

Page 4: ducation - mrsite.comFace to face teachi…  · Web viewRumble (1992) pointed out the vulnerability of dedicated distance teaching universities to competition from dual mode universities

Face to face teaching in Distance Education 2014

Student retention in distance education. It is important to contextualise this study with some basic data on the current state of distance education and the OU in particular with especial reference to its record of student retention and success.

Figure 1 shows the overall graduation rates of a number of higher education systems compared with some distance education institutions (Simpson 2013).

Figure 1 Some conventional and distance institutions’ graduation rates for 1997 entry compared

The selection of distance institutions are just those where graduation rates are relatively easily accessible publicly (many institutions do not release this data). But nevertheless the data appears to fairly represent the overall picture internationally.

It can be seen that there appears to be a substantial graduation gap between conventional UK face-to-face universities and many distance universities - the so-called ‘distance education deficit’ (Simpson, 2013 op cit). In the case of the OU the closest comparison is probably with the part-time UK student graduation rate. Here the difference is between 39% and 22% for UK part-time students and OU students respectively. Even more marked is the difference between the University of London International Programmes institution-supported students and exclusively distance-supported students at 61.5% and 15.7% respectively - see the case study later.

A more detailed picture of OU graduation rates suggests that they have apparently been falling ever since it first started. Its graduation rate has dropped from 59% for its first entry in 1972 steadily down to 22% for 1997 entry (Simpson, 2014) - Figure 2.

Page 3

Page 5: ducation - mrsite.comFace to face teachi…  · Web viewRumble (1992) pointed out the vulnerability of dedicated distance teaching universities to competition from dual mode universities

Face to face teaching in Distance Education 2014

Figure 2 Cumulative OU graduation rates by year of student entry

The decline appears to be continuing and the graduation rate for 2001-2 entry students is now apparently down to around 14%. Figure 3 gives a clearer picture of the trend.

19731976

19791982

19851988

19911994

19972000

20032007

0

10

20

30

40

50

60

7059

5248

2214

Figure 3 OU final graduation rates by year of student entry

There may be many reasons for this apparent steady decline. Certainly the earlier OU graduation rates reflected the pent-up demand from well-qualified students (mostly teachers) well-motivated to gain a degree qualification. But that does not really apply to entry after the early eighties since when the previous entry qualifications of students have remained more or less constant. Whatever the complexity of the reasons for this drop, it does not appear that the increasing introduction of e-teaching from the late 1990’s has been particularly successful in increasing student success.

Literature review

The search term ‘Face to face teaching in distance education’ produces nearly 4 million hits on Google. Related terms such as ‘Blended and distance teaching/learning’ produce a similar number

Page 4

Page 6: ducation - mrsite.comFace to face teachi…  · Web viewRumble (1992) pointed out the vulnerability of dedicated distance teaching universities to competition from dual mode universities

Face to face teaching in Distance Education 2014

although with some overlap. It is quite difficult therefore to make sense of this vast range of findings. This survey has therefore tended to focus on reports of ‘meta-surveys’ - which are essentially surveys of surveys - and has concentrated on a number of different aspects as below.

OU Research into f2f teachingIt appears much OU current research is going into e-teaching. There appears to be very little OU research focused specifically on face to face teaching. For example there are few mentions on the OU’s Open Research Online and the most recent mention on the IET Knowledge Network (which closes on 30 September 2014) is a review by Burt dated 1997, ‘Face to face with distance education’ but which came to the conclusion that more research was needed. Even where blended teaching is specifically addressed (Cameron et al KN 2003) the focus tends to be on other media - video, phone and correspondence rather an examination of specifically f2f contact.

The IET PLUM (‘Programme on learner use of media’) group was closed in 2005 and it is not clear how much attention has been paid to f2f teaching in internal OU research since then. In addition the End of Module Survey issued by the OU Survey Office does not ask a specific question about f2f tutorials. However the remit in this survey was to focus on international findings on f2f teaching: it is clear that there is immense scope for further research within the OU itself.

Blended, traditional and exclusively online teaching.

International studies tend to fall into comparisons between two of three types of teaching - f2f or traditional classroom teaching, ‘exclusive’ online teaching which only uses online methods, and blended or hybrid teaching which combines both - table 2.

Table 1 The range of teaching modes (from Allen, 2013)

While of course it still uses blended teaching, from an Associate Lecturer perspective the Open University currently appears to be moving increasingly towards the online end of this spectrum.

Some educators make considerable claims for blended teaching. Bauk and Scepanovic (2014) claim that blended teaching is now the most popular educational model for teaching, and assert that

Page 5

Page 7: ducation - mrsite.comFace to face teachi…  · Web viewRumble (1992) pointed out the vulnerability of dedicated distance teaching universities to competition from dual mode universities

Face to face teaching in Distance Education 2014

institutions see benefits through increased retention rates and an upsurge in popularity. However their argument seems to apply particularly to dual mode institutions and the evidence in their paper lacks firm data.

Research into blended teaching.

Research into blended teaching tends to focus on comparisons between various forms of distance teaching such as:

1. F2f with exclusive online distance modes 2. Blended with exclusive online modes3. Blended teaching with f2f modes

But there are other aspects worth noting such as the timing of teaching, students’ learning skills and preferences and so on.

Comparisons of blended teaching. Most of the studies detected in this survey compare blended and traditional teaching. One of the most important is the ‘Evaluation of Evidence-Based Practices in Online Learning’ from the US Department of education (2010). This meta-survey of some 50 studies which were deemed to be of sufficient rigour, reported that exclusive online instruction had an edge over classroom instruction, but that both modes of delivery were less effective singly than a “blended” mode using elements of both.

However this report also noted that whilst the studies in the meta-analysis were strong (i.e. experimental or controlled quasi-experimental), many of them ‘suffered from weaknesses such as small sample sizes [and] failure to report retention rates for students in the conditions being contrasted’. For example Dell et al (2009) compared student achievement in online and f2f classes by looking at submitted grades on both graduate and undergraduate courses and found no difference. Yet the report appeared to make no record of the comparative retention rates. This latter point is particularly relevant: many studies identified in this survey appeared to focus on what gave students the ‘best learning experience’ whilst ignoring retention rates. This misses the point: after all the best learning experience a student can have is to pass their course.

Another study which compared exclusive online teaching with f2f teaching was by Bergstrand and Savage (2013) whose results of ‘a series of hierarchical linear models indicated that students felt they had learned less in online courses, believed they were treated with more respect in in-class courses, and rated online courses less highly than in-class courses’. Jaggars (2014) studied community college students who discussed their experiences with online and face-to-face learning as well as their reasons for selecting online (rather than face-to-face) sections of specific courses. Students reported lower levels of instructor presence in online courses and that they needed to ‘teach themselves.’ Accordingly, most students preferred to take only ‘easy’ academic subjects online; they preferred to take ‘difficult or ‘important’ subjects face-to-face.

The most important meta studies may be found on the ‘No significant difference’ website www.nosignificantdifference.org which collates a large number of studies comparing f2f teaching with distance teaching. For example Rovai and Jordan (2006) found evidence to suggest that blended courses produce a stronger sense of community among students than either traditional or fully online courses. There were a number of other meta-surveys which compared blended or hybrid teaching with online and f2f teaching such as Xu and Jaggars (2011) who found that after

Page 6

Page 8: ducation - mrsite.comFace to face teachi…  · Web viewRumble (1992) pointed out the vulnerability of dedicated distance teaching universities to competition from dual mode universities

Face to face teaching in Distance Education 2014

controlling for student characteristics, results indicated that students were more likely to fail or withdraw from exclusive online courses than from face-to-face or hybrid courses.

A small but useful comparison was made by Burns (2013) in a study in Indonesia. A course for 60 primary school educators was presented in three ways - fully online, hybrid (50% each online and face-to-face) and web-facilitated (25% online and 75% f2f) with 20 learners in each. She found that the purely online group experienced a 31% attrition rate while 100% of the other two groups completed the programme. A survey suggested that the greatest factor impacting the attrition in the online programme was the absence of f2f interaction with instructors and other students. It is not possible to draw global conclusions from a small scale study in a country where internet connections are sometimes poor, but it is an excellent example of the kind of comparative study that should be undertaken before investing heavily in one teaching media at the expense of another.

Timing of f2f and blended teaching. There is some evidence that some initial f2f component is a positive element in enhancing subsequent online interaction. Haythornthwaite et al (2000) studied a part-time online Masters degree which started with a f2f ‘boot-camp’ and found that the f2f period helped students get a clear sense of the others in the community. This may be an important finding for the OU: Kear (2010) found concerns amongst OU students about a lack of social presence in online forums and an interesting finding from the international case studies (qv) is that the Dutch OU, Dublin City University, FernUniversität in Germany and the Korean National Open University start their courses with f2f sessions (in the case of both DCU and KNOU these are compulsory).

Clearly a student’s first impression is important in their subsequent experience: research amongst full-time students has recently found that universities who fail to make new students feel welcome suffer higher dropout rates than those who make a good first impression (THE 04/09/2014).

Student skills for online learning. There has long been a belief in distance education institutions that young people brought up in the age of the internet (the ‘Google Generation’) have a high level of skill in using the Web for learning. However a report commissioned by JISC from the Centre for Information Behaviour and the Evaluation of Research (CIBER) research team at University College London (Rowlands et al, 2008) claims that although young people demonstrate an ease and familiarity with computers, they ‘rely on the most basic search tools and do not possess the critical and analytical skills to assess the information that they find on the web’. The report suggests that ‘research-behaviour traits that are commonly associated with younger users – impatience in search and navigation, and zero tolerance for any delay in satisfying their information needs – are now the norm for all age-groups, from younger pupils and undergraduates through to professors’ […!]. A more recent study (Garcia et al 2013) also found that ‘although most university students have a basic set of technological abilities, these do not necessarily translate into sophisticated skills in the use of other technologies or information literacy in general’.

Student preferences. There were a few studies which looked at what students actually preferred such as Miliszewska (2007) who found that students preferred a blended model of teaching to a exclusively online approach. Similarly Shakker and Hu (2008) found that students offered f2f and online teaching were more satisfied than students just offered online teaching and the KNOU f2f sessions were introduced due to students demand (qv).

Jakobsdóttir (2008) in Iceland found that ‘students who liked online asynchronous discussions also tended to like to spend campus time for f2f discussions. There are pros and cons to both types and

Page 7

Page 9: ducation - mrsite.comFace to face teachi…  · Web viewRumble (1992) pointed out the vulnerability of dedicated distance teaching universities to competition from dual mode universities

Face to face teaching in Distance Education 2014

use of one type does, at least not yet, cancel out a need for the other. Other uses of campus time may depend on the type of course and subject (and the teacher) but hands-on experiences and creating a good group spirit should be high on the list’.

Stodel et al (2006) found that learners reported that they miss face-to-face contact when learning online. An investigation into what they missed identifed five themes: robustness of online dialogue, spontaneity and improvisation, perceiving and being perceived by the other, getting to know others, and learning to be an online learner.

Full-time faculty preferences. Much of the literature on teaching faculty preferences concerns resistance from faculty to using new online methods. This resistance is usually put down to technophobia or just innate conservatism. There is seldom a specifically analysis of what faculty reasons are for such resistance, or whether such reasons are well-founded.

Adjunct faculty/part-time tutors/OU Associate Lecturer preferences. There are few findings about the teaching preferences of part-time staff. There is an old (pre-e-teaching) finding from Cooper (1993) who found that the f2f teaching was the most satisfying and motivating elements of an OU AL’s work, but there appears to be no later work on how OU ALs view the current teaching methods they are requested to use. This is important as AL motivation could be a key element in their efforts to maintain retention amongst their students.

Time issues for faculty. Some educators believe that wholly online teaching requires more time from faculty both full and part-time, than f2f teaching. Vord and Pogue (2012) found that, overall, f2f teaching took a little time longer than online teaching, but that certain aspects of online teaching such as assessment took considerably longer than the f2f . Such time comparisons are part of the wider and vital consideration of the costs and benefits of f2f teaching in distance education - see below.

Time issues for students. There appears to be very little research into comparative time issues between blended and online teaching for students. Given that the biggest single issue identified by students is often their lack of time to study this is surprising, although admittedly the research would be difficult. Whilst on the face of it e-learning avoids the problems of f2f session travel time, working with computers is not always trouble-free, and once at a f2f tutorial interactions are likely to be far speedier and richer than similar exchanges online.

Trends. There are also a few studies which asked what the trends in distance education are likely to be - i.e. is there a trend to exclusively online courses in distance education? The answer, at least in the USA, is that there is little evidence that there is a move from blended to exclusively online instruction modes. There are far more blended courses and programs being offered than would be present if institutions were using them only as a transition to fully online.

Other media for blended teaching. Of course there are other media used in teaching in distance education apart from f2f contact. For example the OU Access Programme uses a mix of phone and email contact from tutors to students with no f2f teaching. The % pass rate for the latest presentation of the three Access courses in 2013 together with the pass rates for course modules on similar topics in the undergraduate programme is shown in Table 2.

Access Course modules Undergraduate Level 1 modulesCourse module Pass rate Course module Pass rate

Y031 Arts and Languages 67.4% AA100 Arts Past and Present 66.8%

Page 8

Page 10: ducation - mrsite.comFace to face teachi…  · Web viewRumble (1992) pointed out the vulnerability of dedicated distance teaching universities to competition from dual mode universities

Face to face teaching in Distance Education 2014

Y032 People Work and Society 63.8% DD101 Social Sciences 59.2%Y033 Science, Technology and Maths 66.5% MU123 Discovering mathematics 66.9%

Table 2 Comparative pass rates on a selection of course modules in 2013.

For this selection of modules the pass rates are not very different. The z-scores (a measure of how successful a course is in retention terms given the entry level of its students) for the Access courses are not yet available, but the previous Openings course which had a similar model of phone and email teaching appeared to have much the same range of z-scores as undergraduate courses.

However there are too many variables for this data to suggest that phone and email teaching are clearly as good as f2f teaching. But the data does suggest that there might be a treasure house of data in the OU which could be analysed to get a clearer idea of the retention value of f2f teaching.

Costs and benefits of f2f and online teaching.

Finally, and perhaps the most important for this study are the comparative costs of blended and online teaching. But as Rumble (2014) remarks ‘there are very few comparative studies that that allow one to say with any degree of certainly what the cost implication of a particular socio-technological design will be’.

That certainly turns out to be the case in this study where the search found very little evidence of any effective cost-benefit analysis of the relative financial consequences of f2f and online teaching. This is remarkable given that so many decisions in universities are made on the basis of cutting costs without a full analysis of the consequences. It is clear that spending money on students which increases their success and retention actually produces income or savings benefits to the institution because such students will go on to pay further fees.3 That income can exceed the initial expenditure - that is, it can have a cost benefit ratio of more than one hundred percent.

For example, this author has shown that a simple phone call to new students in the OU can produce a retention increase at the end of their first module which has a cost benefit ratio of up to 500% (Simpson, 2014 op cit). In other words the benefit of the increased retention due to the activity far exceeds the cost of the activity. Applied to all 30,000 new OU students each year this increase in income could amount to several million pounds.

Inversely, a cut in support to students designed to save money but which results in a reduction in retention may actually create an overall loss of income, contrary to the intentions of the authors of the cut. This could be one of the causes, in some part, of the reduction in graduation rates experienced by the OU in the last twenty years. For example cuts were made in the system of continuity of student support which could have affected retention between course modules. It may be that the effects of these cuts are now having to be reversed in the form of the establishment of the University’s ‘Student Support Teams’ who have a remit to support inter-module retention amongst other tasks.

3 The possibility of this saving is increasingly recognised in full-time higher education in the UK. Thomas (2012) noted that each new student who withdraws in the first semester at a full time institution costs the institution at least £24,300 in lost income over the duration of a three year degree course.

Page 9

Page 11: ducation - mrsite.comFace to face teachi…  · Web viewRumble (1992) pointed out the vulnerability of dedicated distance teaching universities to competition from dual mode universities

Face to face teaching in Distance Education 2014

Thus without better analysis of the cost-benefit effects of f2f or blended teaching on retention within an institution it will be dangerous to modify such teaching. Unfortunately this survey has been unable to find any examples of such analyses.

There are general investigations into costs in distance education and, for example, both Rumble (2004) and Hulsmann (2000) both claim that when overheads are taken into account e-teaching is not less expensive to institutions than conventional teaching. Nor is e-learning necessarily a zero cost to students when amortised costs of computers, peripherals and internet access are taken into account, although f2f learning also has its costs to students with transport to tutorials. Again this survey has found no studies which address these issues.

It may be possible to isolate the effects of OU tuition in distance education by noting that the OU’s MOOC ‘FutureLearn’ which uses OU course material with no tuition, has a completion rate of around 12%. This compares with the same material in undergraduate courses taught by Associate Lecturers which have completion rates of around 60%, a difference of nearly 50% points.

Of course FutureLearn courses do not lead to qualifications, but on the other hand the courses are much shorter than undergraduate courses (only a matter of a few weeks compared with several months), and around 80% of the FutureLearn students have degrees - a far higher proportion than new OU undergraduates. Thus it maybe that blended teaching - in whatever forms it takes - does add a very substantial retention value to the OU. It’s very probable that the benefit of that value to the OU in terms of increased subsequent fee income will far outweigh its cost (Simpson 2006).

Page 10

Page 12: ducation - mrsite.comFace to face teachi…  · Web viewRumble (1992) pointed out the vulnerability of dedicated distance teaching universities to competition from dual mode universities

Face to face teaching in Distance Education 2014

International Case Studies

The OU has always tended to regard itself as being at the forefront of progress in distance education - a world leader in the field. However it may still have lessons to learn from other distance institutions. Thus, as a follow-up to the literature review, a number of distance institutions were contacted to enquire about their current and future polices with regards to their f2f teaching operations and regional structures. The selection of institutions was necessarily somewhat random, but they cover a wide range of types. There are replies or information from the following institutions.

1. The University of London International Programmes (ULIP)

The University of London International Programmes is a recent rebranding of the London External Degree which has been in existence for more than 150 years. It has more than 50,000 students in more than 50 countries.

When surveying the role of f2f teaching in distance education ULIP is of particular interest. The programmes are presented in two modes - the first mode entirely at a distance with no f2f teaching and limited online support via email and a VLE, and the second mode which is supported with f2f teaching at local approved institutions. These two modes have very different outcomes - see Figure 3. The f2f mode graduation rate is 61.5% and the distance mode rate is 15.7%, a difference of 45.8% points, the most substantial distance education deficit known to me.

Both modes have identical entry requirements, syllabuses, texts and exams, so the difference in graduation rates between them seems likely to be largely down to the different delivery modes, and in particular to the role of f2f contact in the local approved institution mode. It may not be possible to exactly determine whether that f2f contact is most importantly between teachers and students, or students and students and may well be a mix of both. Nevertheless this data seems to me to be one of the most important pieces of evidence supporting the possibility that it is the lack of the f2f mode that explains a substantial part of the distance education deficit, at least in the ULIP situation.

The extra costs of the f2f mode for the ULIP are borne by the students in the form of the fees to the teaching institutions (students choose the f2f option over the exclusive distance option in roughly the ratio of 4:3, although that must reflect the availability of the f2f option locally).

As an example of comparisons, see Table 3 for the annual fees for the three year law degree (LLB) from two ULIP approved institutions and the OU.

Institute of Law, Jersey Kaplan Holborn College London OUF2f Contact hours

(All these institutions offer

online support via a VLE)

Weekly and all weekend f2f tutorials - total hours not

specified

Weekly f2f tutorials -15 contact f2f hours per week

20 hours per presentation

Annual institutional fee

£4800 £6000

ULIP fee £1700 £1700Total annual fees £6500 £7700 £5200

Table 3 Fees and contact comparisons between ULIP approved institutions and the OU

Page 11

Page 13: ducation - mrsite.comFace to face teachi…  · Web viewRumble (1992) pointed out the vulnerability of dedicated distance teaching universities to competition from dual mode universities

Face to face teaching in Distance Education 2014

The OU’s fees are a little cheaper than the total fees charged jointly by these institutions and ULIP. However it appears that the f2f hours offered by these institutions are considerably greater. This table is included in the report as:

i. It may be one of the few situations where it may be possible to get some kind of estimate of the extra cost of f2f teaching over a exclusively online support - an average of very roughly £5000 per year in the example of the two ULIP-approved institutions.

ii. It may give some idea of the possible competition to the OU. If the OU moves away from f2f teaching whilst students appear to prefer that mode, then that move may well affect recruitment as students choose the competition.

ULIP has no plans to change its mixed model although there are signs of concerns about the need to supply a better level of support to its exclusively online students in future.

2. Dutch Open University

My contact says “Also here at our OU there is a discussion about regional centres. The board is going to put our intentions in a paper which is going to be published around the middle of October.4

“At the moment we still have f2f sessions in the study centres and they are on the increase but always combined with virtual/online sessions. The basic idea is to start with a f2f introduction, a couple of collaborate sessions and to finish off with a f2f session.

“Often only 2 or 3 of the regional centres are involved in f2f sessions. The organization is done in Heerlen. Most centres are only used for exams and information.”

3. Indira Ghandi National Open University (IGNOU)

IGNOU used to offer some programmes through a f2f mode (like their MSc in Chemistry). However it was recently (June 2013) decided that, as the mandate of IGNOU was to teach only through the distance education mode, a decision should be taken to withdraw all courses taught through f2f contact. Currently there is no f2f teaching, except for services continuing for old students to help them complete their degree.

Nevertheless IGNOU maintains a network of Regional Centres (and under them Study Centres). The Study Centres are grassroots contact points for students where they go for study and practical and library or other kind of student support. IGNOU has a network of 67 regional centres, around 2,667 learner support centres

4. University of South Africa (UNISA)

UNISA still undertakes f2f teaching through a network of Study Centres. Though my informant says (quote) “However, e-tutors are the in thing, moving ahead fast!”

5. The Korean National Open University

The Korean National Open University (KNOU) based in Seoul is another useful case-study as South Korea is a roughly similar size to the UK in area and population, but has a much higher level of high speed internet access. Nevertheless KNOU (a similar size to the OU with nearly 200,000 students) still maintains a substantial face to face teaching programme based in 13 campuses around the country5.

4 I have requested a copy of this when it is available, but it may be too late for this report

Page 12

Page 14: ducation - mrsite.comFace to face teachi…  · Web viewRumble (1992) pointed out the vulnerability of dedicated distance teaching universities to competition from dual mode universities

Face to face teaching in Distance Education 2014

As the KNOU introductory 2014 guideline manual for freshmen states:

‘To overcome limitations in distant education and to encourage interactions between faculty members and students and also among students, KNOU requires its freshmen through juniors to take face to face classes for three courses, and seniors are required to take face to face classes for three courses (except for Early Childhood Education majors) at their regional campuses (generally students are supposed to take 6 courses a semester). These face to face classes are held consecutively for two or three days in a semester and there are tests at the end of a course.’

The decision to retain the f2f facilities was partly made on the results of a student survey (see Appendix 3). In the survey students responded that they valued the f2f sessions mostly because of difficulties in understanding study materials on their own. They also wanted to overcome feelings of isolation from studying alone.

Some 50% felt the current proportion of f2f sessions was appropriate whilst nearly 40% wanted more.

6. Dublin City University Connected

The KNOU approach is similar to that of Dublin City University Connected (DCU Connected) in Ireland who comment:

‘On our undergraduate programmes, we offer students the options of all tutorials f2f, all tutorials online or a mixture of both. Due to concerns about socialisation, we insist new students take the 'mixture' option.’

They further comment:

‘The future of f2f teaching is something which greatly exercises us. There is not the level of online interaction in online classrooms that one gets in f2f classes and, as mentioned above, we are concerned about socialisation.’

7. The Open Polytechnic of New Zealand

OPNZ undertakes a variety of f2f teaching activities. The B Teaching (Early Childhood Education - ECE) and Bachelor of Social Work (BSW) students are required to participate in face-to-face workshops (and assessments) throughout their studies. A small number of communication and psychology courses run compulsory face-to-face assessment workshops. ECE and Social Work students are also required to complete several overnight stays at maraes (cultural centres for Maoris) where they gain face-to-face teaching and experiences on Maori culture.

On campus at Lower Hutt some lecturers run optional tutorial sessions for students for some accounting, and statistics courses. There are also weekly study support sessions for some of our trades courses and horticulture programmes have block courses run at hired venues.

For Engineering (B Engineering Technology/NZ Diploma in Engineering): Only the workshop block courses held at other polytechnics, and at the University of Southern Queensland in Australia continue (these qualifications require a practical f2f component).

5 The Regional Campuses are substantial edifices: the one at Incheon which I visited occupies the whole of a big three story building with at least six large teaching rooms in use day and evening, together with a crèche and substantial IT centre and library.

Page 13

Page 15: ducation - mrsite.comFace to face teachi…  · Web viewRumble (1992) pointed out the vulnerability of dedicated distance teaching universities to competition from dual mode universities

Face to face teaching in Distance Education 2014

For Trades: Some voluntary support drop-in sessions are run across the country (this is in areas that either Industry Training Organisations or apprentices have specifically requested it)

A blended model is still preferred by our partners/clients and also appropriate for some learners in the Primary Industries. Examples would include: Horticulture and Agriculture which are distance and supported by face-to-face workshops.

From 2015 level 1 and 2 programmes courses require f2f teaching to maintain funding allocation from the Government. F2f components are also an accreditation requirement for initial teacher education so will continue. This includes a mix of practicum visits to ECE services, workshops and tutorials. This is supported by Open Polytechnic lecturers who are regionally based. ECE also has an office in Christchurch and Auckland. However OPNZ no longer has a network of regional centres. We have offices in Auckland and Christchurch which are the area base for our network of off campus regional lecturers in ECE and a contact centre for some face to face real estate assessments. Our practice is to hire venues in areas most practical for the location and requirements of particular programmes. For B Teaching (ECE), our School of Education Studies organises the venues, which could be a conference centre, hotel venue or some other suitable space. For horticulture courses we use a community facility in the centre of the North Island.

Our agreement with the Department of Corrections sees us send tutors into prisons to provide f2f teaching for level 1&2 courses. We also hire community venues for other Level 1&2 courses that must meet the Government’s requirement to have a f2f component for this level of study. Our model of delivery continues to be open and distance learning in the main with some additional specialist face to face workshops/sessions/study support for some courses/programmes. As long as the funding rules for L1 and 2 require f2f and the Polytechnic wishes to keep teaching at this level then f2f teaching will be maintained.

8. FernUniversität - Germany

Study at the FernUniversität is based on a blended learning concept with written study materials, ICT and media and f2f-teaching and learning. F2f teaching takes place mainly at the campus in Hagen and in regional centers. Use of the virtual classroom or hybrid facilities (real and virtual combined) is increasing.

The FernUniversität has 13 regional centers and 19 study centers. Study centers in Germany belong to their nearest regional center. One Study center is in Hungary (Budapest). We cooperate with the Austrian university of Linz (= 6 study centers and agencies). We offer support in Switzerland.http://www.fernuni-hagen.de/regionalzentren/

There are a) compulsory seminars and b) optional tutorials or other supportive events and lectures. The number of compulsory seminars depends on the study programs. We try to keep them to a minimum so there are probably one or two during a bachelor or master program. They take place in regional centers or on the campus in Hagen or elsewhere (special educational establishments which offer seminar rooms, accommodation and boarding).

More and more the academic staff prefer regional centers, because they find all the facilities they need and get support by the staff of the center.

The regional centers offer tutorial sessions and counselling to the students. For example see the tutorials offered in the regional center in Berlin

Page 14

Page 16: ducation - mrsite.comFace to face teachi…  · Web viewRumble (1992) pointed out the vulnerability of dedicated distance teaching universities to competition from dual mode universities

Face to face teaching in Distance Education 2014

under http://www.fernuni-hagen.de/regionalzentren/ and all events underhttp://www.fernuni-hagen.de/stz/berlin/veranstaltungen/.

Teaching is organized by the faculty. They decide who becomes a mentor or tutor. The regional center does the on-the-spot organization. Regional Centers belong to a unit of the university administration but they have close contact to the faculties and to the faculty staff.

The FernUniversität intends to maintain that network and f2f teaching having reorganized the network of study centers into this current structure of regional centers (with a reduced number of study centers) about 10 years ago.

9. Colorado State University (CSU-global)

CSU is a private exclusively online institution (although it will experiment with some hybrid teaching in 2015). It takes students who have started degrees elsewhere and has around 12,000 students on 27 degree programmes. It runs 8 week courses which start every month and offers 24/7 live tutoring outsourced to Pearson.

CSU believes that new students need an intensive orientation programme right at the beginning of their enrollment. This consist of what is described as a ’boot-camp’ - a synchronous but recorded online ‘lecture’ from their faculty with a phone call which uses a process called MAPS - examining a student’s Motivation, Admissibility, Payment ability and allocating a Start date. This looks a little like a more intensive version of the OU’s ‘Proactive Motivational Support’ programme.

Thus although CSU describes itself as being 100% online it makes extensive use of the phone for both its orientation and teaching. The Provost and Senior Vice President Dr. Jon Bellum believes that “Retention is about people: technology can help cut costs, but it can’t do it all.”

Page 15

Page 17: ducation - mrsite.comFace to face teachi…  · Web viewRumble (1992) pointed out the vulnerability of dedicated distance teaching universities to competition from dual mode universities

Face to face teaching in Distance Education 2014

Conclusions

Doing the research. It will always be challenging to get clear findings from educational research, given that it is almost impossible ethically to devise the large controlled randomized trials characteristic of medical research. When relying on a literature search for evidence probably the best that can be achieved is to find meta-surveys - surveys of surveys - which attempt to control for reliability and experimental validity.

But such searches still throw up a vast literature from which it is difficult to draw strong conclusions. In addition there are difficulties about the best way to measure the effectiveness of different modes of delivery in distance education. Some evaluations use questionnaires to measure student satisfaction, but this seems to me to be inadequate as it is not always clear what the survey population is. If, like the UK National Student Survey, the population is those students who survived to near the end of a course then it is hardly a fair sample. A better measure would be to measure the survival rate and ask the casualties of the process as well.

So this survey has tried to focus on those studies in which the main focus is on student retention. Certainly many studies will have been missed, but using this criterion the following conclusions can be reliably drawn from both the literature and institutional case studies.

Blended teaching. It is clear from the research that for distance institutions to succeed in increasing student retention they need to invest in blended teaching in some form. In particular they must find ways of orienting and integrating new students at the earliest possible point in their studies. They must also try to ensure that students are on the right course and are as well prepared with as many sources of initial anxiety dealt with as possible. An institution which relied solely on online teaching via computer forums and a VLE is likely to do little better than MOOCs where course module completion rates are around 10% (THE 19 June 2014), even given the shortness of the courses and the high level pf previous education of participants. Since a MOOC degree will be made up of a series of such modules each perhaps with a similar dropout rate, the graduation rate of such an institution is likely to be negligibly small.

Media for blended teaching. It is less clear what media are best for blended teaching, whether it is email, phone (including text messaging), f2f or other contact. There appears little research into the relative effectiveness or costs of different media. Even where it would appear possible to make comparisons between different institutions this is not really possible as there are so many other competing variables apart from the media used for blended teaching.

Given that difficulty, perhaps the best evidence that can be offered for the effectiveness of f2f teaching is that the case studies in this report show a number of distance institutions still insist on it. Indeed institutions as different as the Korean National Open University, Dublin City University (Connected) and the FernUniversität in Germany not only offer f2f teaching but make it compulsory. KNOU is a particularly interesting example given the very high level of high-speed broadband in that country.

But perhaps the choice of one media over another does not matter. The biggest variable in any educational system are the students. As Thomas (2012 op cit) notes about retention amongst UK full-time students, “Early engagement is essential to student retention and success. Information may be better delivered via a range of media, as students’ learning styles and needs will differ from

Page 16

Page 18: ducation - mrsite.comFace to face teachi…  · Web viewRumble (1992) pointed out the vulnerability of dedicated distance teaching universities to competition from dual mode universities

Face to face teaching in Distance Education 2014

each other and over time.” So perhaps it is important to use a range of media to support blended learning so that students have a choice of what works best for them. It may not matter therefore if not all students use f2f teaching when it is offered. What matters is if those who do not want it, or cannot avail themselves of it for whatever reasons, have alternative choices of media for support. But note that the evidence strongly suggests that it is especially important to ensure that there is f2f teaching in the first few weeks of a module to enhance student engagement and allay anxieties, even if that is followed by more online teaching activity.

Costs and benefits of various forms of blended teaching. But choice is expensive and one of the disappointments of the findings from this survey is the absence of attempts at the analysis of the costs and benefits of various forms of blended teaching. As noted earlier Rumble and Hulsmann (op cit) maintain that e-teaching is as expensive as ordinary correspondence teaching, but there appear to be few more recent follow-up studies. In particular there appear to be no studies which link blended learning provision to retention outcomes with cost-effective results.

There is evidence for the cost-effectiveness of blended teaching in the form of outreach phone contact (Simpson 2014 op cit), but this is only a small example of the kind of analysis that is needed to decide on the relative proportions in the blend. Within the OU it shouldn’t be too difficult to compare the costs of an online tutorial with a f2f tutorial but it will be more difficult to link that with retention outcomes.

Imponderables. Apart from the effects of f2f teaching on student success there is also the effect on student recruitment which needs analysis as well. Does the availability of local f2f teaching act as a motivator for new student enrolment? Indeed does the existence of f2f teaching act as a motivator for existing students whether they use it or not? Might merely knowing that f2f help is available increase the sense of confidence and security amongst new students even if they then don’t use it? And what is the effect on the morale and commitment of Associate Lecturers of a reduction in the f2f element of their work?

Future research. It is something of a traditional get-out of any survey to end with a conclusion calling for more research. But without such detailed analyses the only possible conclusion that can be drawn from this survey is that it is imperative to maintain blended teaching in its various forms, including f2f teaching. Certainly it would be dangerous to alter the mix fundamentally without considerable additional research into the topics outlined above.

Page 17

Page 19: ducation - mrsite.comFace to face teachi…  · Web viewRumble (1992) pointed out the vulnerability of dedicated distance teaching universities to competition from dual mode universities

Face to face teaching in Distance Education 2014

Appendix 1

Institutions and individuals contacted in the survey

Dutch Open University - Dr. Kathleen H.L.A. Schlusmans

University of South Africa - Mrs Hentie Wilson, Education Consultant, Directorate: Curriculum & Learning Development,

Indira Ghandi National Open University, India - Professor Ramesh Sharma

Open Polytechnic of New Zealand - Leanne Rate, Communications Manager, OPNZ

Korean National Open University - Dr. Young-Sook Jung, Institute for Distance Education, KNOU

FernUniversität, Germany - Dr. Heike Brand

Dublin City University - Seamus Fox

Colorado State University, Denver - Dr. Jon Bellum, Provost and Vice President

University of London International Programmes - this author

Appendix 2.

Some of the Journals and research repositories accessed (but not necessarily quoted)

American Journal of Distance Education

International Review of Research on Open and Distance Learning

Open Research Online - OU

Knowledge Network - OU

‘No significant difference’ website

Open Learning

Distance Learning (Australia)

European Journal of Open, Distance and E-Learning

Academia.edu

Asian Journal of Distance Education

Page 18

Page 20: ducation - mrsite.comFace to face teachi…  · Web viewRumble (1992) pointed out the vulnerability of dedicated distance teaching universities to competition from dual mode universities

Face to face teaching in Distance Education 2014

Appendix 3 Results of a Korean National Open University Student Survey6

Dr. Young-Sook Jung, Institute of Distance Education, Korea National Open University

Q1. Why do you attend f2f classes (sessions)?1) To get help with difficulties in understanding study materials alone 2) To think that taking the test for f2f classes (eg. essays) is easier than the substitution test

(eg. Multiple choice questions)3) To overcome a feeling of loneliness from studying alone4) To think it is a part of the school’s official curriculum5) Others (Please, specify your opinion)

Results of Q1 The results of question No. 1 in 2012 are shown below in comparison with that in 2010. From the results, the graphs are drawn in order of choices listed above, from 1) to 5).

Q2 What is your opinion on the current portion of f2f classes (sessions) in the curriculum?

1) It should be expanded2) It is appropriate now 3) It should be reduced4) It should be abolished

Results of Q2 The results of question No. 2 in 2012 are shown below. From the results, the graphs are drawn in order of choices listed above, from 1) to 4).

6 I have not changed the language or graphs in any way - Ormond

Page 19

Page 21: ducation - mrsite.comFace to face teachi…  · Web viewRumble (1992) pointed out the vulnerability of dedicated distance teaching universities to competition from dual mode universities

Face to face teaching in Distance Education 2014

References

Allen, Seaman and Garrett (2013) Blending in: the promise and extent of blended education in the US’ http://onlinelearningconsortium.org/publications/survey/blended06

Bergstrand and Savage (2013) ‘The Chalkboard Versus the Avatar: Comparing the Effectiveness of Online and In-class Courses.’ Teaching Sociology July 2013 vol. 41 no. 3 294-306 http://tso.sagepub.com/content/41/3/294.abstract

Bellum, J. Presentation at the Centre for Distance Education, University of London. 24 September 2014

Burns, M. (2013) ‘Staying or leaving? Designing for persistence in an online educator programme in Indonesia’ Open Learning Vol 28 no. 2 pp141-152

Burt (1987 updated 2003) ‘Face to face with distance education: a literature review’ OU Knowledge Network (closed 30 September 2014).

Dell, Low and Wilker (2010) ‘Comparing Student Achievement in Online and Face-to-Face Class Formats’ MERLOT Journal of Online Learning and Teaching Vol. 6, No. 1,http://jolt.merlot.org/vol6no1/dell_0310.pdf

Garcia, Escofet, Gros (2013) ‘Students’ Attitude towards ICT Learning Uses: A Comparison between Digital Learners in Blended and Virtual Universities’ EURODL http://www.eurodl.org/index.php?p=special&sp=articles&inum=5&article=624

Haythornthwaite, Kazmer, Robins, and Shoemaker (2000) Community Development among Distance Learners: Temporal and Technological Dimensions. Journal of Computer-Mediated Communication (6) 1

Hulsmann, T. (2000). Costs of open learning: A handbook. Oldenburg, Denmark: Verlag, Bibliotheks und Informations system der Carl von Ossietsky Universitat.

Jaggars (2014) Choosing Between Online and Face-to-Face Courses: Community College Student Voices American Journal of Distance Education 28, 1

Jakobsdóttir (2008) The role of campus-sessions and face-to-face meetings in distance education EURODL http://www.eurodl.org/index.php?p=archives&year=2008&halfyear=2&article=348

Kear (2010) ‘Social presence in online learning communities’ Proceedings of the 7th International Conference on Networked Learning 2010, Edited by: Dirckinck-Holmfeld, Hodgson, Jones, de Laat, McConnell, Ryber. www.lancaster.ac.uk/fss/organisations/netlc/past/nlc2010/.../Kear.pdf

Miliszewska (2007) Is it fully on or partly off? Is it fully on or partly off? The case of fully-online provision of transnational education http://www.jite.org/documents/Vol6/JITEv6p499-514Miliszewska261.pdf

Rovai and Jordan (2004) ‘Blended Learning and Sense of Community: A comparative analysis with traditional and fully online graduate courses’ IRRODL http://www.irrodl.org/index.php/irrodl/article/view/192/274

Rowlands, I; Nicholas, D; Williams, P; Huntington, P; Fieldhouse, M; Gunter, B; Withey, R; Jamali, HR; Dobrowolski, T;Tenopir, C; (2008) The Google generation: the information behaviour of the

Page 20

Page 22: ducation - mrsite.comFace to face teachi…  · Web viewRumble (1992) pointed out the vulnerability of dedicated distance teaching universities to competition from dual mode universities

Face to face teaching in Distance Education 2014

researcher of the future. ASLIB PROC , 60 (4) 290 - 310. 10.1108/00012530810887953. http://discovery.ucl.ac.uk/177993/

Rumble, G (1992) The competitive vulnerability of distance teaching universities, Open Learning, 7 (2), pp 31–45

Rumble, G. (2004). E-education – whose benefits, whose costs? In Papers and debates on the economics and costs of distance and online learning. No. 7 of the series ‘Studien und Berichte der Arbeitsstelle Fernstudienforschung der Carl von Ossietsky Universitat, Oldenburg’ Germany. Retrieved from http://www.mde.uni-oldenburg.de/download/asfvolume7_ebook.pdf

Shanker and Hu (2008) Improving Teaching Effectiveness Using Distance Education Tools, http://personal.kent.edu/~mshanker/personal/Zip_files/06.pdf

Simpson (2006) Rescuing the Personal Tutor’, chapter in ‘Perspectives on Personal Tutoring in Mass Higher Education’ ed. Thomas and Hixenbaugh. Pub. Trentham Books

Simpson (2013) ‘Supporting students for success in online and distance education’. Routledge, New York ISBN 978-0-415-50910-7

Simpson (2014) ‘Student retention in distance education: are we failing our students?’ Open Learning 28 (2) http://www.tandfonline.com/doi/full/10.1080/02680513.2013.847363

Stodel, Thompson and MacDonald (2006) ‘Learners' Perspectives on What is Missing from Online Learning: Interpretations through the Community of Inquiry Framework’ IRRODL http://www.irrodl.org/index.php/irrodl/article/view/325/743

THE - Times Higher Education 4 September 2014 ‘Warm welcome sets tone for staying the course’ Elwell, p8

THE - Times Higher Education 19 June 2014 ‘FutureLearn delighted at response to initial MOOCs’, Parr, p20

Thomas, L. (2012) ‘What works? Student Retention and Success’ Paul Hamlyn Foundation, Action on Access, Higher Education Funding Council for England, Higher Education Academy

Vord and Pogue (2012) Teaching Time Investment: Does Online Really Take More Time than Face-to-Face? International Review of Research in Open and Distance Learning http://www.irrodl.org/index.php/irrodl/article/view/1190/2212

Woodley, A (2013) ‘‘Student dropout – the elephant in the room of distance education’ (with Simpson) chapter in “Online Distance Education - Toward a Research Agenda" pub. Athabasca University Press

Xu and Jaggars (2011), Online and Hybrid Course Enrollment and Performance in Washington State Community and Technical Colleges, http://files.eric.ed.gov/fulltext/ED517746.pdf

Page 21