evaluating the impact of a vle on learning and teaching? dr barbara newland learning design studio...
TRANSCRIPT
![Page 1: Evaluating the Impact of a VLE on Learning and Teaching? Dr Barbara Newland Learning Design Studio Academic Services Bournemouth University](https://reader035.vdocuments.us/reader035/viewer/2022070407/56649e175503460f94b038b1/html5/thumbnails/1.jpg)
Evaluating the Impact of a VLE on Learning and Teaching?
Dr Barbara NewlandLearning Design Studio
Academic ServicesBournemouth University
http://www.bournemouth.ac.uk/lds/[email protected]
![Page 2: Evaluating the Impact of a VLE on Learning and Teaching? Dr Barbara Newland Learning Design Studio Academic Services Bournemouth University](https://reader035.vdocuments.us/reader035/viewer/2022070407/56649e175503460f94b038b1/html5/thumbnails/2.jpg)
Overview Aims of evaluation Durham context Methodology Results Conclusions
![Page 3: Evaluating the Impact of a VLE on Learning and Teaching? Dr Barbara Newland Learning Design Studio Academic Services Bournemouth University](https://reader035.vdocuments.us/reader035/viewer/2022070407/56649e175503460f94b038b1/html5/thumbnails/3.jpg)
Aims of evaluation International widespread
adoption of VLEs Longitudinal study of impact
of a VLE on learning and teaching
Qualitative and quantitative methods
![Page 4: Evaluating the Impact of a VLE on Learning and Teaching? Dr Barbara Newland Learning Design Studio Academic Services Bournemouth University](https://reader035.vdocuments.us/reader035/viewer/2022070407/56649e175503460f94b038b1/html5/thumbnails/4.jpg)
University of Durham 670 academic and 280
academic-related staff 9400 undergraduates
and 2400 postgraduates Founded in 1832 Split campus Research led MIS – outsourced to
Unisys Learning Technologies
Team (LTT)
![Page 5: Evaluating the Impact of a VLE on Learning and Teaching? Dr Barbara Newland Learning Design Studio Academic Services Bournemouth University](https://reader035.vdocuments.us/reader035/viewer/2022070407/56649e175503460f94b038b1/html5/thumbnails/5.jpg)
Images of Durham
![Page 6: Evaluating the Impact of a VLE on Learning and Teaching? Dr Barbara Newland Learning Design Studio Academic Services Bournemouth University](https://reader035.vdocuments.us/reader035/viewer/2022070407/56649e175503460f94b038b1/html5/thumbnails/6.jpg)
Learning Technologies Team aims to encourage, support and develop
elearning throughout the University by: providing advice, information and training supporting the University's learning
environment - duo (Durham University Online)
developing interactive materials 4 members of staff – Juliette Pavey, Kate
Boardman, Malcolm Murray, Victoria Boyd
![Page 7: Evaluating the Impact of a VLE on Learning and Teaching? Dr Barbara Newland Learning Design Studio Academic Services Bournemouth University](https://reader035.vdocuments.us/reader035/viewer/2022070407/56649e175503460f94b038b1/html5/thumbnails/7.jpg)
Duo Durham University Online http://duo.dur.ac.uk
![Page 8: Evaluating the Impact of a VLE on Learning and Teaching? Dr Barbara Newland Learning Design Studio Academic Services Bournemouth University](https://reader035.vdocuments.us/reader035/viewer/2022070407/56649e175503460f94b038b1/html5/thumbnails/8.jpg)
Uptake of duoEstimated use by Xmas 2000
Actual use by Xmas 2000
Use at June 2003
200 students3 departments
4286 students21 departments211 modules
10,000 students700 academics700 modules
![Page 9: Evaluating the Impact of a VLE on Learning and Teaching? Dr Barbara Newland Learning Design Studio Academic Services Bournemouth University](https://reader035.vdocuments.us/reader035/viewer/2022070407/56649e175503460f94b038b1/html5/thumbnails/9.jpg)
Implementation Varied patterns of departmental
implementation All courses All first year courses Individual academics
Administration courses Duo is now embedded into
learning and teaching
![Page 10: Evaluating the Impact of a VLE on Learning and Teaching? Dr Barbara Newland Learning Design Studio Academic Services Bournemouth University](https://reader035.vdocuments.us/reader035/viewer/2022070407/56649e175503460f94b038b1/html5/thumbnails/10.jpg)
Methodology for duo surveys Quantitative and qualitative Online surveys Analyse factors such as gender, year of
study and faculty 3 faculties – Arts and Humanities, Science,
Social Science and Health Relate to C&IT skills in learning and
teaching Data collected from students and staff
in all faculties and both campuses
![Page 11: Evaluating the Impact of a VLE on Learning and Teaching? Dr Barbara Newland Learning Design Studio Academic Services Bournemouth University](https://reader035.vdocuments.us/reader035/viewer/2022070407/56649e175503460f94b038b1/html5/thumbnails/11.jpg)
Response rate All students and staff are in duo so hard
to know the response rate in relation to active users which has increased over the years
Staff April 01 (n = 91)May 02 (n = 153)May 03 (n = 283, c. 40%)
Students May 01 (n = 793)May 02 (n = 2435)May 03 (n = 3668, c. 37%)
![Page 12: Evaluating the Impact of a VLE on Learning and Teaching? Dr Barbara Newland Learning Design Studio Academic Services Bournemouth University](https://reader035.vdocuments.us/reader035/viewer/2022070407/56649e175503460f94b038b1/html5/thumbnails/12.jpg)
Survey questions Similar questions for staff and students
in all 3 years to enable comparisons : between years between staff and students
Software ease of use levels of confidence features used
![Page 13: Evaluating the Impact of a VLE on Learning and Teaching? Dr Barbara Newland Learning Design Studio Academic Services Bournemouth University](https://reader035.vdocuments.us/reader035/viewer/2022070407/56649e175503460f94b038b1/html5/thumbnails/13.jpg)
Survey questions Frequency of use and access
during term time and vacations Learning and teaching
contribution effect
![Page 14: Evaluating the Impact of a VLE on Learning and Teaching? Dr Barbara Newland Learning Design Studio Academic Services Bournemouth University](https://reader035.vdocuments.us/reader035/viewer/2022070407/56649e175503460f94b038b1/html5/thumbnails/14.jpg)
Student responses – support, frequency and access Help documents
19%, 15%, 10% Small and decreasing use
Accessed duo at least once a week 70%, 71%, 71% Consistent pattern but increasing numbers over time
Accessed duo out of term time 60%, 61%, 64% Consistent pattern but increasing numbers over time
![Page 15: Evaluating the Impact of a VLE on Learning and Teaching? Dr Barbara Newland Learning Design Studio Academic Services Bournemouth University](https://reader035.vdocuments.us/reader035/viewer/2022070407/56649e175503460f94b038b1/html5/thumbnails/15.jpg)
Students: Overall, how do you rate the contribution duo has made to your learning?
6.5
53.1
26.6
2.1 1.4
0
10
20
30
40
50
60
%
excellent good nodifference
poor very poor
Contribution
![Page 16: Evaluating the Impact of a VLE on Learning and Teaching? Dr Barbara Newland Learning Design Studio Academic Services Bournemouth University](https://reader035.vdocuments.us/reader035/viewer/2022070407/56649e175503460f94b038b1/html5/thumbnails/16.jpg)
Staff: Overall, how do you rate the effectiveness of duo as a learning resource?
16.3
48.1
11.7
3.2 1.1
0
10
20
30
40
50
60
%
excellent good average poor very poor
effectiveness
![Page 17: Evaluating the Impact of a VLE on Learning and Teaching? Dr Barbara Newland Learning Design Studio Academic Services Bournemouth University](https://reader035.vdocuments.us/reader035/viewer/2022070407/56649e175503460f94b038b1/html5/thumbnails/17.jpg)
Staff: Has duo affected your approach to teaching?
10.90.58.6
80
My basic approach to teaching seems to be changing, in part because of duo
My basic approach to teaching has not changed, but duo helps me do certain things better
There is minimal potential for change
I can't teach as w ell w hen I rely on using duo
![Page 18: Evaluating the Impact of a VLE on Learning and Teaching? Dr Barbara Newland Learning Design Studio Academic Services Bournemouth University](https://reader035.vdocuments.us/reader035/viewer/2022070407/56649e175503460f94b038b1/html5/thumbnails/18.jpg)
91.6
8.4
yes no
Students: If duo is not currently being used for all your modules, would you recommend that it should be?
![Page 19: Evaluating the Impact of a VLE on Learning and Teaching? Dr Barbara Newland Learning Design Studio Academic Services Bournemouth University](https://reader035.vdocuments.us/reader035/viewer/2022070407/56649e175503460f94b038b1/html5/thumbnails/19.jpg)
Staff use of duo features - content 62%, 49% staff have used more features 79%, 76% staff have developed content
since the previous year Course information – 74%, 84%, 92% Course documents – 77%, 89%, 95% External links – 42%, 56%, 55% Bibliography – 35%, 53%, 43%
(plus 17%, 21%, 21% link to OPAC) Staff information – 49%, 63%, 72%
![Page 20: Evaluating the Impact of a VLE on Learning and Teaching? Dr Barbara Newland Learning Design Studio Academic Services Bournemouth University](https://reader035.vdocuments.us/reader035/viewer/2022070407/56649e175503460f94b038b1/html5/thumbnails/20.jpg)
Staff use of duo features - communication Group work – 25%, 23%, 26% Email – 72%, 77%, 80% Announcements – 75%, 84%, 86% Discussion board – 16%, 32%, 31% Chat – 10%, 4%, 6%
![Page 21: Evaluating the Impact of a VLE on Learning and Teaching? Dr Barbara Newland Learning Design Studio Academic Services Bournemouth University](https://reader035.vdocuments.us/reader035/viewer/2022070407/56649e175503460f94b038b1/html5/thumbnails/21.jpg)
Staff use of duo features - assessment Assignments – 53%, 63%, 67% Quizzes – 11%, 20%, 26% Surveys - 15%, 13%, 21% Digital drop box – 10%, 6%, 15%
![Page 22: Evaluating the Impact of a VLE on Learning and Teaching? Dr Barbara Newland Learning Design Studio Academic Services Bournemouth University](https://reader035.vdocuments.us/reader035/viewer/2022070407/56649e175503460f94b038b1/html5/thumbnails/22.jpg)
Reflect current ways of teaching Context – relevant, flexible (start from
where they are at) Collis also found at the University of
Twente that “instructors are most likely to begin by choosing aspects of a system that reflect their current ways of teaching, and then gradually move to new instructional approaches and new features.” (Collis and Messing, 2001)
![Page 23: Evaluating the Impact of a VLE on Learning and Teaching? Dr Barbara Newland Learning Design Studio Academic Services Bournemouth University](https://reader035.vdocuments.us/reader035/viewer/2022070407/56649e175503460f94b038b1/html5/thumbnails/23.jpg)
Lecture notes - staff Is it good practice to put lecture
notes online? Depends on what the lecturer does
with them 2/3 put lecture summaries, handouts,
full notes and/or Powerpoint slides after the lecture
![Page 24: Evaluating the Impact of a VLE on Learning and Teaching? Dr Barbara Newland Learning Design Studio Academic Services Bournemouth University](https://reader035.vdocuments.us/reader035/viewer/2022070407/56649e175503460f94b038b1/html5/thumbnails/24.jpg)
Lecture notes - students Depends how students use them
73% listen more 81% clarify information 72% improve lectures 93% study for exams
2/3 (64%, 64%) students note-taking had stayed the same but for ¼ (27%, 26%) it had decreased
Only 1%, 1% stated no longer attended lectures
![Page 25: Evaluating the Impact of a VLE on Learning and Teaching? Dr Barbara Newland Learning Design Studio Academic Services Bournemouth University](https://reader035.vdocuments.us/reader035/viewer/2022070407/56649e175503460f94b038b1/html5/thumbnails/25.jpg)
Gender No statistical significance between
gender and Staff with effectiveness as learning
resource Students with contribution to learning Both with levels of confidence
Higher percentage of females think duo helps them to manage their learning
![Page 26: Evaluating the Impact of a VLE on Learning and Teaching? Dr Barbara Newland Learning Design Studio Academic Services Bournemouth University](https://reader035.vdocuments.us/reader035/viewer/2022070407/56649e175503460f94b038b1/html5/thumbnails/26.jpg)
Staff by faculty – skills More Science faculty found it very
easy/easy to use compared to Arts Some Arts staff took longer to gain
confidence (6-10 visits) Fewer Science staff chose to have
training Reflects the levels of staff
confidence in C&IT skills found in Audit June 2000
![Page 27: Evaluating the Impact of a VLE on Learning and Teaching? Dr Barbara Newland Learning Design Studio Academic Services Bournemouth University](https://reader035.vdocuments.us/reader035/viewer/2022070407/56649e175503460f94b038b1/html5/thumbnails/27.jpg)
Staff by faculty – teaching 2002 - no significant difference 2003
Basic approach is changing Science and social science 12% compared to
3% arts Basic approach has not changed but
duo helps me do certain things better Science 79%, social science 76% compared
to 98% arts
![Page 28: Evaluating the Impact of a VLE on Learning and Teaching? Dr Barbara Newland Learning Design Studio Academic Services Bournemouth University](https://reader035.vdocuments.us/reader035/viewer/2022070407/56649e175503460f94b038b1/html5/thumbnails/28.jpg)
Students by faculty Contribution that duo has made to their
learning overall 2002 - no significant difference 2003 – higher percentage of science (69%)
and social science (52%) rate “good” or “excellent” compared to 42% arts students
Higher percentage of social science students think duo helps them to manage their learning “greatly” or “quite a lot” compared to science students
![Page 29: Evaluating the Impact of a VLE on Learning and Teaching? Dr Barbara Newland Learning Design Studio Academic Services Bournemouth University](https://reader035.vdocuments.us/reader035/viewer/2022070407/56649e175503460f94b038b1/html5/thumbnails/29.jpg)
Students by year group Contribution that duo has made to
their learning overall 2001 and 2002 - 2nd and 3rd year
students rates more higher than 1st years (1st years have always had duo)
2003 – 1st year students rate slightly higher than 2nd and 3rd years
![Page 30: Evaluating the Impact of a VLE on Learning and Teaching? Dr Barbara Newland Learning Design Studio Academic Services Bournemouth University](https://reader035.vdocuments.us/reader035/viewer/2022070407/56649e175503460f94b038b1/html5/thumbnails/30.jpg)
C&IT skills survey of new undergraduates – 2002 n = 2656 82% of students perceive themselves to
be confident/very confident computer users
60% have brought a computer with them to University
69% have been using a computer for 5+ years (including 21% for over 10 years) and only 2% have been using a computer for <2 years
![Page 31: Evaluating the Impact of a VLE on Learning and Teaching? Dr Barbara Newland Learning Design Studio Academic Services Bournemouth University](https://reader035.vdocuments.us/reader035/viewer/2022070407/56649e175503460f94b038b1/html5/thumbnails/31.jpg)
C&IT skills survey of new undergraduates - 2002 79% perceive themselves to have
good/advanced email skills 65% perceive themselves to have
good/advanced web browser skills 70% stated a preference for a
combination of web and paper based for learning support materials
![Page 32: Evaluating the Impact of a VLE on Learning and Teaching? Dr Barbara Newland Learning Design Studio Academic Services Bournemouth University](https://reader035.vdocuments.us/reader035/viewer/2022070407/56649e175503460f94b038b1/html5/thumbnails/32.jpg)
C&IT skills survey of new undergraduates - 2002 95% own a mobile phone 5% have a PDA 38% take part in online discussions 59% take part in online chat 49% have a qualification in IT 54% regularly switch between several
applications when using a computer
![Page 33: Evaluating the Impact of a VLE on Learning and Teaching? Dr Barbara Newland Learning Design Studio Academic Services Bournemouth University](https://reader035.vdocuments.us/reader035/viewer/2022070407/56649e175503460f94b038b1/html5/thumbnails/33.jpg)
C&IT skills survey of academic staff - 2000 n = 576 (53%) 90% perceive themselves to have
good/advanced email skills 67% perceive themselves to have
good/advanced web browser skills only 17% of staff had taken part in online
discussions only 12% of staff had taken part in online
chat
![Page 34: Evaluating the Impact of a VLE on Learning and Teaching? Dr Barbara Newland Learning Design Studio Academic Services Bournemouth University](https://reader035.vdocuments.us/reader035/viewer/2022070407/56649e175503460f94b038b1/html5/thumbnails/34.jpg)
Catalyst Introduction encouraged
continual reflection on teaching Postgraduate Certificate in HE
compulsory for new academics from Jan 2001
Academics working together within courses
![Page 35: Evaluating the Impact of a VLE on Learning and Teaching? Dr Barbara Newland Learning Design Studio Academic Services Bournemouth University](https://reader035.vdocuments.us/reader035/viewer/2022070407/56649e175503460f94b038b1/html5/thumbnails/35.jpg)
Factors influencing enhancement How used How integrated into curriculum Quality of the parts eg content,
assessment, communication (earlier research)
Enhance learning if the parts enhance learning
Flexibility allows lecturers to choose parts
![Page 36: Evaluating the Impact of a VLE on Learning and Teaching? Dr Barbara Newland Learning Design Studio Academic Services Bournemouth University](https://reader035.vdocuments.us/reader035/viewer/2022070407/56649e175503460f94b038b1/html5/thumbnails/36.jpg)
Is the whole greater than the sum of its parts? But is a VLE more than the pieces
put together?
![Page 37: Evaluating the Impact of a VLE on Learning and Teaching? Dr Barbara Newland Learning Design Studio Academic Services Bournemouth University](https://reader035.vdocuments.us/reader035/viewer/2022070407/56649e175503460f94b038b1/html5/thumbnails/37.jpg)
Evaluating the impact of a VLE on learning and teaching Incredible uptake of duo which has become
embedded into the University Staff and students believe duo has enhanced
teaching and learning Academics are gradually using more features
so this may result in more interactive and innovative teaching
Students expectations are increasing as the basic use of a VLE is assumed
Further analysis is being undertaken
![Page 38: Evaluating the Impact of a VLE on Learning and Teaching? Dr Barbara Newland Learning Design Studio Academic Services Bournemouth University](https://reader035.vdocuments.us/reader035/viewer/2022070407/56649e175503460f94b038b1/html5/thumbnails/38.jpg)
References Collis, B. and Messing, J. (2001) Usage, attitudes and
workload implications for a web-based learning environment, Association of Learning Technology Journal 9 (1)
Collis, B. and Moonen, J. (2001) Flexible Learning in a Digital World, London, Kogan Page
Laurillard, D. (1993). Rethinking University Teaching: a Framework for the Effective Use of Educational Technology. London: Routledge.
Richardson, J., & Turner, A. (2000). A large-scale ‘local’ evaluation of students’ learning experiences using virtual learning environments. Educational Technology and Society, 3 (4).
Salmon, G. (2000). E-moderating: the Key to Teaching and Learning Online. London: Kogan Page.