Download - Evaluating online courses 25 may 2011
Phil Hardcastle
Email: Phil.Hardcastle@@rsc-em.ac.uk
Adam Blackwood
Email: [email protected]
Moving Learning On Line - Evaluating & Reviewing Online courses
April 12, 2023 | slide 1
Plan for the session
Ice Breaking & Introduction Observable Benefits of MLO Measuring Impact and Quality – Tools &
Approaches Review Session Further Resources
Ice Breaking & Introduction Observable Benefits of MLO Measuring Impact and Quality – Tools &
Approaches Review Session Further Resources
April 12, 2023 | slide 2
Who is here…
April 12, 2023 | slide 3
B. Williams RSCEM April 12, 2023 | slide 4
Who’s here
A Adult/Community Learning
B Work Based Learning
C FE College
D Specialist College
E 6th Form College
B. Williams RSCEM April 12, 2023 | slide 5
Online learning – a spectrum of use
A VLE is used to make a variety of documents available as a reference point for the learner when he/she is outside the classroom.
Used to deliver learning activities and to encourage co-operation between students on projects. Assignments may be submitted and marked on-line and there will be assessment exercises.
A fully online course requires the tutor to work online in the same way as he/she would have done face to face. The tutor will have to use digital means to promoteengagement and a sense of community.
B. Williams RSCEM April 12, 2023 | slide 6
Where are you on the scale?
Making
documents
availableLearning activities,
assessment
Creating online community
Plan for the session
Ice Breaking & Introduction Observable Benefits of MLO Measuring Impact and Quality – Tools &
Approaches Review Session Further Resources
Ice Breaking & Introduction Observable Benefits of MLO Measuring Impact and Quality – Tools &
Approaches Review Session Further Resources
April 12, 2023 | slide 7
A checklist for assessing courses 1
Is it likely to work?– Has information been provided to enable prospective
users to understand the course being offered whether it will suit them?
– Have students had an introduction or orientation which prepares them to be successful in the course?
– Do students have the necessary hardware & software to complete the course?
– Have students the necessary study skills and IT skills to complete the course – is there opportunity to practise these skills?
April 12, 2023 | slide 8
A checklist for assessing courses 2
Is it likely to work?– Is the course designed to be interactive?– Are the tutors trained in the techniques that are required
for online tutoring?– Is there a system of support for students (and staff) –
when does it operate 24/7? 7/5? FAQ?– Is data collected about the course so that
administrators, decision makers and course designers can learn from the shared information?
This checklist is adapted from the Appleton Eschool’s online evaluation programme. http://www2.ed.gov/admins/lead/academic/evalonline/evalonline.pdf
April 12, 2023 | slide 9
Online materials presented to students
Course and module title Tutor name, contact details and
photo Brief welcome/introduction to the
course Structured content (chronological
or topic order) A brief introduction to each
section Lecture notes, presentations,
hand-outs - placed in context A clear indication of where and
how to get help
A reading list A glossary of terms, if
appropriate A course timetable (a list or
calendar) Links to specific
resources/websites for research A link to library services and
search Clearly label all resources
indicating what they relate to Assignments details including
how to submit April 12, 2023 | slide 10
A basic course would contain the following:
Interactivity and “stickiness”
The more engaging the course, the more likely the students are to remain on it and complete assignments.
The course should therefore promote interactivity – with the materials, with the tutor and with other students.
Courses can be assessed for the interactive elements they provide.
Varied media presentation will also help – video and podcasts; short spoken introductions to materials; all these help to give a varied experience to the student.
April 12, 2023 | slide 11
Plan for the session
Ice Breaking & Introduction Observable Benefits of MLO Measuring Impact and Quality – Tools &
Approaches Review Session Further Resources
Ice Breaking & Introduction Observable Benefits of MLO Measuring Impact and Quality – Tools &
Approaches Review Session Further Resources
April 12, 2023 | slide 12
Outline of the Evaluation Tool How it works. Criteria Download and use Complete survey for
April 12, 2023 | slide 13
Course Evaluation & ReviewTool
Students
Teaching Practitioners
Data and Quality Assurance
ROI
April 12, 2023 | slide 15
MLOL Evaluation Tool
The Process
April 12, 2023 | slide 16
Determine Evaluation CriteriaDetermine Evaluation Criteria
Create a Survey Monkey Answer Grid
Create a Survey Monkey Answer Grid
Download as Excel FileDownload as Excel File
Add to MLOL Evaluation Tool in the Data Sheet Tab
Add to MLOL Evaluation Tool in the Data Sheet Tab
Provide to Sample Groups…Provide to Sample Groups…
Review PictureReview Picture
April 12, 2023 | slide 17
April 12, 2023 | slide 18
April 12, 2023 | slide 19
MLOL Evaluation Tool
April 12, 2023 | slide 20
Poor Evaluation Picture
MLOL Evaluation Tool
April 12, 2023 | slide 21
Very GoodEvaluation Picture
MLOL Evaluation Tool
The Process
April 12, 2023 | slide 22
Determine Evaluation CriteriaDetermine Evaluation Criteria
Create a Survey Monkey Answer Grid
Create a Survey Monkey Answer Grid
Download as Excel FileDownload as Excel File
Add to MLOL Evaluation Tool in the Data Sheet Tab
Add to MLOL Evaluation Tool in the Data Sheet Tab
Provide to Sample Groups…Provide to Sample Groups…
Review PictureReview Picture
April 12, 2023 | slide 23
Survey Monkey Answer Grid
Option
MLOL Evaluation To
TASK 1 Your Criteria evaluation
http://tiny.cc/mlol7task1
April 12, 2023 | slide 24
TASKS
TASKS Download the MLOL Evaluation Tool – 10 Minutes
– Evaluation Tool Download at http://www.tiny.cc/mlol7tool
Consider what Criteria you might want to add to the different Sectors
Feedback
April 12, 2023 | slide 25
Online Lesson Observation
The main concern when observing a lesson is to check whether learning is taking place.
How could you demonstrate that learning was taking place in an online course?– Data from online interactions – forums etc– Reports showing quiz answers, assignments completed
How do learners feel about the quality of learning?– Learner surveys by web or phone or email– Use an online live forum of learners
Feedback to tutors?
April 12, 2023 | slide 26
Formative and Summative
Online systems and course design are in the very early stages – formative assessment of the course through user surveys, evaluation of online interaction etc can be very useful to help re-focus the course to meet learner needs.
Summative assessment needs to be disseminated around the organisation to make sure that success and failure can be duplicated and avoided.
April 12, 2023 | slide 27
Plan for the session
Ice Breaking & Introduction Observable Benefits of MLO Measuring Impact and Quality – Tools &
Approaches Review Session Further Resources
Ice Breaking & Introduction Observable Benefits of MLO Measuring Impact and Quality – Tools &
Approaches Review Session Further Resources
April 12, 2023 | slide 28
Staff Development Programme and QA Processes
Staff Development Programme and QA Processes
April 12, 2023 | slide 29
Plan for the session
Ice Breaking & Introduction Observable Benefits of MLO Measuring Impact and Quality – Tools &
Approaches Review Session Further Resources
Ice Breaking & Introduction Observable Benefits of MLO Measuring Impact and Quality – Tools &
Approaches Review Session Further Resources
April 12, 2023 | slide 30
References and Resources
Google Docs List of references at: http://tiny.cc/mlol7 Can be added to if you have other links and resources you
can suggest. Survey Monkey Example at http://www.tiny.cc/mlol7task1 Evaluation Tool Download at http://www.tiny.cc/mlol7tool
April 12, 2023 | slide 31