quality on line: what and how should we be measuring for quality assurance in learning?
DESCRIPTION
Prof. dr. Betty CollisEDEN Research Workshop, 26 October 2006, Castelldefels, SpainTRANSCRIPT
![Page 1: Quality on line: What and how should we be measuring for quality assurance in learning?](https://reader033.vdocuments.us/reader033/viewer/2022060202/559c51d41a28abd7458b482e/html5/thumbnails/1.jpg)
Quality on line: What and how should we be measuring for
quality assurance in learning?
Prof. dr. Betty CollisEDEN Research Workshop, 26 October 2006,
Castelldefels, Spain
Moonen & Collis Learning Technology [email protected]
Emeritus professor, University of [email protected]
![Page 2: Quality on line: What and how should we be measuring for quality assurance in learning?](https://reader033.vdocuments.us/reader033/viewer/2022060202/559c51d41a28abd7458b482e/html5/thumbnails/2.jpg)
Quality?
From whose perspective? Against what criteria? What do you do with what you find?
An illustration from practice
![Page 3: Quality on line: What and how should we be measuring for quality assurance in learning?](https://reader033.vdocuments.us/reader033/viewer/2022060202/559c51d41a28abd7458b482e/html5/thumbnails/3.jpg)
From whose perspective?
![Page 4: Quality on line: What and how should we be measuring for quality assurance in learning?](https://reader033.vdocuments.us/reader033/viewer/2022060202/559c51d41a28abd7458b482e/html5/thumbnails/4.jpg)
Institutional perspective?
![Page 5: Quality on line: What and how should we be measuring for quality assurance in learning?](https://reader033.vdocuments.us/reader033/viewer/2022060202/559c51d41a28abd7458b482e/html5/thumbnails/5.jpg)
Institutional perspective
Benchmarking:Self-improvement tool for
organizations Compare with others and/or reflect
internally Review comparative strengths &
weaknesses Identify possible routes to
improvement Rather than statistical indicators
of results (completion rates, unit costs, etc), focus on processes by which results are achieved
Accreditation reviewsExternal review body
• For licensing
• For comparisons among institutions
• Implications for funding
Items are typically responded to on a scale, with labels corresponding to 0=Not applicable, 1= No information available, 2=Not adequate, 3=Partly adequate, 4=Largely adequate, 5=Fully adequate, 6=Best practices processes
![Page 6: Quality on line: What and how should we be measuring for quality assurance in learning?](https://reader033.vdocuments.us/reader033/viewer/2022060202/559c51d41a28abd7458b482e/html5/thumbnails/6.jpg)
http://www.insight.strath.ac.uk/projects/risk/index.htm
![Page 7: Quality on line: What and how should we be measuring for quality assurance in learning?](https://reader033.vdocuments.us/reader033/viewer/2022060202/559c51d41a28abd7458b482e/html5/thumbnails/7.jpg)
How might external funding policies affect your e-learning strategy?
What are the objectives of your investment in e-learning? (ie., widening access?)
How are working practices and staff roles being affected? What are students’ expectations?
What structures are in place to support students? Staff?
Should you make, buy, update, or change?
How do your students access e-learning technologies and resources?
Is your curriculum development being affected by e-learning?
How is the quality of (e)-learning being monitored?
![Page 8: Quality on line: What and how should we be measuring for quality assurance in learning?](https://reader033.vdocuments.us/reader033/viewer/2022060202/559c51d41a28abd7458b482e/html5/thumbnails/8.jpg)
http://www.utdc.vuw.ac.nz/research/emm/index.shtml
How can Institutions help teaching staff to be more successful in their use of technology to support student learning?
![Page 9: Quality on line: What and how should we be measuring for quality assurance in learning?](https://reader033.vdocuments.us/reader033/viewer/2022060202/559c51d41a28abd7458b482e/html5/thumbnails/9.jpg)
Source: S. Marchall, (2005). New Zealand E-Learning Capability Determination. Available at
http://www.utdc.vuw.ac.nz/research/emm/Presentations.shtml
![Page 10: Quality on line: What and how should we be measuring for quality assurance in learning?](https://reader033.vdocuments.us/reader033/viewer/2022060202/559c51d41a28abd7458b482e/html5/thumbnails/10.jpg)
Learner perspective?
![Page 11: Quality on line: What and how should we be measuring for quality assurance in learning?](https://reader033.vdocuments.us/reader033/viewer/2022060202/559c51d41a28abd7458b482e/html5/thumbnails/11.jpg)
Example:
Student perceptions (5-pt scale) of: Flexibility Responsiveness and support Learning Participation/interaction Usefulness and ease of use of technology Overall satisfaction
McGorry, S. Y. (2003). Measuring quality in online programs. The Internet & Higher Education, 6, 159-177
![Page 12: Quality on line: What and how should we be measuring for quality assurance in learning?](https://reader033.vdocuments.us/reader033/viewer/2022060202/559c51d41a28abd7458b482e/html5/thumbnails/12.jpg)
http://www1.tqi.ac.uk/sites/tqi/home/index.cfm
![Page 13: Quality on line: What and how should we be measuring for quality assurance in learning?](https://reader033.vdocuments.us/reader033/viewer/2022060202/559c51d41a28abd7458b482e/html5/thumbnails/13.jpg)
![Page 14: Quality on line: What and how should we be measuring for quality assurance in learning?](https://reader033.vdocuments.us/reader033/viewer/2022060202/559c51d41a28abd7458b482e/html5/thumbnails/14.jpg)
Content perspective?
Content, assembly, metadata, standards
![Page 15: Quality on line: What and how should we be measuring for quality assurance in learning?](https://reader033.vdocuments.us/reader033/viewer/2022060202/559c51d41a28abd7458b482e/html5/thumbnails/15.jpg)
Output oriented
One set of stakeholders may have one (implicit) view on quality
Output oriented
Infrastructure (to achieve output goal)
![Page 16: Quality on line: What and how should we be measuring for quality assurance in learning?](https://reader033.vdocuments.us/reader033/viewer/2022060202/559c51d41a28abd7458b482e/html5/thumbnails/16.jpg)
Another set of stakeholders may have a different view on quality
Learning-process oriented
![Page 17: Quality on line: What and how should we be measuring for quality assurance in learning?](https://reader033.vdocuments.us/reader033/viewer/2022060202/559c51d41a28abd7458b482e/html5/thumbnails/17.jpg)
And some perspectives may not get represented in these clusters…
(Personal) processes?
Theory oriented?
![Page 18: Quality on line: What and how should we be measuring for quality assurance in learning?](https://reader033.vdocuments.us/reader033/viewer/2022060202/559c51d41a28abd7458b482e/html5/thumbnails/18.jpg)
Or may not be fully taken up in the quality priorities of other clusters
Content, assembly, metadata, standards
![Page 19: Quality on line: What and how should we be measuring for quality assurance in learning?](https://reader033.vdocuments.us/reader033/viewer/2022060202/559c51d41a28abd7458b482e/html5/thumbnails/19.jpg)
Thus quality reflects the perspective
![Page 20: Quality on line: What and how should we be measuring for quality assurance in learning?](https://reader033.vdocuments.us/reader033/viewer/2022060202/559c51d41a28abd7458b482e/html5/thumbnails/20.jpg)
The challenge: Try to combine the perspectives
![Page 21: Quality on line: What and how should we be measuring for quality assurance in learning?](https://reader033.vdocuments.us/reader033/viewer/2022060202/559c51d41a28abd7458b482e/html5/thumbnails/21.jpg)
Example of an integrated evaluation approach in a corporate learning context
Collaborative learning
Shell EP Learning Centre
Collis, B., Bianco, M., Margaryan, A., & Waring, B. (2005). Putting blended learning to work: A case study from a multinational oil company. Education, Communication and Information, 5(3), 233-250.
Overall company impact & alignment
Multinational knowledge sharing
Supervisor engagement
Workplace learning, learning- environment design
Merrill’s Principles of instruction
Capture, reuse, from the business and from participant contributions
Competence framework
Participant assessment
LMS, VLE for strategic/pedagogic goals
Shell: overall company
Work-based activities
![Page 22: Quality on line: What and how should we be measuring for quality assurance in learning?](https://reader033.vdocuments.us/reader033/viewer/2022060202/559c51d41a28abd7458b482e/html5/thumbnails/22.jpg)
As part of an integrated approach: Generic principles of learning
“Learning is promoted when: Learners are engaged in solving real-world [i.e., business-
relevant] problems. Existing knowledge [in the learner or in his or her workplace] is
activated as a foundation for new knowledge. New knowledge is demonstrated to the learner [through seeing
it in action in the workplace]. New knowledge is applied by the learner [to his or her current
workplace situation]. New knowledge in integrated into the learner’s world
[workplace].” (Merrill, 2002, pp. 44-45, augmented with linkages relating to
the corporate context)
Collis, B., & Margaryan, A. (2005). Design criteria for work-based learning: Merrill's First Principles of Instruction expanded. British
Journal of Educational Technology, 36(5), 725-738.
Merrill, D. (2003). First principles of instruction. Educational Technology Research and Development, 50(3), 43-59.
![Page 23: Quality on line: What and how should we be measuring for quality assurance in learning?](https://reader033.vdocuments.us/reader033/viewer/2022060202/559c51d41a28abd7458b482e/html5/thumbnails/23.jpg)
Regardless of the context, an approach for quality should include Merrill’s “first principles of instruction”
Merrill’s five first principles of instruction
![Page 24: Quality on line: What and how should we be measuring for quality assurance in learning?](https://reader033.vdocuments.us/reader033/viewer/2022060202/559c51d41a28abd7458b482e/html5/thumbnails/24.jpg)
And finally, what to do with the results?
![Page 25: Quality on line: What and how should we be measuring for quality assurance in learning?](https://reader033.vdocuments.us/reader033/viewer/2022060202/559c51d41a28abd7458b482e/html5/thumbnails/25.jpg)
Use an action research approach to integrate strategic planning, stimulating, implementing, monitoring and interpreting quality
Action research
![Page 26: Quality on line: What and how should we be measuring for quality assurance in learning?](https://reader033.vdocuments.us/reader033/viewer/2022060202/559c51d41a28abd7458b482e/html5/thumbnails/26.jpg)
For further ideas:Prof. dr. Betty Collis & Prof. dr. Jef Moonen
Moonen & Collis Learning Technology Consultants, BVhttp://bettycollisjefmoonen.nl
![Page 27: Quality on line: What and how should we be measuring for quality assurance in learning?](https://reader033.vdocuments.us/reader033/viewer/2022060202/559c51d41a28abd7458b482e/html5/thumbnails/27.jpg)
Footnote: This talk is not related to the report: “Quality on the Line” which is a benchmarking study
Quality On the Line: Benchmarks for Success in Internet-Based Distance EducationBy: Ronald Phipps, Jamie Merisotis, 2000; Institute for Higher Education Policy, Washington DChttp://www.ihep.com/organizations.php3?action=printContentItem&orgid=104&typeID=906&itemID=9239&templateID=1422