how to assess an e-learning institution: methodology ...cipp. process teaching process delivery...
TRANSCRIPT
![Page 1: How to assess an e-learning institution: methodology ...CIPP. Process Teaching process Delivery process All classrooms used for the same object have the same learning documentation,](https://reader033.vdocuments.us/reader033/viewer/2022042310/5ed81109cba89e334c672e8d/html5/thumbnails/1.jpg)
www.aqucatalunya.org
Quality, the assurance of improvement.
www.aqucatalunya.org
Quality, the assurance of improvement.
How to assess an e-learning institution: methodology design and implementation
E. Huertas, A. Prades, S. Rodríguez
Budapest, November 2008
![Page 2: How to assess an e-learning institution: methodology ...CIPP. Process Teaching process Delivery process All classrooms used for the same object have the same learning documentation,](https://reader033.vdocuments.us/reader033/viewer/2022042310/5ed81109cba89e334c672e8d/html5/thumbnails/2.jpg)
www.aqucatalunya.org
Quality, the assurance of improvement.
1. Introduction2. Design of the methodological model
2.1. Characteristics of e-learning2.2. Methodology development2.3. Evaluation process
3. Conclusions
Summary
![Page 3: How to assess an e-learning institution: methodology ...CIPP. Process Teaching process Delivery process All classrooms used for the same object have the same learning documentation,](https://reader033.vdocuments.us/reader033/viewer/2022042310/5ed81109cba89e334c672e8d/html5/thumbnails/3.jpg)
www.aqucatalunya.org
Quality, the assurance of improvement.
1. Introduction2. Design of the methodological model
2.1. Characteristics of e-learning2.2. Methodology development2.3. Evaluation process
3. Conclusions
Summary
![Page 4: How to assess an e-learning institution: methodology ...CIPP. Process Teaching process Delivery process All classrooms used for the same object have the same learning documentation,](https://reader033.vdocuments.us/reader033/viewer/2022042310/5ed81109cba89e334c672e8d/html5/thumbnails/4.jpg)
www.aqucatalunya.org
Quality, the assurance of improvement.
Introduction
Fully complied by conventional universities
e-learning programmes are only now becoming more widespread in Europe
QAin HE
![Page 5: How to assess an e-learning institution: methodology ...CIPP. Process Teaching process Delivery process All classrooms used for the same object have the same learning documentation,](https://reader033.vdocuments.us/reader033/viewer/2022042310/5ed81109cba89e334c672e8d/html5/thumbnails/5.jpg)
www.aqucatalunya.org
Quality, the assurance of improvement.
e-learning evaluation project
Formative assessment of the Universitat Oberta de Catalunya
(UOC)
Aim
Introduction
![Page 6: How to assess an e-learning institution: methodology ...CIPP. Process Teaching process Delivery process All classrooms used for the same object have the same learning documentation,](https://reader033.vdocuments.us/reader033/viewer/2022042310/5ed81109cba89e334c672e8d/html5/thumbnails/6.jpg)
www.aqucatalunya.org
Quality, the assurance of improvement.
1. Introduction2. Design of the methodological model
2.1. Characteristics of e-learning2.2. Methodology development2.3. Evaluation process
3. Conclusions
Summary
![Page 7: How to assess an e-learning institution: methodology ...CIPP. Process Teaching process Delivery process All classrooms used for the same object have the same learning documentation,](https://reader033.vdocuments.us/reader033/viewer/2022042310/5ed81109cba89e334c672e8d/html5/thumbnails/7.jpg)
www.aqucatalunya.org
Quality, the assurance of improvement.
Inputs
Process
Product
Context
CIPP Model
Goals
SWOT
Students
Teachers
Infrastructure
Educational
Organizational
Academic
Personal
Professional
Characteristics of e-learning
![Page 8: How to assess an e-learning institution: methodology ...CIPP. Process Teaching process Delivery process All classrooms used for the same object have the same learning documentation,](https://reader033.vdocuments.us/reader033/viewer/2022042310/5ed81109cba89e334c672e8d/html5/thumbnails/8.jpg)
www.aqucatalunya.org
Quality, the assurance of improvement.
CIPP. Context
UOC
Fully virtual university
Organisation Teaching staffOwn teaching staff: 165
Collaborating staff: 1,969
Type of degree that can be obtained?
Different to distance learning (e.g. UNED)
Include degree programmes
Study Director
Programme Director
Studies (∼ faculties)
Highly centralized E-learning education
![Page 9: How to assess an e-learning institution: methodology ...CIPP. Process Teaching process Delivery process All classrooms used for the same object have the same learning documentation,](https://reader033.vdocuments.us/reader033/viewer/2022042310/5ed81109cba89e334c672e8d/html5/thumbnails/9.jpg)
www.aqucatalunya.org
Quality, the assurance of improvement.
Teachers
Infrastructure
CIPP. Inputs
Work full time.Family responsibilities.Tend to be more mature.
Own teaching staff: propose courses, define contents and aims, look for authors for teaching materials, etc.
Students counsellors: give incentive and impetus to learning activities.
Tutors: support and give advise students on matters connected with running of virtual campus, course enrolment, give guidance regarding professional opportunities.
Collaborating staff
Students
Core of a virtual university. University has to guarantee that services for study and learning purposes are satisfactory
![Page 10: How to assess an e-learning institution: methodology ...CIPP. Process Teaching process Delivery process All classrooms used for the same object have the same learning documentation,](https://reader033.vdocuments.us/reader033/viewer/2022042310/5ed81109cba89e334c672e8d/html5/thumbnails/10.jpg)
www.aqucatalunya.org
Quality, the assurance of improvement.
CIPP. Process
Teaching process
Delivery processAll classrooms used for the same object have the same learning documentation, tools and assessment’ process
Same author for all materials in one particular subject, same learning and assessment activities, same student support system
High degree of homogeneity
High degree of homogeneity
Advantages: the institution can make a cascade of changes quickly and effectivelyRisks: Hegemony of single culture to the detriment of plurality, and possible devaluation of teachers as mediators of knowledge
![Page 11: How to assess an e-learning institution: methodology ...CIPP. Process Teaching process Delivery process All classrooms used for the same object have the same learning documentation,](https://reader033.vdocuments.us/reader033/viewer/2022042310/5ed81109cba89e334c672e8d/html5/thumbnails/11.jpg)
www.aqucatalunya.org
Quality, the assurance of improvement.
CIPP. Product
Personal
Academic
Professional
Outcomes
The evaluation of e-learning programmes should be of the same quality as that of conventional learning degrees.
![Page 12: How to assess an e-learning institution: methodology ...CIPP. Process Teaching process Delivery process All classrooms used for the same object have the same learning documentation,](https://reader033.vdocuments.us/reader033/viewer/2022042310/5ed81109cba89e334c672e8d/html5/thumbnails/12.jpg)
www.aqucatalunya.org
Quality, the assurance of improvement.
1. Identification of the levels of assessment (institutional and degree programme levels)
3. Setting of indicators, standards and evidences
2. Adaptation of the design to e-learning education and structure of the Open University
Methodology development
![Page 13: How to assess an e-learning institution: methodology ...CIPP. Process Teaching process Delivery process All classrooms used for the same object have the same learning documentation,](https://reader033.vdocuments.us/reader033/viewer/2022042310/5ed81109cba89e334c672e8d/html5/thumbnails/13.jpg)
www.aqucatalunya.org
Quality, the assurance of improvement.
Methodology development
Aspects common to all degree programmes (mission, vision, delivery system, infrastructure).
Emphasis on QA policies and mechanisms.
Specifies how the aspects, policies and general mechanisms work.
Degreeevaluation
Degreeevaluation
Degreeevaluation
Degreeevaluation
Degreeevaluation
Degreeevaluation
Degreeevaluation
Degreeevaluation
UOC Assessment at Study’s programme (all or sample)
![Page 14: How to assess an e-learning institution: methodology ...CIPP. Process Teaching process Delivery process All classrooms used for the same object have the same learning documentation,](https://reader033.vdocuments.us/reader033/viewer/2022042310/5ed81109cba89e334c672e8d/html5/thumbnails/14.jpg)
www.aqucatalunya.org
Quality, the assurance of improvement.
Methodology development
1. Institutional mission and vision2. System capacity- Students- Teaching staff- Infraestructure- External relations3. Quality assurance mechanisms- Institutional mission and vision- System capacity- Internal and external strategic position- Learning outcomes and study programme- Instruction design- Learning assessment- Outcomes
1. Strategic position of the degree programme2. Study programme-Definition of the study programme-Adequacy of the study programme3. Instruction design-Teaching methodology- Adequacy of the activities- Organisation of the degree- Student orientation, tutorial & advisory system- Technical set-up for instruction- Interpersonal communication systems4. Learning assessment5. Outcomes
Degree programme / Study levelInstitutional level
![Page 15: How to assess an e-learning institution: methodology ...CIPP. Process Teaching process Delivery process All classrooms used for the same object have the same learning documentation,](https://reader033.vdocuments.us/reader033/viewer/2022042310/5ed81109cba89e334c672e8d/html5/thumbnails/15.jpg)
www.aqucatalunya.org
Quality, the assurance of improvement.
1. Identification of the levels of assessment (institutional and degree programme levels)
3. Setting of indicators, standards and evidences
2. Adaptation of the design to e-learning education and structure of the Open University
Methodology development
![Page 16: How to assess an e-learning institution: methodology ...CIPP. Process Teaching process Delivery process All classrooms used for the same object have the same learning documentation,](https://reader033.vdocuments.us/reader033/viewer/2022042310/5ed81109cba89e334c672e8d/html5/thumbnails/16.jpg)
www.aqucatalunya.org
Quality, the assurance of improvement.
Methodology development
1. Institutional mission and vision2. System capacity- Students- Teaching staff- Infraestructure- External relations3. Quality assurance mechanisms- Institutional mission and vision- System capacity- Internal and external strategic position- Learning outcomes and study programme- Instruction design- Learning assessment- Outcomes
1. Strategic position of the degree programme2. Study programme-Definition of the study programme-Adequacy of the study programme3. Instruction design-Teaching methodology- Adequacy of the activities- Organisation of the degree- Student orientation, tutorial & advisory system- Technical set-up for instruction- Interpersonal communication systems4. Learning assessment5. Outcomes
Degree programme / Study levelInstitutional level
![Page 17: How to assess an e-learning institution: methodology ...CIPP. Process Teaching process Delivery process All classrooms used for the same object have the same learning documentation,](https://reader033.vdocuments.us/reader033/viewer/2022042310/5ed81109cba89e334c672e8d/html5/thumbnails/17.jpg)
www.aqucatalunya.org
Quality, the assurance of improvement.
Methodology development
1. Strategic position of the degree programme2. Study programme-Definition of the study programme-Adequacy of the study programme3. Instruction design-Teaching methodology- Adequacy of the activities- Organisation of the degree- Student orientation, tutorial & advisory system- Technical set-up for instruction- Interpersonal communication systems4. Learning assessment5. Outcomes
Degree programme / Study level
Methodology and learning resources
Learning activities and evaluation of activities
Learning materials.
How students know how to work with the system.
Student satisfaction with counsellors and tutors (speed and functionality of their responses).
![Page 18: How to assess an e-learning institution: methodology ...CIPP. Process Teaching process Delivery process All classrooms used for the same object have the same learning documentation,](https://reader033.vdocuments.us/reader033/viewer/2022042310/5ed81109cba89e334c672e8d/html5/thumbnails/18.jpg)
www.aqucatalunya.org
Quality, the assurance of improvement.
Methodology development
1. Strategic position of the degree programme2. Study programme-Definition of the study programme-Adequacy of the study programme3. Instruction design-Teaching methodology- Adequacy of the activities- Organisation of the degree- Student orientation, tutorial & advisory system- Technical set-up for instruction- Interpersonal communication systems4. Learning assessment5. Outcomes
Degree programme / Study level
Speed of access and stability of the on-line environment.
Adequacy of on-line environment to the goals of the degree programme and learning activities.
Easy use of the on-line campus.
Interpersonal communication between subjects in different university groups.
Intercommunication between studentsin the same group.
![Page 19: How to assess an e-learning institution: methodology ...CIPP. Process Teaching process Delivery process All classrooms used for the same object have the same learning documentation,](https://reader033.vdocuments.us/reader033/viewer/2022042310/5ed81109cba89e334c672e8d/html5/thumbnails/19.jpg)
www.aqucatalunya.org
Quality, the assurance of improvement.
On-line education standards
EHEA standards
1. Identification of the levels of assessment (institutional and degree programme levels)
3. Setting of indicators, standards and evidences
2. Adaptation of the design to e-learning education and structure of the Open University
Methodology development
![Page 20: How to assess an e-learning institution: methodology ...CIPP. Process Teaching process Delivery process All classrooms used for the same object have the same learning documentation,](https://reader033.vdocuments.us/reader033/viewer/2022042310/5ed81109cba89e334c672e8d/html5/thumbnails/20.jpg)
www.aqucatalunya.org
Quality, the assurance of improvement.
Evaluation process
Self-evaluation document
Self-evaluation committee
External report (draft)(External review panel)
Improvement of draft report (self-evaluation committee)
External report (final)
External review process
![Page 21: How to assess an e-learning institution: methodology ...CIPP. Process Teaching process Delivery process All classrooms used for the same object have the same learning documentation,](https://reader033.vdocuments.us/reader033/viewer/2022042310/5ed81109cba89e334c672e8d/html5/thumbnails/21.jpg)
www.aqucatalunya.org
Quality, the assurance of improvement.
Evaluation process. Self-evaluation committee
Institutional Study / Programme
- Vice-chancellors (2)
- Assistant managers (2)
- Directors of studies (3)
- Programme directors (2)
- Director of the study (1)
- Programme directors (2 or 3)
- Study programme administrator (1)
- Teaching staff (2 or 3)
- Student counsellors (2)
- Tutors (2)
- Graduate from each degree programme (2 or 3)
![Page 22: How to assess an e-learning institution: methodology ...CIPP. Process Teaching process Delivery process All classrooms used for the same object have the same learning documentation,](https://reader033.vdocuments.us/reader033/viewer/2022042310/5ed81109cba89e334c672e8d/html5/thumbnails/22.jpg)
www.aqucatalunya.org
Quality, the assurance of improvement.
Evaluation process
On-line interviews
Site visit (interviews, review of documentation,
classrooms examples)
Self-evaluation document
Self-evaluation committee
Advisers, tutors, students
Other stakeholders (teaching staff, graduates, etc.)
External report (draft)(External review panel)
Improvement of draft report (self-evaluation committee)
External report (final)
External review panel
![Page 23: How to assess an e-learning institution: methodology ...CIPP. Process Teaching process Delivery process All classrooms used for the same object have the same learning documentation,](https://reader033.vdocuments.us/reader033/viewer/2022042310/5ed81109cba89e334c672e8d/html5/thumbnails/23.jpg)
www.aqucatalunya.org
Quality, the assurance of improvement.
Institutional
Degree programme
- Academic
- Academic in e-education
- Professional expert in e-education
- Methodologist
- Academic
- Graduate-professional
- Methodologist
Evaluation process. External review panel
![Page 24: How to assess an e-learning institution: methodology ...CIPP. Process Teaching process Delivery process All classrooms used for the same object have the same learning documentation,](https://reader033.vdocuments.us/reader033/viewer/2022042310/5ed81109cba89e334c672e8d/html5/thumbnails/24.jpg)
www.aqucatalunya.org
Quality, the assurance of improvement.
Evaluation process
On-line interviews
Site visit (interviews, review of documentation,
classrooms examples)
Self-evaluation document
Self-evaluation committee
Advisers, tutors, students
Other stakeholders (teaching staff, graduates, etc.)
External report (draft)(External review panel)
Improvement of draft report (self-evaluation committee)
External report (final)
External review panel
![Page 25: How to assess an e-learning institution: methodology ...CIPP. Process Teaching process Delivery process All classrooms used for the same object have the same learning documentation,](https://reader033.vdocuments.us/reader033/viewer/2022042310/5ed81109cba89e334c672e8d/html5/thumbnails/25.jpg)
www.aqucatalunya.org
Quality, the assurance of improvement.
On-line
Site visit
- Students
- Student counsellors
- Tutors
- Self-assessment committee
- Board of Governors
- Teaching staff
- Technical staff
- Studies communication & admi. staff
- Director of studies
Evaluation process. Audiences
- Self-assessment committee
- Teaching staff
- Academic managers / coordinators
- Graduates
- Review of materials
Institutional level Study / Programme level
![Page 26: How to assess an e-learning institution: methodology ...CIPP. Process Teaching process Delivery process All classrooms used for the same object have the same learning documentation,](https://reader033.vdocuments.us/reader033/viewer/2022042310/5ed81109cba89e334c672e8d/html5/thumbnails/26.jpg)
www.aqucatalunya.org
Quality, the assurance of improvement.
1. Introduction2. Design of the methodological model
2.1. Characteristics of e-learning2.2. Methodology development2.3. Evaluation process
3. Conclusions
Summary
![Page 27: How to assess an e-learning institution: methodology ...CIPP. Process Teaching process Delivery process All classrooms used for the same object have the same learning documentation,](https://reader033.vdocuments.us/reader033/viewer/2022042310/5ed81109cba89e334c672e8d/html5/thumbnails/27.jpg)
www.aqucatalunya.org
Quality, the assurance of improvement.
Conclusions
- Methodology well adapted to e-learning institution.
- Two levels of assessment.
- Training in an e-learning education system and specific nature of the university.
![Page 28: How to assess an e-learning institution: methodology ...CIPP. Process Teaching process Delivery process All classrooms used for the same object have the same learning documentation,](https://reader033.vdocuments.us/reader033/viewer/2022042310/5ed81109cba89e334c672e8d/html5/thumbnails/28.jpg)
www.aqucatalunya.org
Quality, the assurance of improvement.
Conclusions
- On-line interviews.
- Access to all resources available to students and some classroom examples.
- Graduate-professional role in the external review panel.
- Lack of benchmarks within the context of evaluation.
![Page 29: How to assess an e-learning institution: methodology ...CIPP. Process Teaching process Delivery process All classrooms used for the same object have the same learning documentation,](https://reader033.vdocuments.us/reader033/viewer/2022042310/5ed81109cba89e334c672e8d/html5/thumbnails/29.jpg)
www.aqucatalunya.org
Quality, the assurance of improvement.
Thank you for your attention