A Life-Cycle Model for the Quality Evaluation

of Educational Content

A. Kameas A. Stefani

Higher Education challenges

A new stage is set for the transition to the paperless University;

Education Institutions’ dual role: publishers and distributors of educational content.

Challenge: Standardization of design

and production procedures

Research Challenge and Methodology: Summary

Research question how to design digital educational content

suitable for open and distance learning based on formal Q&A practices.

Methodology a top-down analysis of Q&A needs through the

definition of a content life-cycle model; a bottom-up synthesis of a standard-like hierarchy

through the definition of certification guidelines.

Goal: to map needs, stakeholders, and quality attributes to

quality characteristics, sub-characteristics and metrics of an ISO-like standard.

Digital Educational Content

We refer to ‘Digital Educational Content’ in the form of:

State of the Art (1/3)

State of the Art (2/3)

State of the Art (3/3)

Research challenges (1/2)

Educational content and learning processes are closely interconnected.

There should exist specifications that allow: compatibility with the educational processes,

to achieve the maximum out of the learning outcomes,

reduction of costs during content use (ideally without compromising educational quality),

reuse of the content.

Research challenges (2/2)

There is a need for a clear grouping of

processes (what should be done),

content (to what) and

stakeholders (by whom);

Design a quality life cycle (QLC) for educational content: adoption of (some) characteristics of quality

standards (formality),

eliciting good practices through the application of benchmarking methods (practicality),

coverage of all types of educational content (completeness).

QLC Design Methodology: insights

What to measure from phase to phase?

diverse nature: educational, pedagogical and technical facets.

numerous parameters: functionality, educational suitability, educational correctness, medium of content delivery…

Different aspects must be assessed through different methods/tools.

What is the best method to asses which part?

How to measure it?

Quality: is neither measurable nor strictly defined (in the field of distance education)

model quality, quality assurance procedures, compliance and quality-of-use.

QLC Development Methodology: insights

The development of QLC requires both

Process View: running a project with analysis, design and development phases - PMBOK

Data View producing a product – ISO standards.

E.g. Product lifecycle according to PMBOK

Top-Down step: The Life cycle Model

Bottom-up Approach: HOU experience

Guidelines/Best Practises derived from HOU experience: Use of standards ISO9001:2008 and ELOT:1429;

12 years of formally applying Q&A in teaching practises and content;

HOU as a publisher: 300 books, 500h of video-lectures;

Several GB of digital content (Wikis, hypertext, ebooks) by 2015;

Multi-step quality control based on in-breed methods

Bottom-up Approach: guidelines

Quality Control practical rules

used by life-cycle stakeholders (authors/evaluators/QC experts) for

the assessment and certification

The certification guide includes

11 guidelines

relating to content accuracy, expression, educational feedback and self-assessment, completeness, scientific accuracy, readability, motivation for further reading and personal research, clarity of expression and up-to-dateness.

Bottom-up Approach: example (1/3)

Bottom-up Approach: example (2/3)

Bottom-up Approach: example (3/3)

Towards a new Quality Model

Initial results have produced a dual, data and process standard with 8 quality characteristics, 30 sub-characteristics and 72 metrics

Future work:

Verification

and Validation Tool

Validate standard

Thank you for your Attention!