Principles for the regulation of e-assessment

An update on developments

Andrew Boyle

Regulations and Standards division

QCA

• Statutory body in England, sponsored by DCSF• Functions:

– Maintains and develops the national curriculum and associated assessments

– Regulates qualifications offered in schools, colleges and workplaces

• Regulatory role covers all qualifications except those awarded by HEIs

• Role is restricted to England– Regulates quals jointly with partners in Wales and NI– Works closely with counterpart in Scotland

Role of QCA

QCA approach to regulation

• Proportionality (interventions are related to risk)• Accountability (the public has a right to see what

QCA does)• Consistency (in judgements made; in data

requested; in criteria used)• Targeting (measures taken related to purpose)• Transparency (openness and visibility)

Regulatory principles for e-assessment

• Published by QCA & regulators in Wales, NI & Scotland– Consultation Oct 06 – Feb 07– Enforceable Oct 07 via monitoring activities– Fully enforceable April 08 via self-assessment

Structure of presentation

• Aims of principles• Background issues that impact on e-regulation

– Few papers addressing regulation of e-assessment specifically

– Standards docs, etc.– Literature from diverse fields

• Discussion– Understanding nature of regulation for e-assessment– Describing further work to be done

Aims of principles

• Robust e-assessment strategy and operations• Guide operations and innovation in consistent way• Extend access to e-assessment for benefit of learners• Identify and address parameters for success and areas at risk for

innovative e-assessment strategy• Ensure all regulation:

– allows for flexibility– promotes and guides innovation– maintains integrity, reliability and validity

• Principles designed to:– maintain public confidence– support ABs who wish to innovate and add value through use of technology

Literature directly describing regulation of e-assessment

• Previous QCA reports– Code of Practice inappropriate for e-assessment

‘Tends to stifle innovation’– Develop ‘higher level principles’– Facilitate dissemination of good practice to ABs

• Exams on Demand predictions – 2004 2014– Drivers of e-assessment uptake

Regular review of regulation to maintain relevance and flexibility Good quality e-learning materials support personalised learning Technical, physical and human resources in schools and colleges Organisational structures in schools and colleges to support personalised learning and

on-demand assessment Greater adoption of MCQs as a valid assessment method Agreed standards for data held in e-portfolios

e-assessment standards, Codes of Practice, etc.

• Sets of documents– ISO23988: International code of practice for use of IT in delivery of assessments– ITC International Guidelines on CB- and Internet-Delivered Testing– ATP Guidelines for CB Testing

• Issues around standards– Scope– Audience– Enforceability– Wording– Granularity of detail– Relationship with other sets of standards– Vision of education quality

Guidance documents

• Include:– QCA, DELLS and CCEA’s: Guide to effective practice in e-

assessment– SQA’s guidelines on e-assessment for schools– SQA’s guidelines for FE– NAA’s effective guide for staff implementing the KS3 ICT test

• Tend to:– Be aimed at learning providers– Give advice about how to run e-assessment sessions

Other relevant topics

History and current trends in regulation

UK government approach to regulation

QCA approach to regulation

New-style regulation as a

transformation of law and decision-

making

Regulation of transformed industries

Discussion

• Need better factual info in several areas• Light regulation might not lead to widespread adoption

– Ways in which pre-existing conditions are non-optimal– Regulator helps create circumstances conducive for innovation

• No ‘one-size-fits-all’ solution that will facilitate e-assessment uptake– Lack of agreed opinions suggest niche solutions– High-stakes, high-volume qualifications present particular issues

Further work

• e-strategy project– Contractor working to define e-strategy for QCA

• Research projects– International approaches to (e-)regulation– Indices for uptake of e-assessment

Continuing to develop the most representative indices Seeking context for indices – e.g. from adoption of other ICT

innovations Using theory to try to understand adoption (e.g. diffusion of

innovations)

– Technical group re e-assessment in general quals