2 dius_self-assessment methodology

7/29/2019 2 DIUS_self-Assessment Methodology

1/18

DIUS: self-assessmentmethodology

Susanna Karakhanyan, PhD

DIUS National Coordinator

ANQAINQAAHE Board of Directors

World Bank Expert

DIUS FINAL CONFERENCESeptember, 2012


2/18

OUTLINE

How it all started: getting to commonunderstanding

Description of processes run by the universities

Mapping a process: identification of gaps

Organization of self-assessment: major steps

Data collection: major sources of data

Production of a self-assessment report.

3/30/2013 DIUS FINAL CONFERENCE2


3/18

How it all started: getting tocommon understanding

Need to establish a common understanding on suchconcepts as quality, quality assurance, self-evaluation and the like;

Quality assurance as fitness for purpose; Interpretation of ESG standards;

Needs analysis per ESG standard;

Allocation of roles among the partners.

Two ESGs per institution; ANQA - external quality assurance;

EU partnersinternational peers in an expert panel;

MoESobservers and evaluators.



4/18

Description of processes run by university

3/30/2013 4

QUALITY QUALITY

ASSURANCE

PERFORMANCE


5/18

Quality management

Planning

Implementing

Evaluationand

feedbackmechanisms

Improving

3/30/2013 5Public Accreditation Agency for Higher Education


6/18

Mapping a process

3/30/2013 Public Accreditation Agency for Higher Education


7/18

Example: ESG 1.4

1.4 Quality assurance of teaching staff:

Institutions should have ways of satisfyingthemselves that staff involved with the teaching

of students are qualified and competent to

do so. They should be available to those

undertaking external reviews, andcommented upon in reports.



8/18

Example: ESG 1.4



9/18

Example: mapping teacher

assessment



10/18

Organization of self-assessment: major steps

Purpose identificationWhy do we need to engage in self-assessment?

Outcomes identificationWhat results do we need to achieve

with the self-assessment?

Measurements and evaluation designWhat methodology,tools and mechanisms for measurement should we use?

Data collectionWhat type of data do we need to obtain?

Analysis and evaluation - What are the methods we are goingto apply to analyse the data?

Decision-making regarding actions to be takenHow are wegoing to use the results of the analysis in the decision-makingregarding new strategies and developments?



11/18

How to put together a valid self-

assessment ?

The questions to be answered are as follows:

What indicators do we have already?

Which indicators do I need to control my process?

Which indicators are out of my influence?

Which indicators do I need for doing the work?

Which indicators do we need for improvement andenhancement?

What is the cost-benefit ratio of collecting indicator data?



12/18

Data collection: major sources

of data

Data collected from the feedback mechanisms, evaluationgroups, which provide for qualitative information;

e.g. questionnaires, which are a powerful element if devised the rightway;

discussion-oriented team workquality circles, peer-reviewers(external and internal);

focus groups, individual interviews and the like;

Statistical data available at the university units and/or statisticaloffice, that is key performance indicators (KPIs) (data like studentsuccess rate, drop-out rate, in-coming out-going students,number of teaching staff, budget allocations and the like);

Performance outcomes (samples) and other evidences (thesispapers, exam questions and respective answers, samples ofpublications, observations of classes and the like).



13/18

Organising a self-assessment


DATE ACTIVITY

8 months before a

planned

assessment

Appoint the leader of the assessment process Compose the

assessment team, including students dividing up the subjects to

be dealt with

The following 6

months

Each person responsible for collecting information and data

collects that informationWriting drafts of the subjects

4 months after the

start

Discussion on the drafts in the group

Second draft

About 5 months

after the start

Discussion of the 2nd draft with all faculty staff and students

during an open hearing

6 months after the

start

Edit the comments of the hearing for the final draft

8 months after the

start

Carry out an (inter-collegial) assessment


14/18

Format: introduction

One page history and current profile of theinstitution;

An overview of the management andorganisational structure, including an organisationchart (or equivalent);

A brief overview of the institutions qualityassurance arrangements and systems,

The scope of the audit, The fitness of QA arrangements for the context of the

institution's charter, profile/objectives and specialcharacteristics.

3/30/2013 14


15/18

Format: the maindocument

The text should be developed per criterion/standard and include

In-depth analysis of quantitative and qualitative data and factualinformation not available in other materials (such as annualreports), cross- referenced to, and within, the main document.

entry qualifications, non-completion rates, degree results,graduate employment data,stakeholder satisfaction data,performance indicators and measures employed that relate tothe scope of the audit.

Effectiveness of a particular approach;

How this information is used in quality control and assuranceprocesses within the institution, giving examples where possible.

Examples of the effectiveness in the operation of qualityassurance systems and instances of particularly good practice

SWOT analysis.

3/30/2013 15


16/18

Appendices

Appendices indicate

how this information is used in quality control

and assurance processes within the institution,giving examples where possible;

examples of the effectiveness in the operationof quality assurance systems and instances ofparticularly good practice;

The referenced documents (e.g. strategic plan,yearly and other plans);

Any other attachments required by the agency.

3/30/2013 16


17/18

Challenges ahead

Re- development of policies and procedures that reallywork;

Re-development of the tools: questionnaires, focus

group discussions and the like;

Wider involvement of stakeholders in the process;

Analytical approach to self-assessment rather thandescriptive;

Using the major findings in the institutional planning andenhancement;

Establishment of a quality culture at institutions.



18/18

THANK YOU and QUESTIONS

[email protected]

[email protected]

3/30/2013 18
mailto:[email protected]:[email protected]

2 dius_self-assessment methodology

Documents