second scoping study - unescounesdoc.unesco.org/images/0023/002301/230149e.pdf · second scoping...
TRANSCRIPT
Arab Network for Quality Assurance in Higher Education
Second Scoping Study
Survey of Quality Assurance and Accreditation in Higher Education in the Arab Region- October 2012
This study was sponsored by the Global Initiative for Quality Assurance Capacity
Building (GIQAC) which is financed by the World Bank and implemented by
the UNESCO.
Arab Network for Quality Assurance in Higher Education (ANQAHE) October, 2012
Table of Contents Table of Contents .................................................................................................................. 2
Foreword............................................................................................................................... 6
ANQAHE Board of Directors................................................................................................... 8
Participants in the scoping study survey................................................................................ 0
Glossary ................................................................................................................................ 1
Executive Summary ............................................................................................................... 2
Objectives of the Study ......................................................................................................... 4
List of Countries that Participated in this Study ..................................................................... 6
Study Design ......................................................................................................................... 7
Good Practice QA Framework .............................................................................................. 10
Survey Development ........................................................................................................... 11
Results and Analysis of the Survey ...................................................................................... 13
Section I. Information about Quality Assurance Agency ...................................................... 16
2. Remit or Scope of QA Agency: .................................................................................... 17
3. Age of the Agency: ...................................................................................................... 18
4. Private VS Public: ........................................................................................................ 19
6. Main Tasks and Goals of QAAs ................................................................................... 22
Section II. Agency Governance/Regulatory Framework ....................................................... 24
2. QAA report: ................................................................................................................. 25
3. Stakeholder groups represented on the Board /Council of QA Agency: ....................... 26
6. Object of Agency communication with its stakeholders................................................ 29
10. Source of QAA budget:.............................................................................................. 31
11. Number of QAA employees: ...................................................................................... 32
12. Financial and human resources: ................................................................................ 33
Section III. Internal Quality Assurance for QAA................................................................... 36
2. Mechanisms that ensure the quality of by subcontractors: ........................................... 36
3. Appeals System:.......................................................................................................... 37
4. Internal feedback and reflection mechanism ................................................................ 38
5. Mechanisms for collection and analysis of feedback from experts and institutions:...... 38
6. Published policy for the assurance of the agency's own quality ................................... 39
7. Processes and results which reflect the agency's mission and goal of quality assuran 40
8. Mandatory cyclical external review of the agency's activities once every five years: ... 40
9. Agency change of approach to evaluation in the past three years ............................... 41
Section IV. Framework for External Quality Assurance ........................................................ 42
1. Compulsory vs. voluntary QA ...................................................................................... 42
2. External quality procedures performed ........................................................................ 42
3. Frequency of periodical external quality assurance procedure:.................................... 43
4. Periodicity of external quality procedure performed: .................................................... 44
5. Formal consequences for HEIs of external quality procedures .................................... 45
6. Formal requirement on the HEI to address recommendations ..................................... 46
7. Importance of different QA constructs/activities in program reviews: ........................... 47
8. Importance of different QA objectives/dimensions in institutional reviews: ................... 48
9. Which model/frame of reference do you use for your external quality procedures? ..... 50
10. Approval of new subjects, programmes and institutions............................................. 52
11. Expected developments in the QA sector in the next three years .............................. 53
Section V. Standards & Methodology for External Quality Assurance................................... 55
1. Formal standards, guidelines, indicators, or criteria to conduct external QA ................ 55
2. Which standards?........................................................................................................ 55
3. Remit of QA procedures: ............................................................................................. 56
4. Stakeholders role in the specification of processes and criteria for the external quality assurance of higher education institutions ....................................................................... 57
5. Final decision in the specification of processes and criteria for the external quality assurance of higher education institutions ....................................................................... 58
6. Publication of relevant processes, criteria and procedures prior to execution of external quality procedures ........................................................................................................... 59
7. Processes used/required within its external quality procedures ................................... 60
8. Appointment of members of the external panels used for external quality procedures . 61
9. Membership of an external review panel ..................................................................... 62
10. Formal terms of reference for external review panels for different external quality procedures? .................................................................................................................... 63
11. Familiarization with external QA procedures: ............................................................. 63
12. Selection of reviewers................................................................................................ 65
Section VI. Site Visit ............................................................................................................ 66
1. Site visit requirement ................................................................................................ 66
2. Who conducts the visit? ............................................................................................... 66
3. Role of members of the QA entity in the site visits ....................................................... 67
4. Duration of the site visit (days only on-site and not briefing or back-office days).......... 68
5. Program of the site visit ............................................................................................... 68
6. Site visit interviews ...................................................................................................... 70
7. Membership of self-evaluation groups in institutions being evaluated .......................... 71
8. Roles during the external QA procedures .................................................................... 72
Section VII. Outcome of Quality Assurance Procedures ....................................................... 74
1. Final decision on the outcome of the external quality procedures ................................ 74
2. Nature of the external QA procedure outcome............................................................. 74
3. Input, consultation, or information about the first draft of the report ............................. 75
4. Publication of the final outcome (assessment/evaluation report).................................. 76
5. Content of the published report.................................................................................... 77
6. Who publishes the report? ........................................................................................... 78
7. Follow up for the implementation of the report's recommendations.............................. 78
8. Nature of follow up....................................................................................................... 79
9. Most problematic areas in the internal quality assurance systems of HEIs .................. 80
10. Development of the quality assurance situation ......................................................... 81
11. Outlook of the higher education/quality assurance situation ...................................... 82
12. Obstacles to the development of quality assurance of higher education .................... 83
Section VIII. Capacity Building in Quality Assurance for Higher Education .......................... 85
1. Responsibility for QA capacity building for HEIs .......................................................... 85
2. Type of QA capacity building ....................................................................................... 86
3. Type of capacity building used:.................................................................................... 87
4. QA capacity building most needed by HEIs ................................................................. 88
5. Importance of different formats of QA capacity building for higher education ............... 90
Section IX. Arab dimension of Quality Assurance ................................................................ 91
2. Relationship/ affiliation with any other national/ transnational/ international agency..... 92
3. Participation in the 2008 scoping study of ANQAHE .................................................... 92
4. Participation to ANQAHE events in the past ................................................................ 93
5. Importance of ANQAHE objectives .............................................................................. 93
6. Relevancy/importance of ANQAHE objectives............................................................. 94
Conclusions and Recommendations: ................................................................................... 96
Specificities of Arab QA Entities ....................................................................................... 97
Internal Quality Assurance for QAAs ................................................................................ 98
Framework for External Quality Assurance....................................................................... 99
Standards & Methodology for External Quality Assurance .............................................. 100
The Site Visit.................................................................................................................. 101
Outcome of Quality Assurance Procedures..................................................................... 102
HEIs’ challenges............................................................................................................. 102
Capacity Building ........................................................................................................... 103
Arab Dimension of Quality Assurance ............................................................................ 104
Recommendations for ANQAHE:........................................................................................ 105
Impact of QA in the Region and Recommendations for QAAs............................................ 107
Appendix A: Details of Participants .................................................................................... 117
Appendix B: Question match between the 2 seeping studies (2008 & 2012) ..................... 122
Appendix C: 2012 Survey Questionnaire............................................................................ 126
Foreword The Arab Network for Quality Assurance in Higher Education (ANQAHE) is an
independent, non-profit, non-governmental organization that was established in
2007. It has 10 full members and 5 associate members which altogether constitute
68% of the Arab Countries, up from 50% three and half years ago when the first
scoping study was conducted. The goals of ANQAHE are: (1) to support and
enhance quality assurance organizations in the Arab region; (2) to establish a
mechanism of cooperation for Quality Assurance (QA) in higher education in the
Arab countries; (3) to initiate and sustain both regional and international initiatives
on quality assurance in higher education; (4) to exchange information and provide
advice; and (5) to develop a system for capacity-building for quality assurance
bodies in the Arab region. The website of ANQAHE is www.anqahe.org
ANQAHE is carrying out a second study on the state of quality assurance in Arab
higher education as part of its activities. This follows a first scoping study which was
published in March 2008. Other than depicting progress in quality assurance of
higher education in the Arab region since the last study was performed, this second
scoping study highlights the following: first, the regulatory and governance set-up of
quality assurance agencies in the region; second, the mechanisms of quality
assurance for the QAA itself; third, the QA framework within which external quality
assurance is performed whether in the presence or absence of a formal QAA; fourth,
the methods and procedures of external QA; fifth, the outcomes of external QAA;
sixth, the QA capacity development needs of QAAs and higher education institutions;
and seventh, the prospects and desirability of an Arab Higher Education Area
(AHEA). These last two points were an addition to the first scoping study.
ANQAHE started this work in December 2011 and completed it in June 2012. By the
end of the study two reports will be sent to each party that has participated in the
study; a collective one and a specific one comparing its performance to the
INQAAHE good practice one.
This study was sponsored by the Global Initiative for Quality Assurance Capacity
Building (GIQAC) which is financed by the World Bank and managed by the
UNESCO.
The number of participants who responded to the survey exceeded three quarters of
the Arab countries. All those who did not respond have not yet established a quality
assurance agency. This denotes the enthusiasm and the keenness present in the
region towards quality assurance in higher education. All respondents predicted a
positive outlook for Quality Assurance in their country’s higher education.
The board of ANQAHE is grateful for the support it has received from all the Quality
Assurance organizations in the Arab Region. It was because of their endeavor that
this study was successfully completed. Without their support and dedication to fulfill
the survey we would never be able to generate this work.
The ANQAHE Board of Directors has strongly supported the production of the
scoping Study. The consultant, Dr. Sofiane Sahraoui, has done a magnificent job in
collecting and analyzing the data and in writing and editing the draft report. The
achievement of this study has taken significant time and effort from our entire staff,
especially Dr. Nadia Badrawi, President of ANQAHE and Dr. Tariq Al-Sindi, Secretary
General. It is due to their dedication and hard work that this project has been
completed in its final successful form.
NADIA BADRAWI,
President,
Arab Network for Quality Assurance in Higher Education
ANQAHE Board of Directors
Dr. Nadia Badrawi President
Dr. Tariq Alsindi Secretary General
Dr. Badr Aboul Ela Vice president
Dr. Jawaher Al Mudhahki Vice President
Dr. Mohammed M Al-Subu Treasurer
Dr. AbdAllah Mussalam Member
Dr. Salim Razvi Member
Dr. Magdi Kassem Member
Dr. Essam Mostafa Member
Dr. Munib Al-Saket Member
Dr. Farida Ali Member
Dr. Sultan Abo-Orabi Co-opted member
Dr. Ahmed Al-Jammal Co-opted member
Dr. Issam Al-Naqib Co-opted member
Participants in the scoping study survey on the Quality Assurance in Higher Education in the Arab Region
Country Agency/Entity Participant
Libya CQAAE&TI Mohamed Elkabir Libya CQAAE&TI Abdullah Abduljalil Muhammad
Sultanate of Oman Oman Academic Accreditation Authority
Salim Razvi
Lebanon Ahmad Al JAMMAl
Kingdom of Bahrain Quality Assurance Authority for Education & Training
Dolina Dowling
UAE Sudan
The Commission for Academic Accreditation
Evaluation and Accredation Commission (EVAC)
Badr Aboul-Ela Isam M Mohamed
Egypt
Tunisia
National Authority for Quality Assurance & Accreditation of Education
Ministry of Higher Education and Scientific Research
Maha Farouk Rashwan
Oussama BEN ABDELKARIM
Kuwait Council of Private Universities Farida Mohamed Ali
Yemen Higher Council of Higher Education Quality Assurance
Muhammad Mahfoudh Abdallah Elhid
Qatar Council for Higher Education Khaled Muhammad Elhasan Syria Ministry of Higher Education Muhammad Nejib Abdelwahed Eritrea Not established yet Muhammad Othman Ali
Jordan
Accreditation Authority for Jordanian Higher Education organizations
Rim Hasan Fayadh Elsokar
KSA
National Commission for Academic Assessment and Accreditation
Saad Al-Zahrani
Palestine Accreditation and quality assurance commission
Mohammed Alsubu
Iraq Salah Al Nuaimy
Glossary
ANQAHE: Arab Network for Quality Assurance in Higher Education
HE: Higher Education HEI: Higher Education Institutions KSA: Kingdom of Saudi Arabia QA: Quality Assurance QAA: Quality Assurance Agency UAE: United Arab Emirates
Executive Summary This study is an update to the scoping study performed by ANQAHE (Arab Network
for Quality Assurance in Higher Education) four years ago (three and half years ago
exactly). The objective of the first study was to assess and report the situation of
Quality Assurance organizations in the –participating- Arab States in order to identify
the gaps in the system and contribute to the construction of a framework for QA in
Higher Education.
This study follows in the footsteps of the first one by reinvestigating the same issues
but also tackling additional ones. The objective is to gauge the development that
took place in the span of three and a half years since the first scoping study was
conducted.
The completion of this study required the development of a new survey, delivering it
to all countries of the region, collecting and analyzing the results, and finally
generating the final report. Further individual country reports and a capacity
building strategy for QA in the region will be generated separately from this report.
17 different countries participated in the survey; one more than in the scoping study
of 2008. Incidentally, the respondents were roughly the same with only one country
dropping (Morocco) and another two joining in, namely Qatar and Saudi. This will
give more meaning and relevance to the comparisons between the 2 scoping
studies. Eleven (11) of the participant countries have established QA entities for
higher education, and were asked to answer all survey questions. The remaining six
(6) were asked to answer only parts of the survey which did not relate to the
existence and activities of a formally established QA agency.
Overall, the study key findings were in line with those of the previous scoping study
lest for the normal growth and progress over the last three and half years: Financial
and governance challenges are still the major obstacles to a faster development in
the region of QA in higher Education. The similarities between the Arab QAAs,
namely in terms of reference standards, review panels, site visits and appeal
mechanisms were confirmed once more. However, there were marked differences
as well as they relate to recognition of national diplomas, the autonomy of the
agencies, etc.
However, this study revealed new trends in the behavior of Arab QAAs such as
getting more involved in and taking more responsibilities in the performance of
external QA procedures (such as site visit and the writing of the final assessment
report). They are also diversifying their sources of revenue namely by charging for
the QA services they offer, but still fall short on securing grants from donors. There
is an increasing concern with the employability of graduates and labor market
representatives are systematically represented in QAA Boards whenever they exist.
Their internal quality assurance systems have also been significantly reinforced
compared to three years ago.
One major contribution of this study is the accurate identification of the most
important and urgent needs of both Arab QAAs and HEIs in QA capacity building.
This will be followed by a proposal for a QA capacity building strategy as an offshoot
of this study.
Finally, the Arab dimension of HE was probed with participants and most have
expressed their wish for an Arab Higher Education area to come into existence,
despite the differences in systems of HE among the different countries. This bodes
well for a sustained role of ANQAHE in the region.
Objectives of the Study Following on the footsteps of the 2008 scoping study, the current study has the
following key objectives:
1. Assess the situation of quality assurance organizations in the Arab Region:
This will be done both for countries with established QA entities as well as
countries with no such entities. In the absence of a formal entity for QA, the
study will look at QA activities in the country. Results will be reported on a
number of QA dimensions that have been embedded in the survey and are
further explained as part of the survey development process.
2. Identify gaps in the deployment of QA in the countries studied: This will be
reported at an aggregate level in this report but will be detailed for each
country in individual country reports that will be sent to each country
separately.
3. Assess the impact of QA in the region: The analysis of results throughout will
highlight changes in the QA landscape in the region. Moreover, a dedicated
section at the end of the report will attempt to draw inferences on the impact
of QA in the region based on a triangulation of results; especially those that
relate to the outcomes of the QA process collected through section VII of the
survey questionnaire.
4. Design a generic plan for future action including a short-term and a long-term
strategy based on a good practice plan that will be used for benchmarking
purposes and plot specific action items for the different countries accordingly.
An overall capacity building strategy will be proposed and will serve as key
input for ANQAHE’s own strategy over the next 3 years. Moreover, specific
capacity development initiatives will be proposed to individual countries as
part of the individual country benchmarking reports that will be produced
following this report.
5. Baseline results of the current scoping study against results of the early
scoping study performed in 2008 and identify changes thereof. These
changes will be identified progressively as results are interpreted. The most
critical changes and their impact on the QA scene will be further reassessed in
the concluding sections of the report as part of the conclusions and
recommendations sessions as well as the overall impact assessment of QA in
the region.
List of Countries that Participated in this Study:
6 7
1 5 17 10
8 11 2 3 12
13 9
14 16
15
4
1. Tunisia
2. Libya
3. Egypt
4. Sudan
5. Palestine
6. Lebanon
7. Syria
8. Jordan
9. Saudi Arabia
10.Kuwait
11.Bahrain
12.Qatar
13.United Arab Emirates
14.Oman
15.Yemen
16.Eritrea
17.Iraq
Study Design
The study was designed following a scientific approach using a conceptual modeling
and empirical design process. The steps involved in the design are described below:
1. Conceptual development of a good practice QA framework based on literature
review and validation with ANQAHE/QA experts in the region:
The scope of the QA assessment was defined based on a good practice QA
framework which provided the QA dimensions that were investigated in this report.
Many of these dimensions were already present in the previous scoping study and
few others have been added to reflect similar surveys in other regions of the world.
Members of the ANQAHE Board were used for validating the dimensions of the QA
framework as part of the survey review; wherein both structure (QA model) and
content (items relating to QA constructs) were verified.
2. The development of a survey questionnaire to assess the QA system in each
of the 22 Arab States and to study the impact of QA systems on HEIs & on
the quality of higher education in general;
Based on the QA framework identified in (1), a survey questionnaire was developed
and undertook several review iterations primarily from ANQAHE President Dr. Nadia
Badrawi and other Board members. Two research assistants provided their feedback
on the different versions of the questionnaire. The process of deriving questions
followed a conventional instrument development process wherein a number of
measurement items were developed for each of the QA constructs identified as part
of the QA framework for this study. The survey was further mapped to the original
instrument used for the scoping survey of 2008 to facilitate comparisons. A test
version of the final draft was sent to a number of persons who are familiar with QA
in higher education to answer it. Final changes were made accordingly and the
questionnaire subsequently posted on an on-line survey engine (KwikSurveys).
3. Data Collection
The survey was translated from English to Arabic and French. Only one
participant answered in French (Tunisia). The rest of the answers were in Arabic
(10) and in English (6). Each potential respondent was sent two versions of the
questionnaire (Arabic and French for the French-speaking countries of North-Africa
and Arabic and English for the rest). Lebanon and Djibouti were sent all three
versions. An invitation link to answer the questionnaire was included within an e-
mail that was automatically fired by the survey engine of KwikSurveys. Kuwait was
the first to answer the survey and Gulf countries in particular were rather prompt in
their responses.
Seventeen (17) countries answered out of a total of 22, making up a response rate
of 77% approximately; which is 5% higher than the scoping study of 2008 (one
additional respondent). The countries that provided answers are (in chronological
order of response): Kuwait, Libya, Yemen, Saudi Arabia, Oman, Tunisia, Lebanon,
Bahrain, Syria, Jordan, Qatar, Eritrea, Sudan, Palestine, United Arab Emirates, Egypt
and Iraq. Countries that did not respond include Mauritania (not reachable),
Morocco (failure to respond despite visit to QA entity premises), Algeria (failure to
respond despite multiple e-mails and follow-up calls), Somalia (not reachable),
Djibouti (no answer), Comores (not reachable). Therefore the actual response rate
is around 85% if we discount countries that could not be reached and did not
receive the survey questionnaire.
Compared to respondents of the scoping study of 2008, one respondent dropped out
(Morocco) and an additional 2 came in (Qatar and Saudi). The rest are the same
which provides a reliable basis for comparison between the two studies.
Informal face-to-face interviews were used to complement survey data and probe
strategic and qualitative issues. These were integrated in the interpretation of the
results along with the survey findings but are not shown separately as study findings
as they were not part of any formal information gathering process.
The data collection process was very tedious and obtaining the 17 responses was no
easy task. It took almost 5 months. The special efforts of ANQAHE President and
Secretary General were instrumental to reach this satisfactory level of answers. One
reason could be the length of the questionnaire which included 90 questions and
required no less than one hour to fill. The second is the lack of motivation and
commitment of respondents. In one instance, the consultant visited a respondent in
his office but failed to receive a response despite the pledged commitment. Some
ANQAHE Board members were equally late to respond. Further recommendations
are made at the end of the report regarding data collection for future similar
surveys.
4. Quantitative analysis of results within the constructs of the good practice QA
framework
The results were automatically compiled by the on-line survey engine
(Kwiksuveys) into data sheets. These were downloaded into Excel to generate
relevant graphics for further analysis as part of the final report.
5. Report of the survey outcomes with a comprehensive analysis of its
implications for QA in the region.
The graphical display of results was combined with qualitative analysis to
generate conclusions about each aspect of QA surveyed. Whenever applicable,
comparisons to results of the 2008 scoping study were made. The final sections of
the report include a synthesis of the overall results and conclusions as to the state
and impact of QA in the region with a special focus on its evolution during the last
three years.
Good Practice QA Framework
The following QA framework was used to derive the constructs of the study which
were then used as a basis for developing questions for the survey. As a framework,
it is not meant to be normative and to indicate any direction of causality; it simply
scopes the domain of QA in higher education.
Survey Development
As stated above, the survey questionnaire was developed based on a good practice
QA framework (see Figure above). A number of QA constructs were subsequently
derived. Questionnaire items were developed for each of the constructs, with some
being borrowed from the survey instrument of 2008 to facilitate comparisons.
Though no formal pilot study was conducted, four (4) people with knowledge about
QA in higher education were asked to fill the survey mostly for purposes of face
validity. The questionnaire was further vetted by the ANQAHE Board and cleared for
distribution accordingly.
The questionnaire is composed of 9 sections that roughly correspond to the QA
constructs of the good practice framework
- Section I: Descriptives about the Quality Assurance Agency, like
denomination, scope or remit of work, year of establishment, mission, etc.
- Section II: Agency Governance/Regulatory Framework trying to outline the
governance framework within which the QAA exists, its mode of inception,
decision-making, reporting structure, stakeholder involvement,
communication, QAA accountability, staffing, funding, etc. .
- Section III: Agency's Internal Quality Assurance. This part investigates the
procedures and processes that Arab QAAs use to assess the quality and
integrity of their own work. This includes questions on conflict-of-interest
provisions, due process and appeals, self-assessment, external review of the
QAA, etc.
- Section IV: Framework for External Quality Assurance. This core component
of the questionnaire examines the various procedures, processes, references,
consequences and outcomes that Arab QAAs had adopted to pursue their
external quality assurance mandate.
- Section V: Standards & Methodology for External Quality Assurance. This
part of the survey explores the standards and methodologies used by Arab
QAAs in their external quality procedures; who contributes to their definition
and how they are implemented.
- Section VI: Site Visit. A cornerstone of the external review procedure, the
site visit is examined from various standpoints of who participates, format and
duration, program and activities, etc.
- Section VII: Outcome of Quality Assurance Procedures. This section of the
survey deals with the outcome of the external QA review procedures and their
consequences and follow-ups. There are also questions about the general
impact and evolution of QA in the country.
- Section VIII: Capacity Building in Quality Assurance for Higher Education.
This section tries to depict the current practices and perceived needs of QA
capacity building;
- Section IX: Arab dimension of Quality Assurance. This final section
investigates the relation that Arab QAAs hold with their counterparts and with
QA networks in the region and beyond. It also seeks to position the mandate
of ANQAHE within the priorities of the different QAAs
Results and Analysis of the Survey
Twelve (12) out of the 17 respondents belonged to a QAA whereas the remaining 5
operated from within alternative structures of QA in higher education, all located
within the Ministry of Higher Education or a similar authority (see Appendix A listing
all QA entities and their details). This gives a 70/30 distribution of countries with
QAAs versus those who do not have one yet. Moreover, it is expected that none of
the remaining 5 countries that did not answer the survey has a formal and self-
standing QA agency. This would put the total number of countries that have not yet
established formal and independent QAAs at a staggering 45%; a rough indication of
the reluctance of some countries to tackle the endemic problem of higher education
quality.
Does your country have a Quality Assurance Agency?
Yes No
29%
71%
The 5 countries that do not have yet a QAA are Lebanon, Tunisia, Eritrea, Qatar and Syria. As a result, no responses will be reported for these countries for the first three sections of the survey which are specific to the existence of a formal QA authority in the country. Respondents that belong to the 5 countries above were asked to branch into Section IV directly “Framework for External Quality Assurance”. The reader has to keep in mind that the first 3 sections include responses from the 12 respondents that indicated a formal QAA in their country. Percentages should be interpreted accordingly. Matching between raw scores and percentages for the first three sections is shown in the table below. Results will be displayed in percentages to facilitate comparison with the 2008 scoping study.
Table of Percentages of Respondents and corresponding number of actual respondents:
For Sections I to III:
N° of Respondents Percentage (%)
1
8
2
17
3
25
4
33
5
42
6
50
7
58
8
67
9
75
10
83
11
92
12
100
For Sections IV to IX:
N° of Respondents Percentage (%)
1
6
2
12
3
18
4
24
5
29
6
35
7
41
8
47
9 53
10
59
11
65
12
71
13
76
14
82
15
88
16
94
17
100
Section I. Information about Quality Assurance Agency
1. Organizational Denomination of the Agency:
%45
40
35
30
25
20
15
10
5 0
0
41.67 25
0.00 0.00
16.67 16.67
Two thirds (67 %) of the organizations are either a commission or an authority. Interestingly enough, none was called agency, unit or department. In the 2008 Scoping Study, there was also no ‘Centers’, whereas two respondents in this study dubbed their organizations ‘Centers’. Furthermore, 2 other names were proposed: Council of Private Universities (Kuwait) and Supreme Council for Quality Assurance of Higher Education (Yemen).
2. Remit or Scope of QA Agency:
A specific discipline or field of learning 8.33
Research Units/Centers 8.33
Pre-university educational institutions
Vocational Education and Training Institutions
41.67
50.00
Universities
91.67
0 20 40 60 80 100
Percentage
Most QA entities cover the quality of teaching and learning at universities (92 %)
with half of them (50%) encompassing vocational education and training. Pre-
university education is also within the remit of about 42% of the respondent’s QA
activities. However, only one entity does quality checks on dedicated research
units/centers (Egypt) and another on a specific discipline or field of studies (Libya).
In Tunisia, the framework of QA activities is scattered and managed by the Ministry
of Higher Education as a portfolio of QA activities rather than a formal entity with a
formal mandate. Beyond the selection offered to respondents, additional remits of
QA that were suggested include ‘Higher Colleges’(Sudan), ‘research institutions that
award qualifications’ (Egypt), ‘all educational levels including maternity’ (Libya) and
‘national examinations’ (Bahrain).
3. Age of the Agency:
%70.00
60.00
64.71
50.00
40.00
30.00
29.41
20.00
10.00
5.88
0.00 Before 2009 In the last 3 years
(2009-2011)
Still being planned
Two thirds of the HE QA bodies were established before 2009 and hence existed at
the time the previous scoping study was conducted. Only one was established since
(Yemen). An additional five are still being planned with 2 (Lebanon and Tunisia) at a
very advanced stage of creation. This suggests that the overall QA scene in the
Arab region is slow to progress. For instance, Tunisia has had a higher education
law prescribing a QA entity since 2008 and has been very slow to implementing it.
Likewise, Lebanon has engaged wide consultations for the institution of a QAA with
all stakeholders in place; however it has not formally come into existence yet.
Per
cen
tag
e
4. Private VS Public: - Covered Institutions:
90.00 80.00 70.00 60.00 50.00 40.00 30.00 20.00 10.00
0.00
16.67
Private Higher
0.00
Public Higher
83.33
Both Private and
Education Institutions Education Institutions (HEI)
Public
Most of QA entities (83%) cover both Private and Public higher education
institutions, with 2 covering only private HEIs (UAE and Kuwait). There has been
very little change from the results of 2008 although private HE is being now given
more scrutiny with dedicated QA bodies/activities.
- Used Procedures: Are public and private HEIs subject to the same QA procedures?
90.00
80.00
70.00
60.00
50.00
40.00
30.00
20.00
10.00
0.00
83.33 16.67
Yes No
The answers to the above question show that in 83% of the cases, public and
private HEIs undergo the same external quality assurance procedures. Only 2
entities (Kuwait and Libya) had different procedures for public and private HEIs.
That is a notable increase from 3 years ago wherein only 69.2 % of the QAA applied
the same standards and processes for external QA irrespective of the public or
nu
mb
er o
f in
stit
uti
on
s
private status of the University. This illustrates a maturation of the QA process
which tends at becoming uniform across the HE field.
- Number of educational institutions within QAA’s primary domain:
Average Number of Educational Institutions per Arab Country
12000
10000
8000
6000
4000
2000
0
96 47
10283 1106
As expected, the number of public HEIs is significantly higher than private ones. On
average, there are twice as many public HEIs as private ones. This number peaks at
tenfold for public schools at pre-university level in comparison with private ones.
These numbers relate only to institutions that come under the purview of the QAA
external control which covers roughly all HEIs in the concerned countries but not all
pre-university schools.
5. Other authorities responsible for the external quality assurance of HEIs:
Are there other agencies/authorities responsible for the external quality assurance of HEIs?
Yes 17%
No 83%
Most of the respondents (83%) confirmed that there are no other QA bodies
responsible for the external quality assurance of HEIs. This means that, once
established, the QA Agency (QAA) is solely responsible for all external quality
assurance activities.
The respondents that answered with ‘Yes’ referred to a ‘Knowledge & Human
Development Authority’ (UAE) and a soon-to-be launched commission by UNESCO
(Jordan). The latter is not thought to be an alternative QA authority however and
rather belongs to the programmatic HE activities of multilateral organizations like
UNESCO and the World Bank for instance.
6. Main Tasks and Goals of QAAs
External quality assurance of… 76.92 7.69
Development and maintenance…
Recognition of national diplomas
46.15 30.77
15.38 30.77
Unimportant
Collecting/ disseminating…
Quality enhancement/improvement
External quality assurance of…
Deciding on the funding of…
0.00
61.54
76.92 84.62
58.33
7.69
0.00
0.00
Moderately important
Important
Very Important
Not a function of my agency
Recognition and licensing of… 46.15 30.77
0% 20% 40% 60% 80% 100%
QA bodies at the participating Arab countries seemed to share the same QA agenda
as illustrated by the common tasks and goals reported:
- Recognition and licensing of higher education institutions: While two
thirds reported it to be as important or very important, 30% considered it not to be
a function of the agency. Generally recognition and licensing are within the remit of
a different licensing authority and not a function of QAAs but many of the external
QA procedures performed by QAAs are seen to be important in connection with it.
- Deciding on the funding of higher education: Again most of the respondents
(59%) did not identify it as a function of their agency and few thought that it was
important (17%), or moderately important (17%). This is again thought to be part
of an overarching HE authority.
- External quality assurance of programmes: There is a consensus here that
this is very important or relevant to the QAAs by 85% of respondents. The
remaining 15% equally think that it is either important or moderately important.
The question was rhetorical in nature and meant to check whether there existed out
there QAAs with non-conventional agendas.
- Quality enhancement/improvement: Again all respondents considered this to
be a core objective of Arab QAAs as three quarters rated it very important while the
fourth quarter considered it important.
- Collecting/ disseminating information on quality of HEI: This also seems to
be of great relevance to Arab QAAs with 62% considering it very important and 23%
considering it important.
- Recognition of national diplomas: Some of the respondents (15%) thought of
this as either unimportant or of little importance. Also, more than 30% do not see it
as a function of their agencies to begin with. The remaining half considered it as
either important or very important. The disparity derives from the differential
mandates that QAAs have in the different Arab countries. Generally, QAAs are not
entrusted with degree recognition but in many cases undertake external QA to verify
such degrees.
- Development and maintenance of the qualification framework: Three
quarters of the respondents thought this as either very important or important. The
remaining quarter thought of it as either not a function of the agency or of little
importance. Yet this result seems to be in contradiction with the number of QAAs
that are entrusted with their countries’ qualification framework. This denotes that
nonetheless, QAAs perceive a forthcoming impact of qualification frameworks on the
external QA of HEIs.
- External quality assurance of institutions: As relevant as the external
assurance of programmes, more than three quarters unanimously consider it as very
important while the rest mostly think of it as important (only one respondent thinks
it should not be a function of the QAA).
Additional tasks that were identified as of great importance for QAAs include
“ranking and classification of universities and programmes”; “assessing faculty
performance”; and “defining faculty qualifications.” All of these emerged from the
Jordanian respondent and denote an idiosyncratic agenda.
Section II. Agency Governance/Regulatory Framework
1. Legal/regulatory basis of QAA:
0%
25%
25%
50%
Law
Presidential/Royal decree
Ministerial decree
Independent entity/NGO
Half of the respondents indicated that their QA agencies were established by law while the rest said they were established either by a Presidential/Royal decree or a ministerial decree in equal proportions (25%). There has been an evolution from results of 3 years ago where most QAA’s were established ‘by Royal or Presidential decree’. However and given that only one new QAA (Yemen) has been established since 2009 (question 6), this begs the question of whether the legal status of existing QAAs has changed in the meantime. This is unlikely however.
Per
cen
tag
e
kin
g/pr
esid
ent
Prim
e M
inis
ter/
Ca
bin
et
Min
ist r
y (h
ighe
r e
duca
tion
aut
horit
y)
Par
liam
ent
/sim
ilar e
lect
ed
body
Gov
erni
ng B
oar
d (e
x. B
oard
of
Dire
ctor
s, B
oar
d o
f Tru
stee
s, B
oard
of
Ad
min
istr
ato
rs)
Fun
din
g e
ntity
Age
ncy
is f
ully
aut
ono
mou
s an
d do
es
not r
epo
rt to
any
ext
ern
al e
ntity
2. QAA report:
50
45
40
35
30
25
20
15
10
5 0.00 0
41.67 41.67
16.67
25.00
0.00 0.00
QAAs generally report to the ministry of higher education in their country and/or the
Prime Ministry/Cabinet Affairs (42% both). This being said, 25% report to a
governing Board. Finally, about 17% report to a Parliament or a similar elected
body. QA Agencies report to more than one entity and all report to their Boards
whenever they exist. The drive of independence is seen in the significant number of
QAAs that report to the Prime Ministry/Cabinet Affairs. However, almost half still
report to the Ministry of Higher Education which sometimes creates a conflict of
interest as the same Ministry runs public higher education, yet oversees the
evaluation of private higher education as well.
In comparison with the last study, the results indicate a change in the reporting of
QAAs as they no longer report to the King or the President (15% used to) but to
institutional bodies instead.
Per
cen
tag
e
Gov
ernm
ent
Hig
her
edu
catio
n in
stitu
tions
Pro
fess
iona
l org
aniz
atio
ns
Na
tiona
l qua
lity
assu
ranc
e co
mm
unity
Inte
rnat
ion
al/r
egi
ona
l qua
lity
assu
ranc
e co
mm
uni
ty (e
x.
AN
QA
HE
/INQ
AA
HE
)
Indu
stry
and
lab
our
mar
ket
repr
ese
ntat
ives
(uni
on,
empl
oye
rs, e
tc.)
N
atio
nal a
ssoc
iatio
n of
HE
Is
(suc
h as
Rec
tors
’ co
nfer
ence
/com
mitt
ee)
Stu
dent
s
3. Stakeholder groups represented on the Board /Council of QA Agency:
70
60 50.00
50
40
30
20
10
0
66.67
8.33
25.00
8.33
50.00
8.33
0.00
From a governance point of view, it is important to depict the composition of the
QAA Board/Council and identify the stakeholders that are directly involved in its
work.
As expected, HEIs are most represented with 67% while the government is
represented in 50% of the cases. Government representation is generally achieved
through the Ministry of Higher Education. Industry and labor market representatives
are also represented in 50% of the cases which is logical given the increasing
importance of the employability imperative for higher education graduates. The
national quality assurance community represented by experts known for their
familiarity with QA and higher education came in fourth with 25%. Only one
institution indicated that they have representation from a professional HE
organization (such as the Rector’s Conference) or an international representation.
This is mostly due to the absence of such organizations in the Arab World. Students
are not represented at all symbolizing the marginal role played by students
throughout the higher education governance system in the Arab World.
Per
cen
tag
e
Pre
side
nt/M
ona
rch
Prim
e M
inis
ter
Min
iste
r of
Hig
her
Edu
catio
n or
oth
er
Min
iste
r
Oth
e r g
over
nmen
tal
auth
orit
y
Ele
cte
d go
vern
ance
bo
dy O
the
r
Egypt’s QAA body includes an Al-Azhar University deputy as ex-officio, a
representative of private HEIs and some eminent figures representing pre-university
education.
4. Appointment of the members of the QAA Governing body:
50 45 40 35 30 25 16.67 20 15 10 5 0
25.00
50.00 0.00 0.00
8.33
Primarily, the members of the QAA governing body are appointed by the Minister of Higher Education or other Minister (50% of the cases). In 25% of the cases, they are appointed by the Prime Minister. In only 17% of the cases were appointments made by a President/Monarch.
In some cases, the nomination process is not straightforward as with Jordan: “The Council of Ministers establishes -upon the recommendation of the Prime Minister- a panel called the Council of Higher Education Accreditation. The Council President is subsequently endorsed by Royal decree”.
In comparison with the last study, the Minister of Higher Education is still the primary responsible for appointing the members of QAA governing body. However, the percentage of Presidential or Royal appointments has decreased, which follows a trend of lesser involvement of the President/Monarch in the work of QA agencies.
Per
cen
tag
e
Ma
inta
ins
a W
ebsi
te/P
orta
l
Pub
lishe
s in
stitu
tiona
l lit
era
ture
Ho
lds
aw
aren
ess
wo
rksh
ops
Inte
grat
es s
take
hold
ers
w
ithin
its
gove
rna
nce
stru
ctur
e
The
re is
no
or li
mite
d co
mm
unic
atio
n
5. Agency communication with its stakeholders
90.00 80.00 70.00 60.00 50.00 40.00 30.00 20.00 10.00
0.00
75.00
83.33 83.33
50.00
16.67
Predominantly, QAAs maintain active communication with their external stakeholders (HEIs, Public, Government, etc.) through three media: an online portal/website, institutional literature (newsletter, brochures, etc.) and awareness workshops. For the sake of institutionalizing such communication, stakeholders are integrated within the governing body of the agency in 50% of the cases (ex. Board of Directors, Consultative Committees, Standing and Ad-hoc committees with stakeholder membership, etc.). Only 16% (2 entities) believe that there is little or no communication between their agency and its external stakeholders. Usually, this is symptomatic of a bureaucratic/punitive approach to QA rather than it being developmental.
This question was not addressed in the last study, but as QAAs mature and become more established, they need to go beyond the technical work of external QA verification to become a communication hub and a knowledge portal about QA. This is all the more difficult within an Arab administrative environment that is premised on the bureaucratic supremacy of the State. The results above show that communication is increasingly accepted as an ongoing process that should not be limited to the dissemination of the agency results. In the absence of a similar question in the study of 2008, there is no basis to draw definite inferences.
Per
cen
tag
e
Mis
sion
, vis
ion,
an
d st
rate
gic
obje
ctiv
es/p
lan
Gov
ern
anc
e an
d m
anag
emen
t st
ruct
ures
an
d p
olic
ies
QA
pro
cess
and
re
spon
sibi
litie
s of
diff
eren
t par
tner
s
QA
eva
luat
ion
resu
lts
Oth
er
6. Object of Agency communication with its stakeholders
90.00
80.00
70.00
60.00
50.00
40.00
30.00
20.00
10.00
0.00
75.00
41.67
83.33 83.33
16.67
While asking about what gets communicated, it is done in a rhetorical way for the “QA process (handbooks, indicators, methodology) and the responsibilities of the different partners” as well as the “QA evaluation results” which are communicated by all QAAs except the Kuwaiti agency for the former and the Sudanese agency for the latter. However, it is instructive that the “Mission, vision, and strategic objectives/plan” of the agency are heavily communicated (75% of the cases). This is a positive development that shows that QAAs are keen on conveying their strategic role in HE and do not only operate as a punitive arm of the HE edifice in their country. Only two agencies however communicate anything about “Governance and management structures and policies” which shows that to a large extent, the inner workings of QAAs remains opaque.
Beyond information specific to QA and the work of the Agency, some agencies communicate a variety of reports and administrative forms relating to HE (KSA), and “Capacity building training materials, National surveys, and good practices” (Egypt).
Per
cen
tag
e
7. Formal accountability procedures for Agency work
Formal Accountability Procedures for Agency Work
No 17%
Yes 83%
As an early sign of the sensitivity of QAAs to their public image of integrity and good governance, 83% of the QA agencies in the participating countries are subject to formal accountability procedures.
8. External review of the QA agencies:
50 45 40 35 30 25 16.67 20 15 10
5 0
My Agency has been
8.33
My Agency doesn't plan to
50.00
My Agency
plans to
25.00 My Agency is
currently externally subject itself to subject itself to preparing for reviewed external
reviews in the future
external reviews in the
future
being externally reviewed
The results show that about 17% of QAAs have already been externally reviewed while a quarter is currently preparing to do so. Half of them plan to undergo external reviews in the future while 8% have no intention to do so.
This shows clearly that Arab QAAs are or plan to be in line with best practice for their governance. In the 2008 scoping study, 61% chose not to answer the question whether their agency undertook either internal or external review of their activities.
Per
cen
tag
e
Per
cen
tag
e
This was probably due to the fact that an external review of a ‘government department’ was probably an alien concept at the time.
9. External reviewer of the agency
70
60
50
40
30 16.67
20
10
0
0.00
66.67 16.67
Ministry Agency funders
International Agency body (ex. ENQA, ANQAHE, INQAAHE)
Other
The results show that 67% of the QA agencies either have been or plan to be externally reviewed by an international body, way ahead of those planning a review by their national Ministry (17%). In two cases, external QA bodies were suggested as reviewing entities, namely the “Bahrain Center of Excellence” for Bahrain, and an “ISO” reviewing body for Libya. Bahrain is a case in point where aside from the Bahrain Center of Excellence whose reviewing is mandated by the Government, also plans to perform an international external review. 10. Source of QAA budget:
80 71.42
70
60
50
40
30
20
10
0 Government
budget
30
HEIs (membership
or levy)
35
Fees for QA
services delivered
6
Grants
Nu
mb
er o
f em
plo
yees
Nu
mbe
r of
man
age
men
t st
aff
Nu
mbe
r of
ad
min
istr
ativ
e st
aff
Nu
mbe
r of
ext
ern
al
qual
ity p
roce
dure
s of
ficer
s
Nu
mbe
r of
re
sear
ch/d
evel
opm
ent
offic
ers
Nu
mbe
r of
tr
aine
es/
inte
rns
Sta
ff co
verin
g fu
nct
ions
un
rela
ted
to e
xte
rnal
QA
The government is still the main funder of QAAs with 71% (almost the same as
three years ago at 69%). Grants still do not represent any significant portion of QAA
funding on par with last study results (about 6%). The change from the last scoping
study is the decrease in levies from HEIs as source of funding (from 54% to 30%) to
be made up by an increase of revenue from QA services delivered to HEIs. While on
the surface, this could suggest a more proactive and diversified approach to QA
service delivery by QAAs through offering value-added services, it could also be an
artifact of questionnaire design, as no specific item on QA service delivery was
probed in the previous study. Overall however, QAAs in the Arab world are still
heavily dependent on government funding.
11. Number of QAA employees:
Average number of employees per task
50 45 40 35 30 25 20 15 6.60 10
5 0
23.20
47.40
3.71
0.00
21.89
The survey reveals that the average number of management staff (with oversight
over external QA) is about 7 employees; the average number of administrative
(support) staff (supporting external QA) is about 23; the average number of external
quality procedures officers (commissioners, reviewers, etc.) is about 47 (except for
Egypt whose respondent stated a whopping 300 employees); the average number of
research/development officers is less than 4 and the average number of staff
covering functions that are unrelated to external QA is close to 22. None reported
the existence of interns or trainees. On aggregate, Arab QAAs are rather large as
they surpass the ‘benchmark’ size of similar entities elsewhere which stand at 20
(Board excluded).
As with most other public service departments, the number of staff with operations
not related directly to QA is quite significant (1/3 approximately). Moreover, there is
hardly any research staff employed which might indicate the lack of a formal
knowledge management and development strategy for QAA activities. On the
positive side, QAAs are largely staffed indicating a commitment to QA in most
countries.
12. Financial and human resources:
- In my opinion, my agency is adequately staffed to conduct external
quality assurance efficiently and effectively:
Strongly Disagree 8.33
Disagree 25.00
Neutral 8.33
Agree 41.67
Strongly Agree 16.67
0 10 20 30 40 50
Percentage
This part of the questionnaire investigates the financial and Human Resources
sustainability of QAA activities. A significant 58% of respondents think that the
agency has enough human resources to conduct its external QA activities properly
whereas 33% disagree.
- In my opinion, my agency is properly funded to fulfill its mandate of
external quality assurance:
Strongly Disagree 8.33
Disagree 41.67
Neutral 8.33
Agree 33.33
Strongly Agree 8.33
0 10 20 30 40 50
Percentage
When it comes to having sufficient funds to conducts their external QA business,
only 42% of respondents believe that they do against 50% who believe that their
agencies are not properly funded.
- In my opinion, my agency is adequately staffed to continue to grow
and develop its structure and processes to meet QA needs in its
primary domain:
Strongly Disagree 8.33
Disagree 33.33
Neutral 16.67
Agree 25.00
Strongly Agree 16.67
0 5 10 15 20 25 30 35
Percentage
The proportion of satisfied respondents with their level of staffing decreases when
projections are made for the future. As many as 46.15% indeed do not believe that
their agencies are adequately staffed to continue to grow and develop their activities
against 38.46% who do. This shows that even if QAAs perceived themselves to be
adequately staffed to fulfill their current mandate, they remain uncertain about
resources allocation for the future.
- In my opinion, my agency is properly funded to continue to grow
and develop its structure and processes to meet QA needs in its
primary domain:
Strongly Disagree 8.33
Disagree 50.00
Neutral 16.67
Agree 16.67
Strongly Agree 8.33
0 10 20 30 40 50
Percentage
The same holds even true when it comes to financial resources as 62% do not
believe their agencies to be properly funded to meet future challenges of growth and
development while 23% do. The resource issue however, both in terms of staff and
funding, has to be interpreted within the usual culture of claims for additional
resources in the public service without a thorough evaluation or forecasting of actual
resources needs either for the present or the future. Moreover, it is likely that more
resources could be made available through enhanced efficiency of QAA management
which is likely to take place through a higher degree of accountability from public
service directorates in many countries and as a result of the external reviews that
are being put in place.
Section III. Internal Quality Assurance for QAA
1. No-conflict-of-interest mechanism:
Established and very effective 41.67
Established and effective 41.67
Established and moderately effective 8.33
Partly established 8.33
Not established/ Not applicable 0.00
0.00 10.00 20.00 30.00 40.00 50.00
Percentage
Overall, most agencies (84%) have developed an effective no conflict-of-interest
mechanism in the work of their external experts. Back in 2009, the percentage was
only at 59% with agencies having a “clear policy on conflict of interest among the
reviewers” while about 40% said they were in the process of establishing it. This
arguably suggests that, three years later, most of those 40% came along with the
said policy.
2. Mechanisms that ensure the quality of any activities and material produced by subcontractors:
Established and very effective
Established and effective
Established and moderately effective
Partly established
Not established/ Not applicable
8.33
8.33
16.67
16.67
50.00
0 10 20 30 40 50
Percentage
50% indicated that mechanisms to ensure the quality of work delivered by
subcontractors have not been established and a further 17% stated that it is either
partly established or established but only moderately effective. The remaining 31%
think their QAAs managed to establish either effective or very effective such policies.
This result can be explained in light of many QAAs not using subcontractors, and
hence not feeling the need to establish related policies.
3. Appeals System:
Established and very effective 8.33
Established and effective 41.67
Established and moderately effective 25.00
Partly established 16.67
Not established/ Not applicable 8.33
0 10 20 30 40 50
Percentage
The establishment of a sound appeals system is quintessential to maximum
transparency and fairness in the quality assurance process. As a result, 75% of QAAs
managed to create such a system, though with varied degrees of effectiveness (25%
moderately effective; 42% effective and 8% very effective). The remaining 25% are
either in the process of establishing it or have not done so yet.
4. Internal feedback and reflection mechanism
Established and very effective 33.33
Established and effective 41.67
Established and moderately effective 16.67
Partly established 0.00
Not established/ Not applicable 8.33
0 10 20 30 40 50
Percentage
Internal quality assurance procedures are very important for self-assessment and to
ensure continuous improvement as part of quality management. Arab QAAs seem to
be quite aware of their importance as 92% have already established such
procedures; 75% of which are either effective (42%) or very effective (33%). Only
8% of respondents have not established such mechanisms yet.
5. Mechanisms for collection and analysis of feedback from experts and reviewed institutions:
Established and very effective 33.33
Established and effective 16.67
Established and moderately effective 16.67
Partly established 33.33
Not established/ Not applicable 0.00
0 5 10 15 20 25 30 35
Percentage
Feeding back from external stakeholders for the purpose of self-development and
continuous improvement is even more important as it provides QAAs with external
control points over their provision. 67% of QAAs have already established such
mechanisms and implemented them in various degrees of effectiveness. All the
remaining agencies are working on it.
6. Published policy for the assurance of the agency's own quality, made available on its website:
Established and very effective 16.67
Established and effective 16.67
Established and moderately effective 8.33
Partly established 33.33
Not established/ Not applicable 25.00
0 5 10 15 20 25 30 35
Percentage
This is also considered to be a further step towards better transparency and
accountability of QAAs. Nevertheless, only 42% have moved into that direction by
publishing their agency’s own quality assurance policy online with 17% of them
considering it very effective. 58% however have either partly published it or did not
at all. This question was not asked in the previous scoping study but it is expected
that most have developed and published their internal QA policy after the last
scoping study.
7. Processes and results which reflect the agency's mission and goal of quality assurance:
Established and very effective 33.33
Established and effective 33.33
Established and moderately effective 16.67
Partly established 8.33
Not established/ Not applicable 8.33
0 5 10 15 20 25 30 35
Percentage
The answers to this question reveal that approximately 67% of surveyed Arab QAAs
managed to successfully establish effective or very effective processes and results
which reflect the agency's mission and goal of quality assurance. In this regard, 17%
think they are moderately effective; 8% are still in the process (partly established)
while another 8% haven’t started yet. This reflects the high level of confidence that
QAAs have on the suitability of their operations to sustain their mission and
objectives.
8. Mandatory cyclical external review of the agency's activities at least once every five years:
Established and very effective 16.67
Established and effective 0.00
Established and moderately effective 0.00
Partly established 33.33
Not established/ Not applicable 50.00
0 10 20 30 40 50
Percentage
Adopting a mandatory cyclical external review of the agency's activities is a further
drive towards ensuring continuous improvement of the agency’s own quality through
an international rigorous review process. Partly due to cost and lower priority, only
17% have established such a process in a very effective manner. The rest have
either partly established it (33%) or did not at all in most cases (50%). It is
expected that with the drive towards external accreditation of QAAs and the ensuing
benchmarking drive, many QAAs will seek to undertake external reviews.
9. Agency change of approach to evaluation in the past three years
Yes 25%
No 75%
During the past three years (after 2008’s study), only 25% of the surveyed QAAs
made significant changes to their approach to evaluation while 75% did not indicate
any action in that regard. Again this a symptomatic of the lower priority of internal
evaluation compared to external QA.
Per
cen
tag
e
Per
cen
tag
e
An
eval
uatio
n of
a
prog
ram
me
An
eval
uatio
n of
an
inst
itutio
n
An
audi
t
An
accr
edita
tion
proc
ess
Ben
chm
arki
ng
Section IV. Framework for External Quality Assurance
1. Compulsory vs. voluntary QA
80.00
70.00
60.00
50.00
40.00
30.00
20.00
10.00
0.00
76.47 23.53
Compulsory Voluntary
The majority of participants stated that QA in higher education in their countries is
mandatory (79%), hence an increase compared to the last study results (69.2%).
This is an expected trend for QA to be compulsory throughout.
2. External quality procedures performed
90 76.92 80 70 60 50 40 30 20 10
0
84.62 53.85
76.92
15.38
Per
cen
tag
e
The respondents’ answers reveal that institutional evaluation is the most prevalent
form of external quality procedures performed by 85%, closely followed by program
evaluation with 77%. Accreditation of both programs and institutions is also
performed by 77%. Furthermore, 54% of QAAs perform audits but relatively few
(15%) do benchmarking. This last percentage has to be interpreted with caution as
those who have answered that they perform benchmarking might have
misinterpreted it to mean the regular benchmarking that is undertook as part of
other external QA procedures. However, this is benchmarking that entails
comparing a program or an institution to others as the objective of the external
review itself. Ranking of programs and institutions is an example of benchmarking.
3. Frequency of periodical external quality assurance procedure:
100 90 80 70 60 50 40 30
61.54
23.08
75.00
62.50
37.50
92.31 42.86
28.57
15.38 16.67 20 8.33 10
0
0.00
7.69
0.00
Never
Periodically
Ad-hoc/upon request
Frequency of external QA depends on the type of procedure performed:
- Programme evaluation: Performed periodically in 62% of the cases and on
an ad-hoc/request basis in 15%. The remaining 23% as transpires from the
previous question do not perform program evaluation altogether.
- Institutional evaluation: Performed periodically in 75% of the cases and
on an ad-hoc/request basis in 17%.
Per
cen
tag
e
- Audit of QA system: For those who undertake audit activities (53%), it is
performed periodically in 63% of the cases and on an ad-hoc/ request basis in
38%.
- Accreditation: Performed periodically in 92% of the cases and never on an
ad-hoc basis.
- Benchmarking: The frequency of benchmarking is equally distributed
between being performed periodically and on an ad-hoc/ request basis (29%
for both).
The results are a little bit surprising with accreditation expected to be the procedure
that is mostly performed upon request. However, it is always performed
periodically. This can be explained by the fact that HEIs that operate within an
accreditation system go systematically for reaccreditation, thereby forcing QAAs to
offer the required service on a systematic basis.
4. Periodicity of external quality procedure performed:
40
35
30
25 22.22
20
15
33.33
36.36 40.00
27.27 18.18
Yearly
Every 2 years
Every 3 years
11.11 9.09
10
5
0
0.00
0.00
9.09 Every 4 years
Every 5 years
More than 5 years
Programme evaluation
Institutional evaluation
Audit of QA system
Accreditation
Following from the previous question, respondents were asked to indicate the
periodicity of the different review procedures:
- Programme evaluation: Performed predominantly every five years or
above (67). In 22 of the cases, it is performed every 2 years and every 3
years in 11of the cases.
- Institutional evaluation: Performed every 5 years or above in 55% of the
cases. 18% of respondents said they perform institutional evaluation every 4
Per
cen
tag
e
App
rova
l/Lic
ensi
ng
De
- lic
ensi
ng/S
uspe
nsio
n
Fun
din
g/w
ithd
raw
al
of fu
ndi
ng
No
form
al
cons
eque
nce
years while an equal number of QAAs do it yearly, every 2 years and every 3
years (9% each).
- Audit of QA system: The audit is most frequently performed either yearly or
every three years with 40% each. The remaining 20% do it on a periodicity
exceeding 5 years.
- Accreditation: In 55% of the cases, the accreditation process is performed
either every 5 years or more. It is performed every 3 years in 18% of the
cases whereas 9% of the respondents indicated that it is performed yearly,
‘every 2 years’ and ‘every 4 years’.
- Benchmarking: Only one respondent (KSA) indicated that his agency
performs benchmarking every two years. The rest did not respond which
goes to confirm that those who indicated that they do benchmarking mistook
it for a regular benchmarking exercise which is not a self-standing external
QA procedure.
5. Formal consequences for HEIs of external quality procedures
80 63.64 70 60 50 40 30 20 10
0
72.73
9.09
27.27
Survey respondents indicated that the most frequent formal consequence for HEIs of
external quality procedures is the de-licensing/suspension (73%) and
approval/licensing (64%). Only 9% indicated any link between the results of external
quality procedures and the funding/withdrawal of funding while a relatively high
percentage 27% said that there were no formal consequences at all.
Per
cen
tag
e
Pro
gram
me
eva
luat
ion
Inst
itutio
nal
eva
luat
ion
Aud
it of
QA
sys
tem
Acc
redi
tatio
n
Ben
chm
arki
ng
and
/or
Ra
nkin
g
The results confirm the importance of QA as it bears serious consequences for three
quarters of institutions in licensing terms mostly. In the remaining one quarter,
there are no consequences mostly in countries where higher education is public and
where licensing and de-licensing are not really viable options.
6. Formal requirement on the HEI to address recommendations
100
90 80 70 60 50 40 30 20 10 0
100 100 100 100
0 0 0 0
100 0
Yes
No
There is a formal requirement for HEIs to address recommendations of the QA
bodies for all the external quality procedures except for Benchmarking. However, it
remains to be seen how this can be implemented in the absence of formal
consequences of external QA in some countries.
7. Importance of different QA constructs/activities in program reviews:
Facilities and resources
Student admission, retention and…
Governance & strategy
Professional and pedagogical…
Quality management system
23.08
15.38
41.67
38.46
41.67
61.54
53.85
50.00
61.54
58.33
Unimportant
Of little importance
Moderately important
Assessment of the learning… 16.67
Student experience
Academic standards &…
25.00
53.85
83.33
58.33
38.46
Important
Very Important
Teaching & Learning
23.08
69.23
0% 20% 40% 60% 80% 100% This question gauges the importance of different QA constructs/activities in program reviews.
- Teaching & Learning: Overall it is considered either very important or
important by 92% of the respondents. The remaining 8% consider it just
moderately important.
- Academic standards & employability of graduates: The same
percentage (92%) considers academic standards of graduate and
employability to be quite important.
- Student experience: 58% of the surveyed sample considers the student
experience very important; 25% considers it important and about 17% sees it
as moderately important.
- Assessment of the learning outcomes attained by graduates: This is
at the very heart of quality assurance and the most important element in the
process with more than 83% considering it as very important and the all rest
as important.
- Quality management system: It is of equal importance for external quality
procedures at programme level, with about 58% considering it very important
and 42% seeing it as important.
- Professional and pedagogical qualifications of staff: The professional
and pedagogical qualifications of staff seem to be even more important than
Educational programs 33.33 66.67
Governance 25.00 75.00
Vision, mission & objectives 15.38 76.92
the quality management itself with about 62% considering it very important
and the rest (about 38%) considering it important.
- Governance & strategy: 50% of the participants consider this item very
important while about 42% consider it important. The rest sees it as
moderately important.
- Student admission, retention and completion rates: Two third of
respondents consider it to be very important to important and another 31%
think it is moderately important.
- Facilities and resources: about 62% of the respondents consider this item
very important; 23% consider it important and only 15% consider it
moderately important.
The high degree of importance ascribed to the above constructs/indicators has to do
with their being generally listed as review indicators for program reviews.
8. Importance of different QA objectives/dimensions in institutional reviews:
Institutional integrity 16.67 75.00
Quality assurance management 16.67 83.33
Community service 16.67 75.00
Research 23.08 69.23 Unimportant
Physical resources 15.38 76.92 Of little importanceFinancial resources
Students & student support
Faculty members
15.38
30.77
16.67
69.23
69.23
83.33
Moderately important
Important
Very Important
0% 20% 40% 60% 80% 100%
The above objectives/dimensions of institutional reviews are ANQAHE’s.
Respondents were asked to rate the importance attached to each by their respective
QAA:
- Vision, mission & objectives: 77% of the respondents consider this
element very important; 15% consider it important and only less than 8%
consider it to be moderately important.
- Governance: Clearly instrumental for external quality procedures at
institutional level, 75% of the participants consider governance as very
important while the remaining 25% considers it important.
- Educational programs: about 67% of the surveyed sample considers
educational programs very important while the remaining 33% considers it
important.
- Faculty members: This is one of the most important elements in the
external quality assurance process with all respondents considering either
very important or important.
- Students & student support: this also turns out to be a very relevant part
to the external quality procedures at institutional level with about 70%
considering it very important and 30% seeing it as important.
- Financial resources: 69% of the participants consider this element very
important while about 15% consider it important. Another 15% sees it as
moderately important.
- Physical resources: 92% of respondents considered physical resources to
be either very important or important. The remaining 8% still consider it of
moderate importance.
- Research: 92% of respondents considered research to be either very
important or important. The remaining 8% still consider it of moderate
importance.
- Community service: 92% of respondents considered community service to
be either very important or important. The remaining 8% still consider it of
moderate importance.
- Quality assurance management: As expected, Quality assurance
management is essential to the external quality procedures at institutional
level with around 83% of the participants considering it very important and
the rest considering it important.
Per
cen
tag
e
Loca
l reg
ulat
ions
Na
tiona
l fra
mew
ork
of q
ualif
icat
ions
Sta
ndar
ds a
nd
gui
del
ines
def
ined
by
pro
fess
iona
l gr
oups
/org
ani
zatio
ns e
ither
loca
l or
inte
rnat
iona
l
Inst
itutio
nal
, pro
gra
m, a
nd s
ubj
ect
be
nchm
ark
s
Bes
t or
goo
d pr
actic
es
Goa
ls a
nd
obje
ctiv
es s
et b
y th
e in
stitu
tion
bein
g e
valu
ated
- Institutional integrity: Just like community service, 75% of respondents
rated institutional integrity at the very important level, and an additional 17%
consider it important. Only 8% gave it a rating of moderate importance.
The above ratings again seem to indicate that the ANQAHE institutional dimensions
are widely used for institutional reviews throughout the Arab region. This can be
considered as a validation of the dimensions as a basis for institutional reviews.
9. Which model/frame of reference do you use for your external quality procedures?
100 90 80 70 60 50 40 30 20 10 0
84.62
53.85
100.00
61.54
100.00 84.62
Respondents were asked to indicate what the basis of their EQA assessments is:
The “Standards and guidelines defined by professional
groups/organizations either local or international” and the “Best or good
practices” are referred to in 100% of the cases while setting up the external quality
assurance procedures. The “Local regulations” and the “Goals and objectives
set by the institution being evaluated” are both considered in 85% of the
cases. “Institutional, program, and subject benchmarks” and the “National
framework of qualifications” were mentioned by 62% and 54% of respondents
respectively.
Two countries indicated that they have developed and use their own proprietary
evaluation models; but these are expected to be based on the references indicated
in the questionnaire. For instance, the Sudanese specified that “National Standards
for Quality Assurance in Higher Education established by EVAC” were used as the
frame of reference for their QA assessments while the Egyptian mentioned
Standards set by NAQAAE.
One can infer from the above results that a standard-based approach is used in most
countries with both standards (100%) and a fitness-for-purpose approach (85%)
mentioned as reference models. A standards-based approach is a combination of
standards, wherein assessments are based on golden standards, and a fitness-for-
purpose approach; wherein assessments are based on the institution’s own mission
and objectives. It is worth mentioning that national qualification frameworks are
climbing up the ladder as a basis for evaluation. This is confirmed in answers to
question 11 below as the development of national qualification frameworks is
mentioned as one of the key developments in the higher education scene of 84% of
respondents’ countries.
Per
cen
tag
e
10. Approval of new subjects, programmes and institutions
90 80 66.67 70 60 50 41.67
83.33 75.00
41.67
The institution itself (self- regulation)
33.33 40 25.00 25.0025.00 30 20 10
0
Ministry
Quality Assurance Agency
- Subjects: The institution itself is responsible for the approval of new subjects
in about 67% of the cases against 42% for the Ministry. The Quality
Assurance Agency is responsible for the approval of new subjects in 25% of
the cases.
- Programmes: The Ministry takes over when it comes to the approval of
programmes with 75%. In 42% of the cases, the institution itself can approve
programmes while 33% of them ascribe such approval to QAAs.
- Institutions: The Ministry leads again in 83% of the cases as the body
responsible for new institutions’ approval. QAA do institutional approval in
25% of the cases, and institutions likewise self-approve in 25% of the cases.
In Sudan the “Standing Committees of the National Council for Higher Education”
is responsible for institutional approval whereas it is the “Supreme Council of
Universities and its Sectorial Committees” who approve programs.
Few remarks can be made here. First, the Ministry in charge of higher education
plays the key role in institutional and programme approvals. It further interferes
with subject approval, which is generally an internal university matter. This
derives from a top-down approach to managing higher education in most Arab
countries. Second, responses show that QAAs also play a significant role in all
three types of approvals. This has to be interpreted with caution as QAAs are
not approving authorities but deliver recommendations for doing so.
Per
cen
tag
e
Est
abl
ishm
ent o
f Na
tiona
l Q
ualif
icat
ions
Fra
mew
ork
Ne
w h
ighe
r ed
uca
tion
legi
slat
ion
(incl
udi
ng
abo
ut Q
A)
Int r
odu
ctio
n of
diff
eren
t met
hod
olo
gie
s fo
r e
xter
nal
qu
ality
pro
cedu
res
Re
visi
on o
f ext
erna
l qua
lity
proc
edur
es
Re
org
anis
atio
n of
qua
lity
ass
uran
ce
agen
cy
Cre
atio
n of
a Q
A a
gen
cy if
cu
rren
tly
inex
iste
nt
11. Expected developments in the QA sector in the next three years (i.e. by December 2014):
100
90
80
70
60
50
40
30
20
10
0
83.33
91.67
58.33 58.33
50.00
16.67
A “New higher education legislation (including about QA)” is the most expected
development in the QA sector as reported by 92% of the respondents. The
“Establishment of National Qualifications Framework” is the second most expected
development being reported in 83% of the cases. The “Introduction of different
methodologies for external quality procedures” and the “Revision of external quality
procedures” came in third with 58%. Moreover, 50% of the respondents expect to
see “Reorganization of their quality assurance agency” in the next three years. The
“Creation of a QA agency” in their countries by December 2014 is expected by 17%
of respondents
In the UAE, “Vocational Education & training” may be devolved to a separate
agency.
The results show clearly that the newly established qualification frameworks in most
countries of the region will come to influence the work of QAAs. ANQAHE has been
working on an overarching Arab qualification framework to mirror its effect on the
practice of QA in higher education.
Per
cen
tag
e
Per
cen
tag
e
Section V. Standards & Methodology for External Quality Assurance
1. Formal standards, guidelines, indicators, or criteria to conduct external QA
88.24
90
80
70
60
50
40
30
20
10
0
11.76
Yes No
The answers to this question show that there are still few Arab countries (12%) that
do not use formal standards, guidelines, indicators, or criteria to conduct external
QA. This is expected to exist among countries that do not have an established QAA
yet.
2. Which standards?
100 80 60 40 20
0
100.00
QA body published
standards and criteria (ex.
Review handbook)
15.38
Other government
entity standards and criteria (ex.
Licensing regulations)
Per
cen
tag
e
Join
t/col
labo
rativ
e pr
ogr
ams
with
re
gio
nal
/inte
rnat
iona
l in
stitu
tions
Str
ateg
ic in
stitu
tiona
l par
tner
ship
s w
ith fo
reig
n in
stitu
tions
Ho
ste
d pr
og
ram
s (i.
e.
inte
rnat
iona
l pro
gram
s d
eliv
ered
w
ithin
loca
l HE
Is)
Bra
nch
cam
pus
ope
ratio
ns
of
inte
rna
tion
al H
EIs
Dis
tanc
e an
d hy
brid
lear
nin
g pr
ogra
ms
No
we
do
not
use
QA
pro
cedu
res
for
any
of t
he a
bove
For the 87% who acknowledged the use of formal standards, guidelines, indicators,
or criteria to conduct external QA, all stated that they use “QA published standards
and criteria (ex. Review handbook)” and 15% use in addition “other government
entity standards and criteria (ex. Licensing regulations)”. This reflects that QAA
work has reached a high degree of formalism.
3. Remit of QA procedures:
57.14 60
50 42.86
40
30
35.71
42.86 35.71
28.57
20
10
0
As QAAs grow and mature, they develop external QA procedures for a variety of
institutional set-ups and programs. The question above tries to identify the
existence of a differentiated QA assessment framework.
QA procedures are used for “Hosted programs (i.e. international programs delivered
within local HEIs)” in 57% of the cases. A further 43% have QA procedures for
“Joint/collaborative programs with regional/international institutions” and “Branch
campus operations of international HEIs.” The above two results mirror the
proliferation of such programs throughout the Arab region. About 36% of
respondents report that they have/use QA procedures for “Distance and hybrid
Per
cen
tag
e
Hig
her
edu
catio
n in
stitu
tions
(w
ithin
you
r p
rima
ry d
om
ain
)
Min
istr
y (o
r o
ther
Hig
her
educ
atio
n au
thor
ity)
Indu
stry
and
labo
ur
mar
ket
repr
ese
ntat
ives
(uni
on,
empl
oyer
s, e
tc.)
Pro
fess
iona
l org
anis
atio
ns
(gui
lds,
trad
e so
ciet
ies,
etc
.)
Nat
iona
l ass
ocia
tion
of H
EIs
(s
uch
as R
ecto
rs’ c
onfe
renc
e)
Oth
er
Qu
ality
ass
uran
ce
entit
ies
in th
e co
untr
y
Hi re
d co
nsu
ltant
s
Inte
rnat
iona
l ass
oci
atio
ns o
f H
EIs
(su
ch a
s A
NQ
AH
E)
Stu
dent
re
pre
sent
ativ
es
learning programs” while 29% said they do not have any differentiated QA
procedures.
Overall, results show that Arab QAAs have gone beyond the basic QA of institutions
and academic programs to consider additional setups through differentiated QA
procedures. This testifies to the maturity of QA in 70% of the countries
investigated.
4. Stakeholders role in the specification of processes and criteria for the external quality assurance of higher education institutions
80 73.33 73.33
70
60
50
40
30
20
10
0
33.33
26.67 26.67
13.33
66.67
13.33
0.00
Higher education institutions and the Ministry (or other Higher education authority)
are the most involved in the specification of processes and criteria for the external
quality assurance of HEIs; both being reported by 73% of the cases. This is a
significant change compared with last study’s results wherein HEIs “participated in
the development of the standards/criteria” in about 5% of the cases only. QAAs are
clearly having a more proactive approach to dealing with HEIs. It might also
indicate a shift of philosophy from sanctioning to developing.
Per
cen
tag
e
Hired consultants are the second most consulted source with 67% of responses,
which is on par with the previous scoping study wherein “experts” were the main
contributor (76.9%). Industry and labour market representatives (union, employers,
etc.) are also consulted 33% of the time. About 27% of respondents say they also
refer to professional organizations (guilds, trade societies, etc.) and national
associations of HEIs (such as Rectors’ conference). The involvement of professional
organizations and syndicates in the specification of processes and criteria for
external QA assessment has remained at about the same level from three years ago
where it stood at 30%.
Only 13% indicated that international associations of HEIs (such as ANQAHE) and
other Quality assurance entities in the country were involved in the specification of
processes and criteria for the external quality assurance of HEI, whereas students
had no input whatsoever.
5. Final decision in the specification of processes and criteria for the external quality assurance of higher education institutions
60 52.94
50
40 29.41
30
20
17.65
10
0 Your QA entity Ministry (or other
Higher Education Authority)
Other
Beyond having an input into the specification of processes and criteria for external
QA, this question asks who makes the final determination.
The QA entity is said to be the ultimate decision maker regarding the specification of
processes and criteria for the external quality assurance of higher education
institutions in its primary domain in 53% of the cases against 29% for the Ministry
(or other Higher Education Authority). Few respondents suggested other bodies such
as “QAAET Board” and “Prime Minister’s Cabinet” (Bahrain), the institution itself
(Lebanon) and the Board of the Authority of HEI Accreditation (Jordan).
6. Publication of relevant processes, criteria and procedures prior to execution of external quality procedures
29%
71%
Yes
Non
71% of the respondents say their QAAs systematically publish relevant processes,
criteria and procedures prior to execution of external quality procedures against 29%
who do not.
Per
cen
tag
e
A s
elf-
ass
essm
ent
repo
rt w
ith p
rope
r ev
iden
ce b
y th
e…
An
exte
rnal
as
sess
men
t by
a gr
oup
of e
xpe
rts…
A s
ite v
isit
A p
ublis
hed
re
port
A f
ollo
w-u
p pr
oced
ure
un
dert
ake
n by
…
7. Processes used/required within its external quality procedures
100 90 80 70 60 50 40 30 20 10 0
73.33
86.67
100.00
53.33
80.00
All QA entities use/require a site visit within their external quality procedures. This
was only at 76.9% in the 2008 study. The use of experts with or without a site visit
was also reported in 87% of the cases. 80% of respondents also reported a follow-
up procedure undertaken by the institution that is subject of the external quality
procedure in light of recommendations contained in the report (ex. Improvement
plan).”
The submission of a self-assessment report with proper evidence by the institution
that is subject of the external quality procedure was only mentioned in 73% of the
cases, although it is generally thought of as a universal requirement of external QA
procedures. Nonetheless, there has been a significant increase over last study
results wherein only 38.5% reported requiring a self-evaluation report by HEIs.
Finally, respondents were equally distributed over the publication of a QA report as a
result of external quality procedures with 53% stating they do so and 47% not.
Qatar requires a feasibility study for the approval of new HEIs or programmes in lieu
of a self-assessment report. Egypt undertakes follow up visits during the validity
period of accreditation.
Overall, the external QA format does correspond to conventional practice in the
international arena with self-assessments, QA experts and site visits forming the
building blocks of external QA proceedings.
Per
cen
tag
e
8. Appointment of members of the external review/audit panels used for external quality procedures
90 81.25
80 70 60 50 40 30 20 10
0 Your QA
entity
0
The institution that is the subject of
the external quality review
12.5 Ministry or
other regulatory authority
0 Funding
entity
6.25
other
The answers to this question reveal a major change of results from 3 years ago. In
2008, HEIs had the primary input for the appointment of the review panel. This has
now shifted to QAAs who appoint members of the external review/audit panels in
81% of the cases. The Ministry has a say only in 13% of the cases. HEIs have been
completely left out of the nomination process. In Jordan, the “Board of the Authority
of HEI Accreditation” appoints panel members.
The shift from the primacy of HEIs in appointing reviewers to that of the QAA has
probably to do with following international practice in the matter as QAAs become
increasingly subjected to external scrutiny as illustrated before.
Per
cen
tag
e
Sub
ject
mat
ter
exp
erts
Hig
her
edu
catio
n an
d Q
A
expe
rts
Sta
ff m
embe
rs o
f the
QA
en
tity
Re
pres
enta
tives
of
prof
essi
ona
l org
anis
atio
ns
Stu
dent
s
Gra
duat
es/a
lum
ni
Fie
ld e
xper
ts/p
ract
ition
ers
Em
ploy
ers
9. Membership of an external review panel
100 90 80 70 60 50 40 30 20 10
0
60.00
100.00 53.33
33.33
13.33
6.67
46.67
0.00
Higher education and QA experts are always sought as members of review panels.
This percentage was only at 61% in the latest study’s results in 2008. Subject matter
experts are still present with 60% although they lost ground to HE and QA experts
compared to 2008 (77% in 2008). This can be explained by the fact that over the
years, countries have developed expertise in QA and HE affairs which they did not
have as readily available three years ago.
Staff members of QA entities continue to be part of review panels which presents a
slight increase compared with the last study (43%). Field experts/practitioners and
representatives of professional organizations are also present in 47% and 33% of
the cases respectively (on par with last study).
As expected, students and graduates/alumni are least represented in review panels
with only 13% and 7%, respectively, mentioning that they can be considered for
membership to these panels. These results remain unchanged from three years ago.
10. Formal terms of reference for external review panels for different external quality procedures (i.e. membership requirements, role, reporting mechanisms, etc.)?
No 12%
Yes 88%
Most QA entities (87%) claim to have formal terms of reference for external review
panels for different external quality procedures (i.e. membership requirements, role,
reporting mechanisms, etc.).
11. Familiarization with external QA procedures:
0%
19%
12%
44%
Compulsory training/briefing prior or after nomination to a panel
Compulsory training/briefing only for members who have not served before in the same external quality procedure Compulsory training/briefing for panel chairs only
6%
0% 19%
Optional training/briefing Self-learning through published material
E-learning
The reliability and validity of external QA depends largely on the competency of
panel members and their understanding of the assessment framework in use. This
question asks how panel members are familiarized with the assessment framework
especially when they come from a different country or have not been involved with
external QA procedures with the concerned QAA. This question goes beyond the
scoping study of 2008 which sought to elicit whether panel members received any
kind of training at all (a yes/no question).
Answers reveal that 44% of QA entities require training/briefing of panel members
prior or after their nomination to the panel while 19% request it only for members
who have not served before in the same external quality procedure. Self-learning
through published material is also prominent with 19% as well. Compulsory
training/briefing for panel chairs only is done in 6% of the cases. Finally, 12% had
no response which probably means that they do not require panel members to
undergo any training. This percentage has not changed much from the 2009 study.
Per
cen
tag
e
No
min
ate
d by
inde
pen
den
t pa
rtie
s
Nom
inat
ed b
y Q
AA
sta
ff
No
min
ate
d by
Min
istr
y/H
E
auth
orit
y
Re
trie
ved
fro
m Q
AA
m
ain
tain
ed d
atab
ase
Re
t rie
ved
fro
m o
the
r re
vie
wer
da
taba
se (
exp.
A
NQ
AH
E)
Thr
ough
ca
ll fo
r re
vie
wer
s
12. Selection of reviewers
80
70 60 50 40 30
20 13.33
10
0
80.00
26.67
46.67
6.67
20.00
The reviewers are mostly nominated by QAA staff (80%), a 18.5% increase from
last study results. In about 47% of the cases, the reviewers are retrieved from QAA
maintained databases. Reviewers may also be nominated by Ministry/HE authority
(27% against just 7.7% in 2008); chosen through a call for reviewers (20% against
only 7.7% in 2008). Independent parties could also nominate reviewers in 13% of
the cases and a final 7% resort to published databases like ANQAHE’s.
Section VI. Site Visit
This section delves deeper into key aspects of the site visit.
1. Site visit requirement for your external QA procedures?
88%
0%
12%
No
Yes for some
Yes for all
The site visit is part of external QA procedures in every country in the region (that has
responded) with 88% having it for every QA procedure and 12% for some.
2. Who conducts the visit?
33%
7%
60%
The QA entity's staff alone
The expert review panel with a member of the QA entity
The expert review panel alone
The site visit is most often conducted by the expert review panel with a member of
the QA entity (60%). That is almost the same number of respondents (61.5%) as in
the 2008 scoping study. In 33% of cases, the visit is performed by the expert review
panel alone which is double what has been indicated in the last study. Finally, 7% of
QAAs perform the site visit themselves.
3. Role of members of the QA entity in the site visits
27%
20%
0% 0%
53%
Facilitator/Coordinator
Full member of the panel but not Chair
Full member of the panel including as Chair
Passive observer/monitor
Secretary/report writer
Whenever a member of the QA entity participates to the site visits, he/she is:
- Facilitator/Coordinator in 53% of the cases.
- Full member of the panel including as Chair 27% of the times.
- Full member of the panel but not Chair in 20% of the cases.
- Never a “passive observer/monitor”.
- Never a “secretary/report writer”.
Being a facilitator/coordinator in 53% of the cases is on par with the last study
(about 60%). However, only 15% used to participate as full members of the panel
(whether as Chair or not) against 47% in the current study. This has to do with the
increasing experience accumulated by QA staff members and the criticality of quality
assuring and normalizing the process.
Per
cen
tag
e n
um
ber
of
day
s
Me
etin
gs w
ith th
e H
EI
Boa
rd a
nd to
p-
ma
nag
emen
t
Me
etin
gs w
ith o
the
r ex
tern
al s
take
hold
ers
(e
x. e
mpl
oyer
s, a
lum
ni)
Mee
tings
with
facu
lty
and
staf
f of
the
inst
itutio
n
Mee
tings
with
st
uden
ts/g
radu
ate
s
Exa
min
atio
n of
do
cum
enta
ry e
vid
ence
Tou
r of
the
faci
litie
s
Cla
ssro
om
ob
serv
atio
ns
Exi
t mee
ting
with
the
ma
nag
emen
t of t
he
inst
itutio
n fo
r a…
4. Duration of the site visit (days only on-site and not briefing or back-office days)
Average duration for site visits
3.5
3.0
2.5
2.0
1.5
1.0
0.5
0.0
3.50 3.41 3.50 2.60
For higher education
institutional reviews
For higher education
programme reviews
For unit (college or
school) reviews
For pre- university
school reviews
Whatever the object of the review, the duration of the visit is always 3 to 4 days,
except for pre-university school reviews which require from 2 to 3 days. The result is
in-line with the findings from the 2008 study where respondents indicated that the
site visit lasts from 3 to 4 days. The last study did not detail results in terms of the
type of visit though although this has been found to be inconsequential as the
duration is the same across.
5. Program of the site visit
100 90 80 70 60 50 40 30 20 10
0
100.00
62.50
100.00 93.75 93.75 100.00
62.50
87.50
Most of the proposed activities are consistently performed during the visit. The
following three are reported to be performed by all the QAAs: “Meetings with the
HEI Board and top-management”, “Meetings with faculty and staff of the institution”
and “Tour of the facilities”. Moreover the following are performed in 94% of the
cases: “Meetings with students/graduates” and “Examination of documentary
evidence”.
The exit meeting with the management of the institution for a preliminary feedback
is also performed by 88% of the participating Arab QAAs. To a lesser extent, the
following two are performed in 63% of the cases: “Classroom observations” and
“Meetings with other external stakeholders (ex. employers, alumni)”.
All the above-stated results are in concordance with the findings of the 2008 scoping
study and are also in line with general practice in site visits.
Per
cen
tag
e
Boa
rd/c
oun
cil m
em
ber
s
Aca
dem
ic a
dmin
istr
ator
s (P
resi
den
t, D
eans
, De
part
men
t…
Adm
inis
tra
tive
sta
ff
Tea
chin
g/re
sear
ch s
taff
Adv
isor
y co
unci
ls/c
onsu
ltan
ts
Em
ploy
ers
Stu
dent
s
Gra
duat
es/a
lum
ni
Stu
dent
par
ents
/gua
rdia
ns
Co
mm
uni
ty re
pres
enta
tive
s
Inte
rvie
ws
are
not
use
d
6. Site visit interviews
100
90
80
70
60
50
40
30
20
10
0
68.75
100.00 87.50
93.75
18.75
56.25
93.75
62.50
12.50
43.75
0.00
Academic administrators (President, Deans, Department Heads, etc.) are
systematically interviewed during site visits. Teaching/research staff and students
are also solicited in 94% of the cases. Another very sought after stakeholder for
interviewing is administrative staff (88%). Board/Council members and
Graduates/Alumni are interviewed in 69% and 63% of the cases respectively.
Employers are part of the interview schedule in 56% of the cases.
A less common practice is meeting and interviewing Community representatives
(44%) while Advisory Councils/Consultants and Student Parents/Guardians are
interviewed in 19% and 13% of the time respectively.
The above results confirm the findings of the last scoping study three years ago.
Per
cen
tag
e
No
form
al g
roup
is u
sed
Fac
ulty
Aca
dem
ic a
dmin
istr
ator
s (e
x. P
resi
den
t, D
ean
s)
Adm
inis
tra
tive
sta
ff
Stu
dent
s
Alu
mni
/gra
du
ate
s
Ext
ern
al c
onsu
ltant
s
7. Membership of self-evaluation groups in institutions being evaluated (i.e. the group responsible to draft the self-evaluation report on behalf of their institution)?
90 80 70 60 50 40 30 16.67 20 10
0
83.33 66.67
50.00
41.67
0.00 0.00
Faculty are the most frequently involved in drafting the self-evaluation report on
behalf of their institution (83%) while academic administrators (ex. President,
Deans) come in second (67%). Administrative staff are also typically members of
self-evaluation groups. Surprisingly, students also participate in 42% of the cases
although it is not clear what the extent of their participation is. Finally, 17%
answered that no formal groups are used within institutions for self-evaluation report
writing.
Saudi QA authority requires institutions to form qualified teams with mixed
memberships (faculty and others). No other country seems to regulate the
formation of such groups although the type of participation in self-assessment teams
is assessed as part of the institutional external QA assessment itself.
Per
cen
tag
e
Ch
oic
e o f
rev
iew
met
hod
if no
t ava
ilabl
e
Pre
para
tion
of th
e gu
ide
lines
for
the
self-
ev
alua
tion
Sco
pin
g o
f the
ext
ern
al
qual
ity p
roce
dur
e
Co
ntac
t with
the
inst
itutio
n
Pla
nnin
g o
f the
site
vis
it
Pre
par
atio
n o
f the
gu
idel
ines
of t
he
site
-vis
it
Writ
ing
of t
he
rep
ort
8. Roles during the external QA procedures
100 90 80 70 60
92.31 92.31
84.62
100.00
76.92
53.85
84.62 84.62
50 40 30 20 7.69 7.69 10
0
38.46
15.38
15.38
30.77
The QA entity
The external expert panel
- “Choice of review method if not available”: This task is predominantly
performed by the QA entity itself (92%). The external expert panel chooses
the review method in only about 8% of the cases.
- “Preparation of the guidelines for the self-evaluation”: This is also the
exclusive prerogative of the QA entity.
- “Scoping of the external quality procedure”: This is largely done by the
QA entity (85% of the time) while the external expert panel plays the same
role in 38% of the cases.
- “Contact with the institution”: this task is always performed by the QA
entity (100%) and only rarely by the external expert panel (15%).
- “Planning of the site visit”: One could say that this role is somewhat
played by both the QA entity and the external expert panel (77% and 54%
respectively).
- “Preparation of the guidelines of the site-visit”: The QA entity bears
this responsibility most of the time (85%), and rarely so the external expert
panel (15%).
- “Writing of the report”: 31% of respondents have reported that the
external expert panel writes the report which is in-line with the results from
the 2008 study (38.5%). The QA entity on the other hand writes the report
most of the time (85%), a staggering increase from the last study results
(23.1%). This could also signify that Arab QAAs have built capacity over the
years to take on more responsibilities in external QA procedures. Having said
that, the write up of external reports is generally a collaborative process
between QA entities and external panels and both are involved at all times.
Per
cen
tag
e
Section VII. Outcome of Quality Assurance Procedures
1. Final decision on the outcome of the external quality procedures (i.e. the decision is official and communicated to the HEI)
0%
6% 29%
The Management of the QA body
The Board of the QA body
59%
The external expert panel
6% Ministry or higher
education authority
The institution that is the subject of the external quality procedure
The Board of the QA body is the major decision maker on the outcome of the
external quality procedure (59%). Otherwise, it is the prerogative of the Ministry or
higher education authority (29%). Only in few cases does the Management of the
QA body or the external expert panel make and issue the final decision (6% each).
2. Nature of the external QA procedure outcome
60 50.00
50
40
30
20
10
0
28.57
50.00
42.86
14.29
57.14
Summative Summative Summative Formative Formative Formative judgment
for program reviews
(pass/fail)
judgment for unit reviews
judgment for
institutional reviews
judgment for program
reviews
judgment for unit reviews
judgment for
institutional reviews
Most often the external QA procedure outcome is a formative judgment for
institutional reviews (57%). In 50% of the cases, the outcome is a summative
Per
cen
tag
e
judgment for either program or institutional reviews (pass/fail). To a lesser extent, it
is a formative judgment for program reviews (43%) or a summative judgment for
unit reviews (29%). Finally, units also receive a formative judgment in 14% of the
cases.
The results seem to indicate a balanced mix of both formative and summative
assessments at all three levels of program, unit, and institution.
3. Input, consultation, or information about the first draft of the report
80 70 60.00 60 50 40 30 20 10 0
73.33 66.67
26.67
0.00
The HEI under review
The Review panel
The QA entity Ministry (or other Higher Education
authority)
Other stakeholders involved in the review
Once a first draft of the report is generated, the Review Panel is usually the first
party to have an input into it or to be consulted and informed about its content
(73%). The QA entity and the HEI under review are also regularly consulted (67%
and 60% respectively). To a lesser extent, the Ministry (or other Higher Education
authority) is also consulted (27%) but no other stakeholder is involved in reviewing
the report which is understandable given the confidentiality of the proceedings.
HEIs are not always consulted because they can do so through the appeals process
and to avoid undue litigation by the HEI while the report is not finalized yet.
4. Publication of the final outcome (assessment/evaluation report)
No, only sent to the HEI
18%
17%
12% 53%
Yes, full version published on the QA entity Website
Yes, but only summary version is published
Other, please specify
52% of QAAs do not publish the final outcome (assessment/evaluation report) but
only send it to the evaluated HEI. 18% publish the full version on the QA entity
website and another 18% publish only a summary version. Finally, 12% of the
respondents indicated that only the summative judgment is published.
Arab QAAs seem to have kept the same approach to review outcome dissemination
since the last scoping study. If we consider that 60% do not publish the report,
either in full or as a summary, then one can question whether QA entities (or their
mandatories) are committed to transparency and accountability. This has to be
interpreted however in light of the dominance of public higher education and the
reluctance of public authorities to put to scrutiny their public higher education
policies, especially if negative reports were published.
Per
cen
tgae
Bac
kgro
und
abo
ut th
e in
stitu
tion/
pro
gram
me
Bac
kgro
und
abo
ut th
e in
tern
al Q
A s
yste
m in
pla
ce
at th
e H
EI
Ana
lysi
s o
f the
evi
den
ce in
re
latio
n to
sta
ndar
ds a
nd
indi
cato
rs u
sed
for
the
revi
ew
Co
nclu
sion
s &
re
com
men
datio
ns
Judg
men
t & c
onse
que
nce
s
5. Content of the published report
90 80 70 60 50.00
50 42.86
40 30 20 10
0
78.57
85.71 71.43
The report is structured in a conventional way and all components that were
suggested were indeed checked as part of a QA assessment report by a large
number of respondents. Analysis of the evidence (79%), conclusions and
recommendations (86%), and judgment and consequences (71%) were the parts
that were most common across. It was rather surprising though that the internal QA
situation of the HEI under review did not appear in more than 50% of the
responses. Probably, this is due to its being integrated with the core assessment
itself while going through the indicators and related evidence. Therefore, no distinct
section on internal QA was part of the final report.
Per
cen
tag
e
6. Who publishes the report?
24%
The QA entity
76% Ministry (or higher education authority in the country)
The report is predominantly published by the QA entity (76%). Otherwise, it is
published by the Ministry (or higher education authority in the country).
7. Follow up for the implementation of the report's recommendations
80 70 60 50 40 30 20 6.67 10
0
80.00 26.67
33.33
No follow up The QA entity Ministry or The HEI that is done is responsible
for the follow up
Higher Education Authority
is the subject of the
external quality
procedure
The QA entity is responsible for the follow up most of the time (80%) and to a lesser
extent the HEI that is the subject of the external quality procedure (33%). It is the
responsibility of the Ministry or Higher Education Authority in 27% of the cases and
no follow up is done in one case. This is probably associated with an emergent QA
system.
Per
cen
tag
e
Per
iodi
cal r
epo
rtin
g fr
om
HE
I to
show
pro
gres
s in
ac
hiev
ing
rep
ort
reco
mm
enda
tions
Sub
mis
sion
of
actio
n/im
pro
vem
ent
pla
n
Fol
low
-up
site
vis
it (s
) to
en
sure
impl
eme
ntat
ion
of
reco
mm
enda
tion
s an
d im
prov
emen
t pla
ns
Re
pea
t of
exte
rnal
qua
lity
proc
edur
e
Moreover in Qatar, a dedicated ‘follow up committee’, independent of the QAA or
higher education authority, is in charge of this.
8. Nature of follow up
100 90 80 70 60 50 40 30 20 10
0
64.29
78.57
100.00 50.00
- Follow-up site visit (s) to ensure implementation of
recommendations and improvement plans is the most common follow
up procedure performed by all the surveyed QA entities (that are responsible
for the follow up).
- Submission of action/improvement plan: About 80% of the surveyed
QAAs require HEIs to submit an action/improvement plan following the
review. That is in line with the findings of the last scoping study where about
90% said they required follow up reports from assessed HEIs.
- Periodical reporting from HEI to show progress in achieving report
recommendations: Other than the action/improvement plan, two thirds of
QAAs require a periodical report showing progress in fulfilling QAA’s report
recommendations.
- Repeat of external quality procedure: 50% of participating QA entities
include a repeat of the external quality procedure as part of their follow up.
9. Most problematic areas in the internal quality assurance systems of HEIs
Program and institutional improvement plans
Governance and strategic plans
Employability of graduates
Periodic self-evaluation
Program and curriculum benchmarking
Program and course Intended Learning…
Teaching and learning strategy and methods
Assessment strategy
Practical training
Connection between teaching and research
Facilities and resources
Research strategy
Partnership with other HEIs
QA management systems and structures
Academic standards of graduates
Operational/financial planning
Community engagement strategy
Student experience
Staff qualification
Internationalization/international partnerships
Student retention and cohort analysis
Admission standards
7.14
7.14
7.14
7.14
14.29
14.29
14.29
21.43
21.43
21.43
21.43
28.57
28.57
28.57
35.71
35.71
42.86
42.86
42.86
50.00
50.00
57.14
0 10 20 30 40 50 60
Percentage
It is expected that QAAs will increasingly orient their activities towards areas of
weaknesses that they perceive among their client constituencies. This question asks
respondents to indicate their perception of such areas of HEI internal QA systems.
The most problematic areas in the internal quality assurance systems of HEIs within
surveyed QA entities' remit are as follows:
1) “Program and institutional improvement plans”: 57.14% of
participating Arab QAAs perceive this area of HEIs internal QA systems to be
the biggest concern.
2) “Governance and strategic plans” and “Employability of graduates”:
both areas were mentioned as problematic by half of the respondents.
3) “Periodic self-evaluation”, “Program and curriculum benchmarking”
and “Program and course Intended Learning Outcomes (ILOs)”: these
three areas were also consistently considered as very problematic (43%).
4) “Teaching and learning strategy and methods” and “Assessment
strategy”: both were reported as very problematic by 36% of the surveyed
sample.
5) “Practical training”, “Connection between teaching and research”
and “Facilities and resources”: are the fifth most reported problematic
areas of internal QA systems of HEIs (29%).
The above findings suggest that QA entities are still concerned with basic issues of
internal QA such as HEI quality management, university governance and strategic
planning, and teaching and learning. Community engagement, student experience,
internationalization, and staff welfare have not yet made it as priorities.
10. Development of the quality assurance situation
0%
24%
41%
35%
Improved significantly
Continued to improve steadily
Did not change significantly
Regressed
The outlook of QA in the Arab countries is rather positive with three quarters of
respondents indicating that it has improved significantly or continues to improve
steadily. The remaining quarter, has answered that it has remained the same over
the last year. Incidentally, no respondent thought that it has regressed although two
respondents indicated so during informal interviews. This sheds some doubt about
the validity of responses. It is thought that some respondents chose not to risk
being identified as stating that the QA situation in their country has regressed and
then be held responsible for such a state of affairs as being in charge of their
country’s QAA.
11. Outlook of the higher education/quality assurance situation
0%
Yes
No
100%
Continuing from answers to the previous question, all respondents are optimistic
about developments in the higher education/quality assurance situation in their
countries. It has to be said that higher education QA has brought a ray of hope to a
situation that looked mired into insurmountable problems in the not very distant
past.
Per
form
ance
Lack
of A
uto
nom
y o
f the
QA
ag
ency
Lim
ited
reso
urce
s/fu
ndi
ng
of Q
A
agen
cy/a
ctiv
itie
s
Lack
of Q
A s
tand
ard
s an
d gu
idel
ines
Lack
of Q
A c
ultu
re a
nd/o
r ex
perie
nce
in H
EIs
Ne
gat
ive
stan
d of
hig
her
educ
atio
n st
ake
hold
ers
(ex
. M
inis
try,
teac
hers
' un
ion,
etc
.)
Abs
ence
of s
upp
ortin
g re
gul
atio
n
12. Obstacles to the development of quality assurance of higher education
70.00
60.00
50.00
40.00
30.00
20.00
10.00
46.67
53.33
20.00
66.67
6.67
40.00
0.00
After a number of years in activity for many Arab QA entities, it is informative to
seek their perceptions of the obstacles they face in the development of quality
assurance of higher education in their countries.
- “Lack of QA culture and/or experience in HEIs”: Typically, two thirds of
QA entities ascribe the biggest obstacle to the lack of a quality culture and
experience within HEIs. This answer can be understood in light of the recent
creation of most QA entities which came into a scene of previously
established HEIs, and they have been playing catch up game since, to bring
HEIs at par with QA standards. This has not always been easy.
- “Limited resources/funding of QA agency/activities”: Slightly more
than half think limited financial resources pose a serious challenge (53%).
- “Lack of Autonomy of the QA agency”: It can be inferred from this
answer that roughly half (47%) of QA entities lack the necessary autonomy to
fulfill their mandates.
- “Absence of supporting regulation”: this is also considered to be a major
hurdle for the development of quality assurance of higher education in the
Arab countries (40%).
- “Lack of QA standards and guidelines”: to a much lesser extent (20%),
the lack of QA standards and guidelines could be a factor that inhibits the
development of quality assurance in HE.
- The “Negative stand of higher education stakeholders (ex. Ministry,
teachers' union, etc.)” is not really thought to be an issue with only one
respondent considering it to be so.
Section VIII. Capacity Building in Quality Assurance for Higher Education
1. Responsibility for QA capacity building for HEIs
Yes, you only
41%
12% 35%
12%
Others only (ex. Higher education authority)
Yes, you and others
This is a new section compared to the scoping study of 2008. It is meant to
investigate the situation of QA capacity building in countries of the region. One of
the objectives of this research is to develop a capacity development plan for QAAs.
Answers to the first question reveal that capacity building is conducted by the QA
entity and other bodies in 41% of cases; the QAA is exclusively in charge of it in
35% of the cases; the QA entity does not participate in any capacity building
endeavors and leaves it to other parties (ex. Higher education authority) in 12% of
the cases and finally, 12% of the respondents indicated that there did not exist an
official entity that is in charge of QA capacity building for higher education
institutions in their countries.
Per
cen
tag
e
2. Type of QA capacity building
80 73.33
70
60 46.67
50
40
33.33
30
20 6.67
10
0 Capacity building
in relation to external QA
procedures (ex. Drafting self-
evaluation reports, benchmarking,
etc.)
Capacity building to fulfill external
QA auditing requirements
(strategic planning, quality management)
No QA capacity
building has been organized so far
There are plans to
organize QA capacity building activities in the
near future
- “Capacity building in relation to external QA procedures (ex.
Drafting self-evaluation reports, benchmarking, etc.)”: is the most
sought after and most organized training either by the QA entities or other
parties (73%). This is typical of early QA training when HEIs are still
struggling with meeting procedural requirements of QAAs.
- “Capacity building to fulfill external QA auditing requirements
(strategic planning, quality management, etc.)”: is the second most
offered QA capacity building activity as HEIs gradually move beyond
procedures to actual substance of QA, as reflected in the review indicators,
both at the program and institutional levels.
- “There are plans to organize QA capacity building activities in the
near future”: Around 33% of the respondents are planning to undertake QA
capacity building activities in the future.
Per
cen
tag
e
- “No QA capacity building has been organized so far”: One respondent
(Eritrea) indicated that no capacity building activities for HEIs have yet been
organized in this country.
3. Type of capacity building used: The dominant format of capacity building offering consists of “1 to 3 day QA
80.00
80 70 60 50 40 30 20 10
0
46.67
53.33 53.33
seminars/workshops offered to all interested HEIs”. Around 53% is “QA training” or
“QA consulting”.
Finally, about 47% of capacity building is provided through “customized
seminars/workshops mediated on behalf of one or several HEIs”. This shows a
balanced mix of several formats and gives an indication of the possibility of
developing capacity in many different ways.
The capacity development plan that will be developed as an off-shoot of this study
will reflect that.
4. QA capacity building most needed by HEIs
Developing a research strategy
Program and curriculum benchmarking
Developing governance and strategic plans
Developing and reviewing program and course ILOs
Measuring and improving the academic standards…
Planning and developing QA management…
Developing teaching and learning strategies and…
Assessing the employability of graduates
Developing program and institutional…
Self-evaluation report writing
Developing an assessment strategy
Planning and Management of university facilities…
Enhancing the student experience
Operational/financial planning
Development of community engagement strategies
Professional development of Staff
20.00
20.00
13.33
6.67
73.33
66.67
66.67
60.00
53.33
53.33
46.67
46.67
46.67
46.67
40.00
33.33
0 10 20 30 40 50 60 70 80
Percentage
- “Developing a research strategy”: QAAs largely perceive that HEIs lack
research strategies and are mostly in need of capacity building in that respect
(73%). This perception comes from the continuous and pervasive failure of
HEIs to meet the research standards published by QAAs. It has to be said
however that QAAs have failed to come to terms with the primacy of teaching
for most HEIs and imposing on them a research agenda that is not part and
parcel of their strategy will not be productive. Capacity building should focus
on helping HEIs tailor research agendas to suit their objectives and the
primacy of the teaching function rather than emphasizing a research agenda
that HEIs are not strategically committed to.
- “Program and curriculum benchmarking” and “Developing
governance and strategic plans”: Two thirds of respondents have
identified these two areas as in need of urgent development in HEIs. They
are indeed the core development issues for program and institutional reviews
respectively.
- “Developing and reviewing program and course ILOs”: this is one of
the most sought after QA capacity building training (60%) and will continue
to be so until HEIs can muster the process of doing so independently. This is
happening gradually anyways.
- “Measuring and improving the academic standards of graduates” and
“Planning and developing QA management systems and structures”:
more than half of the respondents (53%) perceive a real need of HEIs
Capacity Building in the above. The first can be understood in terms of the
increasing importance of providing evidence for the academic standards of
graduates which is a key indicator for the quality provision of higher
education. The second relates to the sustainability of QA management in
HEIs as many have only managed to react to external QA procedures with
limited institutionalization.
- , “Assessing the employability of graduates”, “Developing program
and institutional improvement plans” and “Self-evaluation report
writing”: all were reported to be important by about 47% of respondents
warranting related capacity building. These all have to do with better
fulfilling requirements of external QA procedures.
- “Developing teaching and learning strategies and methods” and
“Developing an assessment strategy” were identified as capacity building
needs by HEIs in 47 and 40% of the cases respectively. Both constitute
central aspects of QA for teaching and learning.
- “Planning and Management of university facilities and resources”:
was rated by a third of respondents as a potential area for capacity
development.
- “Enhancing the student experience” and “Operational/financial
planning” were both rated at 20% as areas where HEIs needed
development.
- “Development of community engagement strategies”: 13% only
believe that HEIs need capacity building in the above.
Av
erag
e S
core
1 to
3 d
ay
sem
inar
s/w
ork
shop
s of
fere
d to
all
inte
rest
ed
HE
Is
Cu
sto
miz
ed
wo
rksh
ops
me
diat
ed o
n b
ehal
f of
one
or
seve
ral H
EIs
QA
tra
inin
g
QA
co
nsul
ting
- “Professional development of Staff”: Only one respondent showed an
interest in the professional development of Staff (Oman). But this could be
due to the fact that HEIs need to dedicate more resources for such an
endeavor rather than capacity building for doing it internally as it is generally
a function that is performed externally.
5. Importance of different formats of QA capacity building for higher education
5 4.50
4
3
2
1
0
3.45
4.42
3.25
QAAs perceive “1 to 3 day seminars/workshops offered to all interested HEIs” and
“QA training” as the most important vehicles for capacity building delivery (scores of
4.5 and 4.42 respectively, on a scale from 1 to 5). Of slightly less importance are
customized workshops mediated on behalf of one or several HEIs and QA
consulting” (scores of 3.45 and 3.25 respectively).
Section IX. Arab dimension of Quality Assurance
ANQAHE is a pan-Arab organization and seeks to embed an Arab dimension in QA activities
across the region. QA entities were asked whether they had already factored such an Arab
dimension and what were their related approach and objectives.
1. Relationship with ANQAHE
6%
23%
6%
65%
Your organization is a full member of ANQAHE
Your organization is an Associate member of ANQAHE
You are personally an individual member of ANQAHE
You have no formal ties to ANQAHE
Membership in ANQAHE is already indicative of the pan-Arab orientation among QA
entities. Two thirds of the surveyed QAAs are indeed full members of ANQAHE, and
one is an associate member. Furthermore, 23% of respondents are individual
members of ANQAHE and only one claims having no formal ties to ANQAHE
(Eritrea). This squares with the official numbers of ANQAHE affiliation which stand
as 13 member countries.
2. Relationship/ affiliation with any other national/ transnational/ international agency/ authority
29%
71% Yes
Non
About three quarters of the surveyed QAAs indicate having a relationship/ affiliation
with other national/ transnational/ international agency/ authority.
Answers include INQAAHE, AFRIQAN, The coordinating committee for Quality in the
Gulf Cooperation Council (GCC) and the Islamic Network for Quality.
3. Participation in the 2008 scoping study of ANQAHE
35%
65% Yes
Non
Two third of QA entities are repeat participants in the scoping study. However this
is in contradiction with the data given in the methods section where it shows that 15
out of the 17 participating countries in this survey did also participate in the 2008
survey. This contradiction is probably due to some respondents being unaware of
their organizations having participated in the previous scoping study.
Av
erag
e S
core
Exc
hang
e in
form
atio
n ab
out
qual
ity a
ssur
ance
Co
nstr
uct
new
qua
lity
assu
ranc
e a
genc
ies
or
orga
niza
tions
Dis
sem
inat
e g
ood
pra
ctic
e in
qu
ality
ass
uran
ce
De
velo
p st
and
ards
to
esta
blis
h ne
w q
ual
ity
assu
ranc
e ag
enc
ies
or
supp
ort t
he a
lrea
dy
pres
ent
ones
Str
eng
then
liai
son
betw
een
qu
ality
ass
uran
ce b
odie
s in
th
e di
ffer
ent
cou
ntrie
s
4. Participation to ANQAHE events in the past All the respondents have answered with “Yes” to this question and gave examples
such as “Building QA Agency - Beirut – 2011” and “the 1st ANQAHE Conference in
2011 at Abu Dhabi”. Respondents were in general a familiar crowd of ANQAHE
events, but this did not help with speeding their response as many had to be chased
at length to get them to respond. ANQAHE has to press upon its members a more
professional attitude towards participating in its activities. This becomes important
as ANQAHE considers a bigger involvement in setting up the premises for an Arab
higher education area.
5. Importance of ANQAHE objectives
5
4 3.23 3.18 3.00
3
2.95 2.89
2
1
0
The order suggested by the average scores calculated from the respondents’
answers is as follows (though the results are very close):
1. Exchange information about quality assurance
2. Construct new quality assurance agencies or organizations
3. Disseminate good practice in quality assurance
Per
cen
tag
e
De
velo
p st
and
ard
s an
d gu
ide
lines
for
QA
acr
oss
the
regi
on
Pro
mot
e an
d lo
bby
for
an
Ara
b H
igh
er E
duc
atio
n A
rea
Lobb
y fo
r an
incr
ease
d in
teg
ratio
n a
nd
stan
dard
iza
tion
acr
oss
the
Ara
b re
gio
n
Co
nstit
ute
a r
esou
rce
poo
l fo
r Q
A e
xte
rnal
pro
cedu
res
acro
ss th
e w
hole
re
gio
n
Pro
vide
ca
pac
ity b
uild
ing
reso
urce
s (p
eopl
e an
d m
ate
rial
) fo
r Q
A a
gen
cies
an
d H
EIs
in th
e re
gion
De
velo
p a
tool
for
ext
ern
al
QA
use
by
QA
bo
die
s
4. Develop standards to establish new quality assurance agencies or support the
already present ones
5. Strengthen liaison between quality assurance bodies in the different
countries.
Given the closeness in scores between the different objectives, all are deemed to be
important.
6. Relevancy/importance of ANQAHE objectives
80 70
60 53.33
50 40 30 20 10
0
60.00 60.00
53.33
73.33 53.33
- “Provide capacity building resources (people and material) for QA
agencies and HEIs in the region” is the objective that QA entities ascribe
the most to ANQAHE (73% of respondents).
- “Promote and lobby for an Arab Higher Education Area” and “Lobby
for an increased integration and standardization across the Arab
region” were both mentioned by 60% of Arab QAAs as an objective that
ANQAHE should pursue vividly
- “Develop standards and guidelines for QA across the region”,
“Constitute a resource pool for QA external procedures across the
whole region” and “Develop a tool for external QA use by QA bodies”
are also believed to be very relevant objectives for ANQAHE to have as stated
by more than half of the participants (53%).
Overall, the Arab dimension of higher education rings a bell with Arab QA entities.
ANQAHE should factor in this finding in its future outreach. This is further discussed
in the conclusions and recommendations section below.
Conclusions and Recommendations:
This study investigated several aspects of quality assurance in higher education in
the Arab world. Most of the findings were in line with the results of the 2008 study,
but emergent trends were spotted as well.
Twelve of the 17 surveyed countries had established QA agencies, and an additional
two were nearing establishment. This is a major development from the last study
wherein only 8 out of 16 respondents had established QA agencies. Yet, the number
of agencies that were created in the last 3 years is only one. This apparent
contradiction has to do probably with many agencies having been created in early
2009, hence falling outside of the 3 year period and past the time of responses to
the scoping study of 2008. The creation of formal and independent QA agencies is
testimony to the positive strides accomplished by Arab higher education.
In terms of governance, the new study revealed that Arab QAAs are increasingly
becoming mainstreamed in the higher education process and structure as they no
longer reported directly to a President/Monarch (against 15% in the last study) but
instead to a higher education authority; nor were QAA members appointed directly
by the highest level of leadership in the country. This confirms a new trend of lesser
involvement of the President/Monarch to the benefit of more institutionalized bodies.
In terms of funding, Arab QAAs are drawing more of their funding from charging
fees for quality assurance services offered. However and with little change from 3
years ago, not much has been done to secure more grants locally and
internationally.
On the operational front, and in relation to the site visit during external QA
procedures, QAA members accompanying the review panel are getting more
involved by participating as full members of the panel (whether as Chair or not).
This has been a significant development compared to the 2008 scoping study and
reflects the development of in-house QA capacity within QAAs.
Arab QAAs now also play a more active role in the write up of external QA reports,
with 85% of respondents indicating so from a mere 23% in the last study. Yet again,
this is a reflection of QA capacity having been built over the last few years.
One indicator of how the QA agenda has increasingly become a national agenda in
most countries is the compulsory nature of QA which has significantly increased in
the last 3 years.
The control framework within QAAs has also been reinforced with the establishment
of fully operational internal no-conflict of interest policies, and the submission to
external reviews of their activities. This clearly illustrates a strategic orientation
among QAAs to be models of quality culture which is what they preach for others.
This study went beyond the scope of the scoping study of 2008 by investigating new
dimensions of Arab QAAs’ work such as the core issues of capacity building and the
Arab dimension of higher education.
Specificities of Arab QA Entities Arab QA entities have generally opted for being called either a commission or an
authority, but not an agency or entity. “Center” or “Council” also appears in QAA
names.
Contrary to common belief, Arab QAA remit is not limited to higher education but
includes vocational education and training (46%) and pre-university education
(38%). As expected and given the dearth of self-standing research centers in the
Arab region, very few have paid any attention to develop related QA procedures.
Overall, Arab QAAs appear to struggle in dealing with assessing research within HEIs
although they highlight it as the core problematic area within HEIs.
QA procedures have extended to cover private and public institutions alike. This is a
reflection of the generalization of QA to all country institutions irrespective of their
governance structure. This is in line with QAAs becoming almost exclusively the
owner of the external QA agenda in almost every country.
In terms of governance, Arab QAAs report primarily to the higher education
authority of their country and secondarily to the Prime Minister/Cabinet Affairs.
Regarding its Board composition, many stakeholder groups are represented with a
dominant presence of HEIs, Government and industry and labour market
representatives. Members of their governing bodies are appointed by the Minister of
Higher Education or the Prime Minister most of the time. QAAs maintain very active
communication with their different stakeholders through a variety of media including
published material, workshops and the Internet. The bulk of what they communicate
consists of QA process and responsibilities of different partners, QA evaluation
results, and the mission, vision and strategic objectives of the agency.
Few agencies have so far undergone an external review of their own activities
(18%). However, this is expected to change in the future as 27% are currently
preparing for being externally reviewed while 45% are planning to be subject to
external review in the near future. For those who already got externally reviewed,
the review was mainly performed by an international agency body, reflecting the
emphasis of Arab QAAs to be internationally legitimized.
Arab QAAs are fundamentally public entities funded through governmental budget
allocations (71%). However, they have consistently increased their income from
fees for QA services offered (35%). Respondents were not optimistic on the financial
outlook of QAAs as there is a general feeling that QAAs are not properly funded to
fulfill their mandates of external quality assurance, and even less so to face future
challenges. Staffing varies greatly from one agency to another (from half a dozen to
few hundreds). While most Arab QAAs feel that they are adequately staffed to
conduct their core activities, they have no dedicated resources for research and
knowledge management activities. This denotes the utterly operational focus of
QAAs at the expense of a developmental dimension.
Internal Quality Assurance for QAAs As a key development since the last scoping study was conducted, most QA entities
have established effective no-conflict-of-interest mechanisms in the work of their
external experts. An appeals system and mechanisms for collecting and analyzing
internal and external feedback for improvement purposes have also been established
by a two third majority of responding QAAs indicating an increased awareness of the
need to set the example for quality management among QAAs.
The above has yet to be complemented with the publication of policies for the
assurance of the agency’s own quality on their websites and the institutionalization
of mandatory cyclical external reviews of their activities. In a fast changing industry,
Arab QAAs relatively stagnate with only 27% having changed their approach
significantly to evaluation in the past three years.
Framework for External Quality Assurance QA in higher education is being increasingly made mandatory by higher education
authorities (79% against 69% in the last study). QA can take several shapes and
formats and mostly consists of reviews, audits and accreditations, both at the
institutional and program levels. An intermediary unit level (ex. College), was not
found to be very prevalent. While only half of the Arab QAAs perform audits, most of
them perform the evaluation of institutions (85%) and programmes (77%) and
undertake accreditations (77%). However, very few if any at all do any
benchmarking. External QA procedures are generally performed periodically rather
than on an ad-hoc/ request basis.
Arab QAAs draw their power from the formal consequences of external quality
procedures for HEIs which consist of de-licensing/suspension (73%) or
approval/licensing (64%). Usually this is done through the channel of a higher
education authority, based upon the recommendation of the QAA.
In the process of external quality assurance, Arab QAAs perceive the following to be
the most important areas:
at programme level: “assessment of learning outcomes attained by
graduates”, “teaching and learning”, “professional and pedagogical
qualifications of staff” and “quality management system”.
At the institutional level: “faculty members”, “quality assurance
management”, “governance”, “educational programs”, “students and students
support”. “Community engagement” and “institutional integrity”.
The Arab QA entities perform their QA activities against the backdrop of several QA
frameworks such as: “best or good practices” and “standards and guidelines defined
by professional groups/organizations either local or international” (both 100%),
“local regulations” and “goals and objectives set by the institution being evaluated”
(both 85%), and to a lesser extent “Institutional, program and subject benchmarks”
and “National frameworks of qualifications”. This clearly denotes a standards-based
approach, combining golden standards with a fitness-for-purpose approach.
Though the QA entity does most of the investigative and evaluative work, the
Ministry of Higher Education is still the prime responsible for the approval of new
subjects, programmes and institutions. The oversight of Ministry approval over
subject approval illustrates the bureaucratic legacy of higher education management
in many Arab countries. Many HEIs are caught between the stringent requirements
of QAAs and the reduced margin of manoeuver afforded by higher education
authorities.
By December 2014, the establishment of “new HE legislation (including about QA)”
is the most expected development in higher education sector in Arab countries
(92%). The “establishment of National Qualifications Framework” is equally highly
anticipated (83%). To a lesser extent, the introduction or revision of external quality
procedures and the “reorganization of the QA agency” are also expected to take
place within the next three years. This shows that Arab QAAs are weary of being
active in a sector where change will continue to be the only constant and thereby
illustrate their readiness to adjust accordingly.
Standards & Methodology for External Quality Assurance All Arab QAAs publish their standards and criteria (ex. Review handbook) for
external quality assurance. Stakeholders play a formal role in the specification of
related processes and criteria. The most involved of such stakeholders include HEIs
(73%), the HE authority (73%), hired consultants (67%) and industry and labour
market representatives (33%). International associations of HEIs such as ANQAHE,
however, are seldom invited to contribute to the specification of processes and
criteria for the external quality assurance of HEI (only 13%). This being said, such
organizations do influence the development of processes and criteria for external QA
through organizing seminars and workshops for their members. The QA entity
makes the final decision about these specifications though; which are subsequently
published prior to implementation.
Site visits are a universal requirement for all Arab QA entities. Moreover, most
require an external assessment by a group of experts (87%), a follow-up procedure
undertaken by the assessed institution (80%) and a self-assessment report (73%).
The QA agency formally appoints the members of the external review/audit panels
most of the time (81%). These are nominated by the QAA staff (80%), or
alternatively retrieved from a QAA maintained database (47%), nominated by the
Ministry of HE (27%) or sometimes chosen through a call for reviewers (20%).
Panelists are primarily HE and QA experts, secondarily subject matter experts and
thirdly staff members of the QA entity. Most of them receive compulsory
training/briefing before performing their QA tasks.
The Site Visit The site visit is now required by all QAAs against only three quarters of them three
years ago. It is generally undertaken by the expert review panel alone (60%) or
accompanied by QAA member(s) (33%). The latter play either a
facilitator/coordinator role (53%) or is a full member of the review panel (47%).
The site visit lasts from 3 to 4 days on average, except for pre-university school
reviews which is shorter. Site visits are structured along a standard structure
including “meetings with faculty and staff of the institution”, “meetings with the HEI
board and top-management”, “tour of the facilities”, “meetings with
students/graduates” and “examination of documentary evidence”.
Outcome of Quality Assurance Procedures It is usually the “Board of the QA body” that makes the final decision on the
outcome of external quality assurance procedures (57%) or “the Ministry or another
HE Authority” (29%).
The outcome could be either formative or summative for institutional or program
reviews, and mostly formative for unit reviews.
Once a first draft of the final report is generated, many parties may have access and
contribute some input to it including the review panel (73%), the QA entity (67%) or
the HEI under review (60%).
The report is only sent to the assessed HEI in half of the countries (47%).
Otherwise, 20% of QAAs publish a summary version while another 20% publish it in
full. The report is structured in a similar way everywhere with sections on
“conclusions and recommendations” (86%), “analysis of the evidence in relation to
standards and indicators used for the review” (79%), and “judgment and
consequences” (71%) as the most common sections between countries. The report
is published by the QAA itself most of the time (79%) and less often by the Ministry
of HE (21%).
The QAA is also the first responsible for the follow up of the report’s
recommendations implementation (80%) while this is left to the own initiative of
reviewed HEIs in third of the cases. The follow up consists mostly of follow-up site
visit(s) in all cases, submission of action/improvements plans by the reviewed
institution (78%), periodical reporting from the HEI (64%) and in half the cases a
repeat of external quality procedure (50%).
HEIs’ challenges QAAs have a first-hand perspective on the key deficiencies in the internal quality
assurance systems of HEIs. This is helpful to define a capacity building strategy for
HEIs QA efforts. The following six areas have been identified as the most
problematic ones: (1) preparing program and institutional improvement plans, (2)
developing governance and strategic plans, (3) enhancing the employability of
graduates, (4) drafting the periodic self-evaluation report, (5) program and
curriculum benchmarking and (6) embedding program and course Intended Learning
Outcomes (ILOs) in the curriculum.
With regard to their own challenges in carrying their QA review mandate, QAAs face
several obstacles, mainly: (1) lack of QA culture and/or experience in HEIs, (2)
limited resources/funding, (3) lack of autonomy from higher education authority in
the country and to a lesser extent, (4) absence of supporting regulation for QA.
Capacity Building Arab QAAs are not always the sole responsible for capacity building activities in their
countries as there are usually other parties that offer such services. But QAAs are
instrumental in helping HEIs define their most important needs in capacity building.
Currently, the most prevalent type of capacity building relates to external QA
procedures (ex. drafting self-evaluation reports, benchmarking, etc.) or to fulfilling
external QA auditing requirements (strategic planning, quality management, etc.).
Typically, capacity building activities are offered through “1 to 3 day QA
seminars/workshop offered to all interested HEIs” (80%), to a lesser extent through
“QA consulting” or “QA training” (53%) and sometimes through “Customized
seminars/workshops mediated on behalf of one or several HEIs” (46%).
Surprisingly, the study revealed that the area where QA capacity building is most
needed is “developing a research strategy” (73%). This has been ascribed to the
lingering struggle that HEIs have had in meeting research requirements of QAAs.
However, this goes beyond capacity building and has to do with the primacy of
teaching in the service delivery strategy of HEIs. Until such strategies are refocused,
if it is at all feasible, the failings in research of most profit-driven institutions will
remain. Other than research, the most important QA capacity building needs as
stated by participating QA entities include: (1) Program and curriculum
benchmarking, (2) developing governance and strategic plans, (3) developing and
reviewing program and course ILOs, (4) measuring and improving the academic
standards of graduates and (5) planning and developing QA management systems
and structures”. This is in line with the problematic areas perceived by QAAs within
HEIs.
Arab Dimension of Quality Assurance As a sign of increasing commitment to the idea of Arab higher education, 2 thirds of
QAAs surveyed are now full members of ANQAHE (against 36.4% three years ago).
ANQAHE also has its first associate member ever (Yemen). About 20% of this
survey’s respondents are individual members of ANQAHE. Moreover, all respondents
or their QA entities attended past events of ANQAHE.
When asked to assess ANQAHE objectives, QAA respondents ranked them in
importance as follows: (1) exchange of information about QA, (2) construction of
new QAAs or organizations, and (3) dissemination of good practices in QA. They
further recommended that ANQAHE should embrace the following three objectives
as a matter of priority (among many others): (1) provide capacity building resources
(people and material) for QA agencies and HEIs in the region, (2) promote and
lobby for an Arab HE area and (3) lobby for an increased integration and
standardization across the Arab region. ANQAHE ought to revisit its strategic
objectives in light of these results.
Beyond ANQAHE, many of the surveyed QAAs have ties with other QA
agencies/networks including INQAAHE and AFRIQAN among others.
Recommendations for ANQAHE:
ANQAHE has not yet reached out to some countries in the region (ex.
Mauritania, Somalia, Comores). This has restricted the response rate as a
result because many of these countries were not reachable. ANQAHE should
try to reach out to these countries by following formal channels such as
through higher education authorities.
Many existing members of ANQAHE were not very responsive to answering
the survey either delaying the study as the data collection period lasted
almost 5 months or failing to answer altogether. ANQAHE should factor this
in financing and sponsoring capacity development activities for its members.
Kuwait, Libya, and Yemen for instance showed the biggest enthusiasm for the
study and were the first three respondents. Any future event in relation to
the dissemination of the study results should be organized in one of these
countries for instance.
Results show that QAAs are predominantly funded by Governments and the
proportion of grants is minimal. Given the national and worldwide support for
enhancing higher education, ANQAHE should work with QAAs to help write
grant proposals and submit them to sources both local and international.
Although Arab QAAs have made significant progress in covering new forms of
higher education such as “distance and hybrid programs,” many are still
unable to assess them properly. ANQAHE could play a role in this regard by
helping transfer successfully benchmarks within the region.
Arab QAAs are struggling with getting to grips with the predominant teaching
orientation of local HEIs. They subject them to research requirements which
the latter fulfill only superficially at best. ANQAHE should work with Arab
QAAs to develop a fitness-for-purpose assessment for the research activities
of primarily teaching institutions. Moreover, Arab QAAs should include
research centers within their remit.
Another possible area of capacity building is the development of proper
policies and processes to ensure the quality of work subcontracted by QAAs.
National Qualification Frameworks are emerging as key artifacts in
reformulating many aspects of higher education in the region. ANQAHE
should activate its Arab qualification framework project and validate it against
national frameworks to ensure convergence of HE systems and the
emergence of a Arab higher education area.
There is a dearth of information on local and regional benchmarks, be it
programs or institutions. Arab HEIs usually resort to benchmarking against
international institutions that operate within completely different
environments. ANQAHE should work with QAAs and independent researchers
to document good practices in the region for use by HEIs.
Arab QAAs have acquired power within the higher education edifice of their
respective countries because of the formal consequences that QA reviews
have in terms of licensing/approval or de-licensing/suspension mainly.
However this is not counter-balanced by a rigorous approach to validating QA
review results. ANQAHE should work with Arab QAAs to undertake such
validation work and be amongst the pioneers in the world to do so.
The concept of an Arab higher education area was appealing to respondents.
ANQAHE should further pursue the subject and detail it for its members.
ANQAHE should use the results of the scoping study to revisit its strategic
objectives, and improve its approach to the development of higher education
QA in the region. A comprehensive capacity development plan should be
elaborated to serve both QAAs and HEIs in their internal QA needs.
Impact of QA in the Region and Recommendations for QAAs
It is undeniable that the QA in higher education tide in the Arab region has had a positive impact on the higher education scene although it would be difficult to provide tangible evidence for it. Likewise, public authorities in the Arab World seem to be cognizant of the critical role that QA agencies can play in improving the quality of higher education in their countries and have invested accordingly. QAAs have come to realize the importance of their mission and are increasingly gearing for self- assessment and for developing their own internal control system. This was a major development compared to the first scoping study. However, another major finding of this study is that Arab QAA’s undertake little institutional research to validate the results of their assessments or develop their practices to make them more effective. This is best illustrated through the lack of research and knowledge management staffing in all QAAs surveyed. Such a state of affairs condemns QAAs to a routine QA operation that cannot evolve to factor in idyosincracies of local academic environments. This is all the more critical as most QAAs have adopted QA frameworks from elsewhere and used external consultants to adapt them. But even this adaptation process was generally done without much evidence collected from the local environment. The fallouts of the ill-conceived QA frameworks can be seen in what is perceived by QAAs as in need of priority capacity building among HEIs. Research is a case in point. Traditionally, research in the Arab region has not been developed enough and most HEIs have a primarily teaching vocation. However, the existing QA frameworks continue to probe for a research environment that cannot exist within the current structure and vocation of higher education in the Arab region. As a result and in their efforts to cope with QA requirements, HEIs have learned to produce the requisite ‘amount’ of research that will sail them through the said QA requirements. It is important therefore to revisit existing QA frameworks and continue to adapt them to local environments. This requires establishing institutional research capacity within QAAs to undertake such a development.
One further scrutiny that QAAs have to undertake has to do with the validity of their QA assessments. Despite the increasing awareness about accountability amongst QAAs, there does not seem to be in place a mechanism to authenticate results of QA reviews. Given the gravity of the QA process consequences for HEIs and for students, this ought to be undertaken urgently. Whilst internal QA processes are well entrenched as sorted out by this second scoping study, QAAs should be weary of not falling in a nominal compliance game where the critical object of compliance is overlooked.
This second scoping study has highlighted critical areas in need of capacity development both for QAAs and for HEIs within their remit. A chart will be generated to plot individual QAAs against the collective evaluation of all responding QAAs. On this basis, a number of capacity building recommendations will be made and corresponding activities illustrated in a capacity development guide that will be produced to guide ANQAHE’s future support to its members.
Appendix A: Details of Participants
Country
Participant
Position/Title Agency/Entity Address
Telephone/FaxEmail address /
Website
Libya
Mohamed Elkabir
Director CQAAE&TI Ben Ashor, Tripoli Libya
+218213617328/ +218213621465
www.qaa.ly
Libya
Abdullah Abduljalil Muhammad
Director of Equivalence Administration
CQAAE&TI
Ben Ashor, Tripoli Libya
[email protected] / www.qaa.ly
Sultanate of Oman
Dr. Salim Razvi
Quality Assurance Expert / Acting CEO
Oman Academic Accreditation Authority
PO Box 1255 Al Khuwair, P.C: 133, Oman
+968 24614361/ +968 24614364
www.oaaa.gov.om
Lebanon
Ahmad Al JAMMAL
Director General of Higher Education
+961 3 333917/ +961 11 772500
[email protected] / http://www.higher-edu.gov.lb/
Kingdom of Bahrain
Dolina Dowling
Executive Director: Higher Education Review Unit
Quality Assurance Authority for Education & Training
PO Box 30347
+97317562305/ + 97317562306
[email protected] / www.qaa.edu.bh
UAE
Badr Aboul-Ela
Executive Director The Commission for Academic Accreditation
Ministry of Higher Education Building, ElNajda Street, Abu Dhabi City
+97126951451/
+97126428488
[email protected] / www.mohesr.gov.ae
Sudan Isam M Mohamed President of
Evaluation and Accreditaion Commission (EVAC, Sudan)
Evaluation and Accredation Commission (EVAC)
Ministry of Higher Education & Scientific Research, Khartoum Sudan, POBox 2081
+249 793469/ [email protected] / www.mohe.gov.sd
Egypt
Maha Farouk Rashwan
National Authority for Quality Assurance & Accreditation of Education
5 Mahmoud el- Meliguy , 6th district Nasr city- Cairo Egypt
002-02-24023988/
002-02-22619389
[email protected] / www.naqaae.eg
Country
Participant
Position/Title Agency/Entity Address
Telephone/FaxEmail address /
Website
Tunisia
Oussama
BEN ABDELKARIM
Directeur du Bureau des Etudes, de la Planification et de la Programmation
Avenue Ouled Haffouz, 1030 Tunis
+216 71783941/
+216 71795358
Kuwait
Dr. Farida Mohamed Ali
Acting Secretary General of Private Universities Council
Council of Private Universities
East Street Khaked Ibn Elwaleed, Borj Elmadina, Kuwait
+965 22240591/ +965 22455326
[email protected] / www.puc.edu.kw
Yemen Muhammad
Mahfoudh Abdallah Elhid
Deputy Director of Higher Education Quality Improvement Project
Higher Council of Higher Education Quality Assurance
Ministry of Higher Education and Scientific research
[email protected] / www.heqipyemen.org
Qatar
Dr. Khaled Muhammad Elhasan
Education Consultant Council for Higher Education
PO Box 35111 Doha Qatar
+ 97444560812/ +97444270598
[email protected] / www.sec.gov.qa
Syria
Muhammad Nejib Abdelwahed
Assistant to the Minister of Higher Education, Scientific Research and Academic Affairs
Ministry of Higher Education
Ministry of Higher Education, Damascus, Syria
+963112130752/
+963112130753
Eretria
Muhammad Othman Ali
Coordinator of Arabic Degree Exams
Not established yet
Center of Research and Consultation, training and tests - University of Asmara
KSA
Dr. Saad
Al-Zahrani
Deputy Director of NCAAA
National Commission for Academic Assess. and Accreditation
Riadh, KSA
+966 0555424196 [email protected]
Palestine
Prof. Mohammed Alsubu
Accreditation and quality assurance commission
West bank, Palestine
+97222980140 / +97222980139
Appendix B: Question match between the 2 scoping studies (2008 & 2012) About half of the 2012 survey’s questions were new. The rest of the questions were somewhat reiterated from the 2008 survey, although reworded:
2008 Survey Questions 2012 Survey Questions The year of establishment of QA Organization When was your agency established? The agency is considered as What is the organizational denomination of the Agency?
Relation of the Agencies to ANQAHE Do you have any relationship with ANQAHE?
The legal status of the QA agency is What is the legal/regulatory basis of your agency? The EQAA is funded by In the year 2011, what was the percentage of your agency's funding
from each of the following sources?
The method of appointing the governing body of QA agency Who appoints members of the QAA Governing body?
QA agency reports to Your agency formally reports to
The functions of the QA agency Based on the definitions above, please indicate which external quality
procedures you perform
The most significant activities of the QA agency For each external quality assurance procedure performed, please indicate frequency
The institutions covered by QA agency are Remit or scope of QA Agency The process of QA agency for HEIs is Is QA in higher education performed in your country compulsory or
voluntary?
The same Quality Assurance standards and processes are applied to all HEIs both public and private
Are public and private HEIs subject to the same QA procedures?
Who participated in the development of the standards/ criteria? Which of the following stakeholders have or had a formal role in the specification of processes and criteria for the external quality assurance of higher education institutions in your primary domain?
The process of QA requires a self-evaluation report by HEIs Which of the following processes does your QA entity use/require within its external quality procedures?
Members of the review panel Who can be a member of an external review panel?
2008 Survey Questions 2012 Survey Questions
Identification of the potential reviewers How are reviewers selected?
Appointment of the review panel Who formally appoints members of the external review/audit panels used for external quality procedures?
A clear policy on conflict of interest among the reviewers An enforced no-conflict-of-interest mechanism in the work of your agency's external experts
The QA process includes a site visit to HEIs by the review panel Do you require a site visit for your external QA procedures?
The duration of the site visit What is the duration of the site visit? (count only days on-site and not briefing or back-office days)
Activities conducted by the review panel during the site visit What is the program of the site visit?
Reviewers meet during the site visit Who is interviewed during the site visit?
- QA agency staff join the site visit panel - Role of the agency staff in the site visit
If a member of the QA entity participates to the site visits, which role does he/she play?
QA agency training strategy / process for the reviewers Members of an external review panel are familiarized with external QA procedures through
Dissemination of policies and procedures to reviewers and HEIs A published policy for the assurance of your agency's own quality, made available on its website
The decision on the outcomes Who makes the final decision on the outcome of the external quality procedures? (i.e. The decision is official and communicated to the HEI)
The inputs that are considered when the decision is made by the QA agency
Once a first draft of the report is generated, who can have an input into it, is consulted or informed about the content?
The outcome of the site visit is given to the institution What is nature of the external QA procedure outcome?
The conclusions in the report include What is the content of the published report? (published content only if full report is not published)
The part of the outcome is made public Is the final outcome (assessment/evaluation report) published?
2008 Survey Questions 2012 Survey Questions
- Formal follow up requirements on the conclusions and recommendations of the review
- Requirements to submit follow up reports, response to recommendations, or other data after the review
- Who does the follow up for the implementation of the report's recommendations?
- What is the nature of this follow up?
Access to appeal mechanisms An appeals system The frequency of the program review How frequently do you perform each external quality procedure?
QA agency does a review of its own activities Internal quality assurance procedures which include mechanisms for self-assessment
QA agency does internal or external or both internal and external review of its own activities
Mandatory cyclical external review of your agency's activities including the external review process at least once every five years
Changes in policies & procedures significantly changed on the basis of the review results
Has your Agency significantly changed its approach to evaluation in the past three years?
The presence of more than agencies in the country Are there other agencies/authorities responsible for the external quality assurance of HEIs?
Collaboration with other QA in other countries Do you have any relationship/ affiliation with any other national/ transnational/ international agency/ authority?
Appendix C: 2012 Survey Questionnaire