www.aqu.cat
ENQA TRAINING OF AGENCY REVIEWERS
Focus on the role of the secretary and agency view
Núria Comet, Bern 12-13 May 2014
ENQA Review : a long way
Agency PanelPanel
AgencyAgency
Report
ENQA
SERENQA
~ 6m – 12 m 2m 2 days 5 years
The point of view of the Agency
The Self evaluation report (SER) is the basic sourceof information for the panel and has to provideclear information, sufficient reflections, critiqueand analysis.
Its contents must be corroborated with evidences.
The reports allows the review team to prepare lines of enquiry in advance of the site visit.
The point of view of the Agency
Some recommendations for drafting the self-evaluation report:
careful study of other agencies’ self-evaluations
meetings with other agencies
make a team responsible (need of human resources)
define the structure of the SER
evidence for each criteria had to be collected in one
document/ links (and translated to English)
draft SER should circulate throught steering bodies for
comments
The point of view of the Agency
SER- strengths:
occasion for self-reflection on strengths and
weaknesses (SWOT analysis)
learning process
engenders improvement mechanisms
“team building“, participation
transparency (SER is public)
consultative process overall acceptance
The point of view of the Agency
SER- weakness
heterogeneous style, repetitions
different level of interpretation of standards
continuous fine-tuning distance from initial
focus
“politically correct” in detriment to self-criticism
weak analytical approach
Second stage
The role of the secretary
The agency will publish the completed self-evaluation document on its website and submit it to the review secretary.
Role of Secretary
Member
Logistics
Communications
Panel
Chair
Agency
ENQAReport
ScheduleBefore
During
After
The secretary has a special role
Role of Secretary
• To get to know the ENQA Guidelines well• Read other ENQA Review Reports
• Read and analyse SER
• Review documentary evidence • Look at Agency website• Examine national documents
Maintain accurate list from the outset of all documents received and considered
As member of the panel:
Role of Secretary
• Time-line for whole review: prior meetingsite visitreport
• Manage logistical arrangements: • Travel arrangements: arrival and departure
times• Hotel booking (+ room for the prior meeting)• Evening meals
As responsible for the logistic:
Role of Secretary
As responsible for communication
• Act as the single conduit between the Panel, ENQA and the Agency throughout the review process.
• Make early introductions to the Institutional Contact (Agency), the Chair, and the Panel.
• Hold telephone meetings/Skype with the Chairregularly (2-3 recommended)
• Establish clear deadlines• Keep team on track
Role of Secretary
As responsible for the preparation of site-visit
Agree visit programme with the Chair and the Agency.
Pay attention to:
. the length of each interview
. assure that all the interest group are interviewed. the people (names/responsability) interviewed. break-times. arrange interpreter support (only if necessary). be sensitive towards the additional request
Role of Secretary
Type of interviews the ENQA review panel required to conduct
• executive board (first – last meeting)
• resp. of quality assurance/ of accreditation department
• chairs (and/or members) of permanent commissions;
• staff
• representatives of the Ministry / Government
• external evaluators
• representatives of public/private universities/ inst. of technology/
• students (preferably who participated in evaluations)
• stakeholders (e.g. employers)
Role of Secretary
As responsible for the preparation of site-visit:things to sort out during the prior meeting
The members must have a clear idea of their role during the site visit
• Produce clear agendas for the meetings
• Prepare a check-list
• Ensure what the panel need to clarify, which information they need, or which evidences need to find
Role of Secretary
During the site visit
• Use time wisely• Use the checklist • Keep notes of formal and informal discussions• Check that sufficient evidence has been
gathered at the end of each session and at the end of each day
• Keep the chair advised of any shortfalls or if sufficient evidence has been gathered
• Use the last internal meeting to agree:• the findings to be presented to the Agency
on the final day• next calendar
Role of Secretary
The final report It’s mainly your task as secretary !!
Start drafting the report as soon as possibleYou must make time available Block out one /two solid weeks
• Review notes, check records of documentation• Summarise team’s analysis and comments on
SER
• Consult all of team for additions and changes• Final consultation with Chair
• Send to the Agency to review accuracy• If necessary, rewrite the Report
Role of Secretary
The final report must
• Accurately reflect the findings of the panel • Provide evidence, analysis and conclusions against
each ENQA membership Criterion• Ensure the requirements of the exercise are met: Type
A or Type B • Reflect the national idiosyncrasy
Provide an clear account of the evidence collected by the team for the ENQA Board to make its decision on membership
Provide a useful tool for the ongoing development of the Agency itself
Feedback of the Agency
The agency welcomed panel members who can understand their HE system and provide
additional insights
who are highly professional and with impartial analysis which include a national panel member
About the site -visit: practical problems with foreign reviewers and
stakeholders’ participation in the site visit
necessary to improve stakeholders’ involvement
Feedback of the Agency
The final meeting : whilst Panel had acquired a global image… the agency
ignores Panel conclusions
it’s a short meeting, not a list of degree of compliance not the final decision
The Report The agency expects that the report includes strengths,
weaknesses and recommendations
Feedback of the Agency
The agencies think that
It’s an European recognition of the Agency
It’s a way to gain legitimacy towards HEIs/ goverment
It’s a way to continue the work of improving the
quality of its evaluations