selecting and training students for the external review panels
DESCRIPTION
Selecting And Training Students For The External Review Panels: The Romanian Experience. Presented at 3rd Quality Assurance Forum in BudapestTRANSCRIPT
Traian Brumă and Violeta Caragea
University of Bucharest
SELECTING AND TRAINING STUDENTS FOR SELECTING AND TRAINING STUDENTS FOR THE EXTERNAL REVIEW PANELS: THE EXTERNAL REVIEW PANELS: THE ROMANIAN EXPERIENCETHE ROMANIAN EXPERIENCE
GREVA
20032003
SBU 2
Students trying to
CHANGE universities
Students trying to
IMPROVE universities
ARACIS starts in 2006
ANOSR was entrusted with selecting, training
and nominating students for evaluation
teams.
Timeline
March 2007: ANOSR’s MethodologyApril and May 2007: training sessions for 64
students. May – June 2007: 11 pilot institutional
evaluations involving 13 student evaluators. August – October 2007: evaluating the impact
of student participation in the review teams. February – March 2008: Rethinking the
student evaluators training.April 2008: 27 more students were trained June – July 2008: 6 more institutional
evaluations
Our aim is to ...
Continue the
European conversation
Selection
analyzing the self-evaluation report and
other documents,
preparing the site visit,
participating to different visit activities
(interviews, visiting academic infrastructure
etc.),
writing the external evaluation report.
Job description:
Job description -> profile:
(1) Knowledge,
(2) Skills,
(3) Personal values and,
(4) Motivation.
Profile -> selection intruments:
A letter of intent,
An essay,
A curriculum vitae.
+ An assessment grid (scoring1 to 100)
main criterion - the score (more than
50pt) ,
participation in training – mandatory,
additional criteria were (eg: to evaluate an
institution with a similar profile)
Nomination:
exclude the criteria related to personal
values;essay replaced by an interview taken after
the training; a peer evaluation exercise was introduced.
Changes:
Training
Learning outcomes for 2007: Understanding the global context of higher
education and the role of QA; Understanding the role of QA in the Romanian
context and the challenges ahead; Operating with basic concepts related to QA; Understanding the emerging principles of the
QA process in Romania;Being able to work with the basic documents
regarding QA; Understanding the students' role in
institutional evaluations reviewers’ team.
Training delivery
Facilitated by two students from
ANOSR.
Seven presentations delivered by
guest speakers;
Participants’ feedback was very positive.
Some improving suggestions:
better time management,
more practical activities.
Feedback
interviews with student evaluators,
evaluation of students' reports and
the report of the independent audit of the
institutional evaluations pilot phase.
Assessing the impact of the training program:
students raised a significant number of
(negative) issues ;
students reports contained specific
recommendations, grounded in the reality
of the university life;
their involvement was seen as mature,
pragmatic and competent.
Results (+):
a need for improvements regarding the site visit organization;
there is little awareness about the students’ role;students reports didn't covered enough the issues for
which their experience and perspective is valuable;in some cases students’ interventions during the site
visit contained an imperative tone, lack of strong arguments, tendentious generalizations of situations encountered in one department for the whole university;
a guide for writing the report would have been necessary.
Results (-):
Rethinking the Training:
the input consisting of: results of the feedback meeting of the 2007 training coordinating team;
student interviews;an independent evaluation report of the pilot phase.
Conclusions
The training for students was
insufficient;
The major gap was the lack of
evaluation skills.
Changes in training:
Learning outcomes focus shifted to practical evaluation skills
Training delivery: roleplays, simulations.
To do list (1)
Better promoting QA to students and the possibility to be involved;
Connecting the training program for the external reviewers with the programs for the students involved in internal QA;
Joint training for students and other experts;
Developing additional modules to the initial training program;
Building an online platform for the student evaluators community;
To do list (2)
Blended learning;Further improving the training methods;Improving the documentation (describing the
role of students in review teams, developing a handbook for student evaluators);
Inviting foreign students evaluators as guest speakers;
Finding a dynamic balance between refreshing the pool and building on the experience;
Recognition of ECTS credits awarded by all Romanian universities.
QUESTIONS
Does it worth to extend the student involvement to the study programmes evaluations? Do we have the capacity to do it?
How should we design the "pool" / "community"/ "system" in order to have it refreshed every year and to build on the expertise and experience that is being generated every year?
How many reviews a student should participate in? How can we improve the integration of the student
in the reviewers’ team? Could we have a paid secretariat (project manager
+ 1 staff); should it be run by ARACIS or by the NUS?
