u-multirank approach to m ultidimensional evaluation of hei performance getalo elena lead expert,...
TRANSCRIPT
U-MULTIRANK Approach to multidimensional evaluation of HEI
performance
Getalo Elena
lead expert,
Development Programs Office,
Tomsk Polytechnic University
28 March, 2014
U-Multirank
lead partners:CHE - Centre
for Higher Education (Germany)
CHEPS - Center for Higher Education Policy Studies
CWTS - Centre for Science and Technology Studies
Elsevier
Bertelsmann Stiftungfolge3
Consortium for Higher Education and Research Performance Assessment
The Project is financed by the European Commission (2009)
Partners:
U-Multirank
New approach to universities ranking (features)
multi-dimensional (orientation not only on research activity (aims, missions, diversity of universities), transparency
user-driven (possibility for users to develop personalised rankings by selecting indicators in terms of their own preferences (practical orientation)
multi-level (combination institutional ranking (of whole institutions) with field-based rankings that focus on particular academic disciplines or groups of programs.
no calculation of composite overall indicators and league tables
comparison institutions with similar institutional activity profiles
U-Multirank
NEW Approach: the comparability of institutions is important; institutions and programs should only be compared when their purposes and activity profiles are sufficiently similar.
Two-step-process:
1. identification of the institutions with similar profiles by “mapping” their activities;
2. ranking of “like with like” can only be applied afterwards.
1.
2.
>
Is the information enough obvious to be properly
comparable?
The information is comparable.
=
U-Multirank
1. Dimensions (5 categories of university performance evaluation): -teaching and learning- research- knowledge transfer- international orientation - regional engagement
2. Fields of study: 2011: Mechanical /Electrical engineering, Business 2013: Business, Electrical Engineering, Mechanical Engineering, Physics 2015: Computer Science, Psychology, Sociology/Social Sciences and Music
3. Data collection:
- Questionnaire Institutional Data
- Questionnaire Field – based Data
- Questionnaire Students Survey
Dimension Indicator Definition
Teaching and Learning
Expenditure on teaching
The percentage of total institutional expenditure dedicated to teaching activities
Degree level focus
The number of higher degrees (master and Ph.D.) awarded as a percentage of the total number of degrees awarded
Scope The number of broad educational subject fields in which a significant number of degrees are awarded
Highest degree awarded
The highest degree level the institution has awarded degrees at
Research
Expenditure on research
The amount of money spent on research activities in the reference year as a percentage of total expenditure
Academic research publications
A count of peer reviewed publications of the institution per FTE academic staff
Professional publications
A count of professional publications of the institution per FTE academic staff
U-Multirank Mapping Indicators
Dimension Indicator Definition
Knowledge Transfer
Income from private sources
The total amount of external research income and income from knowledge transfer from private sources as a percentage of the total income
Patent applications Filed
The number of patent applications for which the university acts as an applicant related to number of academic staff
International Orientation
Income from international sources
Income from international sources, including subsidies and contracts for teaching and research, from both public and private international sources as a percentage of total income
Foreign degree seeking students
The number of degree seeking students with a foreign diploma on entrance as % of total enrolment in degree programs.
U-Multirank Mapping Indicators
Dimension Indicator Definition
Regional Orientation
Income from regional sources
Institutional income from local/regional authorities, local/regional charities and local/regional contracts as a percentage of total institutional income
New entrants from the region
The percentage of new entrants to bachelor programs coming from the region
General
Country The country where the legal seat of the institution is located
Size of the institution
The size of the institution in terms of number of students enrolled
Online programs
The number of programs offered fully online as a percentage of the total number of programs
Legal status The public/private character of the institution
Age of institution
The year the institution or its oldest part was founded
U-Multirank Mapping Indicators
Dimension Institutional ranking
Field-based rankings
TEACHING & LEARNING
Student-staff-ratio X
Graduation rate (BA and - separately - MA) X X
Percentage of academic staff with PhD X
Percentage of students graduating within normative period (BA and –separately - MA)
X X
Rate of graduate employment X X
Inclusion of work experience in degree program X
Indicators from the student survey X
Overall learning experience X
Quality of courses & teaching X
Organisation of program X
Contact to teachers X
Social climate X
Facilities (libraries, laboratories, rooms, IT) X
Research orientation of teaching program X
Inclusion of work experience /practical elements X
U-Multirank Indicators for evaluation
Dimension Institutional ranking
Field-based rankings
RESEARCH
External research income (per fte academic staff) X X
Doctorate productivity
X
Total research publication output (per fte academic staff)*
X X
Art related output X
Field-normalised citation rate* X X
Highly cited research publications * X X
Interdisciplinary research publications* X X
Research orientation of teaching (student survey)
X
Number of post-doc positions X
External research income (per fte academic staff) X X
*Bibliometric and patent indicators will be analysed based on existing data bases
U-Multirank Indicators for evaluation
Dimension Institutional ranking
Field-based rankings
KNOWLEDGE TRANSFER
Income from private sources (service contracts, consultancies, licenses, royalties, clinical trials, etc.)
X X
Joint research publications with industry* X X
Patents (per fte academic staff)* X X
Co-patents with industry (per fte academic staff)* X X
Number of spin-offs (average over three year period) X
Patent citations to research publications* X X
Revenues from Continuous Professional
Development
X
U-Multirank Indicators for evaluation
Dimension Institutional ranking
Field-based rankings
INTERNATIONAL ORIENTATION
Educational programmes (BA, resp. MA) in foreign language X
International orientation of degree programmes
X
Opportunities to study abroad (student survey)
X
Student mobility (composite of incoming, outgoing, joint degree students)
X X
Percentage of international academic staff X X
Percentage of PhDs awarded to foreign students X X
International joint research publications* X X
International research grants X X
REGIONAL ENGAGEMENT
Percentage of graduates working in the region X X
Student internships in regional enterprises X
X
Degree theses in cooperation with regional industry
X
Regional joint research publications* X X
Income from regional sources X X
U-Multirank Indicators for evaluation
U-Multirank
Sunburst representation of an institutional performance profile
U-Multirank
Results
First results : IREG-7 Conference: Employability and Academic Rankings – Reflections and Impacts (14-16 May 2014, London, UK ) Gero Federkeil, CHE - Center for Higher Education; Manager-in-charge of Rankings, Member of U-Multirank Consortium, Vice-President of IREG Observatory, Germany
Practical information:
• Registration for participation in the 2014 edition of U-Multirank is closed.
• Pre-Registration for U-Multirank 2015,
contact form (http://u-multirank.eu/contact-us/pre-registration-2015/)
• phone:+ 49 (0) 5241 9761 58• fax:+ 49 (0) 5241 9761 40• email: [email protected]• web: www.u-multirank.eu• address: CHE Centre for Higher Education
Post Box 105, 33311 Gütersloh, Germany• Twitter: http://twitter.com/UMultirank
U-Multirank
THANK YOU
FOR ATTENTION!
U-Multirank