1
AC 21 International Forum
Competition and Cooperation among Universities in the age of Internationalization
An Analysis of Positions Mobility of Global Rankings:The Effective Use of Global Rankings in Making
Institutional Strategic Plans and Positioning for Building
World Class Universities
Dr. Angela Yung-chi HouDean of Office of Research & Development,
Higher Education Evaluation and Accreditation Council of TaiwanDirector of Faculty Development & Instructional Resources Center ,
Fu Jen Catholic University
18-21 OCT , 2010Shanghai
2
Introduction
• Globalization in the 21st century presents universities and states with a number of challenges and opportunities.
• No matter whether countries are developed or developing ones, they are immensely eager to build at least one world class university, but they don’t know exactly what they look like.
3
What does a world class university look like ?
• In terminology– world class universities are top universities striving for
“Excellence”, in other words, it means “its quality must surpass the expectation of their various stakeholders”
• Philip Altbach– excellence in research, top professors, academic freedom and an
atmosphere of intellectual excitement, governance, adequate facilities and funding.
• Jamil Salmi (World Bank) based on two rankings (Shanghai and QS) – a high concentration of talent (faculty and students)– abundant resources to offer a rich learning environment and
conduct advanced research – favorable governance (features that encourage strategic
vision, innovation and flexibility, and enable institutions to make decisions and manage resources without being encumbered by bureaucracy)
4
Relevance between global rankings and World Class University
• the characteristics of world class universities are inevitably deemed to be strongly correlated to most indicators used by global rankings.
• Many nations tend to use global rankings as a basis of building world class universities despite their well documented methodological flaws.
• Many top administrators at leading universities are learning to use global rankings wisely in order to achieve the institutional short term and long term strategic plans, not just to boycott them. – Minnesota’s initiative to become one of the top three
research institutions in the world– Taiwan National University announced the initiative of
“Moving into the top 100” at its 80th anniversary– Baylor University put the vision on making the institution
one of the U.S. News Top 50 by 2012.
5
Characteristics of 4 Major Global rankings and their methodological limitations
ARWU QS (THE)* Webmetrics HEEACT
Established year 2003 2004 2004 2007
Institution Academicinstitution
Massmedia/PrivateEducationconsulting firm
Governmental research unit
QA Agency
Goal Academiccompetition
Profit making Academic sharing Benchmarking
Number ofindicators
6 6 4 8
Indicatorcategory
Researchoutput/ learning input
Research output /reputation/learning input
Web size/research output/reputation
Research output
Data sources Database Survey/ database/institution
database Database
Outcomes Presentation
Only Top 100 of500 institutions are shown innumerical orders
Top 400 are shownin numerical orders
Top 1000 innumerical order
Top 500 innumerical order
Transparency Highly medium Medium Highly medium Highly medium
6
Methodological limitations of global rankings
• Reductionism / Simplicity
• Research focus
• Unfair for humanities, arts and social science fields
• English domination
• Arbitrary selection of indicators and weightings
7
Popular use of global rankings by stakeholders
• Students are using ranking tables in their decision-making about where to study.
• Governments are taking advantage of rankings to know where to invest
• Scientists use them to know where to work• Institutions use rankings to know where they stand and
whom they can partner with. – OECD survey in 2007 showed:
• over 50 % of respondents regarded rankings as a positive impact on the institution’s reputation and helping its development, such as student recruitment, academic partnerships and collaborations and staff morale.
• Majority of the institutions were found to incorporate the outcomes of rankings into their strategic planning processes at all levels of the organization and to take policy actions based on them.
• 70 % wanted to be in the top 25 internationally
8
Research design and method
• The main purpose is to explore the leading factors in 4 major global rankings which will most affect the rank mobility of an institution in terms of standard deviation and K mean of cluster analysis.
• a sophisticated model of strategic institutional framework for becoming a world class university is proposed
9
Major Findings
• Statistical analysis on the major indicators in 4 global rankings by correlation coefficients
• Rank differences and moving up in 4 global rankings
10
Statistical analysis on the major indicators in 4 global rankings by correlation
coefficients
11
Correlation coefficients among indicators by cluster in ARWU ranking
RankScore onAlumni
Score onAward
Score onHiCi
Score onN&S
Score on PUB
Score onPCP
1~300.812** 0.875** 0.860** 0.900** 0.319 0.728**
0.000 0.000 0.000 0.000 0.086 0.000
31~70-0.151 0.250 0.440** 0.741** 0.129 0.010
0.351 0.120 0.004 0.000 0.426 0.952
71~100
0.171 0.064 0.061 0.100 0.426* 0.235
0.366 0.738 0.747 0.599 0.019 0.211
90~110
-0.075 0.170 0.041 0.184 0.110 -0.090
0.739 0.449 0.856 0.413 0.627 0.692
1~100 0.761** 0.838** 0.871** 0.930** 0.636** 0.783**
0.000 0.000 0.000 0.000 0.000 0.000
12
Correlation coefficients among indicators by cluster in QS ranking
RankAcademic
Peer Review
Employer Review
Faculty Student
Citations per
Faculty
International Faculty
International Student
s
1~300.452* 0.201 0.629** 0.627** 0.059 0.278
0.012 0.286 0.000 0.000 0.758 0.137
31~700.318* 0.486** 0.224 0.135 -0.006 0.210
0.043 0.001 0.159 0.401 0.969 0.187
71~1000.214 -0.047 -0.158 0.221 0.051 0.031
0.266 0.810 0.413 0.249 0.792 0.874
90~110-0.123 0.281 0.206 -0.024 -0.002 0.144
0.584 0.205 0.357 0.915 0.995 0.522
1~1000.700** 0.523** 0.565** 0.363** 0.140 0.341**
0.000 0.000 0.000 0.000 0.165 0.001
13
Correlation coefficients among indicators by cluster in HEEACT ranking
Ranks
Number of
articles in
the last 11
years
Number Of
articles inthe
Currentyears
Number ofcitations inthe last 11years
Number ofcitations inthe last 2
years
NumberOf
citations in the
last11 years
H-index
Number of HighlyCitedpapers
articles in High
\impact journals
inthe
current year
1~300.825** 0.881** 0.987** 0.991** 0.482** 0.903** 0.974** 0.989**
0.000 0.000 0.000 0.000 0.007 0.000 0.000 0.000
31~700.414** 0.422** 0.679** 0.694** 0.031 0.525** 0.662** 0.495**
0.008 0.007 0.000 0.000 0.847 0.000 0.000 0.001
71~100-0.017 0.091 0.349 0.577** 0.238 0.405* 0.177 0.312
0.929 0.632 0.059 0.001 0.205 0.026 0.350 0.094
90~1100.231 0.041 0.363 0.286 0.141 0.022 0.338 0.108
0.314 0.859 0.106 0.209 0.541 0.926 0.134 0.642
1~1000.854** 0.834** 0.984** 0.988** 0.439** 0.920** 0.971** 0.977**
0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
14
Correlation coefficients among indicators by cluster in Webmetrics ranking
Rank SIZE VISIBILITY RICH SCHOLAR
1~300.807** 0.946** 0.606** 0.756**
0.000 0.000 0.000 0.000
31~700.449** 0.797** 0.595** 0.531**
0.004 0.000 0.000 0.000
71~1000.473* 0.331 0.170 0.361
0.011 0.085 0.388 0.059
90~110-0.330 0.578** -0.285 -0.004
0.143 0.006 0.210 0.987
1~1000.845** 0.949** 0.835** 0.822**
0.000 0.000 0.000 0.000
15
Rank differences and moving up
in 4 global rankings
16
ARWU-Numbers of positions moving up by clusters
ClustersNumbers of
Positionsmoving up
No ofinstitutions
%
AveragePositionsimproved
(mean)
SD (No.)
Cluster one 1-17 156 71.6% 6.51 4.65
Cluster two 18-50 55 25.2% 29.33 10.88
Cluster three 0ver 50 7 3.2% 74.71 15.76
total 218 100.0%
Highestmoving uppositions
94
17
ARWU-Numbers of positions moving up and down by indicators
0
10
20
30
40
50
60
70
80
90
100
0 100 200 300 400 500
Rank in 2008
incre
ased p
ositio
ns in 2
009
18
QS RANKINGS Numbers of positions moving up by clusters
ClustersNumbers of
ranksmoving up
No ofinstitutions
%
AverageRanks
improved(mean)
Standard deviation
Cluster one 1-30 144 84.7% 13.15 10.11
Cluster two over 30 26 15.3% 62.84 19.69
total number of moving ups 170 100.0%
Highestmoving uppositions
125
19
WEBOMETRICS: Numbers of positions moving up by clusters
ClustersNumbers of
positions moving up
No ofinstitutions
%
Average positions improved
(mean)
SD (No)
Cluster one 1-39 156 64.5% 16.21 10.90
Cluster two 40-99 76 31.4% 61.45 16.49
Cluster three
Over 100 10 4.1% 137.40 34.03
total number of moving ups 242100.0
%
Highestmoving uppositions
212
20
HEEACT Numbers of positions moving up by clusters
ClustersNumbers of
Positionsmoving up
No of institutions
%
Averagepositions increased(mean)
SD (No. )
Cluster one 1-19 153 66.2% 8.24 5.34
Cluster two 20-45 61 26.4% 30.23 7.11
Cluster three Over 46 17 7.4% 60.18 10.49
total number of moving ups 231 100.0%
Highest moving up positions 82
21
Comparison among 4 global rankings by positions rising
ARWUQS Webmetrics HEEACT
Cluster one 1-17 1-30 1-39 1-19
Cluster two 20-45 Over 30 40-99 20-45
Cluster three Over 46 X Over 100 Over 46
total number of positionsmoving ups
218(500)170 (400)
242 (500) 231(500)
Highest ranks moving up 94 125 212 82
22
Major factors for positions mobility
• Staying on the top 30: – Award of “Nobel Prize” is the most influential indicators to be on top 30 in
ARWU– “Academic peer review” in QS rankings, – ‘Internet visibility’ in Webometrics,– “Citations in the last 2 years” in HEEACT ranking.
• Moving into top 100: – HiCi, N& S and PUB are the most influential indicators in ARWU, – “Academic peer review” in QS rankings, – ‘Size’ in Webometrics, – “Citations in the last 2 years and papers” and “H-Index” in HEEACT
ranking. • Moving up positions:
– PCP, N& S and PUB are the key factors in ARWU– “Academic peer review” in QS rankings – “Visibility’ in Webometrics– H-index in HEEACT rankings
23
Flow Chart of Implication of 4 Global Ranking on Making Institutional Strategic Plans
Webometrics Ranking QS Rankings ARWU/Shanghai Ranking
HEEACT Ranking: Used to inspect the quality and quantity of FACUTLY publications annually
Short term(3-5 years) Mid-term 5-15 years Long-term(15~30years)
Technology/Internet International Reputation Academic Excellence
24
Summary
• The proposal of the strategic planning model above is completely based on the 4 global rankings, so leading factors in the 3 categories are definitely relevant to the research outputs of an institution.
• Some of these indicators will take longer time to improve, such as Nobel Laureates and Academic peer review.
• If all institutions follow the model, it’s highly likely that not all of them will actually move into the ranking in the spots they expect to be.
• it is necessary to note that these are only guidelines and not meant to be used as a rigid cause and effects.
• Academics should not to rely on a single model only to implement in terms of educational policy.
25
Conclusion
• To achieve a good rankings is becoming more and more important
• Global rankings are increasingly being used as a tool for building world class universities
26
Final question raised by the paper
• To what extent can a world class university be replicated by using the factors highlighted in a ranking model and how can it be done?
• The answer is both yes it can be replicated and no it can’t be.
27
Yes and No Answers
• The model, based on statistical analysis, can only provide very rough guidance and clues to institutions on which road to take to achieve academic excellence.
• a clear vision, institutional features, favourable governance, and sufficient resources which were not taken into consideration in the above model (or in the 4 global rankings themselves) are all very crucial if a university is to rise and stay top in the rankings.
28
“THERE IS NO SINGLE ROAD TO EXCELLENCE”
by Jamil Salmi (2010)
29
Thank you for your attention
Question and Comments
Fu Jen Catholic University
Higher Education Evaluation & Accreditation Council of Taiwan