measuring the impact of university- - cesaer · is pleased to present this report on measuring the...
TRANSCRIPT
2014 1
Executive Summary
Measuring the impact of university-business cooperation
2014 2
Europe Direct is a service to help you find answers
to your questions about the European Union
Freephone number ()
00 800 6 7 8 9 10 11
() The information given is free as are most calls (though some
operators phone boxes or hotels may charge you)
More information on the European Union is available on the Internet
(httpeuropaeu)
Luxembourg Publications Office of the European Union 2014
Catalogue Number NC-01-14-299-EN-N
Project Number 20143255
copy European Union 2014
Reproduction is authorised provided the source is acknowledged
2014 3
Measuring the Impact of
University Business Cooperation
(EAC232012)
Authors
Cardiff University Dr Adrian Healy
Imperial Consulting Dr Markus Perkmann
Newcastle University Prof John Goddard
Louise Kempton
Disclaimer
This document has been prepared for the European Commission however it reflects
the views only of the authors and the Commission cannot be held responsible for any
use which may be made of the information contained therein
2014 4
Table of Contents
Introduction 5
Types and drivers of cooperation 6
Benefits of cooperation 8
Assessing the benefits of cooperation 10
Implications for policy and practice 13
2014 5
Introduction
Cardiff University in association with Newcastle University and Imperial Consultants
is pleased to present this report on Measuring the Impact of University-Business
Cooperation to DG Education and Culture (DG EAC)
Promoting and developing cooperation between higher education and business is a
core element of the EUrsquos Agenda for Modernising Higher Education which stresses the
importance of this for pedagogical skills and the development of courses relevant to
social and labour market needs1 The potential to enhance this contribution further
through increased levels of collaboration is now firmly recognized within EU policy
circles and in Member States This is highlighted in the publication of Europe 2020 and
the related Flagship Initiative promoting a new agenda for skills and jobs which
identifies the value of ventures and networks between business and higher educational
establishments to address new skills requirements and labour market needs2
In recognition of the potential benefits to be realized through increased cooperation
between businesses and Higher Educations Institutions (HEIs) in the field of education
the European Commission launched the Knowledge Alliance pilot initiative to create
new multidisciplinary curricula to promote entrepreneurship within education as well
as developing other transferable skills This initially funded a small number of pilot
projects in 2011 and 2012 before embedding the initiative in the Lifelong Learning
Programme in 2013 From 2014 Knowledge Alliances will be part of the Erasmus+
programme
There are many other examples of university-business collaboration in the field of
education across the EU with no involvement of public sector bodies many of which
have been underway for several decades Examples cited in this study include the
Masters Course in Banking Management established in 1989 in Valencia Spain and
the course in Commercial Communications launched in Prague Czech Republic in
1994 As these examples illustrate many cooperation projects are the result of
bilateral agreements between individual companies and HEIs The focus of these
activities can vary widely depending upon the aims and objectives of each particular
example (as identified in the Wilson Review in the UK3)
In contrast to university-business cooperation in the world of research and innovation
the profile of business-university collaboration in the field of education has been
relatively limited This is unfortunate as it is at this level that knowledge exchange
can often most effectively be secured relevant skills developed and the conditions for
future innovation and economic growth laid This low profile has resulted in a limited
literature on the topic and prompted DG EAC to commission this study amongst
others The study had three specific objectives
1 High Level Group on the Modernisation of Higher Education (2013) Improving the quality of teaching and learning in Europersquos higher education institutions European Commission 2 European Commission (2010) An Agenda for new skills and jobs A European contribution towards full employment Com(2010)682 Final 3 Wilson T (2012) A Review of University Business Collaboration
2014 6
To analyse existing and identify emerging types of university-business
cooperation in Europe and their drivers
To identify demonstrate and assess tangible and intangible effects of
university-business-cooperation
To develop an assessment methodology and relevant qualitative and
quantitative indicators for measuring the outcomes and impact of university-
business cooperation and in particular of Knowledge Alliances
The study has analysed the following ten cases
Educckate multinational - led by the University College London in
partnership with other EU Universities and their local companies to
stimulate entrepreneurship in cultural and creative industries
EUEN multinational ndash led by Coventry University with transnational
academic and industry partners to promote entrepreneurial education
KnowFact multinational ndash led by the University of Patras with industry
partners from other EU countries to develop the Teaching Factory paradigm
in manufacturing education
AppCampus Finland ndash an innovative initiative between Aalto University and
NokiaMicrosoft involving app Development National
Qatar Carbonates UK ndash a collaborative arrangement between Qatar
PetroleumShell and Imperial College London promoting post-graduate
education International
Newcastle University Subsea UK ndash cooperation between Newcastle
University and regional businesses developing sector relevant skills
Regional
Masters in Banking Valencia Spain ndash collaboration between Valencia
University and regional banking bodies developing sector relevant skills
Regional
HP Bulgaria ndash Cooperation between Hewlett Packard and various
Universities whereby HP trained university lecturers in specialised
technologies and created the project curriculum National
University of Merseburg Germany ndash regional businesses endow professorial
chairs at University of Merseburg to ensure industry relevant education
AKA Prague ndash professional qualification (Certificate of Communication
Agencies) developed by industry and delivered by University of Economics
Prague
Types and drivers of cooperation
The first decade of the 21st Century has seen an increasing emphasis on the role of
universities in explicitly contributing to social and economic development This can be
attributed to a number of factors including
2014 7
The emergence of global lsquogrand challengesrsquo (eg climate change ageing
terrorism etc) which cannot be solved by government or business alone
but requires a multi-disciplinary and collaborative approach
Increased marketisation of higher education leading to greater competition
between universities and an emphasis on the lsquostudent experiencersquo and
student outcomes (including entrepreneurship)
The global economic crisis which has ushered in an era of austerity in public
finances in many countries
The recognition by many firms that working with Universities provides an
advantage for their future recruitment pipelines and ancillary profile
benefits
This has resulted in a range of policies designed to encourage universities to build
stronger links with business suggesting that businesses should be (more) involved in
designing curricula (including undergraduate postgraduate as well as CPD and other
training) and that universities should work more closely with industry partners to
promote entrepreneurship mobility (between business and academia) and lifelong
learning
In practice many universities and businesses have recognized the value of
cooperation particularly in terms of how this can strengthen student recruitment for
universities improve the educational experience for students and provide a supply of
more experienced labour for companies Some companies and universities also
recognize the wider benefits that this cooperation can bring in the form of higher
levels of innovation the upskilling and retention of existing staff and the higher profile
that successful collaboration activities can bring
A number of reports have usefully set out different lsquotypesrsquo of cooperation between
businesses and universities including student and academic mobility programmes
curriculum development and delivery lifelong learning and entrepreneurship4
However most cooperation activities examined for this work incorporated more than
one of these cooperation types This suggests that it is the purpose of cooperation
that is more important rather than the type of cooperation undertaken
Each cooperation partner (business university student) has their own reason to
engage in the cooperation activity but a number of common themes can be identified
These are illustrated in Figure ES1 Three of these relate to strengthening levels of
human capital either of students or of existing workers within companies and
universities to promote a stronger labour supply enhance individual employability
and heighten levels of entrepreneurship These were common across all the projects
examined for this study Three are more related to innovation primarily through
collaborative working on company projects as part of the educational experience or
through the co-production of new courses and course material these were visible in
each of the cases studied but to varying degrees Finally the remaining three
4 See for example Wilson T (2012) A Review of University Business Collaboration and Science-to-Business Marketing Research Centre (2011) The State of European University Business Cooperation European Commission
2014 8
reasons relate to developing longer-term contacts and the profile of the industry
companies and universities involved for future potential benefits
Figure ES1 Purposes underlying cooperation activities
Benefits of cooperation
The potential benefits of collaboration activities in the field of education can take many
different forms (Figure ES2) Very often it is the qualitative benefits that cooperation
activities bring (such as the deepening of skill sets) that are valued by participants
rather than quantitative benefits (such as increasing the numbers of graduates)
although this does also feature
Figure ES2 Four forms of outcomes
Some benefits are more immediate and occur directly following the end of the
successful completion of a course of studies (such as the number of graduates) or
during it (such as through engagement of staff and students on joint activities) whilst
others may be longer-term (such as new business starts arising out of stronger
entrepreneurial attitudes) Some benefits are direct (such as the creation of a new
Secure labour supply enhance employability
Knowledge exchangeinnov
ation Build contacts
Entrepreneurial skillsattitudes
Pedagogic development
Raise profilebrand
Staff retentionupskill
ing
Institutional modernisation
Attract students
Quantitative vs
qualitative
Immediate vs longer
term
Tangible vs Intangible
Direct vs Indirect
2014 9
course) whilst others are more indirect (such as the value of contacts gained for
potential future research projects) Equally some benefits are very tangible (such as
when a student develops a particular skill) but others are much less so (such as where
student knowledge spills over to workers in a company who work alongside them on a
project) Examples of all these potential benefits were visible throughout the cases
explored for this study
In practice our study illustrates a focus on immediate highly-visible returns by 10
university-business cooperation projects but also emphasises the significance of less
tangible and more indirect returns to participants with different participants placing
differential values on particular outcomes For example in the case of the North East
England sub-sea collaboration businesses were keen to fill an emerging skills gap
upskill staff and develop contacts with the university and other businesses For the
academics it was a means to generate additional student numbers to engage with
the business sector and to explore further research avenues
For students the reason they wished to participate in these collaborative offers in the
field of education was primarily related to the fact they felt it would provide them with
greater skills than a more conventional course that they were more likely to find
subsequent employment or that it would provide them with enhanced entrepreneurial
opportunities
For universities and academics there was a similar emphasis on the enhanced skills
acquired by students the greater employability of these students and the stronger
entrepreneurial outcomes anticipated as highlighted in the EUEN project These
outcomes were expected to feed through into an increase in either the quality or the
quantity of student applications with improvements in student perceptions of course
quality also anticipated Additional outcomes which were cited in some instances
included the opportunity provided for increasing contacts in the business world the
updating of skills through exposure to contemporary businesses practices and the
building of trust-based relationships for future collaborative research
For businesses the primary outcome reported was in terms of an enhanced pipeline of
suitably skilled potential employees together with improvements to the wider labour
force There were also particular outcomes specific to individual cooperation
arrangements such as in the case of AppCampus where one measure of success is the
number of applications developed Businesses also reported that the cooperation
arrangements provided benefits in the form of raising their profile but also helped to
build trust and wider contact networks with academics for future collaborative
research Firms also reported that involvement in live projects did provide the
opportunity for product and process innovations adding economic value to the
company and that the bringing together of students and existing employees served to
raise the knowledge levels of employees as well as students
For other parties such as regional authorities the anticipated (and realised) outcomes
largely revolved around economic benefits such as an enhanced labour supply
increased levels of graduate employment higher levels of new business-starts or
(rarely) higher levels of innovation
2014 10
Assessing the benefits of cooperation
Currently there is very little guidance on approaches to assessing the benefits of
university-business cooperation activities in the field of education The existing
literature is strongly inclined to cooperation in the field of research and innovation
What examples can be found tend to stress measures of inputs and student outcomes
The wider literature can however provide valuable insights This incudes highlighting
the value of less tangible benefits such as contact between students company staff
and university staff often termed as productive interactions
Monitoring and assessment has a number of potential benefits it provides a means to
assess whether a project or programme is performing as expected it enables a
project leader to identify what works and what does not it provides a useful
mechanism to feedback lessons to others and if undertaken in advance it can
provide a means for selecting between different potential options In the evidence
available to this project the main approaches to monitoring and assessment have
focused on project performance often with an emphasis on measures of inputs and
outputs However there is also evidence of the use of results-based formats often of
a qualitative nature
The evidence available from the cases explored in this work suggests that monitoring
activities are undertaken at two levels in the field of university-business collaboration
in the field of education
Firstly through the use of standard student-orientated metrics
Secondly through informal metrics based around joint-perceptions of
success
The cases also suggest that there are a range of outcomes of university-business
collaboration activities in the field of education that are not currently being captured
by existing monitoring and measurement arrangements such as the potential
upskilling of existing staff through their involvement in the educational experience or
the raising of the profile of participating institutions However in no case was this felt
to weaken the cooperation arrangements underway Indeed to the contrary the
suggestion was that widening the formal measurement and monitoring process to
include more indicators would incur an additional burden for limited gains It is
essential that assessment methods are not only fit for purpose but are also seen to be
so
Whilst there are many methods that can be used to assess the outcomes of a
particular cooperation activity (whether that is a project or a programme) the
balanced scorecard approach seems particularly relevant The central idea of the
balanced scorecard is that several metrics can be used to evaluate the process in
question A causal map for educational university-industry collaboration initiatives is
given in Figure ES3 Four areas of measurement form the essential components of a
scorecard for university-industry collaboration The people involved the resources
available the activities undertaken and the results achieved
2014 11
Figure ES3 Scorecard general structure
Although each participant may have their own scorecard for the cooperation (either
held explicitly or implicitly) a combined scorecard can be drawn up for each co-
operation project in the form of a logical framework This allows assumptions to be
made explicit expected relationships to be identified and tests whether the
cooperation planned can meet its intended purpose(s) A stylized logic chain is set out
in Figure ES4
Figure ES4 Stylised causal map for university-industry collaboration
Alongside the scorecard approach an assessment methodology may also wish to set
out the framework for monitoring and evaluation Where evaluation may relate to the
assessment of project applications or the success of a completed project Here a
simple framework such as the success map set out in Figure ES5 may be
advantageous
People
bullThe qualification fit and motivation of the people taking part in the cooperation
Benefits
bullThe beneficial outputs that are generated as a result of the collaboration
Resources
bullThe amount and quality of the resources dedicated to the collaboration
Activities
bullThe quality and appropriateness of processes and organizational set-up
People
bullInstructors students managers
Resources
bullFunding equipment technology
Activities
bullOrganizational processes and structures
Benefits
bullEducation skills employment
2014 12
Figure ES5 Success map for cooperation in the field of education
In developing such a success map due regard should be given to four purposes of
university-business cooperation in the field of education broadly stated
Those that relate to teaching and learning aimed at strengthening the
labour supply and employment
Those that relate to stimulating entrepreneurship through teaching and
learning
Those that relate to knowledge exchange aimed at promoting
productprocess innovation or new pedagogy
Those that relate to raising the profile of an organization
In seeking to assess an actual project the framework can ensure that appropriate
indicators are selected and measurement approaches adopted It is beyond the scope
of this report to suggest a standard set of indicators as this will vary by project
purpose However a range of potential indicators have been identified relating to
each of the above purposes and which might be used to monitor inputs activities
outputs and outcomes Examples of potential indicators are illustrated in Figure ES5
In selecting indicators a number of principles should be observed including those of
parsimony (to seek to limit the data burden) and proportionality (data collected should
not incur costs disproportionate to the benefits received) The collection of data does
incur costs and for an assessment process to operate successfully there must be
agreement of all parties as to the role of particular indicators and the value of their
measurement
Interaction opportunities
Access to resources
Motivated educators
High-quality students
Relevant course delivery
Relevant course design
New ideas developed
Skilled amp trained labour
New pedagogies
Student applicant numbersquality
Non-academic organisations involved
Time inputs
Finance inputs
Newimproved courses
Number of graduates
Student attainment
Entrepreneurial attitudes
New products processes developed
Joint objective setting
Internshipsplacements
Co-delivery and assessment
Interaction intensity
Graduate employment
New business starts
Employer satisfaction
Organisational ranking in lsquoleaguersquo tables
Successful alliance
Human capital
Entrepreneurship
New ideas
Profile raising
Input Activity Output Outcome
2014 13
In the selection of indicators full consideration should also be given to the role of
qualitative measures of success and the potential means of capturing these This
might be through survey techniques but can also be captured by structured review
meetings with inputs from the various parties The value of self-assessment
techniques should also be considered Quantitative measures should not be over-
looked and where these can be part of wider measurement processes should be
welcomed
Implications for policy and practice
This study has proposed a flexible approach to measuring the impact of university-
business cooperation activities using a balanced scorecard approach allied with a
success-map for cooperation activities This recognizes the multiple purposes that
underlie cooperation activities and acknowledges that a one-size-fits-all approach is
unlikely to prove fit for purpose What are the benefits and implications of the
approach proposed to policy-makers and to the collaboration partners themselves
For policy-makers the tool can be used in a variety of manners
In the first instance it forms a framework for the ex-ante development of
the proposed programme By identifying the resources that are available to
the programme or might be available and the desired objectives
programme officials can identify the types of activity that might lead to the
desired results Through a process of iteration the scorecard can be refined
further This can then provide a baseline for a call for applications
The scorecard approach might then be used for the assessment or
appraisal of applications Assessors can check the applications against the
programme scorecard to identify the extent to which each applicant
contributes to the desired approach and objectives In a dynamic approach
the programme scorecard can be updated through the inclusion of novel
and innovative approaches proposed by applicants which were not foreseen
during the development of the programme A blank scorecard template
might also be completed by assessors as a means of summarising the
information contained in applications in a common format The approach
could highlight missed opportunities for collaborative activities or
resourcepeople inputs which can form the basis of negotiation with
programme applicants
Following a successful application a project scorecard can then form the
basis for the review of project progress This can examine the activities
undertaken resources expended and people involved against what was
initially planned together with the benefits realized compared to those that
were expected The review meeting can then explore the reasons for any
differences examining cases where inputs are greater than anticipated
perhaps owing to the cooperation going better than planned or where
results or inputs are less than originally proposed and whether remedial
action needs to be taken
2014 14
The scorecard approach can also deliver significant benefits to the collaboration
partners themselves
As a starting point it can form the basis for robust project design The
scorecard can be used as a mechanism for each partner to set out what
they wish to achieve from the cooperation activity and what they are able
to contribute Each partner can then identify what they expect of the other
partners Using the scorecard approach can also assist in identifying where
potential opportunities might have been overlooked in the original project
design and where small changes might reap large rewards This can then
form the basis of the collaboration agreement between parties
Project partners can then use their own stakeholder scorecard as a means
for internal review and monitoring of the collaboration activity This
provides a useful mechanism for identifying whether desired outcomes are
being achieved and the extent to which anticipated inputs and activities are
occurring As a dynamic process the scorecard can be updated as
activities are added or amended or other inputs change
The partnership as a whole can also use the scorecard as the basis for
ongoing project reviews This can consider whether the original
assumptions remain valid and whether some activities are more appropriate
than others in achieving the desired objectives Crucial to the review
process is to ensure that all partners are realizing the benefits that they
sought initially and if not how this might be rectified The scorecard
approach provides a means of structuring this review process
In order to promote the use of the scorecard approach the Commission may wish to
provide a guide to its use in the context of university-business cooperation activities
This would offer the opportunity to develop worked examples tailored to different
cooperation purposes Such a guide could also incorporate advice on practical
techniques for measuring the outcomes of cooperation activities In the first instance
this could form part of a toolkit available to assist those parties interested in applying
for Knowledge Alliance projects
HOW TO OBTAIN EU PUBLICATIONS
Free publications
bull one copy
via EU Bookshop (httpbookshopeuropaeu)
bull more than one copy or postersmaps
from the European Unionrsquos representations (httpeceuropaeurepresent_enhtm)
from the delegations in non-EU countries (httpeeaseuropaeudelegationsindex_enhtm)
by contacting the Europe Direct service (httpeuropaeueuropedirectindex_enhtm) or
calling 00 800 6 7 8 9 10 11 (freephone number from anywhere in the EU) ()
() The information given is free as are most calls (though some operators phone boxes
or hotels may charge you)
Priced publications
bull via EU Bookshop (httpbookshopeuropaeu)
Priced subscriptions
bull via one of the sales agents of the Publications Office of the European Union
(httppublicationseuropaeuothersagentsindex_enhtm)
NC-0
1-1
4-2
99-E
N-N
doi10
2014 2
Europe Direct is a service to help you find answers
to your questions about the European Union
Freephone number ()
00 800 6 7 8 9 10 11
() The information given is free as are most calls (though some
operators phone boxes or hotels may charge you)
More information on the European Union is available on the Internet
(httpeuropaeu)
Luxembourg Publications Office of the European Union 2014
Catalogue Number NC-01-14-299-EN-N
Project Number 20143255
copy European Union 2014
Reproduction is authorised provided the source is acknowledged
2014 3
Measuring the Impact of
University Business Cooperation
(EAC232012)
Authors
Cardiff University Dr Adrian Healy
Imperial Consulting Dr Markus Perkmann
Newcastle University Prof John Goddard
Louise Kempton
Disclaimer
This document has been prepared for the European Commission however it reflects
the views only of the authors and the Commission cannot be held responsible for any
use which may be made of the information contained therein
2014 4
Table of Contents
Introduction 5
Types and drivers of cooperation 6
Benefits of cooperation 8
Assessing the benefits of cooperation 10
Implications for policy and practice 13
2014 5
Introduction
Cardiff University in association with Newcastle University and Imperial Consultants
is pleased to present this report on Measuring the Impact of University-Business
Cooperation to DG Education and Culture (DG EAC)
Promoting and developing cooperation between higher education and business is a
core element of the EUrsquos Agenda for Modernising Higher Education which stresses the
importance of this for pedagogical skills and the development of courses relevant to
social and labour market needs1 The potential to enhance this contribution further
through increased levels of collaboration is now firmly recognized within EU policy
circles and in Member States This is highlighted in the publication of Europe 2020 and
the related Flagship Initiative promoting a new agenda for skills and jobs which
identifies the value of ventures and networks between business and higher educational
establishments to address new skills requirements and labour market needs2
In recognition of the potential benefits to be realized through increased cooperation
between businesses and Higher Educations Institutions (HEIs) in the field of education
the European Commission launched the Knowledge Alliance pilot initiative to create
new multidisciplinary curricula to promote entrepreneurship within education as well
as developing other transferable skills This initially funded a small number of pilot
projects in 2011 and 2012 before embedding the initiative in the Lifelong Learning
Programme in 2013 From 2014 Knowledge Alliances will be part of the Erasmus+
programme
There are many other examples of university-business collaboration in the field of
education across the EU with no involvement of public sector bodies many of which
have been underway for several decades Examples cited in this study include the
Masters Course in Banking Management established in 1989 in Valencia Spain and
the course in Commercial Communications launched in Prague Czech Republic in
1994 As these examples illustrate many cooperation projects are the result of
bilateral agreements between individual companies and HEIs The focus of these
activities can vary widely depending upon the aims and objectives of each particular
example (as identified in the Wilson Review in the UK3)
In contrast to university-business cooperation in the world of research and innovation
the profile of business-university collaboration in the field of education has been
relatively limited This is unfortunate as it is at this level that knowledge exchange
can often most effectively be secured relevant skills developed and the conditions for
future innovation and economic growth laid This low profile has resulted in a limited
literature on the topic and prompted DG EAC to commission this study amongst
others The study had three specific objectives
1 High Level Group on the Modernisation of Higher Education (2013) Improving the quality of teaching and learning in Europersquos higher education institutions European Commission 2 European Commission (2010) An Agenda for new skills and jobs A European contribution towards full employment Com(2010)682 Final 3 Wilson T (2012) A Review of University Business Collaboration
2014 6
To analyse existing and identify emerging types of university-business
cooperation in Europe and their drivers
To identify demonstrate and assess tangible and intangible effects of
university-business-cooperation
To develop an assessment methodology and relevant qualitative and
quantitative indicators for measuring the outcomes and impact of university-
business cooperation and in particular of Knowledge Alliances
The study has analysed the following ten cases
Educckate multinational - led by the University College London in
partnership with other EU Universities and their local companies to
stimulate entrepreneurship in cultural and creative industries
EUEN multinational ndash led by Coventry University with transnational
academic and industry partners to promote entrepreneurial education
KnowFact multinational ndash led by the University of Patras with industry
partners from other EU countries to develop the Teaching Factory paradigm
in manufacturing education
AppCampus Finland ndash an innovative initiative between Aalto University and
NokiaMicrosoft involving app Development National
Qatar Carbonates UK ndash a collaborative arrangement between Qatar
PetroleumShell and Imperial College London promoting post-graduate
education International
Newcastle University Subsea UK ndash cooperation between Newcastle
University and regional businesses developing sector relevant skills
Regional
Masters in Banking Valencia Spain ndash collaboration between Valencia
University and regional banking bodies developing sector relevant skills
Regional
HP Bulgaria ndash Cooperation between Hewlett Packard and various
Universities whereby HP trained university lecturers in specialised
technologies and created the project curriculum National
University of Merseburg Germany ndash regional businesses endow professorial
chairs at University of Merseburg to ensure industry relevant education
AKA Prague ndash professional qualification (Certificate of Communication
Agencies) developed by industry and delivered by University of Economics
Prague
Types and drivers of cooperation
The first decade of the 21st Century has seen an increasing emphasis on the role of
universities in explicitly contributing to social and economic development This can be
attributed to a number of factors including
2014 7
The emergence of global lsquogrand challengesrsquo (eg climate change ageing
terrorism etc) which cannot be solved by government or business alone
but requires a multi-disciplinary and collaborative approach
Increased marketisation of higher education leading to greater competition
between universities and an emphasis on the lsquostudent experiencersquo and
student outcomes (including entrepreneurship)
The global economic crisis which has ushered in an era of austerity in public
finances in many countries
The recognition by many firms that working with Universities provides an
advantage for their future recruitment pipelines and ancillary profile
benefits
This has resulted in a range of policies designed to encourage universities to build
stronger links with business suggesting that businesses should be (more) involved in
designing curricula (including undergraduate postgraduate as well as CPD and other
training) and that universities should work more closely with industry partners to
promote entrepreneurship mobility (between business and academia) and lifelong
learning
In practice many universities and businesses have recognized the value of
cooperation particularly in terms of how this can strengthen student recruitment for
universities improve the educational experience for students and provide a supply of
more experienced labour for companies Some companies and universities also
recognize the wider benefits that this cooperation can bring in the form of higher
levels of innovation the upskilling and retention of existing staff and the higher profile
that successful collaboration activities can bring
A number of reports have usefully set out different lsquotypesrsquo of cooperation between
businesses and universities including student and academic mobility programmes
curriculum development and delivery lifelong learning and entrepreneurship4
However most cooperation activities examined for this work incorporated more than
one of these cooperation types This suggests that it is the purpose of cooperation
that is more important rather than the type of cooperation undertaken
Each cooperation partner (business university student) has their own reason to
engage in the cooperation activity but a number of common themes can be identified
These are illustrated in Figure ES1 Three of these relate to strengthening levels of
human capital either of students or of existing workers within companies and
universities to promote a stronger labour supply enhance individual employability
and heighten levels of entrepreneurship These were common across all the projects
examined for this study Three are more related to innovation primarily through
collaborative working on company projects as part of the educational experience or
through the co-production of new courses and course material these were visible in
each of the cases studied but to varying degrees Finally the remaining three
4 See for example Wilson T (2012) A Review of University Business Collaboration and Science-to-Business Marketing Research Centre (2011) The State of European University Business Cooperation European Commission
2014 8
reasons relate to developing longer-term contacts and the profile of the industry
companies and universities involved for future potential benefits
Figure ES1 Purposes underlying cooperation activities
Benefits of cooperation
The potential benefits of collaboration activities in the field of education can take many
different forms (Figure ES2) Very often it is the qualitative benefits that cooperation
activities bring (such as the deepening of skill sets) that are valued by participants
rather than quantitative benefits (such as increasing the numbers of graduates)
although this does also feature
Figure ES2 Four forms of outcomes
Some benefits are more immediate and occur directly following the end of the
successful completion of a course of studies (such as the number of graduates) or
during it (such as through engagement of staff and students on joint activities) whilst
others may be longer-term (such as new business starts arising out of stronger
entrepreneurial attitudes) Some benefits are direct (such as the creation of a new
Secure labour supply enhance employability
Knowledge exchangeinnov
ation Build contacts
Entrepreneurial skillsattitudes
Pedagogic development
Raise profilebrand
Staff retentionupskill
ing
Institutional modernisation
Attract students
Quantitative vs
qualitative
Immediate vs longer
term
Tangible vs Intangible
Direct vs Indirect
2014 9
course) whilst others are more indirect (such as the value of contacts gained for
potential future research projects) Equally some benefits are very tangible (such as
when a student develops a particular skill) but others are much less so (such as where
student knowledge spills over to workers in a company who work alongside them on a
project) Examples of all these potential benefits were visible throughout the cases
explored for this study
In practice our study illustrates a focus on immediate highly-visible returns by 10
university-business cooperation projects but also emphasises the significance of less
tangible and more indirect returns to participants with different participants placing
differential values on particular outcomes For example in the case of the North East
England sub-sea collaboration businesses were keen to fill an emerging skills gap
upskill staff and develop contacts with the university and other businesses For the
academics it was a means to generate additional student numbers to engage with
the business sector and to explore further research avenues
For students the reason they wished to participate in these collaborative offers in the
field of education was primarily related to the fact they felt it would provide them with
greater skills than a more conventional course that they were more likely to find
subsequent employment or that it would provide them with enhanced entrepreneurial
opportunities
For universities and academics there was a similar emphasis on the enhanced skills
acquired by students the greater employability of these students and the stronger
entrepreneurial outcomes anticipated as highlighted in the EUEN project These
outcomes were expected to feed through into an increase in either the quality or the
quantity of student applications with improvements in student perceptions of course
quality also anticipated Additional outcomes which were cited in some instances
included the opportunity provided for increasing contacts in the business world the
updating of skills through exposure to contemporary businesses practices and the
building of trust-based relationships for future collaborative research
For businesses the primary outcome reported was in terms of an enhanced pipeline of
suitably skilled potential employees together with improvements to the wider labour
force There were also particular outcomes specific to individual cooperation
arrangements such as in the case of AppCampus where one measure of success is the
number of applications developed Businesses also reported that the cooperation
arrangements provided benefits in the form of raising their profile but also helped to
build trust and wider contact networks with academics for future collaborative
research Firms also reported that involvement in live projects did provide the
opportunity for product and process innovations adding economic value to the
company and that the bringing together of students and existing employees served to
raise the knowledge levels of employees as well as students
For other parties such as regional authorities the anticipated (and realised) outcomes
largely revolved around economic benefits such as an enhanced labour supply
increased levels of graduate employment higher levels of new business-starts or
(rarely) higher levels of innovation
2014 10
Assessing the benefits of cooperation
Currently there is very little guidance on approaches to assessing the benefits of
university-business cooperation activities in the field of education The existing
literature is strongly inclined to cooperation in the field of research and innovation
What examples can be found tend to stress measures of inputs and student outcomes
The wider literature can however provide valuable insights This incudes highlighting
the value of less tangible benefits such as contact between students company staff
and university staff often termed as productive interactions
Monitoring and assessment has a number of potential benefits it provides a means to
assess whether a project or programme is performing as expected it enables a
project leader to identify what works and what does not it provides a useful
mechanism to feedback lessons to others and if undertaken in advance it can
provide a means for selecting between different potential options In the evidence
available to this project the main approaches to monitoring and assessment have
focused on project performance often with an emphasis on measures of inputs and
outputs However there is also evidence of the use of results-based formats often of
a qualitative nature
The evidence available from the cases explored in this work suggests that monitoring
activities are undertaken at two levels in the field of university-business collaboration
in the field of education
Firstly through the use of standard student-orientated metrics
Secondly through informal metrics based around joint-perceptions of
success
The cases also suggest that there are a range of outcomes of university-business
collaboration activities in the field of education that are not currently being captured
by existing monitoring and measurement arrangements such as the potential
upskilling of existing staff through their involvement in the educational experience or
the raising of the profile of participating institutions However in no case was this felt
to weaken the cooperation arrangements underway Indeed to the contrary the
suggestion was that widening the formal measurement and monitoring process to
include more indicators would incur an additional burden for limited gains It is
essential that assessment methods are not only fit for purpose but are also seen to be
so
Whilst there are many methods that can be used to assess the outcomes of a
particular cooperation activity (whether that is a project or a programme) the
balanced scorecard approach seems particularly relevant The central idea of the
balanced scorecard is that several metrics can be used to evaluate the process in
question A causal map for educational university-industry collaboration initiatives is
given in Figure ES3 Four areas of measurement form the essential components of a
scorecard for university-industry collaboration The people involved the resources
available the activities undertaken and the results achieved
2014 11
Figure ES3 Scorecard general structure
Although each participant may have their own scorecard for the cooperation (either
held explicitly or implicitly) a combined scorecard can be drawn up for each co-
operation project in the form of a logical framework This allows assumptions to be
made explicit expected relationships to be identified and tests whether the
cooperation planned can meet its intended purpose(s) A stylized logic chain is set out
in Figure ES4
Figure ES4 Stylised causal map for university-industry collaboration
Alongside the scorecard approach an assessment methodology may also wish to set
out the framework for monitoring and evaluation Where evaluation may relate to the
assessment of project applications or the success of a completed project Here a
simple framework such as the success map set out in Figure ES5 may be
advantageous
People
bullThe qualification fit and motivation of the people taking part in the cooperation
Benefits
bullThe beneficial outputs that are generated as a result of the collaboration
Resources
bullThe amount and quality of the resources dedicated to the collaboration
Activities
bullThe quality and appropriateness of processes and organizational set-up
People
bullInstructors students managers
Resources
bullFunding equipment technology
Activities
bullOrganizational processes and structures
Benefits
bullEducation skills employment
2014 12
Figure ES5 Success map for cooperation in the field of education
In developing such a success map due regard should be given to four purposes of
university-business cooperation in the field of education broadly stated
Those that relate to teaching and learning aimed at strengthening the
labour supply and employment
Those that relate to stimulating entrepreneurship through teaching and
learning
Those that relate to knowledge exchange aimed at promoting
productprocess innovation or new pedagogy
Those that relate to raising the profile of an organization
In seeking to assess an actual project the framework can ensure that appropriate
indicators are selected and measurement approaches adopted It is beyond the scope
of this report to suggest a standard set of indicators as this will vary by project
purpose However a range of potential indicators have been identified relating to
each of the above purposes and which might be used to monitor inputs activities
outputs and outcomes Examples of potential indicators are illustrated in Figure ES5
In selecting indicators a number of principles should be observed including those of
parsimony (to seek to limit the data burden) and proportionality (data collected should
not incur costs disproportionate to the benefits received) The collection of data does
incur costs and for an assessment process to operate successfully there must be
agreement of all parties as to the role of particular indicators and the value of their
measurement
Interaction opportunities
Access to resources
Motivated educators
High-quality students
Relevant course delivery
Relevant course design
New ideas developed
Skilled amp trained labour
New pedagogies
Student applicant numbersquality
Non-academic organisations involved
Time inputs
Finance inputs
Newimproved courses
Number of graduates
Student attainment
Entrepreneurial attitudes
New products processes developed
Joint objective setting
Internshipsplacements
Co-delivery and assessment
Interaction intensity
Graduate employment
New business starts
Employer satisfaction
Organisational ranking in lsquoleaguersquo tables
Successful alliance
Human capital
Entrepreneurship
New ideas
Profile raising
Input Activity Output Outcome
2014 13
In the selection of indicators full consideration should also be given to the role of
qualitative measures of success and the potential means of capturing these This
might be through survey techniques but can also be captured by structured review
meetings with inputs from the various parties The value of self-assessment
techniques should also be considered Quantitative measures should not be over-
looked and where these can be part of wider measurement processes should be
welcomed
Implications for policy and practice
This study has proposed a flexible approach to measuring the impact of university-
business cooperation activities using a balanced scorecard approach allied with a
success-map for cooperation activities This recognizes the multiple purposes that
underlie cooperation activities and acknowledges that a one-size-fits-all approach is
unlikely to prove fit for purpose What are the benefits and implications of the
approach proposed to policy-makers and to the collaboration partners themselves
For policy-makers the tool can be used in a variety of manners
In the first instance it forms a framework for the ex-ante development of
the proposed programme By identifying the resources that are available to
the programme or might be available and the desired objectives
programme officials can identify the types of activity that might lead to the
desired results Through a process of iteration the scorecard can be refined
further This can then provide a baseline for a call for applications
The scorecard approach might then be used for the assessment or
appraisal of applications Assessors can check the applications against the
programme scorecard to identify the extent to which each applicant
contributes to the desired approach and objectives In a dynamic approach
the programme scorecard can be updated through the inclusion of novel
and innovative approaches proposed by applicants which were not foreseen
during the development of the programme A blank scorecard template
might also be completed by assessors as a means of summarising the
information contained in applications in a common format The approach
could highlight missed opportunities for collaborative activities or
resourcepeople inputs which can form the basis of negotiation with
programme applicants
Following a successful application a project scorecard can then form the
basis for the review of project progress This can examine the activities
undertaken resources expended and people involved against what was
initially planned together with the benefits realized compared to those that
were expected The review meeting can then explore the reasons for any
differences examining cases where inputs are greater than anticipated
perhaps owing to the cooperation going better than planned or where
results or inputs are less than originally proposed and whether remedial
action needs to be taken
2014 14
The scorecard approach can also deliver significant benefits to the collaboration
partners themselves
As a starting point it can form the basis for robust project design The
scorecard can be used as a mechanism for each partner to set out what
they wish to achieve from the cooperation activity and what they are able
to contribute Each partner can then identify what they expect of the other
partners Using the scorecard approach can also assist in identifying where
potential opportunities might have been overlooked in the original project
design and where small changes might reap large rewards This can then
form the basis of the collaboration agreement between parties
Project partners can then use their own stakeholder scorecard as a means
for internal review and monitoring of the collaboration activity This
provides a useful mechanism for identifying whether desired outcomes are
being achieved and the extent to which anticipated inputs and activities are
occurring As a dynamic process the scorecard can be updated as
activities are added or amended or other inputs change
The partnership as a whole can also use the scorecard as the basis for
ongoing project reviews This can consider whether the original
assumptions remain valid and whether some activities are more appropriate
than others in achieving the desired objectives Crucial to the review
process is to ensure that all partners are realizing the benefits that they
sought initially and if not how this might be rectified The scorecard
approach provides a means of structuring this review process
In order to promote the use of the scorecard approach the Commission may wish to
provide a guide to its use in the context of university-business cooperation activities
This would offer the opportunity to develop worked examples tailored to different
cooperation purposes Such a guide could also incorporate advice on practical
techniques for measuring the outcomes of cooperation activities In the first instance
this could form part of a toolkit available to assist those parties interested in applying
for Knowledge Alliance projects
HOW TO OBTAIN EU PUBLICATIONS
Free publications
bull one copy
via EU Bookshop (httpbookshopeuropaeu)
bull more than one copy or postersmaps
from the European Unionrsquos representations (httpeceuropaeurepresent_enhtm)
from the delegations in non-EU countries (httpeeaseuropaeudelegationsindex_enhtm)
by contacting the Europe Direct service (httpeuropaeueuropedirectindex_enhtm) or
calling 00 800 6 7 8 9 10 11 (freephone number from anywhere in the EU) ()
() The information given is free as are most calls (though some operators phone boxes
or hotels may charge you)
Priced publications
bull via EU Bookshop (httpbookshopeuropaeu)
Priced subscriptions
bull via one of the sales agents of the Publications Office of the European Union
(httppublicationseuropaeuothersagentsindex_enhtm)
NC-0
1-1
4-2
99-E
N-N
doi10
2014 3
Measuring the Impact of
University Business Cooperation
(EAC232012)
Authors
Cardiff University Dr Adrian Healy
Imperial Consulting Dr Markus Perkmann
Newcastle University Prof John Goddard
Louise Kempton
Disclaimer
This document has been prepared for the European Commission however it reflects
the views only of the authors and the Commission cannot be held responsible for any
use which may be made of the information contained therein
2014 4
Table of Contents
Introduction 5
Types and drivers of cooperation 6
Benefits of cooperation 8
Assessing the benefits of cooperation 10
Implications for policy and practice 13
2014 5
Introduction
Cardiff University in association with Newcastle University and Imperial Consultants
is pleased to present this report on Measuring the Impact of University-Business
Cooperation to DG Education and Culture (DG EAC)
Promoting and developing cooperation between higher education and business is a
core element of the EUrsquos Agenda for Modernising Higher Education which stresses the
importance of this for pedagogical skills and the development of courses relevant to
social and labour market needs1 The potential to enhance this contribution further
through increased levels of collaboration is now firmly recognized within EU policy
circles and in Member States This is highlighted in the publication of Europe 2020 and
the related Flagship Initiative promoting a new agenda for skills and jobs which
identifies the value of ventures and networks between business and higher educational
establishments to address new skills requirements and labour market needs2
In recognition of the potential benefits to be realized through increased cooperation
between businesses and Higher Educations Institutions (HEIs) in the field of education
the European Commission launched the Knowledge Alliance pilot initiative to create
new multidisciplinary curricula to promote entrepreneurship within education as well
as developing other transferable skills This initially funded a small number of pilot
projects in 2011 and 2012 before embedding the initiative in the Lifelong Learning
Programme in 2013 From 2014 Knowledge Alliances will be part of the Erasmus+
programme
There are many other examples of university-business collaboration in the field of
education across the EU with no involvement of public sector bodies many of which
have been underway for several decades Examples cited in this study include the
Masters Course in Banking Management established in 1989 in Valencia Spain and
the course in Commercial Communications launched in Prague Czech Republic in
1994 As these examples illustrate many cooperation projects are the result of
bilateral agreements between individual companies and HEIs The focus of these
activities can vary widely depending upon the aims and objectives of each particular
example (as identified in the Wilson Review in the UK3)
In contrast to university-business cooperation in the world of research and innovation
the profile of business-university collaboration in the field of education has been
relatively limited This is unfortunate as it is at this level that knowledge exchange
can often most effectively be secured relevant skills developed and the conditions for
future innovation and economic growth laid This low profile has resulted in a limited
literature on the topic and prompted DG EAC to commission this study amongst
others The study had three specific objectives
1 High Level Group on the Modernisation of Higher Education (2013) Improving the quality of teaching and learning in Europersquos higher education institutions European Commission 2 European Commission (2010) An Agenda for new skills and jobs A European contribution towards full employment Com(2010)682 Final 3 Wilson T (2012) A Review of University Business Collaboration
2014 6
To analyse existing and identify emerging types of university-business
cooperation in Europe and their drivers
To identify demonstrate and assess tangible and intangible effects of
university-business-cooperation
To develop an assessment methodology and relevant qualitative and
quantitative indicators for measuring the outcomes and impact of university-
business cooperation and in particular of Knowledge Alliances
The study has analysed the following ten cases
Educckate multinational - led by the University College London in
partnership with other EU Universities and their local companies to
stimulate entrepreneurship in cultural and creative industries
EUEN multinational ndash led by Coventry University with transnational
academic and industry partners to promote entrepreneurial education
KnowFact multinational ndash led by the University of Patras with industry
partners from other EU countries to develop the Teaching Factory paradigm
in manufacturing education
AppCampus Finland ndash an innovative initiative between Aalto University and
NokiaMicrosoft involving app Development National
Qatar Carbonates UK ndash a collaborative arrangement between Qatar
PetroleumShell and Imperial College London promoting post-graduate
education International
Newcastle University Subsea UK ndash cooperation between Newcastle
University and regional businesses developing sector relevant skills
Regional
Masters in Banking Valencia Spain ndash collaboration between Valencia
University and regional banking bodies developing sector relevant skills
Regional
HP Bulgaria ndash Cooperation between Hewlett Packard and various
Universities whereby HP trained university lecturers in specialised
technologies and created the project curriculum National
University of Merseburg Germany ndash regional businesses endow professorial
chairs at University of Merseburg to ensure industry relevant education
AKA Prague ndash professional qualification (Certificate of Communication
Agencies) developed by industry and delivered by University of Economics
Prague
Types and drivers of cooperation
The first decade of the 21st Century has seen an increasing emphasis on the role of
universities in explicitly contributing to social and economic development This can be
attributed to a number of factors including
2014 7
The emergence of global lsquogrand challengesrsquo (eg climate change ageing
terrorism etc) which cannot be solved by government or business alone
but requires a multi-disciplinary and collaborative approach
Increased marketisation of higher education leading to greater competition
between universities and an emphasis on the lsquostudent experiencersquo and
student outcomes (including entrepreneurship)
The global economic crisis which has ushered in an era of austerity in public
finances in many countries
The recognition by many firms that working with Universities provides an
advantage for their future recruitment pipelines and ancillary profile
benefits
This has resulted in a range of policies designed to encourage universities to build
stronger links with business suggesting that businesses should be (more) involved in
designing curricula (including undergraduate postgraduate as well as CPD and other
training) and that universities should work more closely with industry partners to
promote entrepreneurship mobility (between business and academia) and lifelong
learning
In practice many universities and businesses have recognized the value of
cooperation particularly in terms of how this can strengthen student recruitment for
universities improve the educational experience for students and provide a supply of
more experienced labour for companies Some companies and universities also
recognize the wider benefits that this cooperation can bring in the form of higher
levels of innovation the upskilling and retention of existing staff and the higher profile
that successful collaboration activities can bring
A number of reports have usefully set out different lsquotypesrsquo of cooperation between
businesses and universities including student and academic mobility programmes
curriculum development and delivery lifelong learning and entrepreneurship4
However most cooperation activities examined for this work incorporated more than
one of these cooperation types This suggests that it is the purpose of cooperation
that is more important rather than the type of cooperation undertaken
Each cooperation partner (business university student) has their own reason to
engage in the cooperation activity but a number of common themes can be identified
These are illustrated in Figure ES1 Three of these relate to strengthening levels of
human capital either of students or of existing workers within companies and
universities to promote a stronger labour supply enhance individual employability
and heighten levels of entrepreneurship These were common across all the projects
examined for this study Three are more related to innovation primarily through
collaborative working on company projects as part of the educational experience or
through the co-production of new courses and course material these were visible in
each of the cases studied but to varying degrees Finally the remaining three
4 See for example Wilson T (2012) A Review of University Business Collaboration and Science-to-Business Marketing Research Centre (2011) The State of European University Business Cooperation European Commission
2014 8
reasons relate to developing longer-term contacts and the profile of the industry
companies and universities involved for future potential benefits
Figure ES1 Purposes underlying cooperation activities
Benefits of cooperation
The potential benefits of collaboration activities in the field of education can take many
different forms (Figure ES2) Very often it is the qualitative benefits that cooperation
activities bring (such as the deepening of skill sets) that are valued by participants
rather than quantitative benefits (such as increasing the numbers of graduates)
although this does also feature
Figure ES2 Four forms of outcomes
Some benefits are more immediate and occur directly following the end of the
successful completion of a course of studies (such as the number of graduates) or
during it (such as through engagement of staff and students on joint activities) whilst
others may be longer-term (such as new business starts arising out of stronger
entrepreneurial attitudes) Some benefits are direct (such as the creation of a new
Secure labour supply enhance employability
Knowledge exchangeinnov
ation Build contacts
Entrepreneurial skillsattitudes
Pedagogic development
Raise profilebrand
Staff retentionupskill
ing
Institutional modernisation
Attract students
Quantitative vs
qualitative
Immediate vs longer
term
Tangible vs Intangible
Direct vs Indirect
2014 9
course) whilst others are more indirect (such as the value of contacts gained for
potential future research projects) Equally some benefits are very tangible (such as
when a student develops a particular skill) but others are much less so (such as where
student knowledge spills over to workers in a company who work alongside them on a
project) Examples of all these potential benefits were visible throughout the cases
explored for this study
In practice our study illustrates a focus on immediate highly-visible returns by 10
university-business cooperation projects but also emphasises the significance of less
tangible and more indirect returns to participants with different participants placing
differential values on particular outcomes For example in the case of the North East
England sub-sea collaboration businesses were keen to fill an emerging skills gap
upskill staff and develop contacts with the university and other businesses For the
academics it was a means to generate additional student numbers to engage with
the business sector and to explore further research avenues
For students the reason they wished to participate in these collaborative offers in the
field of education was primarily related to the fact they felt it would provide them with
greater skills than a more conventional course that they were more likely to find
subsequent employment or that it would provide them with enhanced entrepreneurial
opportunities
For universities and academics there was a similar emphasis on the enhanced skills
acquired by students the greater employability of these students and the stronger
entrepreneurial outcomes anticipated as highlighted in the EUEN project These
outcomes were expected to feed through into an increase in either the quality or the
quantity of student applications with improvements in student perceptions of course
quality also anticipated Additional outcomes which were cited in some instances
included the opportunity provided for increasing contacts in the business world the
updating of skills through exposure to contemporary businesses practices and the
building of trust-based relationships for future collaborative research
For businesses the primary outcome reported was in terms of an enhanced pipeline of
suitably skilled potential employees together with improvements to the wider labour
force There were also particular outcomes specific to individual cooperation
arrangements such as in the case of AppCampus where one measure of success is the
number of applications developed Businesses also reported that the cooperation
arrangements provided benefits in the form of raising their profile but also helped to
build trust and wider contact networks with academics for future collaborative
research Firms also reported that involvement in live projects did provide the
opportunity for product and process innovations adding economic value to the
company and that the bringing together of students and existing employees served to
raise the knowledge levels of employees as well as students
For other parties such as regional authorities the anticipated (and realised) outcomes
largely revolved around economic benefits such as an enhanced labour supply
increased levels of graduate employment higher levels of new business-starts or
(rarely) higher levels of innovation
2014 10
Assessing the benefits of cooperation
Currently there is very little guidance on approaches to assessing the benefits of
university-business cooperation activities in the field of education The existing
literature is strongly inclined to cooperation in the field of research and innovation
What examples can be found tend to stress measures of inputs and student outcomes
The wider literature can however provide valuable insights This incudes highlighting
the value of less tangible benefits such as contact between students company staff
and university staff often termed as productive interactions
Monitoring and assessment has a number of potential benefits it provides a means to
assess whether a project or programme is performing as expected it enables a
project leader to identify what works and what does not it provides a useful
mechanism to feedback lessons to others and if undertaken in advance it can
provide a means for selecting between different potential options In the evidence
available to this project the main approaches to monitoring and assessment have
focused on project performance often with an emphasis on measures of inputs and
outputs However there is also evidence of the use of results-based formats often of
a qualitative nature
The evidence available from the cases explored in this work suggests that monitoring
activities are undertaken at two levels in the field of university-business collaboration
in the field of education
Firstly through the use of standard student-orientated metrics
Secondly through informal metrics based around joint-perceptions of
success
The cases also suggest that there are a range of outcomes of university-business
collaboration activities in the field of education that are not currently being captured
by existing monitoring and measurement arrangements such as the potential
upskilling of existing staff through their involvement in the educational experience or
the raising of the profile of participating institutions However in no case was this felt
to weaken the cooperation arrangements underway Indeed to the contrary the
suggestion was that widening the formal measurement and monitoring process to
include more indicators would incur an additional burden for limited gains It is
essential that assessment methods are not only fit for purpose but are also seen to be
so
Whilst there are many methods that can be used to assess the outcomes of a
particular cooperation activity (whether that is a project or a programme) the
balanced scorecard approach seems particularly relevant The central idea of the
balanced scorecard is that several metrics can be used to evaluate the process in
question A causal map for educational university-industry collaboration initiatives is
given in Figure ES3 Four areas of measurement form the essential components of a
scorecard for university-industry collaboration The people involved the resources
available the activities undertaken and the results achieved
2014 11
Figure ES3 Scorecard general structure
Although each participant may have their own scorecard for the cooperation (either
held explicitly or implicitly) a combined scorecard can be drawn up for each co-
operation project in the form of a logical framework This allows assumptions to be
made explicit expected relationships to be identified and tests whether the
cooperation planned can meet its intended purpose(s) A stylized logic chain is set out
in Figure ES4
Figure ES4 Stylised causal map for university-industry collaboration
Alongside the scorecard approach an assessment methodology may also wish to set
out the framework for monitoring and evaluation Where evaluation may relate to the
assessment of project applications or the success of a completed project Here a
simple framework such as the success map set out in Figure ES5 may be
advantageous
People
bullThe qualification fit and motivation of the people taking part in the cooperation
Benefits
bullThe beneficial outputs that are generated as a result of the collaboration
Resources
bullThe amount and quality of the resources dedicated to the collaboration
Activities
bullThe quality and appropriateness of processes and organizational set-up
People
bullInstructors students managers
Resources
bullFunding equipment technology
Activities
bullOrganizational processes and structures
Benefits
bullEducation skills employment
2014 12
Figure ES5 Success map for cooperation in the field of education
In developing such a success map due regard should be given to four purposes of
university-business cooperation in the field of education broadly stated
Those that relate to teaching and learning aimed at strengthening the
labour supply and employment
Those that relate to stimulating entrepreneurship through teaching and
learning
Those that relate to knowledge exchange aimed at promoting
productprocess innovation or new pedagogy
Those that relate to raising the profile of an organization
In seeking to assess an actual project the framework can ensure that appropriate
indicators are selected and measurement approaches adopted It is beyond the scope
of this report to suggest a standard set of indicators as this will vary by project
purpose However a range of potential indicators have been identified relating to
each of the above purposes and which might be used to monitor inputs activities
outputs and outcomes Examples of potential indicators are illustrated in Figure ES5
In selecting indicators a number of principles should be observed including those of
parsimony (to seek to limit the data burden) and proportionality (data collected should
not incur costs disproportionate to the benefits received) The collection of data does
incur costs and for an assessment process to operate successfully there must be
agreement of all parties as to the role of particular indicators and the value of their
measurement
Interaction opportunities
Access to resources
Motivated educators
High-quality students
Relevant course delivery
Relevant course design
New ideas developed
Skilled amp trained labour
New pedagogies
Student applicant numbersquality
Non-academic organisations involved
Time inputs
Finance inputs
Newimproved courses
Number of graduates
Student attainment
Entrepreneurial attitudes
New products processes developed
Joint objective setting
Internshipsplacements
Co-delivery and assessment
Interaction intensity
Graduate employment
New business starts
Employer satisfaction
Organisational ranking in lsquoleaguersquo tables
Successful alliance
Human capital
Entrepreneurship
New ideas
Profile raising
Input Activity Output Outcome
2014 13
In the selection of indicators full consideration should also be given to the role of
qualitative measures of success and the potential means of capturing these This
might be through survey techniques but can also be captured by structured review
meetings with inputs from the various parties The value of self-assessment
techniques should also be considered Quantitative measures should not be over-
looked and where these can be part of wider measurement processes should be
welcomed
Implications for policy and practice
This study has proposed a flexible approach to measuring the impact of university-
business cooperation activities using a balanced scorecard approach allied with a
success-map for cooperation activities This recognizes the multiple purposes that
underlie cooperation activities and acknowledges that a one-size-fits-all approach is
unlikely to prove fit for purpose What are the benefits and implications of the
approach proposed to policy-makers and to the collaboration partners themselves
For policy-makers the tool can be used in a variety of manners
In the first instance it forms a framework for the ex-ante development of
the proposed programme By identifying the resources that are available to
the programme or might be available and the desired objectives
programme officials can identify the types of activity that might lead to the
desired results Through a process of iteration the scorecard can be refined
further This can then provide a baseline for a call for applications
The scorecard approach might then be used for the assessment or
appraisal of applications Assessors can check the applications against the
programme scorecard to identify the extent to which each applicant
contributes to the desired approach and objectives In a dynamic approach
the programme scorecard can be updated through the inclusion of novel
and innovative approaches proposed by applicants which were not foreseen
during the development of the programme A blank scorecard template
might also be completed by assessors as a means of summarising the
information contained in applications in a common format The approach
could highlight missed opportunities for collaborative activities or
resourcepeople inputs which can form the basis of negotiation with
programme applicants
Following a successful application a project scorecard can then form the
basis for the review of project progress This can examine the activities
undertaken resources expended and people involved against what was
initially planned together with the benefits realized compared to those that
were expected The review meeting can then explore the reasons for any
differences examining cases where inputs are greater than anticipated
perhaps owing to the cooperation going better than planned or where
results or inputs are less than originally proposed and whether remedial
action needs to be taken
2014 14
The scorecard approach can also deliver significant benefits to the collaboration
partners themselves
As a starting point it can form the basis for robust project design The
scorecard can be used as a mechanism for each partner to set out what
they wish to achieve from the cooperation activity and what they are able
to contribute Each partner can then identify what they expect of the other
partners Using the scorecard approach can also assist in identifying where
potential opportunities might have been overlooked in the original project
design and where small changes might reap large rewards This can then
form the basis of the collaboration agreement between parties
Project partners can then use their own stakeholder scorecard as a means
for internal review and monitoring of the collaboration activity This
provides a useful mechanism for identifying whether desired outcomes are
being achieved and the extent to which anticipated inputs and activities are
occurring As a dynamic process the scorecard can be updated as
activities are added or amended or other inputs change
The partnership as a whole can also use the scorecard as the basis for
ongoing project reviews This can consider whether the original
assumptions remain valid and whether some activities are more appropriate
than others in achieving the desired objectives Crucial to the review
process is to ensure that all partners are realizing the benefits that they
sought initially and if not how this might be rectified The scorecard
approach provides a means of structuring this review process
In order to promote the use of the scorecard approach the Commission may wish to
provide a guide to its use in the context of university-business cooperation activities
This would offer the opportunity to develop worked examples tailored to different
cooperation purposes Such a guide could also incorporate advice on practical
techniques for measuring the outcomes of cooperation activities In the first instance
this could form part of a toolkit available to assist those parties interested in applying
for Knowledge Alliance projects
HOW TO OBTAIN EU PUBLICATIONS
Free publications
bull one copy
via EU Bookshop (httpbookshopeuropaeu)
bull more than one copy or postersmaps
from the European Unionrsquos representations (httpeceuropaeurepresent_enhtm)
from the delegations in non-EU countries (httpeeaseuropaeudelegationsindex_enhtm)
by contacting the Europe Direct service (httpeuropaeueuropedirectindex_enhtm) or
calling 00 800 6 7 8 9 10 11 (freephone number from anywhere in the EU) ()
() The information given is free as are most calls (though some operators phone boxes
or hotels may charge you)
Priced publications
bull via EU Bookshop (httpbookshopeuropaeu)
Priced subscriptions
bull via one of the sales agents of the Publications Office of the European Union
(httppublicationseuropaeuothersagentsindex_enhtm)
NC-0
1-1
4-2
99-E
N-N
doi10
2014 4
Table of Contents
Introduction 5
Types and drivers of cooperation 6
Benefits of cooperation 8
Assessing the benefits of cooperation 10
Implications for policy and practice 13
2014 5
Introduction
Cardiff University in association with Newcastle University and Imperial Consultants
is pleased to present this report on Measuring the Impact of University-Business
Cooperation to DG Education and Culture (DG EAC)
Promoting and developing cooperation between higher education and business is a
core element of the EUrsquos Agenda for Modernising Higher Education which stresses the
importance of this for pedagogical skills and the development of courses relevant to
social and labour market needs1 The potential to enhance this contribution further
through increased levels of collaboration is now firmly recognized within EU policy
circles and in Member States This is highlighted in the publication of Europe 2020 and
the related Flagship Initiative promoting a new agenda for skills and jobs which
identifies the value of ventures and networks between business and higher educational
establishments to address new skills requirements and labour market needs2
In recognition of the potential benefits to be realized through increased cooperation
between businesses and Higher Educations Institutions (HEIs) in the field of education
the European Commission launched the Knowledge Alliance pilot initiative to create
new multidisciplinary curricula to promote entrepreneurship within education as well
as developing other transferable skills This initially funded a small number of pilot
projects in 2011 and 2012 before embedding the initiative in the Lifelong Learning
Programme in 2013 From 2014 Knowledge Alliances will be part of the Erasmus+
programme
There are many other examples of university-business collaboration in the field of
education across the EU with no involvement of public sector bodies many of which
have been underway for several decades Examples cited in this study include the
Masters Course in Banking Management established in 1989 in Valencia Spain and
the course in Commercial Communications launched in Prague Czech Republic in
1994 As these examples illustrate many cooperation projects are the result of
bilateral agreements between individual companies and HEIs The focus of these
activities can vary widely depending upon the aims and objectives of each particular
example (as identified in the Wilson Review in the UK3)
In contrast to university-business cooperation in the world of research and innovation
the profile of business-university collaboration in the field of education has been
relatively limited This is unfortunate as it is at this level that knowledge exchange
can often most effectively be secured relevant skills developed and the conditions for
future innovation and economic growth laid This low profile has resulted in a limited
literature on the topic and prompted DG EAC to commission this study amongst
others The study had three specific objectives
1 High Level Group on the Modernisation of Higher Education (2013) Improving the quality of teaching and learning in Europersquos higher education institutions European Commission 2 European Commission (2010) An Agenda for new skills and jobs A European contribution towards full employment Com(2010)682 Final 3 Wilson T (2012) A Review of University Business Collaboration
2014 6
To analyse existing and identify emerging types of university-business
cooperation in Europe and their drivers
To identify demonstrate and assess tangible and intangible effects of
university-business-cooperation
To develop an assessment methodology and relevant qualitative and
quantitative indicators for measuring the outcomes and impact of university-
business cooperation and in particular of Knowledge Alliances
The study has analysed the following ten cases
Educckate multinational - led by the University College London in
partnership with other EU Universities and their local companies to
stimulate entrepreneurship in cultural and creative industries
EUEN multinational ndash led by Coventry University with transnational
academic and industry partners to promote entrepreneurial education
KnowFact multinational ndash led by the University of Patras with industry
partners from other EU countries to develop the Teaching Factory paradigm
in manufacturing education
AppCampus Finland ndash an innovative initiative between Aalto University and
NokiaMicrosoft involving app Development National
Qatar Carbonates UK ndash a collaborative arrangement between Qatar
PetroleumShell and Imperial College London promoting post-graduate
education International
Newcastle University Subsea UK ndash cooperation between Newcastle
University and regional businesses developing sector relevant skills
Regional
Masters in Banking Valencia Spain ndash collaboration between Valencia
University and regional banking bodies developing sector relevant skills
Regional
HP Bulgaria ndash Cooperation between Hewlett Packard and various
Universities whereby HP trained university lecturers in specialised
technologies and created the project curriculum National
University of Merseburg Germany ndash regional businesses endow professorial
chairs at University of Merseburg to ensure industry relevant education
AKA Prague ndash professional qualification (Certificate of Communication
Agencies) developed by industry and delivered by University of Economics
Prague
Types and drivers of cooperation
The first decade of the 21st Century has seen an increasing emphasis on the role of
universities in explicitly contributing to social and economic development This can be
attributed to a number of factors including
2014 7
The emergence of global lsquogrand challengesrsquo (eg climate change ageing
terrorism etc) which cannot be solved by government or business alone
but requires a multi-disciplinary and collaborative approach
Increased marketisation of higher education leading to greater competition
between universities and an emphasis on the lsquostudent experiencersquo and
student outcomes (including entrepreneurship)
The global economic crisis which has ushered in an era of austerity in public
finances in many countries
The recognition by many firms that working with Universities provides an
advantage for their future recruitment pipelines and ancillary profile
benefits
This has resulted in a range of policies designed to encourage universities to build
stronger links with business suggesting that businesses should be (more) involved in
designing curricula (including undergraduate postgraduate as well as CPD and other
training) and that universities should work more closely with industry partners to
promote entrepreneurship mobility (between business and academia) and lifelong
learning
In practice many universities and businesses have recognized the value of
cooperation particularly in terms of how this can strengthen student recruitment for
universities improve the educational experience for students and provide a supply of
more experienced labour for companies Some companies and universities also
recognize the wider benefits that this cooperation can bring in the form of higher
levels of innovation the upskilling and retention of existing staff and the higher profile
that successful collaboration activities can bring
A number of reports have usefully set out different lsquotypesrsquo of cooperation between
businesses and universities including student and academic mobility programmes
curriculum development and delivery lifelong learning and entrepreneurship4
However most cooperation activities examined for this work incorporated more than
one of these cooperation types This suggests that it is the purpose of cooperation
that is more important rather than the type of cooperation undertaken
Each cooperation partner (business university student) has their own reason to
engage in the cooperation activity but a number of common themes can be identified
These are illustrated in Figure ES1 Three of these relate to strengthening levels of
human capital either of students or of existing workers within companies and
universities to promote a stronger labour supply enhance individual employability
and heighten levels of entrepreneurship These were common across all the projects
examined for this study Three are more related to innovation primarily through
collaborative working on company projects as part of the educational experience or
through the co-production of new courses and course material these were visible in
each of the cases studied but to varying degrees Finally the remaining three
4 See for example Wilson T (2012) A Review of University Business Collaboration and Science-to-Business Marketing Research Centre (2011) The State of European University Business Cooperation European Commission
2014 8
reasons relate to developing longer-term contacts and the profile of the industry
companies and universities involved for future potential benefits
Figure ES1 Purposes underlying cooperation activities
Benefits of cooperation
The potential benefits of collaboration activities in the field of education can take many
different forms (Figure ES2) Very often it is the qualitative benefits that cooperation
activities bring (such as the deepening of skill sets) that are valued by participants
rather than quantitative benefits (such as increasing the numbers of graduates)
although this does also feature
Figure ES2 Four forms of outcomes
Some benefits are more immediate and occur directly following the end of the
successful completion of a course of studies (such as the number of graduates) or
during it (such as through engagement of staff and students on joint activities) whilst
others may be longer-term (such as new business starts arising out of stronger
entrepreneurial attitudes) Some benefits are direct (such as the creation of a new
Secure labour supply enhance employability
Knowledge exchangeinnov
ation Build contacts
Entrepreneurial skillsattitudes
Pedagogic development
Raise profilebrand
Staff retentionupskill
ing
Institutional modernisation
Attract students
Quantitative vs
qualitative
Immediate vs longer
term
Tangible vs Intangible
Direct vs Indirect
2014 9
course) whilst others are more indirect (such as the value of contacts gained for
potential future research projects) Equally some benefits are very tangible (such as
when a student develops a particular skill) but others are much less so (such as where
student knowledge spills over to workers in a company who work alongside them on a
project) Examples of all these potential benefits were visible throughout the cases
explored for this study
In practice our study illustrates a focus on immediate highly-visible returns by 10
university-business cooperation projects but also emphasises the significance of less
tangible and more indirect returns to participants with different participants placing
differential values on particular outcomes For example in the case of the North East
England sub-sea collaboration businesses were keen to fill an emerging skills gap
upskill staff and develop contacts with the university and other businesses For the
academics it was a means to generate additional student numbers to engage with
the business sector and to explore further research avenues
For students the reason they wished to participate in these collaborative offers in the
field of education was primarily related to the fact they felt it would provide them with
greater skills than a more conventional course that they were more likely to find
subsequent employment or that it would provide them with enhanced entrepreneurial
opportunities
For universities and academics there was a similar emphasis on the enhanced skills
acquired by students the greater employability of these students and the stronger
entrepreneurial outcomes anticipated as highlighted in the EUEN project These
outcomes were expected to feed through into an increase in either the quality or the
quantity of student applications with improvements in student perceptions of course
quality also anticipated Additional outcomes which were cited in some instances
included the opportunity provided for increasing contacts in the business world the
updating of skills through exposure to contemporary businesses practices and the
building of trust-based relationships for future collaborative research
For businesses the primary outcome reported was in terms of an enhanced pipeline of
suitably skilled potential employees together with improvements to the wider labour
force There were also particular outcomes specific to individual cooperation
arrangements such as in the case of AppCampus where one measure of success is the
number of applications developed Businesses also reported that the cooperation
arrangements provided benefits in the form of raising their profile but also helped to
build trust and wider contact networks with academics for future collaborative
research Firms also reported that involvement in live projects did provide the
opportunity for product and process innovations adding economic value to the
company and that the bringing together of students and existing employees served to
raise the knowledge levels of employees as well as students
For other parties such as regional authorities the anticipated (and realised) outcomes
largely revolved around economic benefits such as an enhanced labour supply
increased levels of graduate employment higher levels of new business-starts or
(rarely) higher levels of innovation
2014 10
Assessing the benefits of cooperation
Currently there is very little guidance on approaches to assessing the benefits of
university-business cooperation activities in the field of education The existing
literature is strongly inclined to cooperation in the field of research and innovation
What examples can be found tend to stress measures of inputs and student outcomes
The wider literature can however provide valuable insights This incudes highlighting
the value of less tangible benefits such as contact between students company staff
and university staff often termed as productive interactions
Monitoring and assessment has a number of potential benefits it provides a means to
assess whether a project or programme is performing as expected it enables a
project leader to identify what works and what does not it provides a useful
mechanism to feedback lessons to others and if undertaken in advance it can
provide a means for selecting between different potential options In the evidence
available to this project the main approaches to monitoring and assessment have
focused on project performance often with an emphasis on measures of inputs and
outputs However there is also evidence of the use of results-based formats often of
a qualitative nature
The evidence available from the cases explored in this work suggests that monitoring
activities are undertaken at two levels in the field of university-business collaboration
in the field of education
Firstly through the use of standard student-orientated metrics
Secondly through informal metrics based around joint-perceptions of
success
The cases also suggest that there are a range of outcomes of university-business
collaboration activities in the field of education that are not currently being captured
by existing monitoring and measurement arrangements such as the potential
upskilling of existing staff through their involvement in the educational experience or
the raising of the profile of participating institutions However in no case was this felt
to weaken the cooperation arrangements underway Indeed to the contrary the
suggestion was that widening the formal measurement and monitoring process to
include more indicators would incur an additional burden for limited gains It is
essential that assessment methods are not only fit for purpose but are also seen to be
so
Whilst there are many methods that can be used to assess the outcomes of a
particular cooperation activity (whether that is a project or a programme) the
balanced scorecard approach seems particularly relevant The central idea of the
balanced scorecard is that several metrics can be used to evaluate the process in
question A causal map for educational university-industry collaboration initiatives is
given in Figure ES3 Four areas of measurement form the essential components of a
scorecard for university-industry collaboration The people involved the resources
available the activities undertaken and the results achieved
2014 11
Figure ES3 Scorecard general structure
Although each participant may have their own scorecard for the cooperation (either
held explicitly or implicitly) a combined scorecard can be drawn up for each co-
operation project in the form of a logical framework This allows assumptions to be
made explicit expected relationships to be identified and tests whether the
cooperation planned can meet its intended purpose(s) A stylized logic chain is set out
in Figure ES4
Figure ES4 Stylised causal map for university-industry collaboration
Alongside the scorecard approach an assessment methodology may also wish to set
out the framework for monitoring and evaluation Where evaluation may relate to the
assessment of project applications or the success of a completed project Here a
simple framework such as the success map set out in Figure ES5 may be
advantageous
People
bullThe qualification fit and motivation of the people taking part in the cooperation
Benefits
bullThe beneficial outputs that are generated as a result of the collaboration
Resources
bullThe amount and quality of the resources dedicated to the collaboration
Activities
bullThe quality and appropriateness of processes and organizational set-up
People
bullInstructors students managers
Resources
bullFunding equipment technology
Activities
bullOrganizational processes and structures
Benefits
bullEducation skills employment
2014 12
Figure ES5 Success map for cooperation in the field of education
In developing such a success map due regard should be given to four purposes of
university-business cooperation in the field of education broadly stated
Those that relate to teaching and learning aimed at strengthening the
labour supply and employment
Those that relate to stimulating entrepreneurship through teaching and
learning
Those that relate to knowledge exchange aimed at promoting
productprocess innovation or new pedagogy
Those that relate to raising the profile of an organization
In seeking to assess an actual project the framework can ensure that appropriate
indicators are selected and measurement approaches adopted It is beyond the scope
of this report to suggest a standard set of indicators as this will vary by project
purpose However a range of potential indicators have been identified relating to
each of the above purposes and which might be used to monitor inputs activities
outputs and outcomes Examples of potential indicators are illustrated in Figure ES5
In selecting indicators a number of principles should be observed including those of
parsimony (to seek to limit the data burden) and proportionality (data collected should
not incur costs disproportionate to the benefits received) The collection of data does
incur costs and for an assessment process to operate successfully there must be
agreement of all parties as to the role of particular indicators and the value of their
measurement
Interaction opportunities
Access to resources
Motivated educators
High-quality students
Relevant course delivery
Relevant course design
New ideas developed
Skilled amp trained labour
New pedagogies
Student applicant numbersquality
Non-academic organisations involved
Time inputs
Finance inputs
Newimproved courses
Number of graduates
Student attainment
Entrepreneurial attitudes
New products processes developed
Joint objective setting
Internshipsplacements
Co-delivery and assessment
Interaction intensity
Graduate employment
New business starts
Employer satisfaction
Organisational ranking in lsquoleaguersquo tables
Successful alliance
Human capital
Entrepreneurship
New ideas
Profile raising
Input Activity Output Outcome
2014 13
In the selection of indicators full consideration should also be given to the role of
qualitative measures of success and the potential means of capturing these This
might be through survey techniques but can also be captured by structured review
meetings with inputs from the various parties The value of self-assessment
techniques should also be considered Quantitative measures should not be over-
looked and where these can be part of wider measurement processes should be
welcomed
Implications for policy and practice
This study has proposed a flexible approach to measuring the impact of university-
business cooperation activities using a balanced scorecard approach allied with a
success-map for cooperation activities This recognizes the multiple purposes that
underlie cooperation activities and acknowledges that a one-size-fits-all approach is
unlikely to prove fit for purpose What are the benefits and implications of the
approach proposed to policy-makers and to the collaboration partners themselves
For policy-makers the tool can be used in a variety of manners
In the first instance it forms a framework for the ex-ante development of
the proposed programme By identifying the resources that are available to
the programme or might be available and the desired objectives
programme officials can identify the types of activity that might lead to the
desired results Through a process of iteration the scorecard can be refined
further This can then provide a baseline for a call for applications
The scorecard approach might then be used for the assessment or
appraisal of applications Assessors can check the applications against the
programme scorecard to identify the extent to which each applicant
contributes to the desired approach and objectives In a dynamic approach
the programme scorecard can be updated through the inclusion of novel
and innovative approaches proposed by applicants which were not foreseen
during the development of the programme A blank scorecard template
might also be completed by assessors as a means of summarising the
information contained in applications in a common format The approach
could highlight missed opportunities for collaborative activities or
resourcepeople inputs which can form the basis of negotiation with
programme applicants
Following a successful application a project scorecard can then form the
basis for the review of project progress This can examine the activities
undertaken resources expended and people involved against what was
initially planned together with the benefits realized compared to those that
were expected The review meeting can then explore the reasons for any
differences examining cases where inputs are greater than anticipated
perhaps owing to the cooperation going better than planned or where
results or inputs are less than originally proposed and whether remedial
action needs to be taken
2014 14
The scorecard approach can also deliver significant benefits to the collaboration
partners themselves
As a starting point it can form the basis for robust project design The
scorecard can be used as a mechanism for each partner to set out what
they wish to achieve from the cooperation activity and what they are able
to contribute Each partner can then identify what they expect of the other
partners Using the scorecard approach can also assist in identifying where
potential opportunities might have been overlooked in the original project
design and where small changes might reap large rewards This can then
form the basis of the collaboration agreement between parties
Project partners can then use their own stakeholder scorecard as a means
for internal review and monitoring of the collaboration activity This
provides a useful mechanism for identifying whether desired outcomes are
being achieved and the extent to which anticipated inputs and activities are
occurring As a dynamic process the scorecard can be updated as
activities are added or amended or other inputs change
The partnership as a whole can also use the scorecard as the basis for
ongoing project reviews This can consider whether the original
assumptions remain valid and whether some activities are more appropriate
than others in achieving the desired objectives Crucial to the review
process is to ensure that all partners are realizing the benefits that they
sought initially and if not how this might be rectified The scorecard
approach provides a means of structuring this review process
In order to promote the use of the scorecard approach the Commission may wish to
provide a guide to its use in the context of university-business cooperation activities
This would offer the opportunity to develop worked examples tailored to different
cooperation purposes Such a guide could also incorporate advice on practical
techniques for measuring the outcomes of cooperation activities In the first instance
this could form part of a toolkit available to assist those parties interested in applying
for Knowledge Alliance projects
HOW TO OBTAIN EU PUBLICATIONS
Free publications
bull one copy
via EU Bookshop (httpbookshopeuropaeu)
bull more than one copy or postersmaps
from the European Unionrsquos representations (httpeceuropaeurepresent_enhtm)
from the delegations in non-EU countries (httpeeaseuropaeudelegationsindex_enhtm)
by contacting the Europe Direct service (httpeuropaeueuropedirectindex_enhtm) or
calling 00 800 6 7 8 9 10 11 (freephone number from anywhere in the EU) ()
() The information given is free as are most calls (though some operators phone boxes
or hotels may charge you)
Priced publications
bull via EU Bookshop (httpbookshopeuropaeu)
Priced subscriptions
bull via one of the sales agents of the Publications Office of the European Union
(httppublicationseuropaeuothersagentsindex_enhtm)
NC-0
1-1
4-2
99-E
N-N
doi10
2014 5
Introduction
Cardiff University in association with Newcastle University and Imperial Consultants
is pleased to present this report on Measuring the Impact of University-Business
Cooperation to DG Education and Culture (DG EAC)
Promoting and developing cooperation between higher education and business is a
core element of the EUrsquos Agenda for Modernising Higher Education which stresses the
importance of this for pedagogical skills and the development of courses relevant to
social and labour market needs1 The potential to enhance this contribution further
through increased levels of collaboration is now firmly recognized within EU policy
circles and in Member States This is highlighted in the publication of Europe 2020 and
the related Flagship Initiative promoting a new agenda for skills and jobs which
identifies the value of ventures and networks between business and higher educational
establishments to address new skills requirements and labour market needs2
In recognition of the potential benefits to be realized through increased cooperation
between businesses and Higher Educations Institutions (HEIs) in the field of education
the European Commission launched the Knowledge Alliance pilot initiative to create
new multidisciplinary curricula to promote entrepreneurship within education as well
as developing other transferable skills This initially funded a small number of pilot
projects in 2011 and 2012 before embedding the initiative in the Lifelong Learning
Programme in 2013 From 2014 Knowledge Alliances will be part of the Erasmus+
programme
There are many other examples of university-business collaboration in the field of
education across the EU with no involvement of public sector bodies many of which
have been underway for several decades Examples cited in this study include the
Masters Course in Banking Management established in 1989 in Valencia Spain and
the course in Commercial Communications launched in Prague Czech Republic in
1994 As these examples illustrate many cooperation projects are the result of
bilateral agreements between individual companies and HEIs The focus of these
activities can vary widely depending upon the aims and objectives of each particular
example (as identified in the Wilson Review in the UK3)
In contrast to university-business cooperation in the world of research and innovation
the profile of business-university collaboration in the field of education has been
relatively limited This is unfortunate as it is at this level that knowledge exchange
can often most effectively be secured relevant skills developed and the conditions for
future innovation and economic growth laid This low profile has resulted in a limited
literature on the topic and prompted DG EAC to commission this study amongst
others The study had three specific objectives
1 High Level Group on the Modernisation of Higher Education (2013) Improving the quality of teaching and learning in Europersquos higher education institutions European Commission 2 European Commission (2010) An Agenda for new skills and jobs A European contribution towards full employment Com(2010)682 Final 3 Wilson T (2012) A Review of University Business Collaboration
2014 6
To analyse existing and identify emerging types of university-business
cooperation in Europe and their drivers
To identify demonstrate and assess tangible and intangible effects of
university-business-cooperation
To develop an assessment methodology and relevant qualitative and
quantitative indicators for measuring the outcomes and impact of university-
business cooperation and in particular of Knowledge Alliances
The study has analysed the following ten cases
Educckate multinational - led by the University College London in
partnership with other EU Universities and their local companies to
stimulate entrepreneurship in cultural and creative industries
EUEN multinational ndash led by Coventry University with transnational
academic and industry partners to promote entrepreneurial education
KnowFact multinational ndash led by the University of Patras with industry
partners from other EU countries to develop the Teaching Factory paradigm
in manufacturing education
AppCampus Finland ndash an innovative initiative between Aalto University and
NokiaMicrosoft involving app Development National
Qatar Carbonates UK ndash a collaborative arrangement between Qatar
PetroleumShell and Imperial College London promoting post-graduate
education International
Newcastle University Subsea UK ndash cooperation between Newcastle
University and regional businesses developing sector relevant skills
Regional
Masters in Banking Valencia Spain ndash collaboration between Valencia
University and regional banking bodies developing sector relevant skills
Regional
HP Bulgaria ndash Cooperation between Hewlett Packard and various
Universities whereby HP trained university lecturers in specialised
technologies and created the project curriculum National
University of Merseburg Germany ndash regional businesses endow professorial
chairs at University of Merseburg to ensure industry relevant education
AKA Prague ndash professional qualification (Certificate of Communication
Agencies) developed by industry and delivered by University of Economics
Prague
Types and drivers of cooperation
The first decade of the 21st Century has seen an increasing emphasis on the role of
universities in explicitly contributing to social and economic development This can be
attributed to a number of factors including
2014 7
The emergence of global lsquogrand challengesrsquo (eg climate change ageing
terrorism etc) which cannot be solved by government or business alone
but requires a multi-disciplinary and collaborative approach
Increased marketisation of higher education leading to greater competition
between universities and an emphasis on the lsquostudent experiencersquo and
student outcomes (including entrepreneurship)
The global economic crisis which has ushered in an era of austerity in public
finances in many countries
The recognition by many firms that working with Universities provides an
advantage for their future recruitment pipelines and ancillary profile
benefits
This has resulted in a range of policies designed to encourage universities to build
stronger links with business suggesting that businesses should be (more) involved in
designing curricula (including undergraduate postgraduate as well as CPD and other
training) and that universities should work more closely with industry partners to
promote entrepreneurship mobility (between business and academia) and lifelong
learning
In practice many universities and businesses have recognized the value of
cooperation particularly in terms of how this can strengthen student recruitment for
universities improve the educational experience for students and provide a supply of
more experienced labour for companies Some companies and universities also
recognize the wider benefits that this cooperation can bring in the form of higher
levels of innovation the upskilling and retention of existing staff and the higher profile
that successful collaboration activities can bring
A number of reports have usefully set out different lsquotypesrsquo of cooperation between
businesses and universities including student and academic mobility programmes
curriculum development and delivery lifelong learning and entrepreneurship4
However most cooperation activities examined for this work incorporated more than
one of these cooperation types This suggests that it is the purpose of cooperation
that is more important rather than the type of cooperation undertaken
Each cooperation partner (business university student) has their own reason to
engage in the cooperation activity but a number of common themes can be identified
These are illustrated in Figure ES1 Three of these relate to strengthening levels of
human capital either of students or of existing workers within companies and
universities to promote a stronger labour supply enhance individual employability
and heighten levels of entrepreneurship These were common across all the projects
examined for this study Three are more related to innovation primarily through
collaborative working on company projects as part of the educational experience or
through the co-production of new courses and course material these were visible in
each of the cases studied but to varying degrees Finally the remaining three
4 See for example Wilson T (2012) A Review of University Business Collaboration and Science-to-Business Marketing Research Centre (2011) The State of European University Business Cooperation European Commission
2014 8
reasons relate to developing longer-term contacts and the profile of the industry
companies and universities involved for future potential benefits
Figure ES1 Purposes underlying cooperation activities
Benefits of cooperation
The potential benefits of collaboration activities in the field of education can take many
different forms (Figure ES2) Very often it is the qualitative benefits that cooperation
activities bring (such as the deepening of skill sets) that are valued by participants
rather than quantitative benefits (such as increasing the numbers of graduates)
although this does also feature
Figure ES2 Four forms of outcomes
Some benefits are more immediate and occur directly following the end of the
successful completion of a course of studies (such as the number of graduates) or
during it (such as through engagement of staff and students on joint activities) whilst
others may be longer-term (such as new business starts arising out of stronger
entrepreneurial attitudes) Some benefits are direct (such as the creation of a new
Secure labour supply enhance employability
Knowledge exchangeinnov
ation Build contacts
Entrepreneurial skillsattitudes
Pedagogic development
Raise profilebrand
Staff retentionupskill
ing
Institutional modernisation
Attract students
Quantitative vs
qualitative
Immediate vs longer
term
Tangible vs Intangible
Direct vs Indirect
2014 9
course) whilst others are more indirect (such as the value of contacts gained for
potential future research projects) Equally some benefits are very tangible (such as
when a student develops a particular skill) but others are much less so (such as where
student knowledge spills over to workers in a company who work alongside them on a
project) Examples of all these potential benefits were visible throughout the cases
explored for this study
In practice our study illustrates a focus on immediate highly-visible returns by 10
university-business cooperation projects but also emphasises the significance of less
tangible and more indirect returns to participants with different participants placing
differential values on particular outcomes For example in the case of the North East
England sub-sea collaboration businesses were keen to fill an emerging skills gap
upskill staff and develop contacts with the university and other businesses For the
academics it was a means to generate additional student numbers to engage with
the business sector and to explore further research avenues
For students the reason they wished to participate in these collaborative offers in the
field of education was primarily related to the fact they felt it would provide them with
greater skills than a more conventional course that they were more likely to find
subsequent employment or that it would provide them with enhanced entrepreneurial
opportunities
For universities and academics there was a similar emphasis on the enhanced skills
acquired by students the greater employability of these students and the stronger
entrepreneurial outcomes anticipated as highlighted in the EUEN project These
outcomes were expected to feed through into an increase in either the quality or the
quantity of student applications with improvements in student perceptions of course
quality also anticipated Additional outcomes which were cited in some instances
included the opportunity provided for increasing contacts in the business world the
updating of skills through exposure to contemporary businesses practices and the
building of trust-based relationships for future collaborative research
For businesses the primary outcome reported was in terms of an enhanced pipeline of
suitably skilled potential employees together with improvements to the wider labour
force There were also particular outcomes specific to individual cooperation
arrangements such as in the case of AppCampus where one measure of success is the
number of applications developed Businesses also reported that the cooperation
arrangements provided benefits in the form of raising their profile but also helped to
build trust and wider contact networks with academics for future collaborative
research Firms also reported that involvement in live projects did provide the
opportunity for product and process innovations adding economic value to the
company and that the bringing together of students and existing employees served to
raise the knowledge levels of employees as well as students
For other parties such as regional authorities the anticipated (and realised) outcomes
largely revolved around economic benefits such as an enhanced labour supply
increased levels of graduate employment higher levels of new business-starts or
(rarely) higher levels of innovation
2014 10
Assessing the benefits of cooperation
Currently there is very little guidance on approaches to assessing the benefits of
university-business cooperation activities in the field of education The existing
literature is strongly inclined to cooperation in the field of research and innovation
What examples can be found tend to stress measures of inputs and student outcomes
The wider literature can however provide valuable insights This incudes highlighting
the value of less tangible benefits such as contact between students company staff
and university staff often termed as productive interactions
Monitoring and assessment has a number of potential benefits it provides a means to
assess whether a project or programme is performing as expected it enables a
project leader to identify what works and what does not it provides a useful
mechanism to feedback lessons to others and if undertaken in advance it can
provide a means for selecting between different potential options In the evidence
available to this project the main approaches to monitoring and assessment have
focused on project performance often with an emphasis on measures of inputs and
outputs However there is also evidence of the use of results-based formats often of
a qualitative nature
The evidence available from the cases explored in this work suggests that monitoring
activities are undertaken at two levels in the field of university-business collaboration
in the field of education
Firstly through the use of standard student-orientated metrics
Secondly through informal metrics based around joint-perceptions of
success
The cases also suggest that there are a range of outcomes of university-business
collaboration activities in the field of education that are not currently being captured
by existing monitoring and measurement arrangements such as the potential
upskilling of existing staff through their involvement in the educational experience or
the raising of the profile of participating institutions However in no case was this felt
to weaken the cooperation arrangements underway Indeed to the contrary the
suggestion was that widening the formal measurement and monitoring process to
include more indicators would incur an additional burden for limited gains It is
essential that assessment methods are not only fit for purpose but are also seen to be
so
Whilst there are many methods that can be used to assess the outcomes of a
particular cooperation activity (whether that is a project or a programme) the
balanced scorecard approach seems particularly relevant The central idea of the
balanced scorecard is that several metrics can be used to evaluate the process in
question A causal map for educational university-industry collaboration initiatives is
given in Figure ES3 Four areas of measurement form the essential components of a
scorecard for university-industry collaboration The people involved the resources
available the activities undertaken and the results achieved
2014 11
Figure ES3 Scorecard general structure
Although each participant may have their own scorecard for the cooperation (either
held explicitly or implicitly) a combined scorecard can be drawn up for each co-
operation project in the form of a logical framework This allows assumptions to be
made explicit expected relationships to be identified and tests whether the
cooperation planned can meet its intended purpose(s) A stylized logic chain is set out
in Figure ES4
Figure ES4 Stylised causal map for university-industry collaboration
Alongside the scorecard approach an assessment methodology may also wish to set
out the framework for monitoring and evaluation Where evaluation may relate to the
assessment of project applications or the success of a completed project Here a
simple framework such as the success map set out in Figure ES5 may be
advantageous
People
bullThe qualification fit and motivation of the people taking part in the cooperation
Benefits
bullThe beneficial outputs that are generated as a result of the collaboration
Resources
bullThe amount and quality of the resources dedicated to the collaboration
Activities
bullThe quality and appropriateness of processes and organizational set-up
People
bullInstructors students managers
Resources
bullFunding equipment technology
Activities
bullOrganizational processes and structures
Benefits
bullEducation skills employment
2014 12
Figure ES5 Success map for cooperation in the field of education
In developing such a success map due regard should be given to four purposes of
university-business cooperation in the field of education broadly stated
Those that relate to teaching and learning aimed at strengthening the
labour supply and employment
Those that relate to stimulating entrepreneurship through teaching and
learning
Those that relate to knowledge exchange aimed at promoting
productprocess innovation or new pedagogy
Those that relate to raising the profile of an organization
In seeking to assess an actual project the framework can ensure that appropriate
indicators are selected and measurement approaches adopted It is beyond the scope
of this report to suggest a standard set of indicators as this will vary by project
purpose However a range of potential indicators have been identified relating to
each of the above purposes and which might be used to monitor inputs activities
outputs and outcomes Examples of potential indicators are illustrated in Figure ES5
In selecting indicators a number of principles should be observed including those of
parsimony (to seek to limit the data burden) and proportionality (data collected should
not incur costs disproportionate to the benefits received) The collection of data does
incur costs and for an assessment process to operate successfully there must be
agreement of all parties as to the role of particular indicators and the value of their
measurement
Interaction opportunities
Access to resources
Motivated educators
High-quality students
Relevant course delivery
Relevant course design
New ideas developed
Skilled amp trained labour
New pedagogies
Student applicant numbersquality
Non-academic organisations involved
Time inputs
Finance inputs
Newimproved courses
Number of graduates
Student attainment
Entrepreneurial attitudes
New products processes developed
Joint objective setting
Internshipsplacements
Co-delivery and assessment
Interaction intensity
Graduate employment
New business starts
Employer satisfaction
Organisational ranking in lsquoleaguersquo tables
Successful alliance
Human capital
Entrepreneurship
New ideas
Profile raising
Input Activity Output Outcome
2014 13
In the selection of indicators full consideration should also be given to the role of
qualitative measures of success and the potential means of capturing these This
might be through survey techniques but can also be captured by structured review
meetings with inputs from the various parties The value of self-assessment
techniques should also be considered Quantitative measures should not be over-
looked and where these can be part of wider measurement processes should be
welcomed
Implications for policy and practice
This study has proposed a flexible approach to measuring the impact of university-
business cooperation activities using a balanced scorecard approach allied with a
success-map for cooperation activities This recognizes the multiple purposes that
underlie cooperation activities and acknowledges that a one-size-fits-all approach is
unlikely to prove fit for purpose What are the benefits and implications of the
approach proposed to policy-makers and to the collaboration partners themselves
For policy-makers the tool can be used in a variety of manners
In the first instance it forms a framework for the ex-ante development of
the proposed programme By identifying the resources that are available to
the programme or might be available and the desired objectives
programme officials can identify the types of activity that might lead to the
desired results Through a process of iteration the scorecard can be refined
further This can then provide a baseline for a call for applications
The scorecard approach might then be used for the assessment or
appraisal of applications Assessors can check the applications against the
programme scorecard to identify the extent to which each applicant
contributes to the desired approach and objectives In a dynamic approach
the programme scorecard can be updated through the inclusion of novel
and innovative approaches proposed by applicants which were not foreseen
during the development of the programme A blank scorecard template
might also be completed by assessors as a means of summarising the
information contained in applications in a common format The approach
could highlight missed opportunities for collaborative activities or
resourcepeople inputs which can form the basis of negotiation with
programme applicants
Following a successful application a project scorecard can then form the
basis for the review of project progress This can examine the activities
undertaken resources expended and people involved against what was
initially planned together with the benefits realized compared to those that
were expected The review meeting can then explore the reasons for any
differences examining cases where inputs are greater than anticipated
perhaps owing to the cooperation going better than planned or where
results or inputs are less than originally proposed and whether remedial
action needs to be taken
2014 14
The scorecard approach can also deliver significant benefits to the collaboration
partners themselves
As a starting point it can form the basis for robust project design The
scorecard can be used as a mechanism for each partner to set out what
they wish to achieve from the cooperation activity and what they are able
to contribute Each partner can then identify what they expect of the other
partners Using the scorecard approach can also assist in identifying where
potential opportunities might have been overlooked in the original project
design and where small changes might reap large rewards This can then
form the basis of the collaboration agreement between parties
Project partners can then use their own stakeholder scorecard as a means
for internal review and monitoring of the collaboration activity This
provides a useful mechanism for identifying whether desired outcomes are
being achieved and the extent to which anticipated inputs and activities are
occurring As a dynamic process the scorecard can be updated as
activities are added or amended or other inputs change
The partnership as a whole can also use the scorecard as the basis for
ongoing project reviews This can consider whether the original
assumptions remain valid and whether some activities are more appropriate
than others in achieving the desired objectives Crucial to the review
process is to ensure that all partners are realizing the benefits that they
sought initially and if not how this might be rectified The scorecard
approach provides a means of structuring this review process
In order to promote the use of the scorecard approach the Commission may wish to
provide a guide to its use in the context of university-business cooperation activities
This would offer the opportunity to develop worked examples tailored to different
cooperation purposes Such a guide could also incorporate advice on practical
techniques for measuring the outcomes of cooperation activities In the first instance
this could form part of a toolkit available to assist those parties interested in applying
for Knowledge Alliance projects
HOW TO OBTAIN EU PUBLICATIONS
Free publications
bull one copy
via EU Bookshop (httpbookshopeuropaeu)
bull more than one copy or postersmaps
from the European Unionrsquos representations (httpeceuropaeurepresent_enhtm)
from the delegations in non-EU countries (httpeeaseuropaeudelegationsindex_enhtm)
by contacting the Europe Direct service (httpeuropaeueuropedirectindex_enhtm) or
calling 00 800 6 7 8 9 10 11 (freephone number from anywhere in the EU) ()
() The information given is free as are most calls (though some operators phone boxes
or hotels may charge you)
Priced publications
bull via EU Bookshop (httpbookshopeuropaeu)
Priced subscriptions
bull via one of the sales agents of the Publications Office of the European Union
(httppublicationseuropaeuothersagentsindex_enhtm)
NC-0
1-1
4-2
99-E
N-N
doi10
2014 6
To analyse existing and identify emerging types of university-business
cooperation in Europe and their drivers
To identify demonstrate and assess tangible and intangible effects of
university-business-cooperation
To develop an assessment methodology and relevant qualitative and
quantitative indicators for measuring the outcomes and impact of university-
business cooperation and in particular of Knowledge Alliances
The study has analysed the following ten cases
Educckate multinational - led by the University College London in
partnership with other EU Universities and their local companies to
stimulate entrepreneurship in cultural and creative industries
EUEN multinational ndash led by Coventry University with transnational
academic and industry partners to promote entrepreneurial education
KnowFact multinational ndash led by the University of Patras with industry
partners from other EU countries to develop the Teaching Factory paradigm
in manufacturing education
AppCampus Finland ndash an innovative initiative between Aalto University and
NokiaMicrosoft involving app Development National
Qatar Carbonates UK ndash a collaborative arrangement between Qatar
PetroleumShell and Imperial College London promoting post-graduate
education International
Newcastle University Subsea UK ndash cooperation between Newcastle
University and regional businesses developing sector relevant skills
Regional
Masters in Banking Valencia Spain ndash collaboration between Valencia
University and regional banking bodies developing sector relevant skills
Regional
HP Bulgaria ndash Cooperation between Hewlett Packard and various
Universities whereby HP trained university lecturers in specialised
technologies and created the project curriculum National
University of Merseburg Germany ndash regional businesses endow professorial
chairs at University of Merseburg to ensure industry relevant education
AKA Prague ndash professional qualification (Certificate of Communication
Agencies) developed by industry and delivered by University of Economics
Prague
Types and drivers of cooperation
The first decade of the 21st Century has seen an increasing emphasis on the role of
universities in explicitly contributing to social and economic development This can be
attributed to a number of factors including
2014 7
The emergence of global lsquogrand challengesrsquo (eg climate change ageing
terrorism etc) which cannot be solved by government or business alone
but requires a multi-disciplinary and collaborative approach
Increased marketisation of higher education leading to greater competition
between universities and an emphasis on the lsquostudent experiencersquo and
student outcomes (including entrepreneurship)
The global economic crisis which has ushered in an era of austerity in public
finances in many countries
The recognition by many firms that working with Universities provides an
advantage for their future recruitment pipelines and ancillary profile
benefits
This has resulted in a range of policies designed to encourage universities to build
stronger links with business suggesting that businesses should be (more) involved in
designing curricula (including undergraduate postgraduate as well as CPD and other
training) and that universities should work more closely with industry partners to
promote entrepreneurship mobility (between business and academia) and lifelong
learning
In practice many universities and businesses have recognized the value of
cooperation particularly in terms of how this can strengthen student recruitment for
universities improve the educational experience for students and provide a supply of
more experienced labour for companies Some companies and universities also
recognize the wider benefits that this cooperation can bring in the form of higher
levels of innovation the upskilling and retention of existing staff and the higher profile
that successful collaboration activities can bring
A number of reports have usefully set out different lsquotypesrsquo of cooperation between
businesses and universities including student and academic mobility programmes
curriculum development and delivery lifelong learning and entrepreneurship4
However most cooperation activities examined for this work incorporated more than
one of these cooperation types This suggests that it is the purpose of cooperation
that is more important rather than the type of cooperation undertaken
Each cooperation partner (business university student) has their own reason to
engage in the cooperation activity but a number of common themes can be identified
These are illustrated in Figure ES1 Three of these relate to strengthening levels of
human capital either of students or of existing workers within companies and
universities to promote a stronger labour supply enhance individual employability
and heighten levels of entrepreneurship These were common across all the projects
examined for this study Three are more related to innovation primarily through
collaborative working on company projects as part of the educational experience or
through the co-production of new courses and course material these were visible in
each of the cases studied but to varying degrees Finally the remaining three
4 See for example Wilson T (2012) A Review of University Business Collaboration and Science-to-Business Marketing Research Centre (2011) The State of European University Business Cooperation European Commission
2014 8
reasons relate to developing longer-term contacts and the profile of the industry
companies and universities involved for future potential benefits
Figure ES1 Purposes underlying cooperation activities
Benefits of cooperation
The potential benefits of collaboration activities in the field of education can take many
different forms (Figure ES2) Very often it is the qualitative benefits that cooperation
activities bring (such as the deepening of skill sets) that are valued by participants
rather than quantitative benefits (such as increasing the numbers of graduates)
although this does also feature
Figure ES2 Four forms of outcomes
Some benefits are more immediate and occur directly following the end of the
successful completion of a course of studies (such as the number of graduates) or
during it (such as through engagement of staff and students on joint activities) whilst
others may be longer-term (such as new business starts arising out of stronger
entrepreneurial attitudes) Some benefits are direct (such as the creation of a new
Secure labour supply enhance employability
Knowledge exchangeinnov
ation Build contacts
Entrepreneurial skillsattitudes
Pedagogic development
Raise profilebrand
Staff retentionupskill
ing
Institutional modernisation
Attract students
Quantitative vs
qualitative
Immediate vs longer
term
Tangible vs Intangible
Direct vs Indirect
2014 9
course) whilst others are more indirect (such as the value of contacts gained for
potential future research projects) Equally some benefits are very tangible (such as
when a student develops a particular skill) but others are much less so (such as where
student knowledge spills over to workers in a company who work alongside them on a
project) Examples of all these potential benefits were visible throughout the cases
explored for this study
In practice our study illustrates a focus on immediate highly-visible returns by 10
university-business cooperation projects but also emphasises the significance of less
tangible and more indirect returns to participants with different participants placing
differential values on particular outcomes For example in the case of the North East
England sub-sea collaboration businesses were keen to fill an emerging skills gap
upskill staff and develop contacts with the university and other businesses For the
academics it was a means to generate additional student numbers to engage with
the business sector and to explore further research avenues
For students the reason they wished to participate in these collaborative offers in the
field of education was primarily related to the fact they felt it would provide them with
greater skills than a more conventional course that they were more likely to find
subsequent employment or that it would provide them with enhanced entrepreneurial
opportunities
For universities and academics there was a similar emphasis on the enhanced skills
acquired by students the greater employability of these students and the stronger
entrepreneurial outcomes anticipated as highlighted in the EUEN project These
outcomes were expected to feed through into an increase in either the quality or the
quantity of student applications with improvements in student perceptions of course
quality also anticipated Additional outcomes which were cited in some instances
included the opportunity provided for increasing contacts in the business world the
updating of skills through exposure to contemporary businesses practices and the
building of trust-based relationships for future collaborative research
For businesses the primary outcome reported was in terms of an enhanced pipeline of
suitably skilled potential employees together with improvements to the wider labour
force There were also particular outcomes specific to individual cooperation
arrangements such as in the case of AppCampus where one measure of success is the
number of applications developed Businesses also reported that the cooperation
arrangements provided benefits in the form of raising their profile but also helped to
build trust and wider contact networks with academics for future collaborative
research Firms also reported that involvement in live projects did provide the
opportunity for product and process innovations adding economic value to the
company and that the bringing together of students and existing employees served to
raise the knowledge levels of employees as well as students
For other parties such as regional authorities the anticipated (and realised) outcomes
largely revolved around economic benefits such as an enhanced labour supply
increased levels of graduate employment higher levels of new business-starts or
(rarely) higher levels of innovation
2014 10
Assessing the benefits of cooperation
Currently there is very little guidance on approaches to assessing the benefits of
university-business cooperation activities in the field of education The existing
literature is strongly inclined to cooperation in the field of research and innovation
What examples can be found tend to stress measures of inputs and student outcomes
The wider literature can however provide valuable insights This incudes highlighting
the value of less tangible benefits such as contact between students company staff
and university staff often termed as productive interactions
Monitoring and assessment has a number of potential benefits it provides a means to
assess whether a project or programme is performing as expected it enables a
project leader to identify what works and what does not it provides a useful
mechanism to feedback lessons to others and if undertaken in advance it can
provide a means for selecting between different potential options In the evidence
available to this project the main approaches to monitoring and assessment have
focused on project performance often with an emphasis on measures of inputs and
outputs However there is also evidence of the use of results-based formats often of
a qualitative nature
The evidence available from the cases explored in this work suggests that monitoring
activities are undertaken at two levels in the field of university-business collaboration
in the field of education
Firstly through the use of standard student-orientated metrics
Secondly through informal metrics based around joint-perceptions of
success
The cases also suggest that there are a range of outcomes of university-business
collaboration activities in the field of education that are not currently being captured
by existing monitoring and measurement arrangements such as the potential
upskilling of existing staff through their involvement in the educational experience or
the raising of the profile of participating institutions However in no case was this felt
to weaken the cooperation arrangements underway Indeed to the contrary the
suggestion was that widening the formal measurement and monitoring process to
include more indicators would incur an additional burden for limited gains It is
essential that assessment methods are not only fit for purpose but are also seen to be
so
Whilst there are many methods that can be used to assess the outcomes of a
particular cooperation activity (whether that is a project or a programme) the
balanced scorecard approach seems particularly relevant The central idea of the
balanced scorecard is that several metrics can be used to evaluate the process in
question A causal map for educational university-industry collaboration initiatives is
given in Figure ES3 Four areas of measurement form the essential components of a
scorecard for university-industry collaboration The people involved the resources
available the activities undertaken and the results achieved
2014 11
Figure ES3 Scorecard general structure
Although each participant may have their own scorecard for the cooperation (either
held explicitly or implicitly) a combined scorecard can be drawn up for each co-
operation project in the form of a logical framework This allows assumptions to be
made explicit expected relationships to be identified and tests whether the
cooperation planned can meet its intended purpose(s) A stylized logic chain is set out
in Figure ES4
Figure ES4 Stylised causal map for university-industry collaboration
Alongside the scorecard approach an assessment methodology may also wish to set
out the framework for monitoring and evaluation Where evaluation may relate to the
assessment of project applications or the success of a completed project Here a
simple framework such as the success map set out in Figure ES5 may be
advantageous
People
bullThe qualification fit and motivation of the people taking part in the cooperation
Benefits
bullThe beneficial outputs that are generated as a result of the collaboration
Resources
bullThe amount and quality of the resources dedicated to the collaboration
Activities
bullThe quality and appropriateness of processes and organizational set-up
People
bullInstructors students managers
Resources
bullFunding equipment technology
Activities
bullOrganizational processes and structures
Benefits
bullEducation skills employment
2014 12
Figure ES5 Success map for cooperation in the field of education
In developing such a success map due regard should be given to four purposes of
university-business cooperation in the field of education broadly stated
Those that relate to teaching and learning aimed at strengthening the
labour supply and employment
Those that relate to stimulating entrepreneurship through teaching and
learning
Those that relate to knowledge exchange aimed at promoting
productprocess innovation or new pedagogy
Those that relate to raising the profile of an organization
In seeking to assess an actual project the framework can ensure that appropriate
indicators are selected and measurement approaches adopted It is beyond the scope
of this report to suggest a standard set of indicators as this will vary by project
purpose However a range of potential indicators have been identified relating to
each of the above purposes and which might be used to monitor inputs activities
outputs and outcomes Examples of potential indicators are illustrated in Figure ES5
In selecting indicators a number of principles should be observed including those of
parsimony (to seek to limit the data burden) and proportionality (data collected should
not incur costs disproportionate to the benefits received) The collection of data does
incur costs and for an assessment process to operate successfully there must be
agreement of all parties as to the role of particular indicators and the value of their
measurement
Interaction opportunities
Access to resources
Motivated educators
High-quality students
Relevant course delivery
Relevant course design
New ideas developed
Skilled amp trained labour
New pedagogies
Student applicant numbersquality
Non-academic organisations involved
Time inputs
Finance inputs
Newimproved courses
Number of graduates
Student attainment
Entrepreneurial attitudes
New products processes developed
Joint objective setting
Internshipsplacements
Co-delivery and assessment
Interaction intensity
Graduate employment
New business starts
Employer satisfaction
Organisational ranking in lsquoleaguersquo tables
Successful alliance
Human capital
Entrepreneurship
New ideas
Profile raising
Input Activity Output Outcome
2014 13
In the selection of indicators full consideration should also be given to the role of
qualitative measures of success and the potential means of capturing these This
might be through survey techniques but can also be captured by structured review
meetings with inputs from the various parties The value of self-assessment
techniques should also be considered Quantitative measures should not be over-
looked and where these can be part of wider measurement processes should be
welcomed
Implications for policy and practice
This study has proposed a flexible approach to measuring the impact of university-
business cooperation activities using a balanced scorecard approach allied with a
success-map for cooperation activities This recognizes the multiple purposes that
underlie cooperation activities and acknowledges that a one-size-fits-all approach is
unlikely to prove fit for purpose What are the benefits and implications of the
approach proposed to policy-makers and to the collaboration partners themselves
For policy-makers the tool can be used in a variety of manners
In the first instance it forms a framework for the ex-ante development of
the proposed programme By identifying the resources that are available to
the programme or might be available and the desired objectives
programme officials can identify the types of activity that might lead to the
desired results Through a process of iteration the scorecard can be refined
further This can then provide a baseline for a call for applications
The scorecard approach might then be used for the assessment or
appraisal of applications Assessors can check the applications against the
programme scorecard to identify the extent to which each applicant
contributes to the desired approach and objectives In a dynamic approach
the programme scorecard can be updated through the inclusion of novel
and innovative approaches proposed by applicants which were not foreseen
during the development of the programme A blank scorecard template
might also be completed by assessors as a means of summarising the
information contained in applications in a common format The approach
could highlight missed opportunities for collaborative activities or
resourcepeople inputs which can form the basis of negotiation with
programme applicants
Following a successful application a project scorecard can then form the
basis for the review of project progress This can examine the activities
undertaken resources expended and people involved against what was
initially planned together with the benefits realized compared to those that
were expected The review meeting can then explore the reasons for any
differences examining cases where inputs are greater than anticipated
perhaps owing to the cooperation going better than planned or where
results or inputs are less than originally proposed and whether remedial
action needs to be taken
2014 14
The scorecard approach can also deliver significant benefits to the collaboration
partners themselves
As a starting point it can form the basis for robust project design The
scorecard can be used as a mechanism for each partner to set out what
they wish to achieve from the cooperation activity and what they are able
to contribute Each partner can then identify what they expect of the other
partners Using the scorecard approach can also assist in identifying where
potential opportunities might have been overlooked in the original project
design and where small changes might reap large rewards This can then
form the basis of the collaboration agreement between parties
Project partners can then use their own stakeholder scorecard as a means
for internal review and monitoring of the collaboration activity This
provides a useful mechanism for identifying whether desired outcomes are
being achieved and the extent to which anticipated inputs and activities are
occurring As a dynamic process the scorecard can be updated as
activities are added or amended or other inputs change
The partnership as a whole can also use the scorecard as the basis for
ongoing project reviews This can consider whether the original
assumptions remain valid and whether some activities are more appropriate
than others in achieving the desired objectives Crucial to the review
process is to ensure that all partners are realizing the benefits that they
sought initially and if not how this might be rectified The scorecard
approach provides a means of structuring this review process
In order to promote the use of the scorecard approach the Commission may wish to
provide a guide to its use in the context of university-business cooperation activities
This would offer the opportunity to develop worked examples tailored to different
cooperation purposes Such a guide could also incorporate advice on practical
techniques for measuring the outcomes of cooperation activities In the first instance
this could form part of a toolkit available to assist those parties interested in applying
for Knowledge Alliance projects
HOW TO OBTAIN EU PUBLICATIONS
Free publications
bull one copy
via EU Bookshop (httpbookshopeuropaeu)
bull more than one copy or postersmaps
from the European Unionrsquos representations (httpeceuropaeurepresent_enhtm)
from the delegations in non-EU countries (httpeeaseuropaeudelegationsindex_enhtm)
by contacting the Europe Direct service (httpeuropaeueuropedirectindex_enhtm) or
calling 00 800 6 7 8 9 10 11 (freephone number from anywhere in the EU) ()
() The information given is free as are most calls (though some operators phone boxes
or hotels may charge you)
Priced publications
bull via EU Bookshop (httpbookshopeuropaeu)
Priced subscriptions
bull via one of the sales agents of the Publications Office of the European Union
(httppublicationseuropaeuothersagentsindex_enhtm)
NC-0
1-1
4-2
99-E
N-N
doi10
2014 7
The emergence of global lsquogrand challengesrsquo (eg climate change ageing
terrorism etc) which cannot be solved by government or business alone
but requires a multi-disciplinary and collaborative approach
Increased marketisation of higher education leading to greater competition
between universities and an emphasis on the lsquostudent experiencersquo and
student outcomes (including entrepreneurship)
The global economic crisis which has ushered in an era of austerity in public
finances in many countries
The recognition by many firms that working with Universities provides an
advantage for their future recruitment pipelines and ancillary profile
benefits
This has resulted in a range of policies designed to encourage universities to build
stronger links with business suggesting that businesses should be (more) involved in
designing curricula (including undergraduate postgraduate as well as CPD and other
training) and that universities should work more closely with industry partners to
promote entrepreneurship mobility (between business and academia) and lifelong
learning
In practice many universities and businesses have recognized the value of
cooperation particularly in terms of how this can strengthen student recruitment for
universities improve the educational experience for students and provide a supply of
more experienced labour for companies Some companies and universities also
recognize the wider benefits that this cooperation can bring in the form of higher
levels of innovation the upskilling and retention of existing staff and the higher profile
that successful collaboration activities can bring
A number of reports have usefully set out different lsquotypesrsquo of cooperation between
businesses and universities including student and academic mobility programmes
curriculum development and delivery lifelong learning and entrepreneurship4
However most cooperation activities examined for this work incorporated more than
one of these cooperation types This suggests that it is the purpose of cooperation
that is more important rather than the type of cooperation undertaken
Each cooperation partner (business university student) has their own reason to
engage in the cooperation activity but a number of common themes can be identified
These are illustrated in Figure ES1 Three of these relate to strengthening levels of
human capital either of students or of existing workers within companies and
universities to promote a stronger labour supply enhance individual employability
and heighten levels of entrepreneurship These were common across all the projects
examined for this study Three are more related to innovation primarily through
collaborative working on company projects as part of the educational experience or
through the co-production of new courses and course material these were visible in
each of the cases studied but to varying degrees Finally the remaining three
4 See for example Wilson T (2012) A Review of University Business Collaboration and Science-to-Business Marketing Research Centre (2011) The State of European University Business Cooperation European Commission
2014 8
reasons relate to developing longer-term contacts and the profile of the industry
companies and universities involved for future potential benefits
Figure ES1 Purposes underlying cooperation activities
Benefits of cooperation
The potential benefits of collaboration activities in the field of education can take many
different forms (Figure ES2) Very often it is the qualitative benefits that cooperation
activities bring (such as the deepening of skill sets) that are valued by participants
rather than quantitative benefits (such as increasing the numbers of graduates)
although this does also feature
Figure ES2 Four forms of outcomes
Some benefits are more immediate and occur directly following the end of the
successful completion of a course of studies (such as the number of graduates) or
during it (such as through engagement of staff and students on joint activities) whilst
others may be longer-term (such as new business starts arising out of stronger
entrepreneurial attitudes) Some benefits are direct (such as the creation of a new
Secure labour supply enhance employability
Knowledge exchangeinnov
ation Build contacts
Entrepreneurial skillsattitudes
Pedagogic development
Raise profilebrand
Staff retentionupskill
ing
Institutional modernisation
Attract students
Quantitative vs
qualitative
Immediate vs longer
term
Tangible vs Intangible
Direct vs Indirect
2014 9
course) whilst others are more indirect (such as the value of contacts gained for
potential future research projects) Equally some benefits are very tangible (such as
when a student develops a particular skill) but others are much less so (such as where
student knowledge spills over to workers in a company who work alongside them on a
project) Examples of all these potential benefits were visible throughout the cases
explored for this study
In practice our study illustrates a focus on immediate highly-visible returns by 10
university-business cooperation projects but also emphasises the significance of less
tangible and more indirect returns to participants with different participants placing
differential values on particular outcomes For example in the case of the North East
England sub-sea collaboration businesses were keen to fill an emerging skills gap
upskill staff and develop contacts with the university and other businesses For the
academics it was a means to generate additional student numbers to engage with
the business sector and to explore further research avenues
For students the reason they wished to participate in these collaborative offers in the
field of education was primarily related to the fact they felt it would provide them with
greater skills than a more conventional course that they were more likely to find
subsequent employment or that it would provide them with enhanced entrepreneurial
opportunities
For universities and academics there was a similar emphasis on the enhanced skills
acquired by students the greater employability of these students and the stronger
entrepreneurial outcomes anticipated as highlighted in the EUEN project These
outcomes were expected to feed through into an increase in either the quality or the
quantity of student applications with improvements in student perceptions of course
quality also anticipated Additional outcomes which were cited in some instances
included the opportunity provided for increasing contacts in the business world the
updating of skills through exposure to contemporary businesses practices and the
building of trust-based relationships for future collaborative research
For businesses the primary outcome reported was in terms of an enhanced pipeline of
suitably skilled potential employees together with improvements to the wider labour
force There were also particular outcomes specific to individual cooperation
arrangements such as in the case of AppCampus where one measure of success is the
number of applications developed Businesses also reported that the cooperation
arrangements provided benefits in the form of raising their profile but also helped to
build trust and wider contact networks with academics for future collaborative
research Firms also reported that involvement in live projects did provide the
opportunity for product and process innovations adding economic value to the
company and that the bringing together of students and existing employees served to
raise the knowledge levels of employees as well as students
For other parties such as regional authorities the anticipated (and realised) outcomes
largely revolved around economic benefits such as an enhanced labour supply
increased levels of graduate employment higher levels of new business-starts or
(rarely) higher levels of innovation
2014 10
Assessing the benefits of cooperation
Currently there is very little guidance on approaches to assessing the benefits of
university-business cooperation activities in the field of education The existing
literature is strongly inclined to cooperation in the field of research and innovation
What examples can be found tend to stress measures of inputs and student outcomes
The wider literature can however provide valuable insights This incudes highlighting
the value of less tangible benefits such as contact between students company staff
and university staff often termed as productive interactions
Monitoring and assessment has a number of potential benefits it provides a means to
assess whether a project or programme is performing as expected it enables a
project leader to identify what works and what does not it provides a useful
mechanism to feedback lessons to others and if undertaken in advance it can
provide a means for selecting between different potential options In the evidence
available to this project the main approaches to monitoring and assessment have
focused on project performance often with an emphasis on measures of inputs and
outputs However there is also evidence of the use of results-based formats often of
a qualitative nature
The evidence available from the cases explored in this work suggests that monitoring
activities are undertaken at two levels in the field of university-business collaboration
in the field of education
Firstly through the use of standard student-orientated metrics
Secondly through informal metrics based around joint-perceptions of
success
The cases also suggest that there are a range of outcomes of university-business
collaboration activities in the field of education that are not currently being captured
by existing monitoring and measurement arrangements such as the potential
upskilling of existing staff through their involvement in the educational experience or
the raising of the profile of participating institutions However in no case was this felt
to weaken the cooperation arrangements underway Indeed to the contrary the
suggestion was that widening the formal measurement and monitoring process to
include more indicators would incur an additional burden for limited gains It is
essential that assessment methods are not only fit for purpose but are also seen to be
so
Whilst there are many methods that can be used to assess the outcomes of a
particular cooperation activity (whether that is a project or a programme) the
balanced scorecard approach seems particularly relevant The central idea of the
balanced scorecard is that several metrics can be used to evaluate the process in
question A causal map for educational university-industry collaboration initiatives is
given in Figure ES3 Four areas of measurement form the essential components of a
scorecard for university-industry collaboration The people involved the resources
available the activities undertaken and the results achieved
2014 11
Figure ES3 Scorecard general structure
Although each participant may have their own scorecard for the cooperation (either
held explicitly or implicitly) a combined scorecard can be drawn up for each co-
operation project in the form of a logical framework This allows assumptions to be
made explicit expected relationships to be identified and tests whether the
cooperation planned can meet its intended purpose(s) A stylized logic chain is set out
in Figure ES4
Figure ES4 Stylised causal map for university-industry collaboration
Alongside the scorecard approach an assessment methodology may also wish to set
out the framework for monitoring and evaluation Where evaluation may relate to the
assessment of project applications or the success of a completed project Here a
simple framework such as the success map set out in Figure ES5 may be
advantageous
People
bullThe qualification fit and motivation of the people taking part in the cooperation
Benefits
bullThe beneficial outputs that are generated as a result of the collaboration
Resources
bullThe amount and quality of the resources dedicated to the collaboration
Activities
bullThe quality and appropriateness of processes and organizational set-up
People
bullInstructors students managers
Resources
bullFunding equipment technology
Activities
bullOrganizational processes and structures
Benefits
bullEducation skills employment
2014 12
Figure ES5 Success map for cooperation in the field of education
In developing such a success map due regard should be given to four purposes of
university-business cooperation in the field of education broadly stated
Those that relate to teaching and learning aimed at strengthening the
labour supply and employment
Those that relate to stimulating entrepreneurship through teaching and
learning
Those that relate to knowledge exchange aimed at promoting
productprocess innovation or new pedagogy
Those that relate to raising the profile of an organization
In seeking to assess an actual project the framework can ensure that appropriate
indicators are selected and measurement approaches adopted It is beyond the scope
of this report to suggest a standard set of indicators as this will vary by project
purpose However a range of potential indicators have been identified relating to
each of the above purposes and which might be used to monitor inputs activities
outputs and outcomes Examples of potential indicators are illustrated in Figure ES5
In selecting indicators a number of principles should be observed including those of
parsimony (to seek to limit the data burden) and proportionality (data collected should
not incur costs disproportionate to the benefits received) The collection of data does
incur costs and for an assessment process to operate successfully there must be
agreement of all parties as to the role of particular indicators and the value of their
measurement
Interaction opportunities
Access to resources
Motivated educators
High-quality students
Relevant course delivery
Relevant course design
New ideas developed
Skilled amp trained labour
New pedagogies
Student applicant numbersquality
Non-academic organisations involved
Time inputs
Finance inputs
Newimproved courses
Number of graduates
Student attainment
Entrepreneurial attitudes
New products processes developed
Joint objective setting
Internshipsplacements
Co-delivery and assessment
Interaction intensity
Graduate employment
New business starts
Employer satisfaction
Organisational ranking in lsquoleaguersquo tables
Successful alliance
Human capital
Entrepreneurship
New ideas
Profile raising
Input Activity Output Outcome
2014 13
In the selection of indicators full consideration should also be given to the role of
qualitative measures of success and the potential means of capturing these This
might be through survey techniques but can also be captured by structured review
meetings with inputs from the various parties The value of self-assessment
techniques should also be considered Quantitative measures should not be over-
looked and where these can be part of wider measurement processes should be
welcomed
Implications for policy and practice
This study has proposed a flexible approach to measuring the impact of university-
business cooperation activities using a balanced scorecard approach allied with a
success-map for cooperation activities This recognizes the multiple purposes that
underlie cooperation activities and acknowledges that a one-size-fits-all approach is
unlikely to prove fit for purpose What are the benefits and implications of the
approach proposed to policy-makers and to the collaboration partners themselves
For policy-makers the tool can be used in a variety of manners
In the first instance it forms a framework for the ex-ante development of
the proposed programme By identifying the resources that are available to
the programme or might be available and the desired objectives
programme officials can identify the types of activity that might lead to the
desired results Through a process of iteration the scorecard can be refined
further This can then provide a baseline for a call for applications
The scorecard approach might then be used for the assessment or
appraisal of applications Assessors can check the applications against the
programme scorecard to identify the extent to which each applicant
contributes to the desired approach and objectives In a dynamic approach
the programme scorecard can be updated through the inclusion of novel
and innovative approaches proposed by applicants which were not foreseen
during the development of the programme A blank scorecard template
might also be completed by assessors as a means of summarising the
information contained in applications in a common format The approach
could highlight missed opportunities for collaborative activities or
resourcepeople inputs which can form the basis of negotiation with
programme applicants
Following a successful application a project scorecard can then form the
basis for the review of project progress This can examine the activities
undertaken resources expended and people involved against what was
initially planned together with the benefits realized compared to those that
were expected The review meeting can then explore the reasons for any
differences examining cases where inputs are greater than anticipated
perhaps owing to the cooperation going better than planned or where
results or inputs are less than originally proposed and whether remedial
action needs to be taken
2014 14
The scorecard approach can also deliver significant benefits to the collaboration
partners themselves
As a starting point it can form the basis for robust project design The
scorecard can be used as a mechanism for each partner to set out what
they wish to achieve from the cooperation activity and what they are able
to contribute Each partner can then identify what they expect of the other
partners Using the scorecard approach can also assist in identifying where
potential opportunities might have been overlooked in the original project
design and where small changes might reap large rewards This can then
form the basis of the collaboration agreement between parties
Project partners can then use their own stakeholder scorecard as a means
for internal review and monitoring of the collaboration activity This
provides a useful mechanism for identifying whether desired outcomes are
being achieved and the extent to which anticipated inputs and activities are
occurring As a dynamic process the scorecard can be updated as
activities are added or amended or other inputs change
The partnership as a whole can also use the scorecard as the basis for
ongoing project reviews This can consider whether the original
assumptions remain valid and whether some activities are more appropriate
than others in achieving the desired objectives Crucial to the review
process is to ensure that all partners are realizing the benefits that they
sought initially and if not how this might be rectified The scorecard
approach provides a means of structuring this review process
In order to promote the use of the scorecard approach the Commission may wish to
provide a guide to its use in the context of university-business cooperation activities
This would offer the opportunity to develop worked examples tailored to different
cooperation purposes Such a guide could also incorporate advice on practical
techniques for measuring the outcomes of cooperation activities In the first instance
this could form part of a toolkit available to assist those parties interested in applying
for Knowledge Alliance projects
HOW TO OBTAIN EU PUBLICATIONS
Free publications
bull one copy
via EU Bookshop (httpbookshopeuropaeu)
bull more than one copy or postersmaps
from the European Unionrsquos representations (httpeceuropaeurepresent_enhtm)
from the delegations in non-EU countries (httpeeaseuropaeudelegationsindex_enhtm)
by contacting the Europe Direct service (httpeuropaeueuropedirectindex_enhtm) or
calling 00 800 6 7 8 9 10 11 (freephone number from anywhere in the EU) ()
() The information given is free as are most calls (though some operators phone boxes
or hotels may charge you)
Priced publications
bull via EU Bookshop (httpbookshopeuropaeu)
Priced subscriptions
bull via one of the sales agents of the Publications Office of the European Union
(httppublicationseuropaeuothersagentsindex_enhtm)
NC-0
1-1
4-2
99-E
N-N
doi10
2014 8
reasons relate to developing longer-term contacts and the profile of the industry
companies and universities involved for future potential benefits
Figure ES1 Purposes underlying cooperation activities
Benefits of cooperation
The potential benefits of collaboration activities in the field of education can take many
different forms (Figure ES2) Very often it is the qualitative benefits that cooperation
activities bring (such as the deepening of skill sets) that are valued by participants
rather than quantitative benefits (such as increasing the numbers of graduates)
although this does also feature
Figure ES2 Four forms of outcomes
Some benefits are more immediate and occur directly following the end of the
successful completion of a course of studies (such as the number of graduates) or
during it (such as through engagement of staff and students on joint activities) whilst
others may be longer-term (such as new business starts arising out of stronger
entrepreneurial attitudes) Some benefits are direct (such as the creation of a new
Secure labour supply enhance employability
Knowledge exchangeinnov
ation Build contacts
Entrepreneurial skillsattitudes
Pedagogic development
Raise profilebrand
Staff retentionupskill
ing
Institutional modernisation
Attract students
Quantitative vs
qualitative
Immediate vs longer
term
Tangible vs Intangible
Direct vs Indirect
2014 9
course) whilst others are more indirect (such as the value of contacts gained for
potential future research projects) Equally some benefits are very tangible (such as
when a student develops a particular skill) but others are much less so (such as where
student knowledge spills over to workers in a company who work alongside them on a
project) Examples of all these potential benefits were visible throughout the cases
explored for this study
In practice our study illustrates a focus on immediate highly-visible returns by 10
university-business cooperation projects but also emphasises the significance of less
tangible and more indirect returns to participants with different participants placing
differential values on particular outcomes For example in the case of the North East
England sub-sea collaboration businesses were keen to fill an emerging skills gap
upskill staff and develop contacts with the university and other businesses For the
academics it was a means to generate additional student numbers to engage with
the business sector and to explore further research avenues
For students the reason they wished to participate in these collaborative offers in the
field of education was primarily related to the fact they felt it would provide them with
greater skills than a more conventional course that they were more likely to find
subsequent employment or that it would provide them with enhanced entrepreneurial
opportunities
For universities and academics there was a similar emphasis on the enhanced skills
acquired by students the greater employability of these students and the stronger
entrepreneurial outcomes anticipated as highlighted in the EUEN project These
outcomes were expected to feed through into an increase in either the quality or the
quantity of student applications with improvements in student perceptions of course
quality also anticipated Additional outcomes which were cited in some instances
included the opportunity provided for increasing contacts in the business world the
updating of skills through exposure to contemporary businesses practices and the
building of trust-based relationships for future collaborative research
For businesses the primary outcome reported was in terms of an enhanced pipeline of
suitably skilled potential employees together with improvements to the wider labour
force There were also particular outcomes specific to individual cooperation
arrangements such as in the case of AppCampus where one measure of success is the
number of applications developed Businesses also reported that the cooperation
arrangements provided benefits in the form of raising their profile but also helped to
build trust and wider contact networks with academics for future collaborative
research Firms also reported that involvement in live projects did provide the
opportunity for product and process innovations adding economic value to the
company and that the bringing together of students and existing employees served to
raise the knowledge levels of employees as well as students
For other parties such as regional authorities the anticipated (and realised) outcomes
largely revolved around economic benefits such as an enhanced labour supply
increased levels of graduate employment higher levels of new business-starts or
(rarely) higher levels of innovation
2014 10
Assessing the benefits of cooperation
Currently there is very little guidance on approaches to assessing the benefits of
university-business cooperation activities in the field of education The existing
literature is strongly inclined to cooperation in the field of research and innovation
What examples can be found tend to stress measures of inputs and student outcomes
The wider literature can however provide valuable insights This incudes highlighting
the value of less tangible benefits such as contact between students company staff
and university staff often termed as productive interactions
Monitoring and assessment has a number of potential benefits it provides a means to
assess whether a project or programme is performing as expected it enables a
project leader to identify what works and what does not it provides a useful
mechanism to feedback lessons to others and if undertaken in advance it can
provide a means for selecting between different potential options In the evidence
available to this project the main approaches to monitoring and assessment have
focused on project performance often with an emphasis on measures of inputs and
outputs However there is also evidence of the use of results-based formats often of
a qualitative nature
The evidence available from the cases explored in this work suggests that monitoring
activities are undertaken at two levels in the field of university-business collaboration
in the field of education
Firstly through the use of standard student-orientated metrics
Secondly through informal metrics based around joint-perceptions of
success
The cases also suggest that there are a range of outcomes of university-business
collaboration activities in the field of education that are not currently being captured
by existing monitoring and measurement arrangements such as the potential
upskilling of existing staff through their involvement in the educational experience or
the raising of the profile of participating institutions However in no case was this felt
to weaken the cooperation arrangements underway Indeed to the contrary the
suggestion was that widening the formal measurement and monitoring process to
include more indicators would incur an additional burden for limited gains It is
essential that assessment methods are not only fit for purpose but are also seen to be
so
Whilst there are many methods that can be used to assess the outcomes of a
particular cooperation activity (whether that is a project or a programme) the
balanced scorecard approach seems particularly relevant The central idea of the
balanced scorecard is that several metrics can be used to evaluate the process in
question A causal map for educational university-industry collaboration initiatives is
given in Figure ES3 Four areas of measurement form the essential components of a
scorecard for university-industry collaboration The people involved the resources
available the activities undertaken and the results achieved
2014 11
Figure ES3 Scorecard general structure
Although each participant may have their own scorecard for the cooperation (either
held explicitly or implicitly) a combined scorecard can be drawn up for each co-
operation project in the form of a logical framework This allows assumptions to be
made explicit expected relationships to be identified and tests whether the
cooperation planned can meet its intended purpose(s) A stylized logic chain is set out
in Figure ES4
Figure ES4 Stylised causal map for university-industry collaboration
Alongside the scorecard approach an assessment methodology may also wish to set
out the framework for monitoring and evaluation Where evaluation may relate to the
assessment of project applications or the success of a completed project Here a
simple framework such as the success map set out in Figure ES5 may be
advantageous
People
bullThe qualification fit and motivation of the people taking part in the cooperation
Benefits
bullThe beneficial outputs that are generated as a result of the collaboration
Resources
bullThe amount and quality of the resources dedicated to the collaboration
Activities
bullThe quality and appropriateness of processes and organizational set-up
People
bullInstructors students managers
Resources
bullFunding equipment technology
Activities
bullOrganizational processes and structures
Benefits
bullEducation skills employment
2014 12
Figure ES5 Success map for cooperation in the field of education
In developing such a success map due regard should be given to four purposes of
university-business cooperation in the field of education broadly stated
Those that relate to teaching and learning aimed at strengthening the
labour supply and employment
Those that relate to stimulating entrepreneurship through teaching and
learning
Those that relate to knowledge exchange aimed at promoting
productprocess innovation or new pedagogy
Those that relate to raising the profile of an organization
In seeking to assess an actual project the framework can ensure that appropriate
indicators are selected and measurement approaches adopted It is beyond the scope
of this report to suggest a standard set of indicators as this will vary by project
purpose However a range of potential indicators have been identified relating to
each of the above purposes and which might be used to monitor inputs activities
outputs and outcomes Examples of potential indicators are illustrated in Figure ES5
In selecting indicators a number of principles should be observed including those of
parsimony (to seek to limit the data burden) and proportionality (data collected should
not incur costs disproportionate to the benefits received) The collection of data does
incur costs and for an assessment process to operate successfully there must be
agreement of all parties as to the role of particular indicators and the value of their
measurement
Interaction opportunities
Access to resources
Motivated educators
High-quality students
Relevant course delivery
Relevant course design
New ideas developed
Skilled amp trained labour
New pedagogies
Student applicant numbersquality
Non-academic organisations involved
Time inputs
Finance inputs
Newimproved courses
Number of graduates
Student attainment
Entrepreneurial attitudes
New products processes developed
Joint objective setting
Internshipsplacements
Co-delivery and assessment
Interaction intensity
Graduate employment
New business starts
Employer satisfaction
Organisational ranking in lsquoleaguersquo tables
Successful alliance
Human capital
Entrepreneurship
New ideas
Profile raising
Input Activity Output Outcome
2014 13
In the selection of indicators full consideration should also be given to the role of
qualitative measures of success and the potential means of capturing these This
might be through survey techniques but can also be captured by structured review
meetings with inputs from the various parties The value of self-assessment
techniques should also be considered Quantitative measures should not be over-
looked and where these can be part of wider measurement processes should be
welcomed
Implications for policy and practice
This study has proposed a flexible approach to measuring the impact of university-
business cooperation activities using a balanced scorecard approach allied with a
success-map for cooperation activities This recognizes the multiple purposes that
underlie cooperation activities and acknowledges that a one-size-fits-all approach is
unlikely to prove fit for purpose What are the benefits and implications of the
approach proposed to policy-makers and to the collaboration partners themselves
For policy-makers the tool can be used in a variety of manners
In the first instance it forms a framework for the ex-ante development of
the proposed programme By identifying the resources that are available to
the programme or might be available and the desired objectives
programme officials can identify the types of activity that might lead to the
desired results Through a process of iteration the scorecard can be refined
further This can then provide a baseline for a call for applications
The scorecard approach might then be used for the assessment or
appraisal of applications Assessors can check the applications against the
programme scorecard to identify the extent to which each applicant
contributes to the desired approach and objectives In a dynamic approach
the programme scorecard can be updated through the inclusion of novel
and innovative approaches proposed by applicants which were not foreseen
during the development of the programme A blank scorecard template
might also be completed by assessors as a means of summarising the
information contained in applications in a common format The approach
could highlight missed opportunities for collaborative activities or
resourcepeople inputs which can form the basis of negotiation with
programme applicants
Following a successful application a project scorecard can then form the
basis for the review of project progress This can examine the activities
undertaken resources expended and people involved against what was
initially planned together with the benefits realized compared to those that
were expected The review meeting can then explore the reasons for any
differences examining cases where inputs are greater than anticipated
perhaps owing to the cooperation going better than planned or where
results or inputs are less than originally proposed and whether remedial
action needs to be taken
2014 14
The scorecard approach can also deliver significant benefits to the collaboration
partners themselves
As a starting point it can form the basis for robust project design The
scorecard can be used as a mechanism for each partner to set out what
they wish to achieve from the cooperation activity and what they are able
to contribute Each partner can then identify what they expect of the other
partners Using the scorecard approach can also assist in identifying where
potential opportunities might have been overlooked in the original project
design and where small changes might reap large rewards This can then
form the basis of the collaboration agreement between parties
Project partners can then use their own stakeholder scorecard as a means
for internal review and monitoring of the collaboration activity This
provides a useful mechanism for identifying whether desired outcomes are
being achieved and the extent to which anticipated inputs and activities are
occurring As a dynamic process the scorecard can be updated as
activities are added or amended or other inputs change
The partnership as a whole can also use the scorecard as the basis for
ongoing project reviews This can consider whether the original
assumptions remain valid and whether some activities are more appropriate
than others in achieving the desired objectives Crucial to the review
process is to ensure that all partners are realizing the benefits that they
sought initially and if not how this might be rectified The scorecard
approach provides a means of structuring this review process
In order to promote the use of the scorecard approach the Commission may wish to
provide a guide to its use in the context of university-business cooperation activities
This would offer the opportunity to develop worked examples tailored to different
cooperation purposes Such a guide could also incorporate advice on practical
techniques for measuring the outcomes of cooperation activities In the first instance
this could form part of a toolkit available to assist those parties interested in applying
for Knowledge Alliance projects
HOW TO OBTAIN EU PUBLICATIONS
Free publications
bull one copy
via EU Bookshop (httpbookshopeuropaeu)
bull more than one copy or postersmaps
from the European Unionrsquos representations (httpeceuropaeurepresent_enhtm)
from the delegations in non-EU countries (httpeeaseuropaeudelegationsindex_enhtm)
by contacting the Europe Direct service (httpeuropaeueuropedirectindex_enhtm) or
calling 00 800 6 7 8 9 10 11 (freephone number from anywhere in the EU) ()
() The information given is free as are most calls (though some operators phone boxes
or hotels may charge you)
Priced publications
bull via EU Bookshop (httpbookshopeuropaeu)
Priced subscriptions
bull via one of the sales agents of the Publications Office of the European Union
(httppublicationseuropaeuothersagentsindex_enhtm)
NC-0
1-1
4-2
99-E
N-N
doi10
2014 9
course) whilst others are more indirect (such as the value of contacts gained for
potential future research projects) Equally some benefits are very tangible (such as
when a student develops a particular skill) but others are much less so (such as where
student knowledge spills over to workers in a company who work alongside them on a
project) Examples of all these potential benefits were visible throughout the cases
explored for this study
In practice our study illustrates a focus on immediate highly-visible returns by 10
university-business cooperation projects but also emphasises the significance of less
tangible and more indirect returns to participants with different participants placing
differential values on particular outcomes For example in the case of the North East
England sub-sea collaboration businesses were keen to fill an emerging skills gap
upskill staff and develop contacts with the university and other businesses For the
academics it was a means to generate additional student numbers to engage with
the business sector and to explore further research avenues
For students the reason they wished to participate in these collaborative offers in the
field of education was primarily related to the fact they felt it would provide them with
greater skills than a more conventional course that they were more likely to find
subsequent employment or that it would provide them with enhanced entrepreneurial
opportunities
For universities and academics there was a similar emphasis on the enhanced skills
acquired by students the greater employability of these students and the stronger
entrepreneurial outcomes anticipated as highlighted in the EUEN project These
outcomes were expected to feed through into an increase in either the quality or the
quantity of student applications with improvements in student perceptions of course
quality also anticipated Additional outcomes which were cited in some instances
included the opportunity provided for increasing contacts in the business world the
updating of skills through exposure to contemporary businesses practices and the
building of trust-based relationships for future collaborative research
For businesses the primary outcome reported was in terms of an enhanced pipeline of
suitably skilled potential employees together with improvements to the wider labour
force There were also particular outcomes specific to individual cooperation
arrangements such as in the case of AppCampus where one measure of success is the
number of applications developed Businesses also reported that the cooperation
arrangements provided benefits in the form of raising their profile but also helped to
build trust and wider contact networks with academics for future collaborative
research Firms also reported that involvement in live projects did provide the
opportunity for product and process innovations adding economic value to the
company and that the bringing together of students and existing employees served to
raise the knowledge levels of employees as well as students
For other parties such as regional authorities the anticipated (and realised) outcomes
largely revolved around economic benefits such as an enhanced labour supply
increased levels of graduate employment higher levels of new business-starts or
(rarely) higher levels of innovation
2014 10
Assessing the benefits of cooperation
Currently there is very little guidance on approaches to assessing the benefits of
university-business cooperation activities in the field of education The existing
literature is strongly inclined to cooperation in the field of research and innovation
What examples can be found tend to stress measures of inputs and student outcomes
The wider literature can however provide valuable insights This incudes highlighting
the value of less tangible benefits such as contact between students company staff
and university staff often termed as productive interactions
Monitoring and assessment has a number of potential benefits it provides a means to
assess whether a project or programme is performing as expected it enables a
project leader to identify what works and what does not it provides a useful
mechanism to feedback lessons to others and if undertaken in advance it can
provide a means for selecting between different potential options In the evidence
available to this project the main approaches to monitoring and assessment have
focused on project performance often with an emphasis on measures of inputs and
outputs However there is also evidence of the use of results-based formats often of
a qualitative nature
The evidence available from the cases explored in this work suggests that monitoring
activities are undertaken at two levels in the field of university-business collaboration
in the field of education
Firstly through the use of standard student-orientated metrics
Secondly through informal metrics based around joint-perceptions of
success
The cases also suggest that there are a range of outcomes of university-business
collaboration activities in the field of education that are not currently being captured
by existing monitoring and measurement arrangements such as the potential
upskilling of existing staff through their involvement in the educational experience or
the raising of the profile of participating institutions However in no case was this felt
to weaken the cooperation arrangements underway Indeed to the contrary the
suggestion was that widening the formal measurement and monitoring process to
include more indicators would incur an additional burden for limited gains It is
essential that assessment methods are not only fit for purpose but are also seen to be
so
Whilst there are many methods that can be used to assess the outcomes of a
particular cooperation activity (whether that is a project or a programme) the
balanced scorecard approach seems particularly relevant The central idea of the
balanced scorecard is that several metrics can be used to evaluate the process in
question A causal map for educational university-industry collaboration initiatives is
given in Figure ES3 Four areas of measurement form the essential components of a
scorecard for university-industry collaboration The people involved the resources
available the activities undertaken and the results achieved
2014 11
Figure ES3 Scorecard general structure
Although each participant may have their own scorecard for the cooperation (either
held explicitly or implicitly) a combined scorecard can be drawn up for each co-
operation project in the form of a logical framework This allows assumptions to be
made explicit expected relationships to be identified and tests whether the
cooperation planned can meet its intended purpose(s) A stylized logic chain is set out
in Figure ES4
Figure ES4 Stylised causal map for university-industry collaboration
Alongside the scorecard approach an assessment methodology may also wish to set
out the framework for monitoring and evaluation Where evaluation may relate to the
assessment of project applications or the success of a completed project Here a
simple framework such as the success map set out in Figure ES5 may be
advantageous
People
bullThe qualification fit and motivation of the people taking part in the cooperation
Benefits
bullThe beneficial outputs that are generated as a result of the collaboration
Resources
bullThe amount and quality of the resources dedicated to the collaboration
Activities
bullThe quality and appropriateness of processes and organizational set-up
People
bullInstructors students managers
Resources
bullFunding equipment technology
Activities
bullOrganizational processes and structures
Benefits
bullEducation skills employment
2014 12
Figure ES5 Success map for cooperation in the field of education
In developing such a success map due regard should be given to four purposes of
university-business cooperation in the field of education broadly stated
Those that relate to teaching and learning aimed at strengthening the
labour supply and employment
Those that relate to stimulating entrepreneurship through teaching and
learning
Those that relate to knowledge exchange aimed at promoting
productprocess innovation or new pedagogy
Those that relate to raising the profile of an organization
In seeking to assess an actual project the framework can ensure that appropriate
indicators are selected and measurement approaches adopted It is beyond the scope
of this report to suggest a standard set of indicators as this will vary by project
purpose However a range of potential indicators have been identified relating to
each of the above purposes and which might be used to monitor inputs activities
outputs and outcomes Examples of potential indicators are illustrated in Figure ES5
In selecting indicators a number of principles should be observed including those of
parsimony (to seek to limit the data burden) and proportionality (data collected should
not incur costs disproportionate to the benefits received) The collection of data does
incur costs and for an assessment process to operate successfully there must be
agreement of all parties as to the role of particular indicators and the value of their
measurement
Interaction opportunities
Access to resources
Motivated educators
High-quality students
Relevant course delivery
Relevant course design
New ideas developed
Skilled amp trained labour
New pedagogies
Student applicant numbersquality
Non-academic organisations involved
Time inputs
Finance inputs
Newimproved courses
Number of graduates
Student attainment
Entrepreneurial attitudes
New products processes developed
Joint objective setting
Internshipsplacements
Co-delivery and assessment
Interaction intensity
Graduate employment
New business starts
Employer satisfaction
Organisational ranking in lsquoleaguersquo tables
Successful alliance
Human capital
Entrepreneurship
New ideas
Profile raising
Input Activity Output Outcome
2014 13
In the selection of indicators full consideration should also be given to the role of
qualitative measures of success and the potential means of capturing these This
might be through survey techniques but can also be captured by structured review
meetings with inputs from the various parties The value of self-assessment
techniques should also be considered Quantitative measures should not be over-
looked and where these can be part of wider measurement processes should be
welcomed
Implications for policy and practice
This study has proposed a flexible approach to measuring the impact of university-
business cooperation activities using a balanced scorecard approach allied with a
success-map for cooperation activities This recognizes the multiple purposes that
underlie cooperation activities and acknowledges that a one-size-fits-all approach is
unlikely to prove fit for purpose What are the benefits and implications of the
approach proposed to policy-makers and to the collaboration partners themselves
For policy-makers the tool can be used in a variety of manners
In the first instance it forms a framework for the ex-ante development of
the proposed programme By identifying the resources that are available to
the programme or might be available and the desired objectives
programme officials can identify the types of activity that might lead to the
desired results Through a process of iteration the scorecard can be refined
further This can then provide a baseline for a call for applications
The scorecard approach might then be used for the assessment or
appraisal of applications Assessors can check the applications against the
programme scorecard to identify the extent to which each applicant
contributes to the desired approach and objectives In a dynamic approach
the programme scorecard can be updated through the inclusion of novel
and innovative approaches proposed by applicants which were not foreseen
during the development of the programme A blank scorecard template
might also be completed by assessors as a means of summarising the
information contained in applications in a common format The approach
could highlight missed opportunities for collaborative activities or
resourcepeople inputs which can form the basis of negotiation with
programme applicants
Following a successful application a project scorecard can then form the
basis for the review of project progress This can examine the activities
undertaken resources expended and people involved against what was
initially planned together with the benefits realized compared to those that
were expected The review meeting can then explore the reasons for any
differences examining cases where inputs are greater than anticipated
perhaps owing to the cooperation going better than planned or where
results or inputs are less than originally proposed and whether remedial
action needs to be taken
2014 14
The scorecard approach can also deliver significant benefits to the collaboration
partners themselves
As a starting point it can form the basis for robust project design The
scorecard can be used as a mechanism for each partner to set out what
they wish to achieve from the cooperation activity and what they are able
to contribute Each partner can then identify what they expect of the other
partners Using the scorecard approach can also assist in identifying where
potential opportunities might have been overlooked in the original project
design and where small changes might reap large rewards This can then
form the basis of the collaboration agreement between parties
Project partners can then use their own stakeholder scorecard as a means
for internal review and monitoring of the collaboration activity This
provides a useful mechanism for identifying whether desired outcomes are
being achieved and the extent to which anticipated inputs and activities are
occurring As a dynamic process the scorecard can be updated as
activities are added or amended or other inputs change
The partnership as a whole can also use the scorecard as the basis for
ongoing project reviews This can consider whether the original
assumptions remain valid and whether some activities are more appropriate
than others in achieving the desired objectives Crucial to the review
process is to ensure that all partners are realizing the benefits that they
sought initially and if not how this might be rectified The scorecard
approach provides a means of structuring this review process
In order to promote the use of the scorecard approach the Commission may wish to
provide a guide to its use in the context of university-business cooperation activities
This would offer the opportunity to develop worked examples tailored to different
cooperation purposes Such a guide could also incorporate advice on practical
techniques for measuring the outcomes of cooperation activities In the first instance
this could form part of a toolkit available to assist those parties interested in applying
for Knowledge Alliance projects
HOW TO OBTAIN EU PUBLICATIONS
Free publications
bull one copy
via EU Bookshop (httpbookshopeuropaeu)
bull more than one copy or postersmaps
from the European Unionrsquos representations (httpeceuropaeurepresent_enhtm)
from the delegations in non-EU countries (httpeeaseuropaeudelegationsindex_enhtm)
by contacting the Europe Direct service (httpeuropaeueuropedirectindex_enhtm) or
calling 00 800 6 7 8 9 10 11 (freephone number from anywhere in the EU) ()
() The information given is free as are most calls (though some operators phone boxes
or hotels may charge you)
Priced publications
bull via EU Bookshop (httpbookshopeuropaeu)
Priced subscriptions
bull via one of the sales agents of the Publications Office of the European Union
(httppublicationseuropaeuothersagentsindex_enhtm)
NC-0
1-1
4-2
99-E
N-N
doi10
2014 10
Assessing the benefits of cooperation
Currently there is very little guidance on approaches to assessing the benefits of
university-business cooperation activities in the field of education The existing
literature is strongly inclined to cooperation in the field of research and innovation
What examples can be found tend to stress measures of inputs and student outcomes
The wider literature can however provide valuable insights This incudes highlighting
the value of less tangible benefits such as contact between students company staff
and university staff often termed as productive interactions
Monitoring and assessment has a number of potential benefits it provides a means to
assess whether a project or programme is performing as expected it enables a
project leader to identify what works and what does not it provides a useful
mechanism to feedback lessons to others and if undertaken in advance it can
provide a means for selecting between different potential options In the evidence
available to this project the main approaches to monitoring and assessment have
focused on project performance often with an emphasis on measures of inputs and
outputs However there is also evidence of the use of results-based formats often of
a qualitative nature
The evidence available from the cases explored in this work suggests that monitoring
activities are undertaken at two levels in the field of university-business collaboration
in the field of education
Firstly through the use of standard student-orientated metrics
Secondly through informal metrics based around joint-perceptions of
success
The cases also suggest that there are a range of outcomes of university-business
collaboration activities in the field of education that are not currently being captured
by existing monitoring and measurement arrangements such as the potential
upskilling of existing staff through their involvement in the educational experience or
the raising of the profile of participating institutions However in no case was this felt
to weaken the cooperation arrangements underway Indeed to the contrary the
suggestion was that widening the formal measurement and monitoring process to
include more indicators would incur an additional burden for limited gains It is
essential that assessment methods are not only fit for purpose but are also seen to be
so
Whilst there are many methods that can be used to assess the outcomes of a
particular cooperation activity (whether that is a project or a programme) the
balanced scorecard approach seems particularly relevant The central idea of the
balanced scorecard is that several metrics can be used to evaluate the process in
question A causal map for educational university-industry collaboration initiatives is
given in Figure ES3 Four areas of measurement form the essential components of a
scorecard for university-industry collaboration The people involved the resources
available the activities undertaken and the results achieved
2014 11
Figure ES3 Scorecard general structure
Although each participant may have their own scorecard for the cooperation (either
held explicitly or implicitly) a combined scorecard can be drawn up for each co-
operation project in the form of a logical framework This allows assumptions to be
made explicit expected relationships to be identified and tests whether the
cooperation planned can meet its intended purpose(s) A stylized logic chain is set out
in Figure ES4
Figure ES4 Stylised causal map for university-industry collaboration
Alongside the scorecard approach an assessment methodology may also wish to set
out the framework for monitoring and evaluation Where evaluation may relate to the
assessment of project applications or the success of a completed project Here a
simple framework such as the success map set out in Figure ES5 may be
advantageous
People
bullThe qualification fit and motivation of the people taking part in the cooperation
Benefits
bullThe beneficial outputs that are generated as a result of the collaboration
Resources
bullThe amount and quality of the resources dedicated to the collaboration
Activities
bullThe quality and appropriateness of processes and organizational set-up
People
bullInstructors students managers
Resources
bullFunding equipment technology
Activities
bullOrganizational processes and structures
Benefits
bullEducation skills employment
2014 12
Figure ES5 Success map for cooperation in the field of education
In developing such a success map due regard should be given to four purposes of
university-business cooperation in the field of education broadly stated
Those that relate to teaching and learning aimed at strengthening the
labour supply and employment
Those that relate to stimulating entrepreneurship through teaching and
learning
Those that relate to knowledge exchange aimed at promoting
productprocess innovation or new pedagogy
Those that relate to raising the profile of an organization
In seeking to assess an actual project the framework can ensure that appropriate
indicators are selected and measurement approaches adopted It is beyond the scope
of this report to suggest a standard set of indicators as this will vary by project
purpose However a range of potential indicators have been identified relating to
each of the above purposes and which might be used to monitor inputs activities
outputs and outcomes Examples of potential indicators are illustrated in Figure ES5
In selecting indicators a number of principles should be observed including those of
parsimony (to seek to limit the data burden) and proportionality (data collected should
not incur costs disproportionate to the benefits received) The collection of data does
incur costs and for an assessment process to operate successfully there must be
agreement of all parties as to the role of particular indicators and the value of their
measurement
Interaction opportunities
Access to resources
Motivated educators
High-quality students
Relevant course delivery
Relevant course design
New ideas developed
Skilled amp trained labour
New pedagogies
Student applicant numbersquality
Non-academic organisations involved
Time inputs
Finance inputs
Newimproved courses
Number of graduates
Student attainment
Entrepreneurial attitudes
New products processes developed
Joint objective setting
Internshipsplacements
Co-delivery and assessment
Interaction intensity
Graduate employment
New business starts
Employer satisfaction
Organisational ranking in lsquoleaguersquo tables
Successful alliance
Human capital
Entrepreneurship
New ideas
Profile raising
Input Activity Output Outcome
2014 13
In the selection of indicators full consideration should also be given to the role of
qualitative measures of success and the potential means of capturing these This
might be through survey techniques but can also be captured by structured review
meetings with inputs from the various parties The value of self-assessment
techniques should also be considered Quantitative measures should not be over-
looked and where these can be part of wider measurement processes should be
welcomed
Implications for policy and practice
This study has proposed a flexible approach to measuring the impact of university-
business cooperation activities using a balanced scorecard approach allied with a
success-map for cooperation activities This recognizes the multiple purposes that
underlie cooperation activities and acknowledges that a one-size-fits-all approach is
unlikely to prove fit for purpose What are the benefits and implications of the
approach proposed to policy-makers and to the collaboration partners themselves
For policy-makers the tool can be used in a variety of manners
In the first instance it forms a framework for the ex-ante development of
the proposed programme By identifying the resources that are available to
the programme or might be available and the desired objectives
programme officials can identify the types of activity that might lead to the
desired results Through a process of iteration the scorecard can be refined
further This can then provide a baseline for a call for applications
The scorecard approach might then be used for the assessment or
appraisal of applications Assessors can check the applications against the
programme scorecard to identify the extent to which each applicant
contributes to the desired approach and objectives In a dynamic approach
the programme scorecard can be updated through the inclusion of novel
and innovative approaches proposed by applicants which were not foreseen
during the development of the programme A blank scorecard template
might also be completed by assessors as a means of summarising the
information contained in applications in a common format The approach
could highlight missed opportunities for collaborative activities or
resourcepeople inputs which can form the basis of negotiation with
programme applicants
Following a successful application a project scorecard can then form the
basis for the review of project progress This can examine the activities
undertaken resources expended and people involved against what was
initially planned together with the benefits realized compared to those that
were expected The review meeting can then explore the reasons for any
differences examining cases where inputs are greater than anticipated
perhaps owing to the cooperation going better than planned or where
results or inputs are less than originally proposed and whether remedial
action needs to be taken
2014 14
The scorecard approach can also deliver significant benefits to the collaboration
partners themselves
As a starting point it can form the basis for robust project design The
scorecard can be used as a mechanism for each partner to set out what
they wish to achieve from the cooperation activity and what they are able
to contribute Each partner can then identify what they expect of the other
partners Using the scorecard approach can also assist in identifying where
potential opportunities might have been overlooked in the original project
design and where small changes might reap large rewards This can then
form the basis of the collaboration agreement between parties
Project partners can then use their own stakeholder scorecard as a means
for internal review and monitoring of the collaboration activity This
provides a useful mechanism for identifying whether desired outcomes are
being achieved and the extent to which anticipated inputs and activities are
occurring As a dynamic process the scorecard can be updated as
activities are added or amended or other inputs change
The partnership as a whole can also use the scorecard as the basis for
ongoing project reviews This can consider whether the original
assumptions remain valid and whether some activities are more appropriate
than others in achieving the desired objectives Crucial to the review
process is to ensure that all partners are realizing the benefits that they
sought initially and if not how this might be rectified The scorecard
approach provides a means of structuring this review process
In order to promote the use of the scorecard approach the Commission may wish to
provide a guide to its use in the context of university-business cooperation activities
This would offer the opportunity to develop worked examples tailored to different
cooperation purposes Such a guide could also incorporate advice on practical
techniques for measuring the outcomes of cooperation activities In the first instance
this could form part of a toolkit available to assist those parties interested in applying
for Knowledge Alliance projects
HOW TO OBTAIN EU PUBLICATIONS
Free publications
bull one copy
via EU Bookshop (httpbookshopeuropaeu)
bull more than one copy or postersmaps
from the European Unionrsquos representations (httpeceuropaeurepresent_enhtm)
from the delegations in non-EU countries (httpeeaseuropaeudelegationsindex_enhtm)
by contacting the Europe Direct service (httpeuropaeueuropedirectindex_enhtm) or
calling 00 800 6 7 8 9 10 11 (freephone number from anywhere in the EU) ()
() The information given is free as are most calls (though some operators phone boxes
or hotels may charge you)
Priced publications
bull via EU Bookshop (httpbookshopeuropaeu)
Priced subscriptions
bull via one of the sales agents of the Publications Office of the European Union
(httppublicationseuropaeuothersagentsindex_enhtm)
NC-0
1-1
4-2
99-E
N-N
doi10
2014 11
Figure ES3 Scorecard general structure
Although each participant may have their own scorecard for the cooperation (either
held explicitly or implicitly) a combined scorecard can be drawn up for each co-
operation project in the form of a logical framework This allows assumptions to be
made explicit expected relationships to be identified and tests whether the
cooperation planned can meet its intended purpose(s) A stylized logic chain is set out
in Figure ES4
Figure ES4 Stylised causal map for university-industry collaboration
Alongside the scorecard approach an assessment methodology may also wish to set
out the framework for monitoring and evaluation Where evaluation may relate to the
assessment of project applications or the success of a completed project Here a
simple framework such as the success map set out in Figure ES5 may be
advantageous
People
bullThe qualification fit and motivation of the people taking part in the cooperation
Benefits
bullThe beneficial outputs that are generated as a result of the collaboration
Resources
bullThe amount and quality of the resources dedicated to the collaboration
Activities
bullThe quality and appropriateness of processes and organizational set-up
People
bullInstructors students managers
Resources
bullFunding equipment technology
Activities
bullOrganizational processes and structures
Benefits
bullEducation skills employment
2014 12
Figure ES5 Success map for cooperation in the field of education
In developing such a success map due regard should be given to four purposes of
university-business cooperation in the field of education broadly stated
Those that relate to teaching and learning aimed at strengthening the
labour supply and employment
Those that relate to stimulating entrepreneurship through teaching and
learning
Those that relate to knowledge exchange aimed at promoting
productprocess innovation or new pedagogy
Those that relate to raising the profile of an organization
In seeking to assess an actual project the framework can ensure that appropriate
indicators are selected and measurement approaches adopted It is beyond the scope
of this report to suggest a standard set of indicators as this will vary by project
purpose However a range of potential indicators have been identified relating to
each of the above purposes and which might be used to monitor inputs activities
outputs and outcomes Examples of potential indicators are illustrated in Figure ES5
In selecting indicators a number of principles should be observed including those of
parsimony (to seek to limit the data burden) and proportionality (data collected should
not incur costs disproportionate to the benefits received) The collection of data does
incur costs and for an assessment process to operate successfully there must be
agreement of all parties as to the role of particular indicators and the value of their
measurement
Interaction opportunities
Access to resources
Motivated educators
High-quality students
Relevant course delivery
Relevant course design
New ideas developed
Skilled amp trained labour
New pedagogies
Student applicant numbersquality
Non-academic organisations involved
Time inputs
Finance inputs
Newimproved courses
Number of graduates
Student attainment
Entrepreneurial attitudes
New products processes developed
Joint objective setting
Internshipsplacements
Co-delivery and assessment
Interaction intensity
Graduate employment
New business starts
Employer satisfaction
Organisational ranking in lsquoleaguersquo tables
Successful alliance
Human capital
Entrepreneurship
New ideas
Profile raising
Input Activity Output Outcome
2014 13
In the selection of indicators full consideration should also be given to the role of
qualitative measures of success and the potential means of capturing these This
might be through survey techniques but can also be captured by structured review
meetings with inputs from the various parties The value of self-assessment
techniques should also be considered Quantitative measures should not be over-
looked and where these can be part of wider measurement processes should be
welcomed
Implications for policy and practice
This study has proposed a flexible approach to measuring the impact of university-
business cooperation activities using a balanced scorecard approach allied with a
success-map for cooperation activities This recognizes the multiple purposes that
underlie cooperation activities and acknowledges that a one-size-fits-all approach is
unlikely to prove fit for purpose What are the benefits and implications of the
approach proposed to policy-makers and to the collaboration partners themselves
For policy-makers the tool can be used in a variety of manners
In the first instance it forms a framework for the ex-ante development of
the proposed programme By identifying the resources that are available to
the programme or might be available and the desired objectives
programme officials can identify the types of activity that might lead to the
desired results Through a process of iteration the scorecard can be refined
further This can then provide a baseline for a call for applications
The scorecard approach might then be used for the assessment or
appraisal of applications Assessors can check the applications against the
programme scorecard to identify the extent to which each applicant
contributes to the desired approach and objectives In a dynamic approach
the programme scorecard can be updated through the inclusion of novel
and innovative approaches proposed by applicants which were not foreseen
during the development of the programme A blank scorecard template
might also be completed by assessors as a means of summarising the
information contained in applications in a common format The approach
could highlight missed opportunities for collaborative activities or
resourcepeople inputs which can form the basis of negotiation with
programme applicants
Following a successful application a project scorecard can then form the
basis for the review of project progress This can examine the activities
undertaken resources expended and people involved against what was
initially planned together with the benefits realized compared to those that
were expected The review meeting can then explore the reasons for any
differences examining cases where inputs are greater than anticipated
perhaps owing to the cooperation going better than planned or where
results or inputs are less than originally proposed and whether remedial
action needs to be taken
2014 14
The scorecard approach can also deliver significant benefits to the collaboration
partners themselves
As a starting point it can form the basis for robust project design The
scorecard can be used as a mechanism for each partner to set out what
they wish to achieve from the cooperation activity and what they are able
to contribute Each partner can then identify what they expect of the other
partners Using the scorecard approach can also assist in identifying where
potential opportunities might have been overlooked in the original project
design and where small changes might reap large rewards This can then
form the basis of the collaboration agreement between parties
Project partners can then use their own stakeholder scorecard as a means
for internal review and monitoring of the collaboration activity This
provides a useful mechanism for identifying whether desired outcomes are
being achieved and the extent to which anticipated inputs and activities are
occurring As a dynamic process the scorecard can be updated as
activities are added or amended or other inputs change
The partnership as a whole can also use the scorecard as the basis for
ongoing project reviews This can consider whether the original
assumptions remain valid and whether some activities are more appropriate
than others in achieving the desired objectives Crucial to the review
process is to ensure that all partners are realizing the benefits that they
sought initially and if not how this might be rectified The scorecard
approach provides a means of structuring this review process
In order to promote the use of the scorecard approach the Commission may wish to
provide a guide to its use in the context of university-business cooperation activities
This would offer the opportunity to develop worked examples tailored to different
cooperation purposes Such a guide could also incorporate advice on practical
techniques for measuring the outcomes of cooperation activities In the first instance
this could form part of a toolkit available to assist those parties interested in applying
for Knowledge Alliance projects
HOW TO OBTAIN EU PUBLICATIONS
Free publications
bull one copy
via EU Bookshop (httpbookshopeuropaeu)
bull more than one copy or postersmaps
from the European Unionrsquos representations (httpeceuropaeurepresent_enhtm)
from the delegations in non-EU countries (httpeeaseuropaeudelegationsindex_enhtm)
by contacting the Europe Direct service (httpeuropaeueuropedirectindex_enhtm) or
calling 00 800 6 7 8 9 10 11 (freephone number from anywhere in the EU) ()
() The information given is free as are most calls (though some operators phone boxes
or hotels may charge you)
Priced publications
bull via EU Bookshop (httpbookshopeuropaeu)
Priced subscriptions
bull via one of the sales agents of the Publications Office of the European Union
(httppublicationseuropaeuothersagentsindex_enhtm)
NC-0
1-1
4-2
99-E
N-N
doi10
2014 12
Figure ES5 Success map for cooperation in the field of education
In developing such a success map due regard should be given to four purposes of
university-business cooperation in the field of education broadly stated
Those that relate to teaching and learning aimed at strengthening the
labour supply and employment
Those that relate to stimulating entrepreneurship through teaching and
learning
Those that relate to knowledge exchange aimed at promoting
productprocess innovation or new pedagogy
Those that relate to raising the profile of an organization
In seeking to assess an actual project the framework can ensure that appropriate
indicators are selected and measurement approaches adopted It is beyond the scope
of this report to suggest a standard set of indicators as this will vary by project
purpose However a range of potential indicators have been identified relating to
each of the above purposes and which might be used to monitor inputs activities
outputs and outcomes Examples of potential indicators are illustrated in Figure ES5
In selecting indicators a number of principles should be observed including those of
parsimony (to seek to limit the data burden) and proportionality (data collected should
not incur costs disproportionate to the benefits received) The collection of data does
incur costs and for an assessment process to operate successfully there must be
agreement of all parties as to the role of particular indicators and the value of their
measurement
Interaction opportunities
Access to resources
Motivated educators
High-quality students
Relevant course delivery
Relevant course design
New ideas developed
Skilled amp trained labour
New pedagogies
Student applicant numbersquality
Non-academic organisations involved
Time inputs
Finance inputs
Newimproved courses
Number of graduates
Student attainment
Entrepreneurial attitudes
New products processes developed
Joint objective setting
Internshipsplacements
Co-delivery and assessment
Interaction intensity
Graduate employment
New business starts
Employer satisfaction
Organisational ranking in lsquoleaguersquo tables
Successful alliance
Human capital
Entrepreneurship
New ideas
Profile raising
Input Activity Output Outcome
2014 13
In the selection of indicators full consideration should also be given to the role of
qualitative measures of success and the potential means of capturing these This
might be through survey techniques but can also be captured by structured review
meetings with inputs from the various parties The value of self-assessment
techniques should also be considered Quantitative measures should not be over-
looked and where these can be part of wider measurement processes should be
welcomed
Implications for policy and practice
This study has proposed a flexible approach to measuring the impact of university-
business cooperation activities using a balanced scorecard approach allied with a
success-map for cooperation activities This recognizes the multiple purposes that
underlie cooperation activities and acknowledges that a one-size-fits-all approach is
unlikely to prove fit for purpose What are the benefits and implications of the
approach proposed to policy-makers and to the collaboration partners themselves
For policy-makers the tool can be used in a variety of manners
In the first instance it forms a framework for the ex-ante development of
the proposed programme By identifying the resources that are available to
the programme or might be available and the desired objectives
programme officials can identify the types of activity that might lead to the
desired results Through a process of iteration the scorecard can be refined
further This can then provide a baseline for a call for applications
The scorecard approach might then be used for the assessment or
appraisal of applications Assessors can check the applications against the
programme scorecard to identify the extent to which each applicant
contributes to the desired approach and objectives In a dynamic approach
the programme scorecard can be updated through the inclusion of novel
and innovative approaches proposed by applicants which were not foreseen
during the development of the programme A blank scorecard template
might also be completed by assessors as a means of summarising the
information contained in applications in a common format The approach
could highlight missed opportunities for collaborative activities or
resourcepeople inputs which can form the basis of negotiation with
programme applicants
Following a successful application a project scorecard can then form the
basis for the review of project progress This can examine the activities
undertaken resources expended and people involved against what was
initially planned together with the benefits realized compared to those that
were expected The review meeting can then explore the reasons for any
differences examining cases where inputs are greater than anticipated
perhaps owing to the cooperation going better than planned or where
results or inputs are less than originally proposed and whether remedial
action needs to be taken
2014 14
The scorecard approach can also deliver significant benefits to the collaboration
partners themselves
As a starting point it can form the basis for robust project design The
scorecard can be used as a mechanism for each partner to set out what
they wish to achieve from the cooperation activity and what they are able
to contribute Each partner can then identify what they expect of the other
partners Using the scorecard approach can also assist in identifying where
potential opportunities might have been overlooked in the original project
design and where small changes might reap large rewards This can then
form the basis of the collaboration agreement between parties
Project partners can then use their own stakeholder scorecard as a means
for internal review and monitoring of the collaboration activity This
provides a useful mechanism for identifying whether desired outcomes are
being achieved and the extent to which anticipated inputs and activities are
occurring As a dynamic process the scorecard can be updated as
activities are added or amended or other inputs change
The partnership as a whole can also use the scorecard as the basis for
ongoing project reviews This can consider whether the original
assumptions remain valid and whether some activities are more appropriate
than others in achieving the desired objectives Crucial to the review
process is to ensure that all partners are realizing the benefits that they
sought initially and if not how this might be rectified The scorecard
approach provides a means of structuring this review process
In order to promote the use of the scorecard approach the Commission may wish to
provide a guide to its use in the context of university-business cooperation activities
This would offer the opportunity to develop worked examples tailored to different
cooperation purposes Such a guide could also incorporate advice on practical
techniques for measuring the outcomes of cooperation activities In the first instance
this could form part of a toolkit available to assist those parties interested in applying
for Knowledge Alliance projects
HOW TO OBTAIN EU PUBLICATIONS
Free publications
bull one copy
via EU Bookshop (httpbookshopeuropaeu)
bull more than one copy or postersmaps
from the European Unionrsquos representations (httpeceuropaeurepresent_enhtm)
from the delegations in non-EU countries (httpeeaseuropaeudelegationsindex_enhtm)
by contacting the Europe Direct service (httpeuropaeueuropedirectindex_enhtm) or
calling 00 800 6 7 8 9 10 11 (freephone number from anywhere in the EU) ()
() The information given is free as are most calls (though some operators phone boxes
or hotels may charge you)
Priced publications
bull via EU Bookshop (httpbookshopeuropaeu)
Priced subscriptions
bull via one of the sales agents of the Publications Office of the European Union
(httppublicationseuropaeuothersagentsindex_enhtm)
NC-0
1-1
4-2
99-E
N-N
doi10
2014 13
In the selection of indicators full consideration should also be given to the role of
qualitative measures of success and the potential means of capturing these This
might be through survey techniques but can also be captured by structured review
meetings with inputs from the various parties The value of self-assessment
techniques should also be considered Quantitative measures should not be over-
looked and where these can be part of wider measurement processes should be
welcomed
Implications for policy and practice
This study has proposed a flexible approach to measuring the impact of university-
business cooperation activities using a balanced scorecard approach allied with a
success-map for cooperation activities This recognizes the multiple purposes that
underlie cooperation activities and acknowledges that a one-size-fits-all approach is
unlikely to prove fit for purpose What are the benefits and implications of the
approach proposed to policy-makers and to the collaboration partners themselves
For policy-makers the tool can be used in a variety of manners
In the first instance it forms a framework for the ex-ante development of
the proposed programme By identifying the resources that are available to
the programme or might be available and the desired objectives
programme officials can identify the types of activity that might lead to the
desired results Through a process of iteration the scorecard can be refined
further This can then provide a baseline for a call for applications
The scorecard approach might then be used for the assessment or
appraisal of applications Assessors can check the applications against the
programme scorecard to identify the extent to which each applicant
contributes to the desired approach and objectives In a dynamic approach
the programme scorecard can be updated through the inclusion of novel
and innovative approaches proposed by applicants which were not foreseen
during the development of the programme A blank scorecard template
might also be completed by assessors as a means of summarising the
information contained in applications in a common format The approach
could highlight missed opportunities for collaborative activities or
resourcepeople inputs which can form the basis of negotiation with
programme applicants
Following a successful application a project scorecard can then form the
basis for the review of project progress This can examine the activities
undertaken resources expended and people involved against what was
initially planned together with the benefits realized compared to those that
were expected The review meeting can then explore the reasons for any
differences examining cases where inputs are greater than anticipated
perhaps owing to the cooperation going better than planned or where
results or inputs are less than originally proposed and whether remedial
action needs to be taken
2014 14
The scorecard approach can also deliver significant benefits to the collaboration
partners themselves
As a starting point it can form the basis for robust project design The
scorecard can be used as a mechanism for each partner to set out what
they wish to achieve from the cooperation activity and what they are able
to contribute Each partner can then identify what they expect of the other
partners Using the scorecard approach can also assist in identifying where
potential opportunities might have been overlooked in the original project
design and where small changes might reap large rewards This can then
form the basis of the collaboration agreement between parties
Project partners can then use their own stakeholder scorecard as a means
for internal review and monitoring of the collaboration activity This
provides a useful mechanism for identifying whether desired outcomes are
being achieved and the extent to which anticipated inputs and activities are
occurring As a dynamic process the scorecard can be updated as
activities are added or amended or other inputs change
The partnership as a whole can also use the scorecard as the basis for
ongoing project reviews This can consider whether the original
assumptions remain valid and whether some activities are more appropriate
than others in achieving the desired objectives Crucial to the review
process is to ensure that all partners are realizing the benefits that they
sought initially and if not how this might be rectified The scorecard
approach provides a means of structuring this review process
In order to promote the use of the scorecard approach the Commission may wish to
provide a guide to its use in the context of university-business cooperation activities
This would offer the opportunity to develop worked examples tailored to different
cooperation purposes Such a guide could also incorporate advice on practical
techniques for measuring the outcomes of cooperation activities In the first instance
this could form part of a toolkit available to assist those parties interested in applying
for Knowledge Alliance projects
HOW TO OBTAIN EU PUBLICATIONS
Free publications
bull one copy
via EU Bookshop (httpbookshopeuropaeu)
bull more than one copy or postersmaps
from the European Unionrsquos representations (httpeceuropaeurepresent_enhtm)
from the delegations in non-EU countries (httpeeaseuropaeudelegationsindex_enhtm)
by contacting the Europe Direct service (httpeuropaeueuropedirectindex_enhtm) or
calling 00 800 6 7 8 9 10 11 (freephone number from anywhere in the EU) ()
() The information given is free as are most calls (though some operators phone boxes
or hotels may charge you)
Priced publications
bull via EU Bookshop (httpbookshopeuropaeu)
Priced subscriptions
bull via one of the sales agents of the Publications Office of the European Union
(httppublicationseuropaeuothersagentsindex_enhtm)
NC-0
1-1
4-2
99-E
N-N
doi10
2014 14
The scorecard approach can also deliver significant benefits to the collaboration
partners themselves
As a starting point it can form the basis for robust project design The
scorecard can be used as a mechanism for each partner to set out what
they wish to achieve from the cooperation activity and what they are able
to contribute Each partner can then identify what they expect of the other
partners Using the scorecard approach can also assist in identifying where
potential opportunities might have been overlooked in the original project
design and where small changes might reap large rewards This can then
form the basis of the collaboration agreement between parties
Project partners can then use their own stakeholder scorecard as a means
for internal review and monitoring of the collaboration activity This
provides a useful mechanism for identifying whether desired outcomes are
being achieved and the extent to which anticipated inputs and activities are
occurring As a dynamic process the scorecard can be updated as
activities are added or amended or other inputs change
The partnership as a whole can also use the scorecard as the basis for
ongoing project reviews This can consider whether the original
assumptions remain valid and whether some activities are more appropriate
than others in achieving the desired objectives Crucial to the review
process is to ensure that all partners are realizing the benefits that they
sought initially and if not how this might be rectified The scorecard
approach provides a means of structuring this review process
In order to promote the use of the scorecard approach the Commission may wish to
provide a guide to its use in the context of university-business cooperation activities
This would offer the opportunity to develop worked examples tailored to different
cooperation purposes Such a guide could also incorporate advice on practical
techniques for measuring the outcomes of cooperation activities In the first instance
this could form part of a toolkit available to assist those parties interested in applying
for Knowledge Alliance projects
HOW TO OBTAIN EU PUBLICATIONS
Free publications
bull one copy
via EU Bookshop (httpbookshopeuropaeu)
bull more than one copy or postersmaps
from the European Unionrsquos representations (httpeceuropaeurepresent_enhtm)
from the delegations in non-EU countries (httpeeaseuropaeudelegationsindex_enhtm)
by contacting the Europe Direct service (httpeuropaeueuropedirectindex_enhtm) or
calling 00 800 6 7 8 9 10 11 (freephone number from anywhere in the EU) ()
() The information given is free as are most calls (though some operators phone boxes
or hotels may charge you)
Priced publications
bull via EU Bookshop (httpbookshopeuropaeu)
Priced subscriptions
bull via one of the sales agents of the Publications Office of the European Union
(httppublicationseuropaeuothersagentsindex_enhtm)
NC-0
1-1
4-2
99-E
N-N
doi10
HOW TO OBTAIN EU PUBLICATIONS
Free publications
bull one copy
via EU Bookshop (httpbookshopeuropaeu)
bull more than one copy or postersmaps
from the European Unionrsquos representations (httpeceuropaeurepresent_enhtm)
from the delegations in non-EU countries (httpeeaseuropaeudelegationsindex_enhtm)
by contacting the Europe Direct service (httpeuropaeueuropedirectindex_enhtm) or
calling 00 800 6 7 8 9 10 11 (freephone number from anywhere in the EU) ()
() The information given is free as are most calls (though some operators phone boxes
or hotels may charge you)
Priced publications
bull via EU Bookshop (httpbookshopeuropaeu)
Priced subscriptions
bull via one of the sales agents of the Publications Office of the European Union
(httppublicationseuropaeuothersagentsindex_enhtm)
NC-0
1-1
4-2
99-E
N-N
doi10