100204 designing a performance measurement system for the research activities a reference framework...
TRANSCRIPT
8/4/2019 100204 Designing a Performance Measurement System for the Research Activities a Reference Framework and an …
http://slidepdf.com/reader/full/100204-designing-a-performance-measurement-system-for-the-research-activities 1/14
Designing a performance measurement system for the
research activities: A reference framework and an
empirical study
Vittorio Chiesa a, Federico Frattini a,*, Valentina Lazzarotti b,Raffaella Manzini b
a Politecnico di Milano, Dipartimento di Ingegneria Gestionale, Milan, Italyb Universita Carlo Cattaneo – LIUC, Castellanza, Varese, Italy
1. Introduction
Defining and implementing a performance measurement system (PMS) within the company is
considered a critical activity for supporting decision making, motivating people, stimulating learning,
improving coordination and communication (Shank and Govindarajan, 1993; Schumann et al., 1995;
Kerssens-van Drongelen and Bilderbeek, 1999). In other words, the PMS is nowadays considered
fundamental for achieving the company’s objectives. As a consequence, all the main activities,
processes and/or functions within companies have recently become the object of a PMS: not only the
J. Eng. Technol. Manage. 25 (2008) 213–226
A R T I C L E I N F O
Article history:
Available online 8 August 2008
JEL classification:
O32
Keywords:
Research & development
Performance
measurement
Performance
measurement system
Biotechnology
Pharmaceutical research
A B S T R A C T
Designing a performance measurement system (PMS) for R&D
activities is a very critical but challenging task for supporting
decision making and people motivation. Therefore, the subject iswidely discussed in literature, but the use of a PMS for R&D is still
uncommon among companies.
The paper aims at making a step further in the field, elaborating a
reference framework that describes the logical steps for the
definition of a PMS for R&D. Moreover, the problem of designing
an effective PMS is in-depth studied in a real context, a biotech
company that operates in the field of pharmaceutical research.
ß 2008 Elsevier B.V. All rights reserved.
* Corresponding author at: Politecnico di Milano, Dipartimento di Ingegneria Gestionale, Via Giuseppe Colombo, 40, 20133Milan, Italy. Tel.: +39 02 2399 2796; fax: +39 02 2399 2720.
E-mail address: [email protected] (F. Frattini).
Contents lists available at ScienceDirect
J. Eng. Technol. Manage.
journal homepage: www.elsevier.com/locate/jengtecman
0923-4748/$ – see front matter ß 2008 Elsevier B.V. All rights reserved.
doi:10.1016/j.jengtecman.2008.07.002
8/4/2019 100204 Designing a Performance Measurement System for the Research Activities a Reference Framework and an …
http://slidepdf.com/reader/full/100204-designing-a-performance-measurement-system-for-the-research-activities 2/14
primary ones, such as production or logistics, but also the supporting ones, such as all the adminis-
trative processes.
According to this trend, R&D as well is being considered as a set of activities and processes whose
performance should be monitored and measured, particularly because in many competitive contexts
technological innovation (and, hence, the results of R&D activities) is the main source of sustainable
competitive advantage.
However, defining a PMS for R&D activities is recognized in literature as a very difficult task, since
effort levels may not be observable in quantitative, measurable terms, success is uncertain (and
influenced by uncontrollable factors) and it can be assessed only after long delays (Tipping et al., 1995;
Brown and Svenson, 1988; Kerssens-van Drongelen and Bilderbeek, 1999; Loch and Tapper, 2002). As
a consequence, in the last years many contributions have been written aimed at discussing the subject
and suggesting possible approaches (Pappas and Remer, 1985; Brown and Svenson, 1988; Chiesa and
Masella, 1996; Werner and Souder, 1997; Hauser, 1998; Driva and Pawar, 1999; Driva et al., 2000; Poh
et al., 2001; Godener and Soderquist, 2004). Such contributions have so far concentrated mainly on
performance measurement , i.e. on defining a set of dimensions of performance to be controlled and the
metrics (or indicators) to be used for the measurement of such performance. Far less contributions are
dedicated to the definition of a whole performance measurement system, i.e. an integrated system not
only able to measure a specific set of performance, but also to explain the managerial and
organizational meaning of each measure, to suggest the most appropriate use of each measure and to
analyze R&D performance with respect to the overall company strategy. The main contributions in this
direction come from Kerssens-van Drongelen and Cook (1997) and Kerssens-van Drongelen and
Bilderbeek (1999), who applied the concept of Balanced Scorecard (Kaplan and Norton, 1992) to R&D,
and from other authors (Bremser and Barsky, 2004) who have adopted more recently a similar
approach.
This paper aims at making a further step in this field, i.e. in-depth studying the problem of defining
a system for performance measurement in R&D units, and not only to identify some metrics and
indicators. In particular, the focus is on the R&D units of companies for which technological
innovation, and, hence, R&D activities are critical for competition. These companies, in fact, aregenerally characterized by a very complex and dynamic R&D environment and, hence, represent a
challenging field for this study.
According to this aim, a reference framework is identified in the paper, that represents a
systematization of literature contributions in the field, and describes the logical steps for the
definition of the PMS for R&D. The framework, starting from the corporate strategy and the R&D
strategy, comes to the definition of:
the dimensions of performance to be measured and the related indicators, the structure to be defined for the measurement system, the process aspects to be implemented for the proper operation of the system.
In a few words, the suggested framework gives a ‘‘practical’’ guide that should help managers in the
definition of all the elements of a PMS, in accordance with the overall company’s strategy.
Then, the framework is applied to a specific case study, with the aim of:
1. verifying the actual applicability of the framework in a real context, in which all the ‘‘contingencies’’
are taken into consideration, exploring in detail the problems and difficulties emerging during the
application;2. enriching, if possible, the framework itself and/or modifying it, according to the evidence emerged
during the application;3. giving a very detailed and concrete example of how the huge literature on performance
measurement , which gives many suggestions in terms of dimensions of performance, indicators,process aspects, can be integrated within a system, internally coherent and adequate for a specific
strategic context;4. discussing the possible generalization of the framework, i.e. verifying whether and how it can be
applied to other contexts than the one analyzed in this paper.
V. Chiesa et al. / J. Eng. Technol. Manage. 25 (2008) 213–226 214
8/4/2019 100204 Designing a Performance Measurement System for the Research Activities a Reference Framework and an …
http://slidepdf.com/reader/full/100204-designing-a-performance-measurement-system-for-the-research-activities 3/14
The case study, as already pointed into evidence, has been selected in a context where
technological innovation represents the critical source of competitive advantage. The analyzed
company, NiKem Research, is a biotech firm that operates in the field of pharmaceutical research,
mainly offering services with a high technical and scientific content and capable of supporting the new
drug’s R&D process. This is an interesting case, in which a company almost collapses on its R&D
function that represents the 90% of the overall activities.
The paper is structured as follows: Section 2 gives an overview of the literature on R&D PMS and
presents the proposed framework; Section 3 describes the case study and the application of the
proposed framework; finally, Section 4 discusses the results and draws some managerial implications.
2. A literature review and a theoretical framework
There are many definitions of technological innovation. Here, we assume the definition given by
Freeman (1976), as re-adapted by Chiesa (2001). Technological innovation is a process which includes
the technical, design, manufacturing , management and commercial activities involved in the marketing
of a new (or improved) product or the first use of a new (or improved) manufacturing process or
equipment . Different generations of the innovation process are identified in literature beginning
from the 1960s to the present (Chiesa, 2001). In particular, the latter two generations emphasise
that technological innovation is not sequential, is cross-functional by nature and often multi-firm.
Links with suppliers and customers are very strong along with the whole innovation process. Links
with firms take place in a variety of forms (joint ventures, consortia, alliances, contracts, etc.). R&D
activities clearly play an important role in the process of innovation. There are many definitions of
R&D activities as well. A traditional classification concerns the macro-phases which make up the
R&D process (Chiesa, 2001). It is possible to identify: basic research, which is ‘‘an activity aimed to
generate knowledge related to the working principles of natural and social science without direct
relation to industrial applications (products, services, production processes)’’; applied research,
which is ‘‘aimed to the production of knowledge required to define the means to fulfil a specific and
explicit need’’; development , which ‘‘consists of the systematic use of knowledge oriented to thedevelopment of materials, methods, tools, systems’’. Development is composed of a series of
phases: design, prototyping and testing, engineering, installation, maintenance and post-
commercialisation service. In parallel with the changed conception of the innovation process
(the generations of the innovation process), different generations of R&D activities have been
identified. From a concept of R&D with no interaction with the rest of the company and considered
as an overhead cost, the importance of integration in several forms has been progressively pointed
out: e.g. in terms of links between strategy and R&D activities, cross-functional teams and
integration with customers. The last generation emphasises that R&D is ‘‘a part of a total innovation
system including competitors, suppliers, customers and distributors’’.
If we examine the R&D activities in terms of their accountability, that is the most relevant topic for
this work, other significant changes can be pointed out. In fact, R&D was once considered to be aunique, creative and unstructured process that was difficult, if not impossible, to manage and control
(Kerssens-van Drongelen and Cook, 1997). Today, R&D consideration has also changed concerning
with this aspect, although it is generally recognized that it is challenging to establish accountability for
many R&D activities (Tipping et al., 1995; Brown and Svenson, 1988; Kerssens-van Drongelen and
Bilderbeek, 1999). Since the 1990s, relevant changes in the business environment (in terms of
intensified competition, shortened product life cycles, advanced technology and automation, etc.)
have focused managers’ attention on R&D’s contribution to competitive advantage (Kerssens-van
Drongelen and Cook, 1997). These changes have brought companies to pay attention to their R&D
processes in terms of efficiency, internal and external customer focus, time to market (TTM),
innovativeness, etc. (Kumpe and Bolwijn, 1994). In other words, R&D is a key strategic issue that must
be aligned with corporate and business strategies and their various expected performance ( Pearsonet al., 2000; Bremser and Barsky, 2004). In order to encourage such alignment, a performance
measurement system is required, composed of the following elements, linked through reciprocal
relations (Kaplan and Norton, 1992; Kerssens-van Drongelen and Cook, 1997; Kerssens-van Drongelen
and Bilderbeek, 1999; Bremser and Barsky, 2004):
V. Chiesa et al. / J. Eng. Technol. Manage. 25 (2008) 213–226 215
8/4/2019 100204 Designing a Performance Measurement System for the Research Activities a Reference Framework and an …
http://slidepdf.com/reader/full/100204-designing-a-performance-measurement-system-for-the-research-activities 4/14
the dimensions of performance to be monitored and the related indicators, qualitative and/or
quantitative; the structure of the system, i.e. the articulation in ‘‘controlled objects’’; the process aspects to be defined for a proper working of the system, i.e. all the norms and rules
governing the PMS, the timing and frequency of measurement for each controlled object and for the
different dimensions of performance, the role and tasks of people involved in the implementation of
the system.
The contributions on the contextual approach give many relevant suggestions for the design of a
PMS for R&D activities, identifying the contextual factors (for a literature review, see Kerssens-van
Drongelen and Cook, 1997) that influence the definition of the PMS’s elements:
the company’s R&D strategy, in terms of long-term objectives (or critical performances) that are
coherent with the business strategy, the competitive context (rules of competition and main
competitive pressures) and the general environmental features (macroeconomic factors, institu-
tional norms, social and cultural characteristics); the entities to be monitored (i.e. R&D division or department, sub-department, project, individual)
organized in a certain structure (i.e. according to scientific disciplines, typology of activities, product
line, project, etc.); the type of activities to be monitored (i.e. basic research; applied research; development, that are
tasks with different degrees of uncertainty); the PMS’s objectives (i.e. the purpose of the measurement, e.g. motivating people, diagnosing
activities, supporting decisions, stimulating learning, improving communication and coordination
between R&D and other company’s organizational units); the resources (time, money, people, competencies) available for the implementation of the PMS.
A tentative systematization of all those suggestions is given in the framework in Fig. 1. Such a
framework describes the logical entities to be considered in the definition of the PMS for R&D and itcan be helpful to practically guide the system design. The framework is made up of two parts:
the contextual factors; the consequent PMS’s elements; and it enlightens the following aspects: within the context, those factors that constitute the R&D ‘‘environment’’ (i.e. R&D strategy, R&D
organization and management, R&D activities) can be distinguished. The R&D strategy drives
Fig. 1. A reference framework for defining a contextual R&D PMS.
V. Chiesa et al. / J. Eng. Technol. Manage. 25 (2008) 213–226 216
8/4/2019 100204 Designing a Performance Measurement System for the Research Activities a Reference Framework and an …
http://slidepdf.com/reader/full/100204-designing-a-performance-measurement-system-for-the-research-activities 5/14
Box 1. The links between the contingencies and the PMS elements
Purposes of the measurement and dimensions of performance If, as it is common especially in highly uncertain and dynamic R&D environments, the measure-
ment system serves the purpose of motivating people , particular cautions are required (Kerssens-van Drongelen and Cook, 1997). In the case of motivational purpose, the assumption is that byfeeding back information about their performance, possibly coupled with incentives, people will bemotivated to change their behaviour. Two conditions should be satisfied in order to design aneffective motivational system. First of all, the system has to include only those factors that can becontrolled by the person subjected to the measurement. Furthermore, if an incentive or a rewardprogram is coupled with the measurement (i.e. bonuses, promotions), the system should measureall the important aspects of a work. This is due to the fact that there is a human tendency to directmore effort towards tasks that are being measured and consequently rewarded, giving lessattention to tasks that are not. However, especially in research activities, tasks may not beobjectively measurable (because of their intrinsic high uncertainty and relevant content of crea-tivity), although they are generally critical in order to obtain appreciable results in the long term.This circumstance, linked to the type of activities contingency, makes it particularly challenging toestablish accountability criteria.Purpose of the measurement and process aspects The purpose of the measurement is a contextual factor that is capable of influencing also somePMS process aspects. For example, whether the main PMS’s objective is to diagnose R&Dactivities, it is necessary to choose standards to measure performance against that give thepossibility to objectively judge the value of a specific indicator and to make comparisons overtime, thus enlightening eventual improvement in R&D performance. At the same time, thefrequency and the timing of the measurement must be chosen so that the PMS is capable of gathering and transferring performance data to R&D and top managers timely and coherently withtheir informational needs.Type of activities and dimensions of performance In a research context, literature suggests that the system should include dimensions of perfor-
mance regarding the research process (a dimension of expected performance could be, forexample, ‘‘the achievement of professionalism standards’’, and a possible indicator ‘‘the frequencyof participation to international conferences’’) rather than the research output , i.e. dimension of performance such as ‘‘the work productivity’’, measured by ‘‘the number of patents’’ indicator(Loch and Tapper, 2002; Hauser, 1998; Chiesa et al., 1996). Rewards should not only be financial, butalso emphasise recognition (Loch and Tapper, 2002).Type of activities and process aspects Also the process aspects of the PMS (e.g. standards/norms to measure performance against)should be adapted to the activities features. For example, if the ‘‘number of patents’’ is the indicatorfor individual performance measurement, two opposite norms could be used to measure perfor-mance against: explicitly recorded standards or subjective and implicit norms. The choice shouldtend to vary with the characteristics of the work. Reasonably, for activities with a high uncertainty,
the subjective techniques should be preferred because they take into consideration a wider range of relevant aspects (qualitative aspects and/or environment conditions, that are not under the controlof the researcher).R&D strategy and dimensions of performance This relationship enlightens that the choice about the dimensions of performance to be monitoredis driven by R&D strategy. In effect, the strategic objectives are the critical performances to bepursued to guarantee company’s competitive success. A PMS is useful if it manages to monitorthese critical performances (or, at least, a part of them) that are assumed as dimensions of performance of the system itself.R&D strategy and process aspects The PMS process aspects can be influenced also by the choices the company makes in terms of R&D strategy and objectives. For example, whether the firm means to undertake a time-basedcompetition that requires short time to market (TTM) and technological pioneering behaviours,
it is necessary that coherent decisions are made in terms of timing and frequency of themeasurement, so that the PMS is capable of supporting R&D and top management decisionprocesses.
V. Chiesa et al. / J. Eng. Technol. Manage. 25 (2008) 213–226 217
8/4/2019 100204 Designing a Performance Measurement System for the Research Activities a Reference Framework and an …
http://slidepdf.com/reader/full/100204-designing-a-performance-measurement-system-for-the-research-activities 6/14
the choice of the R&D activities to be internally carried out and their organization in a certain
structure; the PMS’s available resources can influence the PMS’s objectives, because they can represent a
constraint that limits the informative completeness of the system and thus the achievable purposes
(and the dimensions of performance to be monitored); the PMS’s objectives are driven also by the R&D management, e.g. because of a leadership style that
stresses more or less the motivational aspects of the system; as far as, the objectives are driven by
the type of activities (i.e. a high level of creativity and complexity can require a particular attention
to researchers’ motivation and coordination); all the contextual factors influence the design of the PMS’s elements (see Box 1 for some examples of
relationships); all the PMS’s elements are interrelated and in close relationship, thus stressing their reciprocal
dependence and the systematical nature of the PMS.
Explanations concerning some relations between the contextual factors and the PMS’s elements
are illustrated in Box 1. Whether it may appear obvious that the structure of the system should be
defined in order to collect information about the performance of all R&D organizational units, the link,
for example, between the purpose of the measurement and the PMS’s dimensions of performance is
less evident. The explanations in Box 1 try to clarify these links and enlighten further complexities
deriving from another contextual factor, i.e. the type of activities that can also influence the definition
of the PMS’s process aspects.
3. The case study and the framework application
The framework has been first applied in a real context with the aim of guiding the definition of a
PMS in a pharmaceutical company, NiKem Research. The problems emerged during the application of
the framework have indicated the need to integrate it with a list of key questions, that replicate its
overall structure and can be used to guide the PMS design coherently with the framework underlyinglogic. This has shown to be an important step towards the practical orientation of the model itself.
Fig. 2 summarizes the results of this empirical investigation and allows for a preliminary
explanation of the reference model underlying logic.
After a brief description of the company’s profile, the contextual factors and the associated PMS’s
elements are illustrated more in detail.
3.1. Company’s profile
NiKem Research is an Italian company founded in 2001 that operates in the field of pharmaceutical
research, mainly offering services with a high technical and scientific content and capable of
supporting the new drug’s R&D process. Its origins could be traced back to the merger between Glaxo-Wellcome and SmithKlineBeecham, two big pharmaceutical companies with Italian operative units,
respectively set in Verona and Milan. The merger took place in 2000 and brought to the birth of GSK
(GlaxoSmithKline); the top management established that all the activities would be aggregated in
Verona, thus implying the partial transferring of SmithKlineBeecham’s personnel from Milan to the
Venetian city. A part of SmithKlineBeecham’s discovery team, that was going to be totally cancelled,
decided not to throw away the excellent competences in medicinal chemistry they have developed
during more than 20 years. They thus presented a spin-off project to GSK aimed at the creation of a
small, highly specialized company, capable of supporting pharmaceutical firms’ R&D processes with
specific services.
3.2. Contextual factors
3.2.1. General environment and competitive context
When NiKem Research was born, the features of the pharmaceutical market were particularly
propitious for the birth and the success of this kind of ventures. In fact, the increasing complexity,
V. Chiesa et al. / J. Eng. Technol. Manage. 25 (2008) 213–226 218
8/4/2019 100204 Designing a Performance Measurement System for the Research Activities a Reference Framework and an …
http://slidepdf.com/reader/full/100204-designing-a-performance-measurement-system-for-the-research-activities 7/14
Fig. 2. The framework application.
V. Chiesa et al. / J. Eng. Technol. Manage. 25 (2008) 213–226 219
8/4/2019 100204 Designing a Performance Measurement System for the Research Activities a Reference Framework and an …
http://slidepdf.com/reader/full/100204-designing-a-performance-measurement-system-for-the-research-activities 8/14
costs, risks and investments, typical of new drugs’ discovery and development processes, were forcing
big pharmaceutical companies to reorganize their R&D structures and to outsource R&D activities to
smaller and more flexible firms (Muffatto and Giardina, 2003). In order to understand the
characteristics of the service offered by NiKem Research, it could be useful to briefly describe the
structure of the research and development process in the pharmaceutical industry (for more details
see Muffatto and Giardina, 2003; Chiesa, 2003; Katzung, 2000; Paoletti et al., 2001). The identification
of a new drug in the pharmaceutical industry is the result of a complex R&D activity that is basically
composed of two steps: the first one relates to the identification of a new compound (drug discovery),
the second is concerned with the development of the discovered compound (drug development),
followed by approval of the qualified regulatory authority. The drug discovery phase can be split into
two fundamental stages: the biological activities and the chemical activities. The former is composed
of the target identification and the target validation activities, while the latter includes the lead
generation and the lead optimization tasks.
The identification of the target is the starting point of the process, and it consists in the pointing
out of a particular molecular target (e.g. a gene or a protein), that is responsible for a specific
pathology. The next step of the biological activities (target validation) implies a deep investigation
into the characteristics of the identified target in order to discover how it acts and how it generates
the disease. Once the target has been identified and the process through which it works has been
understood, it is necessary to discover, through an intense screening activity, a set of compounds
(named leads) that, interacting with the target, may have positive impact on the pathology. This
stage of the discovery phase is called lead generation, and it ends with the identification of a limited
number of compounds that need to be optimized. The lead optimization phase of the discovery
process is a very critical, risky and innovative one that requires excellent competences in the field of
medicinal and combinatorial chemistry. It basically consists in the analysis of the generated
chemical compounds in order to identify the most balanced chemical entity in terms of potency at
the molecular target, selectivity, efficacy, defects, pharmacokinetic and toxicological properties. To
evaluate the safety and effectiveness of the optimized pharmaceutical compound before it is
destined to the market, it has to be carefully tested, according to very severe and strict procedures.This phase is called drug development, and it is divided into two macro-activities: the pre-clinical
and the clinical tests. The former consists of a set of laboratory tests (called in vitro) and tests on
animals (named in vivo), with the aim of evaluating the drug’s safety and toxicity level. The clinical
tests, on the other hand, are articulated into three sequential steps: phases I, II and III. They are
conducted on human volunteers in order to fix the proper dosage and to assess the drug’s
effectiveness and tolerability. Once the development stage is ended, the results of the tests are
gathered and analyzed, and, eventually, a request for the market approval of the new drug is
forwarded to the qualified authority, that may approve its commercialisation or not. Going back to
the recent evolutions in R&D processes of the pharmaceutical industry, it should be remembered
that, with the ending of the human genome project, all the 25,000 human genes were identified;
about 3000 were recognized to be ‘drugable’ or ‘physiologically relevant’ molecular targets. Theyrepresented, together with the diffusion of high throughput screening (HTS) techniques, an
enormous incentive for pharmaceutical companies to intensify their screening activity; this
brought to the identification of a huge number of lead compounds that needed to be optimized. It
was especially the large scale to be achieved in the tasks of lead generation and optimization in
order to keep competitive, their high level of inherent risk and costs, and the competences they
require, that forced more and more big pharmaceutical companies to outsource these activities
to highly specialized and flexible organizations, thus externally acquiring the missing skills and
critical mass.1
3.2.2. Strategy and objectives
In the propitious context just described, the spin-off project was approved by GSK, that decided totransfer, through a management-by-out operation, personnel, laboratories and other facilities to the
1 Due to the fact that NiKem almost collapses on its R&D function, in Fig. 2, that represents the applied framework there is no
relevant distinction between business strategy and R&D strategy.
V. Chiesa et al. / J. Eng. Technol. Manage. 25 (2008) 213–226 220
8/4/2019 100204 Designing a Performance Measurement System for the Research Activities a Reference Framework and an …
http://slidepdf.com/reader/full/100204-designing-a-performance-measurement-system-for-the-research-activities 9/14
new-born NiKem Research, and to support the start-up phase of its activity with a research contract of
2 years, capable of covering the 85% of its start-up costs. In 2001, on August 1st, NiKem Research was
officially established and started working as service provider, highly specialized in the activities of
‘lead generation’ and, especially, ‘lead optimization’, with a staff of 25 researchers. NiKem Research is
thus a typical example of Contract Research Organizations (CRO), where ‘contract research’ is defined
as ‘. . . the activity by which a client hires the services of an external organization to carry out a specific
piece of R&D’ (Haour, 1992).
In 2003, NiKem Research’s top management realized that the company’s business model,
completely based on the provision of specific services, would reveal itself too fragile and not capable of
assuring high value and steady profits on the long term. This was mainly due to the fact that the
pharmaceutical market is traditionally exposed to significant sales fluctuations that negatively reflect
on the performances of small CROs like NiKem Research. Another threat was represented by those
service providers operating in emerging countries like India, Russia or China; it was thought they
would be capable, in 4–5 years, of offering high quality services, comparable to European or American
companies’, but at a significantly lower price (from 1/3 to 1/5). This convinced NiKem Research’s top
management to modify the firm’s strategy and to dedicate part of the available resources to internal
research activities, with the aim of generating novel leads to be internally optimized and developed
until they are ready for the clinical stage of the Drug Development process. In order to carry out the
pre-clinical tests, pharmacological competences are necessary; this has brought NiKem to collaborate
with the ‘‘Istituto Nazionale dei Tumori’’ and with various Italian universities (e.g. Firenze, Ferrara). At
that stage of the R&D process, the leads would be licensed to or partnered with big pharmaceutical
companies that own the financial resources and the complementary assets necessary to complete the
development phase and to introduce the new drugs into the market. The internal research activity
assures NiKem Research another advantage, i.e. the possibility to have a buffer of highly specialized
human resources that can be used to face sudden and transitory increases in the request for discovery
services.
Even if the service provision and the internal research activities require that very similar types
of scientific challenges are faced and that similar technologies and competencies are acquired (e.g.pharmacokinetic, preliminary toxicology, MTS, additional ADME profiling), they significantly differ
when we consider the critical success factors where NiKem Research has to excel in order to gain a
competitive advantage. In the business of discovery services, the objectives to be pursued are the
quality level of the service (at a cost level aligned to the competitors’ one), the respect of the
promised delivery time (punctuality) and the confidentiality in the management of the
information concerning the clients’ compounds. On the other hand, these variables do not
represent competitive pressures in the strategic area of internal research, where it is far more
important to adopt a long-term perspective and to try to gain the leadership in the identification
of novel candidates to clinical development. In this area, even if concrete results are obviously
necessary, time constraints are less severe due to the fact that specific clients’ pressures are not
relevant.After all, a critical objective has been identified at the overall company level, that becomes another
specific objective for each business area. This critical dimension consists in the building of an external
technology reputation. This factor is very important first of all because it provides contacts with new
potential clients in the business of discovery services. In addition, it provides relations with big
pharmas and biotechs in order to improve the internal research pipeline.
3.2.3. R&D activities
According to the traditional classification concerning the macro-phases which compose the R&D
process (see, e.g., Chiesa, 2001), it is possible to classify NiKem Research’s activity as basic and applied
research and therefore strongly creative, uncertain and quite unstructured.
3.2.4. R&D organization and management
It can be stated that NiKem Research is organized on the basis of a matrix structure. The researchers
are grouped in departments, according to the discipline in which they are specialized. The
‘‘departmental dimension’’ of the matrix structure is necessary in order to assure a high level of
V. Chiesa et al. / J. Eng. Technol. Manage. 25 (2008) 213–226 221
8/4/2019 100204 Designing a Performance Measurement System for the Research Activities a Reference Framework and an …
http://slidepdf.com/reader/full/100204-designing-a-performance-measurement-system-for-the-research-activities 10/14
specialization in the company’s core scientific competences (the main departments are: medicinal
chemistry, analytical chemistry, computational chemistry, screening). Within each department, a
director is appointed. He is responsible for the management of the activities and the resources in the
department and for its scientific results. Furthermore, each department includes a series of
laboratories. They represent a lower level along the departmental dimension and are defined
according to ‘‘scientific specialization’’ criteria. A ‘‘laboratory head’’ manages the activities of this
organizational unit.
On the other hand, the ‘‘project dimension’’, by means of inter-departmental teams, is important to
put together the transversal skills that are necessary to produce the company outputs. Within each
project, a team leader is appointed as well; he is, in general, the researcher with the major experience,
or charisma, or specialized in the most crucial discipline for the project success. He is responsible for
the results achieved by the team. In this matrix structure, however, a major organizational power is
assigned to the department structure rather than to the project structure. This power can be in
practice appreciated in terms of a wider possibility in the orientation of the researchers’ activities,
through the definition and the valuation of their goals and responsibilities.
After all, NiKem recognizes that human resources are crucial in order to achieve high quality
services. Consequently, the leadership style is surely a participative one with the basic intent of
obtaining people collaboration.
3.2.5. PMS objectives
The leadership approach and the type of activities (i.e. their high level of creativity and complexity
that requires a particular attention to researchers’ motivation and coordination) guide the choice of
the PMS’s objectives towards a clear prevailing of the motivational purpose, even if also the aspect of
diagnosing activities is not ignored. Particular attention has thus been paid to the alignment of the
indicators with the specific goals and responsibilities of the people subjected to the measurement.
This means that the system has to include only those factors that can be controlled by the evaluated
people. This aspect is practically considered when the PMS’s elements are defined, in particular when
the dimensions of performance and the related indicators are assigned to the different objectscomposing the PMS’s structure.
In any case, it has been realized that, in order to enhance the motivational capability of the
performance measurement system, the objectives assigned by a chief to a single researcher must be
collectively defined (‘‘shared objectives’’). Furthermore, they must be measurable, even if they do not
necessarily produce tangible outputs. However, a certain degree of performance ‘‘un-measurability’’ is
expected because of the intrinsic nature of the company’s activity, that is basic and applied research,
and therefore strongly creative, uncertain and quite unstructured. This circumstance influences the
financial incentives (bonuses) that are coupled with the measurement in order to enforce the
motivational purposes of the system. In fact, incentives are depending not only on the specific
individual performances, but they are linked also to the company’s overall earnings. This link is
justified on the basis of the assumption that each researcher (by means of his competences, behaviour,cultural values, etc.), contributes in any case to the company’s success, even if personal, concrete and
measurable results are far from being evident.
The purpose of diagnosing activities can be recognized in the project-progress monitoring. This
type of control is formal and structured, by means of reports and meeting with the clients. The
necessity of a formal approach can be appreciated when we consider in particular the provision of
lead generation and lead optimization services. These stages of the R&D process are, in fact, very
critical for the drug development success and involve customer assets whose intellectual property
must be protected. Besides, the likely client’s criticism is increased by the fact that NiKem has
relationships with companies in competition. Further complexity is caused by the long duration of
the relationships. Theformalization of theapproach is thus considered a mean to limit thecustomer’s
worries.
3.2.6. PMS resources
NiKem Research small dimensions have an inevitable impact on the resources available to design
and then implement the PMS. Above all, human resources availability has been identified as the most
V. Chiesa et al. / J. Eng. Technol. Manage. 25 (2008) 213–226 222
8/4/2019 100204 Designing a Performance Measurement System for the Research Activities a Reference Framework and an …
http://slidepdf.com/reader/full/100204-designing-a-performance-measurement-system-for-the-research-activities 11/14
important constraint in the PMS’s design2 as far as few people will undertake the activities necessary
to the proper operation of the system. This circumstance has lead to the definition of a not very
sophisticated system (i.e. including few dimensions of performance, few indicators, a simple
structure), although representing a good starting point for a further deepening.
3.3. PMS elements
3.3.1. Dimensions of performance
The choice of the dimensions of performance to be monitored is driven by R&D strategic objectives.
In Fig. 2 are reported the selected dimensions of performance (one or more) corresponding to each
strategic objective. For example, in the business area of discovery services, the ‘‘service quality level’’
objective has been translated into two dimensions of performance to be monitored in order to assure
its achievement (i.e. service effectiveness and capability of acquiring new technologies and
competencies), as far as the ‘‘service cost level’’ has been turned into a dimension of internal efficiency
(i.e. respect of the planned service costs). On the other hand, in the business area of internal research,
the dimensions of performance, coherent with the objective of ‘‘gaining the leadership in the
identification of novel candidates to clinical development’’, are basically the quality of identified/
optimized leads and the capability of scouting new collaboration opportunities.
3.3.2. Indicators
It is important to choose the correct qualitative and/or quantitative indicators in order to make the
dimensions of performance operatively measurable. It would be important to select those
performance indicators that allow for a simple and direct measurement and that, even if not
questionable, can be collectively discussed. This indicates a clear preference towards quantitative
indicators. Anyway, the characteristics of uncertainty, risk and un-measurability of R&D activities, as
said above, makes it often necessary to integrate quantitative indicators with non-numeric and
qualitative metrics. This tendency can be clearly recognized in the case of NiKem Research, where both
these types of indicators are applied, even to measure the same dimension of performance. A joint useof quantitative and qualitative indicators has been established, for example, to measure the capability
of acquiring new technologies and competencies, where the numeric indicator based on the ‘‘time
needed to acquire new technologies’’ has been integrated with the qualitative appreciation of the
international relevance of the technologies/competencies acquired.
3.3.3. Structure
The structure of the system reflects the organizational R&D structure as described above. This
means that the monitored objects are basically the ‘‘project’’ and the ‘‘department’’. In addition, the
focus on the motivational purpose of the system leads to dedicate a particular attention also to the
performance of individual researchers.
The assignment of the selected indicators to the structure levels is driven by the basic purpose of the performance measurement system, i.e. motivating people. In other words, it is important to make
a specific organizational unit responsible only for those performance indicators it can directly and
completely influence.
In particular, it has been recognized that the controllable factors tend to vary moving across the
organizational levels. For example, the individual researcher cannot be held responsible for the
success of a specific screening task (i.e. the quality of novel identified leads), due to the fact that the
uncertainty of this activity is too high and out of his control (the chances a molecule has of becoming a
lead are very low). On the other hand, it can be considered a responsibility of a research project as a
whole, seen that the project manager can influence its output, e.g. by improving processes and by
allocating resources to a good portfolio of research activities.
The match between the indicators and the structure levels is a very critical process because of itsmotivational implications. Fig. 2 also reports the final output of the assignment choices.
2 Apart from our contribution, a ‘‘design team’’ has been appointed including the Chief Operating Officer, an external
consultant and two researchers, respectively employed in the discovery services and in the internal research activities.
V. Chiesa et al. / J. Eng. Technol. Manage. 25 (2008) 213–226 223
8/4/2019 100204 Designing a Performance Measurement System for the Research Activities a Reference Framework and an …
http://slidepdf.com/reader/full/100204-designing-a-performance-measurement-system-for-the-research-activities 12/14
3.3.4. Process aspects
It has been observed that the researchers, because of their high level of education, are very
autonomous in carrying out their work, whose creative nature probably stresses this behaviour (they
often say ‘‘we need to be autonomous in our work’’). The autonomy is typical of both the business
areas. The respect of this working condition requires that the measurement system is not too strict and
constraining. This need has influenced the choice of the norms that NiKem Research is going to
implement. Subjective standards are, in fact, prevalent in order to encourage people initiative also in
the case of results that are different from those agreed. For example, an actual number of patents that
is lower than the standard number, can be anyway considered a good performance through subjective
evaluations by superiors, that can be rightly less severe than a quantitative method objectively
applied.
Considering the timing of the system, it has been decided to assign the individual objectives at the
beginning of the year. The evaluation of the objectives’ progressive achievement occurs about every
four months with a final evaluation at the end of the year.
4. Conclusions and managerial implications
This paper aims at making a step further in the understanding of the problems to be faced when
defining a system for performance measurement in R&D units. According to this aim, a reference
framework is identified, that represents a systematization of literature contributions in the field and
describes the logical steps for a proper definition of the PMS for R&D, thus giving a ‘‘practical’’ guide
that should help managers facing this challenge.
The scheme here suggested explicitly establishes relationships between contextual factors (i.e.
dimensions of the R&D ‘‘environment’’- R&D strategy, R&D organization and management, R&D
activities-, measurement system objectives and resources available for the system design and
implementation) and PMS’s elements that are reciprocally interrelated, coherently with the systemic
nature of the PMS.
Then, the framework has been applied to a company specifically selected in a context wheretechnological innovation represents the critical source of competitive advantage. This is clear when
we consider that NiKem Research, the analyzed firm, almost collapses on its R&D function that
represents the 90% of the overall activities. Above all, the case study has surely provided an enriching
experience regarding the aspects of system definition, thanks to the problems emerged during the
application of the suggested approach. In particular, it was clearly perceivable the need to integrate
the framework with a list of key questions, that replicate its overall structure and can be used to guide
the PMS design coherently with the framework underlying logic. This has shown to be an important
step towards the practical orientation of the model itself, allowing the company to deeply understand
its own strategic objectives and the consequent dimensions of performance to be monitored. The
choice of the specific indicators that allow the measurement of the dimensions of performance
revealed itself to be a very critical task, at least as challenging as their assignment to the structurelevels in order to consider the accountability and motivational implications.
Anyway, the case provides a very detailed and concrete example of how the huge literature on
performance measurement , which gives many suggestions in terms of dimensions of performance,
indicators, process aspects, can be integrated within a system, internally coherent and adequate for
a specific strategic context. In fact, various contextual factors emerge as basic aspects that can
influence the measurement system intents. In particular, it has been recognized that dimensions of
performance (and related indicators) should reflect the activity portfolio derived from the
company’s strategy and its critical long-term objectives. Moreover, the main purpose that has
brought to the introduction of the measurement system (i.e. ‘‘motivating people’’) has decisively
guided its definition. This explains why particular attention has been paid to the alignment of the
indicators with the specific goals and responsibilities of the people subjected to the measurement;the system has to include only those factors that can be controlled by the evaluated people.
Furthermore, it has been recognized that the controllable factors tend to vary moving across
the organizational levels. This means that the organizational structure has a great importance in
the measurement system design. The organizational levels to be considered and matched with the
V. Chiesa et al. / J. Eng. Technol. Manage. 25 (2008) 213–226 224
8/4/2019 100204 Designing a Performance Measurement System for the Research Activities a Reference Framework and an …
http://slidepdf.com/reader/full/100204-designing-a-performance-measurement-system-for-the-research-activities 13/14
selected performance indicators, in fact, directly derive from the ‘‘matrix’’ structure applied in
NiKem Research.
In any case, the match between the indicators and the structure levels revels itself to be a very
critical process because of its motivational implications.
The type of activity (basic and applied research) and the peculiarities of the needed human
resources (for example, their autonomy) are the other factors that have had some influence on the
system design. It has been observed that the researchers, because of their high level of education, are
very autonomous in carrying out their work, whose creative nature probably stresses this behaviour.
The respect of this working condition requires that the measurement system is not too strict and
constraining. This need has thus influenced the choice of the norms that NiKem Research is going to
implement. Subjective standards are, in fact, prevalent in order to encourage people initiative also in
the case of results that are different from those agreed.
Finally, NiKem Research small dimensions have had an inevitable impact on the resources available
to design and then implement the PMS. In particular, human resources availability has been identified
as the most important constraint in the PMS’s design and implementation tasks. This circumstance has
lead to the definition of a not very sophisticated system (i.e. including few dimensions of performance,
few indicators and a simple structure), although representing a good starting point for a further
deepening.
Further investigations are surely necessary in order to appreciate the operation of such a system
(‘‘is it accepted by the researchers?’’) and its eventual evolution. From this point of view, the
framework proposed is dynamic in its nature: as far as the contingencies and contextual factors
change, the PMS characteristics (i.e. the dimensions of performance monitored, the structure of the
system, the process aspects) coherently change as well. This is a crucial point, given the dynamic
nature of the competitive context and the rapidly changes in the rules of competition.
The synthetic framework we have elaborated in this article fits the specific characteristics of NiKem
Research, but it represents an important empirical basis for future analysis. First of all, we mean to
further investigate whether it can be usefully applied to other Contract Research Organizations
working in the field of pharmaceutical research, conducting other case studies on the matter. This willgive us the opportunity to improve and correct the proposed scheme and, probably, generalize it, seen
the similarities between CROs in terms of type of activities and strategy. Moreover, it would be
interesting to study, by means of surveys, the possibility to adapt the framework to innovative
companies working in other industries. This will probably require a deep change in the selected
dimensions of performance (and related indicators), due to completely different types of activities and
business models and, therefore, critical long-term objectives.
Another relevant topic for future research concerns the integration of the R&D PMS with the
company’s management control system, i.e. with the control of the overall economic, financial and
non financial performance of the company. This, in fact, poses interesting problems concerning: the
information flows; the style chosen for monitoring performance; the coherence between, on the one
hand, the R&D PMS elements, and, on the other, the company’s PMS elements, particularly in terms of structure and process aspects.
Acknowledgement
We thank Mr. Giuseppe Giardina, Chief Operating Officer in NiKem Research, for his contribution
by means of a deep knowledge of R&D processes and control requirements.
References
Bremser, W.G., Barsky, N.P., 2004. Utilizing the balanced scorecard for R&D performance measurement. R&D Management 34(3), 229–237.
Brown, M.G., Svenson, R., 1988. Measuring R&D productivity. Research-Technology Management 31 (4), 11–15.Chiesa, V., 2003. La Bioindustria. Strategie competitive e organizzazione industriale nel settore delle biotecnologie farm-
aceutiche. ETAS, Milano.Chiesa, V., 2001. R&D Strategy and Organisation. Imperial College Press, London.
V. Chiesa et al. / J. Eng. Technol. Manage. 25 (2008) 213–226 225
8/4/2019 100204 Designing a Performance Measurement System for the Research Activities a Reference Framework and an …
http://slidepdf.com/reader/full/100204-designing-a-performance-measurement-system-for-the-research-activities 14/14
Chiesa, V., Coughlan, P., Voss, C.A., 1996. Development of a technical innovation audit. Journal of Production InnovationManagement 13 (2), 105–136.
Chiesa, V., Masella, C., 1996. Searching for an effective measure of R&D performance. Management Decision 34 (7), 49–57.Driva, H., Pawar, K.S., 1999. Performance measurement for product design and development in a manufacturing environment.
International Journal of Production Economics 60, 61–68.Driva, H., Pawar, K.S., Menon, U., 2000. Measuring product development performance in manufacturing organisations.
International Journal of Production Economics 63 (2), 147–159.Freeman, C., 1976. Economics of Industrial Innovation. Pinter Publisher, London.Godener, A., Soderquist, K.E., 2004. Use and impactof performance measurement results in R&Dand NDP: an exploratory study.
R&D Management 34 (2), 191–220.Haour, G., 1992. Stretching theknowledge-base of the enterprise through contract research. R&D Management 22 (2), 177–182.Hauser, J.R., 1998. Research, development and engineering metrics. Management Science 44 (12), 1670–1689.Kaplan, R.S., Norton, D.P., 1992. The balanced scorecard measures that drive performance. HarvardBusiness Review 70 (1),71–79.Katzung, B.G., 2000. Farmacologia generale e clinica. Piccin, Padova.Kerssens-van Drongelen, I.C., Bilderbeek, J., 1999. R&D performance measurement: more than choosing a set of metrics. R&D
Management 29 (1), 35–46.Kerssens-van Drongelen, I.C., Cook, A., 1997. Design principles for the development of measurement systems for research and
development processes. R&D Management 27 (4), 345–357.Kumpe, T., Bolwijn, P.T., 1994. Towards the innovativefirm-challenge for R&D management. Research-Technology Management
37 (1), 38–44.Loch, C.H., Tapper, S., 2002. Implementing a strategy-driven performance measurement system for an applied research group.
The Journal of Product Innovation Management 19 (3), 185–198.Muffatto, M., Giardina, G., 2003. Innovazioni nei Processi di Ricerca in Campo Farmaceutico. Economia & Management 6, 107–
121.Paoletti, R., Nicosia, S., Clementi, F., Fumagalli, G., 2001. Farmacologia generale e molecolare. UTET, Torino.Pappas, R.A., Remer, D.S., 1985. Measuring R&D productivity. Research-Technology Management 28 (3), 15–22.Pearson, A.W., Nixon, W.A., Kerssens-van Drongelen, I.C., 2000. R&D as a business—what are the implications for performance
measurement? R&D Management 30 (4), 355–366.Poh, K.L., Ang, B.W., Bai, F., 2001. A comparative analysis of R&D project evaluation methods. R&D Management 31 (1), 63–76.Shank, J.K., Govindarajan, V., 1993. Strategic Cost Management: The New Tool for Competitive Advantage. The Free Press, New
York.Schumann, P.A., Ransley, D.L., Prestwood, D.C.L., 1995. Measuring R&D performance. Research-Technology Management 38 (3),
45–54.Tipping, J.W., Zeffren, E., Fusfeld, A.R., 1995. Assessing the value of your technology. Research-Technology Management 38 (5),
22–39.
Werner, B.M.,Souder, W.E., 1997. Measuring R&D performance– state of the art.Research-TechnologyManagement 40 (2), 34–42.
V. Chiesa et al. / J. Eng. Technol. Manage. 25 (2008) 213–226 226