Relatório Final
Dissertação para obtenção do Grau de Mestre em
Engenharia Informática e de Computadores
Júri
Junho de 2009
Dedico este trabalho à minha família e à minha namorada, Ana.
Não teria conseguido sem o vosso apoio.
Muito obrigado.
i
Abstract
Enterprises are growing increasingly hungry for information: the will to empower decisions with carefully
digested data, the wish of transform all implicit knowledge into clearly dened, explicit repositories that
can be shared by all and the desire to monitor all aspects of their internal dynamics. In the past decade,
the rush to technology has created several aws when it comes to managing computers, applications, mid-
dleware and information systems and so organisations struggle to understand how all these elements are
behaving.
Even today, as Enterprise Architectures grow in signicance and are acknowledged as advantageous arti-
facts to help manage change, their benet to the organisation has yet to be fully explored.
We therefore combine our desire for real-time evaluation with the benets of architecture representation
to produce an ontologically supported method of modelling, capturing and evaluating systemic behaviour.
We produce a conceptual framework to performing this task, while avoiding the imprecise denitions of
quality and quality attributes. According to our abstraction, transfering that responsibility to the end user
is essencial since such aspects are subjective and depend on the human context in which they exist. This
conceptualisation was materialised in a model-eval-display loop framework, and implemented using Model
Driven Software Development practices and tools.
Finally, we tested the prototype against a real-world scenario in PT-Comunicações, to observe our con-
ceptual solution in practice.
ogy
iii
Resumo
As empresas têm vindo gradualmente a atribuír uma maior importância à informação: o interesse em
potenciar as decisões usando dados cuidadosamente processados e o desejo de transformar todo o con-
hecimento implícito em repositórios de conhecimento explícito para o fácil acesso de todos. Contudo,
na última década, a corrida à tecnologia criou diversas falhas no que toca à gestão de computadores,
aplicações, middleware e sistemas de informação, que causou diculdades nas organizações em perceber
como todos estes objectos se comportam.
Mesmo nos dias correntes, em que a importância das Architecturas Empresariais cresce e estas são recon-
hecidas como artefactos valiosos para gerir a mudança, o seu benefício não está completamente explorado.
Deste modo, nós combinámos o desejo de ter avaliação de sistemas em real-time com os benefícios da
representação arquitectural para produzir um método, suportado ontologicamente, de modelar, avaliar e
capturar comportamento sistémico. Produzimos uma framework conceptual para executar esta tarefa, evi-
tando as denições impresisas de qualidade e atributos de qualidade. De acordo com a nossa abstracção,
transferir tal responsabilidade para o utilizador nal é essencial uma vez que tais aspectos são subjec-
tivos e dependem do contexto humano onde estes existem. Esta conceptualização foi materializada numa
framework de um ciclo de modelação-avaliação-visualização que foi implementado usando prácticas e fer-
ramentas de Model Driven Software Development.
Finalmente, testámos o protótipo num cenário real da PT-Comunicações, onde pudémos observar a nossa
solução conceptual em práctica.
Modelos, Fenomenologia
1 Introduction 1
1.1 The world we live in and the need for Organizational Self-Awareness . . . . . . . . . . . . 1
1.1.1 Architectural Representation of Organisations . . . . . . . . . . . . . . . . . . . . . 1
1.2 Monitoring Information Technology behaviour: Next steps . . . . . . . . . . . . . . . . . . 1
1.3 Main contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.4 Following chapters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2 Related Work 5
2.1 Enterprise Architectures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.1.1 Unied Enterprise Architecture Modelling Language and LEAN . . . . . . . . . . 5
2.1.2 CEO Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.2 Evaluating Information Technology: metrics and measuring qualities . . . . . . . . . . . . 11
2.2.1 Quality Meta-Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.2.3 Key Performance Indicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.2.4 Software Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.3.2 Eclipse Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.5 On providing quality views over Information System Architectures . . . . . . . . . . . . . 20
vii
3.1.2 Connecting Phenomenology with modelling and evaluating . . . . . . . . . . . . . 23
3.1.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.3 Revisiting the Information System Architecture building pipeline . . . . . . . . . . . . . . 27
3.4 Dening a Model Processing Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.4.1 Model Editor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.4.2 Model Converter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.4.3 Model Analyser . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3.4.4 Results Converter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.5.1 Pre Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.5.3 Instrumenting the MPF cycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
3.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3.6.1 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
4.1 Pulso Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
4.1.2 Measuring IT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
4.1.3 Storing observations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
4.2 Call Center . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
4.2.1 Process View . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
4.2.4 Object diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
4.2.5 Adding Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
4.2.6 In Execution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
Bibliography 51
6.1.1 oAW Conguration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
viii
ix
2.2 Graphical representation of LEAN nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.3 CEO Framework Meta-Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.4 Archimate Meta-Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.6 Quality Meta-Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.7 UML Layers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.8 openArchitectureWare structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.9 The Business Process Management cycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.10 Positioning of BAM systems in relation to the stakeholder and delivery latency . . . . . . 18
3.1 Steps in constructing and applying a High-Level Language . . . . . . . . . . . . . . . . . . 25
3.2 LEAN node pairings with changes to model evaluation
(1) The produces pairing exists only for the Agent → Resource relationship . . . . . . . . . 26
3.3 Evaluation ontology using LEAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.4 a) Standard ISA building approach b) approach proposed in [Vas07] . . . . . . . . . . . . 28
3.5 Proposal for pipeline with Information Systems evaluation . . . . . . . . . . . . . . . . . . 28
3.6 Model Processing Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.7 Model Converter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.8 Model Analyser . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3.10 Results Converter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.12 UML Prole denition in openArchitectureWare . . . . . . . . . . . . . . . . . . . . . . . 36
4.1 Relationship between Client, Operator and IT Infrastructure . . . . . . . . . . . . . . . . 41
4.2 ISA for the Call Center . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
4.3 ISA for the Call Center (on oAW) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
4.4 UML Object diagram for the deployed ISA . . . . . . . . . . . . . . . . . . . . . . . . . . 44
4.5 UML Class diagram for the ISA with added Monitoring Points and a Metric . . . . . . . . 46
4.6 Results after a run of the MPF cycle. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
xi
xiii
Acronyms
KPI Key Performance Indicators
MPF Model Processing Framework
PT-C PT - Comunicações
IS Information Systems
IT Information Technology
xv
1 Introduction
My conception of an information system, which reects the classic work of many MIS
scholars, is that it consists of not just the technology (hardware, software, data, networks) or
the social setting (people, business processes, politics, economics, psychology, culture, orga-
nization, and management), but also the rich phenomena that emerge from the interactions
between the two.
Allen S. Lee [Lee99]
1.1 The world we live in and the need for Organizational Self-Awareness
Laudon describes in [LL06] the importance of knowledge and knowledge management in the today's enter-
prises. Even though humans have the innate trait of self-awareness through consciousness, organizations
becoming more and more living complex social organisms fail to automatically develop similar
capabilities of sensing and learning. This task then falls upon the individuals, who must work together,
interact and share knowledge and, therefore, engage the task of sense-making as to improve the collective
self-awareness.
Nowadays, most companies suer from IT Blindness [Luc04]: the inability to understand which events
(or patterns of events) originated by IT are relevant from a business perspective. This failure has already
been proven disastrous on certain occasions [Luc04].
1.1.1 Architectural Representation of Organisations
To transfer knowledge, communication must be done through known shared boundary objects, that are
described using a set of concepts and with a semantics which cannot be ambiguous.
The need for Organizational Knowledge is already a widely accepted fact, and in order to improve the
holistic view of the Organizational Structure, Enterprise Architectures have been proposed as valuable
artifacts due to their communicative nature [MST08].
With mechanisms to represent organization blueprints, easy-to-use high level notations to describe
high-level requirements and real-time processing, rms will become increasingly aware of themselves and
their ecosystem, and will be able to make quicker, smarter and better informed decisions.
1.2 Monitoring Information Technology behaviour: Next steps
To increase the collective organizational knowledge, information has to be applied, which in turn can only
be produced by processing relevant data in the rst place [Ack89]. The ability to collect more data from
more organizational sources is there unavoidably valuable. We will focus on the use of data produced by
operation systems, middleware, applications and other technological elements to better understand how
the technology and information layers of a deployed enterprise architecture behave in real-time.
Regarding architectural representations as boundary objects, we intend to take advantage of their
creation and maintenance to encourage stakeholders in an enterprise to discuss, dene and perform
evaluation through the denition of quality attributes.
1
This thesis therefore addresses today's inability to evaluate systems in real-time through the eyes
of Enterprise Architectures. To accomplish this, we established four hypothesis that were subject to
validation through the development of our solution:
1. It is possible to extend an existing modelling Information System Architecture framework to express
what can be observed in deployed Information Systems, how to measure and which measurements
inuence (which) qualities;
2. The tasks of modelling an architecture and its metrics and analysing a deployed model's behaviour
can be combined into a framework;
3. Each of these framework's steps can be connected to form an automated circular ow that, once
congured, requires no human interaction;
4. Organizations can have temporal views of their deployed Information System Architecture regarding
previously established metrics.
1.3 Main contributions
This thesis focuses on the problem of providing an unied, architecture-centric evaluation frame-
work of the real-time behaviour of information systems. In order to accomplish this objective,
several tasks were executed, from which the following contributions emerged:
1. An analogy between the sociological and philosophical domain that studies phenomena, or Phe-
nomenology, and enterprise manifestations and evaluations;
2. A conceptual Model Processing Framework (MPF) that comprises a cycle of Modelling, Instantia-
tion, Data Collection and Evaluation;
3. An implementation of the MPF using openArchitectureWare (oAW) in which:
(a) We extended the CEO Framework to support our ontology;
(b) The primitives of the framework were used to populate an oAW's prole diagram;
(c) We created a template project on oAW that implements each step of the MPF, along with an
automation workow that executes the full cycle of the MPF.
1.4 Following chapters
In chapter 2, we cover the state of the art regarding Information System research, Enterprise Architecture
modelling, denition of qualities and metrics in several domains and nally existing instrumentation for
dealing with modelling and business monitoring.
Next, we present the solution in chapter 3, which is divided into three main sub-chapters.
We begin by providing a conceptual setting for the more practical, engineering work. We focus on
the social aspect of our objective, and attempt to gather an abstract, unied view on the dynamics of
the enterprise architecture. After that, we design a conceptual framework that will empower modellers
and analysts with a graphical tool for creating Enterprise Architectures and dene its manifestations of
realtime behaviour,
2
In chapter 4 we apply the implementation of our solution to an existing scenario within the context
of PT-Comunicações (PT-C). By succesfully modelling that evaluation scenario, we were able to partialy
validate the implementation and provide an example so that other can understand how to continue this
work, or simply attempt to replicate the scenario.
We nish by summing up the results of our work, presenting our limitations and intentions for future
work in chapter 5
2 Related Work
After having established the motivation, we skim the academic research areas that construe the domain
of this thesis by presenting key concepts, describing common problems and solutions (and the recurring
tools and frameworks used in them).
This chapter begins with an overview of the concepts revolving around Enterprise Architectures and
provides a description for the Center for Organisational Engineering Framework (the academic setting of
this thesis), Archimate (for its rapid increased use1) and nally the Lightweight Enterprise Architecture
Notation (or LEAN) for its dierent approach on constructing a meta-model. We then investigate, in
section 2.2, the existing means to identify qualities in Software Engineering, Computer Networking and
Enterprise Engineering, as well as the metrics used for measuring said qualities. Since our work is not
exclusively conceptual and we also planned and developped an implementation, section 2.3 gives an
overview of instrumentation trends and tools used to support the theorical works presented throughout
this chapter, specially applications that provide graphic modelling environments. The last section provides
some insight on the driving force behind projects such as the one this thesis leans on. We describe the
evolution of tendencies in the Workow Management Systems, through Business Process Management
and Business Activity Monitoring.
2.1 Enterprise Architectures
As stated in chapter 1, architectural representation is already as a benet to enterprises. These enterprise
architecture meta models are usually constructed in layers (g. 2.1) that separate, for example, hardware
and software from human actors and even information.
A thorough overview of Modelling Frameworks - either for Enterprise, Information System or even
Software Architectures - is documented in [Vas07]. Next, we present three Enterprise Architecture Models:
LEAN [Kho07], CEOF [Vas07] and Archimate [Lan05].
2.1.1 Unied Enterprise Architecture Modelling Language and LEAN
Gerald Khoury produced, in his doctoral thesis [Kho07], the Lightweight Enterprise Architecture Notation
(LEAN) as a unied, human-centered language. By having these unied and human-centered qualities,
the LEAN is able to both represent any Enterprise Architecture (unied) and be easily used in real-world
situations by avoiding too much complexity.
Khoury's ndings on the use of metaphors have shown that EA languages and frameworks possess
underlying metaphors. Problems arise when an author of a framework uses a metaphor unconsciously and
therefore fail to adopt a proper one. To address this problem, Khoury studied metaphor type hierarchies,
and was able to identify that a carefully chosen metaphor augments an Enteprise Architecture' descriptive
power to encompass dierent granularities and support a greater heterogeneity of realities.
1www.opengroup.org/archimate
5
2.1.1.1 The metaphor
The identication of a metaphor has been shown to be an advantageous approach to produce a valid
ontology for a given domain [Kho07]. So too has the link between such metaphors and models not only
been shown to exist but also has its strength been demonstrated by philosophers and language experts.
As already mentioned, Khoury set out to nd a proper metaphor for Enterprise Architectures.
Metaphors are constructed by linking a certain concept (the source) to another (the target) so that
some of source's essential characteristics can be identied when describing the target. To exemplify (using
the example in [Kho07]), the desktop metaphor was applied to desktop pc by dening the source as the
desktop area of an oce and the personal computer that appeared a few decades ago as the target.
The author analysed model hierarchies and extracted the criteria that the source of the metaphor
should be of a higher level of abstraction than the source, in such a way that the metaphor would
not fail to portray every possible manifestation of its target.
His next step was, then, to identify an unifying metaphor that would encase all possible granularities
of an enterprise, including its systems, its actors and its processes. From the several metaphors widely
used in the present day - such as machines, learning organisations and nervous systems - to describe
organisations, society was adopted as the best suited metaphor source since, among other reasons [Kho07],
it complied to the previous described criteria of source-target abstraction level.
6
2.1.1.2 LEAN
Armed with a metaphor, the author produced an ontology that, once formalised, was encoded into a
language and as a result, LEAN was created.
Khoury extracted a set of four high-level concepts from Giddens' Theory of Structuration: Agents,
Resources, Actions and Rules (g. 2.2) and dened their relations, both homogenous and heterogenous
ones. For example, if one wishes to express a specialization, the relationship is a type of should be used
(an example of an exclusively homogenous pairing of LEAN nodes).
Figure 2.2: Graphical representation of LEAN nodes
Using these nodes and pairing relationships, the author successfuly dened the Unied Enterprise
Architecture Modelling Language with positive feedback from case study users.
2.1.1.3 Shortcomings
Having been developed from scratch and being positioned at a higher level of abstraction, the LEAN
does not possess a strong tool support (apart from customized Visio stencils), does not integrate with
other frameworks and warrants renement of the LEAN set and relationships (all problems are stated
in [Kho07]).
2.1.2 CEO Framework
The CEO framework was developed within the Center for Organisational Design and Engineering (CODE)
group at INESC-ID with the objective of dening the set of concepts needed to represent an Information
System Architecture. This framework goes further and provides the means to represent a coherent and
compreensible picture of the enterprise by supplying the modeler with a high level package of Enter-
prise Architecture that includes three sub packages: Business, Organisational and Information System
architectures (g. 2.3).
The Business Architecture represents strategic points of view and business processes. The Organ-
isational Architecture provides the concept of resource as its central element. Resources are essencial
to organisational function and can be materialised in internal policies, roles, competencies and human
resources.
André Vasconcelos [Vas07] greatly improved the 2001 version of this framework by detailing the
Information System Architecture, formalising the primitives in UML and OCL validation expressions.
The ISA's purpose is to represent the structure of information system components that support business
goals and is further subdivided into Application Architecture, Information Architecture and Technology
Architecture:
7
• Application Architecture denes the essencial applications necessary to manage data and
support the business layer;
• Information Architecture identies the critical information needed by the business layer and
provides the data structures (independent of implementation);
• Techonology Architecture support for the application layer is described in this architecture,
focusing on the technology used such as servers, networking, communication infrastructures.
2.1.2.1 Architecture Comparison
Another of Vasconcelos' contributions was the addition of a series of measurements for architectural
qualities. One of the benets of constructing TO-BE Information System Architectures would be the
ability to clearly dene if the new design is better than the old, and quantify the gain. Therefore, to
compare and evaluate structural qualities between the old AS-IS and TO-BE Enterprise Architectures
(or even dierent TO-BE ones), the author created a set of metrics at the meta level, so that they could
be applied to all models.
2.1.2.2 Shortcomings
The CEO framework possesses an implicit problem by creating a rigid structure of concepts, and that is
the inability to guarantee that its 37 primitives are sucient to decribe the multitude of existing sytems.
Concepts such as clusters, network links, and even TCP/IP and UDP ports cannot be expressed without
extending the CEO framework which in turn can change the accuracy of metric results.
8
2.1.3 ArchiMate
The ArchiMate enteprise modelling language was developed by a group of public and private dutch
entities with the common desire to improve communication of enterprise architectures, avoiding informal
diagrams and ambiguous vocabulary. The language describes a taxonomy for mapping architectural
components, along with dierent viewpoints aimed at dierent stakeholders.
Figure 2.4: Archimate Meta-Model
One of Archimate's goals is to ease the integration between the business, application and technology
layers (g. 2.4):
• Business layer relates to the products and services the organisation provides to its (outside)
environment. They are supported by business processes that are executed by actors that play
certain roles;
that are, themselves, realized by the technology layer;
• Technology layer software, hardware infrastructure and networking services and other infras-
tructural services support the realization of the upper layers.
9
2.1.3.1 Evaluation
Another of ArchiMate's goals is to aid Change management. Change has been a concern of the devel-
oppers of Archimate and so Lankhorst's Enterprise Architecture at Work [Lan05] presents a number
of techniques that help architects and stakeholders to compare alternative designs [...] and to be able to
study the impact of a change to the design. The evaluation's purpose is then to study and compare the
performance, quality and cost of architectures prior to their implementation.
For performance measurements, ve views (described in g. 2.5) have been identied:
Figure 2.5: Archimate Performance Views
2.1.4 Summary of Enterprise Architectures
We have described three architectural frameworks with dierent backgrounds.
The Unied Enterprise Architecture Modelling Language(and LEAN), is a highly conceptual meta-
model for enterprise modelling based on a societal metaphor. It can be understood as meta-metamodel for
Enterprise description, from where concepts of CEOF and ArchiMate (for example) can be especialized.
The Organisation Engineering Center (CEO) Framework stems from the academic domain of Organ-
isational Engineering and denes a clear hierarchy of concepts for Enterprise Architectures, views for
dierent desires, and provides structural metrics for TO-BE architecture comparison.
The ArchiMate language stems from a mix of academic and organisational environments, and denes
a taxonomy, along with views that target dierent stakeholders. Regarding evaluation, ArchiMate also
focuses on evaluation of TO-BE architectures, but instead provides analysis of their dynamic behaviour.
What all have yet to provide is the ability to, once the architecture is deployed, perform evaluation
of behaviour of this architecture, using real-time data and events. To be able to execute this action we
can provide benets to the architecture life cycle:
• By executing real-time evaluation, we assert the deviation from the before(model) and the imple-
10
mentation, for events specied using model elements and so we perform a reality-check;
• Perform observations on how the architecture actually behaves after implementation, so that the
evaluations such as ArchiMate's can be corroborated (along the temporal axis).
• Understand changes in behaviour, either from infrastructure degradation or unknown change in
practices, habits or processes.
2.2 Evaluating Information Technology: metrics and measuring qualities
In [Fil99], Robert E. Filman describes the steps towards achieving ilities - typical attributes of systems
such as reliability, performance, availability and maintainability - in compositional architectures. These
concepts exist in a myriad of domains ranging from Networking to Software Development - and recently,
Enterprise Architectures. Even within a given domain, these denitions are not clear. For example, in
multimedia domains, the notion of quality of service is subjective since there isn't a widely accepted QoS
Framework [ACH96] and so QoS must be always understood in the context of the domain or framework
used to measure it.
Filman then states that there is more than one denition for concepts such as reliability; to dene
reliability is to specify the requirements established in a certain environment and by certain people
to attain reliability. The consequence of this looseness is therefore an impossibility to uniquely and
globally dene ilities. Nevertheless, for one to be able to measure the behaviour of an information
system according to a set of qualities, the denition of the set of functional, aesthetic, systematic and
combinatoric requirements must be accomplished by stakeholders.
2.2.1 Quality Meta-Model
A quality meta-model (g. 2.6) is described in [Alb03] to help identify common characteristics in quality
models. This meta-model identies quality attribute as something that is measured through metrics
and, as a source for variables are consumed. These variables relate to external characteristics of the target
entity and can be either observable during execution or not. The latter refers to variables that belong to
static aspects of the monitored entity.
2.2.2 Quality of Service and Quality of Protection
In the domain of networking, intra and internet routing, the notion of Quality of Service represents an end-
to-end property described as a set of service requirements to be met by the network while transporting a
ow [CNRS98]. The multimedia systems domain gave birth to several architectures for assessing Quality
of Service (QoS) through low-level mechanisms such as ow shaping and static policing of hardware and
operating system resources. The constraints, expectancies and tolerances regarding this set are usually
described in Service Level Agreement (SLA).
The QoS/SLA terminology has been transported into other domains (e.g. at the Web Service level
for supporting e-Business transactions in Service Oriented Architectures [DLP03]). A survey on QoS
Architectures [ACH96] presents a series of such frameworks that follow the QoS concepts introduced by
11
Andrew Campbell [CCH94].
Quality of Protection (QoP) surfaced as QoS counterpart regarding security and risk management.
Its measurements and metrics are discussed in the annual QoP Workshop. The emergence of quickly
developed interconnected systems using unstable new technology makes risk [Hol04] assessment a critical
asset.
2.2.3 Key Performance Indicators
Key Performance Indicators represent a set of measures focusing on those aspects of organisational
performance that are most critical for the current and future success of the organisation [Par07] and
therefore measure the critical success factors of the organization [Reh]. KPI are often tuned to each
organisation and reect tacit knowledge and informal metrics for dicult to measure qualities. The KPI
Library2 contains a list of indicators for many domains, from Business to Vertical Industries.
2.2.4 Software Metrics
Software Engineering scholars have always been concerned with the quality of software and so, in order
to measure code quality, several metrics were created and are now widely used:
• Lines of Code and Cyclomatic Complexity measurements were created to access software complexity
and maintainability;
• Weighted Methods per Class, Number of Children, Coupling Between Object Classes (and others)
were found to be useful when measuring Object Oriented code quality;
2KPI Library - http://kpilibrary.com
12
ISO 15408 and IEEE 1061 dene vocabulary, criteria and frameworks for evaluating quality and security
in software engineering and IT.
2.2.5 Conclusion
We were able to observe the multiplicity of meanings attributed to qualities and measurements. Regarding
Enterprise Architectures, as we presented in the previous section, CEOF provides structural metrics, while
ArchiMate focuses on predicting behaviour. Archimate's evaluation, however, isn't built on a well dened
structured of metrics, which is a problem that we will discuss in the next chapter.
2.3 Instrumentation
Throughout the years, assemblers, compilers, linkers and interpreters were created to help developers
handle higher levels of abstration and, therefore, help to create more powerful programs. In the same
way, with UML's broadness and exibility, Model Driven Software Development (MDSD) applications
emerged, ranging Open-Source projects, to low-budget solutions, to full-edged Enterprise suites and even
a development platform who created its own metamodel to escape UML's aws (The Eclipse Modeling
Project).
2.3.1 Unied Modelling Language
The Unied Modelling Language is a general-purpose, layered (g. 2.7) structure of concepts that facili-
tate activity of analysis, design and implementation of systems that are, for example, software-based or
business-based [Gro07]. UML is dened and maintained by the Object Management Group3.
Starting from the rst major revision of UML, extension mechanisms have been improved by means
of using proles and stereotypes. The OMG hosts several UML prole specications that extend UML
in order to model concepts such as Testing, Systems Engineering (SysML), CORBA and CORBA CCM4
and Enterprise Application Integration.
2.3.1.1 Time Specication
The UML Prole for Schedulability, Performance, and Time Specication provides UML the notation
needed to model concepts used in quantitative analysis of software [Fom02]. The Model Processing
Framework (MPF), described in [Fom02], contemplates ve interconnected processing blocks between
whom ow models, conguration data and results of analysis. The blocks, named Model Editor, Model
Congurer, Model Converter, Model Analyzer and Results Converter, execute the model processing
tasks so that conclusions from one iteration of the processing stream can be fed into a new iteration, and
achieve better results.
13
2.3.1.2 UML usage in Enterprise Architecture frameworks
In order to reify abstract concepts, Architecture Model developers have been taking advantage of the
Unied Modelling Language for its exible, heavily scrutinized and broad-scoped structure, although
complexity and cohesion are topics of criticism [Kob04] regarding the UML structure:
• CEO Framework [SCV+06], TOGAF [Pub07] and DoDAF [Gro05] support UML notation.
• ArchiMate framework has a dierent notation, but studies have been made to estimate the porta-
bility of ArchiMate's concepts and semantics to UML [WBB+04].
• Due to their high-level abstract purpose and concepts, frameworks such as the Zachman Framework,
EUP5 and EAP6 are UML-independent.
2.3.2 Eclipse Project
The Eclipse Project evolves around a software development platform that started from a Java Integrated
Development Environment (IDE). This project includes several extensions such as Java Development
Tools7 (the original codebase), C/C++ Development Tools8, Graphical Editing Framework9 and the
Eclipse Modeling Framework10.
10http://www.eclipse.org/emf/
14
The latter, EMF, started as a MOF implementation of a meta-modelling structure [BSM+03], now
called EMF Core or Ecore, but later became a full modeling framework and code generating tool that
relies heavily on OMG's XMI specication for model and metamodel serialization. Taking advantage
of EMF, the Modeling Development Tools project11 was then created to provide several metamodel
implementations for UML2, OCL, BPMN and XSD.
2.3.3 openArchitectureWare
openArchitectureWare is a model driven software development (MDSD) tool that draws from several
components of the Eclipse Project and implements a workow engine at its core (g. 2.8). The workow
components (oAW has several built-in) can read, instanciate and transform models, perform code gen-
eration and execute validation checks (using OCL). oAW relies on the EMF core meta-model but also
supports UML2, XML and Javabeans based models.
Figure 2.8: openArchitectureWare structure
A workow is created as a linearly invoked set of components that perform some kind of activity, and
use slots (that can be named and assigned to), so that the results of one component can be transfered to
another. Some typical components are the Reader and Writer, Xpand, Xtend and Xtext.
The Reader components serve as entry points for the workow: they receive a UML, Ecore or other
supported model and create a named slot that will be available on the global scope of the workow, while
writers have the function of writing a model slot back into a le.
The Xpand component is responsible for code generation in the Eclipse M2T (model-to-text) project
and works a template engine. The engine accepts macro denitions and the places (model elements)
where these macros should be expanded. These macros can have simple control ow structures such as
loop and if statements. Once the engine is parametrized with a metamodel and loaded with a model, it
will navigate the model and apply the macro expansions accordingly, generating code. On a nal note,
11http://www.eclipse.org/mdt/
15
oAW also provides a beautier for some languages so that the code is properly indented after generation.
The Xtend component's purpose is to execute queries on a model and invoke statically-typed func-
tions to perform Model-to-Model (M2M) transformations. Aside from the language provided by oAW,
the Xtend component can be further extended with user-dened Java functions, giving it aditional ex-
pressiveness.
2.3.4 Enterprise Architect
Enterprise Architect is a commercial CASE12 tool developed by Sparx Systems13 that aims to be a
complete tool for Enterprise Architecture design. As of version 7.5 Professional (available to students at
Instituto Superior Técnico), Enterprise Architect implements:
• UML2.1 (along with all the 13 diagrams and the ability to dene custom ones)
• UML proles
• XMI2.1 for importing and exporting models
• a model validation mechanism that can be invoked to ensure conformity to the meta model standard;
Being a CASE tool, EA supports Model Driven Development [MM03] practices of constructing high-
level models, performing transformations for specic platforms, and nally generating artifacts such as
code. Both reverse-engineering (creating models from code) and code generation are available for many
programming languages, such as Java, C, C++, C# and Python.
2.3.5 Comparison and Conclusion
Considering that our solution manipulates models and we wish to provide users with an executable
representation of the enterprise architecture, we identied several necessities for these tools. Support
for metamodels and metamodel extensions is essential since Enterprise Architecture Frameworks
are metamodel level entities. To augment portability and mobility of this solution, import and export
of models is desirable. One way of performing validations is through the verication of constrains on
the model; for UML, the OCL performs this function and therefore it is required for the tool not only
syntax validation of the constraint language but also execution. The last but equally important feature
is the ability to chain model transformation actions, since we're going to perform model edition,
validation and execution of custom tasks.
From this comparison of the two tools we presented above, we can conclude that openArchitectureWare
provides the necessary feature for task at hand. Also, by being Open Source, we have the possibility of
extending the tool to better suit our needs.
12Computer-Aided Software Engineering 13www.sparxsystems.com.au
Graphical Model Edi-
Import / Export XMI 2.1 XMI 2.1
Code Generation Xpand templates Graphical
Constraint Verication oAW's Implementation of OCL Only Syntax Validation
Operation chaining Workow Engine N/A
Table 2.1: Comparison of MDSD tools
2.4 Business Process Management
A new-found way of thinking in Organizations, that stemmed from three trends in Information Systems
(described in [HW03]), propelled the extension of classic Workow Management:
1. From programming to assembling (of Information Systems);
2. From data-oriented to process-oriented applications;
3. From design to redesign and organic growth.
These trends translate the shift from the generic, static and all-purpose solutions to custom-made
applications, capable of evolving along with the enterprise. Traditional Workow Management Systems
- even though process-aware and capable of capturing real-time data from its execution - focused on the
lower section of the BPM lifecycle (shown in g. 2.9).
Figure 2.9: The Business Process Management cycle
In the classic view of Workow Management, processes were (re)designed, followed by the implemen-
tation (usually done by conguring a generic WFMS) followed by the actual IT-supported execution.
Once organizational change occurred, it was back to the drawing board, leaving no room for diagnosis
and adaptation. Business Process Management brought a bigger emphasis on monitoring making use of
collected data from application logs and database activity.
As enterprises grow aware of their processes, their (re)design and diagnosis have become the main
concerns of the BPM life-cycle. Hence, BPA14 (one key aspect of BPM) has evolved to encompass
activities such as simulation, verication and validation, giving birth to Business Activity Monitoring
and BAM tools.
14Business Process Analysis
2.4.1 Business Activity Monitoring
Business Activity Monitoring was (rst) introduced in 2002 by Gartner Inc. [McC02] and, stimulated
by the BPM growth, made several organizations adopt or design solutions to address IT blindness. The
cradle of such tools was the enterprise world (outside academia) and, sprouting from common sense
[Ada02], aimed at providing real-time access to critical business performance indicators that improve the
speed and eectiveness of business operations [McC02].
Figure 2.10: Positioning of BAM systems in relation to the stakeholder and delivery latency
Modern enterprise systems (g. 2.10) are capable of generating large volumes of logs and sources
of measurement, ranging from CPU usage to application transactions. To collect all this data, a BAM
system must monitor orthogonally all the enteprise, and work on more than one time frame (integration
and multiple timescales principles [CCH94]). Regarding the construction of a Business Activity Monitor
system, Joseph DeFee and Paul Harmon provide an overview on the tasks it must accomplish [DH04]:
1. convert data about actual events into digital information;
2. provide context for the digital data being accumulated;
3. apply logic to data to identify problems, diagnose them, and to recommend managerial actions.
A 4th additional step is displaying the near-real-time information produced in the previous step on an
easy to consult location like an online dashboard.
There are several approaches to step 3 (seen in the table below), being the most straight-forward the
design of a rules-based system: data is matched against a set of rules that, when triggered, executes the
appropriate actions.
18
From the domain of Business Intelligence, reliance on Data Warehousing and applying a collection
of powerful tools such as On Line Analytical Processing (OLAP) gives the ability to extract meaningful
patterns from historical data. Knowledge from the AI domain and Rule-based techniques are also used
to Business Intelligence's benet. Simulation-Based Systems use patterns extracted from historical ows
(and sometimes from current data) to perform trend analysis. The results can be used to both eliminate
potential threats and simulate formulated hypothesis. Finally, all these approaches to Decision Making
can be combined into a mixed system, adapting to the needs of the organization.
Advantages Disadvantages
and can operate independent of other
systems
• Is a well understood approach
• Can become complex if the range of the
variables are extensive
• Dicult to graphically validate rela-
tionships to process
trends and patterns
breed15 applications
with large amounts of data
• Must create Data Warehouse as precon-
dition to using BAM capabilities
• Aren't designed to use a process context
for reporting
real-time.
plex and dynamic processes
predicted state if the business, based on
the validated process ows
identied by other techniques
• Can take advantage if both BI and
Rule-Based approaches to provide even
better simulation results at run-time.
• Simulations technology is not so well
understood at the user level
• Development of complex simulation re-
quires specialized knowledge
19
During this chapter we provided an up-to-date overview on Enterprise Architectures, Instrumentation
and also the driving forces that created the need for this thesis.
We were able to establish that, of the Enterprise Architecture Metamodels, none performs real-time
evaluation of deployed architectures. As said before, this feature has potential benets such as helping to
reduce the gap between reality and the modelled architecture, provide constant feedback on its behaviour
and nally reduce the time to assert the necessity of change.
So too exists a lack of proper instrumentation for performing real-time evaluation but there are,
however, tools that can (and will) be used to accomplish this task.
In the next chapter, we will take Gerald Khoury's work on the Lightweight Enterprise Architecture
Notation and André Vasconcelos' improved CEOF Architecture Metamodel and construct a conceptual
framework for processing models to perform real-time evaluation. Next, we will take openArchitecture-
Ware and provide an implementation of this framework so that we can atest its feasibility.
20
3 Solution
In the previous chapter we concluded that evaluation of information systems and information technology
has several problems such as the lack of proper ontologic support, the inability to dene what can be
observed in real-time and nally the inexistance of a generic, unied, meta-model independent framework
of evaluating information system behaviour.
The solution we developed aims to tackle these problems and therefore provide means to describe
architecture run-time manifestations and to perform evaluation modelling based on them.
The rst problem we had to address was how to nd the necessary and sucient abstraction for this
activity so that we would be able to contemplate every possible real-world scenario. For this, we turned
back into Khoury's societal metaphor [Kho07] with the intent of constructing (or extending) an unied
ontology. As we describe next in section 3.1, we approached sociological domains that study experience,
knowledge and phenomena to help us capture essential aspects of enterprise behaviour. By understanding
action and perception through philosophy and (particularly) phenomenology, we sought ways to better
understand what it means to evaluate information systems. The purpose of this thesis was not, however,
to delve deeply into dicult and complex philosophies and so we limited ourselves to the very basic
concepts and understandings that we deemed necessary and sucient to accomplish our task.
An unied description of evaluation from an abstract point of view has well-dened purposes. For one,
it enables the solution designer to translate his ideas into a tangible visual representation. Second, and
most important, it creates a level of indirection between these ideas and their use on an existing Domain
Specic Language or Enterprise Architecture metamodel. Section 3.2 describes how this conceptualization
can be used to drive the design of the evaluation aspect into architectural metamodels, while lessening
the constraints needed to apply our objective to other environments, aside from the one we describe in
section 3.5.
In section 3.3 we modiy the ISA building pipeline described in [Vas07] to encompass the evaluation
activity. To accomplish this, we added an evaluation cycle at its end that takes as input the `AS-IS' ISA
and the Information Systems (of the existing pipeline), add observable variables and measurements and,
through a processing block, determine behaviour issues.
To model the necessary steps that will be ocurring during evaluation, we took advantage of the work
done on the UML Prole of Time Specication [Fom02] concerning the Model Processing Framework,
which we adapted for our purpose. This is described in section 3.4, along with a proof-of-concept imple-
mentation using a UML tool.
3.1 Locating a Proper Metaphor
[...] a good part of the answer to the question `why philosophy?' is that the alternative
to philosophy is not no philosophy but bad philosophy. The `unphilosophical' person has an
unconscious philosophy, which they apply in their practice - whether of science or politics or
daily life.
3.1.1 Establishing the Phenomenological setting
In [RN08], Recker and Niehaves explained the relevance of philosophy in Information System research by
linking several philosophical disciplines such as Ontology, Methodology and Epistemology to IS research
paradigms (e.g. positivism, interpretivism). By understanding how we perceive others, what we observe
and how we judge, we were able to come up with unied, global, overview of the concepts that are involved
when performing evaluation of Information Systems and, therefore, be able to model this domain.
Phenomenology is the study of structures of consciousness as experienced from the rst-person point
of view. [Uni08]. Its key concepts are: intentionality (experience always is directed at something or
is about something), qualia (sensory data), bracketing (putting aside the question of the existence of a
real world) and consciousness itself. We found this philosophical discipline and these concepts to have a
profound link to the Information Systems and Information System research area [RN08]. In the following
subsections of this introduction we describe the mentioned concepts, and then move on to determine in
what way the phenomenological dialectic ts the problem at hand.
3.1.1.1 Phenomenon, Noumenon, Thing-in-itself
Regarding Phenomenology as the study of phenomena as it appears to us in consciousness, it is important
to distinguish the concepts of phenomenon and what Immanuel Kant called noumenon or thing in-itself
[KM34]. Contrary to phenomena, noumena are objects that exist independent of the senses, while the
former refers to appearances and objects produced by senses. This accentuates the subjective aspect
of this philosophy, as to say that a thing of the world can be experienced in dierent ways, from the
phenomenological point of view. As we describe next, the bracketing phenomenon is related to this
duality. As stated before, Information System research paradigms are tightly linked to these philosophical
disciplines. For example, the positivist point of view establishes that the only source of knowledge is that
which stems from phenomena, rejecting noumena altogether.
3.1.1.2 Facticity
Facticity is a concept that grew in signicance during the 20th century within the phenomenology domain,
through philosophers such as Edmund Husserl, Martin Heidegger [Hei96] and Jean-Paul Sartre [Sar55].
Its meaning suered variations throughout history, but as said in the beginning, we did not concern
ourselves with such details and simply focused on how a concept like facticity helps our metaphor, and
will therefore use the Sartrian notion of facticity. As intended by Sartre, facticity includes all those
properties that third-person investigation can establish about me [Uni08]. Such properties can be of
historical and psychological nature, include physical traits and even things known to be true in the future
(e.g. the inevitability of death).
3.1.1.3 The bracketing, noema and qualia
Edmund Husserl strongly advocated that the objects of consciousness do not exist only inside it, but
transcend it, therefore being able to exist independently of the mental activities executed upon them,
22
such as thinking and judgment [Sol00]. This decoupling does not imply that perceptual experiences are
then void of content, but instead generate what he called noema, the perceptual content.
Further still, Husserl established the necessity of describing an object from the rst person standpoint,
in such a way that the nature (e.g. reality, dreams, hallucinations) of the object itself is irrelevant, leaving
only the item as it was experienced by the subject. To the act of suspending judgment of the world,
Husserl called bracketing [Sol00].
Both concepts of noema and bracketing come together in phenomenology since former is that which
results from latter, when experiencing an object while putting aside its nature (e.g. an awaken experience,
a dream, an hallucination).
Qualia stands for the sensory data captured by an entity.
3.1.1.4 Intentionality
Intentionality (not to be confused with intensionality) refers to the directedness of consciousness or mental
states [Uni08]. In other words, it reects the notion of phenomena always being about objects, that exist
within consciousness (once again, independent of its nature, through bracketing).
This concept was used in many schools of thought such as Existentialism, where both Martin Heidegger
and Jean-Paul Sartre used intentionality to proclaim that things such as moods were always intentional
(about something) and, therefore, rejected the notion of being sad, happy, angry without a motive. This
was also used by Sartre to his central theme of responsibility, freedom and bad faith [Sar57].
3.1.2 Connecting Phenomenology with modelling and evaluating
As said in the beginning of this chapter, society has been established as a proper metaphor for enter-
prises. In order to accomplish our goal, we set out to nd a parallel between experiencing, judging,
perceiving-capable agents since, inherently, these activities occur inside society and consequently within
the metaphor.
Since Phenomenology imposes subjectivity upon the world, we can place its rationale within the
interpretivist trend [LG03, Min01]. Considering that the phenomenologic concepts and their relations
apply to society, we then propose that they can also be mapped into the society metaphor and therefore
its target (enterprise systems).
The act of perceiving (noesis) the enterprise structure allows the detection of its meaning, therefore
producing the boundary object (noema) referred in the Organisational Engineering domain as Enterprise
Architecture. And by determining what manifestations are observable on this architecture, we perform
the phenomenological bracketing, for the assumption develops that objects will always be perceived as
modelled, regardless of the particularities relative to their nature.
This construction of an enterprise architecture allows the specication of enterprise elements as simply
`being there' - or noumena - while by describing their manifestations we dene how they appear before
the senses - phenomena.
Modelling evaluation as an activity that is inherently dependent on observations - qualia - we establish
a link to the intentionality aspect of experience, according to phenomenology. And, nally, by having
23
agents record manifestations as observed (from the rst person standpoint) we construct the observed
objects' facticity .
3.1.2.1 Subjectivity of Qualities
One of the problems raised in chapter 2 was the fact that dierent domains establish dierent meanings
the concept quality they wish to monitor. This phenomenological take on evaluation provides evidence
on why it hasn't been possible to pin down notions such as Performance and Availability and also makes
it dicult to determine the metrics used to measure the behavioural attributes of a given system, as
presented in [LCPH00]. The compromise needed to transfer these concepts into the meta plane has the
unavoidable eect of generalizing to a point where the concepts lose their meaning. So too, in phe-
nomenology, the dialectic does not encompass attributes of perception and experience, but exclusively
establishes the abstract concepts of the acts and objects involved (i.e. the essential structures of ex-
perience). As a consequence, we did not attempt to globally dene quality concepts, nor methods of
calculation, for evaluating behavioural characteristics of existing systems. Instead we will consider that
evaluation criteria are phenomena that emerge from consciousness, either through one's preconceptions
or transfer of knowledge from a third party.
Another observation, that we were able to extract, was the importance of accepting the inability to
evaluate something in its essence and, instead, placing our judgement upon what is observed of that thing.
This thinking should be applied to both human-machine and machine-machine levels. For a computer
application A to use another application B, the latter will display a set of manifestations observable by
the former (creating B's facticity) which then can be used by A to evaluate the B's actions.
Finally we established intentionality as a criteria for traceability: measurement methods should never
rely on implicit or tacit knowledge but instead have clear representations of the sensory data used to
calculate them.
3.1.3 Conclusion
In this section we provided insight on how the atoms of society experience themselves and others as a form
of metaphor for evaluating Information Systems. In societies, human beings are constantly perceiving,
evaluating their environment, their peers, and surrounding objects. These agents possess senses that allow
them to perceive things in their environment through action, i.e. by interacting with that environment.
By considering this phenomenological ontology in a level of abstraction greater than the target of
our desired metaphor, we were able to assert its validity and also construct a parallel between evaluation
of Information Systems and the philosophical discipline of phenomenology. Many of the central concepts
of this domain can be understood under the scope of agent interaction, should they be systems, humans,
communities and organisations. We asserted subjectivity as an essential aspect of this problem and, as a
direct consequence, qualities are subjective and should always be understood from the perspective of the
one measuring it.
3.2 Producing a High-Level Language Extension
Given an enterprise architectural model, accomplishing the task of modelling its information systems'
evaluation and execution requires the following actions [Kho07]:
Figure 3.1: Steps in constructing and applying a High-Level Language
Identifying a metaphor has been done in the previous section, where we establish the relationship
between evaluating Information Systems and society's model of behaviour and experience.
According to Khoury's works [Kho07], the sucient lexicon to model an enterprise was drawn from
Giddens' Theory of Structuration and consists of four primitive concepts: Agent, Resource, Action and
Rule, along with their homo and heterogeneous relationships. Using the Lightweight Enterprise Architec-
ture Notation, we described a high-level model for this problem. We began by performing the following
changes on the LEAN node pairings (g. 3.2):
• added an observes relationship between an Agent and a Resource;
• add the Agent → Resource and Agent → Rule to the produces pairing;
These two changes reect the necessity on bringing Agents and Resources closer. This necessity arose
from our closer inspection of the philosophical notions of experience. The causality relation between
Action and Rule is in fact important, but it is also essential to identify the Agent as the producer of
Resources, even if Actions were the means. This close binding also supports the approach of modelling
what can be observed on an entity and how to measure that entity's actions.
Having extended a high level notation to encompass the necessary relationships, we modelled evalu-
ation according to the society metaphor (g. 3.3). Once again, it is a behaviour that exists within it,
and therefore can be expressed using these nodes and pairings. We created three specialisations: Mani-
festation (from Resource), Expectancy (from Rule) and Evaluation (from Action). We used the primary
25
Figure 3.2: LEAN node pairings with changes to model evaluation
(1) The produces pairing exists only for the Agent → Resource relationship
26
Agent node since we didn't have a necessity to constrain the subtypes of Agents that t on this situation.
Manifestations represent the phenomena that are produced by Agents and are, as such, observable and
knowable. They are the essential elements that are consumed when performing evaluation. Expectancies
reect the desired expectations, based on the observed phenomena, regarding an Agent's behaviour. To
this act, we called Evaluation.
Figure 3.3: Evaluation ontology using LEAN
With a proper metaphor for the act of evaluation, we proceeded to construct our Model Processing
Framework. The next (brief) section denes the setting on which the MPF operates.
3.3 Revisiting the Information System Architecture building pipeline
In [Vas07], the traditional approach to building Information System Architectures was extended to encom-
pass iterations of redenition and readjustment of the TO-BE ISA (g. 3.4). Along with the creation of
architectural metrics, this new pipeline created the ability to evaluate Enterprise Architectures regarding
pre-established qualities as means of comparing dierent (TO-BE) ISA.
The solution we created further extends this pipeline so that systems can be monitored from the
perspective of the nal (AS-IS) ISA. The revisited pipeline (g. 3.5) includes an extra cycle located at
its end that feeds the ISA artifact, the manifestations and measurements into a processing framework.
This framework is able to evaluate the state of the deployed ISA regarding dened metrics, as well as the
ability to initiate this evaluation process at will.
In the next section we describe a framework that aims to support this extension of the pipeline.
27
Figure 3.4: a) Standard ISA building approach b) approach proposed in [Vas07]
Figure 3.5: Proposal for pipeline with Information Systems evaluation
3.4 Dening a Model Processing Framework
One of the objectives of this thesis was to create the means to automate the evaluation of Information
System and Information Technology behaviour. By automation, in this context, we mean the ability to
perform all the actions necessary and sucient to execute our extension of the building pipeline without
human intervention. That sequence of actions excludes, however, the initial bootstrapping process of
describing what is the reality, its elements, their actions and how they should be measured.
In [Fom02], the concept of Model Processing (g. 3.6) is described as a set of tasks, namely creating
a model and performing analysis techniques, that are put together in the form of a Model Processing
Framework. In the following subsections we describe each of the MPF's components we used as a base
for our solution. We explain their tasks and input/outputs and how they're connected but, in order to
28
separate the conceptualization of our framework and the tool support (described in section 3.5), we have
kept out all the implementation aspects.
Figure 3.6: Model Processing Framework
3.4.1 Model Editor
The Model Editor supports the creation and edition of models, with proper lexicon and syntactic vali-
dation (according to an Enterprise Architecture meta-model). These meta-models must encompass the
four concepts of the Lightweight Enterprise Architecture Notation: agent, action, resource and rule. The
creation and edition of models must be fully supported by the tool, as to not force the user to understand
lower-level programming concepts.
As we explained in section 3.1.2, evaluation occurs by observing agents while they're performing
actions. Therefore, object representation is necessary so that individual characteristics can be modeled.
Objects as things must be reied within the tool, so that Object diagrams or similar representation of
object-level schemas are possible.
Figure 3.7: Model Converter
The Model Converter's purpose is to act like a pre-processor, transforming the modelled architecture
into a format that can be consumed by the Model Analyser. The converter, loaded with the Enterprise
Architecture meta-model, has the capacity to navigate a model and generate the code needed to capture
the values of monitoring points dened by the modeller.
29
3.4.3 Model Analyser
The core of this MPF is the Model Analyser. It is responsible for performing the evaluation of the Infor-
mations Systems based on the enterprise architecture model (along with manifestations and measurement
specications) that were dened in the Model Editor. To execute this function, the Analyser block must
accomplish the following steps:
1. model two activities are performed in this step: validation and evaluation of the architectural
model:
• validation - the model is rst veried for inconsistencies. Semantic constraints such as cardi-
nality are veried according to predened rules of the meta-model;
• evaluation - structural qualities are calculated in this step as well. For examples of this kind
of metrics, refer to [Vas07];
2. instantiate the model after validation, and in order to accomplish object-level evaluation, the
object diagram must be transformed into an executable set of instructions, whose purpose is to
support the next step. One example of this is the generation of object instances whose types,
names and attributes map into tables in a Database;
3. populate diagrams with manifestation data ranging from CPU usage, web service invocation
timestamps, queue sizes. This step collects all manifestations from a data source and sets the
values on each object of the internal representation of the model;
4. calculate object-level metrics again, a set of constrains - that target the object level - concludes
how the current model instantiation is faring regarding the metrics dened during the model edition
phase (section 3.4.1);
5. generate warnings regarding unmatched acceptance thresholds nally, to help communicate the
results of this iteration, it is desirable to generate notications of which metrics failed to reach
desirable levels of acceptance.
To sum up, this block is responsible for the evaluation of metrics, both at the Model and the Instance
level (g. 3.9). The Model-Level Analyser acts upon metrics that are dened within the meta-model and
don't require user intervention; on the other hand, the Object Level Analyser uses continuously stored
data and user-dened metrics to judge instance behaviour.
In both situations it is desirable to create not only the metrics - the method of calculating a value
that means something for a given domain - but, instead, dene the expectancy along with metric. Since
the purpose of this evaluation is not comparing results with some other architecture in real-time, the
outcome of calculating (for example) a performance index is only useful if there are pre-assumptions on
how low or how high said index should be. Modeling the expectancy can be as simple as changing an
30
index formula into a inequality (so that acceptable/unacceptable correlates to true or false) or, in more
complex domains, include degrees of membership and other fuzzy logic theory [Uni08, Zad65].
3.4.4 Results Converter
Figure 3.10: Results Converter
The nal stage in the MPF cycle is the transportation of information produced by the Model Analyser
back into the visual representation of the model as in the Model Editor. The values (of data collected)
are displayed in each of the objects' manifestation attributes and the generated warnings by the Model
Analyser are displayed to the user.
3.5 Implementing the Model Processing Framework
An abstract framework for evaluating enterprise models was constructed and, to attest its viability, we
describe in this section a possible implementation for the MPF, using an existing architecture meta-model
and an UML tool.
To sum up our objective, the aim was to construct a model driven evaluation framework for the
real-time manifestations of deployed enterprise architectures.
As already stated, this thesis was developed in both an academic and an organisational environment.
As such, this implementation draws from Center of Organisational Engineering meta-model, and from the
31
PT-Comunicações monitoring and data collecting infrastructure. These two domains create constrains
on the abstraction we have already presented, but also helps us implement it.
To better position this implementation, we considered the ODE loop detailed in [Mat07]. According
to this loop, the MPF targets the Domain Monitoring and the Domain Analysis phases of the ODE
Meta-Process (shown in g 3.11).
Figure 3.11: ODE's Double Loop Meta-Process
In the following subsections we describe the set of assumptions on which this implementation relies,
the extension to the CEOF meta-model and the MPF implementation using openArchitectureWare.
3.5.1 Pre Assumptions
The gap between the conceptual Modelling Processing Framework and the reality on which it operates
manifests itself in a requirement we have taken for granted so far: the infrastructure (attached to the
deployed ISA) that collects the values of monitoring points along a temporal axis. This assumption is
due to the fact that such task is outside of the scope of this thesis and was already addressed issue within
the Organisational environment in which this thesis was conceived (further detail on the organisational
structure is presented on the section 4.1 of the case study).
The infrastructure itself should include a central repository and agents that capture (through its
senses) certain dynamic attributes of another agent. By agents, we mean both people (that populate a
table with what they observe) and software (that register state variables from its environment).
Creation and maintenance of this infrastructure may not always be pacic and can generate adverse
reactions simply by means of the Observer Eect alone (e.g. a monitoring application consumes the host's
resources). As such, among other things, agents (by the Heisenberg uncertainty principle) should cast
the least interference possible when performing their observations. For this we suggest other literature
and studies regarding change management [ZMT08].
3.5.2 Extending the Meta-Model
The framework chosen was the Center of Organisational Design and Engineering Framework (CEOF)
since it draws a clear line between the several layers of the Enterprise Architecture, and describes the
32
entities that belong to each.
As already presented in section 2.1, an Enterprise Architecture modelled using CEOF has:
• Business Architecture
• Organisational Architecture
Application Architecture
Information Architecture
Technological Architecture
We have extended this framework at the Information System Architecture level, through its Block
and Service elements since, in this architecture, the concept of agent as action-performer entity does not
exist.
3.5.2.1 Monitoring Point
Vasconcelos [Vas07] denes Block as the representation of a modular component of information systems
that aggregate a set of operations which describe a system or other elements. Looking at the previous
section, we've identied that entities are measurable through their manifestations. Thinking back to
our LEAN extension, Blocks represent Agents and therefore we must dene how (or through what) they
manifest themselves in reality. For this meta-model, we changed the manifestation semantics slightly
in order to better align out concepts with CEO's. The ODE loop we presented in the previous section
shows the concepts Variables and Monitoring Points as the meta-level inputs for Domain Monitoring.
To better suite ODE's syntax we used the latter concept Monitoring Points to refer to the observation
standpoint of a certain variable of an Agent's behaviour. Monitoring points are therefore the equivalent
of the Manifestations, but add a perspective semantic to it. This characteristic is useful due to the nature
of the Case Study monitoring platform, that relies on small footprint agents located near systems that
observe their manifestations.
Denition
A Monitoring Point is a dynamic property of a Block that can be observed over time.
Architectural Attributes
• Block - the Monitoring Point exist within Block.
Semantics
Specializations
a) Exist within the context of a given Block
context Monitoring_Point inv Monitoring_Point_Relations
:
s e l f . c l a s s . oclISKindOf ( Block )
Notation Monitoring Point follows the standard UML representation for At-
tributes
Representation
Options
3.5.2.2 Metric
Looking back into the ODE loop, Domain Analysis is performed adding models (structure) and heuristics
(evaluation) to the results of Domain Monitoring (observable data). To the heuristics we will call Metrics.
Essentially the Metric concept represents an expectancy that is based on collected data from Monitoring
Points.
In CEO Framework, the Service is a collection of operations made available by architectural blocks.
We therefore apply the metric concept to the Service element, so that we can evaluate actions, using
manifestations from the actor, according to LEAN and our LEAN extension.
Denition
Architectural Attributes
Semantics
Specializations
context Metric inv Metr ic_Relat ions :
s e l f . c l a s s . oclISKindOf ( Se rv i c e )
Notation Metric follows the standard UML representation for Operations
Representation
Options
3.5.3 Instrumenting the MPF cycle
As already reviewed in section 2.3, there are several applications designed for modelling information
systems, which is what occurs in this step: using a well dened meta-model to construct a model of a
given architecture. Once described in a structured form, this description can continue the ow of the
Model Processing Framework.
For the purpose of this thesis, the EMF is useful for it provides the modeller the ability to specify
what to observe and how to measure.
The tool selected was openArchitectureWare since, of all those reviewed in chapter 2, it presents the
better support for the needed characteristics: graphical interface, UML2, UML Proles, Code Generation,
XMI import and export, OCL checking and for being Open Source.
3.5.3.1 Model Editor
As referred in section 3.4.1, the model editor requires easy-to-use, UML2 class and object diagram support
as well as support for UML proles.
openArchitectureWare includes the Graphical Modelling Framework1 (from the Eclipse Modelling
Project) that provides this tool with a graphical environment for creating diagrams. Since UML2 is
supported, it is possible to create Class, Deployment, Component, Activity, Composite Structure and
other standard diagrams and therefore it is suitable for implementing this MPF.
As seen in gure 3.12, we've recreated the Information System Architecture of the CEO framework on
a UML Prole (using a Prole diagram in the graphical interface). We have also inserted our meta-model
1http://www.eclipse.org/modeling/gmf/
35
extension in this diagram, by creating two Stereotypes named Monitoring Point and Metric as described
in section 3.5.2.
Figure 3.12: UML Prole denition in openArchitectureWare
Having the CEOF prole and its extension dened, it is now possible to create an UML model (using
the Class Diagram), apply the prole, and model an architecture with the correct taxonomy.
To model object instances in oAW, the eclipse UML2 implementation supplies the InstanceSpecica-
tion element, which is a M1-level representation of the M0 object.
3.5.3.2 Model Converter
The next step in the MPF workow is to convert the UML model into a format than can be used by the
Model Analyzer. In order to manipulate the UML model as an input for other components, a XMIReader
component was used. This component receives a model, a meta-model and allocates a slot so that it can
be read by other components. Then, the Generator component was added to the workow to performe
code generation. The component receives a model (plus the corresponding meta-model) and a Xpand
le, and uses the latter to translate into a programming language each of the Meta-Model, Model and
Object level elements.
The target language chosen for the code generation was Ruby for its simplicity, meta-programming
support, time required to implement and because some of the infrastructure, described later in section
4.1, is already built upon Ruby2. The CEOF meta-model was recreated in Ruby using an Object Oriented
approach so that, when the Generator is invoked, it creates two ruby source les: one for the model, and
another for the instances, and each level is supported by the upper.
3.5.3.3 Model Analyzer
openArchitectureWare supports OCL expressions and validations, that are used to calculate Service
metrics by navigation InstanceSpecications, accessing Monitoring Point values in Blocks, performing
calculations and nally comparing the output with a threshold which warns the user.
2www.ruby-lang.org
36
By bringing objects (M0) into the model (M1) level using the uml::InstanceSpecication, oAW allows
writing metrics as constraints. The object diagram can be navigated from instances of services to the
blocks that support them by using the navigable associations created in said diagram. Basic arithmetic
operations are built-in in the language and, in the necessity of more complex ones (e.g. lters, statistic
functions), it can be extended using Java and the already existing set of libraries for such purposes.
3.5.3.4 Results Converter
Finally, the Results Converter transforms the manipulated model (now possessing values for the moni-
toring points) back into UML. This processing block also has the responsibility to guarantee that the end
of the processing cycle leaves the openArchitectureWare in a state where the process can be run again.
In order to accomplish the necessary idempotency, the only oAW project objects that are altered during
the workow cycle are run-time model slots (which disappear after execution) and the Value elements in
Slots of InstanceSpecications where the monitoring point values are stored. To be able to show the user
the values used for calculating metrics, the model UML(.uml) le is rewritten, and special care is given
so that in subsequent executions of the workow only changes these Values.
3.5.3.5 Tying it all together
The openArchitectureWare tool has, as already referred in section 2.3.3, an workow engine in its core.
This enables the creation of the MPF workow, by executing several tasks and automating the ow
between each of the MPF's steps (the code for this workow is listed in appendices 6.1):
1. Load UML model into a slot (XMIReader component)
2. Verify model and architectural qualities (Check component)
3. Generate Ruby code for model and instances (Generator component)
4. Transform the slotted model by inserting value nodes into InstanceSpecications (Xtend component)
5. Execute checks on the transformed model (Check component)
6. Overwrite the UML model with the slotted model (UML2Writer component)
3.6 Summary
As a base for our implementation, we came up with an abstract picture of how the dynamics of agent
interaction occur and how these should be modelled.
We constructed a conceptual framework for modeling manifestations and evaluations on an Informa-
tion System Architecture. The Model Processing Framework is comprised of a graphical Model Editor
to ease the modelling process, a Model Converter for serialization, a Model Analyser that performs both
model and object level evaluations and nally a Results Converter the presents results to the modeller and
We were able to create a fully automated ow between each of the MPF's steps so that, once congured,
it can be triggered periodically to refresh the current view of the dynamic status.
37
3.6.1 Limitations
The implementation has only the necessary and sucient constructions to perform very basic navigation
on the model. Nevertheless, support for more complex interactions is possible for the implementation
was built upon Eclipse's UML2 and therefore higher level functions can be constructed, to ease the user's
experience of the rule writing process.
38
4 Case Study and Validation
In this chapter we explain how the solution was applied to a real-world situation. This thesis was
developed in an enterprise context, therefore our testbed consisted of the enterprise PT-Comunicações
and its technological infrastructure.
We begin by describing the monitoring framework available to us in this enterprise environment, which
we used and attempt to improve, by using its collected data and providing a better evaluation experience.
Then, describe a proof-of-concept implementation of an existing scenario using a Call Center. Our
purpose is to demonstrate the steps needed to construct a cycle of the MPF using the implementation we
proposed, using the CEOF metamodel and openArchitectureWare. For this, in section 4.2, we describe
a fraction the PT's Call Center system (sucient for an example) and the creation of an evaluation
scenario.
4.1 Pulso Architecture
PT-C has been concerned over the years with measuring IT performance, availability and errors. For
that purpose, the Pulso platform was developed [JA05] to sustain mechanisms of near-realtime measur-
ing, monitoring and storing basic performance indicators for major systems that support key business
processes.
However, the evaluation procedures are still ad-hoc since there's no technical or conceptual framework
that allows - in a simple, declarative and semantically robust way - to dene what entities are being
targetted and how to specify and calculate the relevant indicators for either Quality of Service, Quality
of Protection or even Quality of Maintenance.
The EDS1 division of PT-Comunicações is responsible for this monitoring platform and, as such,
provided an environment that was both resourceful and accessible for us to experiment with our solution
since we were given access to an infrastructure of monitoring, data collector agents attached to every
relevant server, database and network link. By relevant set we mean the entities that are, at the time
of this writing, critical to the business.
4.1.1 Agents and Probes
The Pulso monitoring framework is comprised of system monitoring agents (e.g. Linux, Windows, HP-
UX, Oracle) and network probes (client-server and server-server).
Agents are system-specic software whose job is to periodically capture basic behaviour readings of
their hosts. For instance, Linux agents capture observable variables such as CPU usage, CPU load (for 1,
5 and 15 minutes) and memory usage. For databases, agents designed for Oracle SGBDs capture waiting
times for query processing queues. Depending on the business demands, the agents are further developed
to produce more observable variables.
Probes passively monitor network trac between either servers and clients or between servers. They
capture data regarding to bandwidth consumption and protocol-specic trac (e.g. SQL transaction
1Eciencia, Disponibilidade e Segurança de TI/SI
39
4.1.2 Measuring IT
As previously said, one of the desired features of Pulso is the evaluation of IT and the ability to verify
compliance with Service Level Agreements.
In the Pulso platform the denition of Enterprise entities (ranging from disk partitions to business
processes) is constructed using a graph of things and the links between then. Therefore, there are no
classes, no type hierarchy and no architectural layers. This approach provided some benets due to the
lack of constrains on the evaluation process, making it is faster to implement. However, by allowing any
entity to be related to another a