-
7/29/2019 Towards Complexity Analysis of Software Process Improvement Frameworks
1/15
1
Towards Complexity Analysis of
Software Process Improvement Frameworks
Zador Daniel Kelemen1, Rob Kusters2, Jos Trienekens2, Katalin Balla3, 41ThyssenKrupp Presta Hungary
[email protected],2Eindhoven University of Technology
[email protected],[email protected] ,3Budapest University of Technology and Economics
[email protected],4SQI Hungarian Software Quality Consulting Institute
Abstract Many of quality approaches are described in hundreds of textual pages(see CMMI, SPICE, Enterprise SPICE, ITIL among others). Manual processing
of information consumes plenty of resources. In this report we present the
complexity analysis of CMMI one well known and widely process improvement
framework. The complexity analysis can provide a quick overview on the
coupledness of the elements of quality approaches by an automated analysis of
the cross-references in quality approaches. The result of the analysis could
accelerate the understanding the quality approach and help in setting the starting
points of improvement.
Keywords: quality approach, complexity analysis, CMMI, structure, cross-references, coupledness
1 IntroductionIn this report we present preliminary results of applying a complexity analysis
on software process improvement frameworks, models and standards.
We call each software quality standard, method, technique, model and
(improvement) framework to improve software processes software qualityapproach [1] and we call complexity analysis the analysis of the cross-referencesof quality approaches and discovery of isolated and coupled element instances.The complexity analysis of quality approaches can provide an automated
analysis, thus no manual interaction is needed. It could strengthen the current
primarily qualitative investigations by providing a quick overview on quality
approaches. Complexity analysis could help in understanding focus, decisions in
implementation and training as well as in tool development[2] and appraisals.
We analysed structure and elements of 7 different process based software quality
approaches in [3] and some other versions in [4], [5], which can provide a basis
for further analysis of quality approaches. However, deeper investigation and
mailto:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected] -
7/29/2019 Towards Complexity Analysis of Software Process Improvement Frameworks
2/15
2
analysis (e.g. complexity of) quality approaches were not provided and are not
common in the literature, even though these could contribute to better
understanding and processing and even in comparison [6] of quality approaches
[7], [8]. Preliminary complexity analyses were presented in [9] [13]. These
investigations of quality approaches are mainly focusing on cross-references and
interrelations inside quality approaches (among their elements or element
instances). In this report we present preliminary results of investigating cross-
references in CMMI[12], [14].
The current version of CMMI, v1.3 defines 3 constellations: CMMI for
Development[15], CMMI for Services[16] and CMMI for Acquisition [17]. Figure
1 presents a summary of CMMI versions 1.1-1.3, constellations and their
elements, showing that constellations have about 400-500 pages each, and version
1.3 has 1440 pages in total. Standards such as SPICE, Enterprise SPICE, and
TMMi among others have similar size. The amount of information in thesestandards makes it difficult to understand and apply them; therefore in this
report we show preliminary results of analysing CMMI as an example of possible
complexity analysis of quality approaches.
Figure 1 CMMI versions and constellations and their elements. Source: [18]
In section 2 we briefly present elements of CMMI, in section 3 the search for
cross-references is presented. Section 4 describes the analysis and section 5includes preliminary results. Conclusion is included into 7 after addressing
limitations in 6.
2 Structure of CMMIIn order to understand cross-references, first we need to understand structure
and elements and know which elements can be inter-related (or cross references).
In this section we present the structure and elements of CMMI.
-
7/29/2019 Towards Complexity Analysis of Software Process Improvement Frameworks
3/15
3
CMMI defines its structure as follows:
All CMMI models are produced from the CMMI Framework. This
framework contains all of the goals and practices that are used to produce CMMI
models that belong to CMMI constellations. ... Model components are grouped
into three categories required, expected, and informative that reflect how to
interpret them. Required components are CMMI components that are
essential to achieving process improvement in a given process area. The
required components in CMMI are the specific and generic goals. Goal
satisfaction is used in appraisals as the basis for deciding whether a process area
has been satisfied.
Expected components are CMMI components that describe the activities
that are important in achieving a required CMMI component. Expected
components guide those who implement improvements or perform appraisals.
The expected components in CMMI are the specific and generic practices. Beforegoals can be considered to be satisfied, either their practices as described, or
acceptable alternatives to them, must be present in the planned and implemented
processes of the organization.
Figure 2 The structure of CMMI v1.3
Informative components are CMMI components that help model users
understand CMMI required and expected components. These components can be
example boxes, detailed explanations, or other helpful information. Subpractices,
notes, references, goal titles, practice titles, sources, example work products, and
generic practice elaborations are informative model components. [15].
Figure 2 shows the main elements of CMMI and their interrelations.
Elements of CMMICMMI is a well-structured quality approach, it clearly defines its elements,
and element types are described using different text styles in the document so
-
7/29/2019 Towards Complexity Analysis of Software Process Improvement Frameworks
4/15
4
that it is easy to recognize which text belongs to which element. Table 1 provides
a summary of elements of CMMI v1.3.
3 Searching for cross-references in CMMIIn order to understand relations among CMMI element instances we conducted
a systematic search within CMMI documents.
Search spaceIn this search we focused on CMMI v1.3 and we analyzed all three
constellations. As the goal was to understand relations among element
(instance)s of CMMI, the search was narrowed to the chapter 2 of all three CMMI
documents these contain process areas and their related elements (e.g. specific
and generic practices). We filtered out all the elements which were irrelevant for
our search. These were introductions, appendices, remarks, notes, designelements etc.
Search termsFirst we conducted a manual analysis on CMMI element instances such as
process areas, specific and generic practices. Based on this manual analysis it was
easy to discover cross-references, as they intentionally appeared in the following
patterns:
Refer to the process area
Refer to the specific practice in process area
We have not found other reference patterns in CMMI, therefore we searched
for these two types of references. Some irrelevant search results were filtered out
when searching for patterns such as the organizations set of standard processes
can refer to the standard processes established at the organization level.
Search resultsThe search resulted in 1016 cross-references in total, and 992 after applying
filters. Out of these 311 were found in CMMI-DEV, 388 in CMMI-SVC and 293
in CMMI-ACQ.
As CMMI-DEV is the most common constellation in the following we show
some results gained from this constellation, excluding CMMI-ACQ and CMMI-
SVC from the remaining discussion.
4 Complexity analysis: understanding cross-references
In CMMI-DEV, cross-references were found in various element instances e.g. in
instances of introductory notes, related process areas, specific practices and
generic practices. Besides that references can be found on different levels, these
are pointing to different element instances such as instances of process areas,
specific goals and specific practices.
Figure 3 shows generic practices and process areas referenced in CMMI. For
better visibility, references found below process area level (e.g. in introductory
-
7/29/2019 Towards Complexity Analysis of Software Process Improvement Frameworks
5/15
5
notes, specific goals and specific practices) are represented on process area level.
The figure shows that the least referred element instances are the generic
practices and the PPQA (Process and Product Quality Assurance) process area,
while most referred process areas are IPM (Integrated Project Management), TS
(Technical Solution), QPM (Quantitative Project Management), PI (Product
Integration) and OPM (Organizational Performance Management).
Figure 3 References to CMMI process areas and generic practices
Less referenced elements could be more independent and thus easier to
understand and implement, e.g. at teaching CMMI process areas PPQA could
be useful to start with. At the same time understanding and implementing highly
referred process areas might be crucial, but also more difficult as several other
process areas rely on these. Better understanding of highly referred process areas
could be especially useful in MSPI: e.g. implementing IPM using multiple quality
approaches might be challenging, because other process areas will need to be
taken into consideration when developing the IPM process.
Figure 4 shows process areas to which other CMMI element instances refer
to. The figure shows that the process areas containing a small number of
references are: PPQA, CAR (Causal Analysis and Resolution) and OT
(Organizational Training), and the processes areas containing the most of the
references are: MA (Measurement and Analysis), PP (Project Planning) and
PMC (Project Monitoring and Control). Process areas which contain a few
references are probably more independent from other areas. For instance in
Process and Product Quality Assurance it is important that the process should
not depend on other processes and to be as independent as possible, the
Organizational Training is also usually independent from other processes.
0
5
10
15
20
25
30
35
-
7/29/2019 Towards Complexity Analysis of Software Process Improvement Frameworks
6/15
6
Figure 4 References from CMMI-DEV v1.3 process areas
The Measurement and Analysis, Project Planning and Project Monitoring
and Control process areas contain many references to other CMMI element
instances, therefore these could depend on many other process areas.
Measurements, project planning and project monitoring and control activities are
integral parts of maturity level 2 project and upwards. The implementation of
these process areas may depend on the operation of a number of other process
areas, therefore it deserves special attention to be paid on their appropriate
implementation.
Figure 3 and Figure 4 showed process areas and generic practices which arethe most and the least referenced and those which contain the most and the least
references. Figure 5 goes further and shows coupledness of elements. On the
horizontal (X) axis those process areas and generic practices are represented
which contain references, while on the vertical (Y) axis the referred elements are
represented. The colour of the dots on the figure varies proportionally to the
number of references between 1 and 10. The warmer the colour the more
references are present from X to Y. The figure shows clearly that PMC has the
most references to another process area: the PP. This seems logical as monitoring
and control should rely on planning. For the opposite direction there is only one
reference (from PP to PMC).
Summarizing cross-references in Figure 5, we can see the followings: the mostreferences are between TS (Technical Solution) and RD (Requirements
Development) (8,4). The number of TS-RD references was not expected, and
would be an interesting subject of further investigation. Further highly coupled
elements are PP-PMC, IPM-OPD (8,1), OPM-OPD (6,2) and QPM-OPD (4,3).
Similarly to the previous two figures, Figure 5 also shows the isolated element
instances (e.g. PPQA).
0
5
10
15
20
25
30
35
PPQA
CAR
OT P
I
VAL
OPM
QPM
CM
IPM
OPP
SAM
OPF
TS
RSKM
REQMR
D
DAR
VER
OPD
PMC
PP
MA
-
7/29/2019 Towards Complexity Analysis of Software Process Improvement Frameworks
7/15
7
Figure 5 References among CMMI-DEV v1.3 process areas and generic
practices
Figure 6 A graphical representation of references among process areas and
generic practices in CMMI-DEV v1.3
It could be worthwhile to implement highly coupled process areas together
and independent ones more independently. We also notice that only 16.6% of
elements have more than 6 cross-references (counting both directions) with one
another element, which shows a closely Pareto distribution of cross-references.
-
7/29/2019 Towards Complexity Analysis of Software Process Improvement Frameworks
8/15
8
Figure 6 and Figure 7 in the appendix present various views on CMMI-DEV
v1.3 cross-references. Figure 8 shows a step of complexity analysis by making use
a software tool Cytoscape.
5 Preliminary resultsCross-references are present in quality approaches. In this report we showed
preliminary results of analysing cross-references in CMMI. Based on these
references, highly coupled and independent element instances can be
distinguished and these might need different implementation. A further
systematic analysis could help in understanding the complexity and interrelations
within a quality approach. Understanding cross-references could serve to a better
implementation, training, (e.g. simplified) appraisals or better tool development.
Similar references might be present in other quality approaches; therefore bothCMMI and other approaches would need further complexity-analyses.
6 LimitationsThe approach presented in this report needs further validation, especially to
strengthen external validity: it would be needed to perform the same complexity
analysis on further quality approaches. Different quality approaches have
different structure, therefore before performing the complexity analysis a
discovery of the structure and elements of other quality approaches is needed. In
order to have a basis for this we presented the structure of quality approaches
(see section 2).
Another validation and further research would be needed on the meaning and
usage of coupledness information.
Summarizing, we must admit that both results and limitations motivate
further research of applying complexity analysis on quality approaches.
7 ConclusionThe goal of this report1 was to present a number of issues that arose during a
research aimed at providing a formal solution for the problem of multi-model
software process improvement and are relevant, but outside of the scope [3].Limited number of literature deals with the quantitative analysis of quality
approaches, despite that this can be useful when understanding complexity and
scope of the quality approaches. In this report we presented preliminary results
of analysing CMMI cross-references. The CMMI was chosen as a widely known
and accepted quality approach, however same technique may be applicable to
other quality approaches. Based on the complexity analysis highly and loosely
1 This report has been published as a part of a PhD thesis [3] and it is part of a
series of technical reports which include: [19] [22].
-
7/29/2019 Towards Complexity Analysis of Software Process Improvement Frameworks
9/15
9
coupled element instances of CMMI were identified. Coupledness can be used in
training or when starting the implementation of a quality approach. Highly
coupled element instances may require further investigation, while loosely
coupled elements could be implemented and learned more independently. Effects,
meaning and usage of coupledness of quality approach element instances require
further research and validation. This could be achieved through the analysis of
further quality approaches and validated by various research instruments such
as real-world case studies of teaching, implementation or tool development.
-
7/29/2019 Towards Complexity Analysis of Software Process Improvement Frameworks
10/15
10
Appendices
A.List of AbbreviationsCAR Causal Analysis and Resolution
CM Configuration Management
CMMI Capability Maturity Model Integration
DAR Decision Analysis and Resolution
GP Generic Practice
IPM Integrated Project Management
ITIL Information Technology Infrastructure Library
MA Measurement and Analysis
MSPI Multimodel Software Process Improvement
OPM Organizational Performance Management
OPD Organizational Process Definition
OPF Organizational Process Focus
OPP Organizational Process Performance
OT Organizational Training
PI Product Integration
PMC Project Monitoring and Control
PP Project Planning
PPQA Process and Product Quality AssuranceQPM Quantitative Project Management
RD Requirements Development
REQM Requirements Management
RSKM Risk Management
SAM Supplier Agreement Management
SP Specific Practice
SPICE Software Process Improvement and Capability DeterminationTS Technical Solution
VAL
ValidationVER Verification
-
7/29/2019 Towards Complexity Analysis of Software Process Improvement Frameworks
11/15
11
B.Elements of CMMITable 1 Elements of CMMI-v1.3
Element Description
Constellation
A collection of CMMI components that are used to construct
models, training materials, and appraisal related documents
for an area of interest (e.g., acquisition, development,
services).
Process area
A cluster of related practices in an area that, when
implemented collectively, satisfies a set of goals considered
important for making improvement in that area.
Purpose
statement
A purpose statement describes the purpose of the process
area and is an informative component.
Introductory
note
The introductory notes section of the process area describes
the major concepts covered in the process area and is an
informative component.
Related areas
The Related Process Areas section lists references to related
process areas and reflects the high-level relationships among
the process areas. The Related Process Areas section is an
informative component.
Specific goal
A specific goal describes the unique characteristics that must
be present to satisfy the process area. A specific goal is a
required model component and is used in appraisals to help
determine whether a process area is satisfied.
Generic goal
Generic goals are called generic because the same goal
statement applies to multiple process areas. A generic goal
describes the characteristics that must be present to
institutionalize processes that implement a process area. A
generic goal is a required model component and is used in
appraisals to determine whether a process area is satisfied.
Specific goal
and practice
summaries
The specific goal and practice summary provides a high-level
summary of the specific goals and specific practices. The
specific goal and practice summary is an informative
component.
Specific practice
A specific practice is the description of an activity that is
considered important in achieving the associated specific goal.The specific practices describe the activities that are expected
to result in achievement of the specific goals of a process area.
A specific practice is an expected model component.
Example work
product
The example work products section lists sample outputs
from a specific practice. An example work product is an
informative model component.
-
7/29/2019 Towards Complexity Analysis of Software Process Improvement Frameworks
12/15
12
Element Description
Subpractice
A subpractice is a detailed description that provides
guidance for interpreting and implementing a specific or
generic practice. Subpractices can be worded as if prescriptive,
but they are actually an informative component meant only to
provide ideas that may be useful for process improvement.
Generic
practice
Generic practices are called generic because the same
practice applies to multiple process areas. The generic
practices associated with a generic goal describe the activities
that are considered important in achieving the generic goal
and contribute to the institutionalization of the processes
associated with a process area. A generic practice is an
expected model component.
GenericPractice
Elaboration
Generic practice elaborations appear after generic practices
to provide guidance on how the generic practices can beapplied uniquely to process areas. A generic practice
elaboration is an informative model component.
Addition
Additions are clearly marked model components that
contain information of interest to particular users. An
addition can be informative material, a specific practice, a
specific goal, or an entire process area that extends the scope
of a model or emphasizes a particular aspect of its use.
Note
A note is text that can accompany nearly any other model
component. It may provide detail, background, or rationale. A
note is an informative model component.
Example
An example is a component comprising text and often a listof items, usually in a box, that can accompany nearly any
other component and provides one or more examples to clarify
a concept or described activity. An example is an informative
model component.
Reference
A reference is a pointer to additional or more detailed
information in related process areas and can accompany
nearly any other model component. A reference is an
informative model component.
-
7/29/2019 Towards Complexity Analysis of Software Process Improvement Frameworks
13/15
13
C.A representation of complexity analysis in CMMI
Figure 7 A Circular view of the connections among CMMI process areas
and generic practices
D.Analysing the network within CMMI
Figure 8 Analysing the network within CMMI with Cytoscape
-
7/29/2019 Towards Complexity Analysis of Software Process Improvement Frameworks
14/15
14
References
[1] Z. D. Kelemen, R. Kusters, and J. Trienekens,Identifying criteria formultimodel software process improvement solutions - based on a review of
current problems and initiatives,J. Softw. Evol. Process, vol. 24, no. 8,pp. 895909, Dec. 2012.
[2] Z. D. Kelemen, K. Balla, and G. Bka, Quality Organizer: a support toolin using multiple quality approaches, in Proceedings of 8th InternationalCarpathian Control Conference (ICCC 2007), Strbske Pleso, Slovakia,2007.
[3] Z. D. Kelemen, Process Based Unification for Multi-Model Software ProcessImprovement. Eindhoven: Technische Universiteit Eindhoven, 2013.
[4] Z. D. Kelemen, K. Balla, J. Trienekens, and R. Kusters, Structure ofProcess-Based Quality Approaches - Elements of a research developing acommon meta-model for proces-based quality approaches and methods, in
Proceedings of EuroSPI 2008 Doctoral Symposium, Dublin, Ireland, 2008.[5] Z. D. Kelemen, K. Balla, J. Trienekens, and R. Kusters, Towardssupporting simultaneous use of process-based quality approaches, inProceedings of 9th international Carpathian Control Conference: ICCC2008, Sinaia, Romania, 2008, pp. 291295.
[6] Z. D. Kelemen and K. Balla, A CMMI-DEV v1.2 s az ISO 9001:2000kapcsolata,Magy. Minsg, vol. 17, no. 2, pp. 2740, 2008.
[7] S. Jeners, H. Lichter, and E. Pyatkova, Automated Comparison of ProcessImprovement Reference Models Based on Similarity Metrics, 2012, pp.743748.
[8] S. Jeners and H. Lichter, Smart Integration of Process ImprovementReference Models Based on an Automated Comparison Approach, inSystems, Software and Services Process Improvement, vol. 364, F.McCaffery, R. OConnor, and R. Messnarz, Eds. Springer Berlin Heidelberg,2013, pp. 143154.
[9] X. Chen, M. Staples, and P. Bannerman, Analysis of Dependenciesbetween Specific Practices in CMMI Maturity Level 2, in Software ProcessImprovement, vol. 16, R. V. OConnor, N. Baddoo, K. Smolander, and R.Messnarz, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2008, pp.94105.
[10] A. L. Ferreira, R. J. Machado, and M. C. Paulk, Size and ComplexityAttributes for Multimodel Improvement Framework Taxonomy, inSoftware Engineering and Advanced Applications (SEAA), 2010 36thEUROMICRO Conference on, 2010, pp. 306 309.
[11] A. L. Ferreira, R. J. Machado, and M. C. Paulk, Quantitative Analysis of
Best Practices Models in the Software Domain, presented at the 2010 AsiaPacific Software Engineering Conference, 2010.
[12] Z. D. Kelemen, What is important in CMMI and what are theinterrelations among its elements?, presented at the International ODFSymposium, Budapest, 28-Jun-2011.
[13] P. Monteiro, R. J. Machado, R. Kazman, and C. Henriques, DependencyAnalysis between CMMI Process Areas, in Product-Focused SoftwareProcess Improvement, vol. 6156, M. Ali Babar, M. Vierimaa, and M. Oivo,Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2010, pp. 263275.
-
7/29/2019 Towards Complexity Analysis of Software Process Improvement Frameworks
15/15
15
[14] K. Balla and Z. D. Kelemen, Important Concepts In CMMI and What isDifficult to Understand, presented at the SEPG Europe 2011, Dublin, 07-Jun-2011.
[15] C. S. CMMI Product Team, CMMI for Development, Version 1.3. CMUSEI, Nov-2010.
[16] C. S. CMMI Product Team, CMMI for Services, Version 1.3. CMU SEI,Nov-2010.
[17] C. S. CMMI Product Team, CMMI for Acquisition, Version 1.3. CMUSEI, Nov-2010.
[18] E. Forrester and G. Wemyss, CMMI Version 1.3 and Beyond, Dublin,Ireland, 2011.
[19] Z. D. Kelemen, R. Kusters, J. Trienekens, and K. Balla, A Data Model forMultimodel Process Improvement, Budapest, Technical Report TR201303,Sep. 2013.
[20] Z. D. Kelemen, R. Kusters, J. Trienekens, and K. Balla, Towards Applying
Text Mining Techniques on Software Quality Standards and Models,Budapest, Technical Report TR201302, Sep. 2013.
[21] Z. D. Kelemen, R. Kusters, J. Trienekens, and K. Balla, TowardsComplexity Analysis of Software Process Improvement Frameworks,Budapest, Technical Report TR201301, Sep. 2013.
[22] Z. D. Kelemen, R. Kusters, J. Trienekens, and K. Balla, Selecting a ProcessModeling Language for Unification of Multiple Standards and Models,Budapest, Technical Report TR201304, Sep. 2013.