towards complexity analysis of software process improvement frameworks

7/29/2019 Towards Complexity Analysis of Software Process Improvement Frameworks

1/15

1

Towards Complexity Analysis of

Software Process Improvement Frameworks

Zador Daniel Kelemen1, Rob Kusters2, Jos Trienekens2, Katalin Balla3, 41ThyssenKrupp Presta Hungary

[email protected],2Eindhoven University of Technology

[email protected],[email protected] ,3Budapest University of Technology and Economics

[email protected],4SQI Hungarian Software Quality Consulting Institute

[email protected]

Abstract Many of quality approaches are described in hundreds of textual pages(see CMMI, SPICE, Enterprise SPICE, ITIL among others). Manual processing

of information consumes plenty of resources. In this report we present the

complexity analysis of CMMI one well known and widely process improvement

framework. The complexity analysis can provide a quick overview on the

coupledness of the elements of quality approaches by an automated analysis of

the cross-references in quality approaches. The result of the analysis could

accelerate the understanding the quality approach and help in setting the starting

points of improvement.

Keywords: quality approach, complexity analysis, CMMI, structure, cross-references, coupledness

1 IntroductionIn this report we present preliminary results of applying a complexity analysis

on software process improvement frameworks, models and standards.

We call each software quality standard, method, technique, model and

(improvement) framework to improve software processes software qualityapproach [1] and we call complexity analysis the analysis of the cross-referencesof quality approaches and discovery of isolated and coupled element instances.The complexity analysis of quality approaches can provide an automated

analysis, thus no manual interaction is needed. It could strengthen the current

primarily qualitative investigations by providing a quick overview on quality

approaches. Complexity analysis could help in understanding focus, decisions in

implementation and training as well as in tool development[2] and appraisals.

We analysed structure and elements of 7 different process based software quality

approaches in [3] and some other versions in [4], [5], which can provide a basis

for further analysis of quality approaches. However, deeper investigation and
mailto:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]


2/15

2

analysis (e.g. complexity of) quality approaches were not provided and are not

common in the literature, even though these could contribute to better

understanding and processing and even in comparison [6] of quality approaches

[7], [8]. Preliminary complexity analyses were presented in [9] [13]. These

investigations of quality approaches are mainly focusing on cross-references and

interrelations inside quality approaches (among their elements or element

instances). In this report we present preliminary results of investigating cross-

references in CMMI[12], [14].

The current version of CMMI, v1.3 defines 3 constellations: CMMI for

Development[15], CMMI for Services[16] and CMMI for Acquisition [17]. Figure

1 presents a summary of CMMI versions 1.1-1.3, constellations and their

elements, showing that constellations have about 400-500 pages each, and version

1.3 has 1440 pages in total. Standards such as SPICE, Enterprise SPICE, and

TMMi among others have similar size. The amount of information in thesestandards makes it difficult to understand and apply them; therefore in this

report we show preliminary results of analysing CMMI as an example of possible

complexity analysis of quality approaches.

Figure 1 CMMI versions and constellations and their elements. Source: [18]

In section 2 we briefly present elements of CMMI, in section 3 the search for

cross-references is presented. Section 4 describes the analysis and section 5includes preliminary results. Conclusion is included into 7 after addressing

limitations in 6.

2 Structure of CMMIIn order to understand cross-references, first we need to understand structure

and elements and know which elements can be inter-related (or cross references).

In this section we present the structure and elements of CMMI.


3/15

3

CMMI defines its structure as follows:

All CMMI models are produced from the CMMI Framework. This

framework contains all of the goals and practices that are used to produce CMMI

models that belong to CMMI constellations. ... Model components are grouped

into three categories required, expected, and informative that reflect how to

interpret them. Required components are CMMI components that are

essential to achieving process improvement in a given process area. The

required components in CMMI are the specific and generic goals. Goal

satisfaction is used in appraisals as the basis for deciding whether a process area

has been satisfied.

Expected components are CMMI components that describe the activities

that are important in achieving a required CMMI component. Expected

components guide those who implement improvements or perform appraisals.

The expected components in CMMI are the specific and generic practices. Beforegoals can be considered to be satisfied, either their practices as described, or

acceptable alternatives to them, must be present in the planned and implemented

processes of the organization.

Figure 2 The structure of CMMI v1.3

Informative components are CMMI components that help model users

understand CMMI required and expected components. These components can be

example boxes, detailed explanations, or other helpful information. Subpractices,

notes, references, goal titles, practice titles, sources, example work products, and

generic practice elaborations are informative model components. [15].

Figure 2 shows the main elements of CMMI and their interrelations.

Elements of CMMICMMI is a well-structured quality approach, it clearly defines its elements,

and element types are described using different text styles in the document so


4/15

4

that it is easy to recognize which text belongs to which element. Table 1 provides

a summary of elements of CMMI v1.3.

3 Searching for cross-references in CMMIIn order to understand relations among CMMI element instances we conducted

a systematic search within CMMI documents.

Search spaceIn this search we focused on CMMI v1.3 and we analyzed all three

constellations. As the goal was to understand relations among element

(instance)s of CMMI, the search was narrowed to the chapter 2 of all three CMMI

documents these contain process areas and their related elements (e.g. specific

and generic practices). We filtered out all the elements which were irrelevant for

our search. These were introductions, appendices, remarks, notes, designelements etc.

Search termsFirst we conducted a manual analysis on CMMI element instances such as

process areas, specific and generic practices. Based on this manual analysis it was

easy to discover cross-references, as they intentionally appeared in the following

patterns:

Refer to the process area

Refer to the specific practice in process area

We have not found other reference patterns in CMMI, therefore we searched

for these two types of references. Some irrelevant search results were filtered out

when searching for patterns such as the organizations set of standard processes

can refer to the standard processes established at the organization level.

Search resultsThe search resulted in 1016 cross-references in total, and 992 after applying

filters. Out of these 311 were found in CMMI-DEV, 388 in CMMI-SVC and 293

in CMMI-ACQ.

As CMMI-DEV is the most common constellation in the following we show

some results gained from this constellation, excluding CMMI-ACQ and CMMI-

SVC from the remaining discussion.

4 Complexity analysis: understanding cross-references

In CMMI-DEV, cross-references were found in various element instances e.g. in

instances of introductory notes, related process areas, specific practices and

generic practices. Besides that references can be found on different levels, these

are pointing to different element instances such as instances of process areas,

specific goals and specific practices.

Figure 3 shows generic practices and process areas referenced in CMMI. For

better visibility, references found below process area level (e.g. in introductory


5/15

5

notes, specific goals and specific practices) are represented on process area level.

The figure shows that the least referred element instances are the generic

practices and the PPQA (Process and Product Quality Assurance) process area,

while most referred process areas are IPM (Integrated Project Management), TS

(Technical Solution), QPM (Quantitative Project Management), PI (Product

Integration) and OPM (Organizational Performance Management).

Figure 3 References to CMMI process areas and generic practices

Less referenced elements could be more independent and thus easier to

understand and implement, e.g. at teaching CMMI process areas PPQA could

be useful to start with. At the same time understanding and implementing highly

referred process areas might be crucial, but also more difficult as several other

process areas rely on these. Better understanding of highly referred process areas

could be especially useful in MSPI: e.g. implementing IPM using multiple quality

approaches might be challenging, because other process areas will need to be

taken into consideration when developing the IPM process.

Figure 4 shows process areas to which other CMMI element instances refer

to. The figure shows that the process areas containing a small number of

references are: PPQA, CAR (Causal Analysis and Resolution) and OT

(Organizational Training), and the processes areas containing the most of the

references are: MA (Measurement and Analysis), PP (Project Planning) and

PMC (Project Monitoring and Control). Process areas which contain a few

references are probably more independent from other areas. For instance in

Process and Product Quality Assurance it is important that the process should

not depend on other processes and to be as independent as possible, the

Organizational Training is also usually independent from other processes.

0

5

10

15

20

25

30

35


6/15

6

Figure 4 References from CMMI-DEV v1.3 process areas

The Measurement and Analysis, Project Planning and Project Monitoring

and Control process areas contain many references to other CMMI element

instances, therefore these could depend on many other process areas.

Measurements, project planning and project monitoring and control activities are

integral parts of maturity level 2 project and upwards. The implementation of

these process areas may depend on the operation of a number of other process

areas, therefore it deserves special attention to be paid on their appropriate

implementation.

Figure 3 and Figure 4 showed process areas and generic practices which arethe most and the least referenced and those which contain the most and the least

references. Figure 5 goes further and shows coupledness of elements. On the

horizontal (X) axis those process areas and generic practices are represented

which contain references, while on the vertical (Y) axis the referred elements are

represented. The colour of the dots on the figure varies proportionally to the

number of references between 1 and 10. The warmer the colour the more

references are present from X to Y. The figure shows clearly that PMC has the

most references to another process area: the PP. This seems logical as monitoring

and control should rely on planning. For the opposite direction there is only one

reference (from PP to PMC).

Summarizing cross-references in Figure 5, we can see the followings: the mostreferences are between TS (Technical Solution) and RD (Requirements

Development) (8,4). The number of TS-RD references was not expected, and

would be an interesting subject of further investigation. Further highly coupled

elements are PP-PMC, IPM-OPD (8,1), OPM-OPD (6,2) and QPM-OPD (4,3).

Similarly to the previous two figures, Figure 5 also shows the isolated element

instances (e.g. PPQA).

0

5

10

15

20

25

30

35

PPQA

CAR

OT P

I

VAL

OPM

QPM

CM

IPM

OPP

SAM

OPF

TS

RSKM

REQMR

D

DAR

VER

OPD

PMC

PP

MA


7/15

7

Figure 5 References among CMMI-DEV v1.3 process areas and generic

practices

Figure 6 A graphical representation of references among process areas and

generic practices in CMMI-DEV v1.3

It could be worthwhile to implement highly coupled process areas together

and independent ones more independently. We also notice that only 16.6% of

elements have more than 6 cross-references (counting both directions) with one

another element, which shows a closely Pareto distribution of cross-references.


8/15

8

Figure 6 and Figure 7 in the appendix present various views on CMMI-DEV

v1.3 cross-references. Figure 8 shows a step of complexity analysis by making use

a software tool Cytoscape.

5 Preliminary resultsCross-references are present in quality approaches. In this report we showed

preliminary results of analysing cross-references in CMMI. Based on these

references, highly coupled and independent element instances can be

distinguished and these might need different implementation. A further

systematic analysis could help in understanding the complexity and interrelations

within a quality approach. Understanding cross-references could serve to a better

implementation, training, (e.g. simplified) appraisals or better tool development.

Similar references might be present in other quality approaches; therefore bothCMMI and other approaches would need further complexity-analyses.

6 LimitationsThe approach presented in this report needs further validation, especially to

strengthen external validity: it would be needed to perform the same complexity

analysis on further quality approaches. Different quality approaches have

different structure, therefore before performing the complexity analysis a

discovery of the structure and elements of other quality approaches is needed. In

order to have a basis for this we presented the structure of quality approaches

(see section 2).

Another validation and further research would be needed on the meaning and

usage of coupledness information.

Summarizing, we must admit that both results and limitations motivate

further research of applying complexity analysis on quality approaches.

7 ConclusionThe goal of this report1 was to present a number of issues that arose during a

research aimed at providing a formal solution for the problem of multi-model

software process improvement and are relevant, but outside of the scope [3].Limited number of literature deals with the quantitative analysis of quality

approaches, despite that this can be useful when understanding complexity and

scope of the quality approaches. In this report we presented preliminary results

of analysing CMMI cross-references. The CMMI was chosen as a widely known

and accepted quality approach, however same technique may be applicable to

other quality approaches. Based on the complexity analysis highly and loosely

1 This report has been published as a part of a PhD thesis [3] and it is part of a

series of technical reports which include: [19] [22].


9/15

9

coupled element instances of CMMI were identified. Coupledness can be used in

training or when starting the implementation of a quality approach. Highly

coupled element instances may require further investigation, while loosely

coupled elements could be implemented and learned more independently. Effects,

meaning and usage of coupledness of quality approach element instances require

further research and validation. This could be achieved through the analysis of

further quality approaches and validated by various research instruments such

as real-world case studies of teaching, implementation or tool development.


10/15

10

Appendices

A.List of AbbreviationsCAR Causal Analysis and Resolution

CM Configuration Management

CMMI Capability Maturity Model Integration

DAR Decision Analysis and Resolution

GP Generic Practice

IPM Integrated Project Management

ITIL Information Technology Infrastructure Library

MA Measurement and Analysis

MSPI Multimodel Software Process Improvement

OPM Organizational Performance Management

OPD Organizational Process Definition

OPF Organizational Process Focus

OPP Organizational Process Performance

OT Organizational Training

PI Product Integration

PMC Project Monitoring and Control

PP Project Planning

PPQA Process and Product Quality AssuranceQPM Quantitative Project Management

RD Requirements Development

REQM Requirements Management

RSKM Risk Management

SAM Supplier Agreement Management

SP Specific Practice

SPICE Software Process Improvement and Capability DeterminationTS Technical Solution

VAL

ValidationVER Verification


11/15

11

B.Elements of CMMITable 1 Elements of CMMI-v1.3

Element Description

Constellation

A collection of CMMI components that are used to construct

models, training materials, and appraisal related documents

for an area of interest (e.g., acquisition, development,

services).

Process area

A cluster of related practices in an area that, when

implemented collectively, satisfies a set of goals considered

important for making improvement in that area.

Purpose

statement

A purpose statement describes the purpose of the process

area and is an informative component.

Introductory

note

The introductory notes section of the process area describes

the major concepts covered in the process area and is an

informative component.

Related areas

The Related Process Areas section lists references to related

process areas and reflects the high-level relationships among

the process areas. The Related Process Areas section is an

informative component.

Specific goal

A specific goal describes the unique characteristics that must

be present to satisfy the process area. A specific goal is a

required model component and is used in appraisals to help

determine whether a process area is satisfied.

Generic goal

Generic goals are called generic because the same goal

statement applies to multiple process areas. A generic goal

describes the characteristics that must be present to

institutionalize processes that implement a process area. A

generic goal is a required model component and is used in

appraisals to determine whether a process area is satisfied.

Specific goal

and practice

summaries

The specific goal and practice summary provides a high-level

summary of the specific goals and specific practices. The

specific goal and practice summary is an informative

component.

Specific practice

A specific practice is the description of an activity that is

considered important in achieving the associated specific goal.The specific practices describe the activities that are expected

to result in achievement of the specific goals of a process area.

A specific practice is an expected model component.

Example work

product

The example work products section lists sample outputs

from a specific practice. An example work product is an

informative model component.


12/15

12

Element Description

Subpractice

A subpractice is a detailed description that provides

guidance for interpreting and implementing a specific or

generic practice. Subpractices can be worded as if prescriptive,

but they are actually an informative component meant only to

provide ideas that may be useful for process improvement.

Generic

practice

Generic practices are called generic because the same

practice applies to multiple process areas. The generic

practices associated with a generic goal describe the activities

that are considered important in achieving the generic goal

and contribute to the institutionalization of the processes

associated with a process area. A generic practice is an

expected model component.

GenericPractice

Elaboration

Generic practice elaborations appear after generic practices

to provide guidance on how the generic practices can beapplied uniquely to process areas. A generic practice

elaboration is an informative model component.

Addition

Additions are clearly marked model components that

contain information of interest to particular users. An

addition can be informative material, a specific practice, a

specific goal, or an entire process area that extends the scope

of a model or emphasizes a particular aspect of its use.

Note

A note is text that can accompany nearly any other model

component. It may provide detail, background, or rationale. A

note is an informative model component.

Example

An example is a component comprising text and often a listof items, usually in a box, that can accompany nearly any

other component and provides one or more examples to clarify

a concept or described activity. An example is an informative

model component.

Reference

A reference is a pointer to additional or more detailed

information in related process areas and can accompany

nearly any other model component. A reference is an

informative model component.


13/15

13

C.A representation of complexity analysis in CMMI

Figure 7 A Circular view of the connections among CMMI process areas

and generic practices

D.Analysing the network within CMMI

Figure 8 Analysing the network within CMMI with Cytoscape


14/15

14

References

[1] Z. D. Kelemen, R. Kusters, and J. Trienekens,Identifying criteria formultimodel software process improvement solutions - based on a review of

current problems and initiatives,J. Softw. Evol. Process, vol. 24, no. 8,pp. 895909, Dec. 2012.

[2] Z. D. Kelemen, K. Balla, and G. Bka, Quality Organizer: a support toolin using multiple quality approaches, in Proceedings of 8th InternationalCarpathian Control Conference (ICCC 2007), Strbske Pleso, Slovakia,2007.

[3] Z. D. Kelemen, Process Based Unification for Multi-Model Software ProcessImprovement. Eindhoven: Technische Universiteit Eindhoven, 2013.

[4] Z. D. Kelemen, K. Balla, J. Trienekens, and R. Kusters, Structure ofProcess-Based Quality Approaches - Elements of a research developing acommon meta-model for proces-based quality approaches and methods, in

Proceedings of EuroSPI 2008 Doctoral Symposium, Dublin, Ireland, 2008.[5] Z. D. Kelemen, K. Balla, J. Trienekens, and R. Kusters, Towardssupporting simultaneous use of process-based quality approaches, inProceedings of 9th international Carpathian Control Conference: ICCC2008, Sinaia, Romania, 2008, pp. 291295.

[6] Z. D. Kelemen and K. Balla, A CMMI-DEV v1.2 s az ISO 9001:2000kapcsolata,Magy. Minsg, vol. 17, no. 2, pp. 2740, 2008.

[7] S. Jeners, H. Lichter, and E. Pyatkova, Automated Comparison of ProcessImprovement Reference Models Based on Similarity Metrics, 2012, pp.743748.

[8] S. Jeners and H. Lichter, Smart Integration of Process ImprovementReference Models Based on an Automated Comparison Approach, inSystems, Software and Services Process Improvement, vol. 364, F.McCaffery, R. OConnor, and R. Messnarz, Eds. Springer Berlin Heidelberg,2013, pp. 143154.

[9] X. Chen, M. Staples, and P. Bannerman, Analysis of Dependenciesbetween Specific Practices in CMMI Maturity Level 2, in Software ProcessImprovement, vol. 16, R. V. OConnor, N. Baddoo, K. Smolander, and R.Messnarz, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2008, pp.94105.

[10] A. L. Ferreira, R. J. Machado, and M. C. Paulk, Size and ComplexityAttributes for Multimodel Improvement Framework Taxonomy, inSoftware Engineering and Advanced Applications (SEAA), 2010 36thEUROMICRO Conference on, 2010, pp. 306 309.

[11] A. L. Ferreira, R. J. Machado, and M. C. Paulk, Quantitative Analysis of

Best Practices Models in the Software Domain, presented at the 2010 AsiaPacific Software Engineering Conference, 2010.

[12] Z. D. Kelemen, What is important in CMMI and what are theinterrelations among its elements?, presented at the International ODFSymposium, Budapest, 28-Jun-2011.

[13] P. Monteiro, R. J. Machado, R. Kazman, and C. Henriques, DependencyAnalysis between CMMI Process Areas, in Product-Focused SoftwareProcess Improvement, vol. 6156, M. Ali Babar, M. Vierimaa, and M. Oivo,Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2010, pp. 263275.


15/15

15

[14] K. Balla and Z. D. Kelemen, Important Concepts In CMMI and What isDifficult to Understand, presented at the SEPG Europe 2011, Dublin, 07-Jun-2011.

[15] C. S. CMMI Product Team, CMMI for Development, Version 1.3. CMUSEI, Nov-2010.

[16] C. S. CMMI Product Team, CMMI for Services, Version 1.3. CMU SEI,Nov-2010.

[17] C. S. CMMI Product Team, CMMI for Acquisition, Version 1.3. CMUSEI, Nov-2010.

[18] E. Forrester and G. Wemyss, CMMI Version 1.3 and Beyond, Dublin,Ireland, 2011.

[19] Z. D. Kelemen, R. Kusters, J. Trienekens, and K. Balla, A Data Model forMultimodel Process Improvement, Budapest, Technical Report TR201303,Sep. 2013.

[20] Z. D. Kelemen, R. Kusters, J. Trienekens, and K. Balla, Towards Applying

Text Mining Techniques on Software Quality Standards and Models,Budapest, Technical Report TR201302, Sep. 2013.

[21] Z. D. Kelemen, R. Kusters, J. Trienekens, and K. Balla, TowardsComplexity Analysis of Software Process Improvement Frameworks,Budapest, Technical Report TR201301, Sep. 2013.

[22] Z. D. Kelemen, R. Kusters, J. Trienekens, and K. Balla, Selecting a ProcessModeling Language for Unification of Multiple Standards and Models,Budapest, Technical Report TR201304, Sep. 2013.

towards complexity analysis of software process improvement frameworks

Documents