How does context affect the distribution of software maintainability metrics?

Feng Zhang, Audris Mockus, Ying Zou, Foutse Khomh,

and Ahmed E. Hassan

2

Software Metrics

Numerous Software

3

Various Usage of Software Metrics

4

Contexts !

Motivation

5

In Software Engineering Area?

6

What are the Contexts of Software?

Age (AG)

Number of Changes (NC)

Life Span (LS)

Number of Downloads (ND)

Application Domain (AD)

Programming Language (PL)

7

39 Software Maintainability Metrics

Complexity (14 metrics) Abstraction (5 metrics)

Coupling (8 metrics)

Cohesion (4 metrics)

Encapsulation (4 metrics)

Documentation (4 metrics)

8

Data Collection

56,833

824

9

Data Cleaning

618

506

478

390

320

824

10

31

26

23

29

49

19

16

41

29

14

13

7

7

9

7

Build Tools

Code Generators

Communications

Framework

Games / Entertainmaint

Internet

Network

Software Development

System Administrator

Build & CodeGen

Comm & Internet

Comm & Network

Games & Internet

Internet & SW Dev

SW Dev & Sys Admin

57

85

18

146

14

C

C++

C#

Java

Pascal

Data Description

320 Software Systems

11

Research Questions

12

Separately

RQ1. Analysis Methods

13

RQ1. Analysis Methods (cont’)

For example

C Java Pascal C++ C#

Metric 1

Metric 1

Metric 1

Metric 1

Metric 1

Metric n

Metric n

Metric n

Metric n

Metric n

Kruskal Wallis test

Kruskal Wallis test

14

Complexity (8/14 metrics) Abstraction (1/5 metrics)

Coupling (5/8 metrics)

Cohesion (2/4 metrics)

Encapsulation (1/4 metrics)

Documentation (3/4 metrics)

YES!! the Contexts Matter!

51 % of metrics are impacted by all Six

Contexts

15

and Among the Six Contexts …

at least 72 % of metrics are impacted by a Single Context

16

Does it mean ALL six contexts

should be considered all the time?

17

Research Question 2

18

RQ2. Analysis Methods

19

RQ2. Analysis Methods (cont’)

C Java Pascal C++ C#

Metric i

Metric i

Mann-Whitney U test

Metric i

Metric i

Mann-Whitney U test

Metric i

Metric i

Mann-Whitney U test

Metric i

Metric i

Mann-Whitney U test

Metric i

Metric i

Mann-Whitney U test

Metric i

Mann-Whitney U test Metric i

20

RQ2. Analysis Methods (cont’)

0.147 0.330 0.474 Cliff’s delta 14.7% 33.0% 47.4% % of non-overlap

Small Medium Large Cohen’s standard 0.20 0.50 0.80 Cohen’s d

21

RQ2. Findings for

each Category of Metrics

22

Metric AD PL AG LS NC ND

TLOC - - - - √ -

TNF - - - - √ -

TNC √ √ - - √ -

TNM √ √ - - √ -

TNS - - - - - -

CLOC - - - - - -

NOM - - - - - -

NIM - - - - - -

NIV - - - - - -

WMC - - - - - -

NMP - - - - - -

CC - - - - - -

NPATH - - - - - -

MNL - - - - - -

Contexts Impacting ‘Complexity’

AD: Application Domain PL : Programming Language NC: Number of Changes

23

Metric AD PL AG LS NC ND

CF - - - - √ -

CBO √ √ - - - -

ICP - - - - - -

MPC - - - - - -

RFC √ √ - - - -

NMI - √ - - - -

FANIN - - - - - -

FANOUT - - - - - -

Contexts Impacting ‘Coupling’

AD: Application Domain PL : Programming Language NC: Number of Changes

24

Metric AD PL AG LS NC ND

LCOM √ - - - - -

TCC - - - - - -

LCC - - - - - -

ICH - - - - - -

Contexts Impacting ‘Cohesion’

AD: Application Domain

25

Metric AD PL AG LS NC ND

NACI - √ - - - -

MIF - √ - - - -

IFANIN √ √ - - - -

NOC - - - - - -

DIT √ - - - - -

Contexts Impacting ‘Abstraction’

AD: Application Domain PL : Programming Language

26

Metric AD PL AG LS NC ND

RPA √ - - - - -

RPM - - - - - -

RSA - - - - - -

RSM √ - - - - -

Contexts Impacting ‘Encapsulation’

AD: Application Domain

27

Metric AD PL AG LS NC ND

CLC - - - - - -

RCCC √ √ - - - -

CLM - - - - - -

RCCM - - - - - -

Contexts Impacting ‘Documentation’

AD: Application Domain PL : Programming Language

28

Summary of RQ2 Findings

Metric Category AD PL AG LS NC ND

Complexity √ √ - - √ - Coupling √ √ - - √ -

Cohesion √ - - - - - Abstraction √ √ - - - -

Encapsulation √ - - - - - Documentation √ √ - - - -

AD: Application Domain PL : Programming Language NC: Number of Changes

29

Metric Category Context Groups

Complexity

AD (2) (Framework); and others

PL (3) (C); (Pascal); and others

NC (3) (Low NC;) (moderate NC); and (high NC)

Coupling

AD (3) (Communication, Network); (Build Tools, Code Generators;) and others

PL (3) (Pascal;) (Java;) and others

NC (3) (Low NC); (moderate NC); and (high NC)

Cohesion AD (2) (Communication, Network); and others

Abstraction

AD (4) (Communication, Network); (Games); (Build Tools, Code Generators);

and others

PL (3) (Java;) (C++); and others

Encapsulation AD (3) (Build Tools); (Communication, Network); and others

Documentation AD (2) (Build Tools, Code Generators); and others

PL (2) (Java); and others

Guidelines for Benchmarking Maintainability Metrics

30