1 parastoo mohagheghi- 21 sept.2004 the impact of software reuse and incremental development on the...

Parastoo Mohagheghi- 21 Sept.20041

The Impact of Software Reuse and Incremental Development on the Quality

of Large Systems

Parastoo Mohagheghi

Dept. Computer and Information Science (IDI)

University of Science and Technology (NTNU)

Trondheim, Norway

[email protected]

Doctoral thesis presentation, 21 September 2004


Motivation of the study• Software organizations (and researchers) have always been

looking for effective strategies to develop software faster, better and cheaper: – OO analysis, CASE tools, formal methods, software reuse, agile

methods etc. are proposed, but there is no “silver bullet”.– There is an absence of models and theories that can help to reason

about the software without building it.– Rapid evolution of technologies does not allow proper assessment.– Collecting context-independent knowledge or representing context

is difficult.– Software development is inherently complex, especially for large

systems.• There is a lack of empirical studies in industry on

technologies that can answer what they are good at, and not so good at.

• The INCO- INcremental and COmponent-based Software Development project (2001 to 2004) tries to advance the state-of-the-art and practice for these technologies.


Definitions• “Software reuse is the systematic practice of developing

software from a stock of building blocks..” [Morisio,2002]. – Reusable assets can take many forms: Component libraries, COTS

and OSS components, or entire software architectures and components in a product family approach.

– A reusable component in this study is a component that is reused in more than one product.

– Two distinct activities: development for and with reuse. • Component-Based Development (CDB) covers all the

aspects of developing (reusable, executable?) software components and assembling system from them. Components adhere to a component model.

• Incremental development is covering (or discovering) requirements gradually and developing systems in increments that accumulate functionality. Iterative development is recursive application of development activities and recursive elaboration of artifacts.


Company background

• Ericsson has developed two large-scale telecom systems (470 KSLOC or 1000 KSLOC in equivalent C each) that share software architecture, components and a component model, development environment and software process.

• Systems are developed incrementally (5 releases of one system and 2 of the other). In periods, over 200 developers in different countries were working with design and test.

• Lots of data has been gathered in 5 years in different repositories. Data from several releases are analyzed in this study. It covers trouble reports, change requests, effort, size of components, measures from internal measurement programs and inspections.

• Personnel turnover has traditionally been small, but software development in Norway stopped in mid-2002.


Overview of the Ericsson packet-switched core network- GPRS

Backbonenetwork

SGSN GGSN

BGW

IP network

Other networks

MSC/VLR HLR

SMS-GMSCSMS-IWMSC EIR

OtherSGSN

BSC/RNC

Packet-switchedcore network


Evolution of the GSN software architecture

Wireless Packet Platform (WPP)

4

1 3

65

2

Rev. & Re-eng.

Wireless Packet Platform (WPP)

GPRS for GSMGPRS for GSM

3’

6’

5’

1’

2

4

7

8

The original architecture The hierarchical architecture

GPRS for GSM

GPRS for WCDMA

Reused

Application-specificlayer

Business-specificlayer

Common servicesLayer (includingcomponent framework)

Adaptation of RUP

Initial Process Adapted RUP

and the software process model


Characteristics of the system

• The approach to initiate a product family has been an extractive one and involved high risks regarding cost, quality, training, coordination and management support.

• Quality requirements: Performance, availability, scalability and maintainability (or evolvability) are of great importance. There are no direct user interfaces.

• Large systems face challenges in all phases of development that small systems do not: difficulties in iteration planning, complexity of design, integration and test and maintenance costs. Large systems are designed to be long-lived.

• Components are developed in-house by different Ericsson organizations.


Research questions

• RQ1. Why is a reuse program initiated and how is it implemented?

• RQ2. What is the impact of software reuse, CBD and incremental development on the quality? The impact of development approaches on product quality metrics and on project attributes such as schedule or effort are sought.

• RQ3. How to improve the practice of incremental development of product families in some aspects?

• Research questions are further refined in studies. – Some research questions and hypotheses were identified from

earlier work (top-down style) -> replication in new context and confirmatory studies.

– Other questions and hypotheses are results of exploratory work on available data and practices in the industry, in a bottom-up style -> explorative and descriptive studies.


Research methods

• Quantitative studies by exploring (‘data mining’) company databases, studying distributions and assessing hypotheses.

• Qualitative data collected from web pages, textual documents, discussions with personnel and own experience on the process and practices are combined with results from quantitative studies.

• Several case studies (post-mortem), a small survey and a controlled experiment are performed.

• The rationale for mix of methods has been:– The impact of introducing reuse or incremental development is

widespread.– It is useful to take benefit of all available data.– Triangulation of data/methods.– Answering questions that are not possible to answer otherwise.


Overview of all studies

Study of Software Process & RUP

2001-2002

P9

P6

P2Survey

2002

Study ofReuse Practice

2002

P1

P7

Experiment onInspection

2002P4

Study of MDA2003

Prototype

P3

Study ofTrouble Reports

2003

P8

Study ofChange Requests

2003

P10

Study ofEffort

2003-2004

P12

Phase 1 Phase 2

Developing EffortEstimation Method

2003-2004P13

Developing Data Mining Method

2004

P11

Assessing the Impact ofDev. Approaches & Identifying Metrics

2003-2004

P11

P5

Combining results in Phase 3

July 2004March 2001

Quantitative studyQualitative study

P Paper Input

Contribution

C6b

C6a

C1

C3

C1 C2

C5

C4

C3

C


Contributions and research questionsGroup Contribution RQ1 RQ2 RQ3

Reuse C1. Empirical verification of reuse benefits. √

Incremental dev.

C2. Increased understanding of the origin and type of changes.

√

Incremental dev.

C3. Developing an effort estimation method √

Both C4. Identifying metrics for a combination of

reuse and incremental development. √

Method C5. Developing a data mining method. √

Reuse C6a. Adaptation of the Rational Unified Process for reuse.

√ √

Incremental dev.

C6b. Improving techniques for incremental inspection of UML models.

√


Contributions in software reuse

• C1. Empirical verification of reuse benefits (RQ2):– Quantitative studies of Trouble Reports (TRs) and Change

Requests (CRs). 13 000 TRs and 169 CRs were analyzed.– Reused components have significantly lower defect-density than

non-reused ones, and are more stable between releases (less modified). Data from 3 releases.

– There is no difference in change-proneness between reused and non-reused components (#CRs/KLOC). Data from 4 releases.

– First large-scale industrial study on reuse benefits.• C6a. Adaptation of the Rational Unified Process (RUP)

regarding reuse (RQ1, RQ3):– Proposing changes in workflows and activities for development

for and with reuse. – No empirical studies have been found on evaluating or adapting

RUP in this aspect. Proposals are generic and may be reused in other adaptations of RUP.


Reuse and defect-density, release 3

Defect-density of blocks Mean Median Variance

#TRs/KLOC, Reused 1.32 0.76 1.70

#TRs/KLOC, Non-Reused 3.01 2.44 4.39

#TRs/MKLOC, Reused 3.50 1.78 21.26

#TRs/MKLOC, Non-Reused

5.69 3.73 21.76

•H0: Reused components have the same defect-density as non-reused ones. Rejected.•Reused components have lower defect density, but they have proportionally more defects with severity A (highest severity).•Reused components have proportionally less defectsafter delivery -> more reliable.


Size vs. defect-density

0

1

2

3

4

5

6

7

8

0 5 10 15 20 25 30 35 40 45

KLOC

#TRs/KLOC

Reused

Non-reused

H0: There is no relation between defect-density and component

size for all/reused/non-reused components. Not rejected.


Contributions in incremental development

• C2. Increased understanding of the origin and type of changes in requirements or artifacts in incremental development (RQ2):– A quantitative analysis of CRs showed that perfective changes

(enhancements and optimizations) are most common. Of these, non-functional changes to improve “quality attributes” are more frequent than pure functional changes.

– Most CRs (66%) are issued by the organization itself (and not external actors). Only 34% of CRS are adaptive or customer-initiated.

– Hypotheses are grounded in data. Generalization needs further study.

Perfective/

Functional

Perfective/

Quality Attr.

Adaptive Preventive Other (cost)

CRs 40 74 35 30 8


Contributions in incr. dev.- cont.• C3. Developing an effort estimation method using use

case specifications and effort distribution in different phases of incremental software development (RQ3):– The method was adapted for incremental changes in (large) use

cases. It also includes a factor to estimate effort for building on a previous release.

– The impact of effort-breakdown profile on the estimation results is discussed.

– Method adapted to the context and scaled up.• C6b. Improving techniques for incremental inspection of

UML models (RQ3):– A small experiment (first in industry) was performed, comparing

company’s inspection technique with the new and adjusted OORTs (Object-Oriented Reading Techniques). The results showed no difference in cost-efficiency, but difference in the types of detected defects.

– Useful for improvement of OORTs and adapting these to the context.


Contributions in software reuse and incr. dev., and research methods

• C4. Identifying metrics for a combination of reuse of software components and incremental development (RQ3):– Other literature discusses metrics for CBD. However, these

metrics should be adapted for incremental development and for reuse.

• C5. A data mining method for exploring industrial data repositories (RQ3):– A data mining method is developed that is based on experience

from the quantitative studies. It combines top-down theory search with bottom-up hypotheses generation.

– Challenges in integrating the results of several studies with one another are classified into physical and conceptual ones.


Assessing development approaches

Requirement Modification Planning PrecisionIncremental & Iterative

Development

Reusable Artifacts/Components

Development Approaches Practices Product/process Quality

Needed Effort

Reduced Defect-Density

Changeability

Incremental DeliverySuccess

Incremental Integration

Incremental Planning

Solution Modification

Software Reuse

Component-BasedDevelopment

Component Stability

Leads to

+/-

+/-

+

+

+


The Impact of development approaches and context on practices

Reuse Incremental dev. CBD Large systems

Software evolution

Change-proneness and evolvability.

Increased change rate,

Earlier releases not evolved.

Component granularity is large for CRs.

QA are improved incrementally.

Long-lived.

Effort estimation

ROI? No metrics.

Software from a previous release is evolved.

Incr. changes in requirements, Large CM.

No data for effort spent on components.

Large CM and testing effort.

Data collection

Data should be collected for increments.

Different granularity.

No common repository.


Answers to research questions• RQ1. Why a reuse program is initiated and how is it

implemented? – A product family is initiated because of the similarity between

requirements of the emerging system and an existing system. Software architecture is evolved.

– A lightweight or extractive approach to product family adoption was chosen. Management support, common goals and common infrastructure, and experienced personnel have been critical for the success of reuse. The software process model is not adapted to the same degree.

• RQ2. What is the impact of software reuse, CBD and incremental development on the quality? – Lower defect density and higher stability of reused components.– Product is getting more change-prone (in %of accepted CRs and

reduced requirement stability). Quality is improved incrementally, and functionality is both enhanced and improved. Earlier releases are not evolved.

– High share of integration and testing effort.


Answers to research questions- cont.

• RQ3. How to combine the qualitative and quantitative results to improve the practice in some aspects? – The effort estimation method, metrics for a combination of

development approaches, the research method for data mining, and proposals for adapting RUP.

• Relation to INCO goals:– G1. Advancing the state-of-the-art of software engineering. Better

understanding of approaches is achieved. Contributions C1, C2 and C4.

– G2. Advancing the state-of-the-practice in software-intensive industry and for own students. Some feedback is given to Ericsson. Contributions C3 to C6 are reusable.

– G4. Disseminating and exchanging the knowledge gained. Most results are published. 11 students were involved in the studies. Teaching in courses.


Discussion• Case study research:

– Studies are performed in a real context. Context-independent knowledge is rare in SE and generalization is difficult.

– Affiliation in the company provided access to data (revelatory case).

– The system is a critical case for verifying reuse benefits (extractive approach, only two products and high risks).

– The size of the system allows evaluating techniques for large-scale development.

– In some cases, an example of industrial practice, e.g. RUP.

• Incremental development is combined with reuse of software in multiple products and CBD.– All aspects of software development need adaptation; e.g.

estimation method, software process model, data collection routines etc.


Conclusions

1. Describing different aspects of software development; i.e. the power of example.

– The work also identified areas for improvement like the process model and the data collection routines.

2. Verifying existing theories and assessing existing methods in new contexts; i.e. the power of replication:

– Reuse benefits in a large industrial system, – Estimation method and inspection techniques in a

large project and with incremental development, – RUP in the context of reuse.


Conclusions- cont.

• Generating new theories, hypotheses or methods by analyzing data from new perspectives (as in grounded theory) or combining the results of several studies; i.e. the power of generalization:

– The origin of change requests and the distribution over functional and quality attributes,

– The distribution of effort over development phases for a large project with incremental development,

– The impact of development approaches on quality attributes,

– Metrics for a combination of development approaches,– A data mining method based on experience.


Why such studies are important for research community and industry?

• Methods should be tested in real context and for development in large.

• Companies are increasingly using mainstream methods, tools and programming languages. Results are of interest to a larger community.

• The studies analyzed data that was hardly studied before: evaluation of data collection routines and metrics for the company, in addition to concrete results.


Future work

• Study of RUP adaptations.• Empirical studies on software evolution. • Study of effort distribution. • Estimation of incremental projects• Use cases for effort estimation.• Relevant metrics for incremental development of

large systems.


Acknowledgements

• Thanks to INCO, Simula and NTNU for financial support.

• Thanks to my supervisor, Prof. Reidar Conradi, and colleagues at INCO and IDI.

• Thanks to Ericsson for the permission to perform studies and to publish the results.

1 parastoo mohagheghi- 21 sept.2004 the impact of software reuse and incremental development on the...

Documents