1 07 september 2004 parastoo mohagheghi the impact of software reuse and incremental development on...

07 September 2004Parastoo Mohagheghi1

The Impact of Software Reuse and The Impact of Software Reuse and Incremental Development on the Quality Incremental Development on the Quality of Large Systems of Large Systems

Parastoo Mohagheghi, Parastoo Mohagheghi,

Dept. Computer and Information Science (IDI),Dept. Computer and Information Science (IDI),

University of Science and Technology (NTNU),University of Science and Technology (NTNU),

TrondheimTrondheim

[email protected]@idi.ntnu.no


What is this presentation about?What is this presentation about?

• It presents results of a series of empirical studies It presents results of a series of empirical studies at Ericsson in Grimstad in 2001-2004.at Ericsson in Grimstad in 2001-2004.

• The studies are performed during a doctoral work The studies are performed during a doctoral work in the INCO project and supervised by Prof. in the INCO project and supervised by Prof. Reidar Conradi.Reidar Conradi.

• 11 students were involved in different studies, an 11 students were involved in different studies, an example of university-industry cooperation. example of university-industry cooperation.

• For more details, see the PhD thesis at: For more details, see the PhD thesis at: http://www.idi.ntnu.no/grupper/su/publ/phd/mohahttp://www.idi.ntnu.no/grupper/su/publ/phd/mohagheghi-thesis-10jul04-final.pdfgheghi-thesis-10jul04-final.pdf


MotivationMotivation

• Systematic software reuseSystematic software reuse, , product family product family developmentdevelopment and and incremental developmentincremental development seem seem be potent technologies to achieve benefits in be potent technologies to achieve benefits in productivity, quality and maintainability, and to productivity, quality and maintainability, and to reduce risks of changes.reduce risks of changes.

Quantifiable benefits,Quantifiable benefits,but few empirical studies in industry but few empirical studies in industry

that can verify these benefits that can verify these benefits or possible other impacts. or possible other impacts.


Company backgroundCompany background

• Ericsson has developed two large-scale telecom Ericsson has developed two large-scale telecom systems (470 KSLOC or 1000 KSLOC in systems (470 KSLOC or 1000 KSLOC in equivalent C each) that shareequivalent C each) that share– Software architecture,Software architecture,– Components and a component model,Components and a component model,– Development environment,Development environment,– Software process.Software process.

• Systems are developed incrementally, and lots of Systems are developed incrementally, and lots of data is gathered during 5 years on defects, data is gathered during 5 years on defects, changes, characteristics etc.changes, characteristics etc.


Overview of the Ericsson packet-Overview of the Ericsson packet-switched core network- GPRSswitched core network- GPRS

Backbonenetwork

SGSN GGSN

BGW

IP network

Other networks

MSC/VLR HLR

SMS-GMSCSMS-IWMSC EIR

OtherSGSN

BSC/RNC

Packet-switchedcore network


Evolution of the GSN software architecture Evolution of the GSN software architecture and the software process model and the software process model

Wireless Packet Platform (WPP)

4

1 3

65

2

Rev. & Re-eng.

Wireless Packet Platform (WPP)

GPRS for GSMGPRS for GSM

3’

6’

5’

1’

2

4

7

8

The original architecture The hierarchical architecture

GPRS for GSM

GPRS for WCDMA

Adaptation of RUP

Initial Process Adapted RUP

Reused

Application-specificlayer

Business-specificlayer

Common servicesLayer (includingcomponent framework)


The GSN RUPThe GSN RUP


Research questionsResearch questions

• RQ1.RQ1. Why a reuse program is initiated and how is Why a reuse program is initiated and how is it implemented?it implemented?

• RQ2.RQ2. What is the impact of software reuse, What is the impact of software reuse, Component-Based Development (CBD) and Component-Based Development (CBD) and incremental development on the quality? The incremental development on the quality? The impact of development approaches on product impact of development approaches on product quality metrics and on project attributes such as quality metrics and on project attributes such as schedule or effort are sought.schedule or effort are sought.

• RQ3.RQ3. How to improve the practice of incremental How to improve the practice of incremental development of product families in some aspects? development of product families in some aspects?


Research methodsResearch methods

• QuantitativeQuantitative studies by exploring (‘data mining’) company studies by exploring (‘data mining’) company databases and assessing hypotheses.databases and assessing hypotheses.

• Quantitative results are discussed with experienced Quantitative results are discussed with experienced personnel, and combined with personnel, and combined with qualitativequalitative feedbacks and feedbacks and studies of the process and practice to assess validity of the studies of the process and practice to assess validity of the results.results.

• Several case studies, a small survey and an experiment.Several case studies, a small survey and an experiment.• Combining results of several studies in the interpretation Combining results of several studies in the interpretation

phase:phase:– The impact of introducing reuse or incremental development is The impact of introducing reuse or incremental development is

widespread.widespread.– Taking benefit of all available data,Taking benefit of all available data,– Confirming results by other studies/data. Confirming results by other studies/data.


Qualitative study of RUP in the context of Qualitative study of RUP in the context of reusereuse

• The approach to initiating a product family was an The approach to initiating a product family was an extractive one. extractive one.

• Software architecture has evolved to support reuse, while Software architecture has evolved to support reuse, while GSN RUP has not been adapted for reuse to the same GSN RUP has not been adapted for reuse to the same degree. degree.

• Examples of proposals:Examples of proposals:– ““Domain analysis” and “Make vs. Reuse vs. Buy” decisions in the Domain analysis” and “Make vs. Reuse vs. Buy” decisions in the

inception phase.inception phase.

– ““Update reuse documentation” for reusable components.Update reuse documentation” for reusable components.

– ““Record reuse experience” in the conclusion phase.Record reuse experience” in the conclusion phase.


Quantitative study of Trouble Reports Quantitative study of Trouble Reports (TRs)(TRs)

• For defects detected during integration testing, system For defects detected during integration testing, system testing, or later in maintenance.testing, or later in maintenance.

• For defects detected in previous releases.For defects detected in previous releases.

• For all types of defects (software, hardware, toolbox, and For all types of defects (software, hardware, toolbox, and documentation).documentation).

• We have ‘data mined’ databases:We have ‘data mined’ databases:– 13000 TRs for several systems and releases were parsed, data was 13000 TRs for several systems and releases were parsed, data was

extracted, and inserted in a SQL database by a C# program. extracted, and inserted in a SQL database by a C# program.

– Inconsistencies were resolved -> Data was not analyzed.Inconsistencies were resolved -> Data was not analyzed.

• The company provided us excel sheets on software size The company provided us excel sheets on software size (software module level). (software module level).


Hypotheses: Software Reuse and Quality Hypotheses: Software Reuse and Quality (Defect-density and Stability)(Defect-density and Stability)

H01H01 Reused components have the same Reused components have the same defect-density as non-reused ones.defect-density as non-reused ones.

RejectedRejected

H02H02 There is no relation between number There is no relation between number of defects and component size for of defects and component size for all/reused/non-reused components.all/reused/non-reused components.

Not-rejectedNot-rejected


RejectedRejected

H03H03 There is no relation between defect-There is no relation between defect-density and component size for density and component size for all/reused/non-reused components.all/reused/non-reused components.




H04H04 Reused and non-reused components Reused and non-reused components are equally modified.are equally modified.

RejectedRejected


H1: Reuse and defect-DensityH1: Reuse and defect-Density, Release 3 , Release 3 and blocksand blocks

Defect-density of blocks Mean Median Variance

#TRs/KLOC, Reused 1.32 0.76 1.70

#TRs/KLOC, Non-Reused 3.01 2.44 4.39

#TRs/MKLOC, Reused 3.50 1.78 21.26

#TRs/MKLOC, Non-Reused

5.69 3.73 21.76

•Reused components have lower defect density.Reused components have lower defect density.•Reused components have proportionally more Reused components have proportionally more defects with Severity A (highest severity).defects with Severity A (highest severity).•Reused components have proportionally less defectsReused components have proportionally less defectsafter delivery. after delivery.


H2: Size vs. the number of Defects

R2 = 0,7213

R2 = 0,573R2 = 0,5876

R2 = 0,7698

0

20

40

60

80

100

120

140

160

180

0 5000 10000 15000 20000 25000 30000 35000 40000 45000

LOC

#TR

Reused

Non-reused Blocks

Linear (Non-reused Blocks)

Linear (Reused)

Poly. (Reused)

Poly. (Non-reused Blocks)


H3: Size vs. Defect-DensityH3: Size vs. Defect-Density

0

1

2

3

4

5

6

7

8

0 5 10 15 20 25 30 35 40 45

KLOC

#TRs/KLOC

Reused

Non-reused


DiscussionDiscussion• Reuse benefits: Reused components are less defect-prone and Reuse benefits: Reused components are less defect-prone and

more stable. more stable. • Why? Why? Reused components are designed more thoroughly, and Reused components are designed more thoroughly, and

changed with carechanged with care. . – Confounding factors (conclusion validity): Confounding factors (conclusion validity):

• Programming language. Not applicable. Programming language. Not applicable. • Type of functionality. Not applicable for stability, but may be for Type of functionality. Not applicable for stability, but may be for

defect-density.defect-density.• Developers’ experience: Not applicable. Developers’ experience: Not applicable.

• Internal validity: Missing data, but not systematic.Internal validity: Missing data, but not systematic.• Construct validity: Are defect-density and stability indicators of Construct validity: Are defect-density and stability indicators of

quality?quality?• External validity: At least inside the company for similar systems, External validity: At least inside the company for similar systems,

or in the domain.or in the domain.


ContributionsContributions

• Research:Research:– Empirical verification of reuse benefits. No other Empirical verification of reuse benefits. No other

studies on large industrial systems.studies on large industrial systems.– Combined with other studies in the company to assess Combined with other studies in the company to assess

development approaches and identifying metrics.development approaches and identifying metrics.

• Company:Company:– Evaluating the trouble reporting system.Evaluating the trouble reporting system.– Evaluating the measurement program: Granularity of Evaluating the measurement program: Granularity of

component definition and data for incremental component definition and data for incremental development.development.

– Identifying defect-prone and unstable components.Identifying defect-prone and unstable components.


Software evolution and incremental Software evolution and incremental developmentdevelopment

• With incremental development, software changes:With incremental development, software changes:– during each release (during each release (developmentdevelopment))– between successive releases (between successive releases (evolutionevolution) ) – and after delivery (and after delivery (maintenancemaintenance).).

• Some changes are pre-planned (incremental development) Some changes are pre-planned (incremental development) and some are not (iterative improvement, adaptive changes and some are not (iterative improvement, adaptive changes etc.). etc.).

• The IEEE Standard 1219 on software maintenance defines The IEEE Standard 1219 on software maintenance defines software maintenance as “The modification of a software software maintenance as “The modification of a software product after delivery to correct faults (i.e. product after delivery to correct faults (i.e. correctivecorrective), to ), to improve performance or other attributes (i.e. improve performance or other attributes (i.e. perfective/preventiveperfective/preventive), or to adapt the product to a ), or to adapt the product to a modified environment (modified environment (adaptiveadaptive)”.)”.


Quantitative study of Change Requests Quantitative study of Change Requests (CRs)(CRs)

• Ericsson defines requirements for each release in an ARS Ericsson defines requirements for each release in an ARS (Application Requirement Specification). (Application Requirement Specification).

• Changes to ARS or previous deliveries (releases) are Changes to ARS or previous deliveries (releases) are handled by issuing Change Requests (CRs).handled by issuing Change Requests (CRs).– CRs handle coarse-grained changes to add/delete functionality, CRs handle coarse-grained changes to add/delete functionality,

improve performance or improve design.improve performance or improve design.– CRs cover changes during developing each release as well as CRs cover changes during developing each release as well as

between releases (in addition to new requirements stated in the between releases (in addition to new requirements stated in the ARS). ARS).

• How this data can be used in our research?How this data can be used in our research?– 169 CRs for 4 releases of one system were analyzed.169 CRs for 4 releases of one system were analyzed.– Exploratory study: Origin (internal vs. external), acceptance rate Exploratory study: Origin (internal vs. external), acceptance rate

and classes of changes are studied.and classes of changes are studied.


Origin of CRsOrigin of CRs

Perfective/Perfective/

FunctionalFunctional

Perfective/Perfective/

Quality Quality Attributes Attributes (QA)(QA)

AdaptiveAdaptive PreventivePreventive Other Other (cost)(cost)

NoNo 4040 7474 3535 3030 88

•18 CRs have indicated two reasons.•Most perfective CRs are to improve QA (also preventive CRs improve quality).•23 of 169 CRs are issued because of customer demands. These and adaptive CRs have external origin (34% of all CRs).


Acceptance Rate over ReleasesAcceptance Rate over Releases

0 %

20 %

40 %

60 %

80 %

100 %

Release 1 Release 2 Release 3 Release 4

Rejected CRs (%)

Accepted CRs (%)

Acceptance rate is increasing (59% in total); i.e. the

product is getting more change-prone.


ConclusionsConclusions

• Most changes originate from the project Most changes originate from the project organization itself in order to improve quality and organization itself in order to improve quality and enhance functionality. The share of the first group enhance functionality. The share of the first group is higher. is higher.

• The practice indicates that the iterative realization The practice indicates that the iterative realization and improvement of quality attributes has great and improvement of quality attributes has great impact on the evolution of the products.impact on the evolution of the products.

• We could not observe any significant difference We could not observe any significant difference between reused and non-reused components in the between reused and non-reused components in the number of CRs per KLOC. number of CRs per KLOC.


Conclusions (continued)Conclusions (continued)

• Most CRs are accepted and the acceptance rate Most CRs are accepted and the acceptance rate can have impact on the project plans in terms of can have impact on the project plans in terms of decreasing planning precision (observed). decreasing planning precision (observed). Incremental development opens for (more) Incremental development opens for (more) changes.changes.

• Most CRs are issued pre-delivery, and especially Most CRs are issued pre-delivery, and especially in the short time right after requirement baseline. in the short time right after requirement baseline. The organization possibly takes the decision to The organization possibly takes the decision to baseline requirements too early. baseline requirements too early.


Overview of all studiesOverview of all studies

Study of Software Process & RUP

2001-2002

P9

P6

P2Survey

2002

Study ofReuse Practice

2002

P1

P7

Experiment onInspection

2002P4

Study of MDA2003

Prototype

P3

Study ofTrouble Reports

2003

P8

Study ofChange Requests

2003

P10

Study ofEffort

2003-2004

P12

Phase 1 Phase 2

Developing EffortEstimation Method

2003-2004P13

Developing Data Mining Method

2004

P11

Assessing the Impact ofDev. Approaches & Identifying Metrics

2003-2004

P11

P5

Combining results in Phase 3

July 2004March 2001

Quantitative studyQualitative study

P Paper Input

Contribution

C6b

C6a

C1

C3

C1 C2

C5

C4

C3

C


Impact of dev. approaches and contextImpact of dev. approaches and context

ReuseReuse Incr. dev.Incr. dev. CBDCBD Large Large systemssystems

Software Software evolutionevolution

Change rateChange rate

Earlier rel. Earlier rel. not evolvednot evolved

GranularityGranularity Improved Improved incr.incr.

Long-livedLong-lived

Effort Effort estimationestimation

ROI?ROI? Software Software from a from a previous rel.,previous rel.,

Incr. changes Incr. changes in in requirementsrequirements, Large CM, Large CM

No data for No data for componentscomponents

Large CM Large CM and testing and testing efforteffort

MetricsMetrics yesyes yesyes yesyes Different Different granularitygranularity


Mining industrial databasesMining industrial databases

Theory orHypothesis

Pre-StudyData

Samplingyes/no

ExploreData

ModifyData

Assess

ResearchQuestion

LiteratureSearch

Select/DefineTheory or Hyp.

PackageResults

confirmatoryexploratory

preparation

execution

conclusion

theoretical

Model

Select Methods &

tools

Top-down theory search integrated with bottom-up data exploring.


Assessing development approachesAssessing development approaches

Requirement Modification Planning PrecisionIncremental & Iterative

Development

Reusable Artifacts/Components

Development Approaches Practices Product/process Quality

Needed Effort

Reduced Defect-Density

Changeability

Incremental DeliverySuccess

Incremental Integration

Incremental Planning

Solution Modification

Software Reuse

Component-BasedDevelopment

Component Stability

Leads to

+/-

+/-

+

+

+

Development approaches should be seen as coherent systems of practices, be chosen depending on the attribute that should be optimized and may trade-off for other practices [IEEE software, MacCormack, 2003 ]


Main ContributionsMain Contributions

• Empirical verification of reuse benefits.Empirical verification of reuse benefits.• Increased understanding of the origin and type of Increased understanding of the origin and type of

changes in each release and between releases.changes in each release and between releases.• Identifying metrics for a combination of reuse and Identifying metrics for a combination of reuse and

incremental development.incremental development.• A data mining method for exploring industrial A data mining method for exploring industrial

databases.databases.• An effort estimation method for incremental An effort estimation method for incremental

development.development.


DiscussionDiscussion

• Studies are performed in a real context.Studies are performed in a real context.– Context-independent knowledge is rare in SE and generalization is Context-independent knowledge is rare in SE and generalization is

difficult.difficult.

• Studies combine different methods and data Studies combine different methods and data (triangulation).(triangulation).– Integration of the results is a challenge: Integration of the results is a challenge: physicalphysical and and conceptualconceptual

integration challenges.integration challenges.

• Incremental development is combined with reuse of Incremental development is combined with reuse of software in multiple products and CBD.software in multiple products and CBD.– All aspects of software development need adaptation; e.g. All aspects of software development need adaptation; e.g.

estimation method, software process model, data collection estimation method, software process model, data collection routines etc.routines etc.


Discussion (continued)Discussion (continued)

• Own experience and access to internal dataOwn experience and access to internal data– Confidentiality of some results and other ethical issues should be Confidentiality of some results and other ethical issues should be

respected.respected.

• Some studies are evaluation of others’ observations or Some studies are evaluation of others’ observations or methods in a new context (e.g. estimation method, reuse methods in a new context (e.g. estimation method, reuse benefits or distribution of maintenance categories), while benefits or distribution of maintenance categories), while others contain new observations and methods.others contain new observations and methods.

• Being exposed to organizational changes leaded to an Being exposed to organizational changes leaded to an emerging research design and changes in plans.emerging research design and changes in plans.

• Working in the field needs flexibility and a suitably time-Working in the field needs flexibility and a suitably time-frame.frame.


Why such studies are important for Why such studies are important for industry?industry?

• Methods should be tested for development in Methods should be tested for development in large.large.

• Companies are increasingly using mainstream Companies are increasingly using mainstream methods, tools and programming languages.methods, tools and programming languages.

• The studies analyzed data that was hardly studied The studies analyzed data that was hardly studied before: evaluation of data collection routines and before: evaluation of data collection routines and metrics for the company, in addition to concrete metrics for the company, in addition to concrete results.results.


AcknowledgementsAcknowledgements

• Thanks to INCO, Simula and NTNU for financial Thanks to INCO, Simula and NTNU for financial support.support.

• Thanks to my supervisor, Prof. Reidar Conradi, Thanks to my supervisor, Prof. Reidar Conradi, and colleagues at INCO and IDI.and colleagues at INCO and IDI.

• Thanks to Ericsson for the permission to perform Thanks to Ericsson for the permission to perform studies and to publish the results.studies and to publish the results.

• Thanks to the SPIKE seminar and the audience.Thanks to the SPIKE seminar and the audience.

1 07 september 2004 parastoo mohagheghi the impact of software reuse and incremental development on...

Documents

impact of software reuse

reuse program

support reuse

allreusednonreused components

types of defects software

reusable components

blocksreused components

stabilityh01reused components