[ieee 2013 35th international conference on software engineering (icse) - san francisco, ca, usa...

4
Understanding and Simulating Software Evolution Zhongpeng Lin Computer Science Department University of California, Santa Cruz Santa Cruz, CA 95064, USA Abstract—Simulations have been used in various areas, yield- ing good results, but their application to software evolution is still limited. Simulations of software evolution can help people understand the driving forces that shape software evolution, and predict future evolutionary paths. To move towards simulation of software evolution, this research tries to explore possible models to simulate software evolution, and the applicability of different data to parameterize the models. The simulations will both be based on fine-grained code changes obtained by comparing the abstract syntax trees of source code. The use of fine-grain code changes could reveal information about software evolution that is unavailable by other means. I. I NTRODUCTION Simulations are widely used in many areas to model natural systems, human systems or technologies for various purposes [1]. They are often used to show the eventual effects of alter- native conditions and courses of action, e. g., simulating urban development in response to different government policies [2]. Simulations can also be used when the actual system is not easily accessible, such as durability simulators in IKEA stores [3]. Software evolution is somewhat similar to urban develop- ment in that they both involve a group of people constructing and maintaining useful artifacts (a city or a software system). The design of a city or software system happens continuously, and may change over time. This is very different from the production of many other artifacts (e.g., printers, cars), which are fully designed up-front and put into production according to a static design. Furthermore, design decisions (e.g., zoning policy in a city, or architecture of a software system) may regulate the construction of a city or implementation of a software system, but such control is not deterministic, i.e. there are many possible of outcomes given a design; some of them may even violate the design to certain degrees. In addition, both cities and software can decay (this is called urban decay [4] and code decay [5] respectively). However, some properties of software make software evolu- tion substantially different from urban development. In urban development, many problems can be seen (e.g. building a house in the middle of a street), while in software evolution, problems are not so obvious (e.g. misplacing a method in a class). Also, software is more changeable than cities. For example, moving a method to a class it should belong to is not hard in most cases 1 , but relocating a house from a commercial area to a residential area is costly and may result in legal 1 Although it might be hard when the method is part of a published API. or political issues if the owner is unwilling to move. This fact makes software potentially experience more frequent and extensive changes than cities. Simulations of software evolution could potentially help decide between different design options. There are many software design techniques that are used to structure software. To choose between those design techniques, people need to know the possible outcomes of each. However, outcomes of applying a design to a piece of software are not obvious without simulations. Additionally, because software designs will decay without adequate refactoring [6], software evolution simulations may also help people understand software decay and the benefits of refactoring, by simulating the original and refactored versions of the software. However, few papers have been published on simulating software evolution. The proposed research agenda will examine the changes made to different software systems over years of development and maintenance at the source code level, and based on these observations, simulate those changes. The code changes to be considered are semantic changes extracted by differencing abstract syntax trees (ASTs) [7]. Semantic changes can be simulated by manipulating different parts of an AST according to certain probabilities. For example, simulating the evolution of a Factory class may add new method nodes to the AST more often than renaming existing method nodes, resulting an increase in the number of methods. Simulations of semantic changes will produce plausible future source code that allows analysis using a potentially large number of metrics that can be computed from source code and its changes. This research tries to address questions crucial to simulate software evolution. The first question is about the simulation model: RQ1. How should source code changes be modeled in order to simulate software evolution? How far into the future can simulations remain plausible enough to be useful? A variety of different stochastic models should be tested to simulate the evolution process. Because it is a complex process, accurately simulating code evolution almost certainly requires a complex model. An ideal model should simulate software evolution early in a project, and predict accurately far into the future. However, achieving these properties is a challenge. The proposed research would explore to what extent it is possible to achieve these goals. Simulation models need to be parameterized by change patterns from empirical data. The next two research question explore the applicability of data from history and peers re- 978-1-4673-3076-3/13/$31.00 c 2013 IEEE ICSE 2013, San Francisco, CA, USA Doctoral Symposium 1411

Upload: zhongpeng

Post on 30-Mar-2017

214 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: [IEEE 2013 35th International Conference on Software Engineering (ICSE) - San Francisco, CA, USA (2013.05.18-2013.05.26)] 2013 35th International Conference on Software Engineering

Understanding and Simulating Software EvolutionZhongpeng Lin

Computer Science DepartmentUniversity of California, Santa Cruz

Santa Cruz, CA 95064, USA

Abstract—Simulations have been used in various areas, yield-ing good results, but their application to software evolution isstill limited. Simulations of software evolution can help peopleunderstand the driving forces that shape software evolution, andpredict future evolutionary paths. To move towards simulation ofsoftware evolution, this research tries to explore possible modelsto simulate software evolution, and the applicability of differentdata to parameterize the models. The simulations will both bebased on fine-grained code changes obtained by comparing theabstract syntax trees of source code. The use of fine-grain codechanges could reveal information about software evolution thatis unavailable by other means.

I. INTRODUCTION

Simulations are widely used in many areas to model naturalsystems, human systems or technologies for various purposes[1]. They are often used to show the eventual effects of alter-native conditions and courses of action, e. g., simulating urbandevelopment in response to different government policies [2].Simulations can also be used when the actual system is noteasily accessible, such as durability simulators in IKEA stores[3].

Software evolution is somewhat similar to urban develop-ment in that they both involve a group of people constructingand maintaining useful artifacts (a city or a software system).The design of a city or software system happens continuously,and may change over time. This is very different from theproduction of many other artifacts (e.g., printers, cars), whichare fully designed up-front and put into production accordingto a static design. Furthermore, design decisions (e.g., zoningpolicy in a city, or architecture of a software system) mayregulate the construction of a city or implementation of asoftware system, but such control is not deterministic, i.e. thereare many possible of outcomes given a design; some of themmay even violate the design to certain degrees. In addition,both cities and software can decay (this is called urban decay[4] and code decay [5] respectively).

However, some properties of software make software evolu-tion substantially different from urban development. In urbandevelopment, many problems can be seen (e.g. building ahouse in the middle of a street), while in software evolution,problems are not so obvious (e.g. misplacing a method ina class). Also, software is more changeable than cities. Forexample, moving a method to a class it should belong to is nothard in most cases1, but relocating a house from a commercialarea to a residential area is costly and may result in legal

1Although it might be hard when the method is part of a published API.

or political issues if the owner is unwilling to move. Thisfact makes software potentially experience more frequent andextensive changes than cities.

Simulations of software evolution could potentially helpdecide between different design options. There are manysoftware design techniques that are used to structure software.To choose between those design techniques, people need toknow the possible outcomes of each. However, outcomes ofapplying a design to a piece of software are not obviouswithout simulations. Additionally, because software designswill decay without adequate refactoring [6], software evolutionsimulations may also help people understand software decayand the benefits of refactoring, by simulating the original andrefactored versions of the software. However, few papers havebeen published on simulating software evolution.

The proposed research agenda will examine the changesmade to different software systems over years of developmentand maintenance at the source code level, and based on theseobservations, simulate those changes. The code changes tobe considered are semantic changes extracted by differencingabstract syntax trees (ASTs) [7]. Semantic changes can besimulated by manipulating different parts of an AST accordingto certain probabilities. For example, simulating the evolutionof a Factory class may add new method nodes to the ASTmore often than renaming existing method nodes, resulting anincrease in the number of methods. Simulations of semanticchanges will produce plausible future source code that allowsanalysis using a potentially large number of metrics that canbe computed from source code and its changes.

This research tries to address questions crucial to simulatesoftware evolution. The first question is about the simulationmodel:

RQ1. How should source code changes be modeled in orderto simulate software evolution? How far into the future cansimulations remain plausible enough to be useful?

A variety of different stochastic models should be testedto simulate the evolution process. Because it is a complexprocess, accurately simulating code evolution almost certainlyrequires a complex model. An ideal model should simulatesoftware evolution early in a project, and predict accuratelyfar into the future. However, achieving these properties is achallenge. The proposed research would explore to what extentit is possible to achieve these goals.

Simulation models need to be parameterized by changepatterns from empirical data. The next two research questionexplore the applicability of data from history and peers re-

978-1-4673-3076-3/13/$31.00 c© 2013 IEEE ICSE 2013, San Francisco, CA, USADoctoral Symposium

1411

Page 2: [IEEE 2013 35th International Conference on Software Engineering (ICSE) - San Francisco, CA, USA (2013.05.18-2013.05.26)] 2013 35th International Conference on Software Engineering

spectively:RQ2. Can historical data be used to simulate software

evolution? How consistent are code change patterns overtime?

In order to use the history to predict the future, changepatterns need to be consistent over time. However, [8] showedthat software evolves through the alternation of long periodsof slow, incremental changes (equilibrium) and short periodsof sudden, rapid changes (punctuation). The consistency ofchange patterns should be explored both within equilibriumperiods and between them. It might also be possible toformulate a unifying model to characterize both equilibriumand punctuation.

Besides historical data, the next research question will ex-plore the value of peer classes to simulate software evolution:

RQ3. Can evolutionary data in peer classes be used tosimulate software evolution?

Peer classes are classes having similar functionality ,designsor similar values in some metrics, etc. In order to takeadvantage of peer classes, it is necessary to verify whethersuch similarity will result in similar change patterns.

Change patterns in peer classes may be useful for softwareevolution simulations not only because they can be used tosimulate change, but also because the identification of peerclasses may not rely on the change history2. Once a class isfound to be similar to a set of classes with known changepatterns, the simulations can be started, even if the class hasa very short history.

II. RELATED WORK

Software evolution has been studied for over 40 years,during which many theories have been proposed, with em-pirical studies that either confirmed or rejected the theories,leading to new or revised theories and new empirical studies.Based on these theories and empirical studies, several modelswere proposed to predict some aspects of software evolution.Section II-A reviews majors software evolution theories andtheir related empirical studies. Section II-B describes softwareevolution models that have predictive power.

A. Software Evolution Theories and Empirical Studies

Theories of software evolution have been built and validatedsince the 1970s. Based on empirical observations in largesoftware systems developed in corporate settings, Lehmanproposed laws of software evolution [9]. Empirical studies ofthese laws have been conducted in both closed source settingsand open source communities (e. g., [5], [10]).

Although Lehman’s laws provide insights into softwareevolution, they do not look into universal mechanisms andthe underlying the dynamics of software evolution [11]. Tofill this blank, several researchers turned their eyes to thetheory of self-organized criticality (SOC) [12], which has beenwidely used to model complex systems in many areas, such as

2Some software metrics need to compute from change history. If they areused as similarity measurement, the identification of peer classes will rely onthe change history

evolutionary biology and economics. Punctuated equilibriumis one of the phenomena of SOC. Punctuated equilibrium insoftware evolution had been empirically confirmed by [13] and[8], who found that software evolution alternates between longperiods of equilibrium, when the software is changed slowly,and short periods of punctuation, when the software is changeddrastically.

B. Predictive Models for Software EvolutionBased on the above-mentioned theories and empirical stud-

ies, people have tried to build models to predict some as-pects of software evolution. There are two approaches formaking such predictions: black-box approaches and white-boxapproaches [14]. This section will discuss and compare thesetwo approaches.

1) The Black-Box Approach: Using black-box approaches,people have tried to build statistical models to fit empirical dataderived from different systems and make predictions aboutcertain metrics. Such an approach treats software evolutionas a black box. The focus is on externally observed behaviorinstead of underlying details [15]. Example studies taking thisapproach are [16], [17], etc. The advantage of the statisticalapproach is that it simplifies the problem by looking at aggre-gated behavior, allowing the application of statistical methods,which have been widely used for prediction in other areas suchas the stock market [18]. However, it is this simplification thatlimits the output of the statistical models to a few high level,aggregated results, such as the size of the software or thenumber of changes.

2) The White-Box Approach: White-box approaches tryto identify and model different factors affecting softwareevolution paths [14]. The main focus of these studies isto model the software development processes that organizepeople, technology, and practices to transform information,material and energy into software [19]. Most studies in thiscategory use system dynamics to predict some trends ofsoftware evolution, such as system size and effort [20]. Anexception is [19], which developed a framework to simulatethe evolution of software structure. However, the frameworkworked on a macroscopic level of software (only down to themethod level) and the output of the model is still limited,although it is much richer than previous studies.

The proposed study will take the white-box approach, asthe output will be richer for analysis. However, instead ofmodeling the process factors (e. g., people, practices), thisstudy will focus on source code changes. By simulatingchanges to the source code, the resulting artifact will besource code with valid structure. If it is similar enough to theactual code, existing source code analysis techniques couldbe applied to it. The main difference between the proposedapproach and previous approaches is that the analysis of themodel output is not limited to a few metrics that the authorsof the model intends to measure.

III. PROPOSED WORK

This research will use open source Java projects as examplesto simulate software evolution, although the approach could be

1412

Page 3: [IEEE 2013 35th International Conference on Software Engineering (ICSE) - San Francisco, CA, USA (2013.05.18-2013.05.26)] 2013 35th International Conference on Software Engineering

Fig. 1. The overall simulation process. Each simulation run takes the sourcecode of a class, parses it to construct an AST, applies several batches ofchanges to it, which simulates the evolution over several epochs, resulting inanother AST. The resulted AST can be optionally converted back to a sourcefile.

used in other projects with suitable AST differencing packagessimilar to ChangeDistiller [7]. The simulations will begin withan AST representing a Java class, and generate batches ofchanges to transform the AST into another AST, which canthen be converted to a source file if desired (Figure 1). Theterm epoch will be used in this research to refer to the period oftime during which a batch of changes was made to a Java class.An epoch can be either a revision within a source repositoryor a wall-time duration. As it is unclear which definition ofan epoch is better for simulation purposes, they will be bothtried in this research. The simulation models need to learnfrom historical epochs to generate future epochs. For eachhistorical epoch, ChangeDistiller will be used to compare theAST at the end of the previous epoch with the one at the endof the current epoch, extracting changes made to source codeentities, which represent language constructs in Java such asstatements or method declarations. In the ASTs, these sourcecode entities correspond to sub-trees, with the lowest levelentities considered in this research being statements. A sourcecode change in ChangeDistiller is associated with informationincluding the change type and the changed entity, etc. Thechange types given by ChangeDistiller are according to thetaxonomy in [21]. A special change type NO CHANGE isadded to the taxonomy to indicate there is no change to somesource code entity during an epoch. For each epoch, there arechanges of different change types on different source codeentities. The changes and their patterns in historical epochswill be used to parameterize simulation models.

Each run of a simulation model will generate a batch ofchanges using a Monte Carlo method according to the distribu-tion defined by that model. Consequently, different runs of thesame model under the same conditions will generate differentsets of changes, resulting in different ASTs. The collectiveproperties of ASTs generated using the same model underthe fixed conditions will be studied to explore the trends ofsoftware evolution under those conditions.

The rest of this section will introduce the models andtechniques to be used for simulating software evolution.

A. Independent and Identically Distributed (IID) Model

The IID model assumes all change types are independent,i. e., there are no associations or sequential patterns. Thedistribution of different change types, P (C), can be estimatedusing the frequency of the change types. Classes with fewerchanges will have a higher value of P (NO CHANGE). Thetotal number of changes in each simulated epoch can be esti-

mated using time series analysis, or sequence analysis similarto the approaches used in [22], [23]. With the distributionof different change types and the projected total number ofchanges, source code changes will be generated using MonteCarlo sampling and applied to the ASTs. The application ofthe changes should take into consideration the change type andthe node type. For example, ATTRIBUTE RENAMING shouldonly apply to attributes of a class. The only exception is NOCHANGE, which can be applied to any source code entity.

The IID model needs to be parameterized by empiricaldata, either from the change history of the same class or peerclasses. In order to verify the applicability of historical data,the distribution of different change types will be studied atdifferent periods during the evolution of each software projectunder study. A period consists of many epochs. For eachperiod of time, the frequency of each change type will becalculated. Then the distribution estimated by the frequencyof different change types will be compared between differentperiods. If there is no significant difference in the distributionof change types in different periods, historical data will beused. Similarly, peer data will be used if there is no significantdifference in change type distribution of peer classes.

B. Stochastic Grammar

Stochastic grammars can be used to transform source codeentities, and thus simulate software evolution. The assumptionof this model is that the probability of a change made to aversion of the software only depends on source code entitiesin that version. For each type of source code entities, there willbe probabilities P (C|E) for change types that either updatethe entities, that add new entities as children to the entities,or that delete the entities. P (C|E) will be estimated from thefrequency of different change types made to the entities. Thesimulations will generate changes for each entity of type Ebased on P (C|E) and then apply those changes to the entities.

P (C|E) can be estimated from historical data or peer data.Before using historical data, it is necessary to verify if P (C|E)changes over time. Similar to the approach of subsection III-A,the P (C|E) will be estimated in different periods of softwareevolution, and compared to see whether there is any significantdifference between different periods. To use peer data, it isnecessary to compare the P (C|E) estimated from peer classesfor significance in their difference.

C. Additional Patterns

There are some other patterns that could be used in additionto the stochastic grammar. These patterns include:

• Sequential patterns. The different changes made to asource code entity form a sequence. Sequential changepatterns could be found by mining the change sequencesof source code entities of the same type. These patternscould be use to predict future changes to a source codeentity if the recent changes to the entity form a prefix ofa historical pattern.

• Association rules. Some change types may be madetogether more often than others. Association rule mining

1413

Page 4: [IEEE 2013 35th International Conference on Software Engineering (ICSE) - San Francisco, CA, USA (2013.05.18-2013.05.26)] 2013 35th International Conference on Software Engineering

tries to discover patterns in the co-incidence of differentchange types. When a developer makes a change oftype A to a class, that may increase the likelihood thata change of type B occurs to the same class in thesame epoch. With the association rules, likely co-changescould be added to the changes generated using either IIDmodels or stochastic grammars.

IV. EVALUATION

This section will show how the research questions will beaddressed, as well as how the approaches will be evaluated.

RQ1: Part of the software evolution history will be usedto parameterize the simulation models and techniques insection III, and the rest of the history will be compared tothe simulated evolution. The successfulness of the simulationswill be judged by similarity to the actual software evolutionand the plausibility of the simulated software evolution. Thesimilarity will be based on the changes made to the software,and static metrics of the resulting ASTs, while the plausibilitywill be based on how much the simulations reproduce knownphenomena of software evolution, such as Lehman’s laws ofsoftware evolution [9] and SOC dynamics.

RQ2: In order to use historical data to parameterize abovemodels, it is necessary to verify the consistency of changepatterns used in the models. To that end, the evolution ofexample software will be divided into several periods, eachcomprising many epochs. For each period in the evolution,change patterns (e. g.frequency distributions, sequences, as-sociations) needed for the simulations will be mined. Thenpatterns will be compared between different periods. If thereis no significant difference in the patterns between differentperiods, the historical data can be used.

RQ3: Similar to RQ2, before using change patterns frompeer classes in a specific model, their consistency betweenpeer classes needs to be verified. Peer classes will be identifiedby the similarity in their design, functionality, etc. Thenchange patterns will be examined for each class, and comparedbetween peer classes to decide whether there is any significantdifference.

REFERENCES

[1] Wikipedia, “Simulation,” 2012. [Online]. Available: http://en.wikipedia.org/wiki/Simulation

[2] P. Waddell and G. F. Ulfarsson, “Introduction to urban simulation: designand development of operational models,” Handbook in Transport, vol. 5,pp. 203–236, 2004.

[3] S. Snowman, “Fat-Ass Simulator,” 2011. [Online]. Available: http://livephotoblogging.blogspot.com/2011/09/fat-ass-simulator.html

[4] Wikipedia, “Urban decay,” 2012. [Online]. Available: http://en.wikipedia.org/wiki/Urban decay

[5] S. G. Eick, T. L. Graves, A. F. Karr, J. S. Marron, and A. Mockus,“Does code decay? Assessing the evidence from change managementdata,” IEEE Transactions on Software Engineering, vol. 27, no. 1, pp.1–12, 2001.

[6] M. Fowler and K. Beck, Refactoring: Improving the Design of ExistingCode, 1st ed., ser. Addison-Wesley Object Technology Series. Reading,MA: Addison Wesley, 1999.

[7] B. Fluri, M. Wuersch, M. PInzger, and H. Gall, “Change Distilling:TreeDifferencing for Fine-Grained Source Code Change Extraction,” IEEETransactions on Software Engineering, vol. 33, no. 11, pp. 725–743,Nov. 2007.

[8] J. Wu, C. Spitzer, A. Hassan, and R. Holt, “Evolution spectrographs:visualizing punctuated change in software evolution,” in Proceedings.7th International Workshop on Principles of Software Evolution, 2004.Ieee, 2004, pp. 57–66.

[9] M. M. Lehman, “Rules and Tools for Software Evolution Planning andManagement,” Annals of Software Engineering, vol. 11, no. 1, pp. 15–44, 2001.

[10] M. W. Godfrey and Q. Tu, “Evolution in open source software: a casestudy,” in Proceedings International Conference on Software Mainte-nance ICSM-94. IEEE Comput. Soc. Press, 2000, pp. 131–142.

[11] A. Gorshenev and Y. Pis’mak, “Punctuated equilibrium in softwareevolution,” Physical Review E, vol. 70, no. 6, p. 067103, Dec. 2004.

[12] P. Bak, How Nature Works: the science of self-organized criticality, ser.Copernicus Series. Springer, 1999.

[13] M. Aoyama, “Metrics and analysis of software architecture evolutionwith discontinuity,” in Proceedings of the international workshop onPrinciples of software evolution - IWPSE ’02. New York, New York,USA: ACM Press, 2002, p. 103.

[14] E. Fuentetaja, “A comparison of black-box models for software evolu-tion,” Ph.D. dissertation, Texas Tech University, 2002.

[15] M. Lehman, J. Ramil, and U. Sandler, “An approach to modellinglong-term growth trends in software systems,” in Proceedings IEEEInternational Conference on Software Maintenance. ICSM 2001. IEEEComput. Soc, 2001, pp. 219–228.

[16] C. Woodside, “A mathematical model for the evolution of software,”Journal of Systems and Software, vol. 1, pp. 337–345, Jan. 1979.

[17] I. Herraiz, J. M. Gonzalez-Barahona, G. Robles, and D. M. German,“On the prediction of the evolution of libre software projects,” in 2007IEEE International Conference on Software Maintenance. IEEE, Oct.2007, pp. 405–414.

[18] I. Herraiz, J. M. Gonzalez-Barahona, and G. Robles, “Forecasting theNumber of Changes in Eclipse Using Time Series Analysis,” in FourthInternational Workshop on Mining Software Repositories (MSR’07:ICSEWorkshops 2007), no. January. IEEE, May 2007, pp. 32–32.

[19] B. Stopford and S. Counsell, “A framework for the simulation ofstructural software evolution,” Acm Transactions On Modeling AndComputer Simulation, vol. 18, no. 4, pp. 1–36, 2008.

[20] P. Wernick and M. Lehman, “Software process white box modellingfor FEAST/1,” Journal of Systems and Software, vol. 46, no. 2-3, pp.193–201, Apr. 1999.

[21] B. Fluri and H. Gall, “Classifying Change Types for Qualifying ChangeCouplings,” 14th IEEE International Conference on Program Compre-hension (ICPC’06), pp. 35–45, 2006.

[22] C. Kemerer and S. Slaughter, “An empirical approach to studying soft-ware evolution,” IEEE Transactions on Software Engineering, vol. 25,no. 4, pp. 493–509, 1999.

[23] I. Herraiz, “A statistical examination of the evolution and propertiesof libre software,” in 2009 IEEE International Conference on SoftwareMaintenance. IEEE, Sep. 2009, pp. 439–442.

1414