introduction to software evolution: the software volcano
TRANSCRIPT
![Page 1: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/1.jpg)
1Introduction to Software Evolution
Introduction toSoftware Evolution:
“The Software Volcano”
Paul Klint
![Page 2: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/2.jpg)
2Introduction to Software Evolution
Software Volcano
Mt. Etna, Sicily, Italy
![Page 3: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/3.jpg)
3Introduction to Software Evolution
The Software Volcano: Languages
● For mainframe applications 80% is COBOL!● Figures taken from Capers Jones (Software
Productivity Research)
Distribution of languages in use, worldwide
Language Used in % of total COBOL 30 Assembler 10 C 10 C++ 10 550 other languages 40
![Page 4: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/4.jpg)
4Introduction to Software Evolution
● The total volume of software is estimated at 7 * 109 function points
● 1 FP = 128 lines of C or 107 lines of COBOL● The volume of the volcano is
– 750 Giga-lines of COBOL code, or– 900 Giga-lines of C code
Software Volcano: Volume
Printed on paper we can wrap planet Earth 9 times!
![Page 5: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/5.jpg)
5Introduction to Software Evolution
Software Volcano: Defects
● Observation:– on average 5 errors (bugs) per function point– includes errors in requirements, design, coding,
documentation and bad fixes● The software volcano, world-wide, contains
5 * 7 * 109 Bugs = 35 Giga Bugs
This means 6 bugs per human being onplanet Earth!
![Page 6: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/6.jpg)
6Introduction to Software Evolution
Work distribution of programmers
Year New projects Enhancements Repairs Total1950 90 3 7 1001960 8,500 500 1,000 10,0001970 65,000 15,000 20,000 100,0001980 1,200,000 600,000 200,000 2,000,0001990 3,000,000 3,000,000 1,000,000 7,000,0002000 4,000,000 4,500,000 1,500,000 10,000,0002010 5,000,000 7,000,000 2,000,000 14,000,0002020 7,000,000 11,000,000 3,000,000 21,000,000
Now: 60% of the programmers work on enhancement and repair
In 2020: only 30% of all programmers will work on new software
![Page 7: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/7.jpg)
7Introduction to Software Evolution
Message
● When an industry approaches 50 years of age it takes more workers to perform maintenance than to build new products (ex: automobile industry)
● Maintenance and renovation of existing software become more and more important: avoid that the software volcano explodes
![Page 8: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/8.jpg)
8Introduction to Software Evolution
Software Maintenance & Evolution
● What is Software Maintenance?● Why evolves software?● Problems in Software Maintenance● Solutions
![Page 9: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/9.jpg)
9Introduction to Software Evolution
What is Software Maintenance?
● Modification of a software product after delivery to correct faults, to improve performance or other attributes, or to adapt the product to a modified environment (IEEE 1219, 1993)
● Observe that:– maintenance is seen as after-the-fact activity– no integration with software development process
![Page 10: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/10.jpg)
10Introduction to Software Evolution
A
KNOWN PITFALL
![Page 11: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/11.jpg)
11Introduction to Software Evolution
Another Classification
● Software maintenance– Changes are made in response to changed requirements
– The fundamental software structure is stable
● Architectural transformation– The architecture of the system is modified
– Generally from a centralised to a distributed architecture
● Software re-engineering– No new functionality is added to the system but it is
restructured and reorganised to facilitate future changes
![Page 12: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/12.jpg)
12Introduction to Software Evolution
Why systems change
● Correct errors● Business pull:
– Business / IT alignment– Requirements change (legislation, new insights, efficiency)– Re-organization– Mergers / take-overs– New products, marketing actions– Market hypes (CRM, ERP, BPR, STP)
● Technology push:– Internet– Mobile– Updates of operating system, development environment, databases– Hardware
![Page 13: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/13.jpg)
13Introduction to Software Evolution
Categories of Maintenance
● Corrective: needed to correct actual errors● Adaptive: result from changes in the environment● Perfective: modifications to meet the expanding
needs of the user● Enhancement = Adaptive + Perfective
![Page 14: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/14.jpg)
14Introduction to Software Evolution
Cost Distribution per Category
Corrective 20%
Adaptive25%
Perfective55%
![Page 15: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/15.jpg)
15Introduction to Software Evolution
Costs of Maintenance
● Usually greater than development costs
– 2* to 100* depending on the application● Affected by both technical and non-technical factors● Increases as software is maintained
– Maintenance corrupts the software structure, making further maintenance more difficult
● Ageing software can have high support costs
– old languages, compilers etc.● Think of your software as continuously evolving
![Page 16: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/16.jpg)
16Introduction to Software Evolution
Cost distribution of two systems
0 50 1 00 150 200 2 50 300 35 0 400 450 500
System 1
System 2
Development costs Maintenance costs
$
![Page 17: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/17.jpg)
17Introduction to Software Evolution
Costs and Complexity
● Predictions of maintainability costs can be made by assessing the complexity of system components.
● Most maintenance effort is spent on a relatively small number of system components.
● Complexity depends on– Complexity of control structures;
– Complexity of data structures;
– Object, method (procedure) and module size
– Dependencies
![Page 18: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/18.jpg)
18Introduction to Software Evolution
Lehman’s Laws for Software Evolution
● Lehman observed that software evolves● Law of Continuing Change: software needs to
change in order to stay usefull● Law of Increasing Complexity: the structure of a
program deteriorates as it evolves– the structure of a program degrades until it becomes
more cost effective to rewrite it
![Page 19: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/19.jpg)
19Introduction to Software Evolution
Why Systems Survive
● Organisations have huge investments in their software systems
● Systems are critical business assets● Organizations depend on the system● Organizations know how to use their systems● (Re) building systems is high risk
![Page 20: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/20.jpg)
20Introduction to Software Evolution
Business versus IT inSoftware Maintenance
• Low costs
• Opportunistic / flexible• Quick decision making
• Reliability in short time• IT should understand business• Protect initial investment• Problems with IT systems
make companies careful
• Need for adequate resources
• Requires planning / choices• Hard to predict costs, impact
• Time to deliver quality• Business should understand IT• Want something new• Why all these procedures?
![Page 21: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/21.jpg)
21Introduction to Software Evolution
Major problems in Software Maintenance
● Inadequate testing methods● Performance measurement difficulties● Knowledge management / documentation● Adapting to the rapidly changing business
environment● Large backlog
![Page 22: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/22.jpg)
22Introduction to Software Evolution
Major problems in Software Management
● Lack of skilled staff● Lack of managerial understanding and support● Lack of maintenance methodology, standards,
procedures & tools● Program code is complex and unstructured● Integration of overlapping/incompatible systems
![Page 23: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/23.jpg)
23Introduction to Software Evolution
Solutions
● Architecture● Refactoring● Automated regression testing● Knowledge management● Automated code inspection● Better organization -> ITIL / CMM
![Page 24: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/24.jpg)
24Introduction to Software Evolution
What AboutSoftware Renovation?
● Legacy system:– (information) system that defeats further maintenance,
adjustment or renewal due to its size and age– requires increasing maintenance costs
● System renovation:– understanding and improvement of legacy systems– by means of reverse engineering, program
understanding, design recovery, transformation, ...
![Page 25: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/25.jpg)
25Introduction to Software Evolution
Forward Engineering
ImplementationImplementation
SpecificationSpecification
RequirementsRequirements
GoalsGoals
![Page 26: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/26.jpg)
26Introduction to Software Evolution
Reverse Engineering
ImplementationImplementation
SpecificationSpecification
RequirementsRequirements
GoalsGoals
ImplementationImplementation
SpecificationSpecification
RequirementsRequirements
GoalsGoals
Legacy system Renovated system
![Page 27: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/27.jpg)
27Introduction to Software Evolution
A Typical Legacy System
● Different implementation languages● Job Control Language scripts serve as glue● Part of programs/databases are obsolete● Some source text lost or incomplete; version
unknown● Documentation is incomplete or obsolete
~ 1-100 MLOC
![Page 28: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/28.jpg)
28Introduction to Software Evolution
Typical Renovation Questions
● What is the architecture of this system● Can we improve its structure?● Can we generate documentation for it?● Can we migrate it from COBOL 74 to
COBOL85?● Can we connect it to Internet?● Can we migrate it to a client/server architecture?
![Page 29: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/29.jpg)
29Introduction to Software Evolution
Synergy between Renovated and New Components
Legacy codeLegacy code New business requirementsNew business requirements
Extracted components
New components
![Page 30: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/30.jpg)
30Introduction to Software Evolution
Renovation = Analysis + Transformation
Legacy codeLegacy code
Renovated systemRenovated system
AnalysisAnalysis
TransformationTransformation
Documentation, object model, types,metrics, visualization, components, ...
Documentation, object model, types,metrics, visualization, components, ...
Human insight +
tools
Transformation rules
Transformation rules
![Page 31: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/31.jpg)
31Introduction to Software Evolution
Software Renovation
● Analysis (partly supported by tools):– architecture recovery– system understanding
● Transformation (mostly supported by tools):– systematic repairs– code improvement/dialect conversion/translation– architecture improvement/change
![Page 32: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/32.jpg)
32Introduction to Software Evolution
Software Renovation: Analysis
● Extraction of procedure calls and call graph● Database usage between programs● Dataflow analysis (at program and system level)● Type analysis● Cluster and concept analysis● Metrics● Visualization
![Page 33: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/33.jpg)
33Introduction to Software Evolution
Software Renovation: analysis
Legacy codeLegacy code
Documentation, object model, types,metrics, visualization, components, ...
Documentation, object model, types,metrics, visualization, components, ...
Extract
Abstract
FactsFacts
Enriched by semanticqueries
Elementaryfacts
View
![Page 34: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/34.jpg)
34Introduction to Software Evolution
The Analysis Funnel
Legacy codeLegacy code
Lexical analysisLexical analysis
Syntactic analysisSyntactic analysis
FactsFacts
Volume inhibits detailed analysis
of all code
Semantic analysisSemantic analysis
![Page 35: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/35.jpg)
35Introduction to Software Evolution
Example: DocGen
● Given the sources of a legacy system, web-based documentation is generated containing– overall architecture– module dependencies & internal structure modules– database usage– simple metrics
● Fact: code reading finds two times more defects than testing
● Commercialized by Software Improvement Group
![Page 36: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/36.jpg)
36Introduction to Software Evolution
Software Renovation: Transformation
Legacy codeLegacy code
Renovated systemRenovated systemTransformationTransformationTransformation rules
Transformation rules
FactsFacts
![Page 37: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/37.jpg)
37Introduction to Software Evolution
Typical Transformations
● Year 2000● Euro● Extending bank account numbers to 10 digits● Goto elimination● OO restructuring● Dialect translation (Cobol 74 → Cobol 85)● Language conversion (Cobol → Java, VB →
Java, ...)
![Page 38: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/38.jpg)
38Introduction to Software Evolution
Observations
● Most legacy systems are multi-lingual● A generic approach is needed to describe all
forms of analysis and transformations for all required languages
● Languages like COBOL and PL/I are big:– getting the right grammar is difficult– many parsing techniques break down
● Meta-Programming can help
![Page 39: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/39.jpg)
39Introduction to Software Evolution
How to get How to get
meta-programming
meta-programming
tools for tools for
language processing?
language processing?
Needed
Tools
![Page 40: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/40.jpg)
40Introduction to Software Evolution
Meta-Programming in Rascal
Software Analysis
Software Transformation
DSL Design & Implementation
![Page 41: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/41.jpg)
41Introduction to Software Evolution
Some Problems We Address
● How to parse source code/data files/models● How to extract facts from them● How to perform computations on these facts● How to generate new source code (trafo, refactor,
compile)● How to synthesize other information
EASY: Extract-Analyze-SYnthesize ParadigmEASY: Extract-Analyze-SYnthesize Paradigm
![Page 42: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/42.jpg)
42Introduction to Software Evolution
SystemUnder
Investigation(SUI)
ExtractExtract
Internal RepresentationInternal Representation AnalyzeAnalyze
SynthesizeSynthesize
ResultsResults
?? EASYParadigm
![Page 43: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/43.jpg)
43Introduction to Software Evolution
Here comes Rascal to the Rescue
● Lab infrastructure● Test bed● Social infrastructure● Transfer medium
– Academia– Industry (IBM, Eclipse)
![Page 44: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/44.jpg)
44Introduction to Software Evolution
Rascal Highlights
● Sophisticated built-in data types (immutable)
● Static safety● Local type inference● Pattern Matching● Pattern-directed
invocation● Syntax definitions and
parsing
● Concrete syntax● Visiting/traversal● Comprehensions● Higher-order● Familiar syntax● Java and Eclipse
integration● Read-Eval-Print
(REPL)
![Page 45: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/45.jpg)
45Introduction to Software Evolution
The Rascal Standard Library● Benchmark/test● Boolean, Number,
String, Node, Location, DateTime
● List, Set, Tuple, Relation, Map
● (Labelled) Graph, PriorityQueue
● Exception● IO (text, HTML, XML,
ATERM, RSF, AUC, ...)● JDT (Eclipse only)● Resource (Eclipse only)● Subversion/CVS● Figures (declarative
graphics)
![Page 46: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/46.jpg)
46Introduction to Software Evolution
Applications
● Several refactorings● Typechecking Rascal● Analysis SVN
repositories● Perf. Analysis Hibernate
code (KPMG)● Generic metrics● ECORE models● Magnolia compiler
● Analysis/transformation of service composition automata
● Assertion checking JML
● Various DSLs– Auditing (PWC)– Forensics (NFI)
● UvA Master's courses● OU course
![Page 47: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/47.jpg)
47Introduction to Software Evolution
View occurrences in file
![Page 48: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/48.jpg)
48Introduction to Software Evolution
![Page 49: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/49.jpg)
49Introduction to Software Evolution
![Page 50: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/50.jpg)
50Introduction to Software Evolution
Applications
Technology
Foundations
Main Research Topics
Grammar theoryand
Engineering
Language semantics
Compilationtechniques
Software Engineering
Parsing RewritingMatching TraversingConstraint
solving VisualisationIDE
supportRascal
Specific DSLs Methods for software maintenance
Methodsfor DSL design
Methods for software improvement
![Page 51: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/51.jpg)
51Introduction to Software Evolution
Take Home Messages
![Page 52: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/52.jpg)
52Introduction to Software Evolution
Take Home Messages
● Software evolves in order to stay usefull● Maintenance enables this evolution● Software renovation is needed to extend the life
cycle of a system● Rascal is a meta-programming language for
– software analysis & transformation – DSL design and implementation
![Page 53: Introduction to Software Evolution: The Software Volcano](https://reader033.vdocuments.us/reader033/viewer/2022052619/5550f3aab4c905417d8b5590/html5/thumbnails/53.jpg)
53Introduction to Software Evolution
www.cwi.nl/sen1 www.rascal-mpl.org
www.studeren.uva.nl/master-software-engineering-nl