software architecture erosion and modernization

Page: 1

Bernhard Merkle Frederic Madiot

R&D Software Products Manager

SICK AG, Germany OBEO, France

Contact us on linkedin.com or xing.com

Stop the Software

Architecture Erosion

Page: 2

Agenda

– Software Architecture

– Detect Architectural Erosion

– Eclipse + Open Source Projects

– Model-Driven Reverse-Engineering

Page: 6

Architecture, Erosion

Page: 7

Software-Architecture: Definitions

IEEE 1471-2000:

The fundamental organization of a system,

embodied in its components,

their relationship to each other and the

environment,

and the principles governing its design and

evolution.

Page: 9

Architectural Erosion

“Sometimes the developers manage to

maintain this purity of design through the

initial development and into the first release.

More often something goes wrong.

The software starts to rot like a

piece of bad meat”.

Uncle Bob: “Agile Software Development”

Page: 10

Architectural Erosion

Why should we care ?

– In (lots of) Projects, Architecture decay happens

– We are not alone, as we‘ve some good representatives… ;-)

There is/was a plan

Page: 11

X-raying Software…

Page: 12

Findbugs: the first years

0.7.2

(03/2004)

0.8.6

(10/2004)

Page: 13

Findbugs: first erosion

0.8.7 (05/2005)

Page: 14

Tangle increase…

0.8.8

(05/2005)1.0.0

(06/2006)

Page: 15

Tangle increase…

1.3.0

(07/2007)

Page: 16

ONE BIG Tangle…

1.3.8 (03/2009)

Page: 17

Page: 20

Modeling Subsystems:

Page: 21

Fixing Architectural Violations

Page: 22

Fixing Architectural Violations

Page: 23

Fixing Architectural Violations

Page: 24

Fixing Architectural Violations

Page: 25

Fixing Architectural Violations

Page: 26

Fixing Architectural Violations

Page: 27

Fixing Architectural Violations

Page: 28

Tools for Architecture-Analysis

– SotoArc SonarJ

– Bauhaus Lattix

– Structure101 Klocwork, Coverity

– http://code.google.com/p/architecturerules/

– ODASA, CodeCrawler, Codecity, SeeSoft, XRadar, XDepend, SonarSource, Kalistick, Sqoring, …

– Eclipse-Based: MoDisco (+Modernization of Applications)

http://se-radio.net/

episode-115-architecture-analysis

Page: 29

Basic Approaches (Dependencies)

– makedepend, jdepend

– Eclipse (Java Build Path)

– OSGI (Dependencies)

Page: 30

Basic Approaches

PDE Dependency Visualization

Page: 31

Missing in basic approaches:

Architecture Analysis (Deviation)

Drill-Down + Aggregation

Displaying results

Monitoring changes, trends

Rating of Architecture

� Requirements for Architectural Analysis Tools

Page: 32

Architecture Analysis (deviation)

Should-

Architecture

Requirements

Architecture-

Design

Comparison “Diff-”

ArchitectureActions

Extraction Is-

Architecture

Existing Code

•Violations

•Conformance

Page: 33

Displaying results:

Page: 34

How to replace

key people ?

How to maintain

confidence with users?How to develop

a market ?

How to augment

the development

and support load ?

Open-SourceProject

Architecture: critical for OS-projects

Marketplace

Page: 35

Openess limits Erosion

� Developers expose their reputation

� Names are associated to the architecture

� Community can provide feedback

� Warnings, Recommendations, ...

Page: 36

Risks of Erosion in FOSS

� Contributors from several organizations

� Different cultures, processes, tools, …

� Lower pressure from management

� Indirect Business

� Hazardous funding

� Difficulties to calculate costs and benefits

Page: 37

Eclipse: Architectural Analysis

Page: 38

Eclipse: 10 Years Legacy System

Eclipse Helios = 33M lines of code

Page: 39

Eclipse Architecture

Page: 40

Eclipse Architecture

Page: 41

Eclipse Architecture

Page: 42

E3.4: Plattf:Ant ���� JDT:*

Page: 43

E3.4: Plattf:Ant ���� JDT:UI

Page: 44

Plattform: CVS ���� Workb (internal)

Page: 45

Team-UI����UI-workbench (internal)

https://bugs.eclipse.org/bugs/show_bug.cgi?id=90582

Page: 46

Antipatterns

Page: 47

Dependent BaseClass

– Type:

• Design Problem

– Problem:

• one of more Methods shall implement different

behavior, depending on the type, passed in

– Context:

• make “extensible” systems, frameworks

– Forces:

• Programming languages offer, instanceof/typeid funcs.

– Antipattern:

• Methods of the baseclass, depend on derived classes, e.g. accessing

their members, doing switch/case depending on type information

Page: 48

AntiPatterns / Bad Smells:

Metrics/1000 Classes Eclipse JDK

Dependent Baseclass: 1 16

Multiple Interface Inher. 4 18

Abstractable Methods 80 60

Abstractable Attributes 45 170

Unused Classes 50 150

Unused Methods 950 2500

Unused Attributes 30 20

Page: 49

Code ClonesCode

Clones

Page: 50

Code Clones

– identical Files

• E2.0 JDT,CDT

– jdt\debug\internal\ui\dialogfields\ListDialogField.java

– cdt\debug\internal\ui\dialogfields\ListDialogField.java

• E3.4 CDT: identical packages

– cdt\debug\internal\ui\dialogfields,

cdt\debug\mi\internal\ui\dialogfields

– variation of former identical Files

• E3.4 JDT,CDT

– jdt\debug\internal\ui\dialogfields\ListDialogField.java

– cdt\debug\internal\ui\dialogfields\ListDialogField.java

Page: 51

Rating Eclipse Architecture

Minor Erosions, Code Duplication

– large codebase,

– OSGI helps a lot

– API Police,

– Guidelines

– upcoming PDE tools

– Processes and Tools

Page: 52

•Community Practices

• Engage your user community

•Software Development Practices

• Unit Test, Unit Test, Unit Test

• Continuous Integration

• Integrate in small steps

Architecture Council Recommend.

•Architectural Recommendations

• Be asynchronous

• Think API

• Long-running operations should be

cancelable

• Separate policy and mechanism

• Keep simple things simple

• Create Unittests early

• Minimize plug-in dependencies

• Be aware of the deployment context

• Package coherence

• Putting only related things together

http://wiki.eclipse.org/Architecture_Council/Top_Ten_Project_Development_Practiceshttp://wiki.eclipse.org/Architecture_Council/Top_Ten_Recommendations

Page: 53

Yearly Simultaneous Releases Rules

� Projects should leverage only published APIs of

dependencies

� Version number ends with « qualifier »

� Source code must use ICU4J classes

� The project must contain an « about.html » file

� Packages name should start with the plug-in ID

� Plug-in must not contain JARs files

� Plug-in should contain only one

« message.properties » and « Message.java » files

� Etc…

http://www.eclipse.org/indigo/planning/EclipseSimultaneousRelease.php

Page: 54

New Eclipse projects …

driving to new architectures

Components that are more reusable + customizable� Service-oriented programming model

� GUI represented by a model and configurable with CSS

� Enabling Javascript app to be executed by Eclipse runtime

� Framework to build SWT app declarativeley

Ajax-enabled web applications by using a subset of RCP APIs� Eclipse development model

� Plug-ins & Eclipse workbench extension points

� Widget toolkit using SWT API

Browser-based tools to develop for the web, in the web� Client: loosely coupled components written in JavaScript

� Server: services exposed via REST-oriented HTTP APIs

Page: 55

Manage architectural transition?

Architecture A Architecture B

?

Page: 56

Evolution to e4

Backward Compatibility Layer

• Eclipse 3.x plug-ins run on e4

Challenges

• Not clean APIs usage?

� Refactoring

• Using e4 development model?

� Migration

• Single sourcing?

�Forward Compat. Layer(Session: Singlesourcing for Eclipse 4.x and Eclipse 3.x)

Page: 57

Evolution to RAP

Single sourcing with RAP is not automatic

• API differences

• Missing extension points

• Application startup and Activator scope

• API Differences

• Field validation

• SWT ressources

• Singletons and Scopes

• Jobs and background threads

• Internationalization and localization

(http://eclipsesource.com/fileadmin/doc/2009_product/single_sourcing_guide.pdf)

� Refactoring

Page: 58

How to modernizeexisting Eclipse plug-ins?

Page: 59

Architectural Modernization Process

Legacy

Software

System

Modernized

Software

System

Audit

Reference

Tests

Modernization

Strategy

Transformation

& Integration

Non-Regression

Tests

Transformation

Tooling

Page: 60

Anatomy of an Eclipse 3.x Plug-in

MANIFEST.MF

plugin.xml

Source code

build.properties

plugin.properties

.project

.classpath

files

folders

Eclipse Plug-in

Hetero

geneity!

Page: 61

Model-Driven

Reverse-Engineering

of Eclipse plug-ins

Page: 62

Software artifacts :

- source code- configuration files

- tests- database- …

Existing

Software SystemDiscover

Models

use Models to represent and manipulate

artifacts of existing systems

Understand

Viewpoints

Transform

New

Software System

MoDisco

http://www.eclipse.org/MoDisco/

Page: 63

Supported Technologies

Infrastructure

MoDisco Architecture

Java �Metamodel�Discoverer�Generator�Transfo. to KDM

XML�Metamodel�Discoverer�Generator

Model Browser

Navigation through complex models

Customization& Extensibility

Definition ofspecific Viewpoints

DiscoveryManager

Plug and orchestratetransformations

Eclipse Modeling projects

JSP�Metamodel�Discoverer�Generator

OMG/ADMStandards

Pivot

Metamodels(SMM & KDM)

EclipsePlugin

�Metamodel�Discoverer

Page: 64

Using EMF to describe a Plug-in

MANIFEST.MF

plugin.xml

Source code

build.properties

plugin.properties

.project

.classpath

files

folders

Eclipse Plug-in

Project’s structure

(KDMSource)

.project (XML)

.classpath (XML)

manifest

build.properties

(KDMCore)

plugin(eclipseplugin)

Java source code

(Java)

plugin.properties

(KDMCore)

extensions (XML)Homogeneity

!

Page: 65

Using EMF to describe a Plug-in

Page: 66

What can you do with

the EMF model

of a plug-in ?

Page: 67

EMF Modelof an Eclipse plug-in

Leverage Eclipse Modeling components

Generate code

(Acceleo, Xpand, Jet)

Generate code

(Acceleo, Xpand, Jet)

Transform

(ATL, QVTo)

Transform

(ATL, QVTo)

Compare

(EMF Compare)

Compare

(EMF Compare)

Inspect

(MoDisco Browser, EEF)

Inspect

(MoDisco Browser, EEF)

Vizualize

(GMF, Graphiti)

Vizualize

(GMF, Graphiti)

Query

(EMF Query)

Query

(EMF Query)

Page: 68

Architectural Modernization Process

• Quality Analysis

• Volumetry

• Cartography

Legacy

Software

System

Legacy

Software

System

Modernized

Software

System

Modernized

Software

System

Audit

Reference

Tests

Reference

Tests

Modernization

Strategy

Modernization

StrategyTransformation

& Integration

Transformation

& Integration

Non-Regression

Tests

Non-Regression

Tests

Transformation

Tooling

Transformation

Tooling

Page: 69

Validation

Page: 70

Architecture Visualization

GMF diagram

created with ObeoDesigner

• purple: EMF Model

• green: UI

Page: 71

Architectural Modernization Process

Legacy

Software

System

Legacy

Software

System

Modernized

Software

System

Modernized

Software

System

AuditAudit

Reference

Tests

Reference

Tests

Modernization

Strategy

Modernization

StrategyTransformation

& Integration

Transformation

& Integration

Non-Regression

Tests

Non-Regression

Tests

Transformation

Tooling

• Parsing

• Transformation rules

• Re-generation

Page: 72

Refactoring & Migration

Transformation Rules

What to change + How to change

• Renaming

• Changing code structure

(inheritance, attributes, methods, etc)

• Replacing method calls

• Changing instructions order

• Etc

Transformed

Plug-in

ExistingPlug-in

Model of the existing

Plug-in

Model of the migrated

Plug-in

Page: 73

Architectural Modernization Process

Legacy

Software

System

Legacy

Software

System

Modernized

Software

System

Modernized

Software

System

AuditAudit

Reference

Tests

Modernization

Strategy

Modernization

StrategyTransformation

& Integration

Transformation

& Integration

Non-Regression

Tests

Non-Regression

Tests

Transformation

Tooling

Transformation

Tooling

• Tests coverage analysis

Page: 74

Test coverage analysis

Page: 75

Architectural Modernization Process

Legacy

Software

System

Legacy

Software

System

Modernized

Software

System

Modernized

Software

System

AuditAudit

Reference

Tests

Reference

Tests

Modernization

Strategy

Modernization

Strategy

Transformation

& Integration

Non-Regression

Tests

Non-Regression

Tests

Transformation

Tooling

Transformation

Tooling

• Automatic transformation

• Manual transformation

• Build

Page: 76

Build

B3 Model(build configuration)

Plug-ins to build

Model of plugins to build

Team information(CVS, SVN, etc)

Update sites

(p2)

Model of

update sites content

Build Strategies

IDM++ Research Project (ANR) -> Sept 2011

constraints solver

Page: 77

Summary

Stop the Software Architecture Erosion ?

Analysis is Required

– Evaluate the architectural situation

– Take the right decisions at the right time

Modernization is Required

– Correct erosion consequences

– Go with architectural Evolutions

=> Continuous Analysis + Modernization

Page: 78

Thank you !