Building Modular Software withOSGi

Ulf Fildebrandt

Some words before

This presentation should…

• …give some insights in big software products

• …and how the structure of these products ismanaged

About the presenter

• Ulf Fildebrandt• Studied computer science• Works for SAP since 1998• Starting with C++ and COM• Development in Java for Java EE server and Eclipse

(even Eclipse 2.x)• Product architect of SAP Netweaver Composition

Environment (Java stack)• Program architect of SAP Netweaver 7.30• Program architect of Integration on Java on demand

stack

Agenda

Problem statement

Goals

Implementation of Modularity

Measurement ofModularity

PROBLEM STATEMENT

Findbugs Analysis by Structure 101

SAP Netweaver Structure

„In every phenomenon the beginning remainsalways the most notable moment.“

Thomas Carlyle

Findbugs Analysis

• Famous analysis done by Ian Sutton

• Can be found at http://structure101.com/blog/2008/11/software-erosion-findbugs/

• Used Structure 101 (see also JAX 2012 innovation award for Restructure 101)

Findbugs 0.7.2

Findbugs 0.8.6

Findbugs 0.8.7

Findbugs 1.0.0

Findbugs 1.3.0

Findbugs 1.3.5

Server Analysis

• SAP Java server’s own analysis tool

• It shows the structure of the current Netweaver server

• In contrast to Structure 101 it does not only show Java dependencies (packages) but also:

– Usage Types

– Software Components

– Packages

Structure of Complex System – IOverview of product

http://pwdfm295.wdf.sap.corp:9090/makeresults/ce.lc.reporting/NW730EXT_SP_COR/gen/dbg/java/packaged/full/_doc_pkg/content/index.html

Structure of a Complex System – IIBasic Scenario

Structure of a Complex System – IIIDetailed dependencies

GOALS

Substitutability

Extensibility

„...it is almost always incorrect to begin thedecomposition of a system into modules on thebasis of a flowchart. We propose instead thatone begins with a list of difficult designdecisions or design decisions which are likely to change. Each module is then designed to hidesuch a decision from the others.“

David Parnas

Software Development GoalsModules

A module is a self-contained component of a system, often interchangeable, whichhas a well-defined interface to the other components.

http://en.wiktionary.org/wiki/module

Manage complexity of softwaresystems by „divide and conquer“

(seehttp://firstround.com/article/The-one-cost-engineers-and-product-managers-dont-consider#)

Software Development GoalsPrinciples

Extensibility Substitutability

A module is a self-contained component of a system, often interchangeable, whichhas a well-defined interface to the other components.

http://en.wiktionary.org/wiki/module

Software Development GoalsOne Product

One system can evolve over time and modules can be replaced without impacting the other modules.

Software Development GoalsMultiple Products

Multiple systems can be assembled in different products and do not have to be implemented again and again.

Product 1 Product 2

Common modules

Demo

MODULARITY BY INTENTIONProcess in development

„Programming without an overall architectureor design in mind is like exploring a cave withonly a flashlight: You don't know where you'vebeen, you don't know where you're going, and you don't know quite where you are.“

Danny Thorpe

Process to define modular software

Select modular runtime

Definition ofmodules

Decoupling•SOLID•Design Pattern

Active definitionof system

Measurement ofmetrics

BASIC MODULARITY WITH OSGI

Packages

Services

Lifecycle

Select modular runtime

Definition of modules

Decoupling•SOLID•Design Pattern

Activedefinition of

system

Measurementof metrics

Package Dependencies of a Bundle

(internal packages)Bundle

ExportedPackage

ExportedPackage

Bundle(internal packages)

ImportedPackage

• Bundle exports onlydefined packages

• Packages can beimported from otherarchives (bundles)

• All other packages areonly visible inside thebundle

ImportedPackage

Lifecycle of a Bundle

installed

resolved

uninstalled

starting

active

stopping

start

stop

install

resolve

uninstall

uninstall

How to achieve modularity...

• OSGi is a framework on top of Java

• Complements Java very good

• ...but not suffient, because complex systemrequire higher-level structure

OSGi subsystem specification

Subsystems

http://coderthoughts.blogspot.de/2013/04/osgi-subsystems.html

• Subsystems bundle manyOSGi bundles

• Defined in OSGi R5 Enterprise Specification

• Reference implementation in Apache Aries (http://apache.aries.org)

PRINCIPLES AND PATTERNS- DESIGN PATTERN (GOF)

Facade

Factory

Select modular runtime

Definition of modules

Decoupling•SOLID•Design Pattern

Activedefinition of

system

Measurementof metrics

Factory – I

• A factory externalizes thecreation of objects

• Creation and usage of objects is decoupled

• Pattern definition works forcoding

With factory:IDataAggregator aggregator =

DataAggregatorFactory.getInstance();

Without factory:DataAggregator da = new

DataAggregator();

A factory is an object for creating other objects. It is an abstraction of a constructor.

http://en.wikipedia.org/wiki/Factory_(software_concept)

Factory – II

Apply on component level

• Externalization of creation of objects is a general pattern

How can OSGi help:

• Service registry provides an abstraction between providerand consumer of a service (object instance)

• Creation of instances of a service can be handled byframeworks:– Declarative services

– Blueprint

Facade – IA facade is an object that provides a simplified interface to a larger body of code, such as a class library.

http://en.wikipedia.org/wiki/Facade_pattern

Facade:public interface IDataAggregator {

public List<IDataItem> get();

}

Implementation of facade:final class IdentityDataAggregator extends DataAggregator {

@Override

public ArrayList<IDataItem> get() {

List<IDataItem> itemList = new ArrayList<IDataItem>();

...

return itemList;

}

}

Facade – II

Apply on component level

• Access to a component as a general pattern

How can OSGi help:

1. Services are Java interfaces1. Service implementations are accessed via interface = facade

2. Exported packages are the only external visible entities of a bundle1. Not exported packages are not accessible

2. Clear definition of a facade of the bundle

PRINCIPLES AND PATTERNS- SOLID

Dependency Injection (SOLID)

Liskov (SOLID)

Select modular runtime

Definition of modules

Decoupling•SOLID•Design Pattern

Activedefinition of

system

Measurementof metrics

Dependency Injection – IDependency injection involves at least three elements:

• a dependent consumer,• a declaration of a component's dependencies, defined as interface

contracts,• an injector (sometimes referred to as a provider or container) that creates

instances of classes that implement a given dependency interface on request.

The dependent object describes what software component it depends on to do itswork. The injector decides what concrete classes satisfy the requirements of thedependent object, and provides them to the dependent.

http://en.wikipedia.org/wiki/Dependency_injection

Dependency Injection – II

pro

per

tym

eth

od

Class 1 Class 2

met

ho

d

pro

per

tym

eth

od

met

ho

d

pro

per

tym

eth

od

met

ho

d

pro

per

tym

eth

od

met

ho

d

Dependency Injection – III

Apply on component level

• Reverse the dependency is a general pattern

How can OSGi help:

• Service registry provides an abstraction between providerand consumer of a service (object instance)

• Injecting dependencies can be handled by additional frameworks– Declarative services

– Blueprint

LiskovLet q(x) be a property provable about objects x of type T. Then q(y) should beprovable for objects y of type S where S is a subtype of T.

http://en.wikipedia.org/wiki/Liskov_substitution_principle

Facade = T:public interface IDataAggregator {

public List<IDataItem> get();

}

Implementation of facade = S:final class IdentityDataAggregator extends DataAggregator {

@Override

public ArrayList<IDataItem> get() {

List<IDataItem> itemList = new ArrayList<IDataItem>();

...

return itemList;

}

}

SYSTEM ARCHITECTUREArchitecture Layer

Select modular runtime

Definition of modules

Decoupling•SOLID•Design Pattern

Activedefinition of

system

Measurementof metrics

Architecture Layer – I

System

Platform

System Layer

…

Sub system …

Module

Package

…

…

• A platform (in terms of runtime framework) hosts a system and is a versioned artifact

• A system consists of a set of sub-systems and is a versioned artifact running on a platform

• A system layer groups sub-systems providing structure and direction in terms of dependencies and is a layer which has no associated versioned artifact

• A system layer incarnation is the bill of materials or assembled sub-systems of a system layer relevant for a specific use case

• A sub-system consists of a set of modules and is a versioned artifact

• A module contains packages and is a versioned artifact

• A package contains components

– A component is a File

– A file is of type resource or source

Architecture Layer (as is) – II

System Layer

Package analysis of web application

Uses Structure 101 (see also JAX 2012 innovation award for Restructure 101)

Module = Subsystem = Bundle

Pac

kage

Architecture Layer (to be) – III

Data Source Interface

Data Aggregator Interface

Data Provider

Aggregator

UIData Display

(Servlet)

Data Source 1 Data Source 2

DataAggregator 1

DataAggregator 2

Data AggregatorInterface

Data SourceInterface

Architecture Layer – IV ConQAT (to-be diagram)

• ConQAT (https://www.conqat.org)

• Compared to Structure 101: extensible because of open source

– Used to implement modularity metrics

• Easy integration in build process (Maven) automatic check of to-be and as-is

Demo ConQAT – I

Demo ConQAT – II

MEASUREMENT OF MODULAR SOFTWARE

ConQAT structure check

Metrics

Select modular runtime

Definition of modules

Decoupling•SOLID•Design Pattern

Activedefinition of

system

Measurementof metrics

Demo ConQAT – III

Modularity Metrics – I

Coupling: determines the coupling of this component to other components and is computed based on instability (Robert C. Martin).

– I = Ce / (Ca+Ce) 1..0 (Ce (efferent) = outgoing, Ca (afferent) = ingoing)

– Consequence: the more components THIS component depends upon, the more instable it is (0= stable, 1=instable)

(Internal Packages) Bundle

ExportedPackage

Ca = usages of exported packages

Ce = imported classes in packages

ExportedPackage

ImportedPackage

ImportedPackage

Modularity Metrics – II

Relational Cohesion: a component should consist of cohesive elements, otherwise, it should be splitted. Average number of internal relationships per type.

– rc = Td / T (Td.. type relationships that are internal to this component; T.. number of types within the component)As classes inside an assembly should be strongly related, the cohesion should be high. On the other hand, too high values may indicate over-coupling. A good range for RelationalCohesion is 1.5 to 4.0. Assemblies where RelationalCohesion < 1.5 or RelationalCohesion > 4.0 might be problematic. Therefore rcI is the normalized RC, having value 1 for 1.5 to 4, decreasing to 0 outside this range based on gaussian bell curve.

Modularity Metrics – III

Encapsulation: Components should encapsulate knowledge and offer a slim interface.

– ep = pt / T (pt = private Types, T = all types) pt = types in internal packages, T = types in internal packages +

types in exported packages

The Overall Modularity Score is now defined as:– M = ((1-I) + rcI + ep) / 3.0

(Internal Packages) Bundle

ExportedPackage

ExportedPackage

ImportedPackage

ImportedPackage

SUMMARY

Key take-aways

• Big software systems have to be structured, but more high-level than packages– Concrete implementation: OSGi sub systems

• Use iterative process to structure software during the wholelifecycle of the product

• Implementation best practices support modularity– Facade

– Factory

– Dependency Injection

• Modularity can be measured by metrics