A Journey from a Monolith to a Modularized ApplicationApproach and Key LearningsEmanoel Xavier Senior Software Developer, Intel

Tim Ward CTO, Paremus

1

2

Notices and Disclaimers

Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Learn more at intel.com, or from the OEM or retailer.

No computer system can be absolutely secure.

Tests document performance of components on a particular test, in specific systems. Differences in hardware, software, or configuration will affect actual performance. Consult other sources of information to evaluate performance as you consider your purchase. For more complete information about performance and benchmark results, visit http://www.intel.com/performance.

Intel, the Intel logo and others are trademarks of Intel Corporation in the U.S. and/or other countries.

*Other names and brands may be claimed as the property of others.

© 2017 Intel Corporation.

Modularity through the ages…

3

A little history

4

Adam Smith

The Wealth of Nations (1776)

Division of Labor

“…the trade of a pin-maker: a workman not educated to this business could scarce, perhaps, with his utmost

industry, make one pin in a day, and certainly could

not make twenty.”

“But in the way in which this business is now carried on, it is divided into … about eighteen distinct operations, which, in some manufactories, are all performed by distinct hands”

“Those ten persons, therefore, could make among them upwards of forty-eight thousand pins in a day. Each person, therefore, making a tenth part of forty-eight

thousand pins, might be considered as making four thousand eight hundred pins in a day.”

SMALLER SPECIALIZED CONCURRENT

5

Henry Ford

The Moving Assembly Line (1910)

Modularity & Productivity

parts and car

“come to you”83 parts. 1 step/min

from 12h to 90 mins “continuous build”

interchangeable parts

model

“The major advancement in the area of modular programming has been the development of coding techniques and assemblers which (l) allow one module to be written with little knowledge of the code in another module, and (2) allow modules to be reassembled and replacedwithout reassembly of the whole system. This facility is extremely valuable for the production of large pieces of code, but the systems most often used as examples of problem systems are highly-modularized programs and make use of the techniques mentioned above.”

– David Parnas, On the Criteria to Be Used in Decomposing Systems into Modules (1972)

6

Modularity in Software

7

Apollo 16 Lunar Lander Module (1972)

“... the systems most often used as examples of problem systems are highly-modularized programs and make use of the techniques mentioned above.”

A telling phrase…

“Modularity is too hard to be worth it”

Mixing the problem with the solution

The problems are complex

Modularity makes the solution possible

Modularity today

8

Fast forwarding

9

1970s 2017

Modern modular software – vision

10

1970s 2017

Modern modular software – reality

11

Modularity is not magic

Forgotten past lessons

70s: Low level modularity for low level coupling

Late modularity == expensive tech debt

(Serves One)

12

The recipe for a monolith

Starting a new project

13

Nobody plans to

write bad code

Things will be different

now

Well defined, modular design

we have all been there

Sonar*, Findbugs*, 100% CC

intended modularization

Growing the project

14

Mgmt needs more code

Here, more devs

Flexible processes

more requirements.

6 months deadline

but, Karl, your best coder is

gone

That is ok, we are “Agile*”

Growing the project (6 months later)

15

almost done

processes and tools

some workarounds

needed

worked overtime. “paper” release

people catching up with Karl’s

code. code reviews

that’s ok. track and fix

later

Renewed interest(6-12 months later)

16

customer: shows

interest

sales: delivery in 6

months

no one is familiar with

the code

+4 new members, +20 years worth

work

+1 headache

no previous team member.

Karl’s code is still around

17

Do any parts of this seem familiar?

Gall’s Law:

“A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system.”

Nobody sets out to write bad code

Successful projects are bound to become complex

Complexity must be managed

(Serves One)

18

How OSGi helps

19

imposes

API boundaries

hidesimplementation

details

automatesdependency management

enforcesdependencies at

runtime

facilitatescomposition

20

Won’t magically turn a monolith into a modularized application

Will provide you the tools for that

Won’t automatically fix all coupling issues, “shortcuts” and technical debt

Will make them more evident

Won’t fix API leaks

Will enforce your APIs

21

Getting results

Planned and managed

API changes

Work with it,

not against it

OSGi: ongoing

commitment

How OSGi helps

22

Limits complexity to smaller pieces

– Divide and conquer

Pieces can be more easily and predictably replaced

– No runtime surprises

Easier to understand dependencies fan out

Better testability

23

OSC 101

24

How it started

Intel closed source project

Application and deployment of security virtual functions

Let’s extend it !

Let’s build a community around it!

– Spring 2016 => Summer 2017

Se

em

s fa

mil

iar?

25

OSC in SDI

LOCATIONUTILIZATION LATENCY SECURITY

InfrastructureAttributes POWER PERFORMANCE

InfrastructureAttributes

Servers

Virtualization

ContainersVirtual Router

Virtual SAN

Scale Out

Storage

Apps SDN/NFV SDS

InfrastructurePool

VMsVirtual

FirewallVirtualSwitch

Policy based automation

Self service portal

ServiceOrchestration Compute Network Storage Security

Pool of virtualized resources

Platform capability aware

Centralized and automated

register

setup

26

OSC in action

Plugins

OSC

Security Manager

SDN Controller

Data Center

Plugins

VNF Image

Virtualization Connector

Manager Connector

Security Group

Deployment

Security Policies

IPS, NGFW,

ADC

27

OSC logical components

osc-core

sec-mgr-plugins sdn-ctrlr-plugins

sdn-ctrlr-apisec-mgr-api

OpenStack4j*

RabbitMq*

Fabric8*

OpenStack

KubernetesH2 DB

VNF Catalog

Plugins

BL services Sync tasks

Security Mgr Services SDN Ctrlr Services

UI REST API

OSGi , h2db-jpa, apache.aries.jpa, vaadin, logback,

…

28

The path to open source

Community members diversity

SDN Providers and Security Managers neutral

! OSC Plugins Model

Dependency conflicts

REST Services?

Redundant endpoints

Not conducive to early adoption

Legacy plugins

29

The journey starts…

30

Modularizing the plugins

osc-server

sec-mgr-plugins

1. Build dependencies

2. Runtime dependencies

3. Startup dependencies

osc-controller

sdn-ctrlr-plugins

sec-mgr-apisec-mgr-api

pom.xml pom.xml

whiteboard pattern

whiteboard pattern

OSGi Resolver

BAR Installer

31

Modularizing the plugins

osc-server

sec-mgr-plugins

osc-controller

sdn-ctrlr-plugins

sec-mgr-apisec-mgr-api

whiteboard pattern

whiteboard pattern

OSGi Resolver

BAR Installer

Smaller footprint

Simpler service registration

Earlier plugin validation

64 /110packaged

dependenciesOSGi Http

Service Whiteboard

Plugins Lifecycle

32

Are we modular yet?

33

Perceived modularization

Implemented Lacking

"…Curved Walking in Humans”

Gregoire Courtine, Journal of Neurophysiology

34

Does it matter?

SDN/SDI

maintainability

clean dependencies

semantic versioning

isolation

extensibility

Open Source

incremental build

clean dependencies

semantic versioning

runtime enforcement

unit testing

35

The journey continues….

36

OSC uber bundle anatomy

Common monolithic to modularized migration path

Flat classpath

Transparent non-bundle dependencies

No isolation between internal packages

osc-server

uber bundle

uber bundle

core

BL services Sync tasks

DB access

UI REST API

37

Database access: osc-domain

Only @Entity classes

JPA API & Apache Aries JPA

OSGi Transaction Control Service

Unidirectional dependency

osc-server

uber bundle

core

BL services Sync tasks

DB access

UI REST API

osc-domain

hibernate-core, h2databasehibernate-jpa-api

38

User interface: osc-ui

New project: osc-service-api

UI views as DS components

OSGi managed views

osc-server

uber bundle

core

BL services Sync tasks

UI

REST API

osc-domain

osc-ui

osc-service-api

39

REST API: osc-rest-server

Uses the osc-service-api

JAX-RS service classes

No more domain entity references from the DTOs

New service APIs and DTOsosc-server

uber bundle

core

BL services Sync tasks

REST API

osc-domain

osc-ui

osc-service-api

osc-rest-server

40

Modularized OSC

5 new smaller bundles

DS, Interfaces, clear contracts

Improved dependencies scopes

Auxiliary uber bundles

– Non OSGi dependencies

Incremental build

osc-server

osc-server

core

BL services Sync tasks

osc-domain

osc-ui

osc-service-api

osc-rest-server

+

41

Are we modular yet?End of journey?

No…

42

Modularization, like good code, is an ongoing effort

There are multiple degrees of modularization.

– A gradual approach is recommended

– But don’t go back.

OSGi is not a magic wand, but makes the job possible

A monolith will only getyou so far. For a longer,pleasant (and modular)journey you will neednot only the right toolsand frameworks. Theright mindset is just asimportant.

Thank you!Emanoel Xavier Senior Software Developer, Intel

Tim Ward CTO, Paremus

43

www.opensecuritycontroller.orgCome join our community!