OSGi In AngerA Case Study

Aepona?

• Instil customer for 3 years

• Provider of products into Telco space

• Focus on ‘Network as a Service’

• Think abstractions over operator capabilities

Telecom Web Services(In a Nutshell)

• SOAP/REST web services offering simplified APIs over SMS, MMS, Location, Conference Calling, etc

• Endpoints provisioned on a per-partner basis

• Endpoints backed by per-partner runtime policy enforcement

And Much More

• Billing

• Reporting

• Alarms

• Fault Reporting

• System deployed with null implementations of optional features

nodeA

OAM

SMS

Architecture

TLX

MMS

nodeB

OAM

MMS

TLX

SMS

container(i.e. JVM)

Portal(JBoss)

DEVKIT(Eclipse)

containergroup

Container Groups

MMS MMS

Load Balancer

Container Group

Runtime

• Equinox-based stack

• Initial bundles defined by config.ini

• Bundles can be remotely installed, uninstalled, activated, etc

• Managed by Eclipse RCP plug-in or command line (telnet & Command Provider)

A Potted History

• Project started in late 2006

• Began life as Artix-based stack

• Fairly large, ~10000 unit tests

• Evolved to OSGi 2007

The OAM

• Operations, Administration and Maintenance component spec’ed in 2007

• Used to create, delete, monitor & manage a multi-container (JVM) environment

• Two key requirements: (a) 99.999% uptime (b) ability to DYNAMICALLY install, uninstall and patch components

A Leap of Faith

• But for the wrong(ish) reasons

• OAM provided compelling reason for OSGi

• OSGi chosen mainly because it enabled hot-deploys

• Not because of enforced modularity or service-oriented design

Perceived Downside

• OSGi insists on everything being made ‘modular’

• But aren’t we already modular?

• Don’t we partition our software effectively?1000s of unit tests say we do!

Umm..NO!

• In retrospect, system far from modular

• Layers not partitioned cleanly

• Code not open for extension

• Free-for-all on code base

Life Before OSGi

• Single source tree

• Spring Configuration Hell

• Extensibility issues - PS work polluting core product

• Build issues

Evolving to OSGi

• How do we develop? Tools?

• How do we build? Ant, Maven?

• What to do with Artix, Spring & Hibernate?

• How to modularise existing code stream?

• How much time ($) is this going to take?

• No prior OSGi experience

Eclipse PDE

• OSGi is first class citizen

• Single project defines ‘target platform’

• Per-customer PSF (Team Project Set) files

• Export bundles directly to TWS

• We like PDE - but we don’t know any better!

Ant Build

• build.xml copies projects into launcher-prescribed directory structure and then kicks off launcher

• Heavy use of Eclipse features

• Considerable effort migrating projects over, but minimal effort in maintaining

• 400+ projects, features, fragments

Baby Steps

• No big bang

• Artix converted to single bundle

• Common code refactored into core bundle

• Services deployed as mega-bundles, built from single source tree

• Gradual migration to OSGi services

Hibernate

• Used bnd to create hibernate bundle

• Used DynamicImport-Package: * to find all exported mapping files & pojos

• Added Antlr & all hibernate jar files to internal classpath

• Works, but less than ideal

Spring DM

• Consistent programming model with existing Spring infrastructure/knowledge

• No issues with proxy-based approach

• Outside of core, only use Spring DM

• Inside core, mixture of Spring DM and direct OSGi, but only when necessary

Early Mistakes

• Failed to appreciate beauty of whiteboard pattern: publish/subscribe really should be replaced with export/track

• Modules not delineated using OSGi services - services enable dynamism & enforce better design

• Forgetting to unget services

Gradual Modularisation

• Services migrated one at a time

• Initially, layers broken into modules with public and private parts

• Eventually, modules refactored to be service oriented

• Code base significantly cleaner

Service(e.g. SMS)

ISmsActivityFactory

Life Before OSGi

ParlayX

ISmsDaoSmsMessage

SmsActivity

ICallback

Adapter

Gateway/Network

* ALL PUBLIC

Service(e.g. SMS)

Life With OSGi

ParlayX

SmsMessage

SmsActivity

ISmsSenderAdapter - public

Gateway

SmsSender

ISmsDaoISmsDaoISmsDaoAdapter - private

OSGi service POJO

Note to Self

• OSGi did not make me this way!

• But it encouraged me to step back...

• And be even more clinical in my separations

Reality Check

• Developers rarely have the good grace to hide implementation details!

• Developers who struggle with partitioning and layering will struggle even more with OSGi

• OSGi requires that you think more & be more disciplined

Today

• Extensive use of services & properties

• Heavy use of Spring DM

• Moderate use of the OSGi API

• Little use of compendium services

Design• Layered

• Service-Oriented

• Event-Driven

• Loosely-Coupled

• Dynamic/Modular!

Service(e.g. SMS)

core

WebService(e.g. SMS)

Adapter

OSGi services

OSGi services

Shared OSGi

services

Overall Philosophy

• Avoid the API at all costs

• Necessarily complex and wonderfully subtle - e.g. returning null from a tracker

• Well documented but good practice hard to come by

• Hard to unit test - especially trackers

• Instead use Spring DM wherever possible

Using the OSGi API

• Only 1 activator across 150 production bundles

• BundleContext injected via Spring DM BundleContextAware

• Primarily used to track & register services

• Also used to track bundles e.g. update property accessor when bundle installed, signal OAM when bundle fails to install

Idiomatic OSGi

• What is good practice? A catalog of idioms/patterns would be useful, very useful

/** * Property accessor tracker that creates describer's associated * policy description based on properties extracted from exported * policy accessor service. Once created the service is immediately * 'untracked'. */ private class PropertyAccessorTracker implements ServiceTrackerCustomizer {

public Object addingService(final ServiceReference serviceReference) { final IPropertyAccessor propertyAccessor = (IPropertyAccessor) bundleContext.getService(serviceReference); policyDescription = createPolicyDescription(propertyAccessor); bundleContext.ungetService(serviceReference); debug("Created policy description ", policyDescription); return null; }

public void modifiedService(final ServiceReference serviceReference, final Object service) { // Do nothing }

public void removedService(final ServiceReference serviceReference, final Object service) { // Do nothing - no need for unget } }

Fragments

• Used extensively to configure bundles in customer specific ways

• Used to house unit tests (10000+)

• One test fragment for every production bundle

• Fragments containing code cannot be unit tested. Therefore avoid.

Testing

• Focus mostly on behavioural & interaction-based unit testing

• Unit tests live in fragments

• Dedicated team for end-to-end testing

• Many parts of API mockable, except for trackers. Grumble.

Features

• Eclipse features describe pre-canned applications/features

• Parsed into config.ini files at build time

• Deployed under <install-dir>/conf/apps/<feature>/config.ini

• Each feature in conf/apps presented as a deployable application on front-end

OSGi Services

• Modules (mostly) communicate through services

• Services enable ALL extension points

• Lower layers define and export interfaces; consumed by higher layers

• Lower layers define and track interfaces; implemented & exported by higher layers

serviceweb service proxy

MethodInterceptorInvoker

IMethodInterceptorIMethodInterceptorIMethodInterceptor

Exported OSGi serviceWith service properties:order=1targetMethods=sendSms

Dynamic Method Interception

IMethodInterceptorimport org.aopalliance.intercept.MethodInterceptor;

/** * Method interceptor that enables any bundle to intercept * a service invocation. Implementations of this interface * are expected to be exported as an OSGi, along with 2 * properties: * <ul> * <li>an 'order' property used to determine the order in which * exported interceptors are invoked compared to each other</li> * <li>an optional set of comma separated 'targetMethods' method * names defining the method(s) that the interceptor should be * applied to</li> * </ul> * * Typically implementations will extend {@link AbstractMethodInterceptor} * rather than implement this interface directly. * * @see AbstractMethodInterceptor */public interface IMethodInterceptor extends MethodInterceptor {}

Canned Method Interceptors

• Address Normalisation: Enables white & black listing

• Group Address Resolution: Send to ‘group:abc’ => send to many

• Reporting: On every system transaction

serviceweb service proxy

PolicyEnforcer

EnforceablePolicyEnforceablePolicyEnforceablePolicy

Created using IPolicyDescriber (an OSGi service)

Policy Enforcement

Tracking Lists/** * Provider of policy descriptions ({@link PolicyDescription}s), derived * from their service policy description ({@link IPolicyDescriber}) * counterpart. */public class PolicyProvider implements IPolicyProvider {

private List<IPolicyDescriber> describers;

/** * Sets the list of policy describers being tracked. * The supplied list is expected to be a Spring-backed * list that automatically proxies and tracks the actual * policy describers. * @param describers a set of policy describers to be used */ @Required public void setPolicyDescribers(final List<IPolicyDescriber> describers) { this.describers = describers; } ...

<bean id="policyProvider" class="com.aepona.tws.core.policy.support.PolicyProvider"> <property name="policyDescribers" ref="policyDescribers" /> </bean> <osgi:list id="policyDescribers" interface="com.aepona.tws.core.policy.IPolicyDescriber" cardinality="0..N" />

Mobile Originated Events

• How to publish asynchronous SMS notifications and delivery receipts to third party applications?

• More generally - how do we publish events from lower layers to higher layers?

NetworkThird PartyApplication

Service

Mobile Originated Events

?

Back to the Whiteboard

• Lower layer defines and tracks ‘publisher’ interface

• Higher layer implements and exports publisher

Whiteboard

SmsNotificationPublisher

ISmsNotificationPublisherSmsNotificationHandler

<<implements>>

to TPA

from network

Exported as OSGi service

Tracks ISmsNotificationPublisher

Lifecycle Events

• Web service endpoints come and go

• How do we inform interested parties of such events?

• They may need to stop ongoing transactions e.g. in the network layer

OSGi as an Event Bus

• Whiteboard pattern pervades system

• Core defines and tracks ‘listener’ services

• Bundles implement and export listener implementations

• Layers are massively decoupled from one another

Service Listener/** * Interface to be implemented classes interested in service events. * Any instance of a type that implements this interface and that is * exported as an OSGi service will be informed of service lifecycle * events. Events are fired <i>asynchronously</i> but sequentially. * <p> * Implementations should not tie up the invoking thread. In other * words the {@link #serviceChanged(ServiceEvent)} method should * return as quickly as possible. */public interface IServiceListener {

/** * Receives notification that a service lifecycle event * has occurred. * * @param serviceEvent the service event */ void serviceChanged(ServiceEvent serviceEvent);}

Service Event/** * A service event describing a service lifecycle change. * Service events are categorized as: * <pre> * create - signals that a service has been provisioned * update - signals that a service has been updated * dispose - signals that a service has been disposed * destroy - signals that a service has been destroyed * </pre> * An 'update' event signals that a service has been updated * without having to recreate the endpoint from scratch. * If an update requires that a service is re-provisioned * (because its service name has changed), then a 'destroy' * followed by a 'create' event will be emitted. */public final class ServiceEvent {

/** * Returns a service create event. * @param serviceName the newly created service name * @return a create service event */ public static ServiceEvent createEvent(final ServiceName serviceName) { return new ServiceEvent(CREATE, serviceName); }...

Handling Service Events/** * Purges expired location notification requests from the system. * An expired notification is one that has sent the maximum number * of notifications or has reached its duration limit - both limits * are specified when the notification is created. */public class LocationNotificationTerminator implements ILocationNotificationTerminator, IServiceListener, InitializingBean, DisposableBean {

private ILocationNotificationCriteriaDao criteriaDao;

/** * Deletes data associated with a destroyed service */ public void serviceChanged(final ServiceEvent serviceEvent) { if (serviceEvent.isDestroyEvent()) { logger.debug("Stopping all notifications for " + serviceEvent.getServiceName()); stopAll(criteriaDao.findByServiceName(serviceEvent.getServiceName())); } }...

Service Adapters

• When deployed inside operator TWS talks to single CORBA-based adapter

• When deployed outside operator TWS proxies onto multiple operator APIs

• Goal is unified API - OneAPI

• Each operator has own service adapter OSGi service

OneAPI WebService

Service

IServiceAdapter

IServiceAdapter

IServiceAdapter

Exported as OSGi ServiceWith ‘name’ property name=orange

Service Adapters

sprint

vodafone

orange

• Need to support new operator, simply drop new adapter bundle

Modularity Matters

• TWS is large - without OSGi it would, right now, be a mess

• Modularity enforces better discipline

• Modular designs are more extensible

• Modular designs are easier to test

• Modular designs are cheaper

Personal Opinion

• First I couldn’t live without Design By Contract

• Then I couldn’t live without BDD & DI

• Now I dread life without OSGi

• Enforced modularity and OSGi services are indispensable design tools