when architecture meets data

24
endava.com QUALITY. PRODUCTIVITY. INNOVATION. When Architecture Meets Data Self-Aware Systems for an Imperfect World Eoin Woods IEEE Software Experts Summit Beijing, October 2015

Upload: eoin-woods

Post on 16-Apr-2017

151 views

Category:

Software


0 download

TRANSCRIPT

Page 1: When Architecture Meets Data

endava.com

QUALITY. PRODUCTIVITY. INNOVATION.

When Architecture Meets DataSelf-Aware Systems for an Imperfect World

Eoin WoodsIEEE Software Experts Summit

Beijing, October 2015

Page 2: When Architecture Meets Data

2

When Architecture Meets Data

Self-Aware Systems for an Imperfect World

Defining Architecture Architecture Today Where Architecture is Going Self-Adaptive Systems Examples Implications For Us All

QUALITY. PRODUCTIVITY. INNOVATION.

Page 3: When Architecture Meets Data

3

3

To begin: what is “architecture”?

Many uses of the term “architecture”For our purposes:

Software Architecture is the set of software related elements that comprise a software-intensive system, their relationships, and the set of design decisions that allow the system to meet its system quality requirements (“non-functionals”)

Software architecture tends to be the decisions that are difficult to change once implemented [Fowler]

QUALITY. PRODUCTIVITY. INNOVATION.

Page 4: When Architecture Meets Data

4

4

Architecture Today

Software architecture today can be characterised as:

• Static yet Ephemeral• Set in stone, difficult to discern in the running system

• Depends on “Up-Front” Platform Design• The least agile part is the runtime platform design

• Difficult to change• Architectural change is a manual, time-consuming affair

QUALITY. PRODUCTIVITY. INNOVATION.

Page 5: When Architecture Meets Data

5

5

Architecture – Static yet Ephemeral

QUALITY. PRODUCTIVITY. INNOVATION.

uml-diagrams.org

http://msdn.microsoft.com

Page 6: When Architecture Meets Data

6

6

Architecture – Big Platform Up Front

QUALITY. PRODUCTIVITY. INNOVATION.

http://www.clipartpanda.com/http://kenwoodrecords.com/http://www.aberdeeninc.com/http://www.tbngconsulting.com/

Page 7: When Architecture Meets Data

7

7

Architecture – Difficult to Change

QUALITY. PRODUCTIVITY. INNOVATION.

http://pubs.opengroup.org/

https://www.thehouseshop.com

Page 8: When Architecture Meets Data

8

8

Where Architecture is Going

Where do current trends point for architecture?

QUALITY. PRODUCTIVITY. INNOVATION.

Data Driven

Dynamic

http://daily.captaindash.com/

Page 9: When Architecture Meets Data

9

9

Learning from Infrastructure

Infrastructure is undergoing a revolution

• Static, device centric, manual change, request based… to dynamic, virtualised, automated, policy based

• Enabled by the “API for Infrastructure” – “SDx”QUALITY. PRODUCTIVITY. INNOVATION.

http://www.cxotoday.com/story/cisco-vmware-deliver-next-generation-cloud-infrastructure/http://www.bbc.co.uk/news/business-22879160

Page 10: When Architecture Meets Data

10

10

Democratisation of Data

Data and analytics are also undergoing major change

• Centralised very expensive warehouses… to scalable open source and SaaS analysis platforms

• Platform and algorithms becoming accessible commodities• Allow predictive analytics on large data sets

QUALITY. PRODUCTIVITY. INNOVATION.http://www.iconshut.com/http://www.dbtekpro.com/2014/10/21/scaling-data-apps-cloud/

Page 11: When Architecture Meets Data

11

11

Analytics Providing System Insight

Analytics possibilities• Descriptive – what has happened so far?• Predictive – what is likely in the future?• Prescriptive – how do we optimise in response?

Example using Predictive Analytics• Transaction profiling of system activity metrics• Machine learning to “teach” the system what

peaks in activity look like• Resulting model predicting future activity peaks

QUALITY. PRODUCTIVITY. INNOVATION.

https://azure.microsoft.com

Page 12: When Architecture Meets Data

12

12

Impact on Software Architecture?

What could the effect be on software architecture?

• SDx gives measurement and control of platform• Big data allows insight into system operation• SDx + Big Data == dynamic platform• Where does this leave the software architecture?• The weak link?

QUALITY. PRODUCTIVITY. INNOVATION.

Picture: infralayer – analytical controller – Software architecture static on top

Page 13: When Architecture Meets Data

13

13

Impact on Our Systems

• SDx + Analytics == dynamic platform• Software architecture is still static – left behind?

QUALITY. PRODUCTIVITY. INNOVATION.

DynamicInfrastructure

Platform

Analytics Based

Controller

Observations

Changes

http:

//m

sdn.

mic

roso

ft.co

m

Page 14: When Architecture Meets Data

14

14

• What if we had an API for our architecture?• Needs first class architecture in the code

• SDI + Analytics + SDA == dynamic system

ArchitectureAPI

Control

The Missing API - Architecture

QUALITY. PRODUCTIVITY. INNOVATION.

PlatformAPI

AnalyticsBased

Controller

Control

DynamicInfrastructure

Platform

Measurements

Page 15: When Architecture Meets Data

15

Not a new idea

1999 – IEEE Intelligent Systems Article• An Architecture Based Approach to Self-Adaptive Systems• Oreizy, Gorlick, Taylor, Heimbigner, Johnson, Medvidovic,

Quilici, Rosenblum and Wolf

2009 – QoSA Conference theme• Architectures for Self-Adaptive Software Systems

Research still going on today• Recent survey paper “A survey on engineering

approaches for self-adaptive systems” by Krupitzer, Roth, VanSyckel, Schiele, and Becker

Self-Adaptive Systems

QUALITY. PRODUCTIVITY. INNOVATION.

Page 16: When Architecture Meets Data

16

16

Self-Adaptive Systems

QUALITY. PRODUCTIVITY. INNOVATION.

Evolution

Maintain Integrity

Make changes; collect observations

Adaptation Evaluate Operation

Plan Change

Apply Change

Model

(Adapted fromOreizy 99)

System

Page 17: When Architecture Meets Data

17

17

Self-Adaptive Systems

QUALITY. PRODUCTIVITY. INNOVATION.

ConditionalStatements

OnlineAlgorithms

GenericAlgorithms

AlgorithmSelection

GeneticAlgorithms &

Machine Learning

(Adapted fromOreizy 99)

Page 18: When Architecture Meets Data

18

18

What problems might this solve?

We could have systems that …

• Auto-scale reactively• Add capacity when load increases (we have this today)

• Intelligently auto-scale predictively• Adjust capacity for predicted peaks and troughs

• Change their functional behaviour• Predictively replace processing components to meet

predicted requirements

QUALITY. PRODUCTIVITY. INNOVATION.

Page 19: When Architecture Meets Data

19

19

Risk Estimator

An Example

QUALITY. PRODUCTIVITY. INNOVATION.

Transaction Processor

Request Handler

Demand

Message Bus

Risk Calculator

Risk Dashboard

Performance Analytics

PredictiveControllerRisk Estimator

Risk Calculator

Page 20: When Architecture Meets Data

20

20

An API for Architecture

Controlling our architecture would involve:• Knowing the current structure (a model)• Adding & removing elements and connectors• Being able to “rewire” elements & connectors• Constraints to reject invalid structures• API implemented by the platform (PaaS)

Could extend the programming language or be an external language – but in the code & “compiled”• Maybe related to existing “ADLs”?

QUALITY. PRODUCTIVITY. INNOVATION.

Page 21: When Architecture Meets Data

21

21

An API for Architecture

Our example using an imaginary API

QUALITY. PRODUCTIVITY. INNOVATION.

arch = getArchitecture("MyArch")riskCalc = arch.getElement("RiskCalculator")msgBus = calc.getConnector("inboundMsg")dashBrd = calc.getConnector("dashboard")riskEst = arch.getElement("RiskEstimator")riskEst.setConnector("dashboard", dashBrd)riskEst.setConnector("inboundMsg", msgBus)riskCalc.setConnector("dashboard", null)riskCalc.setConnector("inboundMsg", null)riskCalc.deactivate()riskEst.activate()arch.commitChanges()

Page 22: When Architecture Meets Data

22

22

Implications

What does this mean for research and practice?

• More cross disciplinary research than is normal• Data, machine learning, adaptive systems, middleware, …

• More industry and academic cooperation• Industry has the systems, academia the models

• First class runtime software architecture• You need a model of the architecture to change it• No (general) runtime architecture models in use today• The challenge Software Architecture research needs?

QUALITY. PRODUCTIVITY. INNOVATION.

Page 23: When Architecture Meets Data

23

23

Our Future Systems Architectures

QUALITY. PRODUCTIVITY. INNOVATION.

Handler

Processor Calculator

Store

SDx Infrastructure System Analytics Architecture API Self-Adaptive Systems

Dynamic Data-Driven

Systems

Page 24: When Architecture Meets Data

24

Thank you

QUALITY. PRODUCTIVITY. INNOVATION.

Eoin WoodsEndava

[email protected]+44 207 367 1000en_ewoods