manage complexity

24
IT to Manage Complexity The Challenge of Complexity Modeling and Abstractions Metasafe as a Solution Conclusion www.Metasafe-Repository.com 2013_01_10

Upload: reinhold-thurner

Post on 28-Nov-2014

305 views

Category:

Documents


1 download

DESCRIPTION

"Complexity" threatens growth, render decisions difficult and are handy as an excuse. Managing, containing, even reducing complexity is feasable, but it dies not come for free and requires through thinking. Complexity combined with size requires even more - a concept and tools to represent a complex system in such a manner that it can be described, analyzed and queried. Most efforts in data management are directed at "performance" and the thinking is focused downwards on the efficient physical representatin of tuples. We have the upper part in mind - the clear and understandable representation of complex facts and the means to design, store and analyze these facts to derive insight and answer questions. There is an incredible number of efficient ways to physically store data (e.g. incore databases like Hana or Volt). Metasafe provides the missing link between these solutions and the challenge of complex systems.

TRANSCRIPT

Page 1: Manage complexity

IT to Manage Complexity

The Challenge of ComplexityModeling and AbstractionsMetasafe as a SolutionConclusion

www.Metasafe-Repository.com 2013_01_10

Page 2: Manage complexity

The World Is Complex

Nature is a complex eco-system.We must understand and respect itto be able to use and to preserve it.

Our society is a complexsocio-economic-technical system.

A complex infrastructure isrequired to support it.

IT-systems are an indispensablepart of this infrastructure.

IT

Page 3: Manage complexity

IT – the Victim of its own Success

IT-Systems became complex• Dynamic business requirements• Complex compliance rules• Hundreds of applications – legacy, new…• Conflict between “Open” and “Secure”• Complex user interfaces• Numerous flavors of architectures• Several DBMSs in parallel• Hundreds of databases with lots of columns• Diverse system-software and hardware

The consequence of this complexity• IT – a business driver or an obstacle for business development• Information gap between top management and operations level widens• Application backlog grows, cost of application acquisition and maintenance grows• Users resort to Shadow IT

Page 4: Manage complexity

Complicated vs Complex

Complicated Too Simple Complex

ManyManyNot Visible

ManyMissingIncomplete

ManyManyOrganized

EntitiesRelationships

Structure

(the world is complex, discarding information is not an acceptable solution)

Page 5: Manage complexity

Abstraction of Reality usinga Conceptual Data Model

Reality – describedby a Picture

OrderOrderNrDate

CustomerNameGender

PaymentDateAmount

pays

places

refersTo

1

0..n

0..n0..n

1

1..n RAmount

The conceptual data modeldescribes the data ->

The data describe relevant factsabout the reality.

Abstraction – to reduce Complexity

Reality – described by TextCustomer places 0..n orders and pays 0..n payments.Orders and payments belong to exactly one customer.A payment may refer to 1..n orders (split by RAmount).An order can be paid in several installments (payments).An order may be placed without simultaneous paymentPayments without orders are not accepted.

Page 6: Manage complexity

“Models” an Abstraction of Reality

Abstraction

- Entities (for real and virtual things, e.g. Customer, Order)

- Relationships (between entities, e.g. places, pays)

- Attributes (properties of entities or relationships, e.g. Gender, Amount…)

a Relationship

InstancesEntity-Type

Relationship-Type

Attribute-Typean Attribute

Storage Model Conceptual Model

Entities

Entity-RelationshipDiagram*)

(* Entity-Relationship Diagrams “ER” are the standard visualization technique for semantic models)

Page 7: Manage complexity

Complexity-Reduction – The Idea

DATA(0)

Model(1)

metaModel(2)

metametaModel(3)

1012

106

102

100

describesData

Describethe world

describesModels

describesmeaModels

Name Size Purpose

enti

ty

relation

attrib

ute

entity

relation

attribute

Reduction of complexity and size by abstraction with multiple levels of models

Page 8: Manage complexity

Information Models

Conceptual Model

(ER) Entity-Relationship(RL) Relational (Tables)(oo) Object(HI) Hierarchical(Do) Document(NS) NOSQL………..

Storage Models*)

NS

Do

HI

DatabaseServices

OO-Programming

RL

(ER) Entity-RelationshipER

ER

Storage Model aka “logical” model or “internal” model – describes the implementation concepts

(the de facto Standard for visualization)

Page 9: Manage complexity

Models and ViewsConceptual Model Submodels User Access

Sales

Procurement

Customers

Submodels of the conceptual model describe individual views and access rightsSubmodels (aka external models) make large models transparent and manageable

Page 10: Manage complexity

Representations and Storage

Models (languages, e.g. UML)

- Entity-Types (for classes of things..)

- Relationship-Types (e.g.has, belongs…)

-Attribute-Types (e.g. size, color…)

Instances (data, e.g. Text, XLS..)- Entities (for things..)

- Attributes (size, color…)

- Relationships (has, belongs…)

Graphical Represention

places

Instances

Customer

John

Order

N3245

Model

Customer

Name

Order

OrderNr

places

Textual Representation

Storage in a Repository

Instances

Models

Data about the reality are complex and voluminous-> we store them in databases-> we describe them by models

Models about the reality are (less, but still) complex and voluminous-> we store them in a repository

Page 11: Manage complexity

Part 2: The Metasafe-Repository

Metasafe-Repository

Instances

Models

Model based Tool Set Integrated• Eclipse and Web-based model- and instance-data editor• erSQL-query language with graphical query builder• Java-API for model and data access

Models AND Instance Data in the Database• Conceptual model and submodels (derived views)• Model-based access rights for models and instances• Instance data with versioning

Integration in Environment• Import / Export facility XLS, XML, Graphics• Transactions for Multi-Entity and Multi-User access• Large models (nK) and Instance Data (GB)• Persistence with Standard RDBMS

Tool-Set

ClientApplication

MetasafeCore

Database

RDMS

MetasafeDataServer

XML,…XLS,…

Page 12: Manage complexity

Metasafe-Architecture

Conceptual Data Model(Entity-Relationship-Model)

Persistence(a relational DBMS)

Instances (Versioning)

(Entity-Relationship-Model)

Dic

tio

nar

yEl

em

en

ts+

Do

cExternal ModelsM

etas

afe

AP

I

Page 13: Manage complexity

Physical Architecture

ClientApplication

MetasafeCore

Database

RDMS

ClientApplication

MetasafeCore

Database

RDMS

MetasafeDataServer

WEB -Client

Web-Application

Browser

Web-Server

MetasafeDataServer

MetasafeCore

RMI

Web

Co

vers

the

full

ran

ge

Page 14: Manage complexity

Full Life-Cycle Support1. Design – conceptual models:

Design types and rights with the design dictionaryDesign instance data structure with the master modelDesign views with submodels (external model)

3. Implement – the model is blueprint:

Make complex information transparentInstances with versions, variants and access rights

4. Use – API + Toolset + erSQL query language

Powerful API to process models and instancesToolset for modeling, data edit and exchangeModel based query language with graphical query builder

2. Test the Data Model – to assure the applicability:

Load instance data into the repositoryCheck data against requirement documents (eg. with erSQL)Improve and maintain the model during the lifetime of the data

Page 15: Manage complexity

Metasafe Tool Set

API Core API - Framework for Tools and Applications

Model Manager - manage models in the database

Data Browser - Edit, View, Visualize Instances

erSQL Query – Query-Builder, Executor

Exchange – Im/Export with XML, XLS, DBMS, Files

Page 16: Manage complexity

(1) Model Editor Toolset

Dictionary of Types

Design the model

Document the Model

Generated Graphics

metaModeler: design, document …

MetasafeDatabase

….and store the model in the database

Page 17: Manage complexity

(2) Data Editor Toolset

Browse Instance DataEdit Attributes

Navigate with Graphics

metaEditor• no programming• edit/view your data• automatic - model driven• set of windows• configurable• based on Eclipse• extensible• easy to use• protected with access rights• desktop-version• web 2.0 version

Page 18: Manage complexity

(3) erSQL and the Query Builderre

sult

s@

you

rfi

nge

rtip

s

Execute

(B) select attributes

(C) generate query

(D) generate graphics

(A) define navigation

(B)

(C) (D)

(A)

Display ResultsExport Results – XLS, XML

Page 19: Manage complexity

erSQL vs SQL

“select Customer.country,Customer.Name,OrderDetail.quantityOrdered, OderDetail.price,Product.productCode, Product.productLine;

start with type = ˈCustomerˈ, name = ˈ*ˈwhere Customer.country = ˈUSˈ;

follow ˈplaceˈ to type ˈOrderˈ as Order;follow ˈhasˈ to type ˈOrderDetailˈ as OrderDetail;follow ˈrefersToˈ to type ˈProductˈ as Product;”

place

has

refersTo

CustomercustomerNamecountry

OrderorderNumber

OrderDetailquantityOrderedprice

ProductproductCodeproductLine

Request: “Report the orders placed by US-customers”

The Model The erSQL-Query:

SELECT Customer.country, Customer.Name,OrderDetail.quantityOrdered, OrderDetail.price,Product.productCode, Product.productLine

FROM Customer INNER JOIN OrderON Customer.customerNumber= Order.customerNumber

INNER JOIN OrderDetailON Order.orderNumber= OrdertDetail.orderNumber

INNER JOIN ProductON OrderDetail.productCode

= Product.productCodeWHERE Customer.country = ˈUSˈ

The SQL-Query:

Created by the userin a few minutes

using the query builder

Requires a specialisttakes a few days(to reach him)

Page 20: Manage complexity

Application Szenarios

Enterprise Architecture Management

Data and Meta-Data Management

Repository for Software-Tools

Platform for Configuration Management

DBMS for User friendly Applications

Where Management of Complexity Matters

Page 21: Manage complexity

Enterprise Architect

Report/Export XLSXML

BusinessServices

ApplicationsProjects

Report

EDITEdit – Browse – View

Import – ConsolidateFrom multiple sources

MetaData

DashBoard

Models +Metadata

AccessRights

Page 22: Manage complexity

Tool Developer

ProprietaryService Layer

RelationalDatabase

Tool-A

Files

Tool-B

Silo-Architecture

…..

Etc.

XML_Exchange

P2P-Copies

Model drivenRepository

Tool-A

Repository Services and Tools

Tool-B

Integrated Repository

View BViewA

………..

View-A View-B

… time consuming… expensive… complicated tools

… better tools faster… less time and cost… extensible, efficient

MOVE!

Page 23: Manage complexity

ConclusionComplexity

is the challenge in modern societycreates misunderstandings and costcreates the decision dilemma

Transparency BY IT and IN IT(meta) IT is required to manage ITsells and provides better service to customersreduces cost – avoids redundancy & waste

Metasafe - a Dedicated Platformto make complex information transparent,to make information useful and easily and fast accessiblein an efficient and user friendly manner

Page 24: Manage complexity

Visit us @ www.metasafe-repository.comQuestions @ info/at/metasafe-repository.com2013-01-10