manage complexity
DESCRIPTION
"Complexity" threatens growth, render decisions difficult and are handy as an excuse. Managing, containing, even reducing complexity is feasable, but it dies not come for free and requires through thinking. Complexity combined with size requires even more - a concept and tools to represent a complex system in such a manner that it can be described, analyzed and queried. Most efforts in data management are directed at "performance" and the thinking is focused downwards on the efficient physical representatin of tuples. We have the upper part in mind - the clear and understandable representation of complex facts and the means to design, store and analyze these facts to derive insight and answer questions. There is an incredible number of efficient ways to physically store data (e.g. incore databases like Hana or Volt). Metasafe provides the missing link between these solutions and the challenge of complex systems.TRANSCRIPT
IT to Manage Complexity
The Challenge of ComplexityModeling and AbstractionsMetasafe as a SolutionConclusion
www.Metasafe-Repository.com 2013_01_10
The World Is Complex
Nature is a complex eco-system.We must understand and respect itto be able to use and to preserve it.
Our society is a complexsocio-economic-technical system.
A complex infrastructure isrequired to support it.
IT-systems are an indispensablepart of this infrastructure.
IT
IT – the Victim of its own Success
IT-Systems became complex• Dynamic business requirements• Complex compliance rules• Hundreds of applications – legacy, new…• Conflict between “Open” and “Secure”• Complex user interfaces• Numerous flavors of architectures• Several DBMSs in parallel• Hundreds of databases with lots of columns• Diverse system-software and hardware
The consequence of this complexity• IT – a business driver or an obstacle for business development• Information gap between top management and operations level widens• Application backlog grows, cost of application acquisition and maintenance grows• Users resort to Shadow IT
Complicated vs Complex
Complicated Too Simple Complex
ManyManyNot Visible
ManyMissingIncomplete
ManyManyOrganized
EntitiesRelationships
Structure
(the world is complex, discarding information is not an acceptable solution)
Abstraction of Reality usinga Conceptual Data Model
Reality – describedby a Picture
OrderOrderNrDate
CustomerNameGender
PaymentDateAmount
pays
places
refersTo
1
0..n
0..n0..n
1
1..n RAmount
The conceptual data modeldescribes the data ->
The data describe relevant factsabout the reality.
Abstraction – to reduce Complexity
Reality – described by TextCustomer places 0..n orders and pays 0..n payments.Orders and payments belong to exactly one customer.A payment may refer to 1..n orders (split by RAmount).An order can be paid in several installments (payments).An order may be placed without simultaneous paymentPayments without orders are not accepted.
“Models” an Abstraction of Reality
Abstraction
- Entities (for real and virtual things, e.g. Customer, Order)
- Relationships (between entities, e.g. places, pays)
- Attributes (properties of entities or relationships, e.g. Gender, Amount…)
a Relationship
InstancesEntity-Type
Relationship-Type
Attribute-Typean Attribute
Storage Model Conceptual Model
Entities
Entity-RelationshipDiagram*)
(* Entity-Relationship Diagrams “ER” are the standard visualization technique for semantic models)
Complexity-Reduction – The Idea
DATA(0)
Model(1)
metaModel(2)
metametaModel(3)
1012
106
102
100
describesData
Describethe world
describesModels
describesmeaModels
Name Size Purpose
enti
ty
relation
attrib
ute
entity
relation
attribute
Reduction of complexity and size by abstraction with multiple levels of models
Information Models
Conceptual Model
(ER) Entity-Relationship(RL) Relational (Tables)(oo) Object(HI) Hierarchical(Do) Document(NS) NOSQL………..
Storage Models*)
NS
Do
HI
DatabaseServices
OO-Programming
RL
(ER) Entity-RelationshipER
ER
Storage Model aka “logical” model or “internal” model – describes the implementation concepts
(the de facto Standard for visualization)
Models and ViewsConceptual Model Submodels User Access
Sales
Procurement
Customers
Submodels of the conceptual model describe individual views and access rightsSubmodels (aka external models) make large models transparent and manageable
Representations and Storage
Models (languages, e.g. UML)
- Entity-Types (for classes of things..)
- Relationship-Types (e.g.has, belongs…)
-Attribute-Types (e.g. size, color…)
Instances (data, e.g. Text, XLS..)- Entities (for things..)
- Attributes (size, color…)
- Relationships (has, belongs…)
Graphical Represention
places
Instances
Customer
John
Order
N3245
Model
Customer
Name
Order
OrderNr
places
Textual Representation
Storage in a Repository
Instances
Models
Data about the reality are complex and voluminous-> we store them in databases-> we describe them by models
Models about the reality are (less, but still) complex and voluminous-> we store them in a repository
Part 2: The Metasafe-Repository
Metasafe-Repository
Instances
Models
Model based Tool Set Integrated• Eclipse and Web-based model- and instance-data editor• erSQL-query language with graphical query builder• Java-API for model and data access
Models AND Instance Data in the Database• Conceptual model and submodels (derived views)• Model-based access rights for models and instances• Instance data with versioning
Integration in Environment• Import / Export facility XLS, XML, Graphics• Transactions for Multi-Entity and Multi-User access• Large models (nK) and Instance Data (GB)• Persistence with Standard RDBMS
Tool-Set
ClientApplication
MetasafeCore
Database
RDMS
MetasafeDataServer
XML,…XLS,…
Metasafe-Architecture
Conceptual Data Model(Entity-Relationship-Model)
Persistence(a relational DBMS)
Instances (Versioning)
(Entity-Relationship-Model)
Dic
tio
nar
yEl
em
en
ts+
Do
cExternal ModelsM
etas
afe
AP
I
Physical Architecture
ClientApplication
MetasafeCore
Database
RDMS
ClientApplication
MetasafeCore
Database
RDMS
MetasafeDataServer
WEB -Client
Web-Application
Browser
Web-Server
MetasafeDataServer
MetasafeCore
RMI
Web
Co
vers
the
full
ran
ge
Full Life-Cycle Support1. Design – conceptual models:
Design types and rights with the design dictionaryDesign instance data structure with the master modelDesign views with submodels (external model)
3. Implement – the model is blueprint:
Make complex information transparentInstances with versions, variants and access rights
4. Use – API + Toolset + erSQL query language
Powerful API to process models and instancesToolset for modeling, data edit and exchangeModel based query language with graphical query builder
2. Test the Data Model – to assure the applicability:
Load instance data into the repositoryCheck data against requirement documents (eg. with erSQL)Improve and maintain the model during the lifetime of the data
Metasafe Tool Set
API Core API - Framework for Tools and Applications
Model Manager - manage models in the database
Data Browser - Edit, View, Visualize Instances
erSQL Query – Query-Builder, Executor
Exchange – Im/Export with XML, XLS, DBMS, Files
(1) Model Editor Toolset
Dictionary of Types
Design the model
Document the Model
Generated Graphics
metaModeler: design, document …
MetasafeDatabase
….and store the model in the database
(2) Data Editor Toolset
Browse Instance DataEdit Attributes
Navigate with Graphics
metaEditor• no programming• edit/view your data• automatic - model driven• set of windows• configurable• based on Eclipse• extensible• easy to use• protected with access rights• desktop-version• web 2.0 version
(3) erSQL and the Query Builderre
sult
s@
you
rfi
nge
rtip
s
Execute
(B) select attributes
(C) generate query
(D) generate graphics
(A) define navigation
(B)
(C) (D)
(A)
Display ResultsExport Results – XLS, XML
erSQL vs SQL
“select Customer.country,Customer.Name,OrderDetail.quantityOrdered, OderDetail.price,Product.productCode, Product.productLine;
start with type = ˈCustomerˈ, name = ˈ*ˈwhere Customer.country = ˈUSˈ;
follow ˈplaceˈ to type ˈOrderˈ as Order;follow ˈhasˈ to type ˈOrderDetailˈ as OrderDetail;follow ˈrefersToˈ to type ˈProductˈ as Product;”
place
has
refersTo
CustomercustomerNamecountry
OrderorderNumber
OrderDetailquantityOrderedprice
ProductproductCodeproductLine
Request: “Report the orders placed by US-customers”
The Model The erSQL-Query:
SELECT Customer.country, Customer.Name,OrderDetail.quantityOrdered, OrderDetail.price,Product.productCode, Product.productLine
FROM Customer INNER JOIN OrderON Customer.customerNumber= Order.customerNumber
INNER JOIN OrderDetailON Order.orderNumber= OrdertDetail.orderNumber
INNER JOIN ProductON OrderDetail.productCode
= Product.productCodeWHERE Customer.country = ˈUSˈ
The SQL-Query:
Created by the userin a few minutes
using the query builder
Requires a specialisttakes a few days(to reach him)
Application Szenarios
Enterprise Architecture Management
Data and Meta-Data Management
Repository for Software-Tools
Platform for Configuration Management
DBMS for User friendly Applications
Where Management of Complexity Matters
Enterprise Architect
Report/Export XLSXML
BusinessServices
ApplicationsProjects
Report
EDITEdit – Browse – View
Import – ConsolidateFrom multiple sources
MetaData
DashBoard
Models +Metadata
AccessRights
Tool Developer
ProprietaryService Layer
RelationalDatabase
Tool-A
Files
Tool-B
Silo-Architecture
…..
Etc.
XML_Exchange
P2P-Copies
Model drivenRepository
Tool-A
Repository Services and Tools
Tool-B
Integrated Repository
View BViewA
………..
View-A View-B
… time consuming… expensive… complicated tools
… better tools faster… less time and cost… extensible, efficient
MOVE!
ConclusionComplexity
is the challenge in modern societycreates misunderstandings and costcreates the decision dilemma
Transparency BY IT and IN IT(meta) IT is required to manage ITsells and provides better service to customersreduces cost – avoids redundancy & waste
Metasafe - a Dedicated Platformto make complex information transparent,to make information useful and easily and fast accessiblein an efficient and user friendly manner
Visit us @ www.metasafe-repository.comQuestions @ info/at/metasafe-repository.com2013-01-10