on-demand visibility copyright © 2005 composite software, inc. all rights reserved. ian pestell...
Post on 20-Jan-2016
217 Views
Preview:
TRANSCRIPT
ON-DEMAND VISIBILITY
Copyright © 2005 Composite Software, Inc. All Rights Reserved.
Ian Pestell
Director EMEA Operations
Data Virtualization an Overview
22Copyright © 2006 Composite Software, Inc. All Rights Reserved.
Is Your Data an Asset or a Liability?
“Managers spend two hours a day looking forinformation they need, and almost half thedata is useless once they get it”
Source: Accenture
“Managers spend two hours a day looking forinformation they need, and almost half thedata is useless once they get it”
Source: Accenture
33Copyright © 2006 Composite Software, Inc. All Rights Reserved.
Data within SOA
SOA Architectures implement Business Process Through Application Logic Operating on Data
Data from many sources Existing Legacy systems Databases, Web Services,
Files Complicated by package
applications, ERP and CRM
Data Services are often addressed late in SOA
SOA
Business
Process
Application
Logic
Business
Process
Application
Logic
LegacyData
ApplicationData
Data
44Copyright © 2006 Composite Software, Inc. All Rights Reserved.
Data Warehouses
RDBMS Flat FilesPackaged
Apps
Web Services
Data Integration Options
ExistingData
ConstantChange
Siloed&
Complex
DataIntegrationChallenge
BusinessSolutions
ReportingDashboards SOA
Code within the Application?
File Extracts / Database Replications?
ETL / Physical Warehouses, Marts, Stores?
EII / Virtual?
55Copyright © 2006 Composite Software, Inc. All Rights Reserved.
Data Consolidation / ETL
Physical data movement via scripts Data is extracted from source
systems Transformed
Data Cleansing, OLTP to OLAP Loaded into target system
Physical Data Warehouse, Mart, Operational Data Store
Very large volumes of data High degree of transformation Data is only as current as last
load
ETL
Server
ETL
Server
DataWarehouse
Data Mart/ ODS
ApplicationApplication
MDMHub
ApplicationsFiles
Databases
66Copyright © 2006 Composite Software, Inc. All Rights Reserved.
Data Federation / EII
Virtual data views via modelling Data is left in-situ within data
sources EII server federates and
transforms data on demand
No Data Warehouse required High degree of flexibility in adding
new data sources Lower development & operating
costs
Moderate to high data volumes Low to medium transformations Data is ‘near’ real-time
EII
Server
EII
Server
ApplicationApplication
cache
ApplicationsFiles
Databases
77Copyright © 2006 Composite Software, Inc. All Rights Reserved.
Solutions to Data Integration – Application Integration
Integration performed in the application logic Data sources require WS API
Vendor – bespoke Access to WS API via ESB
Supported by standards ws-orchestration, ws-choreography
Disadvantages Complex logic with Application High develop time Maintenance of change Enabling WS APIs for data High load on ESB Latency of ESB
Enterprise Service Bus
Application
Logic
LegacyData
ApplicationData
WS API WS API WS API WS API
88Copyright © 2006 Composite Software, Inc. All Rights Reserved.
Solutions to Data Integration – Data Services Layer
Implement Data Services Layer Provides set of reusable
data services WS access into ESB Connects to data sources
using native interfaces Integration / optimization
only done within data layer Reduces load
Enterprise Service Bus
Application
Logic
LegacyData
ApplicationData
Data Services Layer
99Copyright © 2006 Composite Software, Inc. All Rights Reserved.
Customers
Orders
Invoices
Trades
Positions
Vendors
Employees
POs
GLs
Data Warehouses
RDBMS Flat FilesPackaged
Apps
Web Services
You Can Integrate Data Differently
ExistingData
Real-timeInformation
Siloed&
Complex
VirtualizesVirtualDataLayer
Abstracts
BusinessSolutions
ReportingDashboards Applications
Federates
1010Copyright © 2006 Composite Software, Inc. All Rights Reserved.
Composite’s Mission
Composite helps IT teams who need to build new applications faster,
BusinessApplications
ReportingDashboards SOA
Data Warehouses
RDBMS XML Docs Flat FilesPackaged Apps
ExistingData
by providing data services middleware that lets them access any data in the enterprise… and deliver it on demand.
1111Copyright © 2006 Composite Software, Inc. All Rights Reserved.
What We Do
Data EngineData Engine
ForFor
Operational BIOperational BI
Operational ReportingInformation
Infrastructure
Drug Discovery Order Management Portfolio Management Inventory Management
Drug Discovery Order Management Portfolio Management Inventory Management
Single Source for all Data Standardized Access Reusable Data Services
Single Source for all Data Standardized Access Reusable Data Services
NA
Sal
es
Data ServicesData Services
(SOA)(SOA)
Access
App
Combine
Deliver
>>> On Demand <<<
1212Copyright © 2006 Composite Software, Inc. All Rights Reserved.
Composite Software at a Glance
Privately-held, Silicon Valley firm Strong Business Model & Team
Best-of-Breed Virtual Data Integration / EII Discover, Virtualize, Abstract & Federate
Proven Business and IT Value Increased Flexibility Greater Agility Lower Costs
Public Sector
Financial ServicesFinancial Services Life SciencesLife Sciences
Top 5 I-Bank
Top 5 I-Bank
Top 5 CommBank
Top 5 Inv Bank
Top 5I-Bank
MutualFund
Life Sciences TechnologyFinancial Services Consumer/Retail
IE 2008Editor’s Award
Energy and Utility
1313Copyright © 2006 Composite Software, Inc. All Rights Reserved.
Data Source 1Data Source 1 Data Source 2Data Source 2
Composite ViewComposite View
Step 1 - Development: Build and Publish a View (A View contains Metadata on the actual data source – format, type, location, access, etc.)
How we do it – Data Views
1414Copyright © 2006 Composite Software, Inc. All Rights Reserved.
Data Source 1Data Source 1 Data Source 2Data Source 2
Composite ViewComposite View
1 A B
2 C D
3 E F
4 G H
1 a b
2 c d
3 e f
4 g h
1 A B a b
2 C D c d
3 E F e f
4 G H g h
Step 2 – Runtime: Access and Combine(Queries are optimized for speed and efficiency)
Step 3 – Runtime: Deliver Information(Combined data creates meaningful information)
Reporting Analytics Composite Applications
Step 1 - Development: Build and Publish a View (A View contains Metadata on the actual data source – format, type, location, access, etc.)
How we do it – Data Views
1515Copyright © 2006 Composite Software, Inc. All Rights Reserved.
Composite Lets You Easily Construct Data Services
Access DataAccess DataCombine Data/
Model
Combine Data/Model
Request Results
Deliver ResultsDeliver Results Auto-Generated WSDLAuto-Generated WSDL
Access DataAccess DataCombine Data/Model
Combine Data/Model
Request Results
Deliver ResultsDeliver ResultsAuto-Generated WSDLAuto-Generated WSDL
1616Copyright © 2006 Composite Software, Inc. All Rights Reserved.
•S.
Data Services
Query Processing Engine
COMPOSITE INFORMATION SERVERCOMPOSITE INFORMATION SERVERCOMPOSITE INFORMATION SERVERCOMPOSITE INFORMATION SERVER
Cache
ApplicationRDBMS Excel Web Service Flat File
Composite Information Server Architecture Easily Works Within Complex Environments
Client and Enterprise Applications
Pre-Built
Custom
Security
Security
Customer-Built
(Views, SQL Script, XQuery, Java)
Mainframe
Composite StudioComposite Studio
Modeler
Manager
Composite Web Composite Web ManagerManager
Relational SQL (ODBC/JDBC/ADO.NET)Web Services (SOAP/JMS)
APIsWS (SOAP/JMS)Relational (SQL) Java MF Adapter
Ad
min
istratio
n A
PI
1717Copyright © 2006 Composite Software, Inc. All Rights Reserved.
Why Composite Application Data Services™
Composite Apps
ExcelBI Tools
SQL / WS
Complex API
Enterprise Data
AdaptorApplication
Views
1818Copyright © 2006 Composite Software, Inc. All Rights Reserved.
Composite Applications
Data Services (Info Server)
XML/HTTPSOAP/HTTPCustom
AppsXML FilesFilesPackaged Apps
Transaction Services
App Server
1) Data services stand alone2) Transaction services stand
alone3) Data services integrated
with transaction services4) Data services with
orchestration using a business process management tool or ESB
5) Data services WSDL stored in a UDDI registry
Orchestration
ESB / BPMDATADATA DATA
DATA
DATADATA DATA
DATA
Data Services
Info ServerDATADATA
DATADATA
DATADATA
DATADATA
DATADATA DATA
DATA
Data Services are a Critical SOA Component
Registry
“Over 60% of SOA services will be data services”
Source: IDC
“Over 60% of SOA services will be data services”
Source: IDC
1919Copyright © 2006 Composite Software, Inc. All Rights Reserved.
NY Investment Bank – New e-business application
Implementation of new e-business application
Initial estimate 2 man years 1 year application 1 data integration Using EAI approach
2020Copyright © 2006 Composite Software, Inc. All Rights Reserved.
NY Investment Bank – New e-business application
Implementation of new e-business application
Initial estimate 2 man years 1 year application 1 data integration Using EAI approach
Using Composite Data services Reduces Data integration from 12
Month to 4 Months Created optimized and re-usable
data services Introduced abstraction layer
between application and data
2121Copyright © 2006 Composite Software, Inc. All Rights Reserved.
Data Services At Work - New York Bank
DATADATA
DATADATA
DATADATA DATA
DATA DATADATA
DATADATA
DATADATA DATA
DATADATADATA
1. Prime Brokerage Reporting
2. Risk Management
3. Back Office Reconciliation
4. Portfolio Balancing
1. Prime Brokerage Reporting
2. Risk Management
3. Back Office Reconciliation
4. Portfolio Balancing
Multiple Projects
Data Required (100%) Reuse (nearly 40%) Lower Impact (75% or more) Faster Response (up to 80%)
Data Required (100%) Reuse (nearly 40%) Lower Impact (75% or more) Faster Response (up to 80%)
Business and IT Benefits
ExistingData
DataServices
Layer
Data Warehouses
RDBMS XML Docs Flat FilesPackaged Apps
2222Copyright © 2006 Composite Software, Inc. All Rights Reserved.
CompositeInformationServer
Single View of Payroll and VariancesHusasmidjan, Inc.
Situation & Requirements: Scandinavian construction materials
retailer/wholesaler, 25 stores
Business requirement: detect and address variances between planned and actual payroll expense
Business user: Office of the CFO
Technical requirement: Expose and combine SAP R/3 and in-house budget data
Solution:
1. Create views to SAP payroll information
2. Create views to other payroll information
3. Build roll-up view for analysis
4. Expose view into budget system
Benefits: More accurate variance information sooner
Rapid application development Legacy
Rollup View
Local views normalize across different stores
and apps
Payroll data sources
Files
top related