frying your infrastructure
DESCRIPTION
Frying your infrastructure. Are RDBMS really useful in a distributed realtime enterprise system ?. Overview. Quick SAP application infrastructure runthrough SAP application evolution: Enterprise Services & Smart Items Oops: frying the old infrastructure - PowerPoint PPT PresentationTRANSCRIPT
Are RDBMS really useful in a distributed realtime enterprise system ?
Frying your infrastructure
SAP AG 2005, Frying your infrastructure, Burkhard Neidecker-Lutz / 2
Overview
Quick SAP application infrastructure runthrough
SAP application evolution: Enterprise Services & Smart Items
Oops: frying the old infrastructure
Lessons from the Web Services rewrite
Conclusions and SAP Research ideas
SAP AG 2005, Frying your infrastructure, Burkhard Neidecker-Lutz / 3
SAP: Lots and lots of business applications...
SAP AG 2005, Frying your infrastructure, Burkhard Neidecker-Lutz / 4
...on a common infrastructure
Portal (People, Roles, KM, Collaboration)
Exchange Infrastructure (Process Integration, Messaging)
Business Warehouse (Analytics)
APO (Planning, Optimization)
Search (TREX)
Web Application Server (Everything...)
(i.e. SAP takes a pretty broad definition of „application server“)
SAP AG 2005, Frying your infrastructure, Burkhard Neidecker-Lutz / 5
Anatomy of a workhorse: SAP Web Application Server
Web BrowserWeb Client
Web Application ServerABAP/J2EE
Database Server
Internet Communication Manager (HTTP/SOAP/etc.)
Connectivity Layer
Database Interface (JDBC/OpenSQL)
Persistence Layer
Dispatcher
Presentation Layer (JSP/etc.)
• Presentation Logic• Navigation• Interaction with Client
Dispatcher
Business Layer (J2EE/ABAP)
• Business Logic• Status Administration• Locking/Enqueue
Dispatcher
Integration Layer
Integration Tools:
• Java Connector• .NET Connector• XML/XSLT• SOAP• WSDL Proxies• BAPI/RFC• IDoc• etc.
SAP AG 2005, Frying your infrastructure, Burkhard Neidecker-Lutz / 6
Remarks on SAP Applications
SAP Web Application Server
Very robust, complete resource isolation (even for J2EE)
RDBMS (ab-) used as a persistence layer for almost everything
Not particularly lightfooted Heavyweight sessions
Accrued portability layering over the years
SAP Application Design
Database transactions only used for error rollbacks
Locking mostly handled at AppServer level (Enqueues)
Isolation mostly handled at application level
SAP AG 2005, Frying your infrastructure, Burkhard Neidecker-Lutz / 7
Global Industry Survey: The world in 2010
The ability to adapt strategy and business models quickly will be a critical source of competitive advantage
Firms will focus increasingly on speed of innovation and customer retention to create long-term value
Market consolidation will continue, lots of mergers and splits
People are the biggest asset and the biggest cost factor
Interviewing more than 4000 executives and managers
SAP AG 2005, Frying your infrastructure, Burkhard Neidecker-Lutz / 8
Flexibility: From monolithic applications to Web Services
Information TechnologySolutions
ProcessInnovationSolutions
SAP NetWeaver
UI UI UI
Functionalcomponents
DB DB
Composite Application
3 tier applications Coded transactions Best practices
Service-oriented applications Model-driven applications Best practices and platform
SAP AG 2005, Frying your infrastructure, Burkhard Neidecker-Lutz / 9
Attaching the real world: Eliminating media breaks
Source: M-Lab, 2001 (http://www.m-lab.ch/)
Degree of automation
RF
ID
Digital world („Bits“):• Inter- and cross-company
information systems (e.g.: ERP systems)
• Local, regional and global communication networks (e.g.: Internet)
Physical world („Atoms“):• Human beings • Products • Production means
State-of-the-art Next Generation Smart Items
Ma
nu
al
da
ta e
ntr
y
Vo
ice
in
pu
t
Ba
rco
de
sc
an
nin
g
Sm
art
Ite
ms
SAP AG 2005, Frying your infrastructure, Burkhard Neidecker-Lutz / 10
Performance matters: Realtime Application Scenarios
Asset Tracking
Hospital inventory
Machines and parts
Supply Chains
Emergency Response
Vehicle dispatch & tracking
Workplace safety
Security
Behavior mining
Intrusion detection
SAP AG 2005, Frying your infrastructure, Burkhard Neidecker-Lutz / 11
Putting it all together: Retail example
Enterprise & Merchandise Operations
Distribution Center Operations
Store Operations Multi-Channel Sales Operations
POS
CustomerLoyalty Mgmt
e-Commerce,Catalog &
Call CenterStore
Inventory Mgmt
In-StoreCustomer Service
PersonalizedPromotions
WarehouseManagement
FinanceHuman
Resource
RetailMaster Data Operational
Purchasing
StoreWorkforce/Task
Mgmt
InventoryMgmt
Bu
siness D
ata W
areho
use
…
Multi-ChannelOrder
Fulfillment Glo
bal
Mas
ter
Dat
a M
ana
gem
ent
(Ex
chan
ge
& C
on
soli
dat
ion
)
TransportationManagement
CustomerOrder Mgmt
Supply ChainEvent Mgmt
Sourcing
Product Design & Introduction
Foreign TradeManagement
EnterpriseOperations
Demand Intelligence & Analytics
Merchandise & Assortment Planning &
Optimization
RevenuePlanning &
Optimization
Multi Echelon Inventory Planning
& Optimization
Workforce Planning &
Optimization
Collaborative
Collaborative Collaborative
SupplyPlanning
Demand
Forecast
SAP AG 2005, Frying your infrastructure, Burkhard Neidecker-Lutz / 12
Infrastructure challenges
Event Rates Hundreds of readers in a warehouse
Bursts of hundreds of events per reader per second
Reaction times on the order of seconds in some scenarios
Data Volumes Billions of objects in a large organization (retail forecast & replenishment)
Some organizations are required to maintain history over 3 or more years
Analytics need to deal
Cost efficiency Sometimes do this on a shoestring budget
Deal with unpredictable load patterns (the curse of business flexibility)
Keep it all manageable
SAP AG 2005, Frying your infrastructure, Burkhard Neidecker-Lutz / 13
Net result: Frying your infrastructure
General purpose RDBMS no longer a good fit for these tasks
Good at
Transactions on a few records at a time
Search with well (pre-) designed queries
Not so good at:
Very long running transactions
Mass operations
Ad hoc analytics
Streams
SAP AG 2005, Frying your infrastructure, Burkhard Neidecker-Lutz / 14
Business Process Platform: The big rewrite
SAP AG 2005, Frying your infrastructure, Burkhard Neidecker-Lutz / 15
Observations from the Enterprise Services rewrite
You can do without distributed transactions for even very complex business systems
Database-based consistency can be replaced with message-based consistency
Extremely stylized use of DB model generated
transaction design patterns
A lot of DB functionality factored out (by design): No cross-component transactions (design rule)
Search (capturing updates allows for separate engine)
Analytics designed for separate engine
SAP AG 2005, Frying your infrastructure, Burkhard Neidecker-Lutz / 16
SAP Enterprise Services
SAP comitted to Enterprise Services across the board
New design opens up lots of flexibility in the lower stack Sufficiently coherent data replication in multiple special purpose
engines possible
Web services semantics require application level recovery already, negating a lot of RDBMS „comforts“
Special engines already (partly) there: Search (TREX)
Analytics (EUCLID)
Planning/Optimization (APO)
SAP AG 2005, Frying your infrastructure, Burkhard Neidecker-Lutz / 17
Research Idea: Special purpose engines for everything
Mass operations (prediction) Computation -> Data (integrated processing, LiveCache)
Data -> Computation (mass data services, SAP Research)
Streams Luckily, not yet a big problem for us (other than prediction)
SAP Research looking for partners
Very fast „simple“ RDBMS for the rest Fast keyed search, update
Speaking of device evolution: Give me a few Mbyte of MRAM per machine