2008 wwfo performance - chiboug.org · on sybase ase •more than 55% ... •high query performance...
TRANSCRIPT
1 – Company Confidential – September 26, 2011
Smarter Analyticsfor Marketing Services
Providers
Introduction to Sybase IQ
Jason Lawrence – Territory Account Manager773‐864‐7253 / [email protected]
Doug Hoover – Principle Systems Consultant
Marketing Optimized:Intelligence to Action
Unleash Business Analytics Across the Enterprise with Sybase® IQ
2 – Company Confidential – September 26, 2011
WHO IS SYBASE?Sybase delivers enterprise software to manage, analyze and mobilize information.
Sybase delivers mission‐critical enterprise software to manage, analyze and mobilize information. Sybase is recognized globally as a performance leader, proven in the most data‐intensive industries and across all systems, networks and devices.
Over 27 years providing enterprise software ‐ 4000+ employees in 60 countries ‐ July 2010 – Became part of SAP
Excellent financial stability ‐ 2009 Best year in company history ($1.132B)‐15% of Revenues are invested back into R&D
Market Leadership ‐ 9 Gartner Magic Quadrants ‐ 3 Forrester Waves‐ #1 IDC Mobile device mgmt‐ #1 LinuxWorld best database‐ #1 Best Wall St data solution‐ 1 TPC ASE performance record‐ 1 TPC SQL Anywhere price record‐ 1 Guinness World Record
3 – Company Confidential – September 26, 20113
ONE SIZE DOES NOT FIT ALL IN THE DATABASE WORLDMobile and Embedded Analytics and ReportingOLTP and Decision Support
SQL Anywhere (Mobile Leader)
• Mobile database leader with 73% market share
• Mobile middleware market leader
• Leading embedded database vendor
• More than 20,000 corporate customers in
50 countries
• Over 80% of the Fortune 100
• Largest mobile application in the world with
more than 10 million users
Adaptive Server Enterprise – Online Transaction Processing
• Worlds first programmable RDBMS
• More than 50% of worlds stock trades run
on Sybase ASE
• More than 55% of Wall Street
• Awarded “Most Favorite” Linux Database by
Linux World Magazine
• Performance Leader
Sybase IQ – Column Based Analytics
• 10 to 100s times faster than traditional RDBMS
• Data compression instead of data explosion
• Unique Architecture designed for analytics
• Over 2000 customersworld wide
• Easy to implement and maintain
• Scalable from gigabytes to petabytes and from
10s of users to 1000s of users
IQCPUCPUMemCPUCPUMem
IQCPUCPUCPUMemCPUMem
IQCPUCPUMemMemMemMem
IQCPUCPUCPUCPUMemMem
IQCPUMemCPUMemCPUMem
Fiber Channel
Dashboards
Database Management Requirements
Trading
Order Processing
Mobile WorkflowMobile SalesCustom Mobile App
Calendar
InBoxTimecardExpenses
4 – Company Confidential – September 26, 2011
Data Services
Real-Time Report Server
Reporting /Dashboarding
ReplicationServer
Transactions Data Warehouse
SYBASE BI ARCHITECTURE & ECOSYSTEM
Mainframe
IBM DB2
Microsoft SQL Server
Oracle
Sybase ASE
Trading
Order Processing
Provisioning
Inventory
Financials
Enterprise Modeling
Analytics
PowerDesigner
5 – Company Confidential – September 26, 2011
SAP AND SYBASE ANALYTICS: DELIVERING A COMPLETE BI
STACK THAT INTEGRATES BOTH DATA PREPARATION AND DATA USAGE
Answers
SAP EIM SAP BI / Apps
IQ
Sybase IQ
Data Warehousing
Data preparation Data Usage
Data Modeling / Database Design Tool
Data Replication
Event Processing
Sybase IQColumn-based analytics server
IQ
Sybase PowerDesignerLeading Data Modeling Tool
Data Warehousing
Sybase Replication Server, Data movement & real-time synchronization
Sybase Aleri Streaming Platform (CEP), stream event data enabling Operational BI
+EIM
6 – Company Confidential – September 26, 2011
SYBASE IQ: THE MARKET LEADING COLUMN BASED ANALYTICS SERVERWhat Is Sybase IQ• A highly optimized analytics server/engine designed to
address challenging reporting and advanced analytics infrastructure requirements
Business Challenges Addressed• System performance bottlenecks and unacceptable
delays in business analysis due to exploding data volumes and increasing data complexity
Competitive Advantage• Speed and Flexibility – 10x‐100x faster query
performance, Ad‐hoc queries, In‐DB Analytics• Lower TCO – Compression, Data Partitioning, Commodity
Hardware, Better Price‐Performance• Mature & Proven – 1900 customers, 3300+ unique
deployments
Market Presence• Designated as a market leader in the data warehouse
space by Gartner and Forrester
–
BNP ParibasReporting (3x)
TransunionAnalytics (6x)
Oracle Sybase IQ
Spice TelecomDisc savings (50%)
DB2
Sybase IQ
Wellmark(Health insurance)4x
5x
3x5x
7 – Company Confidential – September 26, 2011
REAL WORLD BENEFITS OF SYBASE IQ FOR BI SUITESpeed • Run quarterly financial reports in hours, not days
• Analyze end of day sales analysis for thousands of stores in time for next day store opening• Business managers will run intra‐day analysis instead of waiting for end‐of‐day reports• Reports are run against data that is minutes old, not days or weeks old • Answers can be provided to customers in seconds, while they are still on the phone with the call centre• Faster responses allow for train‐of‐thought analysis, making analysts more productive• Operational decision‐making in seconds• IT can focus on new challenges (new sources of data, new types of analysis) rather than spending most of
their time fine‐tuning to improve performance• Users are not afraid to ask questions, confident that response times will be short and predictable
Scale & Flexibility
• Reports and ad‐hoc queries are run against detailed data instead of aggregates• Trending can be done against 5 years of data, rather than 6‐12 months • BI and ad‐hoc query capability can be delivered to thousands of front‐line workers concurrently • Open up BI environment to self‐service and mobile platforms, knowing that neither the volume of users
or rogue queries from untrained users will cripple the system• Data doesn’t need to be pre‐structured to give reliable response times• You can store structured, transactional data alongside unstructured qualitative data in one place• Multiple sources of data can be easily stored in a single system allowing richer questions• The addition of a new attribute takes minutes, not days (no unload, add, reload, rebuild)
Low TCO • Budget for BI projects can be spent on analysis, ETL, BI platforms and user‐training• The compression of data provided by IQ results in significantly lower storage costs• The ability to load at the same time as querying reduces admin pressure on batch windows and reduces
SLA breach penalty costs• Sybase IQ uses on average one‐third the number of CPUs of relational databases and runs on cost‐
effective commodity hardware and operating systems• Database administration costs are typically one‐third of comparable relational or pure MPP environments• Oracle and Microsoft DBAs usually require one week of training to become Sybase IQ DBAs
9 – Company Confidential – September 26, 2011
SYBASE IQ:DESIGNED FOR FAST ANALYTICS
Data is stored by columns• Each column is stored separately
– Bit‐mapped index– Index on every column– Indexes are compressed
• Columns can be partitioned
Optimized storage • Normal compression of 40%‐90%
– Database smaller than input data– Includes indexes
Fast load and query• Large 128K+ page sizes versus 2K page‐size in traditional
• Produces high performance even with less expensive and larger SATA disks
Engine only retrieves columnsused in the query• Reduces system I/O dramatically
– Average 90% less than competition• Permits better data manipulation
– Easy to alter and manage• VLDB Support
– Partition data on any attribute
Unrestricted schema design• Cost savings on warehouse design• Can greatly reduce ETL workload and latency
• Design based on application use– Flat, Star, Relational, Snowflake– Any schema
• Also allows simplified, flattened RCUBE‐type designs
10 – Company Confidential – September 26, 2011
THE SYBASE IQ DIFFERENCE:Column‐Based Architecture
Conventional Database
c5
c4
c3
c2
c1 …c
9c8
c7
c6
r1
r2
r3
r4
r5
SYBASE IQ
c5
c4
c3
c2
c1 …c
9c8
c7
c6
r1
r2
r3
r4
r5
Data is stored & retrieved vertically
Each column is stored separately – the data is the index
Retrieve only columns used in the specific query; no unnecessary processing
Dramatically reduce system I/O – dramatically increase query speed and accuracy
Data is stored & retrieved horizontally
Querying without indexes and views is extremely I/O intensive
Building indexes and views is a huge time and resource drain, and views/summaries do not give complete picture
Database footprint must be dramatically expanded to make the environment efficient for querying
11 – Company Confidential – September 26, 2011
SYBASE IQ: REDUCED DISK ‘HITS’= FASTER QUERY RESULTS
Example Query: How many MALES buy INSURANCE in OHIO?
800 Bytes/Row
Gender
M
M
F
M
M
‐
10M
ROWS
State
NY
TX
CT
OH
IL
VA
Row‐based RDBMSInsurance
Y
Y
N
Y
N
10M Rows x 800 Bytes4KB Page = 2,000,000 I/Os
Process large amounts of unused data
Often requires full table scan or space‐intensive indexes and materialized views
M Y VA
M N OH
F Y NY
M N OH
Gender Insured State
+ +1101
1101
10M
Bits
10M Bits x 3 column16KB Page
= 234 I/OsColumn‐based Sybase IQ
0101
10M
ROWS
12 – Company Confidential – September 26, 2011
EXAMPLE (ACTUAL CUSTOMER)
Sybase IQ improves query performance 50x over Oracle
Oracle Total: 519 minutes(31,196 seconds)Sybase IQ Total: 10 minutes (603 seconds)
13 – Company Confidential – September 26, 2011
PROVEN DATA COMPRESSION SAVINGS WITH IQSybase IQ Data Compression: saves money – saves time
Compression over Raw Data: 3.45xCompression over Row‐Based (Oracle, DB2, etc.) Warehouses: 10x
14 – Company Confidential – September 26, 2011
LOAD
Conventional DBMS
SummariesAggregates
1 – 2 TB
Indexes0.5 – 3 TB
Base table“RAW data”no indexes0.9 – 1.1 TB
2.4-6TB
Same INPUT data:“Conventional DW”is 3x-6x larger than
Sybase IQ DW
Base table: 0.2 - 0.5 TB
Indexes: 0.05 - 0.3 TB
Aggr/Summ: 0 - 0.1 TB0.25 - 0.9
TB
INPUT DATA:
1 TBSource: flat files,
ETL, replication, ODSLOAD
DATA COMPRESSIONDRAMATICALLY REDUCES STORAGE AND MAINTENANCE
15 – Company Confidential – September 26, 2011
MPP with Virtual Data Marts (VDM) — a deeper look
SYBASE IQ 15.3 PLEXQ®
SAN Fabric
Full Mesh High Speed Interconnect
VDM1
VDM2
Shared
Characteristics• VDM and works via login permission control
• VDM can isolate applications, workload, users
• DQP within VDM boundaries only
• Dynamic/scheduled resource reassignment b/w VDM
Advantages• Effective user group assignment to resources
• Effective workload isolation and partitioning• Elastic, flexible resource provisioning/utilization• Economical scalability
• Ideal for MPP AND concurrent workload
16 – Company Confidential – September 26, 2011
Scenario 1
Application intent: delinquency trend balance by periodExecution characteristics: distributed scans, joins; near linear speed up ~35% per node
Beta program findings
DQP Response Times
Configuration: 8 machines with 8 cores each
SYBASE IQ 15.3 PLEXQ®
18 – Company Confidential – September 26, 2011
SYBASE IQ: RECAP
How Sybase IQ leverages multiple approaches to scalability and performance
•Dynamic load balancing ability• Superior performance under heavy concurrent usage• Scale compute and storage separately
• Reasonable cost to scale out•Avoid being locked into a single vendor• Flexibility to adapt to technology trends
• Built from the ground up for analytic workloads•High query performance with minimal tuning• Efficient compression lowers storage requirement
Shared nothing MPP
Non‐proprietary hardware platform
Column‐based RDBMS
19 – Company Confidential – September 26, 2011
SAP AND SYBASEA complete BI analytics stack
Operational
Packaged Applications
SAP BusinessObjects EPM, SAP BusinessObjects BAS,SAP BusinessObjects GRC
Business Intelligence Tools
Web Intelligence, Crystal, Xcelsius, Explorer
SAP BusinessObjectsBusiness Intelligence
Platform
Event Processing Event InsightSybase Event Stream
Processing
Data Warehouse / Analytics
SYBASE IQ / SAP HANA / SAP BW
Data IntegrationData Services(Integration &
Quality)
Master Data Management
Information Steward
20 – Company Confidential – September 26, 2011
IQ & SAP BUSINESSOBJECTS SYNERGIES
•Certified access with and without Semantic layer
•Faster query times• For ad‐hoc queries and analyses
• For batch creation of reports (keeping time windows, SLAs, etc. ...)
•DBA support for the optimization of queries no longer necessary • Self‐service BI without IT bottleneck
• More flexibility for query definition
•No aggregation tables, high data compression• More (detail) data available for analysis purposes
• Clear modeling
•More sophisticated queries and reports
In combination with SAP BusinessObjects front‐ends
21 – Company Confidential – September 26, 2011
SYBASE IQ – SUMMARY
Sybase IQ is a proven, high‐performance, and scalable analytics server that can maximize the performance of SAP BusinessObjects applications when analyzing large historical, persistent data sets.
Optimize your SAP BusinessObjects application performance and scalability with Sybase IQ.
LOB ITAUDIENCE
BUSINESS NEEDS
KEY MESSAGE
• Just‐in‐time business decisions – near real‐time, ad hoc analysis and data mining
• Predictive analytics (“windshield views”) – enriched in‐database analytics
• Full text search and federated analytics• More analysis for more users in less time – on‐demand, self‐service access
• Faster ROI – leading best‐in‐class products work together out‐of‐box
• Speed and flexibility – 10‐100x faster ad hoc query performance
• Lowest TCO – compression, data partitioning, commodity hardware, price‐performance, resource utilization
• Less burden on IT – fewer resources, faster implementations, less hardware
• Performance optimization – integration across the BI stack
.• Mature and proven – 1,900 customers and 3,300+ unique deployments• Market Leader position in DW – Gartner Magic Quadrant and Forrester Wave
MARKET PRESENCE
23 – Company Confidential – September 26, 2011
WHAT ANALYSTS SAY
Gartner
• “During the past few years, Sybase has shown increased ability to move from offering an analytic data mart to offering an enterprise data warehouse DBMS. “
• “The company has consistently won POCs with analytic applications, sometimes with performance 100 times greater than its competitors. “
• “We believe Sybase will achieve increased market share growth in 2011 and beyond. “
Gartner, Jan 2011 MQ for DW Database Mgt Systems
Forrester• “Sybase’s mature columnar EDW platform, coupled with a fresh injection of development capital from new parent SAP, gives it a competitive advantage.”
• “Expect SAP Sybase to leverage these strengths, plus intensify its focus on in-memory and in-database analytics, to grow its share of the EDW market among large enterprises requiring robust real-time analytics solutions.”
Forrester, Jan 2011 EDW Platform Wave
25 – Company Confidential – September 26, 2011
JOINT SAP BOBJ AND SYBASE IQ CUSTOMERS
BNP was experiencing substantial growth in the use of their Web-based BOBJ reporting application and chose Sybase IQ to accelerate online transactions.RESULT: online customers generated twice as many reports 3x faster, freeing up OLTP resources.
Spice Telecom was able to populate a data warehouse with raw mobile and landline call data, analyzing data sets not previously possible, opening access to all departments.RESULT: achieved a 10x performance improvement with Sybase IQ.
REPORTING (3x)
Oracle Sybase IQ Oracle Sybase IQ
DISC SAVINGS (50%)TransUnion was able to process more than half a billion records each month with different data formats from different sources. RESULT: achieved enhanced data management, higher query speeds, lower costs, and a future-proofed system with Sybase IQ.
DB2 Sybase IQ
ANALYTICS (6x)
Sybase enables Business Objects users to dramatically improve their analytics performance with higher query speeds and faster reporting.
Wellmark chose Sybase IQ as its analytics server to decrease reporting times and claims processing, as well as centralizing data silos at the enterprise level.RESULT: reduced software costs by 200-300% and dramatically reduced people and hardware costs.
DB2 Sybase IQ
SOFTWARE SAVINGS (300%)
26 – Company Confidential – September 26, 2011
THANK YOU
JASON LAWRENCE 773‐864‐7253 OR [email protected]