db2 tech talk puredata systems presentation.pdf

2012 IBM Corporation

Formerly

1

Technical Tour of PureData Systems for Transactions and Operational Warehousing

James Cho, Architect IBM PureData Systems November 15, 2012

2012 IBM Corporation Need webcast troubleshooting help? Click attachments 2

DB2 Tech Talk series host and todays presenter: Serge Rielau

SQL Architect,

DB2 for Linux, UNIX, and Windows at IBM

Serges blog: www.sqltips4db2.com

1. Access the presentation for this Tech Talk:

bitly.com/ttfilenov

2. Next steps and troubleshooting guide:

click Attachments in this webcast window

A few details .

James Cho

Architect, PureData Systems

Todays technical presenter . . .

2012 IBM Corporation Need webcast troubleshooting help? Click attachments

Disclaimer

The information contained in this presentation is provided for informational purposes only.

While efforts were made to verify the completeness and accuracy of the information contained in this presentation, it is provided as is, without warranty of any kind, express or implied.

In addition, this information is based on IBMs current product plans and strategy, which are subject to change by IBM without notice.

IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this presentation or any other documentation.

Nothing contained in this presentation is intended to, or shall have the effect of:

Creating any warranty or representation from IBM (or its affiliates or its or their suppliers and/or licensors); or

Altering the terms and conditions of the applicable license agreement governing the use of IBM software.

Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.

3


Agenda

IBM PureSystems Family Review

PureData

Database Architecture

PureData for Transactions

Product Offering

New Features

Simplified Experience Pure Console

PureData for Operational Analytics

Single Part number simplified building blocks

New Features and Enhancements

Simplified Experience Pure Console

Need webcast troubleshooting help? Click attachments 4


Integration by Design

Deeply integrating and tuning hardware and software

Simplified Experience

Making every part of the IT lifecycle easier

IBMs family of expert integrated systems has expanded

Systems with integrated expertise and built for cloud

Built-in Expertise

Capturing and automating what experts do



PureSystems simplify each phase of the systems lifecycle

Streamlined:

Pre-integrated, pre-tested, ready to run

Optimized configurations with high availability

Single part to order

Accelerated:

System Up and running in hours

Easily deploy new application and data services

Rapidly replicate environments for development and test

Simplified:

Dynamically scale to meet changing demand

Automatic failover and high availability

Fully integrated management

Integrated:

Single support number

System level updates and upgrades

Simplified problem determination

Patterns of expertise



Infrastructure Application Platform Data Platform

New

IBM PureSystems Family How much flexibility, integration and workload optimization do you want out of the box?

New PureSystem with models optimized exclusively for data workloads

Integrated and optimized application platform Built on IBM middleware to accelerate deployment of your choice of applications

Integrated and optimized infrastructure with flexibility Runs your choice of applications and middleware

Integrated and optimized data platform Delivers high performance data services to transactional and analytics applications

Delivering application platform services

Delivering IT infrastructure services



Database services that handle

large volumes of transactions with

high availability, scalability and integrity

Data Warehouse services for

complex analytics and reporting

on data up to petabyte scale -

with minimal administration

Operational Warehouse services for

continuous ingest of operational data,

complex analytics, and a large volume

of concurrent operational queries

Different data workloads have different characteristics

System for Transactions

System for Analytics

System for Operational Analytics

powered by Netezza technology



Database Architectures Single, Shared Storage, Shared Nothing

Single Database View

Log

DB2

Log

DB2

Log

DB2

Part 1 Part 2 Part 3

SQL 1 SQL 1 SQL 1

SQL 1

DB2 InfoSphere Warehouse (aka DPF) Ideal for data warehousing with MPP scale out

for near linear scalability

PureData for Analytics

PureData for Operational Analytics

Log

DB2 DB2 DB2

Single Database View

DB2 pureScale Data Sharing Ideal for active/active OLTP/ERP

scale out

PureData for Transactions

Tran 1 Tran 2 Tran 3

Shared Data Access

DB2

Tran

Log

Database

Core DB2 Ideal for OLTP and

data marts

Pure Applications

Optimized for OLTP Optimized for Analytics



Scalable Transactional Database Analytics Data Warehouse Operational Data Warehouse

Real Time Fraud Detection Customer Analysis E-commerce

Specialized data systems deliver greater performance and efficiency

Transaction Processing Reporting and Analytics Operational Analytics

Random reads &

random updates

Many transactions

with narrow data scope

accessing the same database

Shared access to all data

Sequential reads &

sequential data loads

Analytics with broad data

scope, split into many parts

across

data partitions to run in

parallel

Partitioned data access

Random and sequential reads

& data loads + continuous

ingest

Analytics split into many

parts and narrow scope

operations,

all running in parallel

Partitioned data access



Todays Data Challenges Demand Transactional Performance and Efficiency

Data systems optimized for transactions

High performance and throughput

Efficient scalability - for traditional and cloud

environments

Simplicity - across development and operations



Uninterrupted access to data with consistent performance

PureData System - built-in expertise

Traditional systems

- build it yourself

In minutes, 1. Just specify database,

description and

topology pattern

Over several days/weeks: 1. Define High Availability topology

2. Configure HW/SW/Network

3. Set up storage pools

4. Install multiple operating systems

5. Install database instances

6. Set up primary and secondary

management systems

7. Set up database members

8. Set up backup processes

9. Test, tune, reconfigure

6-node database cluster




Automatic node recovery in seconds

Log

Shared Data

Log

Log

Log

CF CF

Design point is to maximize availability during failure recovery processing

When a database member fails, only in-flight data remains locked until

member recovery completes

In-flight = data being updated on the failed member at the time it

failed

Target time to row availability


Deploy

Software application

Pattern Based Deployment Across Systems

HA Proxy

Caching

Server Database

Application

Server

HA Proxy

Application

Server

Caching

Server

Database



PureData System for Transactions

Full Rack Capabilities (Large)

384 processor cores More cores to optimize transactional performance

6.1 TB of memory

(DRAM) Allows more queries to execute entirely in-memory

19.2 TB of flash (SSD)

Allows placement of your most important

transactional database objects

on the fastest storage (48x400B)

128 TB of disk (HDD) High performance storage for todays growing data demands (144x900GB)

1,500,000 IOPS Provides sustained high performance transactional

throughput

Advanced Storage

Tiering

Automatically migrates most important data to the

fastest storage

Advanced Adaptive

Compression

up to 10x

Allows you to store more data in less space while

speeding queries

through reduced I/O transfers

High Speed

Interconnect

RDMA (Remote Direct Memory Access) for low latency

and near-linear scalability

Dual 10 GB network High speed redundant database connectivity speeds

application performance * Raw storage space



Configurations Small

Rack

Medium

Rack

Large

Full Rack

Compute Chassis 1 1 2

Blacktip ITEs

(16 cores per ITE)

6

12

24

Cores 96 192 384

Memory 1.5 TB 3.1 TB 6.1 TB

V7000 + Exp 2 4 8

User Capacity

Raw SSD Storage (400 GB

drives)

Raw HDD Storage (900 GB

drives)

18.6 TB

4.8 TB

32.4 TB

37.2 TB

9.6 TB

64.0 TB

74.4 TB

19.2 TB

128.0 TB

PureData System for Transactions

3 standard configurations to choose from Upgrade Upgrade



Database

Deploy Virtual Ap

plia

nce

Metadata

Application Server

Operating system

Vir

tua

l

Ap

pli

an

ce Metadata

Application Server

Operating system

Vir

tua

l

Ap

pli

an

ce Metadata

HTTP Server

Operating system Cluster

Topology

Consolidate more that 100 database servers

to a single system for

optimal resource efficiency

and easier administration

Optimize data compression with up to

10x storage space savings

Innovate faster by deploying

new databases in minutes

Accelerate deployment of new database

services in cloud environments

using patterns of expertise

Deploy

Capture your expertise in patterns

for consistent, reliable deployment

of critical database services

Pattern Based Database Deployment



Consistent IBM PureSystem console to manage all resources and workloads

Easy integration with data center monitoring tools and processes

Role-based security and tasks

Single point of contact for support

System firmware and OS updates applied with no planned downtime




PureData login page



Welcome page System Console



Welcome page Workload Console



Workload Management Pure Data for Transactions



Database Cluster Topology: Deploy



Database Cluster Topology: View

24 Need webcast troubleshooting help? Click attachments


Manage a pS instance (con.)



User management



Manage a pS instance - users



Manage a pS instance - user groups



Create a new DB pattern



Database patterns



Deploy a DB from a DB pattern



Databases



Manage a Database

Need webcast troubleshooting help? Click attachments Need webcast troubleshooting help? Click attachments

33


Manage a Database - backup



Manage a Database - storage



Manage a Database storage (con.)



Manage a Database - users



Manage a Database user groups



System Maintenance Sparta Data System for Transactions



Fix Pack Download



Fix Pack management



Error event log



Diagnostic levels



A closer look at PureData System for Operational Analytics



10-100x faster than traditional custom systems4

20x greater concurrency and throughput for tactical queries than previous Netezza technology5

Pattern based database deployment in minutes, not hours1

Handles more than 100 databases on 1 system2

IBM PureData System

Continuous ingest of operation data

Handles 1000+ concurrent operational queries3

Up to 10x storage savings with adaptive compression6

System for Transactions

System for Analytics

System for Operational Analytics

powered by Netezza technology

1. Based on IBM internal tests and system design for normal operation under expected typical workload. Individual results may vary. 2. Based on one large configuration 3. Based on IBM internal tests of prior generation system, and on system design for normal operations under expected typical workload. Individual results may vary. 4. Based on IBM customers' reported results. "Traditional custom systems" refers to systems that are not professionally pre-built, pre-tested and optimized. Individual results may vary. 5. Based on IBM internal performance benchmarking 6. Based on client testing is the DB2 10 Early Access Program



Embedded 7710 Single box solution scales down as well as up oThe starting building block for 7700 scalable

Consolidated GUI Console significantly improved operational experience oSingle pane of glass oAlerts for all HW/SW components oMaintenance for all firmware/software cluster wide oIntegrated OPM application management

Enhanced availability fewer components, reduced outage time oIntegrated Backup leveraging 900 GB HDD oRoving Standby, Hot swap SSD o56x fewer managed resources; 1/3rd fewer relationships

Enhanced next generation SSD and enclosure oDense (1U) Dual Controller SSD enclosure oRAID-5 at high IOPS and bandwidth

Higher Density simpler manufacturing, shipment, service oOne HA group per rack: no FC cables cross racks oP730 2U server: the rack space oDense SSD enclosure, Double Dense IO cards

Higher I/O Bandwidth high ingest capability, high scaling o40 Gbps etherchannel interconnect o64 Gbps FC storage bandwidth

New Software Stack DB210 capabilities oAIX 7.1 + DB2 10

IBM PureData System for Operational Analytics New Features



PureData for Operational Analytics Building Blocks

Server Nodes IBM 8205-E6C

IBM 8231-E2C

Storage IBM SSD 5888

IBM V7 2076-124

IBM V7 2076-224

Ethernet

Switches IBM 1G 1455-64C

IBM 10G 1455-48E

SAN switches

SAN48B-5

HMC

7042-CR6

Figures are not to scale

1 U2 U3 U4 U5 U6 U7 U8 U9 U10U11U12U13U14U15U16U17U18U19U20U21U22U23U24U25U26U27U28U29U30U31U32U33U34U35U36U37U38U39U40U41U42U

PD

U

PD

UP

DU

PD

U

Cable ingress / egress


HMC 1

HMC 2Keyboard/Monitor/KVM

Switch Pwr Cord Routing

Filler panels

Cable Ingress / Egress


Foundation Server740_2

SSD Enclosure 2

10G Enet SW 10G Enet SW 2

1G Enet SW 3

1G Enet SW 4

10G Enet SW 3

10G Enet SW 4

10G Enet SW 1

Document Drawer

1G Enet SW 1

1G Enet SW 2

SAN Switch 2

SAN Switch 1

Storage Controller 1

SSD Enclosure 1

Storage Expansion 2

Storage Expansion 1

Foundation Standby740_1

Filler panels

1 U2 U3 U4 U5 U6 U7 U8 U9 U10U11U12U13U14U15U16U17U18U19U20U21U22U23U24U25U26U27U28U29U30U31U32U33U34U35U36U37U38U39U40U41U42U

Filler panels

PD

U

PD

UP

DU

PD

U



Data Server730_2

Storage Controller_2

SSD Enclosure_1

Storage Expansion_2

Storage Expansion_1

SSD Enclosure_2

Data Standby730_1


Data Server730_3


Storage Expansion_4

Storage Expansion_3

SSD Enclosure_3


Data Server730_4


Storage Expansion_6

Storage Expansion_5

SSD Enclosure_4


Switch Pwr Cord Routing



SAN Switch 1

SAN Switch 2

XL Foundation Rack Data Rack (s)

Capacity

Add-on

Data Racks

Data

Module 1

Data

Module 2

Data

Module 3

Roving

Standby

Module

Foundation

Modules



Operational Analytics Configuration Sizes

Size System MTM Modules

Primary

Data (TB)

Backup

Cold Data

(TB)

Comments Foundation Data

XS A1791-01 8279-A01 1 0 10.8 21.6 0.5D

S A1791-02 8279-A02 1 1 32.4 64.8 1.5D

M A1791-03 8279-A03 1 2 54.0 108.0 2.5D

L A1791-04 8279-A04 1 3+ 75.6 151.2

3.5 to 9.5D

(add up to 2 full add-on

racks AD3)

XL A1791-X4 8279-A05 1 +

switches 3+ 75.6 151.2

3.5 to 18.5D

(add up to 5 full add-on

racks AD3)

Add-on

data 1 A1791-E1 8279-AD1 n/a 1 21.6 43.2 1D

Add-on

data 2 A1791-E2d 8279-AD2 n/a 2 43.2 86.4 2D

Add-on

data 3 A1791-E3 8279-AD3 n/a 3 64.8 129.6

3D

full rack

Capacity per data module (foundation module has half) 21.6 TB Primary Hot Active data 43.2 TB free space (for backups, cold data, etc.)



PureData System for Operational Analytics System Console

Provide Easy to Use, Common interfaces for management of Pure System Family

Unified Appliance UI

Alerting via SNMP & Email (Single System Software Status,

Hardware)

Platform Monitoring (Hardware and non workload

specific)

Maintenance Wizard (Launch point integration)

User Authentication (For console roles/access only)

Workload Monitoring OPM (SSO, Debranding, LIC)

License Acceptance On initial interface access

System Console

Management

Monitoring

Maintenance



Simplified maintenance with pre-integrated fixes

Single point of contact for support

Automated updates for faster maintenance

All hardware firmware and OS software patches integrated

and tested together at the

factory



Enhanced Information Center

Information is organized according to topic area and operational lifecycle

Overview, Planning, Getting Started, Operational Tasks, Advanced Tasks,

Troubleshooting, Reference

Each section has further detail, and links to relevant information

Easily searchable within the Information Center

Easily searchable from any search engine

Ability for users to enter comments on topics and participate in

discussions



Information Center Sample comment



HA tools

hareset

Soft reset will bring the domain to a base state

Clear failed, stuck, pending states - remove resource group requests across entire configuration

Rebuild HA configuration completely

drop the domain, create domain, create all resources/groups/relationships/equivs etc.

hachkconfig

Detect and repair resource model problems

Detect OS configuration issues that impact HA and alert

Can be run automatically or manually



High Availability Configuration

Nodes are grouped into HA groups consisting of 4 servers where 3 are active

Each and every server can access the storage of all other servers in its group

One failure per HA group is supported.

No performance hit when running in failover

Failback is NOT required after failed node is repaired

Passive Node

Active Node

Active Node

Failover

Active Node

Failover Failover

Active Node

Active Node

Active Node

Failover

Passive Node

Failover Failover


2012 IBM Corporation 58

Improved

Cost Efficiencies

Real-time

Operational Analytics

Improved Productivity

and Manageability

Clients have experienced cases of 10x storage space

savings via Adaptive

Compression

Multi-Temperature data management provides

improved efficiencies and

performance

Integrated and simplified system monitoring and maintenance

Simplified support for multi-tenant operational warehouses

Row and Column access controls to support multiple tenant operational

warehouses

Faster, more accurate decision making with real-

time operational analytics

Continuous Ingest of data

Built-In Time Travel query enabling faster historical and

trend analytical queries

DB2 10 Key Technology Improved efficiencies, performance and Manageability

Need webcast troubleshooting help? Click attachments


Multiple Instances of at least 3x Faster Query Performance*

Multi-core parallelism enhancements

Performance improvements for:

Queries over star schemas

Queries with joins and sorts

Queries with aggregation

Hash joins

Higher performance

Up to 35% faster out-of-the-box performance

Multiple instances of at least 3x faster when using new features *

* Based on both external tests by partners, as well as internal tests of IBM DB2 9.7 FP3 vs. DB2 10.1 with new compression features on P6-550 systems with

comparable specifications using data warehouse / decision support workloads, as of 29 Mar 2012.

IBM and Intel have collaborated over a decade to optimize DB2 performance with Intel Parallel Studio 2011, software development suite on Intel Xeon processors. We are excited to see a ~10x improvement in query processing performance using DB2 10 over the previous DB2 version, running on IBM System x3850 using Intel Xeon Processor E7. Customers can now realize dramatically greater performance boost at

lower cost per query running IBM DB2 10 on servers powered by Intel Xeon processors. Pauline Nist, GM Software Strategies, Intels Datacenter & Connected Systems Group



Save time and cost Reduce Storage Requirements

Database Compression: Clients have experienced cases of 10x storage space savings

Backup Compression: Shrink backup and disaster recovery storage

Improved Performance

Workloads run against compressed data result in performance improvements

Significantly reduces I/O requirements to process queries

Reduce Costs

Less storage hardware,

Smaller footprint

reduced power consumption

Advanced row compression technique with two compression dictionary levels:

table-level

page-level

Improves compression as data changes

Adaptive compression



Your Feedback is Important!

Access SmartSite to complete your session surveys

Any web or mobile browser at iodsmartsite.com

Any SmartSite kiosk onsite

Each completed session survey increases your chance to win an Apple TV with daily drawing sponsored by

Alliance Tech



Visit the PureData Systems Web page

bit.ly/pure_data

Read the materials

PureData Systems White Paper: The Big Data Zoo Taming the Beast ibm.co/TDl9sV

Read the data sheet ibm.co/TDlhsh

Watch IBM developerWorks for additional content at ibm.com/developerworks/data

Take the Test Drive ibm.co/RVYaeI

Connect with the community!

Information Management bit.ly/InfoMgmtCommunity

Business Analytics bit.ly/AnalyticsCommunity

Enterprise Content Management bit.ly/ECMCommunity

IBM Champions ibm.com/champion Recognizing individuals who have made the most outstanding contributions to Information Management, Business Analytics, and Enterprise Content Management communities

DB2 Tech Talk: Next Steps Roadmap: PureData Systems

Reference

Call IBM to schedule a demo or learn more

1 800 966-9875 (U.S) 1-888-746-7426 (Canada) 1800-425-3333 (India) Or visit http://www.ibm.com/planetwide/ for contact information worldwide

Information Management

Tech forums on developerWorks

ibm.co/ZEt7Yw

Serge Rielau SQL Tips Blog:

www.sqltips4db2.com

Step Three

Step Two

Step One

Step Four


2012 IBM Corporation 63 63

Upcoming Tech Talks Dont miss these in-depth DB2 10.1 feature talks!

Row and Column Access Control (RCAC) in DB2 10 and InfoSphere Warehouse 10

Walid Rjaibi, IM Data Security Chief Architect

December 12, 2012

Reg: bit.ly/tt2012dec

Stay Tuned for the 2013 schedule!

Dates and topics subject to change and modification.

How to register :

DB2 Tech Talks web site



Questions

Listening in replay?

Questions: www.sqltips4db2.com

Click submit a question.



Thanks for attending!

Please rate the session

Presentation download: bitly.com/ttfilenov

Next Steps document: click the attachments button


db2 tech talk puredata systems presentation.pdf

Documents

ibm corporation puresystems

architect ibm puredata

ibm corporation integration

use of ibm software

standard ibm benchmarks

ibm serges blog

systems lifecycle

software simplified