season 1 : the foundations -...

05/08/09

1

© 2009 Oracle Corporation

Oracle/IBM Integrated Solutions Center productionpresents

Oracle’s Maximum Availability Architecture (MAA) on IBM System z running Linux

Season 1 : The Foundations

2 05/08/09

Many Thanks toFrank Almarcha, IT Specialist, IBM System z benchmark Center

Jesus nunez, Oracle ACS Principal Service Delivery Engineer

Pierre Yves Urena, IT Specialist, IBM Testing Center

Fabienne Lepetit, Senior Performance Specialist, Oracle, JSC

Sanjay Singh, Senior Principal Technical Support Engineer, RAC Assurance Team

Special thanks toNicolas Marescaux, IT Specialist - z/VM - Linux on IBM System z, JSC

Casting.David J Simpson, Certified IT Specialist, Oracle Solutions, IBM ATS

Marc Connolly, Technical Development Director: Oracle/IBM Integrated Solutions

Sam Amsavelu, System z Linux Oracle Solutions Pre-sales Support, IBM ATS.

Frederic Dubois, IT specialist, Oracle Solutions on Linux, JSC

Paul Bramy, Technical Development Director: Oracle/IBM Integrated Solutions

05/08/09

2

3 05/08/09

Agenda• Oracle/IBM Joint Solutions Center• IBM System z running Linux: Standardize & Virtualize Oracle

Solutions• IBM System z running Linux & Oracle Maximum Availability

ArchitectureOracle Clusterware (CRS)Oracle Automated Storage Manager (ASM)Oracle Real Application Cluster (RAC)Oracle DataguardOracle Enterprise Manager (OEM)

• IBM/Oracle Integrated Architecture summary.• Customer projects

4 05/08/09





05/08/09

3

5 05/08/09

IBM/Oracle Joint Solutions Center: The team

Technical foundationsNicolas Marescaux - System z Thierry Plumeau - System p Fréderic Dubois - System x

Fabienne Lepetit Daniel Freneuil

Oracle Grid / MAA Database Infrastructure & Performances

Oracle Middleware/SOA Infrastructure

Fréderic Michiara

Oracle Applications AIA

Marc Connolly

Christophe Menichetti Didier Wojciechowski

Alain Cyr

Carla Gamba

Paul Bramy

Christophe Maldy

MatthewPuccini

Managers

Architects

ProjectsManagement

6 05/08/09

Customers are already « Building Up »…

Business Logic Protection

Integrate existing z/OS applications through Oracle Integration tools (OAM for CICS)

Middleware integration

Infrastructure Modernization

Oracle Solutions integrated to z/OS legacy System

Simplify global infrastructure on IBM System z with Oracle Solutions

Infrastructure Simplification to reduce Complexity & increase resources utilization.

Standardize your Database pool, Oracle & MS SQL, to simplify management.

Infrastructure Simplification

Increase footprint with System z

Application based on Oracle 10g Database & 10g AS + SOA can be fully implemented on

System z running Linux

More ISV evaluate their application on System z running Linux

New Workloads on System z - Linux

Applications on System z LinuxLeverage technology expertise

for their core Applications

Home made Oracle application -> Full System or split tier

Oracle iFlex -> Full System z architecture

Oracle EBS, PeopleSoft, Siebel -> Split Tier

05/08/09

4

7 05/08/09

Oracle/IBM Integrated Solutions CentersCustomer Services

Technical Training

Demonstrations

Design Workshops

Light Benchmark for Oracle

Proof of Concept

Benchmarks

8 05/08/09

IBM Light Benchmark offering for Oracle solutions on System z

Purpose: Proof of Concept• Validation of new System z solutions • Validation of new Oracle releases• Validation of new System z features usage• Validation of customer applications before a

benchmark.

Terms• 1 to 4 weeks• Platform (remotely through Open VPN)• Oracle AS on one Linux guest + Oracle RAC on 2

guest Linux (cluster), 4GB and 4CP per guest

Support• First level support provided by IBM tech center staff• Second level support from System z Benchmark

Center and the Oracle/IBM Joint Solution Center

hipersocket

Oracle SOA

J2EE engine

http ServerASMASM

RAC

Linux guestSLES 9 4CPs shared 4GB Mem

Linux guestSLES 9/10RHEL 4/5 4CPs shared 4GB Mem

Linux guestSLES 9/10RHEL 4/5 4CPs shared 4GB Mem

05/08/09

5

9 05/08/09

Customer expectations

Oracle / IBM Integrated SolutionsWhat we learned from our customer engagements.

Reliability11

Open Standards22

High-Availability33

Agility / Provisioning 55

Security66

Performance77

Virtualization44

Integration 99

Manageability / Monitoring88

Application Portfolio1010

IBM

Z10 Architecture

LINUX

Z10 + Z/VM + LPAR

z/VM

Crypto Card

Z10 processor and Architecture

IBM Hardware

Oracle

LINUX

Oracle RAC, CRS

Oracle’s Grid Architecture (ASM,CRS,RAC)

Data Vault / Crypto Integration

Oracle DB and Options

Audit Vault / GridControl

Oracle TG

Oracle Applications

10 05/08/09




• IBM/Oracle Integrated Architecture summary.• References

05/08/09

6

11 05/08/09

Virtualized Oracle Solutions with IBM System z running Linux (1/2)

• Application support layerOpen, reliable operating systemVirtual server awareness infrastructureEnterprise applications

• Hypervisor layer (z/VM)Shared-memory based virtualization modelHighly granular resource sharing and simulationFlexible virtual networkingResource control and accountingServer operation continuity (failover)Server maintenance tools and utilities

• Hardware layer (z/Architecture)Legendary reliability, scalability, availability, securityLogical partitioning (LPAR)Processor and peripheral sharingInter-partition communicationVirtualization support at the hardware instruction level (PR/SM)

Application Layer

Hypervisor Layer

Hardware Layer

12 05/08/09

Virtualized Oracle Solutions with IBM System z running Linux (2/2)

Phase I Phase II Phase III Phase IV

Project

Database designDatabase schema

User responsibilitiesData protection

…/…

ProjectImage

project Layer

z/VM Memory,

ProcessorsNetwork

DisksSecurity

…/…

z/VMImage

Z/VM Layer

Linux Linux distribution

Linux configurationRPM

OS User privilèges…/…

OSimage

Linux Layer

Oracle

Oracle productsOracle setupsOracle patches

Database security…/…

OracleImage

Oracle Layer

CustomizedBuilding

Block

CustomizedBuilding

Block

Marketplace unique capability: Linux provisioning using z/VM virtualization

05/08/09

7

13 05/08/09

What does System z bring to Linux• The most reliable hardware platform available

Redundant processors and memoryError detection and correctionRemote Support Facility (RSF)

• Designed to support mixed workloads Allows consolidation while maintaining one server per applicationComplete workload isolationHigh speed inter-server connectivity

• Centralized Linux systems are easier to manage• Scalability

System z10 EC scales to 64 application processors System z9 EC scales to 54 application processorsSystem z9 BC scales to 7 application processorsUp to 8(z9) 11(z10) dedicated I/O processors Hundreds of Linux virtual servers

14 05/08/09

Linux gives Choice

OpenPowerSUN, HP

iSeries

z/OS OS/400

Power(PA RISC, SPARC)Intel, AMD z/ArchitectureChip

H/W

OS Windows

BladeCenterIBM – x86, PowerHP/Dell – x86 only

iSeries/i5pSeries &

OpenPower(HP, Sun)

z/OS

xSeriesHP, Sun, Dell

J2EE and Open Standards

Middleware

Applications

Linux

IBM System z

Support of Linux on all IBM servers

05/08/09

8

15 05/08/09

Integrate your System z in your SAN infrastructure

• z/VM support for SCSI industry-standard devicesSystem z attachment to SCSI devices is provided by FCP devicesz/VM provides native support to SCSI disks for paging, spooling and other system devicesSupport is provided by emulating SCSI disk LUNs to VM as 9336 FBA 512-Bytes blocks DASDIPL, Dump, and Service from/to SCSI disk LUNs is provided to achieve a SCSI-only VM environment.SCSI-only as well as mixed SCSI and ECKD environments are supportedz/VM provides support for multipath in order to take advantage of hardware redundancy.

16 05/08/09


z/VM multipath implementation

05/08/09

9

17 05/08/09


• Linux on IBM System z supports FCP devices in the following configuration:

Native LPAR modeAs a z/VM Guest

• When running as z/VM guests, Linux systems can use SCSI devices by the following ways:

LUNs Direct attachment As emulated 9336 FBA 512-byte block DASD

• Linux uses zfcp driver to exploits SCSI architecture• SCSI-only as well as mixed SCSI and ECKD

environments are supported

18 05/08/09


• Linux on IBM System z supports :Multipath to SCSI devices:

multibus mode (round Robin)failover mode (recommended)

N_PORT ID Virtualization (NPIV)Allows FCP WWPN VirtualizationEnable standard method for SAN infrastructure access restriction (LUN masking, Zoning)Available on z9EC, z9BC and z10 processor typesRequires NPIV-enabled SAN switches

SAN Volume ControllerVirtualizes your storageUse multiple Storage subsystems (different models from differents vendors)

05/08/09

10

19 05/08/09

System p

System zSystem x DS8000

EMC HP Hitachi SUN IBMSystem z in switched fabric topology

20 05/08/09

Customers are already « Building Up »…

Business Logic Protection

Integrate existing z/OS applications through Oracle Integration tools (OAM for CICS)

Middleware integration

Infrastructure Modernization

Oracle Solutions integrated to z/OS

Simplify global infrastructure on IBM System z with Oracle Solutions

Infrastructure Simplification to reduce Complexity & increase resources utilization.

Standardize your Database pool, Oracle & MS SQL, to simplify management.

Infrastructure Simplification

Increase footprint with System z

Application based on Oracle 10g Database & 10g AS + SOA can be fully implemented on System z running Linux

More & more ISVs evaluate their applications on System z running Linux

New Workloads on System z - Linux

Applications on System z Linux

Leverage technology expertise

for their core Applications

Oracle iFlex : Full System z architecture

Oracle EBS, PeopleSoft, Siebel : Split Tier

05/08/09

11

21 05/08/09

Available Oracle solutions on IBM System z running Linux

Linux on System z

Oracle Database

FusionMiddleware

Applications

Enterprise Manager

• Data SolutionsOracle Database EEOracle Database SEOracle Database clientOracle Warehouse BuilderOracle Business Intelligence EE (Split tier)

• Middleware SolutionsOracle Application ServerOracle Containers for J2EE (OC4J)Oracle Top LinkOracle AS Metadata Repository Creation Assistant

• Management SolutionsOracle ClusterwareOracle Configuration ManagerOracle Grid Control Agent

• Integration SolutionsOracle Transparent Gateway for DRDA

• ApplicationsPeoplesoft Enterprise (Split tier)Siebel (Split tier)eBusiness Suite (Split tier)

• Industry SolutionsiFlex

10gR2 64-bit10gR2 64-bit10gR2 64-bit10gR2 64-bit10gR2 64-bit

10g (10.1.3.1.0)10g (10.1.3.1.0)10g (10.1.3.1.0)10g (10.1.3.1.0)

10gR2 64-bit10.2.6.0.010gR2 64-bit

10gR2 64-bit

8.498R12

Flexcube

22 05/08/09

Supported Linux Distributions on IBM System z for Oracle

05/08/09

12

23 05/08/09





24 05/08/09

Oracle MAA: Clusterware Architecture (1/3)

• Oracle Clusterware is a complete cluster software solution

• Including advanced functionality:Failure Notification (FaN)Support for 3rd party cluster softwareHA-API for all kind of applicationsFully integrated with Oracle RAC

• Low cost and flexibility:No need to purchase additional softwareEasy to install & to manageSupports 100 nodes on all OS’ certified for Oracle RAC Single-vendor support

Applications

Services Framework

Cluster Control/Recovery APIs

Automatic Storage Management

Messaging and Locking

Membership

Connectivity

Database

Even

t Ser

vice

s

Man

agem

ent A

PIs

Hardware/OS Kernel

RAC APPS

05/08/09

13

25 05/08/09

140.84.20.05 ORA-VIP140.84.20.04 ORA-VIP

140.84.20.06 ORA-VIP

SQL Cache

Large Pool

Buffer Cache

Java Pool

SGA Pool

SQL Cache

Large Pool

Buffer Cache

Java Pool

SGA Pool

Cluster Synchronisation Services (CSS)Basis for Inter-process & -node-operations, Node Monitoring & Membership

Hardware / Operating System Kernel

Cluster Ready Services (CRS)HA Framework

Virtual IP addresses Instances ProcessesONS

140.84.20.02 APP-VIP140.84.20.01 APP-VIP

140.84.20.03 APP-VIP

Oracle MAA: Clusterware Architecture (2/3)

26 05/08/09

Monitor

Protect

Restart

Failover

Relocate

Oracle MAA: Clusterware Architecture (3/3)Increase Linux serviceability and availabilityImprove the availability of your applications thru CRS services

SCRIPTS

Application VIPRules / Dependencies

Protected ApplicationCustomized scripts

Linux 1

Shared disks for cluster management

Cluster interconnect (private network)heartbeat

Oracle Clusterware

Virtual IP 1

R/W R/W

SCRIPTS

Application VIPRules / Dependencies

Protected ApplicationCustomized scripts

Linux 2

Oracle Clusterware

Virtual IP 1Virtual IP 2

05/08/09

14

27 05/08/09





28 05/08/09

Oracle MAA: Automated Storage Manager (1/2)

ASMNon ASM

0010 00100010 0010

Raw Disk Groups LVs

FileSystems

Files Tablespace

Tables

Eliminates need for conventional file system and volume managerASM extends SAME (Stripe and Mirror Everything)Improved performance, scalability, and reliability

Disk 3

Disk 2

Disk 1

Disk 3

Disk 2

Disk 1

Without ASM With ASM

Provisioning storage when you need it… and save money

Conventional wisdom

Disk group

Oracle DB Instance

ASM Instance

ASM is Oracle’s integrated Clusterware

• Capacity on demandAdd/drop disks onlineAutomatic I/O load balancingStripes data across disks to balance loadBest I/O throughputAutomatic mirroring and stripping

• Easy to manage• Can only host Datafiles, not binaries

05/08/09

15

29 05/08/09

Oracle MAA: Automated Storage Manager (2/2)

Disk Group DB 01

Shared Flash Recovery Aera

Disk Group DB 02

ASM

Z/VM

SingleInstance

DB01

CRS

+ASM

IFL

Memory

OSAZ/VM

LPAR

hipersocket

VIP01

Voting OCR

CRS

+ASM

VIP02

Failoverconfiguration

Integrated in CRS to be used as a Cluster Logical Volume Manager

Take advantage of running Linux under z/VM to add/remove storage in a maximum dynamic way

Manage your Disk Groups directly from a central point of administration: the Grid Control Server (Oracle Enterprise Manager)

30 05/08/09





05/08/09

16

31 05/08/09

Oracle MAA: Real Application Cluster (1/4)

• Complete Oracle cluster software solutionActive/Active Cluster Databases Up to 64 nodes

• Single-vendor support• Cheaper

No need to purchase additional softwareEasy to install, manage

• High quality and functionality across all platforms• Support for third-party Cluster Filesystem

32 05/08/09


AvailabilityActive-Active solutionOnly fractional capacity lost

when a node in the cluster failsScalability

Add additional Instance as your workload grows

PerformanceQuery parallelization across

nodes in the clusterWorkload Distribution

Automatic servers-side load balancing

Manageability Reduce Database downtime

during maintenance phases

ASM

Z/VM

RACInstanceRAC01

CRS

+ASM1

IFL

Memory

OSA

Z/VM

LPAR

hypersocket

VIP01

Voting OCR

Physicalresources

RACInstanceRAC02

CRS

+ASM2

VIP02

RAC – Cache Fusion

Services

Cluste-rware

DatabaseInstances

StorageManagement

RedologsUndo TBS /instance

05/08/09

17

33 05/08/09


Database FilesDatabase Files

RedoRedoLogs (1)Logs (1)

RedoRedoLogs (N)Logs (N)

Shared Disk SubsystemShared Disk SubsystemSystem z

Linux

IFL IFL IFL IFL

Linux Linux Linux Linux

OracleClusterwar

e

OracleClusterwar

e

OracleClusterwar

e

OracleClusterwar

e

OracleClusterwar

e

ASM Instance 1

ASM Instance 2

ASM Instance 1

ASM Instance 3

ASM Instance 4

ASM Instance 5

10g RACnode1

LPAR 5LPAR 1 LPAR 2 LPAR 3 LPAR 4

10g RACnode5

10g RACnode4

10g RACnode3

10g RACnode2

IFL IFLIFL IFL IFLIFLIFL IFLIFL

34 05/08/09

Oracle RAC on an IBM system z“Virtualized Machines”

“Extended” Oracle RAC on IBM System z servers

Oracle RAC in a LPAR

z/VM

RAC

z/VM z/VM

RAC

……

z/VM z/VM

RAC

Application and datacenter Scalability

Availability


05/08/09

18

35 05/08/09





36 05/08/09

Apps

VirtualizationOracle Standby

DatabaseOn virtual

environment

Apps

Virtualization

Oracle RAC on Virtualized environment

Apps

Virtualization

Disaster SiteProduction SiteDataguard

Real Application Clusters

Oracle MAA: Dataguard (1/2)

05/08/09

19

37 05/08/09

Oracle MAA: Dataguard (2/2)

ASM

RACInstanceRAC01

CRS

+ASM1

Interconnect

VIP01

RACInstanceRAC02

CRS

+ASM2

VIP02

RAC – Cache Fusion

Services

Clusterware

DatabaseInstances

StorageManagement

Redo Apply

Logical Standby Database

SQLApplyRedo to SQL

Transform

Physical Standby Database

Broker

Data Center A

Data Center B

Data Center C

Disaster Recovery

Failover and Switchover Capabilities

Flexibility in data protection

Safeguard against data corruptions and users errors

Centralized and simple management

38 05/08/09





05/08/09

20

Template Documentation

Grid ControlConsole

Agent Agent Agent

ManagementRepository

Grid ControlManagement

Server

HTTP(S)

HTTP(S) HTTP(S) HTTP(S)

10g DatabaseControl Console

HTTP(S)10g Application Server

Control Console

HTTP(S)

ThinJDBC

Oracle 10g/11gDatabase

Server

OS/ Third-PartyApplication

Oracle 10gApplication

Server

Managed Targets

Firewall

Firewall

Agent

Oracle 8i/9iDatabase

Server

Oracle MAA: Enterprise Manager (Grid Control)

40 05/08/09

Z/VM

SingleInstance

DB01

CRS

Z/VM

CRS

+ASM

RACRACDatabaseDatabase

Z/VM

RACInstancerac01

CRS

+ASM

CRS

+ASM

SingleInstanc

eDB01

RACinstanceRAC01

+ASM

Data Data GuardGuardDatabaseDatabase

SingleDataguard

DG01

IFL

Memory

OSA

Z/VM

LPAR

IFL

Memory

OSA

LPARZ/VM

+ASM +ASM +ASM +ASM +ASM

SingleSingleDatabasesDatabases

SingleDatabases Data Data GuardGuard

DatabaseDatabase

RACRACDatabaseDatabase

z/VMControls Virtualized Secure access to secure shared resources

System z & z/VMServer provisioning, secure access, shared resources

Oracle DB zVM Virtual Linux Server

Oracle ClusterwareServer Clustering Oracle Real Application Cluster

( RAC )Protect Against Data and

Application Server Failures

Oracle Automatic Management ( ASM)

Oracle Automatic Storage Management ( ASM)

Protect Against Storage Failure

Oracle Real Application Cluster( RAC )

Oracle Data GuardProtect Against Total System

and Site Failures

Oracle Grid Control Applicationand System Monitoring

Oracle Grid AgentOracle Grid Agents

Oracle VM

Oracle Grid AgentIBM System x & Oracle VM

Maximum Availability Architecture (MAA)IBM/Oracle Integrated Architecture

05/08/09

21

41 05/08/09





42 05/08/09

References• Oracle Solutions on System z running Linux

Experiences with Oracle 10gR2 Solutions on Linux for IBM System zhttp://www.redbooks.ibm.com/abstracts/sg247191.html?Open

Using Oracle Solutions on Linux for System z http://www.redbooks.ibm.com/redpieces/abstracts/sg247573.html?Open

Experiences with Oracle Solutions on Linux for IBM System zhttp://www.redbooks.ibm.com/redpieces/abstracts/sg247634.html?Open

Linux multipathing: failover mode with preferred pathshttp://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP101420

• System z I/OI/O Connectivity

http://www-03.ibm.com/systems/z/hardware/connectivity/products/fc.htmlIBM System Storage SAN Volume Controller

http://www-03.ibm.com/systems/storage/software/virtualization/svc/index.html

05/08/09

22

43 05/08/09

References• FCP/SCSI implementation on IBM System z

System z Fiber Channel Protocol for Linux and z/VM on IBM System zhttp://www.redbooks.ibm.com/abstracts/sg247266.html

Introducing N_Port Identifier Virtualization for IBM System z9http://www.redbooks.ibm.com/abstracts/redp4125.html

Cloning FCP-attached SCSI Disks on SLES9 Linux for zSeries Systems

http://www.redbooks.ibm.com/abstracts/redp3871.html• Oracle

Products Certification Matrixhttp://www.oracle.com/technology/index.html

Oracle Technology Networkhttp://otn.oracle.com

Metalinkhttps://metalink.oracle.com/CSP/ui/index.html

44 05/08/09

Questions

Thank you.

05/08/09

23

© 2009 Oracle Corporation

Oracle IBM Integrated Solutions center productionpresents

Oracle’s Maximum Availability Architecture (MAA) on IBM System z running Linux

Season 2 : Customer Study Cases


• Database hosted on Linux on system z

• Application tier potentially on Linux on System z

• Additional components or application on Linux on System x

• But always … an End to End

• Virtualized

• Linux

• Provisioned

Architecture

Application Architecture within System z running z (1/2).

05/08/09

24

47 05/08/09

z/VM

IFLs

IBM System z10

LPAR

z/VM

LPAR

OVM ServerOracle VM Hypervisor

Oracle VM

Server(Dom 0)

OVM ServerOracle VM Hypervisor

Oracle VM

Server(Dom 0)

Linu

x

Virt

ualiz

atio

n

IBM Storage

Dat

abas

e Ti

erA

pplic

atio

n Ti

er

IBM System x3850M2

Production Site

Provisioning

Linux Guest

Interconnect

Hipersocket

Dataguard

DR Site

VM

Interconnect

VM

Oracle ASMOracle CRSOS / Linux

Oracle ASMOracle CRSOS / Linux

Template Documentation 48 8-May-09

Use Case: Oracle DatabaseAdvance Security SSL

andSystem z Crypto Support

(PKCS11)

Marc Connolly

05/08/09

25


• Database to Database to Client Authentication

• File System Wallet (Credential) Support

• PKCS #11 openCryptoki Support for Linux

• HSM Support for Secure Key Storage

Oracle Database Advanced Security SSL


Symetric Algorithms (Shared Secret Key)

Asymmetric algorithms (Public Key Cryptography)

Cryptographic "checksum" with one way algorithms (data integrirty)

MD5 (128-bit hash)

M AC, MDC

SHA-1(160-bit hash)

SHA-224 (224-bit hash)

SHA-256 (256-bit hash)

SHA-384, SHA 512

RSA (512, 1024, 2048, 4096 bits)

DSA (512, 1024 bits)

Authentication, key

distribution (VPN,

SSL/TLS handshake, ..)

DES (56 bits)

T-DES (168 bits)

AES (128, 192, 256 bits)

Data transfer (VPN, SSL/TLS, ..)

Data storage (database,

archives, …)

05/08/09

26


SSL Handshake

PKI (Asymmetric Key) Operations are Very Expensive!

Linux

DBInstance

Listener

PKCS#11openCryptoki*

libica

z90crypt

z/VM

PCI CryptoAdapter

JavaIBMPKCS11Impl Provider

Class

OracleASO

JavaJCE

OracleSQL*Plus

/oracle/admin/DB01/wallet/Wallet_serverOracle PKCS#11 Wallet

./build/conf/JKS/client.jksJava PKCS#12 keystore

/oracle/admin/DB01/wallet/Wallet_clientOracle PKCS#12 Wallet

OracleJDBC

CPACF

*/usr/lib/pkcs11/PKCS11_API.so64

PKCS#11openCryptoki*

• Accelerated Asymmetric Key Operations

• Secure Key Operations

05/08/09

27

53 05/08/0953 05/08/09

Agenda

SAM AMSAVELU• Customer overview• PoC

• Environment• Testing• Issues and Solutions

• IBM / Oracle MAA Design Review• Recommendations• Q&A

54 05/08/0954 05/08/09

Customer overview• US based financial company• Customer in Unix environment

• 132 processors• 164 GB memory

• SAN storage• 500 GB database size

• Application• Custom developed• Over million transactions during peak hours

• Problem• Vertical Scalability

05/08/09

28

55 05/08/0955 05/08/09

PoC Environment• System z (30 + IFLs)• Linux (SLES 10)• Oracle 10.2.0.3• Stand alone database• Grid agent• Custom developed application

• Lot of chattering between application and database • Application was divided into a sub-set application layer to co-

exist along with database layer

56 05/08/0956 05/08/09

PoC Testing • Simulated work load• Single LPAR with just Linux

• Co-existing sub set application• Database Server

• Two Linux guests under z/VM• sub set application at one Linux guest• Database Server at another Linux guest

• Production test to simulate expected peak load for a period of 20 minutes

• 25%, 50%, 75%, 100% load• Throughput time, pending txns at any time

05/08/09

29

57 05/08/0957 05/08/09

Problems / solutions during PoC• Every transaction needs a commit (logs filled too

quickly, log I/O activity in the top 5 time event)• Redistributed the redo log files,• Increased the redo log file size• Relocate undo tablepsace

• I/O issues• Increased the PCTFREE for some indexes• disk_async_io=true• filesytemio_options = setall• unset DB_FILE_MULTIBLOCK_READ_COUNT

58 05/08/0958 05/08/09

Problems / solutions during PoC• Flash recovery

• Disable the flashback recovery • SGA adjustment:

• Shared_pool, • db_cache_size (fixed)

• Identified poor performing SQL queries• Recommended to customer to change the select statements

• Lot of buffer gets • TB Scan

• Upgraded from oracle 9i • Removing the index hints

05/08/09

30

59 05/08/0959 05/08/09

Oracle MAA design review• Design Review

• Oracle and IBM conducted a MAA design review with the customer

• Two days on-site review

• What we found• No virtualization • No ASM• No High Availability• No DR• Basically No MAA

60 05/08/0960 05/08/09

Oracle MAA design review• IBM / Oracle joint recommendations

• Upgrade to 10.2.0.4• Fix the identified queries• Explore the possibility to use Stored Procedures • Optimize some more Oracle initialization parameters (9i to 10g)• Work with the customer to optimize SQL*NET messages• Segregate DB server from AS• Implement ASM for storage management• Implement CRS for single instance protection• Implement RAC for the long run• Implement disaster recovery using Data Guard

05/08/09

31

61 05/08/09

Client #David J Simpson

• 24-IFL z10 – Oracle RAC and ASM running under Red Hat 4 with Disaster Recovery to another Data Center.

• Mission Critical System.• DASD Storage – RAC under z/VM using ASM

• Biggest Pitfall – Mini Disk Cache On when using 2 LPARs different SCN (System Change Number) between LPARs caused database crash.

• Why Project was so successful. ASM, RAC, z/VM IFLs, hit the dates with flying colors by being able to provision quickly.

62 05/08/09

Client #2

• 8 - z9 IFLS – db2 transactional database via Hipersockets to Oracle Data warehouse under System z Linux, VM 5.3

• Cut Database load time from db2 to distributed servers from days, and their key seasonal peak load update period from a week to complete to one day.

• Biggest Pitfall – Testing and inefficient code, threw hardware at the problem which did not scale well.

• Why Project was successful. ASM’s ability to spread workload across many disk storage devices. The ability to leverage Hipersockets, and the fact they had no more data center floor center space, they had the mainframe with capacity and they simply added IFLs.

05/08/09

32

63 05/08/09

Almost Live Client #3

• 100 IFL – z10 Oracle RAC environment across – 2 z10’s with Oracle ASM. 16 IFLs per machine for Oracle Production.

• 35 TB Database and 45 TB Flash Recovery Area (FRA)

• Pitfall – Upgraded to 10.2.0.4 and hit oprocd.bin reboots, set Diagwait to 13, and had issues with z/VM cleaning pages suspending the Linux guest for up to 12 seconds causing node evictions.

• Success Factor - Project is getting very high I/O throughput inserting 5.79 billion records in a 7 hour window and updating another 320 million records which allowed them to hit their SLAsfor the 5 year projection levels – with room to grow.

64 05/08/09

Large European company , with branches in many countries.

Large Oracle customer: running Oracle on Unix and z/OS.Main site : Production z9 713 + 6 zIIPs + Development z9 709 + 5 zIIPsSecond Site: Disaster Recovery + Development z9 703 + 3 zIIPs + CBUs

Core business application, completed in live

in October 2007, Oracle announced end of developments for z/OS.

Customer consulted partners and market influencers move to Unix.Stay on system z (running Linux)

Customer experiencesCustomer’s context

05/08/09

33

65 05/08/09

• Step 1 : First Architecture Workshop – Kick-Off – on site - May 2008Study different architecture options Define Project plan (System z Oracle Light Benchmark, On-site PoC)

• Step 2 : System z Oracle Light Benchmark – June 2008Remotely accessible PoC Environment offering created by PSSC.Goals: test different technical options & components, validate Migration process

• Step 3 : Design Workshop – on site – July 2008Validate Outcomes Light benchmark Review PoC architecture and define migration planBuild project plan for performance & sizing benchmark in MOP

• Step 4 : Customer’s onsite Proof-of-Concept – July/August 2008In depth additional functional tests + MOP Benchmark Kit development With remote support from PSSC

Customer experiencesProject plan

66 05/08/09

Customer experiencesBenchmark scope : Customer objectives

Assess the feasibility of migration and performance of main Customer application to a new platform based on Linux on System z and Oracle. The benchmark should allow platform configuration sizing, in order to deliver the Application backend service with no degradation, from the user's perception, compared to current production on zOS.

System z10

05/08/09

34

67 05/08/09

The benchmark was made of multiple assessments:

• OLTP: Based on “Customer in house” workload generation tool reproducing transactions for a typical working day

• Batch: End of month process. End of week process. Critical daily process.

• Migration: Export and rebuild all databases objects from zOS to linux on system z within a dedicated window timeframe (< 40h)

• Scalability: Assess the platform behavior with a workload increased up to 150% of the actual OLTP workload.

Customer experiencesBenchmark objectives and phases

68 05/08/09

Customer experiencesTargets and achievements

OK

OK

Status

Comparison with production elapsed time: • End of month: 46% improvement• End of weekl: 60% improvement• Daily process: 33% improvement

95% successfully injected transactions (>1.18M):• 90% with better response time

Results

Measure the elapsed time of 3 batch processes scheduled from zOS ctrl-M to ORACLE linux on System z databases.

Batch

Assess the capability of the new platform to support the OLTP workload based on injected transactions captured during a peak day on production system

OLTPObjective

Infrastructure: Linux on System z LPAR: 18CPs / 80GB - 30 % CPU utilization

Infrastructure: zOS LPAR: 4CPs / 80GBLinux1 on System z LPAR: 12CPs / 40GBLinux2 on System z LPAR: 12CPs / 40GB

CPU at peak load: 40% on zOS, 60% on Linux1, 100% on Linux2

05/08/09

35

69 05/08/09

Customer experiencesTargets and achievements

OK

OK

Status

Linearity up to 125% scaling factor.Additional tuning has to be done to improve 150% results.

Migration process achieved in 15 hoursincluding Database statistics updates

Results

Validate the new Infrastructure capability in terms of scalability

Scalability

Migrate the full databases from zOS to linux on System z within 40 hours.

Migration

Objective

Infrastructure: zOS LPAR: 4CPs / 80GBLinux1 on System z LPAR: 12CPs / 40GB

CPU utilization at peak load: 85% on zOS, 100% on Linux1

Infrastructure: Linux on System z LPAR: 18CPs / 80GB, except for the 150% test (28 CPs)

CPU utilization: 70%

70 05/08/09

OK

OK

Status

more than 95% successfully injected transactions~75% with better response time ** Only 4 runs have been done on RAC, Additional Database tuning detected.

more than 95% successfully injected transactions~80% with better response time

Results

Validate Application in ORACLE RAC environment with 2 nodes** Application never tested on RACFirst experience in Cluster mode.

RAC

Assess zVM capability to host multiple Linux guests Running application with workload 125%

zVMObjective

Infrastructure: zVM LPAR 18CPs / 80GBLinux on System z LPAR: 18CPs / 80GBCPU utilization: 80%

Infrastructure: zVM LPAR 28CPs / 84GB2 guest Linux: 18CPs / 40GB,

CPU utilization: 55% for each Linux & 70% for zVM

Customer experiencesTargets and achievements ( Additional test added in the test plan)

05/08/09

36

71 05/08/09

Questions

Thank you.

season 1 : the foundations -...

Documents