hpe reference configuration for oracle 12c on hpe synergy

HPE Reference Configuration for Oracle 12c on HPE Synergy composable infrastructure using HPE 3PAR StoreServ storage

Technical white paper


Contents Executive summary .................................................................................................................................................................................................................................................................................................................................3 Introduction ....................................................................................................................................................................................................................................................................................................................................................3 HPE Synergy for Oracle ..................................................................................................................................................................................................................................................................................................................... 4 HPE 3PAR StoreServ all-flash array ........................................................................................................................................................................................................................................................................................ 5 HPE Synergy Composer .................................................................................................................................................................................................................................................................................................................... 6 Solution overview ..................................................................................................................................................................................................................................................................................................................................... 6 Solution components ............................................................................................................................................................................................................................................................................................................................ 9

Software ...................................................................................................................................................................................................................................................................................................................................................10 Best practices and configuration guidance for the Oracle Database solution ...............................................................................................................................................................................10 Capacity and sizing ............................................................................................................................................................................................................................................................................................................................... 11

Workload description .................................................................................................................................................................................................................................................................................................................... 11 Workload results ............................................................................................................................................................................................................................................................................................................................... 12 Analysis and recommendations ......................................................................................................................................................................................................................................................................................... 13

Summary ........................................................................................................................................................................................................................................................................................................................................................ 13 Implementing a proof-of-concept ........................................................................................................................................................................................................................................................................................... 13 Appendix A: Bill of materials ........................................................................................................................................................................................................................................................................................................ 14 Appendix B: Oracle configuration parameters – HPE Synergy 480 Gen9 ........................................................................................................................................................................................ 15 Appendix C: Oracle configuration parameters – HPE Synergy 660 Gen9 ........................................................................................................................................................................................ 15 Appendix D: Linux kernel configuration ........................................................................................................................................................................................................................................................................... 16 Appendix E: 3PAR permission udev rules ....................................................................................................................................................................................................................................................................... 16 Appendix F: 3PAR udev rules for SSD drives ............................................................................................................................................................................................................................................................... 17 Resources and additional links .................................................................................................................................................................................................................................................................................................. 18


Executive summary The demands of database implementations continue to escalate. Faster transaction processing speeds, scalable capacity, and increased flexibility are required to meet the needs of today’s business. At the same time, enterprises are looking for cost-effective, open-architecture, industry standard solutions that don’t include vendor lock-in or carry the high price tag attached to proprietary solutions.

HPE Synergy is a single infrastructure of pools of compute, storage, and fabric resources, along with a single management interface that allows IT to rapidly assemble, disassemble and re-assemble resources in any configuration. HPE Synergy architecture eliminates hardware and operational complexity so IT can deliver infrastructure to applications faster and with greater precision and flexibility.

The introduction of the HPE Synergy composable infrastructure products simplify this model by reducing the quantity of components and required skillsets needed for deployment. Deployment is accelerated by using HPE Synergy Composer for all interactions with the infrastructure. Pools of resources consisting of compute, storage and fabric can be deployed, managed and reconfigured using templates that outline the resources required for each server in a solution. Once the initial template has been built, servers can be deployed quickly by specifying the template, a unique host name and network address. This allows many steps in the configuration chain to be completed in a consistent and quick manner, allowing the entire infrastructure to be built out with as little administrative action as possible. For database solutions like Oracle 12c which use the same configuration for every server, the overall deployment process can be expedited and the possibility of administrative error reduced significantly. Additionally, tiered applications can each have their own template that can be applied to different classes of compute modules. For example, we might deploy an HPE Synergy 660 Compute Module as the Oracle database server and use the HPE Synergy 480 Compute Module as the application tier.

Customers today also require high performance, highly available flexible database solutions. HPE Synergy Gen9 Compute Modules coupled with an HPE 3PAR StoreServ 7450c all-flash array running Oracle 12c deliver just that by providing a fully tested, flexible, high performance, and highly available reference architecture.

This Reference Configuration will demonstrate the ease of reconfiguring HPE Synergy and HPE 3PAR to bring to bear resources at the time they are needed. It will further provide a proof point regarding the elasticity of HPE Synergy that allows an enterprise to provision exactly what’s needed and expand or contract based on changes in the business’ environment, eliminating the need to over provision at the time of purchase.

Target audience: This Hewlett Packard Enterprise white paper is designed for IT professionals who use, program, manage, or administer large databases that require high availability and high performance. Specifically, this information is intended for those who evaluate, recommend, or design new IT high performance architectures.

This white paper describes testing completed in February 2016.

Document purpose: The purpose of this document is to describe a Reference Configuration, highlighting recognizable benefits to technical audiences.

Introduction This Reference Configuration is one in a series of white papers that demonstrate the use of the HPE Synergy composable infrastructure to support enterprise applications. Composable infrastructure is designed around three core principles:

• A pool of resources including compute, storage and network that can be deployed, managed and reconfigured as business and application needs change.

• The ability to use templates or profiles to simply and quickly provision infrastructure resources in a repeatable and consistent manner.

• A unified API that provides access to all infrastructure components and hardware resources and allows for the management of hardware in an integrated manner.

This series of projects demonstrates the concepts and solution designs used to place applications such as Oracle Database, Microsoft® Exchange Server, SQL Server, and other applications on the HPE composable infrastructure pools of compute, storage and fabric.


HPE Synergy for Oracle HPE Synergy offers a variety of capabilities to provide a better experience with Oracle Database server lifecycle management. The following criteria is used in choosing the infrastructure for database servers.

• High Performance

Databases are configured to support a variety of workloads. Infrastructure has to be set up to meet these workload’s performance needs. To process a large number of transactions, OLTP database servers must support a high number of Input/Output operations. HPE Synergy with its ability to scale the number of disk drives in the internal storage or with external SAN storage can meet the IOPS needs.

For those environments focusing on In-memory databases, the critical requirement is the high end-to-end memory access speeds and memory density. By using the latest Intel® processor technology, with up to 24 DIMMs per socket support, large databases can be stored in memory and performance targets can be achieved.

In OLAP configurations, bandwidth is the key requirement. With the midplane supporting multiple Tb/s, and support for multiple 40 Gb/s external ports, DAS as well as SAN performance can meet large OLAP performance needs.

With the supporting infrastructure for both OLAP and OLTP environments, one can cohost applications that need these two database technologies and can take advantage of the high performance infrastructure.

• Minimal downtime

Database servers are expected to be available all the time. HPE Synergy offers that reliability with its robust design – solid mechanical design, efficient electrical design and smart thermal and power management. The goal is not only to minimize planned downtime but also to ensure that the system is highly available even in the case of component failures.

HPE Synergy has redundant components – fabric, management modules and others, ensuring that the system is functional even when one of the components fails. During planned maintenance updates, HPE Synergy upgrades its infrastructure, including all the components, ensuring that the services are available without any disruption. The only time a system has to be booted is in those cases where an operating system driver update needs reboot.

• Ease of management

HPE Synergy’s composable infrastructure offers compelling benefits for the database environment.

– Reduces overprovisioning of resources

The HPE Synergy 12000 Frame is flexible in supporting compute, storage and network resources. Users can add any combination of those resources depending on their needs. With HPE Synergy’s software defined intelligence, one can pick the right amount of resources optimized for the database workload and can deploy the needed resources only, i.e., the right amount of compute, storage and network.

– Faster deployment

Typically, once a first server is defined with the right set of resources, then additional servers are deployed with the same configuration. With a template optimized for a database workload defined, deploying the server is very fast with HPE Synergy Composer for infrastructure, and Insight Control Server Provisioning or Image Streamer for the upper stack. With all of these capabilities integrated, the deployment experience is very simple and you can start using the new infrastructure in minutes.

In the day-to-day management, administrators have to support their end users by adding new servers for scale-out performance or for temporary test environments. They may also update servers with a new database version for testing, upgrade the processing capability for scale-up performance, or update storage capacity as the database needs change. All of these activities are disruptive and need long lead times. With HPE Synergy’s composable infrastructure adding compute modules, upgrading from a two socket to four socket compute module or increasing storage or network capacity is simple and quick.

Further, in this environment, HPE Synergy Composer powered by OneView can be used to create LUNs on 3PAR storage and export those LUNs to a compute module within the HPE Synergy environment. This removes the need to have dedicated administrators for servers and storage. One administrator can perform both administrative functions.


– Simplifies lifecycle management

HPE Synergy Composer offers appropriate tools to manage the infrastructure lifecycle management – through deployment, and on-going management and monitoring. With software defined intelligence, deployment is simple and quick. It offers a comprehensive view of all the resources and the ability to monitor and troubleshoot the infrastructure from a single console. Firmware and device drivers are integrated into a single package for all the components and updating these is automated, minimizing downtime due to human errors.

– Easy to integrate with the current environment

HPE Synergy offers a unified API for all the resources. Due to the simplicity and ability to address all the resources in the same way, developing scripts for various workflows is made easy. Now, integrating HPE Synergy with the current environment has become an easy task.

• Low TCO

HPE Synergy provides the ability to move an application’s personality from server to server quickly, seamlessly and easily. The ability to rapidly move applications means that an enterprise doesn’t need to acquire equipment specifically for peak utilization requirements, but rather flex the environment during times when the peak need is present. With the right provisioning of resources, capital expenses can be reduced with the initial purchase, as well as ongoing hardware purchases. Ease of management, and the ability to have an end-to-end view through the management console, reduces operational expenses.

With all these benefits from HPE Synergy and its ability to compose the infrastructure to meet various application needs, HPE Synergy provides the ideal platform for database servers and other applications.

HPE 3PAR StoreServ all-flash array All-flash HPE 3PAR StoreServ Storage is the only platform in the industry to deliver the performance advantages of a purpose-built, flash-optimized architecture without compromising resiliency, efficiency, or data mobility.

HPE 3PAR StoreServ 7450c is an all-flash based storage array that combines accelerated performance with all the enterprise-class, Tier 1 features and functionality expected for mission-critical application environments. This flash-optimized architecture relies on several unique HPE 3PAR StoreServ innovations:

• Mesh-active architecture—Fine-grained virtualization and system-wide striping.

• Purpose-built HPE 3PAR StoreServ ASIC—Supports mixed workloads with extremely high performance levels.

• HPE 3PAR Adaptive Read and Write—Matches host I/O size reads and writes to flash media at a granular level to avoid unnecessary data reads and writes to reduce latency.

• Autonomic cache offload—Reduces cache bottlenecks by automatically changing the frequency at which data is offloaded from cache to flash media based on utilization rate.

• Multi-tenant I/O processing—Enables performance improvement for mixed workloads by breaking large I/O into smaller chunks.

A unique suite of persistent technologies power HPE 3PAR StoreServ 7450c Storage in delivering high availability and Tier 1 resiliency to performance-critical applications. These include:

• HPE 3PAR Persistent Cache—Preserves service levels so they are not impacted by unplanned component failures.

• HPE 3PAR Persistent Ports—Allows non-disruptive upgrades without relying on multipathing software and without initiating failover.

• HPE 3PAR Peer Persistence—Ability to federate storage across data centers without being constrained by physical boundaries.

• HPE 3PAR StoreServ Data at Rest Encryption—Protects data from both internal and external security breaches.

• Flash-based media failure reconstruction—This enables the system to provide consistent performance levels even under situations of flash media failure.

• HPE 3PAR Remote Copy software enables low recovery time objectives (RTOs) and zero data-loss recovery point objectives (RPOs) with complete distance flexibility.


HPE Synergy Composer To eliminate infrastructure complexity, HPE Synergy Composer powered by OneView automates the delivery and operation of IT services—transforming everyday management of compute, storage, and network resources in physical and virtual environments. HPE Synergy Composer improves IT administrator efficiency by converging management of HPE compute, storage, and networking resources. It speeds IT service roll-outs and helps actively prevent error-induced downtime through a template-based, software defined approach to management. HPE Synergy Composer also saves time by acting as an automation hub that performs infrastructure configuration and management tasks at the request of other applications.

HPE Synergy Composer reduces OPEX and improves agility, so you can free-up resources to focus on new initiatives that will help grow your business.

You can use HPE Synergy Composer to automate the deployment, update, and ongoing management of HPE Synergy components. The customizable HPE Synergy Composer dashboard provides an easy-to-understand summary/status of servers, storage pools, and enclosures. Color-coded icons tell you which systems are functioning properly and which ones need help—whether you manage five systems or 500.

• One platform manages HPE Synergy Compute Modules, storage modules and networking, and HPE 3PAR StoreServ storage.

• Smart Search instantly finds what you are looking for without forcing you to search through long and complicated tree views.

• 3D Power/Thermal Mapping provides intuitive power management.

• Pre-configured reports help catalogue inventory and alerts, facilitating asset management and creation of internal status reports to IT and business leadership.

• Email event notification alerts administrators of potential problems in real time.

Solution overview This white paper outlines the architecture and performance you can expect from a solution built on HPE Synergy Compute Modules and the HPE 3PAR StoreServ 7450c all-flash array running Oracle 12c and Red Hat® Enterprise Linux® 7.1.

The HPE Synergy 12000 Frame offers the following composable infrastructure for deploying Oracle:

• Two or four socket compute modules – Intel Xeon® E5 and E7 platforms

• HPE Synergy Network Interconnects. HPE Synergy offers many different network switch options. The HPE Virtual Connect SE 40Gb F8 Module for HPE Synergy was chosen for this set of tests.


Figure 1. HPE Synergy Compute Modules left to right, the two socket 480 Gen9, four socket 660 Gen9 and the four socket 680 Gen9

Figure 2. HPE Virtual Connect SE 40Gb F8 Module for HPE Synergy

The HPE Synergy 12000 Frame includes a comprehensive management environment running on a redundant pair of HPE Synergy Composer Modules powered by HPE OneView. Frame Link Modules connect up to 20 frames that define the management domain. The HPE Synergy Composer is an environment where all the components of a specific solution are defined.

The use of the HPE Synergy composable infrastructure for applications allows the deployment and management of a set of hardware and subsequent adjustment of the balance of compute, storage and fabric using HPE Synergy Composer in a simple and straightforward manner, by either licensing the desired feature or the installation of additional compute modules or storage modules and drives. The wire-once model combined with the ability to expand compute, fabric and storage gives the user the ability to maintain the balance of hardware needs as application requirements change over time, thus driving faster business results, more agile IT environments and reducing the cost and complexity to deploy and support applications running in support of the enterprise.


Figure 3. HPE Synergy management components

The HPE 3PAR StoreServ 7450c offers the following configuration options

• Two-node, upgradeable to four-node configurations. Each node pair provides 24 drive slots

• Up to 18 expansion shelves for a total of 240 drive slots when deploying a 4-node 7450c

• Up to 921.6TiB of raw capacity

• 16Gb/8Gb Fibre Channel (24) Ports

• 10GbE iSCSI (8) Ports

• 192GiB of cache

• The same operating system and interface across all models of HPE 3PAR 7000 arrays

HPE Synergy Composer HPE Synergy Frame Link Module

Frame uplink and crosslink connection

Management networkconnection


Figure 4. HPE 3PAR StoreServ 7450c (4-node)

In addition, HPE OneView 3.0 can provision storage on the 3PAR 7450c array and that storage can then be attached to server profiles on which the Oracle database is run. This makes the provisioning of compute, network and storage seamless across all technologies utilized to provide Oracle database services.

Solution components The HPE Synergy components used in this solution are based on a balanced architecture that uses components that fit within a single HPE Synergy 12000 Frame. The use of components that reside within the HPE Synergy Frame allow for a simplification of design by using the fewest number of components necessary to deliver a high performance Oracle environment.

The HPE Synergy 12000 Frame had the following components:

• 1 X HPE Synergy 480 Gen9. This is a two socket compute module and was populated with 16-core processors for a total of 32 cores.

• 1 X HPE Synergy 660 Gen9. This is a four socket compute module and it was populated with 20-core processors for a total of 80 cores. We used this compute module as a target to move our HPE Synergy 480 Gen9 environments when we needed additional throughput provided by the additional sockets and cores.

• 1 x HPE Synergy D3940 Storage Module. Two mirrored drives were used for the operating system to allow the OneView server profile to be easily moved from one compute module to another.

HPE 3PAR 7450c configured as follows:

• 1 X 3PAR StoreServ 7450c all-flash array

– 4-node

– 192GB of cache

– 6 X expansion shelves

– 80 X 480GB MLC SSD

– 24 X 8Gb Fibre Channel ports


Figure 5 shows the environment used for testing.

12

110 23

12

110 23

3PARStoreServ

7450

3PARStoreServ

7450SAS SAS SAS SAS

10K 900GB 10K 900GB 10K 900GB 10K 900GB

SAS SAS SAS SAS10K 900GB 10K 900GB 10K 900GB 10K 900GB



SAS10K 900GB

SAS10K 900GB

SAS10K 900GB

SAS10K 900GB

12

110 23

12

110 23

3PARStoreServ

7450

3PARStoreServ

7450SAS SAS SAS SAS

10K 900GB 10K 900GB 10K 900GB 10K 900GB




SAS10K 900GB

SAS10K 900GB

SAS10K 900GB

SAS10K 900GB

12

110 23

12

110 23

3PARStoreServ

7450

3PARStoreServ

7450SAS SAS SAS SAS

10K 900GB 10K 900GB 10K 900GB 10K 900GB




SAS10K 900GB

SAS10K 900GB

SAS10K 900GB

SAS10K 900GB

12

110 23

12

110 23

3PARStoreServ

7450

3PARStoreServ

7450SAS SAS SAS SAS

10K 900GB 10K 900GB 10K 900GB 10K 900GB




SAS10K 900GB

SAS10K 900GB

SAS10K 900GB

SAS10K 900GB

• 3PAR StoreServ 7450c All Flash Arrayo 4-nodeo 192GB of cacheo 6 X Expansion Shelveso 24 X 8Gb FC portso 80 X 480GB MLC SSD

• 8 X RAID-5 LUNs for Tables and Indexes• 8 X RAID-5 LUNs for Redo Logs• 8 X RAID-5 LUNs for Undo Tablespace

StoreServ 7450c Controller Nodes

StoreServ 7450c Expansion Drive Shelves

FrontPanel

Bay16

Bay6

Bay7

Bay1

ApplianceBay 2

ApplianceBay 1

Synergy12000Frame

UID

OneViewUID

Active

Power

Synergy Composer

UID

Synergy D3940Storage Module

2

4

1

3

UID

Synergy660

Gen9

OneViewUID

Active

Power

Synergy Composer

1

2

25

26

23

24

15

18

17

16HPE 5930 Switch

40GbE Link FCoE Traffic

4 X 8Gb Fibre Channel

40GbE Link IP Traffic

HPE Synergy 480 Gen9HPE Synergy D3940o 2 X 450GB 15K RPM drives

• 2 X RAID-1 for OS

HPE Synergy 660 Gen9HPE Synergy Composer

HPE Synergy 12000

UID

Synergy480

Gen9

HP SN6000B 16Gb FC Switch

47434642454144403935383437333632312730262925282423192218211720161511141013912873625140HPE SN6000B Switch

Figure 5. Logical diagram of HPE Synergy environment

Software • Red Hat Enterprise Linux version 7.1

• Oracle 12c Enterprise Edition

Best practices and configuration guidance for the Oracle Database solution HPE Synergy Compute Module BIOS • Hyper-Threading—Enabled

• Intel Turbo Boost—Enabled

• HPE Power Profile—Maximum Performance


Storage configuration best practices – 3PAR all-flash array • Create udev rules to set the following device options for the SSD LUNs and required settings for the Oracle volumes (per values in Appendix E

and F).

– Set the sysfs “rotational” value for SSD disks to 0.

– Set the sysfs value rq_affinity to 2 for each device. Request completions all occurring on core 0 caused a bottleneck. Setting rq_affinity to a value of 2 resolved this problem.

– Set I/O scheduler to noop (no operation).

– Set permissions and ownership for Oracle volumes.

• SSD loading—Load SSDs in groups of four per enclosure at a minimum.

• Volume size—Virtual volumes should all be the same size and SSD type for each Oracle ASM group.

• Multiple paths to maintain high availability while also maximizing performance and minimizing latencies. Use recommended multipath parameters (see Appendix E).

Oracle configuration best practices For specific Oracle database parameter settings, see Appendix B and C.

• Disable RHEL automatic NUMA balancing (see Appendix D).

• Disable automatic memory management if applicable.

• Set buffer cache memory size large enough per your implementation to avoid physical reads.

• Create two large redo log file spaces of 450GB each to minimize log file switching and reduce log file waits.1

• Create an undo tablespace of 512GB.2

• Configure huge pages (see Appendix D) and set Oracle to only use huge pages.

Capacity and sizing One of the more compelling features of HPE Synergy is frictionless movement of resources from one application to another. When additional resources are required, the profile of a two socket compute module can be moved to a four socket compute module to easily configure more compute capacity. This allows a customer to provide the right sized server to the Oracle Database and then when peak amounts of resources are required (perhaps at the end of the month or year), they can apply the same server profile to a more robust compute module and have the resources needed. When the peak resource requirement has passed, simply move the profile back to the original compute module and return to a normal run rate. There is no need to over size or over provision compute or storage resources for the worst case scenario.

This feature also opens up a number of options. As an example, you could have a primary Oracle Database server that replicates to a read-only database that is used for reporting or decision support during the day. Then when it is backup time, you could apply a profile for a server specifically tuned to perform backups or even shift the storage to another backup-specific server. When the backup is complete, you simply shift back to the original read-only database configuration and continue on with processing.

Workload description The Oracle workload was tested using HammerDB, an open-source tool. The tool implements an OLTP-type workload (60 percent read and 40 percent write) with small I/O sizes of a random nature. The transaction results have been normalized and are used to compare test configurations. Other metrics measured during the workload come from the operating system and/or standard Oracle Automatic Workload Repository (AWR) statistics reports.

1 During our testing, we used redo log file spaces of 350GB, 400GB and 450GB. We found no performance difference. It is, however, important that the size and quantity of redo log

file spaces be large enough so that a constraint is not encountered when closing one redo log and opening another, such as the new redo log has not had enough time to have been flushed since it was last used.

2 During our testing we used undo tablespace sizes of 300GB and 512GB. We saw no performance difference as long as the size was sufficiently large enough that it didn’t cause the queries to throw errors.


The OLTP test, performed on a 500GB database, was both highly CPU and moderately I/O intensive. The environment was tuned for maximum user transactions. After the database was tuned, the transactions were recorded at different connection levels. Because customer workloads vary so much in characteristics, the measurement was made with a focus on maximum transactions.

Oracle Enterprise Database version 12.1.0.1 was used in this test configuration.

The fibre channel attached 3PAR StoreServ 7450c storage was configured with 8 x RAID-1 LUNs for the indexes and tablespace, 8 x RAID-1 LUNs for the redo logs, and 8 x RAID-1 LUNs were used for the undo tablespace.

We used several different Oracle connection counts for our tests, and found the optimal number to be 150 connections for the two socket compute modules. The optimal number of connections was twice that number on the four socket compute module because it had more memory and more than double the number of cores.

Workload results We tested an HPE Synergy 480 Gen9, two socket compute module and compared the results with those delivered by an HPE Synergy 660 Gen9, four socket compute module.

The HPE Synergy 480 Gen9 two socket compute module had 32 cores and 256GB of memory. The HPE Synergy 660 Gen9, four socket compute module had 80 cores and 1TB of memory.

Both the HPE Synergy 480 Gen9 and HPE Synergy 660 Gen9 were tested with the same storage configuration, in order to demonstrate the performance improvement achieved when replacing the HPE Synergy 480 Gen9 with the HPE Synergy 660 Gen9.

The following graph shows the results of our testing using an HPE 3PAR StoreServ 7450c all-flash array. The number of connected users is the count of users that drove the test. Each user had no think time and as a result represented hundreds of actual users. The results have been normalized such that the 25 User result on the HPE Synergy 480 Gen9 Compute Module was set to 100%. All other results are relative to the initial result.

As you can see from the graph, HPE Synergy provided a good scaling factor when moving from a two socket compute module to a four socket compute module.

050

100150200250300350400450500

25Users

50Users

75Users

100Users

150Users

200Users

250Users

300Users

350Users

400Users

Perc

enta

ge

Number of Oracle Connections

Performance results using HPE 3PAR

SY480 SY660


The following graph represents the transactional latency incurred during the test run. Note that the two socket compute module provided similar performance to the four socket compute module up through 150 Oracle connections, and the additional capacity of the four socket compute module provided lower transaction latency at 200 Oracle connections and higher.

Analysis and recommendations HPE Synergy is a new computing paradigm, and a new paradigm introduces options that were heretofore unavailable.

The testing has demonstrated that an organization does not have to purchase compute and storage resources based on a worst case scenario and leave those resources underutilized or lying idle. Being able to quickly and easily bring additional compute and storage options to bear, allows an organization to right-size the environment for the typical deployment scenario, while being able to flex the environment when additional resources are required to fulfil the customer’s needs.

Being able to change the personality of a server on the fly by applying different profiles at the touch of a button, means that each server profile can be specifically tuned for its unique job requirements. There is no longer the requirement to make compromises to satisfy two different workloads as long as those workloads are not concurrent.

Additionally, being able to tailor a server profile to the unique requirements and then being able to apply that profile to multiple compute modules means that we could add or delete Oracle RAC servers in minutes rather than hours or days. This allows the organization to be able to flex the environment as the needs of the organization change.

As our performance results demonstrate, HPE Synergy is a highly performant, highly scalable, highly available environment that will perform well in any Oracle environment, even if the business is stable, and as a result Hewlett Packard Enterprise recommends HPE Synergy in those environments. Hewlett Packard Enterprise also recommends HPE Synergy for Oracle environments where business needs fluctuate.

Summary The deployment of the Oracle Database on the HPE Synergy composable infrastructure and HPE 3PAR all-flash arrays provides options that have not existed before now. The new environment allows the deployment of a modular design that affords the ability to deploy the environment quickly and flex the environment rapidly and seamlessly as needs change. The use of integrated networking interconnect modules allows the deployment of a simple networking model with a reduced number of components.

Implementing a proof-of-concept As a matter of best practice for all deployments, HPE recommends implementing a proof-of-concept using a test environment that matches as closely as possible the planned production environment. In this way, appropriate performance and scalability characterizations can be obtained. For help with a proof-of-concept, contact an HPE Services representative (hpe.com/us/en/services/consulting.html) or your HPE partner.

0123456789

10

25Users

50Users

75Users

100Users

150Users

200Users

250Users

300Users

350Users

400Users

Milli

seco

nds

Number of Oracle Connections

Transaction Latency

SY480 SY660

http://www.hpe.com/us/en/services/consulting.html


Appendix A: Bill of materials

Note The bill of materials does not include complete support options or rack and power requirements. For questions regarding ordering, please consult with your HPE Reseller or HPE Sales Representative for more details. hpe.com/us/en/services/consulting.html

Table 1a. Bill of materials for HPE Synergy

QTY DESCRIPTION

Rack and Network Infrastructure

1 HPE 642 1200 mm Shock Intelligent Rack

1 HPE 42U 1200mm Side Panel Kit

1 HPE Air Flow Optimization Kit

3 HPE 1U Black Universal 10-pk Filler Panel

HPE Synergy Frame Components

1 HPE Synergy 12000 Configure-to-order Frame with 1x Frame Link Module 10x Fans

1 HPE 6X 2650W AC Titanium Hot Plug FIO Power Supply Kit

1 HPE Synergy Composer

1 HPE VC SE 40Gb F8 Module

2 HPE Synergy 12Gb SAS Connection Module with 12 Internal Ports

HPE Synergy Compute Module Components

1 HPE Synergy 480 Gen9 Configure-to-order Compute Module

2 HPE Synergy 480 Gen9 Intel Xeon 16-core processors

16 HPE 16GB (1x16GB) Dual Rank x4 DDR4-2133 RDIMMs

1 HPE Synergy Smart Array P542D Controller

1 HPE Synergy 3820C 10/20Gb Converged Network Adapter

1 HPE Synergy 660 Gen9 Configure-to-order Compute Module

4 HPE Synergy 660 Gen9 Intel Xeon 20-core processors

32 HPE 32GB Dual Rank DDR4-2133 RDIMMs

1 HPE Synergy Smart Array P542D Controller

1 HPE Synergy 3820C 10/20Gb Converged Network Adapter

HPE Synergy Storage Module Components (only used two disks for these tests)

1 HPE Synergy D3940 Storage Module

2 HPE Synergy D3940 Redundant I/O Adapter

4 200GB SSD

36 450GB 15K RPM Disk Drives



Table 1b. Bill of materials for HPE 3PAR 7450c

QTY DESCRIPTION

1 HPE 642 1075mm Shock Intelligent Rack

1 HPE 3PAR StoreServ 7450c 4N St Cent Base

20 HPE M6710 480GB 6G SAS 2.5in MLC 5yr SSD

1 HPE 3PAR 7450 Reporting Suite LTU

1 HPE 3PAR 7450 OS Suite Base LTU

80 HPE 3PAR 7450 OS Suite Drive LTU

2 HPE SN6000B 16Gb 48/24 FC Switch

48 HPE B-series 16Gb SFP+SW XCVR

6 HPE M6710 2.5in 2U SAS Drive Enclosure

60 HPE M6710 480GB 6G SAS 2.5in MLC 5yr SSD

1 HPE 3PAR 7000 Service Processor

1 HPE Rack Front Door Cover Kit

48 HPE Premier Flex LC/LC OM4 2f 15m Cbl

8 HPE Premier Flex LC/LC OM4 2f 5m Cbl

4 HPE 4.9kVA 208V 20out NA/JP bPDU

1 HPE 600mm Rack Stabilizer Kit

1 HPE 42U 1075mm Side Panel Kit

1 HPE 3PAR StoreServ Mgmt/Core SW Media

1 HPE 3PAR OS Suite Media

1 HPE 3PAR Service Processor SW Media

1 HPE 3PAR Reporting Suite Media

1 HPE Intelligent Inft Analyzer SW v2 LTU

Appendix B: Oracle configuration parameters – HPE Synergy 480 Gen9 pga_aggregate_target=51546M processes=1500 result_cache_max_size=794304K sga_target=155136M _high_priority_processes='VKTM*|LG*' lock_sga=TRUE use_large_pages='ONLY' _max_outstanding_log_writes=4

Appendix C: Oracle configuration parameters – HPE Synergy 660 Gen9 pga_aggregate_target=103173M processes=3000 result_cache_max_size=1585984K sga_target=309760M _high_priority_processes='VKTM*|LG*' lock_sga=TRUE use_large_pages='ONLY' _max_outstanding_log_writes=4


Appendix D: Linux kernel configuration kernel.sem = 250 32000 100 128 kernel.shmall = 4294967295 kernel.shmmax = 332859965440 fs.file-max = 6815744 kernel.shmmni = 4096 fs.aio-max-nr = 1048576 net.ipv4.ip_local_port_range = 9000 65500 net.core.rmem_default = 262144 net.core.rmem_max = 4194304 net.core.wmem_default = 262144 net.core.wmem_max = 1048586 vm.nr_hugepages = 77572 vm.hugetlb_shm_group = 1003 kernel.numa_balancing=0

Notes 1. The vm.nr_hugepages is dependent on the amount of memory installed in the server. We had 256GB in the HPE Synergy 480 Gen9, so we

set it to 77572. However, when we had 1TB of memory in the HPE Synergy 660 Gen9, we set it to 155144. This is required by Oracle if you wish to set the sga_target higher, which we did when we were running more connected users and higher throughput numbers.

2. For RHEL 7, automatic NUMA balancing should be disabled (by setting kernel.numa_balancing=0) for optimal Oracle performance. This resulted in a significant performance improvement for our testing.

Appendix E: 3PAR permission udev rules For 3PAR configurations, a udev rules file /etc/udev/rules.d/12-dm-permission.rules was created to set the required ownership of the Oracle ASM LUNs:

ENV{DM_NAME}=="mpathaj", OWNER:="oracle", GROUP:="oinstall", MODE:="660" ENV{DM_NAME}=="mpathak", OWNER:="oracle", GROUP:="oinstall", MODE:="660" ENV{DM_NAME}=="mpathal", OWNER:="oracle", GROUP:="oinstall", MODE:="660" ENV{DM_NAME}=="mpatham", OWNER:="oracle", GROUP:="oinstall", MODE:="660" ENV{DM_NAME}=="mpathan", OWNER:="oracle", GROUP:="oinstall", MODE:="660" ENV{DM_NAME}=="mpathao", OWNER:="oracle", GROUP:="oinstall", MODE:="660" ENV{DM_NAME}=="mpathap", OWNER:="oracle", GROUP:="oinstall", MODE:="660" ENV{DM_NAME}=="mpathaq", OWNER:="oracle", GROUP:="oinstall", MODE:="660" ENV{DM_NAME}=="mpathl", OWNER:="oracle", GROUP:="oinstall", MODE:="660" ENV{DM_NAME}=="mpathm", OWNER:="oracle", GROUP:="oinstall", MODE:="660" ENV{DM_NAME}=="mpathn", OWNER:="oracle", GROUP:="oinstall", MODE:="660" ENV{DM_NAME}=="mpatho", OWNER:="oracle", GROUP:="oinstall", MODE:="660" ENV{DM_NAME}=="mpathp", OWNER:="oracle", GROUP:="oinstall", MODE:="660" ENV{DM_NAME}=="mpathq", OWNER:="oracle", GROUP:="oinstall", MODE:="660" ENV{DM_NAME}=="mpathr", OWNER:="oracle", GROUP:="oinstall", MODE:="660" ENV{DM_NAME}=="mpaths", OWNER:="oracle", GROUP:="oinstall", MODE:="660"


Appendix F: 3PAR udev rules for SSD drives For the 3PAR configurations, a udev rules file /etc/udev/rules.d/10-3par.rules was created to set device parameters to appropriate settings for SSD drives:

ACTION=="add|change", KERNEL=="dm-*", PROGRAM="/bin/bash -c 'cat /sys/block/$name/slaves/*/device/vendor | grep 3PARdata'", ATTR{queue/rotational}="0", ATTR{queue/scheduler}="noop", ATTR{queue/rq_affinity}="2", ATTR{queue/nomerges}="1", ATTR{queue/nr_requests}="12


Sign up for updates

Rate this document © Copyright 2016 Hewlett Packard Enterprise Development LP. The information contained herein is subject to change without notice. The only warranties for HPE products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HPE shall not be liable for technical or editorial errors or omissions contained herein.

Oracle is a registered trademark of Oracle and/or its affiliates. Microsoft is either a registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. VMware is a registered trademark or trademark of VMware, Inc. in the United States and/or other jurisdictions. Intel and Xeon are trademarks of Intel Corporation in the U.S. and other countries. Red Hat is a registered trademark of Red Hat, Inc. in the United States and other countries. Linux is the registered trademark of Linus Torvalds in the U.S. and other countries.

4AA6-4736ENW, March 2016

Resources and additional links HPE & Oracle Alliance http://h22168.www2.hpe.com/us/en/partners/oracle/

HPE & Oracle Alliance Technical page http://h17007.www1.hpe.com/us/en/enterprise/converged-infrastructure/info-library/index.aspx?app=oracle

HPE Synergy hpe.com/info/synergy

HPE Composable Infrastructure hpe.com/info/composableinfrastructure

HPE Storage hpe.com/storage

HPE Enterprise Information Library hpe.com/info/convergedinfrastructure

HPE Technology Consulting Services hpe.com/us/en/services/consulting.html

To help us improve our documents, please provide feedback at hpe.com/contact/feedback

http://www.hpe.com/info/getupdated

http://www.facebook.com/sharer.php?u=http://h20195.www2.hpe.com/V2/GetDocument.aspx?docname=4AA6-4736ENW

http://twitter.com/home/?status=HPE%20Reference%20Configuration%20for%20Oracle%2012c%20on%20HPE%20Synergy%20composable%20infrastructure%20using%20HPE%203PAR%20StoreServ%20storage+@+http://h20195.www2.hpe.com/V2/GetDocument.aspx?docname=4AA6-4736ENW

http://www.linkedin.com/shareArticle?mini=true&ro=true&url=http://h20195.www2.hpe.com/V2/GetDocument.aspx?docname=4AA6-4736ENW&title=HPE%20Reference%20Configuration%20for%20Oracle%2012c%20on%20HPE%20Synergy%20composable%20infrastructure%20using%20HPE%203PAR%20StoreServ%20storage+&armin=armin

https://hpresearch.az1.qualtrics.com/SE/?SID=SV_6EuQKOr2Ku1CUzH&Pubnumber=4AA6-4736ENW

http://h22168.www2.hpe.com/us/en/partners/oracle/

http://h17007.www1.hpe.com/us/en/enterprise/converged-infrastructure/info-library/index.aspx?app=oracle

http://hpe.com/info/synergy

http://hpe.com/info/composableinfrastructure

https://www.hpe.com/storage

http://hpe.com/info/convergedinfrastructure/


http://www.hpe.com/contact/feedback

hpe reference configuration for oracle 12c on hpe synergy

Documents