using smarttier of oceanstor v3 in oracle database to ... · oracle database to improve performance...
TRANSCRIPT
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
This document explains how to use the SmartTier feature of the Huawei OceanStor V3 converged storage to improve the
online transaction processing (OLTP) application for Oracle databases and reduce the total cost of ownership (TCO). This
document introduces the working principle of SmartTier, and verifies whether SmartTier can improve database application
performance and reduce the TCO in a typical scenario, and provides the best practices of SmartTier. When deploying Oracle
databases, users can follow the instructions in this document to use SmartTier.
Author: Wang Yaohui
IT Storage Solutions, Huawei Enterprise BG
February 7, 2015 Version 1.0
Huawei Technologies Co., Ltd.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 2
2
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
Contents
1 About This Document .................................................................................................................. 4
1.1 Overview ...................................................................................................................................................................... 4
1.2 Purpose ......................................................................................................................................................................... 4
1.3 Intended Audience ........................................................................................................................................................ 4
1.4 Business case ................................................................................................................................................................ 5
1.5 Customer Benefits ........................................................................................................................................................ 5
1.6 Key Components .......................................................................................................................................................... 5
1.7 Workload Models .......................................................................................................................................................... 6
2 Products and technologies ........................................................................................................... 7
2.1 Huawei OceanStor V3 converged storage .................................................................................................................... 7
2.1.1 OceanStor OS ............................................................................................................................................................ 8
2.1.2 SmartTier ................................................................................................................................................................. 10
2.2 Oracle Database .......................................................................................................................................................... 12
2.2.1 Oracle Database Architecture .................................................................................................................................. 12
2.2.2 Oracle RAC ............................................................................................................................................................. 13
2.2.3 Oracle ASM ............................................................................................................................................................. 14
2.2.4 Workloads Types ...................................................................................................................................................... 14
3 SmartTier solution for Oracle database .................................................................................. 16
3.1 Business Challenges ................................................................................................................................................... 16
3.2 Solution ....................................................................................................................................................................... 17
3.2.1 Solution Architecture ............................................................................................................................................... 17
3.2.2 Solution Configuration ............................................................................................................................................ 20
3.3 Test Planning............................................................................................................................................................... 21
3.3.1 Environment build-up .............................................................................................................................................. 21
3.3.2 Improve performance test ........................................................................................................................................ 23
3.3.3 Reduce TCO test ...................................................................................................................................................... 24
3.4 Improve performance test ........................................................................................................................................... 25
3.4.1 Baseline test ............................................................................................................................................................. 25
3.4.2 5% SSD capacity test ............................................................................................................................................... 25
3.4.3 10% SSD capacity test ............................................................................................................................................. 28
3.4.4 15% SSD capacity test ............................................................................................................................................. 28
3.4.5 20% SSD capacity test ............................................................................................................................................. 29
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 3
3
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
3.4.6 25% SSD capacity test ............................................................................................................................................. 29
3.4.7 30% SSD capacity test ............................................................................................................................................. 30
3.4.8 35% SSD capacity test ............................................................................................................................................. 30
3.4.9 40% SSD capacity test ............................................................................................................................................. 31
3.4.10 45% SSD capacity test ........................................................................................................................................... 31
3.4.11 50% SSD capacity test ........................................................................................................................................... 32
3.4.12 Clean up the environment ...................................................................................................................................... 32
3.4.13 Test results summary.............................................................................................................................................. 33
3.5 Reduce TCO test ......................................................................................................................................................... 34
3.5.1 Baseline test ............................................................................................................................................................. 34
3.5.2 5% SSD capacity test ............................................................................................................................................... 34
3.5.3 10% SSD capacity test ............................................................................................................................................. 36
3.5.4 15% SSD capacity test ............................................................................................................................................. 36
3.5.5 25% SSD capacity test ............................................................................................................................................. 37
3.5.6 35% SSD capacity test ............................................................................................................................................. 37
3.5.7 45% SSD capacity test ............................................................................................................................................. 38
3.5.8 Test results summary................................................................................................................................................ 38
3.6 Conclusion .................................................................................................................................................................. 39
3.6.1 Improve performance ............................................................................................................................................... 39
3.6.2 Reduce TCO ............................................................................................................................................................ 41
3.7 Best practices .............................................................................................................................................................. 43
3.7.1 Selecting acceleration areas ..................................................................................................................................... 43
3.7.2 Estimite the hot spot ratio ........................................................................................................................................ 43
3.7.3 Selecting an I/O Monitoring Policy ......................................................................................................................... 44
3.7.4 Selecting a Data Migration Policy ........................................................................................................................... 45
3.7.5 Selecting an Initial Capacity Allocation Policy ....................................................................................................... 46
4 Appendix ...................................................................................................................................... 48
4.1 Reference Documents ................................................................................................................................................. 48
4.2 Terminology ................................................................................................................................................................ 48
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 4
4
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
1 About This Document
1.1 Overview
SmartTier is a dynamic storage tiering (DST) feature independently developed by Huawei
based on RAID 2.0+ block level virtulization. By monitoring I/Os, SmartTier automatically
detects the activity levels of data blocks and dynamically matches data blocks with proper
storage media based on their activity levels, improving performance of mission-critical
applications and reducing total cost of ownership (TCO).
As one of the key application scenarios, Oracle databases require storage devices to provide
high performance. This document describes how to use SmartTier to improve the performance
of Oracle databases and reduce the TCO in a typical Oracle OLTP business scenario.
This document covers the following contents:
OceanStor V3 technologies and features
Principle of SmartTier
Introduction to the solution
Implementation steps of the solution
Test steps and results
1.2 Purpose
This document describes how to use SmartTier to improve the performance of Oracle
databases, thereby providing partners and customers with reference about IT system solution
planning to improve database performance and reduce investment costs.
1.3 Intended Audience
The audience of this document includes Huawei's partners, customers, and employees. In
particular, this document is well-suited for storage and database administrators who plan to
use the SmartTier feature of V3 storage to improve the performance or reduce the TCO of
Oracle database with OLTP type workload.
It’s assumed that the readers are familiar with the following products and technologies.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 5
5
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
OceanStor V3 converged storage
OceanStor OS and SmartTier feature
OceanStor UltraPath
Red Hat Enterprise Linux 6
Oracle 12c RAC and Database
SwingBench Test Tool
1.4 Business case
As IT systems continue to evolve, enterprises and organizations have increasingly demanding
requirements on database application performance. As a result, how to improve database
performance becomes a major challenge for enterprises to address. The SmartTier feature of
the OceanStor V3 series converged storage helps customers maximize database performance
under a tight budget.
Oracle database is one of the most widely used databases, and OLTP type database is a typical
I/O-intensive service that requires extremely high storage performance but a moderate storage
capacity. SmartTier employs HDDs and SSDs to dynamically migrate data between tiers and
allows customers to significantly enhance database application performance by adding a small
number of SSDs. Customers do not need to purchase a large number of SSDs at a high cost,
maximizing return on investment (ROI).
1.5 Customer Benefits
SmartTier combines NL-SAS disks, SAS disks, and SSDs to intelligently migrate data blocks
to proper tiers based on their activity levels, striking a perfect balance between performance
and cost, ensuring high-performance access to mission-critical data, and reducing customers'
costs.
The test of this whitepaper proved that, for a typical Oracle OLTP database, configure 10%
SSD capacity to hold the “hot area” of user tables and indexes, the transaction throughput is
improved to 190%, the relative cost reduced to 68%, and the relative power consumption
reduced to 66%.
1.6 Key Components
The solution is based on the following hardware and software:
OceanStor V3 converged storage V300R001
OceanStor OS with SmartTier feature
OceanStor UltraPath for Linux: 8.01.023
Red Hat Enterprise Linux 6.5
Oracle Database 12.1.0.2 and SwingBench 2.5.0.928
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 6
6
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
1.7 Workload Models
The OLTP workload in this document is simulated using SwingBench Order Entry 2.0
workload model on Oracle Database 12C. The model is a widely used OLTP type workload
with the similar character with ERP application, in which lots of online users doing
transaction processing, with and I/O character of small block random read and write with
an ratio of 5:5.
The Order Entry model defines an online order service and simulates a scenario where a
number of users are querying products, placing orders, processing orders, and viewing orders
online. Those operations are the most common operations in transaction systems. In this
workload model, there are two main performance indicators: TPS (transactions per second)
and transaction response time. The TPS indicates the number of transactions processed per
second. A higher TPS indicates higher productivity. The transaction response time directly
impacts the speed of user operations. Shorter response time indicates better user experience.
Order Entry 2.0 model defines 10 tables, storing information about products, customers,
orders, warehouses, and login. During the load test, 50% of operations are allocated to
SELECT, 30% to INSERT, and 20% to UPDATE, and none to DELETE operations. From the
perspective of I/O layer, the workload model is one of the most typical OLTP workload
models, where small data blocks are accessed at random and the ratio between reads and
writes is 5:5.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 7
7
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
2 Products and technologies
2.1 Huawei OceanStor V3 converged storage
Huawei OceanStor V3 mid-range converged storage systems (V3 storages for short) offer a
cloud architecture oriented operating system, high-performance hardware platform, and a
complete suite of smart management software. V3 storages are on-demand converged storage
systems with secure, reliable, efficient features for multiple applications, heterogeneous
device integration, data disaster recovery, fast data growth.
Enables convergence in five areas: SAN and NAS, heterogeneous devices, entry-level to
high-end storage, HDD and SSD, and storage and backup
Provides industry-leading specifications: Up to 8 controllers, 1 TB cache, 5 PB storage
capacity, and various interface types, including 16 Gbit/s FC, 56 Gbit/s InfiniBand,PCIe
3.0, 12 Gbit/s SAS, and smart I/O card
Simplest management platform handles multiple product models with a graphical
interface and is available for Windows, iOS, and Android versions
Protects customer initial investments and reduces TCO for multiple applications, various
product models, and fast business growth
For more information about V3 storages, refer to the follow link:
http://e.huawei.com/en/products/cloud-computing-dc/storage/unified-storage/mid-range
Figure 2-1 OceanStor V3 Converged Storage
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 8
8
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
2.1.1 OceanStor OS
OceanStor OS is a cloud oriented storage platform used in V3 storage, with wealth of
software features and convergence of storage technologies, provides on-demand, simple,
efficient, secure and reliable storage service.
Figure 2-2 OceanStor OS
Convergence of SAN and NAS – On-demand service deployment
Figure 2-3 Convergence of SAN and NAS
Block- and file-level data storage is unified, requiring no additional file engines,
simplifying deployment, and reducing purchase cost
The efficiency of databases and file sharing services is notably improved
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 9
9
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
Convergence of SSDs and HDDs – On-demand performance and capacity
Figure 2-4 Convergence of HDDs and SSDs
All-flash arrays are optimized to put the high performance and low latency advantages
of SSDs into full play
HDDs and SSDs are converged to meet the performance requirements of complex
services
Convergence of primary and backup storage – On-demand data protection
Figure 2-5 Convergence of primary and backup storage
Backup storage is perfectly integrated into primary storage, requiring no additional
backup software and reducing your purchase cost
Primary storage and backup storage are managed in a unified manner, simplifying the
O&M of backup solutions
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 10
10
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
Convergence of high-end, mid-range, and entry-level storage systems – On-demand data value
Figure 2-6 Convergence of high-end, mid-range and entry-level storage
High-end, mid-range, and entry-level storage systems interwork, enabling free data flow
High-end, mid-range, and entry-level storage systems are managed in a unified manner,
multiple times the efficiency
Convergence of heterogeneous storage systems – On-demand storage resource allocation
Figure 2-7 Convergence of heterogeneous storage
Legacy storage systems are reused, protecting the original investment
Pools of third-party storage resources simplify storage management and achieve
cloud-based storage
2.1.2 SmartTier
SmartTier is a piece of storage tiering software developed based on RAID 2.0+. It selects an
appropriate storage tier for each data block based on the data activity level, improving system
performance and reducing TCO. SmartTier is designed to cope with the following challenges
facing customers:
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 11
11
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
High performance and multi-service support: various services and differentiated service
performance demands
Flexibility and ease-of-use: convenient configuration and migration of data to a suitable
tier
High efficiency and low power: energy-saving and efficient space usage
SmartTier divides storage capacity into various storage tiers to meet performance
requirements. The high-performance tier supports SSDs, the performance tier supports SAS
disks, and the capacity tier supports NL-SAS disks. Each storage tier can be used
independently, or two or three storage tiers can be combined to provide data storage space.
After performance statistics and analysis, SmartTier automatically relocates data according to
the data access frequency. SmartTier stores right data onto the right media at the right time.
SmartTier performs intelligent data storage based on LUNs and identifies LUNs based on a
data relocation granularity from 512 KB to 64 MB. The data relocation granularity is called
extent. SmartTier collects statistics on and analyzes the activity levels of data based on extents
and matches data of various activity levels with storage media. Data that is more active will
be promoted to higher-performance storage media (such as SSDs), whereas data that is less
active will be demoted to more cost-effective storage media with larger capacities (such as
NL-SAS disks).
Figure 2-1 Working principle of SmartTier
A complete SmartTier service process involves three phases:
Phase I: Collecting hotspot statistics
SmartTier allows user-defined I/O monitoring periods. During the scheduled periods, it
collects I/O statistics. Activity levels of data will change throughout a data lifecycle. By
comparing the activity level of one extent with that of another, the storage system
determines which data block is more or less frequently accessed. The activity level of
each extent is obtained based on the performance indicator statistics of data blocks.
Phase II: Analyzing data placement
The collected hotspot statistics are analyzed. This analysis ranks extents within the
storage pool. The ranking progresses from the most frequently accessed extents to the
least frequently accessed extents in the same storage pool. Note that only extents in the
same storage pool are ranked. Then a data migration solution is created. Before data
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 12
12
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
migration, SmartTier determines the direction of migrating extents according to the latest
data migration solution.
Phase III: Migrating data
SmartTier has two migration triggering modes: manual and periodic. The manual
triggering mode has a higher priority than the periodic mode. In manual triggering mode,
data migration can be triggered immediately when necessary. In periodic triggering mode,
data migration is automatically triggered based on a preset migration start time and
duration. The start time and duration of data migration are user-definable.
For more information about SmartTier, please refer to the document:
HUAWEI OceanStor V3 converged storage systems SmartTier technical white paper
2.2 Oracle Database
2.2.1 Oracle Database Architecture
As shown in the following figure, an Oracle database server consists of two parts: Oracle
instance and Oracle database. An Oracle instance contains software processes and their
occupied memory space (SGA, PGA, and program stack), whereas an Oracle database is a set
of data files. Data files record IT application system data and logical data such as indexes and
views created based on the system data. The logical data is created to assist in quickly
obtaining data. In addition, data files include logs that are saved to safeguard data reliability.
An application system (client) uses the listener (Listener) to send data storage and access
requests to server processes (Shared Process and Dedicated Process). The server processes
read the requested data from data files to the cache (Buffer Cache) and stores data changes
and newly generated data (dirty data) temporarily in the cache. After a short period of time,
background processes will write dirty data from cache to reliable storage. Dirty data is not
written to storage in real time. To ensure data reliability, dirty data generated by an atomic
operation (transaction) is written to online redo log files before the operation is complete. If
unexpected software and hardware faults (such as power failure, restart, and software crash)
occur, those redo log files can be used to recover dirty data that has not been written into the
data files.
Oracle database files include control files, data files, online redo log files, and archived redo
log files. A control file records the locations of other files, a data file records user data and
indexes, an online redo log records user operations, and an archived redo log is a copy of the
online redo log.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 13
13
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
Figure 2-2 Oracle database architecture
2.2.2 Oracle RAC
Oracle Real Application Clusters (RAC) provides a high-availability clustering mechanism
that ensures Oracle database continuity. As shown in the following figure, Oracle RAC is a
shared storage cluster. Each node runs an Oracle database instance, and all the database
instances share the same database. Operations between instances are synchronized by voting
disks and private network message communication. Instances provide external services by
using virtual IP addresses (VIPs). If one instance is down, the VIP of this instance is switched
to another node, and the instance on that node continues to provide external services. Oracle
RAC 11g can be deployed in NAS share directories or in shared block storage managed by
Oracle Automatic Storage Management (ASM).
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 14
14
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
Figure 2-3 Oracle RAC cluster architecture
2.2.3 Oracle ASM
Oracle ASM is launched together with Oracle database 10g and 11g. The ASM provides a
simple storage management interface for database administrators. As a built-in file system and
volume manager, the ASM is exclusively used by Oracle database files. The ASM simplifies
file system management, optimizes asynchronous I/O performance, reduces management time
for database administrators (DBAs), and provides an elastic and efficient database
environment.
The ASM can combine multiple raw devices into disk groups and allocate space based on the
Allocation Unit (AU). The ASM disk groups are classified into the following types:
External: Data is not mirrored among failure groups and the storage system provides a
data protection mechanism.
Normal: A normal disk group contains two failure groups and data is mirrored between
two raw devices.
High: A high disk group contains three failure groups and data is mirrored between three
raw devices.
2.2.4 Workloads Types
Database workloads are classified into OLTP and OLAP. The two types are described as
follows:
OLTP: Massive online users perform transactions. For example, online shopping
websites mainly run OLTP services.
OLAP: A few online users perform long-time complex statistical query.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 15
15
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
The character of OLTP type workload is described as follows:
From the perspective of database administrators:
Service characteristics:
− Massive online users perform transactions. For example, online shopping websites
mainly run OLTP services.
− Only a small amount of data is read, written, and modified in each transaction.
− Usually one transaction needs to be completed within seconds. Therefore, databases
must respond to requests quickly.
From the perspective of storage administrators:
I/O characteristics
− The I/O size is small, usually ranging from 2 KB to 8 KB.
− The locations of accessed data are random.
− The ratio of reads to writes is about 6:4.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 16
16
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
3 SmartTier solution for Oracle database
3.1 Business Challenges
In the IT industry, enterprises and administration departments are faced with challenges
concerning capacity, performance, and prices in data storage. Enterprises' storage devices
become inadequate to store fast-growing data. In addition, stored data keeps mounting, which
imposes a serious impact on the performance of IT systems.
For enterprises' services and applications, not all the data has the same importance. Among
the data, some is frequently accessed, some is seldom accessed, and some even has not been
accessed for several years.
Statistics and analysis show that a data importance has its life cycle. New data is frequently
accessed and has a high value. As time goes by, the data is accessed less frequently and is not
accessed at last. The data importance decreases year by year. A large amount of data with a
low value occupies high-performance system resources and large storage space, causing a
waste of resources. However, the data cannot be deleted due to related policies and data
warehouse construction requirements. Where to store infrequently accessed data becomes one
of data management questions that enterprises must answer. Enterprises often migrate
infrequently accessed data from expensive disk arrays to economical backup or archiving
systems. However, rocketing data amount results in a large amount of infrequently accessed
data, which incurs the following problems:
Data life cycles cannot be managed flexibly. Large data amount complicates data
management. Manual allocation of data to desired storage space is time consuming.
Large infrequently accessed data occupies precious high-performance storage media,
which results in the purchase of more high-performance devices.
Enterprises must consider the previous problems in their IT development process, especially
at the initial stage of building IT systems. Dynamic storage tiering (DST) well resolves the
previous problems by automatically migrating data to proper storage space and is widely
applied in various industries.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 17
17
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
3.2 Solution
3.2.1 Solution Architecture
This solution utilize Huawei OceanStor V3 storages’ SmarTier feature to resolve the
performance issues brought by business growth or workload changes in data life cycle.
SmartTier is also able to reduce the TCO of OLTP oracle database.
Two scenarios are tested in this solution:
Improve performance
The first scenario “improve performance” defines a database on 75 SAS disks which has
bad performance that limited the business growth of an enterprise, the solution is expanding
110GB ~ 1.1TB SSD capacity on the storage pool step by step, to meet the performance needs
when the business growing on demand.
Figure 3-1 Using SmartTier to improve performance
Initially the Storage Pool is allocated 4000GB space (6*600GB LUNs used to store 2.2TB user tables
and indexes), we expand the Storage Pool for 10 times each with 110GB (5% of user data) to test the
performance improvement brought by SmartTier.
The Disk Domain initially contains 75 SAS disks. Each time to expand the Storage Pool, the Disk
Domain is expanded first from 3 to 12 SSD disks.
The RAID level of performance tier (SAS disks) and high performance tier (SSD disks) are RAID10.
The test result shows that the TPS performance is improved from 1702 to 7103, and the “db file
sequential read” wait time (read latency) is reduced from 9.2 ms to 2.7ms. The steps and result of each
step will be described in detail in later sections.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 18
18
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
Table 3-1 Storage Pool and Disk Domain expand steps
0 1 2 3 … 10
Disk Domain SAS 75 75 75 75 75
SSD 0 3 4 5 12
Storage Pool SAS 4000GB 4000GB 4000GB 4000GB 4000GB
SSD 0GB 110GB 220GB 330GB 1100GB
LUN SAS 3600GB 3500GB 3400GB 3300GB 2600GB
SSD 0GB 100GB 200GB 300GB 1000GB
Data SAS 2200GB 2100GB 2000GB 1900GB 1300GB
SSD 0GB 100GB 200GB 300GB 1000GB
10% of the SSD capacity of SmartTier is reserved for data moving, so only about 100GB is used when
110GB is configured.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 19
19
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
Reduce TCO
The second scenario “reduce TCO” compares 7 different storage configurations which meet
the same performance needs, but with different costs and resource utilization. The 7 storage
configurations contain different number of SAS and SSD disks. The tested solution is to prove
that customer could reduce the TCO using SmartTier of OceanStor V3 storage for typical
Oracle database with OLTP workload.
Figure 3-2 Using SmartTier to reduce TCO
Initially the Storage Pool is allocated 4000GB space (6*600GB LUNs used to store 2.2TB user table
and index) on a Disk Domain contains 75 SAS disks, we remove the SAS disks and expand SSD disks
step by step, and expand SSD capacity to the Storage Pool each time.
The RAID level of performance tier (SAS disks) and high performance tier (SSD disks) are RAID10.
The test result shows that the number of SAS disks is reduced from 75 disks to 16 by adding 3 to 12 SSD
disks step by step. The steps and result of each step will be described in detail in later sections.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 20
20
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
Table 3-2 Storage Pool and Disk Domain expand/reclaim steps
0 1 2 3 4 5 6
Domain SAS 75 55 45 40 35 25 16
SSD 0 3 4 5 7 9 12
Pool SAS 4000GB 4000GB 4000GB 4000GB 4000GB 4000GB 4000GB
SSD 0GB 110GB 220GB 330GB 550GB 770GB 1100GB
LUN SAS 3600GB 3500GB 3400GB 3300GB 3100GB 2900GB 2600GB
SSD 0GB 100GB 200GB 300GB 500GB 700GB 1000GB
Data SAS 2200GB 2100GB 2000GB 1900GB 1700 1500 1300GB
SSD 0GB 100GB 200GB 300GB 500GB 700GB 1000GB
For each step, remove SAS disk from the Disk Domain one by one (remove next when the previous
rebalance complete) until to the target number, then expand SSD disks to the Disk Domain and expand
SSD capacity to the Storage Pool.
10% of the SSD capacity of SmartTier is reserved for data moving, so only about 100GB is used when
110GB is configured.
3.2.2 Solution Configuration
Table 3-3 Hardware configuration
Item Specifications Quantity
Server Huawei Tecal RH5885 V2 each with:
256GB memory
4 – Intel Xeon E5-2620 CPUs
1 – Intel 10Gbps Ethernet HBA card
2 – Qlogic 16Gbps FC HBA card
2
Private 10GE Switch Huawei S6700 1
Storage Huawei OceanStor 5800 V3 with:
2 – controllers each with 64GB cache
4 – 2U 25 slots disk enclosures
75 – 600GB 10K RPM SAS disks
12 – 200GB eMLC SSD disks
4 – 16Gbps FC I/O modules (2 ports)
2 – 24Gbps SAS I/O modules (4 ports)
1
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 21
21
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
Table 3-4 Software configuration
Item Software
OS Red Hat Enterprise Linux 6.5
Database Oracle RAC & Database 12.1.0.2
Test Tool Huawei Order Entry test suite 1.1
Storage Huawei UltraPath 8.01.023
Huawei OceanStor V3 SmartTier License
Huawei OceanStor V3 SmartMotion License
3.3 Test Planning
3.3.1 Environment build-up
Table 3-5 Environment build-up steps
Task Step Description
1 Prepare physical environment (hardware & cabling)
2 Install and configure OS on the two servers
2.1 Install Red Hat Enterprise Linux 6.5 basic server on the two RH5885 servers
2.2 Configure IP address (/etc/sysconfig/network-scripts/ifcfg-eth0 & ifcfg-eth1)
2.3 Configure host name (/etc/sysconfig/netwok)
2.4 Configure for host name resolve (/etc/hosts)
2.5 Configure I/O scheduler and TPH settings (/boot/grub/menu.lst)
2.6 Create users and groups (user oracle, group dba and oinstall)
2.7 Create installation directory target ($ORACLE_BASE, $GRID_HOME)
2.8 Configure SSH connection authentication (/home/oracle/.ssh/authenticated_keys)
2.9 Configure environment variables (/home/oracle/.bash_profile)
2.10 Configure kernel parameters (/etc/sysctl.conf)
2.11 Configure user limits (/etc/security/limits)
2.12 Configure firewall (/etc/selinux/config and iptables)
2.13 Configure NTP service (/etc/sysconfig/ntpd)
2.14 Configure software source (/etc/yum.repos.d/rhel-source.repo)
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 22
22
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
Task Step Description
2.15 Install software (yum install …) and Oracle JRE
2.16 Install OceanStor UltraPath package and reboot system
3 Configure storage sub-system
3.1 Import SmartTier and SmartMotion License
3.2 Provision storage resource (Disk Domain, Storage Pool, LUNs, LUN group)
3.3 Scan hosts, create host group, mapping view
3.4 Scan LUNs on hosts
3.4 Configure UDEV policies on hosts (/etc/udev/rules.d/99-huawei-devices.rules)
4 Install and configure Oracle RAC and database
4.1 Upload and unzip Oracle 12.1.0.2 grid and database software
4.2 Install Oracle Grid Infrastructure (grid/runInstaller)
4.3 Install Oracle Database (database/runInstaller)
5 Load test data
5.1 Create ASM disk groups (asmca)
5.2 Create container database (dbca)
5.3 Create pluggable database and load 2.2TB test data (oe12c/0.load.sh, scale 2000)
5.4 Backup database (rman)
Table 3-6 IP configuration
Site Host Mngmt IP / 16 Virtual IP / 16 Private IP / 25 ASM IP/25
Server
cloud1 100.148.42.1 100.148.42.11 192.168.42.1 192.158.42.1
cloud2 100.148.42.2 100.148.42.12 192.168.42.2 192.158.42.2
100.148.42.10
Storage 5800 V3 A 100.148.52.200
5800 V3 B 100.148.52.201
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 23
23
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
3.3.2 Improve performance test
Table 3-7 Improve performance test steps
Task Step Description
1 Base line test
1.1 Try a user session configuration, run the script “1.run.sh”, check the read I/O latency
1.2
Tune the user session until the read I/O latency go to about 10 ms, record the test
results (assume SESSIONS_BASE) and collect the IOPS performance per SAS disk
(assume HDD_IOPS_BASE)
2 5% SSD capacity test
2.1 Expand 3 SSD disks for the Disk Domain
2.2 Expand the Storage Pool, create high performance layer with 110GB capacity
2.3 Invoke “1.run.sh” with SESSION_BASE for 1 hours
2.4 Execute SmarTier data migration for the storage pool, record the distribution result
2.5 Run the script “1.run.sh”, tune the user session until the IOPS per SAS disk go to
HDD_IOPS_BASE, record the test results
3 10% SSD capacity test
3.1 Expand 1 SSD disks for the Disk Domain, wait until the rebalance complete
3.2 Expand the Storage Pool by 110GB capacity
3.3 Execute SmarTier data migration for the storage pool, record the distribution result
3.4 Repeat step 2.5
4 15% SSD capacity test (repeat step 3)
5 20% SSD capacity test (repeat step 3)
6 25% SSD capacity test (repeat step 3)
7 30% SSD capacity test (repeat step 3)
8 35% SSD capacity test (repeat step 3)
9 40% SSD capacity test (repeat step 3)
10 45% SSD capacity test (repeat step 3)
11 50% SSD capacity test (repeat step 3)
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 24
24
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
3.3.3 Reduce TCO test
Table 3-8 Reduce TCO test steps
Task Step Description
1 Base line test (75 SAS disks)
1.1 Try a user session configuration, run the script “1.run.sh”, check the read I/O latency
1.2
Tune the user session until the read I/O latency go to about 10 ms, record the test
results (assume SESSIONS_BASE) and collect the IOPS performance per SAS disk
(assume HDD_IOPS_BASE)
2 5% SSD capacity test
2.1 Expand 3 SSD disks for the Disk Domain
2.2 Expand the Storage Pool, create high performance layer with 110GB capacity
2.3 Invoke “1.run.sh” with the load of SESSION_BASE for 1 hours
2.4 Execute SmarTier data migration for the storage pool, record the distribution result
2.5 Remove 1 SAS disks, wait until the rebalance complete
2.6 Run the script “1.run.sh” with the load of SESSIONS_BASE, record the IOPS
performance per SAS disks
2.7 Repeat step 2.5 and 2.6 until the IOPS performance per SAS disks go to
HDD_IOPS_BASE
3 10% SSD capacity test
3.1 Expand 1 SSD disks for the Disk Domain, wait until the rebalance complete
3.2 Expand the Storage Pool by 110GB capacity
3.3 Execute SmarTier data migration for the storage pool, record the distribution result
3.4 Repeat step 2.5, 2.6, 2.7
4 15% SSD capacity test (repeat step 3)
5 25% SSD capacity test (expand 2 SSD disks and 220GB capacity, repeat step 3)
6 35% SSD capacity test (expand 2 SSD disks and 220GB capacity, repeat step 3)
7 45% SSD capacity test (expand 2 SSD disks and 220GB capacity, repeat step 3)
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 25
25
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
3.4 Improve performance test
3.4.1 Baseline test
Try a user session configuration, run the script “1.run.sh”, check the read I/O latency. Tune
the user session until the read I/O latency go to about 10 ms, record the test results (assume
SESSIONS_BASE) and collect the IOPS performance per SAS disk (assume
HDD_IOPS_BASE).
cd /home/oracle/oe12c
sh 1.run.sh -p cloud -c 1 -d pdb2t -n 300 -i 10 -r 20 -w 10 -o OSPSWD -s DBAPSWD
Parameters of script 1.run.sh
-p NODE_PREFIX: Oracle RAC node prefix
-c NODE_COUNT: Number of Oracle RAC nodes to be run the test
-d PDB1,... PDBs to connect
-n NumUsers1,... Sessions to create for PDBs
-i INTERVAL Performance statistic & swingbench verbose print interval seconds
-r RUN_TIME_MINS Run minutes, include warmup time
-w WARMUP_MINS Warm up minutes
-o OS_PSWD Current OS user password
-s SYSTEM_PSWD Password of dba user 'system'
Table 3-9 Baseline test result
SSDs SSD% Users TPS IOPS Read Lat SSD Load% IOPS/SAS
0 0% 300 1702 20797 9.20 ms 0 243.96
SESSION_BASE = 300, HDD_IOPS_BASE = 243.96.
3.4.2 5% SSD capacity test
Step 1 Expand Disk Domain
Login to 5800 V3 via Device Manager, go to “Provisioning > Disk Domain”, select
“DomainOracle”, click “Expand” to add 3 SSD disks to the disk domain.
Step 2 Expand Storage Pool
On Device Manager, go to “Provisioning > Storage Pool”, select “PoolData”, click “Expand”,
create “High performance layer” with 110GB capacity.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 26
26
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
Step 3 Configure SmartTier policy
On Device Manager, go to “Provisioning > Storage Pool”, select “PoolData”, click
“Properties”, go to “SmartTier Policy”, click “Enable I/O monitoring”.
Figure 3-3 Configure SmartTier policy
Step 4 Warm up SmartTier
Execute “1.run.sh” with the load of SESSIONS_BASE for one hour to warm up SmartTier
(monitoring the hot blocks).
sh 1.run.sh -p cloud -c 1 -d pdb2t -n 300 -i 10 -r 60 -w 10 -o OSPSWD -s DBAPSWD
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 27
27
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
Step 5 SmartTier data migration
On Device Manager, go to “Provisioning > Resource Performance Tuning > SmartTier”,
change the “Data Migration Speed” to “High”, choose “PoolData”, click “More > Start” to
execute the data migration.
Figure 3-4 SmartTier data migration
After the migration complete, click “Properties”, go to “SmartTier Status”, check the data
distribution of each storage tier.
Table 3-10 SmartTier data distribution
Tier Total Capacity Allocated Capacity Free Capacity
SSD 110 GB 99.347 GB 10.652 GB
SAS 4000 GB 3500.653 GB 499.347 GB
Step 6 Improve performance test
Run the script “1.run.sh”, tune the user session until the IOPS per SAS disk go to
HDD_IOPS_BASE, record the test results.
sh 1.run.sh -p cloud -c 1 -d pdb2t -n 450 10 -r 20 -w 10 -o OSPSWD -s DBAPSWD
Table 3-11 5% SSD capacity test result
SSDs SSD% Users TPS IOPS Read Lat SSD Load% IOPS/SAS
0 0 300 1702 20797 9.20 ms 0 243.96
3 5% 450 2724 32633 7.51 ms 38.3% 246.654
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 28
28
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
3.4.3 10% SSD capacity test
Expand 1 SSD disk to the disk domain “DomainOracle” and 110GB SSD capacity to the
Storage Pool “PoolData”. Then repeat the steps of 3.5.2.
Table 3-12 SmartTier data distribution
Tier Total Capacity Allocated Capacity Free Capacity
SSD 220 GB 198.003 GB 21.996 GB
SAS 4000 GB 3401.997 GB 598.003 GB
Table 3-13 10% SSD capacity test result
SSDs SSD% Users TPS IOPS Read Lat SSD Load% IOPS/SAS
0 0 300 1702 20797 9.20 ms 0 243.96
4 10% 530 3318 39585 6.77 ms 50.6% 240.973
3.4.4 15% SSD capacity test
Repeat the steps of 3.5.3.
Table 3-14 SmartTier data distribution
Tier Total Capacity Allocated Capacity Free Capacity
SSD 330 GB 297.160 GB 32.839 GB
SAS 4000 GB 3302.84 GB 697.160 GB
Table 3-15 15% SSD capacity test result
SSDs SSD% Users TPS IOPS Read Lat SSD Load% IOPS/SAS
0 0 300 1702 20797 9.20 ms 0 243.96
5 15% 625 3989 46450 6.09 ms 59.1% 239.946
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 29
29
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
3.4.5 20% SSD capacity test
Repeat the steps of 3.5.3.
Table 3-16 SmartTier data distribution
Tier Total Capacity Allocated Capacity Free Capacity
SSD 440 GB 396.050 GB 43.949 GB
SAS 4000 GB 3203.95 GB 796.050 GB
Table 3-17 20% SSD capacity test result
SSDs SSD% Users TPS IOPS Read Lat SSD Load% IOPS/SAS
0 0 300 1702 20797 9.20 ms 0 243.96
6 20% 675 4357 50127 5.16 ms 62.6% 239.053
3.4.6 25% SSD capacity test
Repeat the steps of 3.5.3.
Table 3-18 SmartTier data distribution
Tier Total Capacity Allocated Capacity Free Capacity
SSD 550 GB 494.886 GB 55.113 GB
SAS 4000 GB 3105.114 GB 894.886 GB
Table 3-19 25% SSD capacity test result
SSDs SSD% Users TPS IOPS Read Lat SSD Load% IOPS/SAS
0 0 300 1702 20797 9.20 ms 0 243.96
7 25% 650 4319 53249 4.38 ms 63.7% 236.936
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 30
30
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
3.4.7 30% SSD capacity test
Repeat the steps of 3.5.3.
Table 3-20 SmartTier data distribution
Tier Total Capacity Allocated Capacity Free Capacity
SSD 660 GB 594.312 GB 65.687 GB
SAS 4000 GB 3005.688 GB 994.312 GB
Table 3-21 30% SSD capacity test result
SSDs SSD% Users TPS IOPS Read Lat SSD Load% IOPS/SAS
0 0 300 1702 20797 9.20 ms 0 243.96
8 30% 700 4633 57223 4.13 ms 67.6% 235.08
3.4.8 35% SSD capacity test
Repeat the steps of 3.5.3.
Table 3-22 SmartTier data distribution
Tier Total Capacity Allocated Capacity Free Capacity
SSD 770 GB 693.003 GB 76.996 GB
SAS 4000 GB 3.229 TB 1.067 TB
Table 3-23 35% SSD capacity test result
SSDs SSD% Users TPS IOPS Read Lat SSD Load% IOPS/SAS
0 0 300 1702 20797 9.20 ms 0 243.96
9 35% 800 5524 67786 3.33 ms 71.2% 243.013
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 31
31
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
3.4.9 40% SSD capacity test
Repeat the steps of 3.5.3.
Table 3-24 SmartTier data distribution
Tier Total Capacity Allocated Capacity Free Capacity
SSD 880 GB 792.003 GB 87.996 GB
SAS 4000 GB 3.132 TB 1.164 TB
Table 3-25 40% SSD capacity test result
SSDs SSD% Users TPS IOPS Read Lat SSD Load% IOPS/SAS
0 0 300 1702 20797 9.20 ms 0 243.96
10 40% 960 6447 80442 3.26 ms 75.5% 241.92
3.4.10 45% SSD capacity test
Repeat the steps of 3.5.3.
Table 3-26 SmartTier data distribution
Tier Total Capacity Allocated Capacity Free Capacity
SSD 990 GB 891.050 GB 98.949 GB
SAS 4000 GB 3.036 TB 1.260 TB
Table 3-27 45% SSD capacity test result
SSDs SSD% Users TPS IOPS Read Lat SSD Load% IOPS/SAS
0 0 300 1702 20797 9.20 ms 0 243.96
11 45% 1020 6841 83840 2.98 ms 80.3% 210.186
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 32
32
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
3.4.11 50% SSD capacity test
Repeat the steps of 3.5.3.
Table 3-28 SmartTier data distribution
Tier Total Capacity Allocated Capacity Free Capacity
SSD 1100 GB 989.992 GB 110.007 GB
SAS 4000 GB 2.939 TB 1.357 TB
Table 3-29 50% SSD capacity test result
SSDs SSD% Users TPS IOPS Read Lat SSD Load% IOPS/SAS
0 0 300 1702 20797 9.20 ms 0 243.96
12 50% 1020 7103 89294 2.69 ms 82.8% 185.773
3.4.12 Clean up the environment
Stop Oracle Database, dismount ASM disk groups, remove LUNs “OracleData_001~004”
from LUN group, delete the LUNs, delete Storage Pool “PoolData”.
Re-create Storage Pool “PoolData” and LUNs “OracleData_001~004”, add LUNs to LUN
group, rescan LUNs on the two hosts, re-create ASM disk groups, restore database from
backup set.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 33
33
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
3.4.13 Test results summary
Table 3-30 Improve performance test results
SSDs SSD% Users TPS IOPS Read Lat SSD Load% IOPS/SAS
0 0 300 1702 20797 9.20 ms 0 243.96
3 5% 450 2724 32633 7.51 ms 38.3% 246.654
4 10% 530 3318 39585 6.77 ms 50.6% 240.973
5 15% 625 3989 46450 6.09 ms 59.1% 239.946
6 20% 675 4357 50127 5.16 ms 62.6% 239.053
7 25% 650 4319 53249 4.38 ms 63.7% 236.936
8 30% 700 4633 57223 4.13 ms 67.6% 235.08
9 35% 800 5524 67786 3.33 ms 71.2% 243.013
10 40% 960 6447 80442 3.26 ms 75.5% 241.92
11 45% 1020 6841 83840 2.98 ms 80.3% 210.186
12 50% 1020 7103 89294 2.69 ms 82.8% 185.773
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 34
34
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
3.5 Reduce TCO test
3.5.1 Baseline test
Repeat the steps in section 3.5.1.
3.5.2 5% SSD capacity test
Step 1 Expand Disk Domain
Login to 5800 V3 via Device Manager, go to “Provisioning > Disk Domain”, select
“DomainOracle”, click “Expand” to add 3 SSD disks to the disk domain.
Step 2 Expand Storage Pool
On Device Manager, go to “Provisioning > Storage Pool”, select “PoolData”, click “Expand”,
create “High performance layer” with 110GB capacity.
Step 3 Configure SmartTier policy
On Device Manager, go to “Provisioning > Storage Pool”, select “PoolData”, click
“Properties”, go to “SmartTier Policy”, click “Enable I/O monitoring”.
Step 4 Warm up SmartTier
Execute “1.run.sh” with the load of SESSIONS_BASE for one hour to warm up SmartTier
(monitoring the hot blocks).
sh 1.run.sh -p cloud -c 1 -d pdb2t -n 300 -i 10 -r 60 -w 10 -o OSPSWD -s DBAPSWD
Step 5 SmartTier data migration
On Device Manager, go to “Provisioning > Resource Performance Tuning > SmartTier”,
change the “Data Migration Speed” to “High”, choose “PoolData”, click “More > Start” to
execute the data migration.
Figure 3-5 SmartTier data migration
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 35
35
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
After the migration complete, click “Properties”, go to “SmartTier Status”, check the data
distribution of each storage tier.
Table 3-31 SmartTier data distribution
Tier Total Capacity Allocated Capacity Free Capacity
SSD 110 GB 99.347 GB 10.652 GB
SAS 4000 GB 3500.653 GB 499.347 GB
Step 6 Reduce TCO test
Remove 1 SAS disks, wait until the rebalance complete.
Run the script “1.run.sh” with the load of SESSIONS_BASE, record the IOPS performance
per SAS disks
sh 1.run.sh -p cloud -c 1 -d pdb2t -n 300 10 -r 20 -w 10 -o OSPSWD -s DBAPSWD
Continue to remove SAS disks one by one until the IOPS performance per SAS disks go to
HDD_IOPS_BASE
Table 3-32 5% SSD capacity test result
SSDs HDDs SSD% TPS IOPS Read Lat SSD Load% IOPS/SAS
0 75 0 1702 20797 9.20 ms 0 243.96
3 55 5% 2000 24290 5.64 ms 38.5% 248.745
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 36
36
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
3.5.3 10% SSD capacity test
Expand 1 SSD disk to the disk domain “DomainOracle” and 110GB SSD capacity to the
Storage Pool “PoolData”. Then repeat the steps of 3.6.2.
Table 3-33 SmartTier data distribution
Tier Total Capacity Allocated Capacity Free Capacity
SSD 220 GB 198.003 GB 21.996 GB
SAS 4000 GB 3401.997 GB 598.003 GB
Table 3-34 10% SSD capacity test result
SSDs HDDs SSD% TPS IOPS Read Lat SSD Load% IOPS/SAS
0 75 0 1702 20797 9.20 ms 0 243.96
4 45 10% 2157 26108 4.34 ms 54.3% 240.089
3.5.4 15% SSD capacity test
Repeat the steps of 3.6.3.
Table 3-35 SmartTier data distribution
Tier Total Capacity Allocated Capacity Free Capacity
SSD 330 GB 297.160 GB 32.839 GB
SAS 4000 GB 3302.84 GB 697.160 GB
Table 3-36 15% SSD capacity test result
SSDs HDDs SSD% TPS IOPS Read Lat SSD Load% IOPS/SAS
0 75 0 1702 20797 9.20 ms 0 243.96
5 40 15% 2243 27342 3.37 ms 60.4% 245.925
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 37
37
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
3.5.5 25% SSD capacity test
Repeat the steps of 3.6.3.
Table 3-37 SmartTier data distribution
Tier Total Capacity Allocated Capacity Free Capacity
SSD 550 GB 494.886 GB 55.113 GB
SAS 4000 GB 3105.114 GB 894.886 GB
Table 3-38 25% SSD capacity test result
SSDs HDDs SSD% TPS IOPS Read Lat SSD Load% IOPS/SAS
0 75 0 1702 20797 9.20 ms 0 243.96
7 35 25% 2395 28918 2.25 ms 67.4% 243.685
3.5.6 35% SSD capacity test
Repeat the steps of 3.6.3.
Table 3-39 SmartTier data distribution
Tier Total Capacity Allocated Capacity Free Capacity
SSD 770 GB 693.003 GB 76.996 GB
SAS 4000 GB 3.229 TB 1.067 TB
Table 3-40 35% SSD capacity test result
SSDs HDDs SSD% TPS IOPS Read Lat SSD Load% IOPS/SAS
0 75 0 1702 20797 9.20 ms 0 243.96
9 25 35% 2513 30285 1.54 ms 77.8% 247.6
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 38
38
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
3.5.7 45% SSD capacity test
Repeat the steps of 3.6.3.
Table 3-41 SmartTier data distribution
Tier Total Capacity Allocated Capacity Free Capacity
SSD 990 GB 891.050 GB 98.949 GB
SAS 4000 GB 3.036 TB 1.260 TB
Table 3-42 45% SSD capacity test result
SSDs HDDs SSD% TPS IOPS Read Lat SSD Load% IOPS/SAS
0 75 0 1702 20797 9.20 ms 0 243.96
12 16 45% 2605 31249 1.02 ms 88% 217
3.5.8 Test results summary
Table 3-43 Reduce TCO test results
SSDs HDDs SSD% TPS IOPS Read Lat SSD Load% IOPS/SAS
0 75 0 1702 20797 9.20 ms 0 243.96
3 55 5% 2000 24290 5.64 ms 38.5% 248.745
4 45 10% 2157 26108 4.34 ms 54.3% 240.089
5 40 15% 2243 27342 3.37 ms 60.4% 245.925
7 35 25% 2395 28918 2.25 ms 67.4% 243.685
9 25 35% 2513 30285 1.54 ms 77.8% 247.6
12 16 45% 2605 31249 1.02 ms 88% 217
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 39
39
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
3.6 Conclusion
3.6.1 Improve performance
Using OceanStor V3 storage’s SmartTier feature, administrators are able to improve Oracle
database’s transaction throughput and transaction response time by add few number of SSD
disks to an existing Disk Domain. During the lifecycle of enterprise information system,
SmartTier is useful to solve the performance issues brought by business growth or data
changes.
On database administrators’ perspective; allocating SSD to hold 5% ~ 50% capacities of user
data (tables and indexes) can improve the transaction throughput to 160% ~ 417%.
Figure 3-6 Transaction throughput improvement
100%
160% 195%
234% 256% 254%
272%
325%
379% 402% 417%
0%
100%
200%
300%
400%
500%
0
2000
4000
6000
8000
10000
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50%
Imp
rove
me
nt
TPS
SSD capacity% of user data
Improvement TPS
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 40
40
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
On storage administrators’ perspective, allocating SSD to hold 5% ~ 50% capacities of user
data (tables and indexes) is able to improve the I/O throughput to 157% ~ 429%, to improve
the I/O latency to 82% ~ 29%.
Figure 3-7 I/O throughput improvement
Figure 3-8 I/O latency improvement
100%
157% 190%
223% 241% 256%
275%
326%
387% 403% 429%
0%
100%
200%
300%
400%
500%
0
20000
40000
60000
80000
100000
120000
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50%
Imp
rove
me
nt
IOP
S
SSD capacity% of user data
Improvement IOPS
100%
82% 74%
66% 56%
48% 45% 36% 35% 32% 29%
0%
20%
40%
60%
80%
100%
120%
-1
1
3
5
7
9
11
13
15
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50%
Imp
rove
me
nt
Re
ad L
ate
ncy
(m
s)
SSD capacity% of user data
Improvement Read Lat
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 41
41
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
3.6.2 Reduce TCO
Using OceanStor V3 storage’s SmartTier feature, CIOs are able to reduce the TCO (total cost
of ownership) of information system. Large number of HDDs can be replaced by few
numbers of SSDs, the cost of device, power consumption, space and management is reduced.
The solution provides 7 storage configurations supporting an Oracle database with 1700 TPS
performance needs with different number of SAS disks (HDDs) and SSD disks (SSDs).
Although the test on the 7 configuration is based on the same number of SwingBench users,
TPS performance is improved a little because SSD disks makes the read I/O latency reduced.
Figure 3-9 Performance needs satisfaction
Figure 3-10 Read I/O latency
1702 2000 2157
2243 2395
2513 2605
0
500
1000
1500
2000
2500
3000
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6 7
TPS
Nu
mb
er
of
dis
ks
SSDs HDDs TPS
9.2
5.64
4.34 3.37
2.25 1.54
1.02
0
1
2
3
4
5
6
7
8
9
10
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6 7
I/O
Lat
en
cy (
ms)
Nu
mb
er
of
dis
ks
SSDs HDDs Read Lat
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 42
42
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
The following figure shows the estimated relative cost of the 7 configurations, which is
reduced from 75 to 34 when adding more SSD disks and reclaiming SAS disks.
Figure 3-11 Relative cost
For simple measurement, assume the relative price of HDD is 1, SSD is 1.5.
The following figure shows the relative power consumption of the 7 configurations, which is
reduced from 75 to 30 when adding more SSD disks and reclaiming SAS disks.
Figure 3-12 Power consumption
For simple measurement, assume the relative power of HDD is 1 and SSD is 1.2.
75
60
51 48 46
39 34
0
10
20
30
40
50
60
70
80
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6 7
Re
lati
ve P
rice
Nu
mb
er
of
dis
ks
SSDs HDDs Relative Cost
75
59
50 46 43
36 30
0
10
20
30
40
50
60
70
80
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6 7
Po
we
r
Nu
mb
er
of
dis
ks
SSDs HDDs Relative Power
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 43
43
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
3.7 Best practices
3.7.1 Selecting acceleration areas
Oracle database files include control files, data files, online logs, and archived logs.
For OLTP applications, data files require performance acceleration areas because these files
feature frequent, small, random read and write I/Os. Online logs involve frequent I/O
operations but almost without hotspots, while control files and archived logs have less
frequent I/O operations. Therefore, in OLTP applications, it is recommended that SmartTier
be used to store Oracle data files on different tiers for performance acceleration.
For OLAP applications, data files feature large and multi-channel sequential I/Os. Since
database query and analysis are concurrently implemented in multiple flows and rarely
involve hotspot data, it is recommended that SmartTier not be used to accelerate the OLAP
application performance.
3.7.2 Estimite the hot spot ratio
There’s no obvious demarcation between hot or cold for SmartTier’s blocks. SmartTier
monitors the I/O frequency on each block (default 4MB), and tag the frequency value for each
block as the hot degree. During data migration phase, SmartTier migrate data to SSD as much
as possible base on the relative degree.
Based on the test result of this solution, migrating 5% ~ 10% user data (Oracle tables and
indexes) to SSD disks, 38.3% ~ 50.6% I/O load will be hold by SSD tier. We recommend
administrators allocate SSD capacity as of 5% ~ 10% of user data to get the best ROI.
Figure 3-13 Load profile vs. SSD capacity%
0.00%
38.30%
50.60% 59.10% 62.60% 63.70%
67.60% 71.20% 75.50%
80.30% 82.80%
0.00%
20.00%
40.00%
60.00%
80.00%
100.00%
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50%
SSD capacity% of user data
SSD Load% HDD Load%
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 44
44
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
Besides, SmartTier provides a function called “Forecast Analysis” to forecast best ROI based
on the I/O frequency history. The forecasting analysis module uses statistics formulas to
compute the total disk I/Os based on the performance statistics of each extent, average I/O
size of each data block, average IPOS, and properties of each tier (RAID levels and logical
member disks). Based on the performance properties and capacity of disks on each tier and
the computed total disk I/Os, heuristics is used to compute the number of disks required by
each tier and find out the capacity configuration with the highest cost-effectiveness.
To use the function, login to OceanStor V3 Device Manager, go to “Provisioning > Resource Performance Tuning > SmartTier”, choose the storage pool, click “Properties”, go to
“Forecast Analysis”, click “Result” to see the forecast.
Figure 3-14 Forecast analysis
3.7.3 Selecting an I/O Monitoring Policy
SmartTier allows service monitoring periods to be set for each storage pool based on
application scenarios. Service Monitoring Period specifies the time during which
performance data is collected for SmartTier performance analysis. The object of Service
Monitoring Period is a specific storage pool. The default period is 24/7 hours. Users can
customize this period to specific days or hours and can change it later.
The monitoring time range setting must be aligned with the characteristics of the target
application workloads. If an application runs from 9:00 to 21:00, it is recommended that its
I/Os be monitored during that period.
If applications stay inactive or I/Os are generated by unimportant tasks (for example,
background backup and data synchronization) during a specific period of time, this period
must be excluded from the service monitoring period to prevent the I/Os from affecting the
identification of hotspot data and non-hotspot data.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 45
45
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
3.7.4 Selecting a Data Migration Policy
Data Migration Granularity
SmartTier identifies LUNs based on a data migration granularity. The activity levels of data
blocks are identified by the granularity, and then the data blocks are migrated to appropriate
storage tiers based on their activity levels.
SmartTier supports a number of granularities, including 256 KB, 512 KB, 1 MB, 2 MB, 4 MB,
8 MB, 32 MB, and 64 MB, where the default granularity is 4 MB.
Choosing a smaller granularity helps improve storage resource utilization and reduces the
possibilities of migrating cold data together with hot data, bringing storage resources into full
use. However, big granularities, such as 32 MB and 64 MB, are recommended for streaming
media and video surveillance applications.
In most cases, the recommended granularity is 4 MB.
Migration Triggering Mode
A migration triggering mode indicates how data is migrated. SmartTier provides manual and
periodic migration modes, both of which apply to a single storage pool. The default mode is
Manual. The manual mode allows data migration at any time, while the periodic mode allows
data migration during a specific preset period. The migration triggering mode can be changed
later.
Manual data migration may be required when a storage pool needs to be reconfigured (for
example, after a new tier is added), new LUN properties need to take effect immediately, or
constantly changing hotspot data is distributed to all tiers. After a manual migration task is
initiated, SmartTier analyzes the current performance data and determines the destination tiers
of data based on the analysis.
If you choose the periodic migration mode, first set a data migration period. The data
migration schedule specifies when and where SmartTier migrates data.
Periodic migration is recommended, and data migration during off-peak hours, for example
from 1:00 to 5:00, is preferable.
Migration Policy
A migration policy specifies the data migration direction and applies to one LUN in a storage
pool. Four migration modes are optional: Automatic migration, Migration to a higher
performance tier, Migration to a lower performance tier, and No migration.
Automatic migration
This policy is the default policy set upon the creation of a LUN. SmartTier migrates data
blocks in a storage pool based on the ranking order of activity levels. Automatic
migration is implemented based on the analysis data in I/O monitoring. Therefore, for
automatic migration to take effect, you must enable I/O monitoring and set a service
monitoring period.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 46
46
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
Migration to a higher performance tier
If a LUN has high requirements for response, IOPS, and bandwidth, a high available
performance policy is appropriate for the LUN. This policy preferentially promotes data
blocks to the high performance tier and the performance tier. If the capacity of data
blocks to be migrated exceeds that of the high-performance tier (or performance tier),
only data blocks with a higher activity level are promoted to the high-performance tier
(or the performance tier).
Migration to a lower performance tier
This policy is recommended for the applications (such as file sharing) insensitive to
performance. This policy preferentially demotes data blocks to the performance tier and
the capacity tier, regardless of the activity levels of these data blocks.
No migration
This policy does not migrate data blocks among storage tiers. Data can be migrated only
when the data migration policy is changed.
It is recommended that the data migration policy be set and dynamically adjusted based
on the service environment and performance requirements of LUNs.
Migration Rate
A data migration rate specifies how fast data is migrated among storage tiers and applies to all
storage pools. SmartTier supports high, medium, and low data migration rates, where the
default migration rate is low. The following lists the migration speeds of the three rates.
High: 300 GB/hour to 350 GB/hour
Medium: 60 GB/hour to 72 GB/hour
Low: 30 GB/hour to 36 GB/hour
As periodic migration has been selected to migrate data during off-peak hours, medium or
high migration rate is recommended if you want to compete the hotspot data migration in a
short period of time. However, if the effect on performance is a more important consideration
during the migration, a low migration rate is recommended.
3.7.5 Selecting an Initial Capacity Allocation Policy
An initial capacity allocation policy specifies where new data is written into a LUN. The
policy applies to one LUN in a storage pool, and optional policies include Automatic
migration, Allocate from the high performance tier first, Allocate from the performance
tier first, Allocate from the capacity tier first.
The default policy is Automatic migration. It enables the storage system to distribute new
data to the capacity tier, the performance tier, and then the high performance tier in sequence.
Allocate from the high performance layer first, Allocate from the performance layer
first, or Allocate from the capacity layer first enables the storage system to distribute new
data to the high performance tier (performance tier or capacity tier) first, and only migrates
data to other performance tiers when the capacity of the high performance tier (performance
tier or capacity tier) is insufficient.
When data is initially written, SmartTier has no collected performance statistics about the data.
Without performance analysis, SmartTier cannot identify the activity level of the data.
Therefore, the automatic allocation policy is recommended. If you want to obtain optimal data
performance during the initial deployment, allocation from the high performance layer
first is recommended.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 47
47
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
Note that the initial capacity allocation policy cannot be changed after it is set.
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 48
48
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
4 Appendix
4.1 Reference Documents
HUAWEI OceanStor V3 converged storage systems SmartTier technical white paper
4.2 Terminology
Table 4-1 Terminology
Term Description
RAC Real Application Clusters
Flex Cluster A new cluster feature in Oracle 12c that supports the deployment of Leaf nodes
ASM Automatic Storage Management
Flex ASM A new feature in Oracle 12c that supports cross-node database access
CDB Container Database that can house multiple pluggable databases
Non-CDB Non-Container Database, a traditional database
PDB Pluggable Database that runs in a CDB
Hub Node Nodes in a Flex Cluster that can directly access shared storage
Leaf Node Nodes in a Flex Cluster that indirectly access shared storage
Server Pool A group of servers in a Flex Cluster (in this document)
VIP Virtual IP address, a type of IP address that floats between database nodes
OCR Oracle Cluster Registry that contains registry information about an Oracle cluster
Voting Disk Used as the cluster quorum disk
AU Oracle ASM Allocation Unit
SGA System Global Area
PGA Program Global Area
DBWn Oracle disk writing process
Huawei Proprietary and Confidential Copyright © Huawei Technologies Co., Ltd. 49
49
Using SmartTier of OceanStor V3 in Oracle database to improve performance and reduce TCO
- Implementation Guide
Term Description
LGWR Oracle log writing process
ARCn Oracle archiving process
RAID2.0+ Enhanced virtualization technology of Huawei unified storage systems
SmartTier Storage tiering technology of Huawei unified storage systems
Thick LUN Pre-allocation of storage space
Thin LUN Storage space allocation on write
Extent A basic unit of LUN space migration (as in this document)
SLA Service Level Agreement, a service agreement between a customer and a service
provider
ERP Enterprise Resource Planning, a type of enterprise operation and management software
EIP Enterprise Information Portal, a platform for integrated message transmission in
enterprise IT systems
OLTP Online Transaction Processing
TPM Transaction per minute
ROI Return of investment
TCO Total cost of ownership