andrzej jakowski, armoun forghan apr 2017 santa clara, ca · validated solution: apache hadoop*...

Andrzej Jakowski, Armoun Forghan

Apr 2017Santa Clara, CA

Non-Volatile Memory Solutions Group

Legal Disclaimers

Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Learn more at intel.com, or from the OEM or retailer.

No computer system can be absolutely secure.

Tests document performance of components on a particular test, in specific systems. Differences in hardware, software, or configuration will affect actual performance. Consult other sources of information to evaluate performance as you consider your purchase.

Cost reduction scenarios described are intended as examples of how a given Intel-based product, in the specified circumstances and configurations, may affect future costs and provide cost savings. Circumstances will vary. Intel does not guarantee any costs or cost reduction.

This document contains information on products, services and/or processes in development. All information provided here is subject to change without notice. Contact your Intel representative to obtain the latest forecast, schedule, specifications and roadmaps.

No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document.

Intel does not control or audit third-party benchmark data or the web sites referenced in this document. You should visit the referenced web site and confirm whether referenced data are accurate.

Intel, 3D XPoint, Xeon, and the Intel logo are trademarks of Intel Corporation in the U.S. and/or other countries. *Other names and brands may be claimed as the property of others.

© 2017 Intel Corporation.

2

Non-Volatile Memory Solutions Group 3

Agenda

Introduction

Intel® CAS Overview & Benefits

Ceph* Use Cases

Swift* Use Cases

Intel® CAS with Intel® Optane™ SSDs

Summary

*Other names and brands may be claimed as the property of others.


New Data Centers & Software-Defined Storage

Orchestrated Data Center

DPDK/SPDK Storage Libraries (ISA-L)

Ceph*, Swift*Hadoop*,Lustre*

Intel®CAS

Rack Scale

Data Analytics Software-Defined StorageCloud Storage Converged Infrastructure

Memory I/O Bandwidth

NetworkingMulti-Core Processor Quick Assist

New storage architectures (commodity hardware and open source software solutions)

Storage SW Acceleration

Flexible, cost effective solutions to drive value-add and differentiation

4

SSDNVM



The Problem: Software has Become the Bottleneck

Historically, storage media was many orders of magnitude slower than software and memory

3D XPoint™ memory media approaches DRAM latencies, blurring the lines between storage media and memory latency

Storage software stacks optimized for efficiency on spinning disks will obscure the value of new storage media

The Opportunity: Optimize for new storage technologiesDisclaimer: Software and workloads used in performance tests may have been optimized for performance only on Intel® microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components,software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products.

5

0%

20%

40%

60%

80%

100%

7200 RPM 15000 RPM SATA NAND Enterprise

NAND

Intel®

Optane™

SSD

3D XPoint™

Hardware vs. Software Latency

Drive Read Latency Software Overhead


Intel® CAS: Cost-Effective Way to Benefit from NVM Technology

1400x IOPS1

3x OLTP2

57x OLAP2

Intel® Cache Acceleration

Software (Intel® CAS)

Up to…

Intel® Data Center SSDP3700/P3600

S3700

Quickly Improve Application SLA

No Application Modification

No External StorageModification

1. Configuration used: Intel® Server model 2600GZ (Grizzly Pass); Dual Intel® E5-2680 processor (2.7GHz), 32GB memory; Seagate* ST1000NC000 SATA HDD Microsoft* Windows 2012R2 SP1, Intel® SSD DC P3700 -800GB, Intel® CAS 2.6 release, L2 cache on; IOmeter 10.22.2009; 4K Random Read test; 8-queue depth x 8 workers

2. Configuration used: Intel server model 2600GZ (Grizzly Pass); Dual Intel® Xeon® E5-2680 processor (2.7GHz), 96GB DDR3, VMware* 5.5, Intel® SSD DC P3700 -800GB, Intel® CAS 2.6, L2 cache off, 8x Seagate 146GB SAS in RAID5, VMs: Microsoft* Server 2008R2, 8GB, 2 Cores, IOMeter workloads: Media Player, Exchange Server, Web Server, 4K OLTP using QD4.1 worker *Other names and brands may be claimed as the property of others.

../CAS Collateral/videos/enterprise_install dec 2014.mp4


1. Data is read from backend storage and copied to the cache partition on the SSD

2. All writes are completed at SSD speeds3. Subsequent reads of cached data are returned at high-

performance SSD speed4. Dirty data is written opportunistically to backend storage

Intel® CAS Linux* Overview – Features Intel® CAS Architecture – Write-Back Mode(Other modes: WT, WA, PT)

Accelerate Your Data Center… Without Application or Infrastructure Changes

Caching approach featuring I/O classification

‒ The ability to selectively cache I/O by size & type

‒ Improves I/O efficiency overall -> spinning disks remain spinning for large data blocks

Read & write caching addresses I/O bottlenecks

Selectable cache line sizes

In-flight upgrades

Multi-Level caching for added performance

Application

Intel® CAS

Cache (SSD)Backend Storage (HDD)

Writes Initial Read Subsequent Reads

2

1

3

7



Intel® CAS Linux* Benefits & Recent Features

In Flight Upgrade Capability:

‒ Enables SW upgrade w/o impacting application/service availability

Selectable Cache Line Size:

‒ Reduces DRAM footprint via increased cache block size

‒ Improved cost efficiency

Optimized for low latency medium – Intel® Optane™ SSD

‒ I/O path in the SW is optimized for the latest, next gen storage -> Intel® Optane™ SSD

8



Performance counters:

‒ Cache usage statistics

‒ I/O request statistics

‒ Block statistics

Health information:

‒ Number of I/O errors on SSD, primary storage

Ability to export data to tabular and CSV formats

Option to drill-down: per cache, per primary store, per I/O-class

Enables performance analysis & tuning

Intel® CAS Linux* Telemetry Information


Ability to apply different I/O processing policies based on the type of I/O

Operator defines policies for I/O classes

SW detects I/O class and enforces the policy

Possibility to control eviction priority and cache insertion

Benefits

Improved cache space usage

Improved cost-effectiveness

CAS I/O Classes

Unclassified

Filesystem metadata (SB, inode, directory)

<=4KiB

<=16KiB

<=64KiB

<=256KiB

<=1MiB

<=4MiB

<=16MiB

<=64MiB

<=256MiB

<=1GiB

>1GiB

O_DIRECT

MISC

Intel® CAS Linux* – I/O Classes



Software Defined Storage: Ceph*/Swift*

Industry Challenge

Standard industry architectures result in high latency, low throughput, and limited flexibility for specific workloads

Software Defined Storage Challenge

Large number of metadata from Replication or Erasure Coding

Small random files create large performance issues for spinning drives

Gateway/proxy bottlenecks

I/O bottlenecks due to expanding apps

Continuous Storage Acquisitions

Regulatory need to store data

Must Meet Customer SLAs

11


Ceph* Challenges: (Yahoo!* Use Case)

Environment & Problem

Yahoo* Mail, Flickr*, and Tumblr* storage requirements (cold, warm, hot data)

Cost-effective solution to meet user SLAs

Solution – Intel® CAS + NVMe* SSDsAn Intel® DC SSD with NVMe* and Intel® CAS utilizes a tunable and selective I/O classification caching approach increasing storage cluster performance while saving cost.

Technology Benefits Cost reduction by less overprovisioning

Improved deployment with more predictable & less varied latencies

Ceph cluster performance: >2X throughput & <½ the latency1

12

1 Based on internal Yahoo measurements, 2015. Using a 600 OSD Ceph* cluster, 3PB storage, erasure coding 8+3 (Ten) 8T SATA disk, 1MB object size, Intel® CAS 3.0, Intel® P3600 1.6TB NVMe* SSD, single reads and writers. Testing done with and without Intel® CAS/SSD components. Each OSD node in the cluster contained: SERVER:HP ProLiant DL180 G6 ySPEC 39.5; CPU:2x Intel® Xeon® X5650 2.67GHz (HT enabled, total 12 cores, 24 threads); NETWORK: (2) HP NC362i/Intel® 82576 Gigabit, (2) Intel® 82599EB10Gbe; OS:RHEL 6.5, kernel 3.10-0-123.4.4.el7. The Intel® CAS 3.0 configuration added an SSD to each node in the system: SSD:(1)1.6TB Intel® P3600 SSD (10GB journal per OSD, 1.5TB cache).



Swift* Challenges:

Environment & Problem

Large amount of file system metadata required for reading & writing objects

Replication processes are time consuming & require intensive use of metadata

Solution – Intel® CAS + NVMe* SSDs

Cache filesystem metadata to eliminate inefficient disk accesses

Improve replication by caching the Swift* replication metadata and hash files

13

1 Based on internal Intel measurements, 2017. Using a 15 node Swift* cluster with replication, 8x 2T SATA disks, 64k to 1MB object sizes, Intel® CAS 3.1.1 using one Intel® SSD DC P3520 Series 1TB with NVMe*. Testing done with and without Intel® CAS/SSD components. Each storage node server in the cluster; CPU:2x Intel® Xeon® E5-2699 (2.3 GHz, 18 cores, 45MB cache); NETWORK:2x Intel® X540 & 2x Intel® X520 10Gbe; OS:Ubuntu 14.04.5, kernel 4.4.0-47-generic, Swift 2.9.0.2-4-trusty & controller 4.7.0.1.

Technology Benefits• Swift cluster performance: >3x throughput &

<½ the latency1

• Swift replication performance > 5x

(ex. 29hrs decreased to 5hrs. )1



Validated Solution: Apache Hadoop* with Ceph* StorageOptimized proof-of-concept (POC) has been built by Intel and QCT

14

Based on Intel & QCT measurements, 2016. Using an 8 storage node Ceph* cluster with replication, each with 24x6T SATA disks, 2x Intel® SSD DC P3700 Series, CPU: Intel® Xeon® E5 2680 (2.5 GHz, 12 cores, 30MB cache), NETWORK:2x Intel® X540. 8 Hadoop* data nodes using Ceph* RBD for storage, with CPU: Intel Xeon E5-2690 (2.6GHz, 12 cores, 30MB cache), NETWORK:2x Intel® X540. Testing done with and without Intel® CAS/SSD components. OS: Ubuntu 14.04, CephHammer* .94, Cloudera* CDH 5.7, Intel® CAS 3.0

**

Optimize performance with Intel® CAS and Intel® SSDs using NVMe*

Resolve input/output (I/O) bottlenecks

Provide better customer service-level-agreement (SLA) support

Provide up to a 60-percent I/O performance improvement2

Disaggregate storage and compute in Hadoop* by using Ceph* storage instead of direct-attached storage (DAS)

HDFS replication 1, Ceph* replication 2



0

10000

20000

30000

40000

50000

60000

70000

80000

0

50

10

0

15

0

20

0

25

0

30

0

35

0

40

0

45

0

50

0

55

0

60

0

65

0

70

0

75

0

80

0

85

0

90

0

95

0

10

00

10

50

11

00

11

50

12

00

12

50

13

00

13

50

14

00

14

50

15

00

15

50

16

00

16

50

17

00

17

50

18

00

18

50

19

00

19

50

20

00

20

50

21

00

21

50

Qu

eri

es

pe

r S

eco

nd

Time (s)

Accelerating All-Flash Database Storage

with Intel® Optane™ SSD and Intel® CAS

Time to Complete 1 Million Transactions in MySQL

Intel® SATA SSD RAID w/ Intel® Optane™ SSD + Intel® CAS

Intel® SATA SSD RAID

15

Intel® Optane™ SSD + Intel® CASFaster Transactions in MySQL*

faster results1

Sustained performance under load accelerates large batch workloads 5.6x

1. System configuration - Intel Server System R2208WT2YS, 2x Intel® Xeon® E5 2699v4, 128 GB DDR4 DRAM, boot drive- 1x Intel® SSD S3500 Series (240GB), database drives- 3x RAID 5 Intel® SSD S3520 Series (1.6 TB) with 1x Intel® RAID Controller RS3DC080, cache drive- Intel® SSD DC P4800X Series (375 GB), CentOS 7.3, MySQL Server 5.7.17, Sysbench 0.5 configured for 70/30 R/W OLTP transaction split using a 1500GB database.

*Other names and brands names may be claimed as the property of others

up to


Intel® Optane™ SSD DC P4800X Series

Form Factors and Capacities

Quality of Servicevs. Intel® SSD DC P3700 @ 70/301

70/30 Mixed up to 8x faster

Random Read up to 10x faster

Random Write up to 3x faster

99% QoS up to 60x better

99.99% QoS up to 77x better

Low Queue Depth Performancevs. Intel® SSD DC P37001

Responsiveness Under Loadvs. Intel® SSD DC P37001

200MB/s write pressure up to 12.5x faster

400MB/s write pressure up to 25x faster

600MB/s write pressure up to 35x faster

1. Common Configuration - Intel® 2U PCSD Server (“Wildcat Pass”), OS CentOS 7.2, kernel 3.10.0-327.el7.x86_64, CPU 2 x Intel® Xeon® E5-2699 v4 @ 2.20GHz (22 cores), RAM 396GB DDR @ 2133MHz. Intel® Optane™ Configuration – Intel® Optane™ SSD DC P4800X 375GB. NAND Configuration – Intel® SSD DC P3700 1600GB. Performance – measured under 4K 70-30 workload at QD1-16 using fio-2.15. QoS – measures 99% QoS under 4K 70-30 workload at QD1 using fio-2.15. Latency – Average read latency measured at QD1 during 4K Random Write operations using fio-2.15.

Form Factor Capacity Availability

AIC

375GB Initial Mar 19, broad 2H

750GB Initial Q2

1.5TB 2H

U.2

375GB Initial Q2

750GB Initial Q2

1.5TB 2H

Tests document performance of components on a particular test, in specific systems. Differences in hardware, software, or configuration will affect actual performance.


Summary & Call to Action

Intel® CAS:

Performance:

‒ Optimized for the latest storage technologies – Intel® Optane™ SSD

‒ Shows significant improvement (2x throughput & ½ latency) in SDS solutions

Compelling features:

‒ I/O classification

‒ Robust installation and SW upgradability

Call to Action: Download Intel® CAS free 120 day trial software and try it out today!

http://www.intel.com/content/www/us/en/software/intel-cache-acceleration-software.html

17

http://www.intel.com/content/www/us/en/software/intel-cache-acceleration-software.html

andrzej jakowski, armoun forghan apr 2017 santa clara, ca · validated solution: apache hadoop*...

Documents