data center survey: how many iops is enough. compares to site capacity and system latency

1

OBJECTIVE ANALYSIS – Semiconductor Market Research

How Many IOPS is

Enough?

Tom Coughlin

Coughlin Associate

&

Jim Handy

Objective Analysis

OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com

Outline

• The Survey

• Application Distribution and Attributes

• More Survey Results: IOPS, Capacity and

Latency

• Developing tiers of storage for enterprise

(and client) applications

• Implications/Projections

• Authors & Sources

2

2


Outline

• The Survey



Latency




• Authors and Sources

3


Our Survey

• Five-minute survey asked end users

– IOPS needs

– Capacity

– Latency

– System bottleneck IOPS

– Primary application

• Nearly 200 respondents

• Report analyzes and interprets the results

in depth4

3


Outline

• The Survey



Latency





5


Applications Breakdown

6

Archiving and backup

Video Creation or Distribution

Cloud storage or services

OLTP

Scientific or Engineering

Databases

Mail server and mail storage

4


Application Types

• Cloud Storage/Service-Virtualization

• Databases

• On-Line Transaction Processing (OLTP)

• Video Creation and Distribution

• Science & Engineering

• Exchange Servers

7


Cloud Storage/Services& Virtualization

• The “IO Blender”– Many streams

– Scrambled I/O

– Highly random

• Suits SSDs better than

HDDs for rapid access

• Many VM and VDI

systems using flash cache

to meet demand speed

needs Image courtesy of Waring Corp.8

5


Databases

• Large data sets

• Random traffic

• High I/O load

• Early SSD adopter

(and before that

used DRAM-based

SSDs)

• Some users load

their entire DB on

flash memory9


OLTP(On-Line Transaction Processing)

• Verified writes– Write/read back

– Doubles I/O load

• No room for errors

• Speed is

imperative– Delays lose customers

10

6


Video Creation or Distribution

• Large data sets

• Multiple video

streams– Randomizes access

• High bandwidth

required

• Expensive talent– Don’t want them sitting

around waitingImage courtesy of the US Library of Congress

11


Recording Media Share inProfessional Video Cameras

2012 Digital Storage for Media and Entertainment Report, Coughlin Associates 12

7


Science & Engineering

• Complex problems– Genome sequencing

– CAD/CAM

– Natural Resources

– Nuclear modeling

• Large data sets

• Expensive talent– Don’t want them sitting

around waiting

Image courtesy of Wikimedia Commons

13


Exchange Server

• Multiple tasks– e-mail

– Scheduling/calendars

– Data storage

• Scads of users

• e-mail chaos– Multiple mailboxes

– Asynchronous sends & receives

– Spam & virus filters

Image courtesy of Dell Computer

14

8


Outline

• The Survey



Latency





15


Let’s Look At the Overall Results

Report breaks analysis down by application

– That’s 54 charts

– Too much for this presentation!

• We’ll just look at Database

9


IOPS Required

0%

10%

20%

30%

40%R

esp

on

den

ts

10 100 1K 10K 100K 1M 10M

IOPS


Storage Capacity Needed

0%

5%

10%

15%

20%

25%

Resp

on

den

ts

1GB

10GB

50GB

100G

B

500G

B1T

B5T

B10

TB50

TB

100T

B

Capacity

10


How Do The TwoCompare To Each Other?


IOPS vs. Capacity

1

10

100

1,000

10,000

100,000

1.E+00 1.E+01 1.E+02 1.E+03 1.E+04 1.E+05 1.E+06 1.E+07

IOPS

Ca

pa

cit

y (

GB

)

20

11


System Bottleneck IOPSWhat is the Fastest Storage Your System Can Use?

0%

10%

20%

30%

40%R

esp

on

den

ts

10 100 1K 10K 100K 1M 10M

IOPS


Minimum Latency Requirement

0%

10%

20%

30%

40%

Resp

on

den

ts

10ns 1µs 100µs 10ms 1 sec

Latency

12


Outline

• The Survey



Latency





23


DRAMs Now 6,000x HDD Speed!

From : HDDs and Flash Memory: A Marriage of Convenience

24

13


Memory and StoragePrice vs. Bandwidth

1.E+00

1.E+01

1.E+02

1.E+03

1.E+04

1.E+05

1.E+06

1.E+07

1.E-01 1.E+00 1.E+01 1.E+02 1.E+03 1.E+04 1.E+05 1.E+06

Price per Gigabyte

Ban

dw

idth

(M

B/s

)

Tape

HDD

NAND

DRAM

L3

L2

L1

101

100

102

103

104

105

106

107

101

102

103

104

105

106

100

10-1

From Objective Analysis: Are Hybrid Drives Finally Coming of Age?

25


Price/GB Roughly Follows IOPS

$0.01

$0.10

$1.00

$10.00

$100.00

$1,000.00

0 200 400 600 800 1,000 1,200

IOPS

$/G

B

26

14


Device IOPS by Form Factor

102 103 104 105 106

HDD SATA SAS/FC 2-Hop 1-Hop

27


Outline

• The Survey



Latency





28

15


Implications/Projections (1)

• SSDs adoption will increase

– Usually more IOPS is better

• Fast storage is changing

– From short-stroked HDDs to SSDs

• HDDs becoming a tier behind SSDs

• Other system elements will become the

bottleneck

– Network, software, servers…

29


Implications/Projections (2)

• Users will focus more attention on IOPS

– Understanding will be greater than it is today

• Higher IOPS will support data/content

growth

– This means more storage

• SSD

• HDD

• Even tape

30

16


Report Compiles Survey Results

• An analysis of complete survey results by application

• Report, published July 2014, can be purchased for immediate download at www.Objective-Analysis.com.

• Orders can also be processed through Coughlin

Assocaites at:

http://www.tomcoughlin.com/techpapers.htm.

• You can contact Coughlin Associates by calling Tom at 408-871-8808, or e-mailing: [email protected].

31


Analysts

32

Tom Coughlin, President, Coughlin Associates is a

highly-respected storage analyst and consultant with

over 30 years in the data storage industry in

engineering and management at high profile

companies.

Jim Handy is a widely recognized semiconductor

analyst, has over 30 years in the electronics industry.

His background includes marketing and design

positions at market-leading suppliers.Jim Handy

Objective Analysis

Thomas Coughlin

Coughlin Associates

17


Sources

• How Many IOPS do You Really Need?

• 2012 Digital Storage for Media and Entertainment, Coughlin Associates: (www.tomcoughlin.com/techpapers)

• HDDs and Flash Memory: A Marriage of Convenience,

Coughlin Associates and Objective Analysis, 2011

(www.tomcoughlin.com/techpapers)

• Two may be Better than One: Why HDD and Flash Belong

• SNIA SSSI White Paper, Coughlin/Handy 2010

• Are Hybrid Drives Finally coming of Age?, Objective

Analysis, 2010

33

data center survey: how many iops is enough. compares to site capacity and system latency

Data & Analytics