data center survey: how many iops is enough. compares to site capacity and system latency
Post on 25-Dec-2014
170 Views
Preview:
DESCRIPTION
TRANSCRIPT
1
OBJECTIVE ANALYSIS – Semiconductor Market Research
How Many IOPS is
Enough?
Tom Coughlin
Coughlin Associate
&
Jim Handy
Objective Analysis
OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com
Outline
• The Survey
• Application Distribution and Attributes
• More Survey Results: IOPS, Capacity and
Latency
• Developing tiers of storage for enterprise
(and client) applications
• Implications/Projections
• Authors & Sources
2
2
OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com
Outline
• The Survey
• Application Distribution and Attributes
• More Survey Results: IOPS, Capacity and
Latency
• Developing tiers of storage for enterprise
(and client) applications
• Implications/Projections
• Authors and Sources
3
OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com
Our Survey
• Five-minute survey asked end users
– IOPS needs
– Capacity
– Latency
– System bottleneck IOPS
– Primary application
• Nearly 200 respondents
• Report analyzes and interprets the results
in depth4
3
OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com
Outline
• The Survey
• Application Distribution and Attributes
• More Survey Results: IOPS, Capacity and
Latency
• Developing tiers of storage for enterprise
(and client) applications
• Implications/Projections
• Authors and Sources
5
OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com
Applications Breakdown
6
Archiving and backup
Video Creation or Distribution
Cloud storage or services
OLTP
Scientific or Engineering
Databases
Mail server and mail storage
4
OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com
Application Types
• Cloud Storage/Service-Virtualization
• Databases
• On-Line Transaction Processing (OLTP)
• Video Creation and Distribution
• Science & Engineering
• Exchange Servers
7
OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com
Cloud Storage/Services& Virtualization
• The “IO Blender”– Many streams
– Scrambled I/O
– Highly random
• Suits SSDs better than
HDDs for rapid access
• Many VM and VDI
systems using flash cache
to meet demand speed
needs Image courtesy of Waring Corp.8
5
OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com
Databases
• Large data sets
• Random traffic
• High I/O load
• Early SSD adopter
(and before that
used DRAM-based
SSDs)
• Some users load
their entire DB on
flash memory9
OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com
OLTP(On-Line Transaction Processing)
• Verified writes– Write/read back
– Doubles I/O load
• No room for errors
• Speed is
imperative– Delays lose customers
10
6
OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com
Video Creation or Distribution
• Large data sets
• Multiple video
streams– Randomizes access
• High bandwidth
required
• Expensive talent– Don’t want them sitting
around waitingImage courtesy of the US Library of Congress
11
OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com
Recording Media Share inProfessional Video Cameras
2012 Digital Storage for Media and Entertainment Report, Coughlin Associates 12
7
OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com
Science & Engineering
• Complex problems– Genome sequencing
– CAD/CAM
– Natural Resources
– Nuclear modeling
• Large data sets
• Expensive talent– Don’t want them sitting
around waiting
Image courtesy of Wikimedia Commons
13
OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com
Exchange Server
• Multiple tasks– e-mail
– Scheduling/calendars
– Data storage
• Scads of users
• e-mail chaos– Multiple mailboxes
– Asynchronous sends & receives
– Spam & virus filters
Image courtesy of Dell Computer
14
8
OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com
Outline
• The Survey
• Application Distribution and Attributes
• More Survey Results: IOPS, Capacity and
Latency
• Developing tiers of storage for enterprise
(and client) applications
• Implications/Projections
• Authors and Sources
15
OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com
Let’s Look At the Overall Results
Report breaks analysis down by application
– That’s 54 charts
– Too much for this presentation!
• We’ll just look at Database
9
OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com
IOPS Required
0%
10%
20%
30%
40%R
esp
on
den
ts
10 100 1K 10K 100K 1M 10M
IOPS
OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com
Storage Capacity Needed
0%
5%
10%
15%
20%
25%
Resp
on
den
ts
1GB
10GB
50GB
100G
B
500G
B1T
B5T
B10
TB50
TB
100T
B
Capacity
10
OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com
How Do The TwoCompare To Each Other?
OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com
IOPS vs. Capacity
1
10
100
1,000
10,000
100,000
1.E+00 1.E+01 1.E+02 1.E+03 1.E+04 1.E+05 1.E+06 1.E+07
IOPS
Ca
pa
cit
y (
GB
)
20
11
OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com
System Bottleneck IOPSWhat is the Fastest Storage Your System Can Use?
0%
10%
20%
30%
40%R
esp
on
den
ts
10 100 1K 10K 100K 1M 10M
IOPS
OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com
Minimum Latency Requirement
0%
10%
20%
30%
40%
Resp
on
den
ts
10ns 1µs 100µs 10ms 1 sec
Latency
12
OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com
Outline
• The Survey
• Application Distribution and Attributes
• More Survey Results: IOPS, Capacity and
Latency
• Developing tiers of storage for enterprise
(and client) applications
• Implications/Projections
• Authors and Sources
23
OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com
DRAMs Now 6,000x HDD Speed!
From : HDDs and Flash Memory: A Marriage of Convenience
24
13
OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com
Memory and StoragePrice vs. Bandwidth
1.E+00
1.E+01
1.E+02
1.E+03
1.E+04
1.E+05
1.E+06
1.E+07
1.E-01 1.E+00 1.E+01 1.E+02 1.E+03 1.E+04 1.E+05 1.E+06
Price per Gigabyte
Ban
dw
idth
(M
B/s
)
Tape
HDD
NAND
DRAM
L3
L2
L1
101
100
102
103
104
105
106
107
101
102
103
104
105
106
100
10-1
From Objective Analysis: Are Hybrid Drives Finally Coming of Age?
25
OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com
Price/GB Roughly Follows IOPS
$0.01
$0.10
$1.00
$10.00
$100.00
$1,000.00
0 200 400 600 800 1,000 1,200
IOPS
$/G
B
26
14
OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com
Device IOPS by Form Factor
102 103 104 105 106
HDD SATA SAS/FC 2-Hop 1-Hop
27
OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com
Outline
• The Survey
• Application Distribution and Attributes
• More Survey Results: IOPS, Capacity and
Latency
• Developing tiers of storage for enterprise
(and client) applications
• Implications/Projections
• Authors and Sources
28
15
OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com
Implications/Projections (1)
• SSDs adoption will increase
– Usually more IOPS is better
• Fast storage is changing
– From short-stroked HDDs to SSDs
• HDDs becoming a tier behind SSDs
• Other system elements will become the
bottleneck
– Network, software, servers…
29
OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com
Implications/Projections (2)
• Users will focus more attention on IOPS
– Understanding will be greater than it is today
• Higher IOPS will support data/content
growth
– This means more storage
• SSD
• HDD
• Even tape
30
16
OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com
Report Compiles Survey Results
• An analysis of complete survey results by application
• Report, published July 2014, can be purchased for immediate download at www.Objective-Analysis.com.
• Orders can also be processed through Coughlin
Assocaites at:
http://www.tomcoughlin.com/techpapers.htm.
• You can contact Coughlin Associates by calling Tom at 408-871-8808, or e-mailing: Tom@TomCoughlin.com.
31
OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com
Analysts
32
Tom Coughlin, President, Coughlin Associates is a
highly-respected storage analyst and consultant with
over 30 years in the data storage industry in
engineering and management at high profile
companies.
Jim Handy is a widely recognized semiconductor
analyst, has over 30 years in the electronics industry.
His background includes marketing and design
positions at market-leading suppliers.Jim Handy
Objective Analysis
Thomas Coughlin
Coughlin Associates
17
OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com
Sources
• How Many IOPS do You Really Need?
• 2012 Digital Storage for Media and Entertainment, Coughlin Associates: (www.tomcoughlin.com/techpapers)
• HDDs and Flash Memory: A Marriage of Convenience,
Coughlin Associates and Objective Analysis, 2011
(www.tomcoughlin.com/techpapers)
• Two may be Better than One: Why HDD and Flash Belong
• SNIA SSSI White Paper, Coughlin/Handy 2010
• Are Hybrid Drives Finally coming of Age?, Objective
Analysis, 2010
33
top related