attributed consistent hashing for heterogeneous storage system the virtual nodes (a2) will be...
Post on 28-Sep-2020
3 Views
Preview:
TRANSCRIPT
Ø Attributed Consistent Hashing Ring Ø Add two attributes to nodes on the hashing ring Ø Capacity: the overall data capacity of a node Ø Bandwidth: the maximum bandwidth of a node
• Given a set of n nodes with attributes of E.g., mean node A1 has 1TB capacity and 146 MB/s bandwidth, node B1 has 256 GB capacity and 540 MB/s bandwidth • Virtual node has the same attributes with the physical node, e.g., A1 and A2
on the ring Ø Consider Capacity Attribute
Ø Making virtual nodes according to node capacity • Making virtual nodes for each physical node, where
is the capacity of node
• Proportionally adjust virtual node numbers if they are too large • To avoid the declustering of data movement on each node
Ø Consider Bandwidth Attribute Ø Define a range for each virtual node according to node bandwidth Ø Each node has a range on the hashing ring. The virtual node has the same
range with the physical node. Range length is computed with a predefined parameter p with node configuration
Ø Data Placement algorithm Ø Step 1: data is hashed to a position on the ring with hash value s Ø Step 2: data is assigned to a node with pseudo-number algorithm val = hash(x, s, max), where x is data ID, s and max are lower and
upper range limits of the hashing ring.
Ø Objectives Ø To design a data placement algorithm for heterogeneous storage
composing with hard disk driver (HDD) and solid state disk (SSD) Ø Based on consistent hashing to keep its inherent features while making
better utilization of heterogeneous storage devices Ø Background
Ø Cloud-scale storage system is an important building block of the cloud infrastructure, which demands efficient and flexible data distribution for performance improvement
Ø Heterogeneous storage system is a very promising trend in data centers Ø Contributions
Ø A novel data distributed placement algorithm based on consistent hashing for heterogeneous storage systems
Ø Improve the I/O performance while keeping storage space load balanced and well utilized
Methodology
Attributed Consistent Hashing for Heterogeneous Storage System
Abstract
Ø Heterogeneous device characteristics A heterogeneous storage system, which contains nodes with both HDDs and SSDs, is often desired to combine merits of different storage devices.
Ø I/O bandwidth comparison of the HDD (Seagate ST9500620NS) and SSD (Intel SSDSC2BA200G3T)
Ø SSD favors workloads with small I/O size • The bandwidth of SSD is 65 times higher than that of HDD when
the request size is 4KB Ø SSD excels in random I/Os generally
• The SSD achieves nearly 2 to 9 times higher throughput than that of HDD when the request size varies from 64KB to 4MB
Ø Consistent hashing algorithm Consistent hashing (used in Dynamo, Chord and Sheepdog) conceptually assigns nodes and data to a ring. The data is assigned to the “K closest” nodes with the hashing value. Virtual nodes can be added to improve load balance. Data x and its replicas will be mapped to 3 “closest” nodes (A, B, C) with clockwise way. The virtual nodes (A2) will be skipped for data reliability. Ø Challenge Consistent hashing is mainly designed for a homogeneous environment and lacks efficient method to distinguish heterogeneous device characteristics.
Motivation
Ø Simulation tests Ø Make tests in a modular consistent hashing library libch-placement Ø Results show our algorithm achieves similar calculation time and
memory usage with traditional consistent hashing Table 1 The calculation time of different algorithms (s)
Table 2 The memory usage in different configs (MB)
Ø Performance Evaluation on Sheepdog storage system
Ø On a cluster of 30 nodes (half SSDs) with FIO and Filebench Ø Use FNV-1 hash function for consistent hashing and SIMD-oriented
Fast Mersenne Twister (SFMT) for generate pseudo random numbers Ø Results show the performance can be improved more than 3 times
Performance Results
We propose a consistent hashing based data placement algorithm to promote better utilization of heterogeneous storage devices. This work is supported by the National Science Foundation under grant CNS-1338078 and IIP-1362134 through the Nimboxx membership contribution.
Conclusion
A
B
C
Dxr1
r2r3 A1
B1A2
C1xr1
r3 r2
D1
(a) Without virtual nodes (b) Using virtual nodes
DataVirtual node
DataNode
V = v1,!,vn{ } vi = ci,bi .a1 = 1000,146 ,b1 = 256,540 ,
A1
B1
C1
A2
B2
C2
<c1, b1>
<c2, b2>
<c3, b3>
Virtual node
<c1, b1>
<c2, b2>
<c3, b3>
cimin j∈V cj{ }
"
#
##
$
%
%%
ci
vi.
Node number Our algorithm Consistent hashing CRUSH 64 0.0131 0.0124 1.4591 512 0.0218 0.0204 9.1039 4096 0.0361 0.0332 70.2185 32768 0.1783 0.1948 582.4153
Node No. ×Virtual Node No. Our algorithm Consistent hashing 100 ×256 0.98 0.78 1000 ×256 9.77 7.81 10000 ×256 97.65 78.13 1000 ×1024 39.06 31.25 10000 ×4096 1953.0 1562.4
0
20
40
60
80
100
120
140
1K 4K 16K 64K 256K 1M 4M
Agg
rega
ted
Ban
dwid
th (
MB
/s)
Request sizes
Our algorithmConsistent hashingCRUSH (straw buckets)
0200400600800
100012001400160018002000
1 3 5 7 9 11 13 15
Aggr
egat
ed B
andw
idth
(MB/
s)
Number of clients
Our algorithmConsistent hashingCRUSH (straw buckets)
0
50
100
150
200
250
fileserver varmail webservervideoserver
Aver
age
late
ncy
(ms)
Applications
CRUSH (straw buckets)Consistent hashingOur algorithm
0
1000
2000
3000
4000
5000
6000
fileserver varmail webserver videoserver
IOPS
(op/
s)
Applications
Our algorithmConsistent hashingCRUSH (straw buckets)
(a) Random read performance with different request sizes
(b) Random read performance with different clients (scalability)
(c) Average latency (d) IOPS
0Max
A1
B1
A2
C1
xf(x, s, max)
C2
<1000, 146>
<500, 219>
<1000, 146>
<256, 540>
range
0 1 7.12.5 3.5
DataVirtual node
B2
Node RangeA [0, 1), [2.5, 3.5)B [1, 2.5)C [3.5, 7.1)
range length = node bandwidthp
p =146
top related