Download - State of Gluster Performance
State of Gluster PerformanceManoj Pillai - [email protected] Hat Perfomance and Scale Engineering
Agenda● Motivation● Gluster performance by workload class● EC for video storage use-cases● Work in Progress
Motivation● Understand gluster performance characteristics, for
different workload classes and workload parameters● Identify areas for improvement
Configuration● Systems:
○ 12 drives per server
○ Replica-2 results with single 12-disk RAID-6 device on each server
○ EC results with brick on each drive○ Upto 6 servers, 4 clients in these tests○ 1 10GbE link each
● Software versions○ Glusterfs-3.8.2-1.el7.x86_64, RHEL-7
● Standard tuning○ lookup-optimize, client-io-threads: on○ event-threads=4
Sequential I/O: Replica Volume
● With sufficient number of requests concurrently in flight, close to network line speed on both writes and reads.
Sequential I/O: Disperse Volume
● Similar to dist-rep, but larger gap in throughput between low thread count and high thread count.
Seq I/O: Comparison of Volume Types
● Replica-2 performs better for low thread count, especially on writes.
● Disperse excels for writes with higher no. of threads.
Seq Write: Comparison with kNFS
● kNFS better on single-thread tests, gluster does well with higher number of threads.
Seq Read: Comparison with kNFS
● kNFS dominates on single-thread test, hits line speed
Random Read: Comparison with kNFS
● “1-cached” is single thread test where server-side cache has not been dropped.○ Good test of latency in the I/O path
Random Write: Comparison with kNFS
● All of these tests bottleneck on disk○ RAID-6 not a good choice for random writes.
Random Read: Comparison of Volume Types
Random Write: Comparison of Volume Types
● Replica-2 suffers possibly because of RAID-6 poor performance on random writes.
Small File Read: Comparison of Volume Types
● Disperse at a clear disadvantage in these tests
Small File Create: Comparison of Volume Types
● Small file performance still under investigation
EC for Video Storage Use-Cases
● Lot of interest seen in this area● Significantly lower cost for EC, good performance
Work In Progress
● Rebalance Performance○ Crawl, migration and throttle settings
● Small-file Performance○ Performance degradation with increasing number of
files
● Tiering performance for small files○ Small file performance with EC cold tier
Theme
Thanks!!