cloud performance benchmarking

Cloud Performance Benchmarking

The Problem Statement

Test Performance of a Compute Instance on the cloudThe testing focuses on disk IO, network IO, and some index of CPU. This report outlines the testing procedure ,result data and analysis on the findings

Test Performance of a DB Service on the cloudThe testing focuses on database IOPS, QPS, Connection number and others, This report outlines the testing procedure ,result data and analysis on findings

Application Deployment Try out WordPress based web application is deployed on cloud. The deployment involve 1 instance of Compute and 1 Database service. This presentation outlines the procedure of deployment and any learnings associated with it etc.

Testing Steps

Set up

• For the purpose of this tests Alicloud is used

• For compute : An ECS instance with 2 virtual cores and 4GB is used

• For Database an RDS service used. Both being in the same region.

Steps

12

34

56

Launch an ECS and an RDS instance in the same regionWhitelist RDS for accessing from the ECS

Install httpd, PHP modules and Wordpress on the ECS instance. Configure Database access in wordpress config file

Run CPU, Memory and IO test on ECS. Monitor with top and then with AliCloud default Monitoring

Create Database instance for Wordpress app, create user for database access: Using Alicloud Console

Install Sysbench tool in ECS instance.

Run IOPS and Connections test for the RDS.

Test Report

IOPS Test on Compute Node

IOPS on Compute Node: Test Method FIO utility command is used# throw-away: 5 min warm-upfio --runtime=300 --time_based --clocksource=clock_gettime --name=randread --numjobs=8 \ --rw=randread --random_distribution=pareto:0.9 --bs=8k --size=10g --filename=fio.tmp

# Sample 1min runs with varying job threads and block sizefio --runtime=60 --time_based --clocksource=clock_gettime --name=randread --numjobs=1 \ --rw=randread --random_distribution=pareto:0.9 --bs=8k --size=10g --filename=fio.tmp

IOPS on Compute Node : Monitor from Cloud Console

IOPS on Compute Node: Results and Analysis

• Random Writes are optimized at 4k blocks

• Random Reads work best at 8k blocks

• Writes perform better than reads in this test

Issues

• IOPs for the writes were observed higher than that of in cloud console monitoring of Alicloud. This is possibly to due to write buffering ignored by fio

• Increasing number of threads does not increase IOPs. This is possibly due to what is supported in the virtualized disk controller/driver in the hypervisor.

CPU Test on Compute Node

CPU Performance Test on Compute Node : Testing Steps

Tools Used sysbench and top

sysbench --test=cpu --cpu-max-prime=40000 --num-threads=1 run

Top is used to observe peak CPU utilization in parallel to the sysbench run

CPU Performance Test on Compute Node : Monitor from Cloud Console

CPU Performance on Compute Node: Results & Analysis

# CPU Events

Number of Threads

Peak CPU Utilization % During Test

Total Time Taken in s

40000 1 50.1 99.86

40000 2 99.5 50.19

40000 3 99.8 50.12

40000 4 99.5 50.13

• No performance anomalies observed. Behaviour expected for CPU with 2 cores

• Comparing the time taken for the same test with other known CPUs the performance can be benchmarked

Issues

• There is no direct relation between CPU events / sec and real CPU clock speed. Only relative estimate indicates benchmarking against other processors

• This test is not suitable for observing occasional CPU bursts / CPU hogging due to shared nature of virtual instances. That needs real time monitoring tool for a longer period.

Network Performance Test

Testing MethodOn Server [root@iZu1iy3602qZ ~]# iperf -s -l 128k

------------------------------------------------------------

Server listening on TCP port 5001

TCP window size: 85.3 KByte (default)

------------------------------------------------------------

[ 4] local xx.xx.xx.xx port 5001 connected with xx.xx.xx.xx port 42490

[ ID] Interval Transfer Bandwidth

[ 4] 0.0-30.0 sec 175 MBytes 48.8 Mbits/sec

On Client# iperf -c xx.xx.xx.xx -l 128k -i 4 -t 30

------------------------------------------------------------

Client connecting to xx.xx.xx.xx, TCP port 5001

TCP window size: 45.0 KByte (default)

------------------------------------------------------------

[ 3] local 172.31.54.251 port 42489 connected with xx.xx.xx.xx 5001

[ ID] Interval Transfer Bandwidth

[ 3] 0.0- 4.0 sec 23.2 MBytes 48.8 Mbits/sec

[ 3] 0.0-30.2 sec 176 MBytes 49.0 Mbits/sec

Results & Analysis

With 1 Connections -> 0.0-30.2 sec 176 MBytes 48.9 Mbits/sec

With 4 Connections -> 0.0-30.2 sec 176 MBytes 49.0 Mbits/sec

• The bandwidth limit observed at 49 Mbps could be a throttling on the client or the server. The server was the ECS instance on Alicloud. The client was external to Alicloud on another cloud provider.

• Within Alicloud the speed was much better - 18 Gbps

RDS Performance

Using sysbench OLTP test mode

#Preparation

sysbench --test=oltp --oltp-table-size=200000 --mysql-host=<host> --mysql-db=perfdatabase --mysql-user=perfuser --mysql-password=password1 --db-driver=mysql prepare

# Read Test

sysbench --test=oltp --oltp-table-size=200000 --mysql-host=<host> --mysql-db=perfdatabase --mysql-user=perfuser --mysql-password=password1 --db-driver=mysql --max-time=60 --oltp-read-only=on --max-requests=0 --num-threads=8 run

Run 1 : 1 Connection # Read only Test

read: 55594

write: 0

other: 7942

total: 63536

transactions: 3971 (66.17 per sec.)

read/write requests: 55594 (926.38 per sec.)

other operations: 7942 (132.34 per sec.)

# Read & Write Test

read: 43120

write: 15400

other: 6160

total: 64680




Run 2 : 4 Parallel Connections # Read only Test

read: 232890

write: 0

other: 33270

total: 266160




# Read & Write Test

read: 154042

write: 55015

other: 22006

total: 231063





read: 471324

write: 0

other: 67332

total: 538656




# Read & Write Test

read: 290360

write: 103700

other: 41480

total: 435540





read: 707126

write: 0

other: 101018

total: 808144




# Read & Write Test

read: 482804

write: 172430

other: 68972

total: 724206





read: 707126

write: 0

other: 101018

total: 808144




# Read & Write Test

read: 708680

write: 253100

other: 101240

total: 1063020




Maximum Performance

Eventually the following maximums were reached

Maximum Number of Queries per second (QPS): 22 k

IoPS: Could not measure

Concurrent Connections: Tested upto 600 concurrent connections. After which server threw error FATAL: error 1040: Too many connections

RDS Monitoring Metrics

Issues

• Mysqlslap could not be run due to lack of database admin privilege : Error: mysqlslap: Cannot drop database 'mysqlslap' ERROR : OPERATION need to be executed set by ADMIN.

• IOPs could not be reliably measured without local access to mySQL instance.

• Only query performance could be measured

• The number of parallel connections can be indirectly measured by running netstat | grep <hostname>| wc -l when the test is going on

• Cloud console monitoring collects data every 300 seconds. For short tests running for about 1 min each, those test data are not reflected in the console monitoring graphs.

Default ECS Disk Usage Metrics

IOPS

Bytes / Sec

Sample Application Deployment on ECS and RDS

Wordpress Deployment• Create WordPress DB in RDS & whitelist the ECS Server• Create Wordpress DB and DB user in RDS• Install HTTPD and PHP module on the ECS instance• Configure Wordpress to use the RDS BD information• Following this use apache ab tool performance test the wordpress site for

concurrent requests, max TPS and latency measures.

ECS Instance

httpd

PHP Module

Wordpress (php)RDS Service

Wordpress DB

ConnectionWhitelisting

Thank You

cloud performance benchmarking

Software