microsoft azure web sites performance analysis lessons learned
TRANSCRIPT
Microsoft Azure Web SitesPerformance Testing Lessons Learned
Christopher [email protected]
@microsofttrend
www.microsofttrends.com
December, 2014
2
What is Azure Web Sites?Azure Websites is a fully managed Platform-as-a-Service (PaaS) that enables you to build, deploy and scale enterprise-grade web Apps in seconds.
Azure Web Sites is Available in Different Hosting Plans:FreeSharedBasic (Small, Medium and Large)Standard (Small, Medium and Large)
3
Azure Web Site Pricing (Dec 2014)Hosting Plan Features ~ Price per
Month
Free 5 connections, shared VM, no SSL, 1 GB disk space, lots of usage quotas $0
Shared 35 connections, 1 GB disk space, shared VM, SSL support, 100 web sites per plan, some usage quotas, custom domain support, no SLA, scale up to 10 instances
$10
Basic 350 connections, 500 web sites per plan, dedicated VM, 64 bit, custom domain SSL support, scale up to 3 instances, 10 GB storage, no usage quotas, 99.9% SLA.
Basic Small 1 Core, 1.75 GB RAM $60
Basic Medium
2 Cores, 3.5 GB RAM $119
Basic Large 4 Cores, 7 GB RAM $235
Standard Unlimited connections, 500 web sites per plan, dedicated VM, 64 bit, custom domain SSL support, scale up to 10 instances, 50 GB storage, no usage quotas, 99.9% SLA.
Standard Small
1 Core, 1.75 GB RAM $79
Standard Medium
2 Cores, 3.5 GB RAM $157
Standard Large
4 Cores, 7 GB RAM $314Prices subject to change…
4
Testing Methodology Implemented the ASP.NET Bakery Site in two configurations:Out of the box using SQL Compact (e.g. File system based database) (average page load time unloaded is 300 ms)
Migrated database to Azure SQL (average page load time unloaded is 29 ms) Apache Jmeter used to create load test to simulate requests for home page
and product pages Tested 1, 2, 4, 8, 12, 20, 50, 100, 200 and 500 concurrent users on all Azure
Web Site scale up options Tested scale out with 3 x Shared, 3 x Basic Medium and 3 x Standard Medium
instances In each test, ran 300 page requests per thread or until ~30,000 page views
5
Jmeter Example
6
New Relic for Monitoring
Web Requests Queuing Up…
7
Planning for Performance What is your unloaded web page performance? 30 ms or 300 ms? How many average concurrent users do you expect? 4 or 20 or 200? How many peak concurrent users do you expect? 10 or 50 or 200? How long do you expect to run at peak? 5 minutes or 1 hour or 12
hours? What is an acceptable maximum page load time? 1 second? 3
seconds? 10 seconds? What is your target hosting spend per month?
8
Calculating Projected Page ViewsMaximum Page Views per Day =
8,640,000 ms per day / average page load time * number of concurrent users.
For example, if we can serve 200 concurrent users a page every 27 ms that means we have capacity to serve 675 million page views per day!
9
Results: Bakery Running SQL CompactAverage Page Load Speed (ms)
Threads
Hosting Plan 1 2 4 8 12 20 50 100 200
Free 384 344 343 324 360 346 713 Fail Fail
Shared 399 353 345 338 383 350 404 1290 1788
Basic Small 427 434 795 1542 2246 3802 Fail Fail Fail
Basic Medium 410 387 412 415 445 655 1133 66001170
0
Basic Large 329 299 293 308 303 403 650 1543 2500
Standard Small 366 428 880 1816 2502 434110061 Fail Fail
Standard Medium 318 312 273 287 369 492 1212 5147 5024
Standard Large 319 292 275 288 272 350 741 1435 2644
10
Bakery Running SQL Compact
One Two Four Eight Twelve Twenty Fifty One Hundred
Two Hundred
0
2000
4000
6000
8000
10000
12000
Azure Web Site Average Page Load (SQL Compact)
FreeSharedStandard SmallStandard MediumStandard Large
Avera
ge P
ag
e L
oad
Tim
e (
ms)
11
Results: Bakery Running SQL AzureAverage Page Load Speed (ms)
Threads
Hosting Plan 1 2 4 8 12 20 50 100 200 500
Shared 42 36 29 30 26 28 29 36 49Fail
Basic Small 35 34 34 34 49 93 270 500 1115 2843
Basic Medium 30 26 26 29 29 40 108 251 534 1320
Basic Large 27 27 27 28 32 37 116 237 540 1367
Standard Small 29 25 32 35 70 136 356 717 1351 3000
Standard Medium 26 25 29 30 32 34 107 215 563 1300
Standard Large 26 26 26 27 28 31 55 118 200 620Shared 3 Instances 45 28 30 27 28 27 27 26 27 64Basic Medium x 3 Instances 28 27 27 27 29 29 33 53 68 124Standard Large x 3 Instances 30 27 26 27 26 27 32 31 36 40
12
Bakery Running Azure SQL
One Two Four Eight Twelve Twenty Fifty One Hundred
Two Hundred
Five Hundred
0
500
1000
1500
2000
2500
3000
3500
Azure Web Site Average Page Load (Azure SQL)
Shared with SQLStandard Small with SQLStandard Medium with SQLStandard Large with SQL
Avera
ge P
ag
e L
oad
Tim
e (
ms)
13
Bakery with Scale Out
One Two Four Eight Twelve Twenty Fifty One Hundred
Two Hundred
Five Hundred
0
20
40
60
80
100
120
140
Azure Web Site Average Page Load with Scaled Out Instances (Azure SQL)
Shared with SQLShared with SQL x 3 InstancesBasic Medium with SQL x 3 InstancesStandard Large with SQL x 3 Instances
Avera
ge P
ag
e L
oad
Tim
e (
ms)
14
Maximum Page Views per Hosting PlanHosting Plan Price
Maximum Page Views Per Day with Azure SQL
~ Cost Per Million Page Views
Maximum Page Views Per Day with SQL Compact
~ Cost per Million Page Views
Shared with SQL $10 240,000,000* $0.0054 10,693,069* $0.031
Basic Small with SQL $60 17,280,000 $0.1157 461,620 $4.333
Basic Medium with SQL $119 43,200,000 $0.0918 2,638,167 $1.504
Basic Large with SQL $235 36,455,696 $0.2149 4,287,841 $1.827
Standard Small with SQL $79 12,705,882 $0.2073 414,388 $6.355
Standard Medium with SQL $157 50,823,529 $0.1030 3,564,356 $1.468
Standard Large with SQL $314 55,741,935 $0.1878 4,937,142 $2.120
Shared with SQL x 3 $30 675,000,000* $0.0015
Basic Medium with SQL x 3 $357 348,387,096 $0.0342
Standard Medium with SQL x 3 $471 360,000,000 $0.0436
Standard Large with SQL x 3 $942 1,080,000,000 $0.0291
* Theoretical maximum only, will likely overload quota limits before achieving this target.
15
Real World: WordPress on Azure Web SitesAverage Page Load Speed (ms) Threads
Hosting Plan 1 2 4 8 12 20 50 100
Shared 1111 986 Fail Fail Fail Fail Fail Fail
Basic Small 1131 1617 2911 5890 8850 14700 35000 Fail
Basic Medium 1139 1155 1485 2825 4185 4268 17000 Fail
Basic Large 1021 1063 1012 1445 2186 3743 9000 17300
Shared x 3 1063 1122 1065 999 Fail Fail Fail Fail
Basic Medium x 3 1132 1119 1219 1355 1650 2377 5747 0
Standard Large x 3 1116 1127 1259 1050 1130 1250 3059 6253
16
Key Conclusions: Performance Baseline performance of page makes a big difference in ability to scale and the size of
hosting plan required to maintain performance under load. Azure SQL ran flawlessly without any bottlenecks under all scenarios – web server was
always slower than the DB. Shared works awesome – it scaled better than Standard Large! However, it can be quickly
overloaded because of quota restrictions. Shared x 3 instances means also 3x the usage quotas – good approach to handling sites that
need additional CPU or memory. Performance of Basic and Standard are similar – you’re paying more for features, not
performance. Basic Small and Standard Small are too small to scale – performance degrades very quickly. Scaling out number of instances improves load better than scaling up to the next size of VM.
Basic Medium x 3 performed significantly better than Standard Large x 1.
17
Key Conclusions: Economics Even running SQL Compact on Basic Small, we could still crank out more than 400K page views per day! Baseline performance makes a big difference in $$ / million page views. Investments in optimization can
lower your hosting costs by allowing you to run leaner and cheaper. Only use what you absolutely need to sustain your traffic – you can always scale up to a more expensive
plan and you only pay for when you’re using the higher performance. Clear winner in delivering pages per $$ is Shared hosting plan, if you can live within the quotas. Shared x 3 at $30 / month scales much better than a single Standard Large at $314.00 / month and quotas
are tripled! Basic / Standard Medium more economical than Basic / Standard Small under load because they can scale
better for the price. Scaling out instances is cheaper than scaling up to a bigger VM – e.g. Basic Medium X 3 instances
performed much better than a single Standard Large VM and is only a little bit more expensive ($357 / month vs. $314 / month).
Scaling out instances is also better because it supports auto-scaling. No auto-scaling support for moving up and down sizes of VMs.
