![Page 1: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/1.jpg)
Comparing the Network Performance of AWS, Azure, GCP,
IBM Cloud, and Alibaba CloudAngelique Medina
![Page 2: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/2.jpg)
Cloud Research Journey
2018
• Microsoft Azure
• AWS
• Google Cloud
2019
• YoY changes (Azure, AWS, GCP)
• AWS Global Accelerator
• Broadband providers
• China
• Alibaba Cloud
• IBM Cloud
![Page 3: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/3.jpg)
Research Methodology
![Page 4: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/4.jpg)
Data Collection• Customized traceroute
– TCP-based
• Bidirectional measurements– Accounts for variation in forward
and return routes– Unidirectional metrics computed at
target agent
• Network path– Layer 3 per-hop AS path, metrics,
QoS re-markings
• End-to-end network metrics– Latency, loss, jitter TARGET MONITORING AGENT
SOURCE MONITORING AGENT
TCP TracesLoss
Latency Jitter
Bi-directional Tests
Network Path Data
![Page 5: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/5.jpg)
Data Processing• Managed vantage points
– Consistent measurement– Hosted in data centers
• 10 minute testing interval• 30 day data collection
– Data collection spans 09.01.2019 -10.30.2019
– Zero outages occurred during the data collection timeframe
– Loss and Jitter negligible– Latency differences most prominent
TARGET MONITORING AGENT
SOURCE MONITORING AGENT
TCP TracesLoss
Latency Jitter
Bi-directional Tests
Network Path Data
![Page 6: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/6.jpg)
Research Scope
END USER MEASUREMENTS
BROADBAND ISP MEASUREMENTS
MULTI-CLOUD CONNECTIVITY
AWS GLOBAL ACCELERATOR
INTER-REGION MEASUREMENTS
INTER-AZ MEASUREMENTS
![Page 7: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/7.jpg)
End User Measurements
98 Global Locations
1595 Regions 191525 21
![Page 8: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/8.jpg)
Broadband Measurements
Ashburn, VA
Chicago, IL
Dallas, TX
Los Angeles, CA
San Jose, CA
Seattle, WA
US East
US Central
US West
Canada
![Page 9: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/9.jpg)
Inter-Region Measurements
• Inter-region performance measured per-provider
• Provider performance compared against baseline latency metrics derived from vantage points proximal but external to each cloud provider
![Page 10: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/10.jpg)
Intra-Region (Inter-AZ) Measurements
• AWS: 6 regions
• Azure: 6 regions
• GCP : 6 regions
• Alibaba Cloud: 7 regions
• IBM Cloud: 4 regions
Region X
![Page 11: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/11.jpg)
Are Cloud BackbonesCreated Equal?
![Page 12: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/12.jpg)
Inter-Region Performance Reveals Connectivity
• Performance baselined with Internet averages to reflect relative performance
• 97% of IBM inter-regions pairs performed better than the Internet
• Alibaba Cloud had ~ 15% of inter-regions pairs perform worse than the Internet
• Inter-Region traffic for compute resources typically stays within the cloud provider network • Except for Alibaba Cloud
![Page 13: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/13.jpg)
What Does Strong Inter-AZ Performance Mean?
0.74ms
0.75msEast US 0.77ms North Europe
0.71ms
French Central
0.74msCentral US
1.63ms
us-west-1
1.6ms cn-shanghai
1.13ms
ap-southeast-1
1.51ms
ap-south-1
0.52ms
eu-central-1
0.92ms
us-east-1
0.81ms
0.75ms
us-east-1
0.77ms
eu-west-3
0.74ms eu-west-2
0.71ms
ap-south-1
0.91ms
sa-east-1
0.99msus-west-1
1.68ms
1.0msWashington
1.68ms Sydney
2.05ms London
2.02msDallas
.52ms
0.56ms
us-east4
0.4ms
eu-west2
0.63ms asia-south1
0.57ms asia-southeast1
0.61msus-west1
southamerica-east1
0.37ms
eu-west-1
0.27ms0.27ms
0.37ms
1.22ms
![Page 14: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/14.jpg)
BackboneFriendly
InternetIntensive
Two Types of User <-> Region Connectivity
SingaporeDATA CENTER
FrankfurtUSER
Internet Backbone
HongKong
Internet
Frankfurt
Backbone
![Page 15: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/15.jpg)
But Three Approaches
Backbone Friendly Internet IntensiveHybrid
![Page 16: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/16.jpg)
2018 vs. 2019 Performance
![Page 17: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/17.jpg)
450
0
50
100
150
200
250
300
350
400
ms
USER LOCATION
2018: AWS had High Performance Variability
Asia Europe North America Oceania South America
HOSTING REGION: MUMBAI, INDIABI-DIRECTIONAL LATENCIESEND-USER MEASUREMENTS2018 AWS Azure GCP
![Page 18: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/18.jpg)
450
0
50
100
150
200
250
300
350
400
ms
USER LOCATION
2019: AWS Improved Performance Variability
Asia Europe North America Oceania South America
HOSTING REGION: MUMBAI, INDIABI-DIRECTIONAL LATENCIES
2018 AWS Azure GCP
2019 ■ AWS ■ Azure ■ GCP
![Page 19: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/19.jpg)
2018: Sub-Optimal Connectivity to AWS India
Internet Path AWS Backbone
![Page 20: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/20.jpg)
2019: AWS Optimized Routing
Internet Path AWS Backbone
![Page 21: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/21.jpg)
450
0
50
100
150
200
250
300
350
400
ms
USER LOCATION
2018: GCP is 3x Slower From Europe to India
Asia Europe North America Oceania South America
HOSTING REGION: MUMBAI, INDIABI-DIRECTIONAL LATENCIES
2018 AWS Azure GCP
![Page 22: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/22.jpg)
No direct route from Europe to India
Why was GCP 3x Slower to Asia in 2018?
![Page 23: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/23.jpg)
450
0
50
100
150
200
250
300
350
400
ms
USER LOCATION
2019 Measurements Still Show High Latency
Asia Europe North America Oceania South America
HOSTING REGION: MUMBAI, INDIABI-DIRECTIONAL LATENCIES
2018 AWS Azure GCP
2019 ■ AWS ■ Azure ■ GCP
GCP has 3x the network latency
![Page 24: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/24.jpg)
GCP Infrastructure Updated in 2019
GCP infrastructure updated in 2019
![Page 25: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/25.jpg)
WARNING:Low Cloud Visibility Ahead
![Page 26: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/26.jpg)
2018: Reverse Path Visibility
2019: Reduced Visibility
GCP
![Page 27: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/27.jpg)
Root cause: GCP Changes in TTL Handling
![Page 28: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/28.jpg)
The Performance Impact of China’s Great Firewall
![Page 29: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/29.jpg)
All Cloud Providers Pay Performance Toll
![Page 30: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/30.jpg)
Viable Hosting Locations Outside of China
![Page 31: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/31.jpg)
Viable Hosting Locations Outside of China
![Page 32: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/32.jpg)
US Broadband to Cloud Connectivity
![Page 33: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/33.jpg)
US Broadband to Cloud Connectivity Is StrongBi
dire
ctio
nal L
aten
cy V
aria
tions
(ms)
■ AT&T ■ CenturyLink ■ Charter ■ Comcast ■ Cox ■ Verizon
USER CITIES
0
10
20
30
40
50
60
70
80
90
Chicago, IL Dallas, TX Los Angeles, CA San Jose, CA Seattle, WA
Bidirectional Latency to Azure East
![Page 34: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/34.jpg)
But, Exceptions Can Occur
0.00
10.00
20.00
30.00
40.00
50.00
60.00
70.00
80.00
Los Angeles, CA San Jose, CA Seattle, WA
BIDI
RECT
IONA
L LA
TENC
Y (M
S)
USER CITIES
Bidirectional Latency to GCP LA (us-west2)AT&T CenturyLink Charter Comcast Cox Verizon
![Page 35: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/35.jpg)
Routing Anomaly: SJC to LA via NJ
Traffic hairpins via East Coast
![Page 36: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/36.jpg)
Fast MTTR from GCP – Routing Anomaly Resolved!
Routing Anomaly Resolved
![Page 37: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/37.jpg)
AWS Global Accelerator Performance
![Page 38: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/38.jpg)
AWS Global AcceleratorInternet
AWS Global Accelerator
Users
![Page 39: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/39.jpg)
AWS Global Accelerator
San Francisco,CA, US
Bangalore,India
Seoul, South Korea
Ashburn, VA, US
UserData Center
Latency Jitter
7.46 ms 0.01 ms
Improvements
Latency JItter
-100.50 ms 0.45 ms
Improvements
Latency Jitter
82.7 ms 14.23 ms
Improvements
Your Mileage May Vary
Data collection Period: 10/05/2019 - 11/01/2019Optimizations in effect by AWS
![Page 40: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/40.jpg)
Continuous Optimizations in Progress
Vantage Points LATENCY
SourcesBaseline
Internet ConnectionGlobal Accelerator
ConnectionGlobal Accelerator
Connection
Bangalore, India (Reliance) 219.74 323.27 243.98
Los Angeles, CA 74.92 74.45 59.43
Oct. 2019 Dec. 2019
![Page 41: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/41.jpg)
Wrap Up
![Page 42: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/42.jpg)
Summary Findings• Cloud routing preferences continue to vary
o Backbone vs. Internet-centric vs. hybrid
• Inter-Region connectivity stays within the cloud provider networko Exception: Alibaba Cloud
• Inter-AZ latency < 2ms• AWS Global Accelerator
o Performance varies but optimization continues
• GCP Europe-to-India backbone route still pre-rollout for most geos
![Page 43: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/43.jpg)
Takeaways
• No steady state in the cloud
• Trust, but verify performance and routing expectations
• Visibility key to provider oversight and accountability
![Page 44: Comparing the Network Performance of AWS, Azure, GCP, IBM Cloud, and Alibaba … · 2020-02-11 · Inter-Region Performance Reveals Connectivity •Performance baselined with Internet](https://reader033.vdocuments.us/reader033/viewer/2022042100/5e7c3c1714e18f55cc3be431/html5/thumbnails/44.jpg)
Download the full report: ThousandEyes.com/cloud
@bitprints @archana_k7