big data trends: cloud cost benchmark study
DESCRIPTION
A study of 250 companies, 250,000 instances, and 2.5 petabytes of data. Key insights about use of reserved instances, the instance marketplace, and top spend categories.TRANSCRIPT
DAVID FEINLEIBTHE BIG DATA GROUP, LLC
BIG DATA TRENDS:PUBLIC CLOUD COST BENCHMARK STUDY
250 COMPANIES250,000 INSTANCES2.5PB OF STORAGE
THIRD QUARTER 2012
SPONSORED BY
thebigdatagroup.com Copyright © 2012 The Big Data Group, LLC bigdatalandscape.com
THE BIG DATA GROUP 2
Web Browser Market Share Study Goals
• Benchmark industry public cloud costs and utilization
• Enable buyers to improve operational efficiency based on cloud utilization best practices
• Leverage previously unavailable sources of Big Data to provide market insights
THE BIG DATA GROUP
THE BIG DATA GROUP 3
Web Browser Market Share Study Data
• 250 companies
• 250,000 Amazon EC2 instances
• 7,543,955 instance hours
• 2.5PB of storage
• Measured Oct 1 - 31, 2012
THE BIG DATA GROUP
THE BIG DATA GROUP 4
Web Browser Market Share
Public cloud buyers can achieve
significant savings by optimizing usage
Key Findings
• 33% of instances are running as reserved instances but some 94% should be
• 17% of storage is unattached (unused)
• Elastic Cloud compute resources account for more spend (at 62% of total spend) than any other resource type
THE BIG DATA GROUP
Many instances are underutilized. Significant storage goes unattached. Many buyers are reluctant to purchase reserved instances, which can provide cost savings when used for at least three months.
THE BIG DATA GROUP 5
Web Browser Market Share Instance Definitions
THE BIG DATA GROUP
Type Instances
Small t1.micro, m1.small, c1.medium, m1.medium
Medium m1.large, m2.xlarge
Large m1.xlarge, c1.xlarge, m2.2xlarge, m2.4xlarge
Huge cc1.4xlarge, cc2.8xlarge, hi1,4xlarge, cg1.4xlarge
For the purposes of this study, instances were grouped into Small, Medium, Large, and Huge instance types. The instances included in each type are shown below.
THE BIG DATA GROUP 6
Web Browser Market Share
Small instances account for about 42% of launched instances, medium for about 40%, large for just over 17%, and huge instances for under 1%.
The majority of launched instances
are small and mid sized.
Launched Instances By Type
0.44%17%
40%
42%
Small Medium Large Huge
THE BIG DATA GROUP 7
Web Browser Market ShareSmall instances account for 53% of instance runtime hours while medium instances account for 29%.
Instance Runtime By Type
Huge1%Large
18%
Medium29%
Small53%
THE BIG DATA GROUP 8
Web Browser Market ShareSmall instances account for 53% of instance runtime hours compared with 42% of launched instances. Medium instances account for 29% of instance runtime hours compared with 40% of launched instances.
Smaller instances account for more
hours run compared with the number that
are launched
Launched And Runtime Instances
Huge0.64%Large
18%
Medium29%
Small53%
Runtime InstancesThird Quarter 2012
Huge0.44%
Large17%
Medium40%
Small42%
Launched InstancesThird Quarter 2012
THE BIG DATA GROUP 9
Web Browser Market ShareMedium instances are only about 12% utilized while small instances are just under 17% utilized. The low utilization rates of medium instances combined with medium instances accounting for some 40% of launched instances means that most customers use medium instances when they should use small instances instead.
Most customers use medium instances when they should
use small instances
Average Instance Utilization Rates
Small Medium Large Huge
3.9%
12.8%11.9%
16.7%
THE BIG DATA GROUP 10
Web Browser Market ShareAmazon Web Services customers in the sample group spend more on EC2 instances than on any other type of resource.
EC2 Instances account for the
majority of spend
Top Cloud Spend Categories
S3 Estimated26%
RDS Databases3%
EBS Volumes7%
EC2 Instances64%
Note: S3 Data is based on a smaller set of randomly selected customers
THE BIG DATA GROUP 11
Web Browser Market ShareSome 44% of customers operate 1 - 10 instances per month while 45% operate 11 - 100 instances per month.
The majority of public cloud customers
operate between 1 and 100 instances
per month
Deployment Size Breakdown
1%10%
45%
44%
1-10 instances/mo11-100 instances/mo101-500 instances/mo501+ instances/mo
THE BIG DATA GROUP 12
Web Browser Market ShareOnly 33% of instances are reserved today, while some 94% of instances, if converted to reserved instances, would result in expected cost savings.
Reserved and Unreserved InstancesThird Quarter 2012
Reserved And Unreserved Instances
Unreserved67%
Reserved33%
Actual Recommended
94%
33%
Reserved Instances - Actual and RecommendedThird Quarter 2012
Most customers underutilize reserved instances, losing out
on significant cost savings
THE BIG DATA GROUP 13
Web Browser Market Share
Many reasons buyers cite for not
purchasing reserved instances prove
financially unsound
Top Reasons Customers Avoid Reserved Instances
• Expect project to run only for a short time
• Undecided about proper instance size
• Lack of time to purchase
• Fear of commitment to reserved instances
THE BIG DATA GROUP
Many buyers are reluctant to purchase reserved instances, which provide significant cost savings when used for between three and seven months. Projects typically last twice as long as buyers expect and buyers tend to stay with the same instance size for at least half a year, making the purchase of reserved instances worthwhile.
THE BIG DATA GROUP 14
Web Browser Market Share Emerging Trends
• Amazon RDS has not gained widespread adoption, likely due to the difficulty of moving to and from RDS databases and the inability to sync only specific database tables
• Amazon Glacier, which is 10X cheaper than Simple Storage Service (S3) but significantly slower, is expected to replace a lot of S3 storage for digital archiving
• The Reserved Instance Marketplace will need changes to gain widespread adoption
The marketplace lacks significant volume today because it is relatively new and because a company needs to have a US bank account to use it. In the future, without changes, we do not expect the marketplace to gain significant volume because prices tend to go down over the reservation term, making the resale of reserved instances at originally committed prices uncompetitive with lower prices at the time of resale
THE BIG DATA GROUP
THE BIG DATA GROUP 15
Web Browser Market ShareCustomers can
achieve significant cost savings through
proper public cloud management
Recommendations For Buyers
• Evaluate
• Applicability of reserved instances
• Any unattached storage
• Instance sizes relative to utilization
• Consider
• Smaller instances
• Reserved instances
• Retiring unattached storage
• Ensure total response time stays within acceptable range (1.5 - 2.5 seconds)
THE BIG DATA GROUP
THE BIG DATA GROUP 16
Web Browser Market Share For More Information
David [email protected]
THE BIG DATA GROUP