Cloud SUT proposal

OSGcloud group

Objective

• To fill in the Research the group about the thinking within the OSG working group

• To solicit new ideas/proposals

Definition of cloud (NIST)

“Cloud computing is a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources that can be rapidly provisioned and released with minimal management effort or service provider interaction. The model promotes availability and includes five essential characteristics, three service models and four deployment models.”

Characteristics• Cloud Characteristics from the NIST list

– On-demand self service– Broad network access– Resource pooling– Elasticity– Measured service

• Characteristic components separated by layer (By osgcloud group)– Client

• Standard client• Network connectivity• Access• Security• identity

– Server and Storage• Processing power• Memory• Storage• Network• Power consumption• Performance measurement

– Internet• Speed• Reliability• Availability

Metrics

• Elasticity– Provisioning Interval– Agility

• Durability• Response time• Throughput• Reliability• Power• Price

Potential Workloads/Use cases (some workloads listed)

• Data Analytics Expert Search Clustering Customer Segmentation

• Data Warehousing Pipelines bring in data feed and clean and transform it (example: logs from web servers) Iterative processing : one very large data set is maintained (often in the form of a graph).

• Business OLTP• Mail• Memory Cloud

– Key-value pair databases• Social Networking

– Web2.0 based application– Write/read workload– Memory cloud– Search engine

Issues with defining the SUT

• Cloud offerings may be different.• Inability on the user’s side to find out what

exact hardware is being used and how much is being used.

• Migration of hardware from one platform to another.

• Temporal aspect.

Multiple ways of classifying Cloud SUTs

• Virtualized, non-virtualized, mixedWhether the cloud offering is virtualized, non-virtualized, or mixed. Non-virtualized clouds can also be elastic (think grid computing)

• IaaS, PaaS, SaaSWhether the cloud offering is IaaS, PaaS, SaaS, or a mixture of these (e.g., Azure). It is easier to compare providers that do not mix service offerings.

• white box vs. black boxWhether benchmark and measurements will measure (1) and (2) as a white box (from hardware to the software stack) or as a black box (what customer purchases).

• private, public, or hybrid, and whether it is solely for enterprises.

Cloud Offering categories

• Cloud as a black box– No control over what is the exact hardware/software,

whether it changes with time etc.– User gets a description/QOS guarantees of what he/she is

paying for. – Unique identifier tied to the offering.

• Cloud as a white-box– Full control and knowledge of part numbers and software.– Description available in terms of exact hardware and

software.

Who is interested in what + Issues

Hardware vendors selling systems to Cloud vendors – White-box approach

End customer trying to buy a Cloud service – Black box approach

Resources to quantify

• IAAS– Compute resource– Memory Resource (capacity/bandwidth)– Network capacity– Disk/Storage Capacity

• PAAS– Hardware may not be described by vendor– Possible to measure as in IAAS

• SAAS– Hardware not described– Is it important to know about the hardware resources?

Proposal to Describe a Cloud based SUT

• Users would describe SUT based on what they pay for. Yet, there is a need for other engineering details.

• Use micro-measurement tools to measure capacities of various elements.

• The micro-benchmarks should be easy and quick to run and provide numbers describing different types of resources.

• This would then create a common denominator for comparisons between systems; irrespective of the specific aspect being measured in a given benchmark.

• Characterizations must include temporal variations in resources as well as limitations if there are any

Why is this needed?

• SUT description necessary for reproducibility as well as comparisons

• Having a common denominator approach on engineering components would help with comparisons of known systems with black boxes on the cloud.

Examples/suggestions

• Use of virus detection scans to determine memory capacity

• Use of Transcoders between media formats to measure cpu capacity.

• SPECcpu programs to measure cpu capacity.

Questions to the Research group

• Can we work on putting together a standard set of such tools?

• Is there any accepted set of similar tools that people are using currently?

• Other thoughts/discussions