1. Cloud SUT proposal OSGcloud group

2. Objective To fill in the Research the group about the thinking within the OSG working group To solicit new ideas/proposals

3. Definition of cloud (NIST) “Cloud computing is a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources that can be rapidly provisioned and released with minimal management effort or service provider interaction. The model promotes availability and includes five essential characteristics, three service models and four deployment models.”

4. Characteristics Cloud Characteristics from the NIST list – On-demand self service – Broad network access – Resource pooling – Elasticity – Measured service Characteristic components separated by layer (By osgcloud group) – Client Standard client Network connectivity Access Security identity – Server and Storage Processing power Memory Storage Network Power consumption Performance measurement – Internet Speed Reliability Availability

5. Metrics Elasticity – Provisioning Interval – Agility Durability Response time Throughput Reliability Power Price

6. Potential Workloads/Use cases (some workloads listed) Data Analytics  Expert Search  Clustering  Customer Segmentation Data Warehousing  Pipelines bring in data feed and clean and transform it (example: logs from web servers)  Iterative processing : one very large data set is maintained (often in the form of a graph). Business OLTP Mail Memory Cloud – Key-value pair databases Social Networking – Web2.0 based application – Write/read workload – Memory cloud – Search engine

7. Issues with defining the SUT Cloud offerings may be different. Inability on the user’s side to find out what exact hardware is being used and how much is being used. Migration of hardware from one platform to another. Temporal aspect.

8. Multiple ways of classifying Cloud SUTs Virtualized, non-virtualized, mixed Whether the cloud offering is virtualized, non-virtualized, or mixed. Non-virtualized clouds can also be elastic (think grid computing) IaaS, PaaS, SaaS Whether the cloud offering is IaaS, PaaS, SaaS, or a mixture of these (e.g., Azure). It is easier to compare providers that do not mix service offerings. white box vs. black box Whether benchmark and measurements will measure (1) and (2) as a white box (from hardware to the software stack) or as a black box (what customer purchases). private, public, or hybrid, and whether it is solely for enterprises.

9. Cloud Offering categories Cloud as a black box – No control over what is the exact hardware/software, whether it changes with time etc. – User gets a description/QOS guarantees of what he/she is paying for. – Unique identifier tied to the offering. Cloud as a white-box – Full control and knowledge of part numbers and software. – Description available in terms of exact hardware and software.

10. Who is interested in what + Issues  Hardware vendors selling systems to Cloud vendors – White-box approach  End customer trying to buy a Cloud service – Black box approach

11. Resources to quantify IAAS – Compute resource – Memory Resource (capacity/bandwidth) – Network capacity – Disk/Storage Capacity PAAS – Hardware may not be described by vendor – Possible to measure as in IAAS SAAS – Hardware not described – Is it important to know about the hardware resources?

12. Proposal to Describe a Cloud based SUT Users would describe SUT based on what they pay for. Yet, there is a need for other engineering details. Use micro-measurement tools to measure capacities of various elements. The micro-benchmarks should be easy and quick to run and provide numbers describing different types of resources. This would then create a common denominator for comparisons between systems; irrespective of the specific aspect being measured in a given benchmark. Characterizations must include temporal variations in resources as well as limitations if there are any

13. Why is this needed? SUT description necessary for reproducibility as well as comparisons Having a common denominator approach on engineering components would help with comparisons of known systems with black boxes on the cloud.

14. Examples/suggestions Use of virus detection scans to determine memory capacity Use of Transcoders between media formats to measure cpu capacity. SPECcpu programs to measure cpu capacity.

15. Questions to the Research group Can we work on putting together a standard set of such tools? Is there any accepted set of similar tools that people are using currently? Other thoughts/discussions