1. Dynamically Scaling Applications in the Cloud Presented by Paul

2. Problem The cloud offers o Near infinite computing capabilities o Near infinite storage o On-demand access to new VMs o Pay-as-you-go model But how do you build an application to automatically take advantage of those resources? [1]

3. Overview of scaling options [1]

4. Horizontal Scaling VM Replication o Duplicate servers and create load balancers to distribute incoming request o Most common approach to scaling o Will look in more detail... Network Scalability o As an application scales up, the bandwidth requirements for intercloud communication and app requests increases, [1] assesses the possibility of requesting network resources in conjunction with cloud resources Network as a Server (NaaS) [1]

5. Vertical Scaling VM resizing (live) o Taking a VM with given processing power, and upgrading it with more resources while it's still running VM replacement o Replacing less powerful VM with more powerful VM (shutting down original VM) Might require reboots, and application tolerance [1]

6. "Rule of thumb" CPU-Intensive o Best to load balance and split computation among many instances Network intensive o Better to use single CPU heavy instance More network intensive o DNS-based load balancing on powerful CPUs [1]

7. Decisions, decisions 1 Cloud to rule them all? o Do you want to use multiple cloud providers? o Does part of your application require running on private cloud? How much control over automated scaling would you like? o If you don't want any control, maybe PaaS is better option o Do you simply want horizontal scaling, and will handle load balancing on your own? o How much $$$ do you have?

8. Methods for Scaling Automatic VM Scaling Dynamic Workload-pattern Matching Whole Application Scaling Non-scalable Load Balancing (e.g. Elastic Load Balancing) DNS-based Load Balancing [1]

9. Basic: Automatic VM Scaling Services that scale based on predefined VM- related performance metrics Offered by Amazon and Rightscale o Amazon Autoscaling comes with Cloudwatch  Set conditions based on Cloudwatch variables, react to latency, CPU speed, etc. What you might imagine for scaling - like a state machine o If conditions a,b,c are met, do x,y,z [1]

10. The "Controller"

11. "Non-Scalable" Load Balancers Amazon offers load balancing service (Elastic Load Balancing) o The reason it's "non-scalable" is this is a single load balancer Could be used in conjunction with horizontal scaling o Scale up/down the number of instances, configure load balancer to balance across running instances

12. Elastic Beanstalk Automatically scale up your application depending on services it's using, and beanstalk controller parameters o Only pay for underlying AWS resources Can upload.NET, PHP/Python, and Java apps for integration with Elastic Beanstalk Closer in function to a PaaS o Still have access to AWS instances, can interact with elastic beanstalk controller

13. Whole Application Scaling [3]

14. Whole Application Scaling [3]

15. Dynamic Workload-pattern Matching [2] Scalability Controller

16. Dynamic Workload-pattern Matching [2]

17. Final Observations You must first decide how many clouds you want to use Scaling is still an art o Systems for automated scaling still in their infancy At the core, there is always a scaling controller o Either you configure and operate the controller, or the IaaS provider does (e.g. Elastic Beanstalk) You pay for automation o Either with your time or your money o More customized automation == more complexity

18. Sources [1] L. M. Vaquero, L. Rodero-Merino, and R. Buyya. "Dynamically scaling applications in the cloud," SIGCOMM Comput. Commun. Rev., 41:45-52. [2] P. Marshall, K. Keahey, and T. Freeman, "Elastic site:Using clouds to elastically extend site resources," Cluster Computing and the Grid, IEEE International Symposium on, vol. 0, pp. 43-52, 2010. [3]R. Buyya, R. Ranjan, and R. Calheiros, “Intercloud: Utility-oriented federation of cloud computing environments for scaling of application services.” in ICA3PP 2010: The 10th International Conference on Algorithms and Architectures for Parallel Processing, 2010, pp. 19–24.

19. Links for papers [1] http://www.cloudbus.org/papers/ScalabilityI nCloud2011.pdf [2] http://www.nimbusproject.org/files/elasticsit e_ccgrid_2010.pdf [3] http://arxiv.org/pdf/1003.3920.pdf