Presentation is loading. Please wait.

Presentation is loading. Please wait.

Mastering Cloud Computing Rajkumar Buyya, Christian Vecchiola, Thamarai Selvi.

Similar presentations


Presentation on theme: "Mastering Cloud Computing Rajkumar Buyya, Christian Vecchiola, Thamarai Selvi."— Presentation transcript:

1 Mastering Cloud Computing Rajkumar Buyya, Christian Vecchiola, Thamarai Selvi

2 Chapters Part I: Foundations Chapter 1 — Introduction Chapter 2 — Principles of Parallel and Distributed Computing Chapter 3 — Virtualization Chapter 4 — Cloud Computing Architecture Part II: Cloud Application Programming and the Aneka Platform Chapter 5 — Aneka: Cloud Application Platform Chapter 6 — Concurrent Computing: Thread Programming Chapter 7 — High-Throughput Computing: Task Programming Chapter 8 — Data Intensive Computing: Map-Reduce Programming Part III: Industrial Platforms and New Developments Chapter 9 — Cloud Platforms in Industry Chapter 10 — Cloud Applications Chapter 11 — Advanced Topics in Cloud Computing

3 Chapter 1 - Introduction

4 I need to grow my infrastructure, but I do not know for how long… I cannot invest in infrastructure, I just started my business…. I want to focus on application logic and not maintenance and scalability issues I want to access and edit my documents and photos from everywhere.. I have a surplus of infrastructure that I want to make use of I have a lot of infrastructure that I want to rent … I have infrastructure and middleware and I can host applications I have infrastructure and provide application services

5

6 Compute Storage Applications Development and Runtime Platform Private Resources Cloud Manager Private Cloud Private Cloud (Government) Public Clouds Government Agencies Organization Personnel All users, on any device

7 Cloud Deployment Models Private/Enterprise Clouds * A public Cloud model within a company’s own Data Center / infrastructure for internal and/or partners use. Public/Internet Clouds * 3rd party, multi-tenant Cloud infrastructure & services: * available on subscription basis to all. Hybrid/Inter Clouds * Mixed usage of private and public Clouds: Leasing public cloud services when private cloud capacity is insufficient

8 Virtualization IT outsourcing Pay as you go Provisioning on demand Provisioning on demand Elasticity Utility computing Scalability IaaS PaaS SaaS Virtual Datacenters Green computing Internet No capital investments Quality of Service Cloudbusrting Security Privacy & Trust IT outsourcing Service Level Agreement Billing Cloud Computing?

9 Runtime Environment for Applications Development and Data Processing Platforms Examples: Windows Azure, Hadoop, Google AppEngine, Aneka Runtime Environment for Applications Development and Data Processing Platforms Examples: Windows Azure, Hadoop, Google AppEngine, Aneka Platform as a Service Virtualized Servers Storage and Networking Examples: Amazon EC2, S3, Rightscale, vCloud Virtualized Servers Storage and Networking Examples: Amazon EC2, S3, Rightscale, vCloud Infrastructure as a Service End user applications Scientific applications Office automation, Photo editing, CRM, and Social Networking Examples: Google Documents, Facebook, Flickr, Salesforce End user applications Scientific applications Office automation, Photo editing, CRM, and Social Networking Examples: Google Documents, Facebook, Flickr, Salesforce Software as a Service Web 2.0 Interfaces

10 1950196019701980199020002010 Mainframes Clusters 1999: Grid Computing Grids Clouds 1966: Flynn’s Taxonomy SISD, SIMD, MISD, MIMD 1969: ARPANET 1970: DARPA’s TCP/IP 1984: DEC’s VMScluster 1984: IEEE 802.3 Ethernet & LAN 1975: Xerox PARC Invented Ethernet 1990: Lee-Calliau WWW, HTTP, HTML 2004: Web 2.0 2005: Amazon AWS (EC2, S3) 1960: Cray’s First Supercomputer 2010: Microsoft Azure 1997: IEEE 802.11 (Wi-Fi) 1989: TCP/IP IETF RFC 1122 2007: Manjrasoft Aneka 2008: Google AppEngine 1951: UNIVAC I, First Mainframe

11 Chapter 2 – Parallel and Distributed Computing

12

13 Computer and network hardware Hardware Support for heterogeneous resource sharing, communication, and programming environments for application development Middleware User interface for presentation Applications Distributed System Stack Execution platform including network connectivity services Operative System

14 Paradigms / Architectural Models Paradigms / Architectural Models Message Based Communication Message Based Communication Client - Server Object Space RPC Distributed Objects RPC Distributed Objects Peer –to-Peer Java Spaces Java RMI, Remoting, CORBA, ORBIX.DCOM, COM+ RPC / Specific Tasks MPI Programming, JMS, MSMQ, MQS Istant Messaging, Video Conferencing Istant Messaging, Video Conferencing Distributed Agents Mobile Agents

15 19401950 Applications Parallel Era Sequential Era Compilers Architectures 1960 19701980 199020002010 2020 2030 Applications Problem Solving Environments Compilers Architectures Problem Solving Environments

16 Processor Data Input Data Output Instruction Stream

17 Processor N Data Output N Single Instruction Stream Processor 2 Processor 1 Data Input 1 Data Input 2 Data Input N Data Output 2 Data Output 1

18 Processor N Instruction Stream 1 Instruction Stream 1 Processor 2 Processor 1 Single Data Input Stream Single Data Output Stream Instruction Stream 2 Instruction Stream 2 Instruction Stream N Instruction Stream N

19 Processor N Instruction Stream 1 Instruction Stream 1 Processor 2 Processor 1 Instruction Stream 2 Instruction Stream 2 Instruction Stream N Instruction Stream N Data Input 1 Data Input 2 Data Input N Data Output 1 Data Output 2 Data Output 3

20 Cache L1

21 + Large Level (Processes, Tasks) Large Level (Processes, Tasks) Task 1 Task 2 Task N Function 1 Function 2 Function J Statements x load function f1() {…} function f2() {…} function fJ() {…} a[0] = … b[0] = … a[1] = … b[1] = … a[k] = … b[k] = … Shared Memory Messages IPC Medium Level (Threads, Functions) Medium Level (Threads, Functions) Fine Level (Processor, Instructions) Fine Level (Processor, Instructions) Very Fine Level (Cores, Pipeline, Instructions) Very Fine Level (Cores, Pipeline, Instructions)

22 Hardware Operating System Middleware Applications Networking and Parallel Hardware IPC primitives for control and data. Frameworks for distributed programming

23 Hardware and OS (IaaS) Middleware (PaaS) Applications (SaaS) Social Networks, Scientific Computing, Enterprise Applications Virtual hardware, networking, OS images, and storage. Frameworks for Cloud Application Development Frameworks for Cloud Application Development

24 Client-server client Server/client server Server/client server client Server/client server client server request response Two Tier (Classic Model) Two Tier (Classic Model) Three Tier N Tier

25 P2P peer

26 RPC

27 DOF interaction

28 Application WS Client Invocation Query Response (WSDL) Web Server Web Server UDDI Registry WSDL(s) Web Server Web Server WSDL Web Service Application WSDL Web Service Application WS Client

29 WS technology Stack Web Service Flow Service Discovery Service Description XML-based Messaging Network Security Service Publication Quality of Service Management WSDL SOAP HTTP, FTP,e-mail, MQ, IIOP, …. Direct  UDDI Static  UDDI WSFL

30 SOAP Messages Envelope POST /InStock HTTP/1.1 Host: www.stocks.com Content-Type: application/soap+xml; charset=utf-8 Content-Length: IBM Header: Metadata & Assertions Body: Method Call

31 SOAP Messages Envelope POST /InStock HTTP/1.1 Host: www.stocks.com Content-Type: application/soap+xml; charset=utf-8 Content-Length: 34.5 Header: Metadata & Assertions Body: Execution Result

32 Chapter 3 – Virtualisation

33 Virtualization Layer Virtual Hardware Virtual Networking Virtual Storage Software Emulation Host Physical Hardware Physical Storage Physical Networking Guest Applications Virtual Image

34 Operative Systems Execution Stack Hardware - level Virtualization Hardware Programming Languages Applications OS- level Virtualization Programming Language level Virtualization Application - level Virtualization

35 Aggregation Sharing Emulation Isolation Virtualization Physical Resources Virtual Resources

36 Virtualization Execution Environment Storage Network …. Emulation High-Level VM Multiprogramming Hardware-assisted Virtualization Hardware-assisted Virtualization Process Level System Level Paravirtualization Full Virtualization How it is done? Technique Virtualization Model Application Programming Language Operating System Hardware Partial Virtualization

37 Libraries API ABI Hardware Operative System ISA Applications Operative System Hardware Libraries Applications API calls System calls ISA User ISA User ISA

38 Ring 3 Ring 2 Ring 1 Ring 0 Least privileged mode (user mode) Privileged modes Most privileged mode (supervisor mode)

39

40 ABI Hardware Operative System ISA Virtual Machine Manager ISA VM Hardware ISA Virtual Machine Manager ISA VM

41 Virtual Machine Manager ISA Virtual Machine Instance Instructions (ISA) Interpreter Routines Interpreter Routines Interpreter Routines Interpreter Routines Allocator Dispatcher

42 User Instructions Sensitive Instructions Privileged Instructions

43 Server A (running) Server A (running) VM Server B (running) Server B (running) Virtual Machine Manager VM Server A (running) Server A (running) VM Server B (inactive) Server B (inactive) Virtual Machine Manager VM Before Migration After Migration

44 Xen Hypervisor (VMM) Memory management CPU state registers Devices I/O Xen Hypervisor (VMM) Memory management CPU state registers Devices I/O User Domains (Domain U) Guest OS Modified codebase Hypercalls into Xen VMM User Domains (Domain U) Guest OS Modified codebase Hypercalls into Xen VMM User Applications (unmodified ABI) User Applications (unmodified ABI) Management Domain (Domain 0) VM Management HTTP interface Access to the Xen Hypervisor Management Domain (Domain 0) VM Management HTTP interface Access to the Xen Hypervisor Ring 3 Ring 2 Ring 1 Ring 0 Hardware (x86) Privileged instructions Hardware trap

45 Hypervisor Binary translation Instruction caching Hypervisor Binary translation Instruction caching Guest Operating System Unmodified codebase VMM unaware Guest Operating System Unmodified codebase VMM unaware User Applications (unmodified ABI) User Applications (unmodified ABI) Ring 3 Ring 2 Ring 1 Ring 0 Hardware (x86) Hardware trap (sensitive instructions) Dynamic / cached translation (sensitive instructions)

46 Hardware (x86) Host Operating System VMware Hypervisor (VMM) Direct access to hardware I/O, memory, networking for guests Save/Restore CPU state for host OS VMware Hypervisor (VMM) Direct access to hardware I/O, memory, networking for guests Save/Restore CPU state for host OS VMware Driver Virtual Machine Instance User Applications VMware Workstation Guest Operating System User Applications I/O

47 Hardware (x86) Host Operating System VMware Hypervisor (VMM) Direct access to hardware I/O, memory, networking for guests Save/Restore CPU state for host OS VMware Hypervisor (VMM) Direct access to hardware I/O, memory, networking for guests Save/Restore CPU state for host OS VMware Driver VM Instance serverd (daemon) serverd (daemon) VMware Web Server VM Instance

48 Hardware VMkernel hostd VMX CIM broker VM User world API Resource scheduling Resource scheduling Device drivers Storage stack Network stack Distributed VM file system Virtual Ethernet adapter and switch VM VMM VMX DCUI syslog vxpa SNMP Third-party CIM plug-ins

49 Server ESXi ESX vSphere Server ESXi ESX vSphere Data Center vCenter Server ESXi ESX vSphere Server ESXi ESX vSphere Data Center vCenter vCloud Cloud Infrastructure Virtualization Infrastructure Virtualization vFabric Platform Virtualization Platform Virtualization Zimbra Application Virtualization Application Virtualization

50 Latest Pictures. Hardware (x86) Hypervisor (Ring -1) Hypervisor (Ring -1) Hypercalls MSRs APIC Scheduler Address Management Partition Management Partition Management Root / Parent Partition VMWPs VMMS WMI Hypervisor-aware Kernel (Ring 0) VSPs VID WinHv I/O Stack Drivers VMBus Enlightened Child Partition User Applications (Ring 3) User Applications (Ring 3) Hypervisor-aware Wndows Kernel (Ring 0) Hypervisor-aware Wndows Kernel (Ring 0) VSCs / ICs WinHv I/O Stack Drivers VMBus Enlightened Child Partition User Applications (Ring 3) User Applications (Ring 3) Hypervisor-aware Linux Kernel (Ring 0) Hypervisor-aware Linux Kernel (Ring 0) VSCs / ICs LinuxHv I/O Stack Drivers VMBus Unenlightened Child Partition User Applications (Ring 3) User Applications (Ring 3) Hypervisor-unaware Kernel (Ring 0) Hypervisor-unaware Kernel (Ring 0) ProcessorMemory

51 Chapter 4 – Cloud Computing Architecture

52 Pure SaaS

53 Physical Infrastructure Datacenter Cluster Desktop / Heterogeneous Resources Desktop / Heterogeneous Resources Third Party IaaS Cloud Infrastructure Management Software Scheduling Pricing / Billing Monitoring QoS SLA Management Reservation VM Pool Management VM Image Repository Provisioning Web Services, Portals, REST API Web based Management Interface

54 Programming API / Libraries Physical Infrastructure IaaS Providers PaaS Core Middleware Resources Management Application Management Application Management User Management User Management QoS / SLA Management & Billing QoS / SLA Management & Billing Elasticity & Scaling Runtime Web Services, Portals, REST API Web based Interface

55

56 Datacenter Public Cloud Cluster Desktop Grids / NOWs Desktop Grids / NOWs Public Cloud Dynamic Provisioning IaaS (M) Paas Private Cloud Hybrid / Heterogeneous Cloud

57 Private Users Private Users Public Services Industries Enterprises Industries Enterprises 3 rd Party Cloud Public Cloud Private Cloud Public Cloud Community Cloud Federal and Government Bodies Federal and Government Bodies Development Platforms Development Platforms Infrastructure Application Services Application Services

58 Chapter 5 – Aneka: Cloud Application Platform

59 Multiple Infrastructures Multi-coreClusterGridCloud ThreadTask...MapReduce 2100 Aneka Multiple Applications 1. SDK 2. Runtime

60 ECMA 334:.NET or Mono / Windows, Linux, & Mac Infrastruscture Enterprise Desktop Grid Data Centers Clusters Public Cloud Persistence and Security Fabric Services Hardware Profiling Dynamic Resource Provisioning Foundation Services Billing & Reporting Resource Reservation Storage Licensing & Accounting Execution Services Independent Bag of Tasks Distributed Threads MapReduce other models… Middleware - Container Application Development & Management SDK: APIs & Tools Management: Tools, Interfaces and APIs PAL – Platform Abstraction Layer

61 Aneka Repository Aneka Containers Aneka Containers Node Manager Node Manager Node Manager Node Manager Aneka Containers Aneka Containers Aneka Containers Aneka Containers Node Manager Node Manager Node Manager Node Manager Aneka Containers Aneka Containers Management Console update HTTP File Share ……

62

63

64

65 ECMA 334:.NET or Mono / Windows, Linux, & Mac Infrastruscture Enterprise Desktop Grid Data Centers Clusters Public Cloud Persistence & Security Fabric Services Hardware Profiling Resource Provisioning High-Availability Membership Foundation Services Billing & Reporting Resource Reservation Storage Licensing & Accounting Application Services Bag of Tasks Distributed Threads MapReduce Other models… PSM Middleware - Container Application Development & Management Software Development Kit: APIs & Tools Management: Tools, Interfaces and APIs PAL – Platform Abstraction Layer

66 Reservation Provisioning Reporting Accounting Mandatory Index (master) Scheduling Failover Mandatory Index (slave) Storage Storage Node Allocation Mandatory Index (slave) Execution Worker Node Reservation Provisioning Reporting Accounting Mandatory Index (master) Scheduling Master Node Allocation Mandatory Index (slave) Execution Worker Node Allocation Mandatory Index (slave) Execution Worker Node Mandatory Index (slave) Storage Storage Node

67 Dynamic Resources Amazon EC2 Storage Service Provisioning Service Task Scheduling Service Reporting, Billing, Accouting Master Node Slave Node Task Execution Service Static Resources AWS

68

69 Resource Provisioning Application Management & Scheduling Resource Reservation Master Node Desktop Machines Virtual Clusters Workstations Clusters

70 Resource Provisioning Application Management & Scheduling Master Node Reporting, Billing, Accouting Amazon EC2 Instance Amazon EC2 Instances Slave Nodes EC2 Boundaries

71 Provisioning Service Application Management & Scheduling Reporting, Billing, Accouting Public Clouds Desktops & Workstations Clusters Virtual Cluster Resources Resource Reservation Master Node Local Infrastructure

72 Unknown Initialized Running Paused Stopped Pausing Resuming Starting Stopping IService.Start() IService.Stop() IService.Continue() IService.Pause()

73 Chapter 6 – Concurrent Computing: Thread Programming

74

75 Execution Timeline Thread Local Storage Instructions (program counter) Instructions (program counter) Thread Main Thread Process Shared Memory

76 Process Units of work Result Process Units of work Result a. Embarrassingly parallel b. Inherently sequential

77 A 4 x 6 B 6 x 3 C 4 x 3 2 1 2,1

78 Process Units of work Result Unit 1 Unit 2 Unit 5 Unit 4 Unit 3

79 Unstarted Running Stopped Wait SleepJoin Start() Interrupt() Abort() Suspended Suspend Requested Suspend() Resume () Abort Requested Wait()/Sleep()/Join() Normal Completion Unstarted Started Running Completed StagingIn Queued Staging Out Aborted Rejected Failed Start() a. System.Threading.Thread life cycle. b. Aneka.Threading.AnekaThread life cycle.

80 Chapter 7 – High-Throughput Computing: Task Programming

81 Middleware Access Node(s) Computing Nodes Local Node Tasks Remote Environment

82 1 6 4 5 2 7 8 9 3 0 11 10 1 2 4 3 0 1 6 4 5 2 3 0 7 9 3 0 11 4 2 1 5 6 8 10 MPI_COMM_WORLD Group A Group B Communicator A Communicator B

83 MPI Include File MPI Environment Initialization MPI Environment Shutdown Prototypes declaration Do Work and Message Passing Do Work Serial Code Parallel Code Code Section Header Section Header Section

84 Workflow Application Modeling & Definition Tools Workflow Specification Workflow Specification Task + Data Scheduler Data Movement Data Provenance Dependency Manager Runtime Manager Runtime Manager Language Interpreter Fault Management Workflow Enactment Engine Middleware Supporting Clusters, Grids, and Clouds Distributed Computing Resources Interaction with Distributed Resources Interaction with Information Services Users Workflow Design & Definition Workflow Execution & Control Resource Info Service Application Info Service Information Services Run Time Build Time

85 Local Node (… more tasks) AnekaApplication Task1 : ITask Task2 : ITask AnekaTask Index Node Membership Catalogue Membership Catalogue StorageService TaskScheduler TaskExecution Service TaskExecution Service Aneka Cloud TaskManager

86 Client Machine File dependencies Index Node Membership Catalogue Membership Catalogue StorageService TaskScheduler TaskExecution Service TaskExecution Service Aneka Cloud Web Service TaskManager AnekaTask Remote FTP Result Files Input/Output Files Input/Output Files

87 IJobManager TaskManager AnekaApplication PSM Model PSMJobInfo PSMCommandInfo PSMParameterInfo PSMFileInfo PSMContext AnekaTask ITask Aneka Application Model Aneka Application Model

88 Offspring Environment IDistributionEngine AppInstance IOffspringPlugin DefaultPlugin LogPlugin AutoPlugin Strategy Controller Strategy Controller ITaskStrategy Workflow Logic 1 Workflow Logic N TaskStrategyBase Local Engine Local Engine AnekaEngine DistributionEngineBase TaskManager Aneka Cloud

89 StrategyController Control Thread (Strategy Management) Control Thread (Strategy Management) Monitoring Thread (Feedback from Distribution Engine) Monitoring Thread (Feedback from Distribution Engine) Init Application Init Strategy Submit Tasks Wait Release Strategy Shutdown Application Yes No Complete or Stop? Task Feedback Task Failed? Invoke OnSuccess Invoke OnFailed Yes No New Task? Signal Iteration Complete? Yes Submit No Yes No

90 Chapter 8 – Data Intensive Computing: Map-Reduce Programming

91 Data/compute intensive problems Data/compute intensive problems Compute intensive problems Compute intensive problems Current Problems Data intensive problems Data intensive problems Heterogeneous Formats Distributed Petabytes Heterogeneous Formats Distributed Petabytes Data Complexity (volume/format/distribution) Data Complexity (volume/format/distribution) Homogeneous Formats Non-distributed Megabytes or Gigabytes Homogeneous Formats Non-distributed Megabytes or Gigabytes Computational Complexity Computational Complexity Model Solvers Simulations Simple search Model Solvers Simulations Simple search Statistical Models Decision support Knowledge generation Statistical Models Decision support Knowledge generation

92 Compute Site Instrument Storage Facility Compute Site Scientist Instrument Scientist Storage Facility Information & Discovery Compute Site

93 Storage Peer Key Ring. Pluggable Storage Engine (BDB, MySQL) Failure and Membership Detection Failure and Membership Detection Storage Peer Request Coordinator Request Coordinator Storage Peer

94 Tablet Server Master Server Chubby GFS Logs Tablet Files Logs Tablet Files Logs Tablet Files Logs Tablet Files

95 MapReduce Infrastructure / Runtime Map Task Map Task Reduce Task Reduce Task Map Task Map Task Map Task Map Task Map Task Map Task Map Task Map Task Reduce Task Reduce Task Aggregation by Key Input Output Input Data File Output Data File Distributed File System

96 Worker Master Split 0 Split 1 Split 2 Split 3 Split 4 Output File 0 Output File 1 User Program (1) fork (2) assign map (2) assign reduce (3) read (4) local write (5) remote read (6) write input filesmap phasereduce phase output files intermediate files

97 R R R R R R R R X X X X X X X X X X X X M M M M M M M M M M M M Stage Input File Output File Vertices (processes) Channels

98 Aneka Cloud MapReduce Scheduling Service MapReduce Scheduling Service MapReduceApplication Reducer Mapper Input Data File Client Machine MapReduce Execution Service MapReduce Execution Service Map Task Map Task Reduce Task Reduce Task MapReduce Execution Service MapReduce Execution Service Map Task Map Task Reduce Task Reduce Task MapReduce Execution Service MapReduce Execution Service Map Task Map Task Reduce Task Reduce Task DFS Implementation Split 0 Split 1 Split 2 Split 3 Split 4 intermediate files Output File 0 Output File 1

99 MapReduceSchedulerService MapReduceScheduler upload queue initialization queue dispatch queue failure queue resubmission queue reporting queue upload thread initialization thread dispatch thread report thread DFS Implementation Message translation and Event Triggering Aneka Middleware MapReduceApplication MapReduce Execution Services MapReduce Execution Services

100 MapReduceExecutorService ExecutorManager job and task information command thread report thread completion thread DFS Implementation Message translation and Event Triggering Aneka Middleware MapReduce Scheduling Service MapReduce Scheduling Service MapReduceExecutor Map Task MapReduceExecutor Reduce Task Sandbox command queue

101 Header: 3 bytes magic number, 1 byte version ……. Record Block: Integer (4 bytes) Length of the block Integer (4 bytes) Length of the block Integer (4 bytes) Length of the key Integer (4 bytes) Length of the key Key byte array Value byte array

102

103

104 User Requests Wireless / Mobile 3G Network ECG Sensor ECG Sensor Module Embedded Bluetooth Enabled Data Communication & Processor Module Embedded Bluetooth Enabled Data Communication & Processor Module Bluetooth connectivity IaaS PaaS Dynamically Scalable Runtime (QoS-based scaling of computing resources) Dynamically Scalable Runtime (QoS-based scaling of computing resources) Security Runtime SaaS ECG Data Analysis as a Service User Requests User (detailed view) Large Number 1 1 2 2 3 3 6 6 4 4 5 5

105 Jeeva Portal A A B B C C D D E E F F G G H H I I A : BLAST B : Create Data Vector C : HH Classifier D : SS Classifier E : TT Classifier F : HS Classifier G : ST Classifier H : TH Classifier I : Predict Final Secondary Structure Initial Phase Classification Phase Final Phase Task Graph Aneka

106 Chapter 9 – Cloud Platforms in Industry

107 Communication Services Amazon Simple Queue Service (SQS) Amazon Simple Notification Service (EBS) Amazon Simple Email Service (SES) Amazon Route 53 Amazon Virtual Private Cloud (VPC) Amazon Direct Connect Amazon Elastic Load Balancing Storage Services Amazon Simple Storage Service (S3) Amazon Elastic Block Store (EBS) Amazon ElastiCache Amazon SimpleDB Amazon Relational Database Service (RDS) Amazon CloudFront Amazon Import/Export Compute Services Amazon Elastic Compute Cloud (EC2) Amazon Elastic MapReduce AWS Elastic Beanstalk AWS Cloudformation Autoscaling Amazon AWS Platform Additional Services Amazon GovCloud Amazon Flexible Payment Service (FPS) Amazon DevPay Amazon Fullfillment Web Service (FWS) Amazon Mechanical Turk Alexa Web Information Service Amazon CloudWatch Alexa Top Sites

108 Development Runtime Local Machine Python SDK Google AppEngine Infrastructure Sandboxed Runtime Environment Data Store Url Fetch Image Manipulation Task Queue Cron Jobs Web App

109 Compute Enterprise Level Infrastructure Storage Networking Identity Marketplace Management portal Windows Azure Platform Local Development Environment Performance Development Tools Windows Azure Compute Windows Azure Storage Windows Azure Connect Content Delivery Network (CDN) AppFabric Caching AppFabric Caching AppFabric Service Bus AppFabric Service Bus AppFabric Integration AppFabric Integration AppFabric Access Control AppFabric Access Control Tools Windows Azure SDK Windows Azure SDK AppFabric SDK AppFabric SDK SQL Azure DataMarket Applications MarketplaceSQL AzureWindows Azure AppFabricWindows Azure

110 Infrastructure Layer Platform Layer Tabular Data Stream (TDS) TDS + Secure Socket Layer SQL Server SQL Azure Fabric Management Services SQL Server SQL Azure Fabric Management Services SQL Server SQL Azure Fabric Management Services Services Layer Provisioning Billing and Metering Connection Routing Provisioning Billing and Metering Connection Routing Provisioning Billing and Metering Connection Routing ODBC ADO.NET PHP SQL Server Applications and Tools WCF Data Services HTTP / REST Client Layer Client Premises or Windows Azure Platform

111 Chapter 10 – Cloud Applications

112 Aneka Cloud-CoXCS (Task orchestrator) Cloud-CoXCS (Task orchestrator) Elastic Sizing (EC2,…, ) CoXCS Tasks Datasets

113 Local Storage Aneka Private Cloud Public Cloud Archiving Distribution Portal (SaaS)

114 Force.com Platform Metadata Cache Bulk Processing Engine Multitenant-aware Query Optimiser Runtime Application Generator Full-text Search Engine Shared Database Data Tables Pivot Tables Metadata Tables (UDD) Virtual Application Components COMMON APPLICATION SCREENS TENANT- SPECIFIC SCREENS TENANT- SPECIFIC SCREENS OBJECTS (TABLES) Indexes User-base 3: Application 3 User-base 1: Application 1 User-base 2: Application 2

115 User EyeOS Server eyeos apps extern extras system users workgroups Client-side scripts

116 Scalability engine EC2 instances: video rendering EC2 instances: web front end EC2 instances: web front end Amazon SQS Amazon S3: Video storage Amazon S3: Video storage

117 Aneka Networked LAN Aneka MayaRender Maya: Train Design Render Files Scenarios Render Tasks

118 Gamer Titan Gaming Portal Game Server Aneka Log Processing Task Log Processing Task Log Processing Task Log Processing Task Log Processing Task Log Processing Task

119 Dropbox Folder Automatic synchronization

120 Chapter 11 – Advanced Topics in Cloud Computing

121 Reducing Power Cost and Carbon Emission Reducing Power Cost and Maximizing Revenue Revenue Power Consumption Cloud Data Center User Applications Green Cloud Solution

122 Users Brokers Compute Cloud Pool Node Cluster (VM Pool) Compute Cloud Storage Cloud Cloud Exchange Directory Bank Auctioneer VMM Enterprise IT Consumer Enterprise Resource Server (Proxy) Request Capacity Negotiate Bid = Cloud Coordinator

123 Local Global Allocation Decision Profit-based System-based Hybrid Objective Market Model Application Model Participant Focus Bag of Tasks Workflow Parallel Applications Others eg. Storage User Centric Resource Centric System Centric Game Theory Proportional Share Bargaining Commodity Market Posted Price Auction Contract Net Single Sided Double Auction Combinatorial Auction Vickery Auction Dutch Auction English Auction First Price Sealed Auction Market-Oriented Models

124 Service Request Examiner and Admission Control - Customer-driven Service Management - Computational Risk Management - Autonomic Resource Management Service Request Examiner and Admission Control - Customer-driven Service Management - Computational Risk Management - Autonomic Resource Management Pricing Accounting Dispatcher VM Monitor Service Request Monitor Users Brokers Physical Machines Virtual Machines Virtual Machines SLA Allocator Cloud Data Center

125

126 Cloud Federation Stack Motivations Advantages Opportunities Obligations Motivations Advantages Opportunities Obligations Federation Model Cloud Service, Provider, Agreements Market and Pricing Models Service Level Agreements Federation Model Cloud Service, Provider, Agreements Market and Pricing Models Service Level Agreements Protocol, Interfaces, and Standards Programmatic Interoperation Federation Platforms (RESERVOIR, InterCloud) Protocol, Interfaces, and Standards Programmatic Interoperation Federation Platforms (RESERVOIR, InterCloud) Infrastructural Level Logical and Operational Level Conceptual Level

127 OCCI Interface Provider’s APIs Resources Resource Management Framework Resource Management Framework OCCI Clients Other Clients Provider’s Infrastructure any channel, any protocol Most interoperable Least interoperable Communication is internal and proprietary Communication is related to IMF interface requests

128 Container Information Services (Future) Data Services Storage Services Cloud Data Management Storage Cloud Data Management Interface (CDMI) Data Storage Cloud Draw Resources on Demand Multiple, Proprietary Interfaces iSCSI, FC, FCoE, LUNs, Targets POSIX (NFS, CIFS, WebDAV) CDMI Export Data to the Cloud Block Storage Client File System Client Object Storage Client XAM Client Database / Table Client Clients Acting in the Role of Using Data Storage Interface Clients can be in the Cloud or Enterprise and provide additional services (data, computing, etc…) Clients can be in the Cloud or Enterprise and provide additional services (data, computing, etc…) Data Storage Management Client Clients Acting in the Role of Managing Data / Storage Management of the Cloud Storage can be standalone or part of the overall management of your Cloud Computing

129 VEE Host Computational Resource Virtualizer VEE Host Computational Resource Virtualizer VEE Host Computational Resource Virtualizer RESERVOIR Site 1 RESERVOIR Site 2 Service Application 1 Service Application 3 Service Application 2

130 VEE Manager (VEEM) VEE Manager (VEEM) Service Manager VEE Hosts (e.g. hypervisor, VJSC Host, …) VEE Hosts (e.g. hypervisor, VJSC Host, …) VEE Manager (VEEM) VEE Manager (VEEM) VEE Manager (VEEM) VEE Manager (VEEM) VEE Hosts Service Manager Site A Site B Site C SLA Service Provider manifest

131 Cloud Exchange Cloud Coordinator Cloud Coordinator Cloud Coordinator Cloud Coordinator Cloud Coordinator Cloud Coordinator VM Manager (e.g. Open Nebula) VM Manager (e.g. Open Nebula) VM Manager (e.g. EC2) VM Manager (e.g. EC2) VM Manager (e.g. Eucalyptus) VM Manager (e.g. Eucalyptus) User Interface User Interface User Interface User Interface User Interface User Interface User Broker User Resource Negotiation Publish / Query Offers / Request User

132 Nirvanix Coral CDN Windows Azure Amazon S3 XYZ Web Service Access Web Service Access Web Portal Nirvanix Connector Nirvanix Connector Coral Connector Coral Connector Azure Connector Azure Connector S3 Connector S3 Connector XYZ Connector XYZ Connector MetaCDN Manager MetaCDN Manager MetaCDN Database MetaCDN Database MetaCDN Allocator MetaCDN Allocator MetaCDN Load Redirector MetaCDN Load Redirector SLA Users Applications SLA

133 XYZ Nirvanix Coral CDN Windows Azure Amazon S3 Web Service Access Web Service Access Web Portal Nirvanix Connector Nirvanix Connector Coral Connector Coral Connector Azure Connector Azure Connector S3 Connector S3 Connector XYZ Connector XYZ Connector MetaCDN Manager MetaCDN Manager MetaCDN Database MetaCDN Database MetaCDN Allocator MetaCDN Allocator MetaCDN Load Redirector MetaCDN Load Redirector Users Applications SLA

134 Comparative Listing of Offerings Comparative Listing of Offerings Buyers Capacity Provider (Compute) SpotCloud Connector SpotCloud Connector Capacity Provider (Storage) SpotCloud Connector SpotCloud Connector VM Package Repository VM Package Repository Transactions Bookkeeping and Billing Engine QoS and SLA Monitoring User Profiles Database User Profiles Database SLA Deployment

135 Other Stuff

136

137

138


Download ppt "Mastering Cloud Computing Rajkumar Buyya, Christian Vecchiola, Thamarai Selvi."

Similar presentations


Ads by Google