Presentation is loading. Please wait.

Presentation is loading. Please wait.

Metrics and Techniques for Quantifying Performance Isolation in Cloud Environments Rouven Krebs (SAP AG), Christof Momm (SAP AG), Samuel Kounev (KIT) SPEC.

Similar presentations


Presentation on theme: "Metrics and Techniques for Quantifying Performance Isolation in Cloud Environments Rouven Krebs (SAP AG), Christof Momm (SAP AG), Samuel Kounev (KIT) SPEC."— Presentation transcript:

1 Metrics and Techniques for Quantifying Performance Isolation in Cloud Environments Rouven Krebs (SAP AG), Christof Momm (SAP AG), Samuel Kounev (KIT) SPEC RG Cloud, May 2012

2 ©2012 SAP AG. All rights reserved.2 Isolation and Shared Resources provides Service Provider High overhead, low utilization need to share Hardware Operating System Middleware Application Hardware Operating System Middleware Application Hardware Operating System Middleware Application

3 ©2012 SAP AG. All rights reserved.3 Isolation and Shared Resources provides Service Provider Performance guarantees Different performance isolation methods. Hardware Virtualization Operating System Middleware Application

4 ©2012 SAP AG. All rights reserved.4 Questions How to quantify isolation? Performance isolation methods Q1: How strong is one tenants influence onto the others? Q2:How much is a system better isolated than a non- isolated system? Q3:How much potential has the method to improve? Introduction Metrics Isolation Methods Conclusion/Related Work

5 ©2012 SAP AG. All rights reserved.5 Definition of Performance Isolation Tenants working within their assigned quota (e.g., #Users) should not suffer from tenants exceeding their quotas. Load t1 > Quota Time Load t2 < Quota Response Time t1 Response Time t2 Isolated Non-Isolated Load t1 > Quota Time Load t2 < Quota Response Time t1 Response Time t2 Introduction Metrics Isolation Methods Conclusion/Related Work

6 ©2012 SAP AG. All rights reserved.6 Contributions Contribution III Approaches for performance isolation at the architectural level in SaaS environments. Contribution III Approaches for performance isolation at the architectural level in SaaS environments. Contribution I Metrics to quantify the performance isolation of shared systems. Contribution I Metrics to quantify the performance isolation of shared systems. Contribution II Measurement techniques for quantifying the proposed metrics. Contribution II Measurement techniques for quantifying the proposed metrics. Introduction Metrics Isolation Methods Conclusion/Related Work

7 ©2012 SAP AG. All rights reserved.7 Performance Isolation Metrics: Basic Idea D is a set of disruptive tenants exceeding their quotas. A is a set of abiding tenants not exceeding their quotas. Workload Time Response Time Time Impact of increased workload of the disruptive tenants onto the response time of the abiding ones. Introduction Metrics Isolation Methods Conclusion/Related Work

8 ©2012 SAP AG. All rights reserved.8 Metric I: Based on QoS Impact t1t3t2t4 Load t1t3t2t4 Load Avg. Response Time for all Tenants in A W ref W disr seconds A Reference Workload W ref Disruptive Workload W disr Different Response Times Tenants Introduction Metrics Isolation Methods Conclusion/Related Work Workload

9 ©2012 SAP AG. All rights reserved.9 Metric I: Based on QoS Impact Difference in Workload Difference in Response Time Perfectly Isolated = 0 Non-Isolated = ? Answers Q1: How strong is a tenants influence onto the others? Perfectly Isolated = 0 Non-Isolated = ? Answers Q1: How strong is a tenants influence onto the others? Introduction Metrics Isolation Methods Conclusion/Related Work

10 ©2012 SAP AG. All rights reserved.10 Metrics Based on Workload Ratio - Idea Non-Isolated Isolated By decreasing workload of the abiding tenant, it is possible to maintain the QoS for them. Introduction Metrics Isolation Methods Conclusion/Related Work

11 ©2012 SAP AG. All rights reserved.11 Metrics Based on Workload Ratio - Idea Workload Time Response Time Time Workload Time Response Time Time Introduction Metrics Isolation Methods Conclusion/Related Work

12 ©2012 SAP AG. All rights reserved.12 Metrics Based on Workload Ratio Disruptive Workload Non-Isolated Abiding Workload Stable QoS for the abiding tenants residual users. Pareto optimum with regards to total workload. Introduction Metrics Isolation Methods Conclusion/Related Work

13 ©2012 SAP AG. All rights reserved.13 Metrics Based on Workload Ratio Disruptive Workload Isolated Abiding Workload We maintain the QoS for the abiding tenant without decreasing his workload. Introduction Metrics Isolation Methods Conclusion/Related Work

14 ©2012 SAP AG. All rights reserved.14 Metrics Based on Workload Ratio Abiding Workload Disruptive Workload Isolated Non- Isolated Observed System W d base W d end W a base W d ref W a ref W a ref = W d base - W d ref Introduction Metrics Isolation Methods Conclusion/Related Work

15 ©2012 SAP AG. All rights reserved.15 Metric II: Based on Workload Ratio I end Perfectly Isolated = ? Non-Isolated = 0 Answers Q2: Is the system better isolated than a non- isolated system. Perfectly Isolated = ? Non-Isolated = 0 Answers Q2: Is the system better isolated than a non- isolated system. Introduction Metrics Isolation Methods Conclusion/Related Work

16 ©2012 SAP AG. All rights reserved.16 Metrics Based on Workload Ratio Integrals Abiding Workload Disruptive Workload Isolated Non- Isolated Observed System W d base W d end W a base W d ref W a ref A measured Introduction Metrics Isolation Methods Conclusion/Related Work

17 ©2012 SAP AG. All rights reserved.17 Metrics Based on Workload Ratio Integrals Abiding Workload Disruptive Workload Isolated Non- Isolated Observed System W d base W d end W a base W d ref W a ref A nonIsolated Introduction Metrics Isolation Methods Conclusion/Related Work

18 ©2012 SAP AG. All rights reserved.18 Metrics Based on Workload Ratio Integrals Abiding Workload Disruptive Workload Isolated Non- Isolated Observed System W d base W d end W a base W d ref W a ref A Isolated p end Introduction Metrics Isolation Methods Conclusion/Related Work

19 ©2012 SAP AG. All rights reserved.19 Metrics Based on Workload Ratio Integrals: Basic Idea Abiding Workload A nonIsolated = W a ref * W a ref / 2 I = (A measured – A nonIsolated )/A isolated - A nonIsolated Disruptive Workload Isolated Non-Isolated Observed System W d base W d end W a base W d ref W a ref A nonIsolated A measured A Isolated Introduction Metrics Isolation Methods Conclusion/Related Work

20 ©2012 SAP AG. All rights reserved.20 Metrics Based on Workload Ratio Integrals: I intBase and I intFree Perfectly Isolated = 1 Non-Isolated = 0 Answers Q3: How much potential has the isolation method to improve. Perfectly Isolated = 1 Non-Isolated = 0 Answers Q3: How much potential has the isolation method to improve. Introduction Metrics Isolation Methods Conclusion/Related Work Areas within W d ref and predefined bound. Areas within W d ref and W d base.

21 ©2012 SAP AG. All rights reserved.21 Approaches for Performance Isolation in MT Applications Add Delay Round Robin Blacklist Separate Thread Pools Introduction Metrics Isolation Methods Conclusion/Related Work

22 ©2012 SAP AG. All rights reserved.22 Results: Workload QoS Based Metrics Introduction Metrics Isolation Methods Conclusion/Related Work

23 ©2012 SAP AG. All rights reserved.23 Results: Workload Ratio Based Metrics Introduction Metrics Isolation Methods Conclusion/Related Work

24 ©2012 SAP AG. All rights reserved.24 Discussion/Conclusion QuestionsMetricsSemanticsLimitations Q1: influence I QoS Reduced QoS based on workload. No ranking. Only value for isolated system is known. Q2: relation to non- Isolated I end How many times better than non-isolated system. Not available when system is good isolated. Q3: potential to improve Integral based Ranking within isolated/non-isolated. Quantification needs two values. Introduction Metrics Isolation Methods Conclusion/Related Work Q1: How strong is one tenants influence onto the others? Q2:How much is a system better isolated than a non isolated system? Q3:How many potential has the method to improve?

25 ©2012 SAP AG. All rights reserved.25 Discussion/Conclusion of Approaches Isolation capabilitiesLimitations Round RobinFairNo QoS differentiation, inefficient scenarios possible. BlacklistFairToo hard to disruptive tenant. Burstiness for disruptive and abiding ones. DelayIneffective for high loadBurstiness for abiding tenants. ThreadpoolFairCan lead to inefficiency in overcommitted scenarios. Introduction Metrics Isolation Methods Conclusion/Related Work

26 ©2012 SAP AG. All rights reserved.26 Related Work Concerning Metrics VMmark [3]: Scores a normalized overall throughput Focus on hypervisors No impact of varied load Georges et al. [2]: Reflect throughput when additional VMs are deployed. Do not set the changed workload in relation. Huber et al. [4]/Koh et al. [5]: Closely characterize the performance inference of workloads in different VMs. No metric derived by these results. Introduction Metrics Isolation Methods Conclusion/Related Work

27 ©2012 SAP AG. All rights reserved.27 Related Work Concerning Performance Isolation Fehling et al. [1]/ Zhang [8]: Tenant placement onto locations with different QoS. Tenant placement onto a restricted set of nodes with awareness of SLAs. Do not guarantee isolation. Lin et al. [7]: Request Admission Control Provide different QoS on a tenants base One test case evaluated the system regarding tenant specific workload changes and their interference. No setup with high utilization for reference workload. Introduction Metrics Isolation Methods Conclusion/Related Work

28 ©2012 SAP AG. All rights reserved.28 to non isolated Recap Performance Isolation is a challenge in shared systems. Metrics with expressiveness concerning QoS Metrics with ranking capabilities Introduction Metrics Isolation Methods Conclusion/Related Work How to quantify performance isolation methods. potential to improve Observed QoS by increasing workload. Variable workloads and constant QoS.

29 ©2012 SAP AG. All rights reserved.29 Ongoing / Future Work MT Performance Isolation Benchmark Mapping these approaches to real existing benchmarks/reference application. MT Performance Isolation Mechanisms Identification + Evaluation of different performance isolation mechanisms Introduction Metrics Isolation Methods Conclusion/Related Work

30 ©2012 SAP AG. All rights reserved.30 References [1] Fehling, C., Leymann, F., and Mietzner, R. A framework for optimized distribution of tenants in cloud applications. In Cloud Computing (CLOUD), 2010 IEEE 3rd International Conference on (2010), pp. 252 –259. [2] Georges, A., and Eeckhout, L. Performance metrics for consolidated servers. In HPCVirt 2010 (2010). [3] Herndon, B., Smith, P., Roderick, L., Zamost, E., Anderson, J., Makhija, V., Herndon, B., Smith, P., Zamost, E., and Anderson, J. Vmmark: A scalable benchmark for virtualized systems. Tech. rep., VMware, 2006. [4] Huber, N., von Quast, M., Hauck, M., and Kounev, S. Evaluation and modeling virtualization performance overhead for cloud environments. In Proceedings of the 1st International Conference on Cloud Computing and Services Science (CLOSER 2011), Noordwijkerhout, The Netherlands (May 7-9 2011), pp. 563 – 573. [5] Koh, Y., Knauerhase, R., Brett, P., Bowman, M., Wen, Z., and Pu, C. An analysis of performance interference effects in virtual environments. In Performance Analysis of Systems Software, 2007. ISPASS 2007. IEEE International Symposium on (april 2007), pp. 200 –209. [6] Koziolek, H. The SPOSAD architectural style for multi-tenant software applications. In Proc. 9 th Working IEEE/IFIP Conf. on Software Architecture (WICSA'11), Workshop on Architecting Cloud Computing Applications and Systems (July 2011), IEEE, pp. 320–327. [7] Lin, H., Sun, K., Zhao, S., and Han, Y. Feedback-control-based performance regulation for multi-tenant applications. In Proceedings of the 2009 15th International Conference on Parallel and Distributed Systems (Washington, DC, USA, 2009), ICPADS 09, IEEE Computer Society, pp. 134–141. [8] Zhang, Y., Wang, Z., Gao, B., Guo, C., Sun, W., and Li, X. An effective heuristic for on-line tenant placement problem in SaaS. Web Services, IEEE International Conference on 0 (2010), 425–432.

31 Thank you Contact information: Rouven Krebs:Rouven.Krebs@sap.com Christof Momm:Christof.Momm@sap.com Samuel Kounev:Kounev@kit.edu http://www.sap.com/research http://www.descartes-research.net

32 ©2012 SAP AG. All rights reserved.32 Scenario - Simulation Our simulated server Poolsize configured for 38 Threads to ensure optimal throughput. At 80 users the system achieves 3500ms response time. Normalovercommited referencedisruptivereferencedisruptive T0824, 40, 2512440, 56, 251 T18888 T28888 T38888 T48844 T58811 T68811 T78811 T88811 T98824

33 ©2012 SAP AG. All rights reserved.33 Metrics based on Workload Ratio Relation of Significant Points: I base Perfectly Isolation = 1 Non-Isolated = 0 Describes the decrease of abiding workload at the point at which a non-isolated systems abiding load is 0. Perfectly Isolation = 1 Non-Isolated = 0 Describes the decrease of abiding workload at the point at which a non-isolated systems abiding load is 0.

34 ©2012 SAP AG. All rights reserved.34 Performance in Cloud matters [Bitcurrent2011]

35 ©2012 SAP AG. All rights reserved.35 Results: QoS Impact Based Metrics Negative results as the QoS increased when the disruptive tenant increase load. This happes if disruptive tenant gets completely blocked for a while.

36 ©2012 SAP AG. All rights reserved.36 Architectures for Performance Isolation Application Tier Application Threads Client TierDatabase Tier Web Browser Rich Client Cache (optional) Load Balancer Application Threads Meta-Data Manager Data (Shared Table) Meta- Data REST / SOAP Data transfer customizes Relates to 123 4 5 6 1 2 3 4 5 6 Admission Control Cache Restrictions Load Management Thread Priorities Thread Pool Sizes Database Admission Architectural Style based on [6]

37 ©2012 SAP AG. All rights reserved.37 Approach 1: Add Delay for Users Exceeding Quotas Request Manager Quota checker checks if the quota for a tenant is exceeded Quotas and current usage information are maintained in tenant data If user is exceeds quota, request delayer adds custom delay After delay requests are forwarded to Server New Request App. Server Request Processor R Quota checker Tenants Request delayer R

38 ©2012 SAP AG. All rights reserved.38 Approach 2: Request-Queueing per Tenant + Round-Robin Request Manager Requests are queued in separate queues for each tenant Round-robin support used for getting next request if Request Processor has free resources. t 1 Queue request adder R R R t n Queue R R New Request Next request provider Round Robin Strategy R App. Server Request Processor

39 ©2012 SAP AG. All rights reserved.39 Approach 3: Request-Queueing with Blacklist Queue App. Server Request Manager Triggered by each incoming request, the quota checker checks if the quota is exceeded and blacklists users Quotas and blacklist information are maintained in tenant data Requests by blacklisted users are put in separate queue Requests from blacklist queue are only returned by next request provider if normal queue is empty Normal Queue request adder R R R Blacklist Queue R R New Request R FIFO Queues Quota checker Tenants R Next request provider Normal queue always first Request Processor

40 ©2012 SAP AG. All rights reserved.40 Approach 4: Separate Thread Pools App. Server Request Processor Request Manager Simple FIFO queue for all tenants Work controller only assigns request to leader if no busy worker is already working for this user. If tenant is already served, work controller adds request to queue as last element request adder New Request Next request provider Pool t 1 W W W Pool t n W W R Worker Controller W t 1 Queue R R R t n Queue R R

41 ©2012 SAP AG. All rights reserved.41 © 2012 SAP AG. All rights reserved. No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained herein may be changed without prior notice. Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors. Microsoft, Windows, Excel, Outlook, PowerPoint, Silverlight, and Visual Studio are registered trademarks of Microsoft Corporation. IBM, DB2, DB2 Universal Database, System i, System i5, System p, System p5, System x, System z, System z10, z10, z/VM, z/OS, OS/390, zEnterprise, PowerVM, Power Architecture, Power Systems, POWER7, POWER6+, POWER6, POWER, PowerHA, pureScale, PowerPC, BladeCenter, System Storage, Storwize, XIV, GPFS, HACMP, RETAIN, DB2 Connect, RACF, Redbooks, OS/2, AIX, Intelligent Miner, WebSphere, Tivoli, Informix, and Smarter Planet are trademarks or registered trademarks of IBM Corporation. Linux is the registered trademark of Linus Torvalds in the United States and other countries. Adobe, the Adobe logo, Acrobat, PostScript, and Reader are trademarks or registered trademarks of Adobe Systems Incorporated in the United States and other countries. Oracle and Java are registered trademarks of Oracle and its affiliates. UNIX, X/Open, OSF/1, and Motif are registered trademarks of the Open Group. Citrix, ICA, Program Neighborhood, MetaFrame, WinFrame, VideoFrame, and MultiWin are trademarks or registered trademarks of Citrix Systems Inc. HTML, XML, XHTML, and W3C are trademarks or registered trademarks of W3C ®, World Wide Web Consortium, Massachusetts Institute of Technology. Apple, App Store, iBooks, iPad, iPhone, iPhoto, iPod, iTunes, Multi-Touch, Objective-C, Retina, Safari, Siri, and Xcode are trademarks or registered trademarks of Apple Inc. IOS is a registered trademark of Cisco Systems Inc. RIM, BlackBerry, BBM, BlackBerry Curve, BlackBerry Bold, BlackBerry Pearl, BlackBerry Torch, BlackBerry Storm, BlackBerry Storm2, BlackBerry PlayBook, and BlackBerry App World are trademarks or registered trademarks of Research in Motion Limited. Google App Engine, Google Apps, Google Checkout, Google Data API, Google Maps, Google Mobile Ads, Google Mobile Updater, Google Mobile, Google Store, Google Sync, Google Updater, Google Voice, Google Mail, Gmail, YouTube, Dalvik and Android are trademarks or registered trademarks of Google Inc. INTERMEC is a registered trademark of Intermec Technologies Corporation. Wi-Fi is a registered trademark of Wi-Fi Alliance. Bluetooth is a registered trademark of Bluetooth SIG Inc. Motorola is a registered trademark of Motorola Trademark Holdings LLC. Computop is a registered trademark of Computop Wirtschaftsinformatik GmbH. SAP, R/3, SAP NetWeaver, Duet, PartnerEdge, ByDesign, SAP BusinessObjects Explorer, StreamWork, SAP HANA, and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and other countries. Business Objects and the Business Objects logo, BusinessObjects, Crystal Reports, Crystal Decisions, Web Intelligence, Xcelsius, and other Business Objects products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of Business Objects Software Ltd. Business Objects is an SAP company. Sybase and Adaptive Server, iAnywhere, Sybase 365, SQL Anywhere, and other Sybase products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of Sybase Inc. Sybase is an SAP company. Crossgate, m@gic EDDY, B2B 360°, and B2B 360° Services are registered trademarks of Crossgate AG in Germany and other countries. Crossgate is an SAP company. All other product and service names mentioned are the trademarks of their respective companies. Data contained in this document serves informational purposes only. National product specifications may vary. The information in this document is proprietary to SAP. No part of this document may be reproduced, copied, or transmitted in any form or for any purpose without the express prior written permission of SAP AG.

42 ©2012 SAP AG. All rights reserved.42 © 2012 SAP AG. Alle Rechte vorbehalten. Weitergabe und Vervielfältigung dieser Publikation oder von Teilen daraus sind, zu welchem Zweck und in welcher Form auch immer, ohne die ausdrückliche schriftliche Genehmigung durch SAP AG nicht gestattet. In dieser Publikation enthaltene Informationen können ohne vorherige Ankündigung geändert werden. Die von SAP AG oder deren Vertriebsfirmen angebotenen Softwareprodukte können Softwarekomponenten auch anderer Softwarehersteller enthalten. Microsoft, Windows, Excel, Outlook, und PowerPoint sind eingetragene Marken der Microsoft Corporation. IBM, DB2, DB2 Universal Database, System i, System i5, System p, System p5, System x, System z, System z10, z10, z/VM, z/OS, OS/390, zEnterprise, PowerVM, Power Architecture, Power Systems, POWER7, POWER6+, POWER6, POWER, PowerHA, pureScale, PowerPC, BladeCenter, System Storage, Storwize, XIV, GPFS, HACMP, RETAIN, DB2 Connect, RACF, Redbooks, OS/2, AIX, Intelligent Miner, WebSphere, Tivoli, Informix und Smarter Planet sind Marken oder eingetragene Marken der IBM Corporation. Linux ist eine eingetragene Marke von Linus Torvalds in den USA und anderen Ländern. Adobe, das Adobe-Logo, Acrobat, PostScript und Reader sind Marken oder eingetragene Marken von Adobe Systems Incorporated in den USA und/oder anderen Ländern. Oracle und Java sind eingetragene Marken von Oracle und/oder ihrer Tochtergesellschaften. UNIX, X/Open, OSF/1 und Motif sind eingetragene Marken der Open Group. Citrix, ICA, Program Neighborhood, MetaFrame, WinFrame, VideoFrame und MultiWin sind Marken oder eingetragene Marken von Citrix Systems, Inc. HTML, XML, XHTML und W3C sind Marken oder eingetragene Marken des W3C ®, World Wide Web Consortium, Massachusetts Institute of Technology. Apple, App Store, iBooks, iPad, iPhone, iPhoto, iPod, iTunes, Multi-Touch, Objective-C, Retina, Safari, Siri und Xcode sind Marken oder eingetragene Marken der Apple Inc. IOS ist eine eingetragene Marke von Cisco Systems Inc. RIM, BlackBerry, BBM, BlackBerry Curve, BlackBerry Bold, BlackBerry Pearl, BlackBerry Torch, BlackBerry Storm, BlackBerry Storm2, BlackBerry PlayBook und BlackBerry App World sind Marken oder eingetragene Marken von Research in Motion Limited. Google App Engine, Google Apps, Google Checkout, Google Data API, Google Maps, Google Mobile Ads, Google Mobile Updater, Google Mobile, Google Store, Google Sync, Google Updater, Google Voice, Google Mail, Gmail, YouTube, Dalvik und Android sind Marken oder eingetragene Marken von Google Inc. INTERMEC ist eine eingetragene Marke der Intermec Technologies Corporation. Wi-Fi ist eine eingetragene Marke der Wi-Fi Alliance. Bluetooth ist eine eingetragene Marke von Bluetooth SIG Inc. Motorola ist eine eingetragene Marke von Motorola Trademark Holdings, LLC. Computop ist eine eingetragene Marke der Computop Wirtschaftsinformatik GmbH. SAP, R/3, SAP NetWeaver, Duet, PartnerEdge, ByDesign, SAP BusinessObjects Explorer, StreamWork, SAP HANA und weitere im Text erwähnte SAP-Produkte und ­ Dienstleistungen sowie die entsprechenden Logos sind Marken oder eingetragene Marken der SAP AG in Deutschland und anderen Ländern. Business Objects und das Business-Objects-Logo, BusinessObjects, Crystal Reports, Crystal Decisions, Web Intelligence, Xcelsius und andere im Text erwähnte Business- Objects-Produkte und ­Dienstleistungen sowie die entsprechenden Logos sind Marken oder eingetragene Marken der Business Objects Software Ltd. Business Objects ist ein Unternehmen der SAP AG. Sybase und Adaptive Server, iAnywhere, Sybase 365, SQL Anywhere und weitere im Text erwähnte Sybase-Produkte und -Dienstleistungen sowie die entsprechenden Logos sind Marken oder eingetragene Marken der Sybase Inc. Sybase ist ein Unternehmen der SAP AG. Crossgate, m@gic EDDY, B2B 360°, B2B 360°Services sind eingetragene Marken der Crossgate AG in Deutschland und anderen Ländern. Crossgate ist ein Unternehmen der SAP AG. Alle anderen Namen von Produkten und Dienstleistungen sind Marken der jeweiligen Firmen. Die Angaben im Text sind unverbindlich und dienen lediglich zu Informations- zwecken. Produkte können länderspezifische Unterschiede aufweisen. Die in dieser Publikation enthaltene Information ist Eigentum der SAP. Weitergabe und Vervielfältigung dieser Publikation oder von Teilen daraus sind, zu welchem Zweck und in welcher Form auch immer, nur mit ausdrücklicher schriftlicher Genehmigung durch SAP AG gestattet.

43 ©2012 SAP AG. All rights reserved.43 Headline area Drawing area White space The Grid


Download ppt "Metrics and Techniques for Quantifying Performance Isolation in Cloud Environments Rouven Krebs (SAP AG), Christof Momm (SAP AG), Samuel Kounev (KIT) SPEC."

Similar presentations


Ads by Google