1. Azure Awesomeness Dr. Zoran B. Djordjevic’s CSCI E-175 Cloud Computing and Software as a Service class, Harvard University, 04-Nov-2011 Copyright (c) 2011, Bill Wilder – Use allowed under Creative Commons license http://creativecommons.org/licenses/by-nc-sa/3.0/ http://creativecommons.org/licenses/by-nc-sa/3.0/ Boston Azure User Group http://www.bostonazure.org @bostonazure Bill Wilder http://blog.codingoutloud.com http://blog.codingoutloud.com @codingoutloud Let’s talk about a few ways the Windows Azure Cloud Platform is simply fabulous

2. Bill Wilder Microsoft MVP for Windows Azure Platform Authoring book on Cloud Architecture Patterns Founder of Boston Azure Cloud User Group Independent Windows Azure Consultant @codingoutloud codingoutloud@gmail.com http://blog.codingoutloud.com

3. Awesomness Highlights The Windows Azure Cloud Platform provides Geographically distributed, cost-effective, reliable, scalable, performant, and secure services spanning... Compute, storage, messaging, connectivity, and authentication services Broad device and technology integration support Productive and “natural” model for loosely coupled CQS-like Patterns such as CQRS + advanced patterns like Pub/Sub

4. Key Concepts & Patterns GENERAL 1.Scale vs. Performance 2.Scale Up vs. Scale Out 3.Shared Nothing 4.Design for Failure DATABASE ORIENTED 5.ACID vs. BASE 6.Eventually Consistent 7.Sharding 8.Optimistic Locking COMPUTE ORIENTED 9.CQRS Pattern 10.Poison Messages 11.Idempotency

5. Key Terms 1.Scale Up 2.Scale Out 3.Horizontal Scale 4.Vertical Scale 5.Scale Unit 6.ACID 7.CAP 8.Eventual Consistency 9.Strong Consistency 10.Multi-tenancy 11.NoSQL 12.Sharding 13.Denormalized 14.Poison Message 15.Idempotent 16.CQRS 17.Performance 18.Scale 19.Optimistic Locking 20.Shared Nothing 21.Load Balancing 22.Design for Failure

6. Old School Excel and Word

7. Scale != Performance Scalable iff Performance constant as it grows Scale the Number of Users … Volume of Data … Across Geography Scale can be bi-directional (more or less) Investment α Benefit What does it mean to Scale?

8. Options: Scale Up (and Scale Down) or Scale Out (and Scale In) Terminology: Scaling Up/Down == Vertical Scaling Scaling Out/In == Horizontal Scaling Architectural Decision – Big decision… hard to change

9. Scaling Up: Scaling the Box.

10. Scaling Out: Adding Boxes “Autonomous” units scale best  “shared nothing” db “stateless” compute

11. How do I Choose???? ?????? … Scale Up (Vertically) Scale Out (Horizontally). Not either/or! Part business, part technical decision (requirements and strategy) Consider Reliability (and SLA in Azure) Target VM size that meets min or optimal CPU, bandwidth, space

12. Essential Scale Out Patterns Data Scaling Patterns Sharding: Logical database comprised of multiple physical databases, if data too big for single physical db NoSQL: “Not Only SQL” – a family of approaches using simplified database model Computational Scaling Patterns CQRS: Command Query Responsibility Segregation

13. PRODUCTIVE

14. Windows Azure is Feature Rich iOS ToolkitAndroid ToolkitWindows Phone Toolkit

15. Windows Azure Platform Data Centers

16. NIST – Cloud Platform Taxonomy Essential Characteristics On-demand self-service Broad network access Resource Pooling Rapid Elasticity Measured service Infrastructure as a Service Platform as a Service Software as a Service Deployment Models Public Cloud Private Cloud Hybrid Cloud Community Cloud

17. Cloud Service Models Twitter-sized Description: IaaS = BYO VMs, PaaS = BYO Apps, SaaS = BYO Users

18. PaaS com IaaS Some of the Players SaaS AppHarbor

19. “Bring Your Own” ____ as a Service

20. ___________________ as a Service Apps, $/user, LDAP, Expertise, SLA System SW is OpEx, Auto Scale Out, Geo-LB, Failover, HA, OS Patching, Environment Monitoring, Geo-Repl (not back-up), Expertise, SLA Hardware OpEx, Networking, DB/OS Licenses, Virtualization, Automation, Geo Distribution, CDN, Geo Replication, Elasticity, Managed Facility, Expertise, SLA Public Cloud Rental Models

21. Application Ownership Simplified with PaaS Slide stolen from Chris Bowen’s talk: Windows Azure: What? Why? And a Peek Under the Hood 21 Application Development Network Addressing Network Load Balancing Hardware Repair OS updates & Patches OS Installation Computational Scalability Storage Scalability Hardware Provisioning Staging / Production High Availability Fault Tolerance Data Center Management Stuff We Might Rather Not Deal With Stuff We Like

22. SECURE

23. Proposition Big-vendor public cloud offerings will emerge as the most secure platforms available – more secure than vast majority of non- cloud datacenters

24. Reality is Resource-Constrained “Security is always a tradeoff; it must be balanced with the cost.” - Bruce Schneier http://www.schneier.com/essay-207.html

25. Members of Windows Azure Security Team

26. PaaS in Azure also adds… (Just examples…) Key Management for Compute Ability to specify base OS + patch level – Allow auto-management – Enabler: (more) Homogenous Platform – “one throad” – Alternative: Amazon lists 1000+ AMI images: http://aws.amazon.com/amis http://aws.amazon.com/amis – Choice vs. Responsibility: interesting tradeoffs here!

27. Azure Data Storage… Access Controls – Storage keys, with rollover – Shared Access Signatures (Blobs) – Container-level Access Policies (Blobs) Strong Consistency in Data Access – Eventual Consistency challenges: Privacy settings, deletion of sensitive data No automatic, at-rest encryption – Amazon offers this

28. Reach: How CloudIdentityConnectivity Identity and Access Management (IAM)Amazon Virtual Private Cloud AWS Direct Connect AppFabric Access Control Service (SAML, OAuth) App Fabric Service Bus Windows Azure Connect (CTP) Windows Azure Traffic Manager (CTP) Google Account Google Apps for domain Open ID Google Secure Data Connector Salesforce infrastructure Delegated authentication Federated authentication (SAML) Amazon hosted AppCloud: Amazon hostedxCloud: Private Virtual LAN OneLogin is highlighted option on Rackspace site RackConnect app engine

29. Remember Me?

30. Public Cloud Platform My Data Center Public Cloud Hybrid Cloud Private Cloud Public  Hybrid  Private

31. Secure Hybrid Cloud Connectivity – Windows Azure AppFabric Connect (VPN) – Windows Azure Message Bus (Secure Message Relay, Pub/Sub) Identity / SSO / Claims-based AuthZ – Access Control Service – Active Directory Federation Services

32. Data Defense in Depth Approach Physical Application* Host Network  Strong storage keys for access control  SSL support for data transfers between all parties  Front-end.NET framework code running under partial trust  Windows account with least privileges  Hardened version of Windows Server 2008 OS for both VM Host and VM Guest operating systems  Host boundaries enforced by external hypervisor  Host firewall limiting traffic to VMs  VLANs and packet filters in routers  World-class physical security  ISO 27001 and SAS 70 Type II certifications for datacenter processes Layer Defense-in-Depth

33. Defenses Inherited by Windows Azure Platform Applications Spoofing Tampering/ Disclosure Elevation of Privilege Configurable scale-out Denial of Service VM switch hardening Certificate Services Shared- Access Signatures HTTPS Sidechannel protections VLANs Top of Rack Switches Custom packet filtering Partial Trust Runtime Hypervisor custom sandboxing Virtual Service Accounts Repudiation Monitoring Diagnostics Service

34. data more secure in cloud compute more secure More homogenous than iaas clouds ACS Mature virtualization stack Mature SQL Mature Windows Server 2008 OS Partial Trust compute available Internal & External endpoints 3 copies x 2 geo http://download.microsoft.com/download/7/3/E/73 E4EE93-559F-4D0F-A6FC- 7FEC5F1542D1/SecurityBestPracticesWindowsAzure Apps.docx Developers also should not store private keys associated with SSL/TLS certificates in Windows Azure Storage. Instead, upload them through the Developer Portal and access them via thumbprint references in the Service Configuration. Windows Azure will not only store these certificates encrypted at all times, but also securely provision them into the certificate stores of the service’s web roles upon boot. Developers should not attempt to store certificates anywhere on their own as these actions would constitute re-inventing a protection already supplied by the platform.

35. PaaS and cloud make strong security accessible to mere mortals Less complex, more cost-effective, competitive pressure (“everyone’s doing it”)

36. Simplified Security Interesting matrix Appendix B: http://download.microsoft.com/download/7/ 3/E/73E4EE93-559F-4D0F-A6FC- 7FEC5F1542D1/SecurityBestPracticesWindows AzureApps.docx

37. NoSQL

38. Storage Taxonomy Type of DataTraditionalAzure Way RelationalSQL ServerSQL Azure BLOB (“Binary Large Object”) File System, SQL Server Azure Blobs FileFile System(Azure Drives) Azure Blobs LogsFile System, SQL Server, etc. Azure Blobs Azure Tables Non-RelationalAzure Tables NoSQL ?

39. NoSQL  SQL Server Excel  Microsoft Access

40. NoSQL Databases (simplified!!!), CouchDB: JSON Document Stores Amazon Dynamo, Azure Tables: Key Value Stores – Dynamo: Eventually Consistent – Azure Tables: Strongly Consistent Many others! Fewer constraints allows going Faster Cheaper cost-per-byte (>66x SQL Azure) Scale Out model “Simpler”

41. Consistency “A foolish consistency is the hobgoblin of little minds” - Ralph Waldo Emerson, Self-Reliance (essay)

42. Eventual Consistency Property of a system such that not all records of state guaranteed to agree at any given point in time. – Applicable to whole systems or parts of systems (such as a database) As opposed to Strongly Consistent (or Instantly Consistent) Eventual Consistency is natural characteristic of a useful, scalable distributed systems

43. Why Eventual Consistency? #1 ACID Guarantees: – Atomicity, Consistency, Isolation, Durability AtomicityConsistencyIsolationDurability – SQL insert vs read performance? How do we make them BOTH fast? Optimistic Locking and “Big Oh” math BASE Semantics: – Basically Available, Soft state, Eventual consistency Basically Available, Soft state, Eventual consistency From: http://en.wikipedia.org/wiki/ACID and http://en.wikipedia.org/wiki/Eventual_consistencyhttp://en.wikipedia.org/wiki/ACIDhttp://en.wikipedia.org/wiki/Eventual_consistency

44. Why Eventual Consistency? #2 CAP Theorem – Choose only two guarantees 1.Consistency: all nodes see the same data at the same timeConsistency 2.Availability: a guarantee that every request receives a response about whether it was successful or failedAvailability 3.Partition tolerance: the system continues to operate despite arbitrary message lossPartition tolerance From: http://en.wikipedia.org/wiki/CAP_theoremhttp://en.wikipedia.org/wiki/CAP_theorem

45. Cache is King Facebook has “28 terabytes of memcached data on 800 servers.” http://highscalability.com/blog/2010/9/30/facebook-and-site- failures-caused-by-complex-weakly-interact.html http://highscalability.com/blog/2010/9/30/facebook-and-site- failures-caused-by-complex-weakly-interact.html Eventual Consistency at work!

46. Relational (SQL Azure) vs. NoSQL (Azure Tables) ApproachRelational (e.g., SQL Azure) NoSQL (e.g., Azure Tables) NormalizationNormalizedDenormalized (Duplication)(No duplication)(Duplication okay) TransactionsDistributedLimited scope StructureSchemaFlexible ResponsibilityDBA/DatabaseDeveloper/Code KnobsManyFew ScaleUp (or Sharding) Out

47. NoSQL Storage Suitable for granular, semi-structured data (Key/Value stores) Document-oriented data (Document stores) No rigid database schema Weak support for complex joins or complex transaction Usually optimized to Scale Out NoSQL databases generally not managed with same tooling as for SQL databases

48. Azure Table Storage – NoSQL in Azure Not a good home for Relational Data Best place for granular, semi-structured data – No rigid database schema – No complex joins or complex transaction Fast and easy to instantiate – Strongly Consistent – No performance lag Programming model is WCF Data Services – All data access and data updates – LINQ Not SQL, not managed with SQL Server tooling

49. Azure Table Storage Storage Account Table [*] Entity [*, 1 MB data] PartitionKey + RowKey Property [255x] Name/Value/Type Up to 64k

50. Azure Table Storage Partition Key – A “logical grouping” (a shard with less baggage) – PartitionKey value of up to 64 KB Row Key – Identify specific row within a partition – RowKey value is String of up to 64 KB Table access requires Partition Key + Row Key – Not to mention (cryptographic) Access Key (for the digital signing of the http header)

51. SQL Azure Federations Semi-automated sharding – Tell SQL Azure how to shard, it does the hard work – Rebalance shards – Smart caching, connection management Coming soon to a datacenter near you! In CTP currently Due end of 2011 (?)

52. GRACEFUL #fail is okay

54. Failure IS an Option

56. Failure actually *is* an option… MTBF -or- MTTR

57. How much did we #fail? Small #fail: transient connectivity or service availability issue (including throttling) – Client-side retries – Retry concepts supported with client-side libraries Medium #Fail: hardware fail, software bugs – Failure cause could be your bug, or in DC (bug, hardware failure) – (Solution? See CQRS pattern) Bigtime #FAIL: datacenter comes down – Geo-replication helps a lot – Multi-datacenter architecture

58. Failure actually *is* an option… http://stackoverflow.com/questions/31466/d oes-amazon-s3-fail-sometimes http://stackoverflow.com/questions/31466/d oes-amazon-s3-fail-sometimes Perhaps “easier” than not failing? Does not take team of “rocket scientists” to avoid failure Some architecture patterns enable all at once: RESILIENCE, SCALE OUT, and a CLEAN SEPARATION of CONCERNS

59. CQRS

60. CQRS Architecture Pattern Command Query Responsibility Segregation Based on notion that methods which update (“Commands”) are separate architectural concerns from those actions which ask for data (“Query”) In distributed systems, leads to Front End (UI) and Backend (Business Logic) being Loosely Coupled

61. CQRS in Windows Azure WE NEED: Compute resource to run our code Web Roles (IIS) and Worker Roles (w/o IIS) Reliable Queue to communicate Azure Storage Queues Durable/Persistent Storage Azure Storage Blobs & Tables; SQL Azure

62. CQRS in Action Web Server Compute Service Reliable Queue Reliable Storage

63. Canonical Example: Thumbnails Web Role (IIS) Web Role (IIS) Worker Role Worker Role Azure Queue Azure Blob Key Point: at first, user does not get the thumbnail (UX implications)

64. Reliable Queue & 2-step Delete (IIS) Web Role (IIS) Web Role Worker Role Worker Role queue.AddMessage( new CloudQueueMessage( urlToMediaInBlob)); CloudQueueMessage msg = queue.GetMessage( TimeSpan.FromSeconds(10)); … queue.DeleteMessage(msg); Queue

65. General Case: Many Roles, Many Queues Web Role (IIS) Web Role (IIS) Worker Role Worker Role Web Role (IIS) Web Role (IIS) Web Role (IIS) Web Role (IIS) Web Role (IIS) Web Role (IIS) Worker Role Worker Role Worker Role Worker Role Worker Role Type 1 Worker Role Type 1 Worker Role Worker Role Worker Role Worker Role Worker Role Worker Role Worker Role Type 2 Worker Role Type 2 Queue Type 1 Queue Type 2 Queue Type 3 Queue Type 1 Queue Type 2 Queue Type 3 Remember: Investment α Benefit Watch your scale units! Logical vs. Physical Architecture

66. CQRS requires Idempotent If we perform idempotent operation more than once, end result same as if we did it once Example with Thumbnailing (easy case) App-specific concerns dictate approaches – Compensating transactions – Last in wins – Many others possible – hard to say

67. CQRS expects Poison Messages A Poison Message cannot be processed – Error condition for non-transient reason – Queue feature: know your dequeue count CloudQueueMessage.DequeueCount property in Azure Be proactive – Falling off the queue may kill your system Message TTL = 7 days by default in Azure Determine a max Retry policy – May differ by queue object type or other criteria – Delete, Move to Special Queue

68. CQRS enables Responsive Response to interactive users is as fast as a work request can be persisted Time consuming work done off-line Comparable total resource consumption, arguably better subjective UX UX challenge – how to express Async to users? – Communicate Progress – Display Final results

69. CQRS enables Scalable Loosely coupled, concern-independent scaling – Getting Scale Units right Blocking is Bane of Scalability – Decoupled front/back ends insulate from other system issues if… – Twitter down – Email server unreachable – Order processing partner doing maintenance – Internet connectivity interruption Queue Length is important signal

70. CQRS enables Distribution Scale out systems better suited for geographic distribution – More efficient and flexible because more granular – Hard for a mega-machine to be in more than one place – Failure need not be binary

71. CQRS requires Plan for Failure There will be VM (or Azure role) restarts – Hardware failure, O/S patching, crash (bug) Bake in handling of restarts – Idempotent Not an exception case! Expect it! Restarts are routine, system “just keeps working”

72. What about the DATA? Azure Web Roles and Azure Worker Roles – Taking user input, dispatching work, doing work – Follow CQRS pattern – Stateless compute nodes “Hard Part” – persistent data, scalable data – Azure Queue, Blob, Table, SQL Azure – 3x copies of each byte – Blobs and Tables geo-replicated – Retry and Throttle!

73. Division of Labor Client- facing code dealing with #fail Backoffice code dealing with #Fail Reliable Queuing Reliable Storage #fail, #Fail, #EpicFail

74. Typical SiteAny 1 Role InstOverall System Operating System Upgrade Application Update / Deploy Change Topology Hardware Failure (e.g., disk) Software Bug / Crash / Failure Azure Storage hardware failure What’s Up? Aspirin-free Reliability as EMERGENT PROPERTY

75. Questions? Comments? More information? ?

76. Other resources http://sigops.org/sosp/sosp11/current/2011-Cascais/printable/11- calder.pdf http://sigops.org/sosp/sosp11/current/2011-Cascais/printable/11- calder.pdf http://www.microsoft.com/windowsazure/whitepapers/ http://www.microsoft.com/windowsazure/free- trial/?WT.srch=1&WT.mc_id=CMG001_Search http://www.microsoft.com/windowsazure/free- trial/?WT.srch=1&WT.mc_id=CMG001_Search http://www.microsoft.com/windowsazure/pricing/ www.bostonazure.org  in-person meetings www.bostonazure.org blog.codingoutloud.com  this PPT deck will appear here soon

77. 4 Big Ideas to Take Home 1.Code for #fail ; architect for #Fail; architect (or not!) for #EpicFail! 2.Consider flexibility of Scale Out architecture – Scalable, Resilient, Testable, Cost-appropriate – Computation: Queues, Storage, CQRS – Data: SQL Azure Federations, NoSQL (Azure Tables) 3.Look for Eventual Consistency opportunities – Caching, CDN, CQRS, Non-transactional Data Updates, Optimistic Locking 4.Embrace platforms with affordances for future-looking architecture – e.g., Windows Azure Platform (PaaS)

78. BostonAzure.org Boston Azure Cloud User Group Focused on the Windows Azure Cloud Platform Microsoft’s PaaS cloud platform Meetings monthly, 6:00 ish -8:30 ish PM at NERD – Food; wifi; free; great topics; growing community Boston Azure Boot Camp: 2012 (in planning) Follow on Twitter: @bostonazure More info or to join our Meetup.com group: http://www.bostonazure.org

79. Contact Me Looking for … consulting help with Windows Azure Platform? someone to bounce Azure or cloud questions off? a speaker for your user group or company technology event? Just Ask! Bill Wilder @codingoutloud http://blog.codingoutloud.com

80. Contact Me Bill Wilder @codingoutloud codingoutloud@gmail.com http://blog.codingoutloud.com Swiss bank acct # 3141-5926-5358-9793-2382