1. Exchange Deployment Planning Services Exchange 2010 High Availability

2. The Exchange 2007 High Availability has the following goals:  Review of Exchange Server 2007 Availability solutions  Overview of Exchange Server 2010 High Availability  Exchange Server 2010 High Availability fundamentals  Exchange Server 2010 High Availability architecture scenarios  Exchange Server 2010 site resilience

3. Ideal audience for this workshop  Messaging SME  Network SME  Security SME Exchange 2010 High Availability

4. During this session focus on the following :  How will we leverage this functionality in our organization?  What availability and service level requirements do we have around our messaging solution?

5. Agenda  Review of Exchange Server 2007 Availability solutions  Overview of Exchange Server 2010 High Availability  Exchange Server 2010 High Availability fundamentals  Exchange Server 2010 High Availability architecture scenarios  Exchange Server 2010 site resilience

6. Exchange Server 2007 Single Copy Clustering Single Copy Cluster (SCC) out-of-box provides little high availability value −On Store failure, SCC restarts store on the same machine; no CMS failover −SCC does not automatically recover from storage failures −SCC does not protect your data, your most valuable asset −SCC does not protect against site failures −SCC redundant network is not leveraged by CMS Conclusion −SCC only provides protection from server hardware failures and bluescreens, the relatively easy components to recover −Supports rolling upgrades without losing redundancy

7. 1. Copy logs E00.log E0000000012.log E0000000011.log 2. Inspect logs 3. Replay logs Log Log shipping to a local disk Local File Share Log shipping within a cluster Cluster Log shipping to a standby server or cluster Standby DatabaseDatabase DatabaseDatabase

8. DB1 Client Access Server CCR #1 Node A CCR #1 Node B CCR #2 Node B CCR #2 Node A SCR Outlook (MAPI) client Windows cluster OWA, ActiveSync, or Outlook Anywhere AD site: San Jose AD site: Dallas Client Access Server Standby Server SCR managed separately; no GUI Manual “activation” of remote mailbox server Clustering knowledge required DB2 DB3 DB1 DB2 DB3 DB4 DB5 DB6 DB4 DB5 DB6 Database failure requires server failover DB4 DB5 DB6 Mailbox server can’t co-exist with other roles

9. Core Architectural Shift Windows Failover Cluster Default Cluster Group Cluster IP Address Cluster IP Address Cluster Name Cluster Name Cluster Quorum Cluster Quorum Default Cluster Group Cluster IP Address Cluster IP Address Cluster Name Cluster Name Cluster Quorum Cluster Quorum Cluster Database Clustered Mailbox Server (CMS) CMS IP Address CMS IP Address CMS Name CMS Name CMS resources (exres.dll) CMS resources (exres.dll) CMS disk resources CMS disk resources Clustered Mailbox Server (CMS) CMS IP Address CMS IP Address CMS Name CMS Name CMS resources (exres.dll) CMS resources (exres.dll) CMS disk resources CMS disk resources Cluster Networks

10. Database Availability Group Core Architectural Shift Windows Failover Cluster Default Cluster Group Cluster IP Address Cluster IP Address Cluster Name Cluster Name Cluster Quorum Cluster Quorum Default Cluster Group Cluster IP Address Cluster IP Address Cluster Name Cluster Name Cluster Quorum Cluster Quorum Cluster Database Active Manager PAM PAM SAM SAM Active Manager PAM PAM SAM SAM DAG Networks

11. Database Availability Group Core Architectural Shift Mailbox Server Get- MailboxDatabaseCopyStatus Primary Active Manager Move- ActiveMailboxDatabase StorageStorage Mailbox Server Get- MailboxDatabaseCopyStatus Standby Active Manager Move- ActiveMailboxDatabase StorageStorage Mailbox Server Get- MailboxDatabaseCopyStatus Standby Active Manager Move- ActiveMailboxDatabase StorageStorage

12. Agenda Review of Exchange Server 2007 Availability solutions Overview of Exchange Server 2010 High Availability Exchange Server 2010 High Availability fundamentals Exchange Server 2010 High Availability architecture Scenarios Exchange Server 2010 site resilience

13. Exchange Server 2010 HA Goals Reduce complexity Reduce cost Native solution - no single point of failure Improve recovery times Support larger mailboxes Make High Availability Exchange deployments mainstream!

14. Improved failover granularityImproved failover granularity Simplified administrationSimplified administration Incremental deploymentIncremental deployment Unification of CCR + SCRUnification of CCR + SCR Easy stretching across sitesEasy stretching across sites Up to 16 replicated copiesUp to 16 replicated copies Improved failover granularityImproved failover granularity Simplified administrationSimplified administration Incremental deploymentIncremental deployment Unification of CCR + SCRUnification of CCR + SCR Easy stretching across sitesEasy stretching across sites Up to 16 replicated copiesUp to 16 replicated copies Easier and cheaper to deploy Easier and cheaper to manage Better Service Level Agreements (SLAs) Reduced storage costs Larger mailboxes Further IO reductionsFurther IO reductions RAID-less/JBOD supportRAID-less/JBOD support Further IO reductionsFurther IO reductions RAID-less/JBOD supportRAID-less/JBOD support Exchange Server Improvements Key Benefits Online mailbox movesOnline mailbox moves Improved transport resiliencyImproved transport resiliency Online mailbox movesOnline mailbox moves Improved transport resiliencyImproved transport resiliency Easier and cheaper to manage Better SLAs Improved mailbox uptime More storage flexibility Better end-to-end availability

15. DB2 DB3 DB2 DB3 DB4 DB5 Client Access Server Mailbox Server 1 Mailbox Server 2 Mailbox Server 3 Mailbox Server 6 Mailbox Server 4 AD site: Dallas AD site: San Jose Mailbox Server 5 DB5 DB2 DB3 DB4 DB5 DB1 Failover managed within Exchange Database centric failover Easy to stretch across sites Client Access Server All clients connect via CAS servers DB3 DB5 DB1

16. Mailbox Server −Evolves Continuous Replication technology −Combines parts of Cluster Continuous Replication (CCR) and Standby Continuous Replications (SCR) into unified platform −Simplifies deployment and management of solution −Allows up to 16 replicated copies of each mailbox database −Provides full redundancy of Exchange roles on as few as two servers DB1 DB3 DB2 DB4 DB5 Mailbox Server DB1 DB2 DB4 DB5 DB3 Mailbox Server DB1 DB2 DB4 DB5 DB3 San Jose Dallas Recover quickly from disk and database failures Replicate databases to remote datacenter

17. Agenda Review of Exchange Server 2007 High Availability solutions Overview of Exchange Server 2010 High Availability Exchange Server 2010 High Availability fundamentals Exchange Server 2010 High Availability architecture scenarios Exchange Server 2010 site resilience

18. Exchange Server 2010 Active Directory Schema Organization Exchange Administrative Group Server 1 ServersServers Database Availability Groups DatabasesDatabases DAG 1 Database 1 Database Copy 1

19. Exchange Server 2010 Active Directory Schema Organization

20. Exchange Server 2010 HA Fundamentals Database Availability Group (DAG) Server Database Database Copy Active Manager (AM) RPC Client Access Service DAG

21. Exchange Server 2010 HA Fundamentals Database Availability Group (DAG) A group of up to 16 servers hosting a set of replicated databases Wraps a Windows Failover Cluster −Manages servers’ membership in the group −Heartbeats servers, quorum, cluster database Defines the boundary of database replication Defines the boundary of failover/switchover (*over) Defines boundary for DAG’s Active Manager Mailbox Server 1 Mailbox Server 2 Mailbox Server 3 Mailbox Server 4 Mailbox Server 16

22. Exchange Server 2010 HA Fundamentals Server Unit of membership for a DAG Hosts the active and passive copies of multiple mailbox databases Executes Information Store, CI, Assistants, etc., services on active mailbox database copies Executes replication services on passive mailbox database copies DB2 DB3 DB4 Mailbox Server 1 Mailbox Server 2 Mailbox Server 3 DB1 DB3 DB4 DB2

23. Exchange Server 2010 HA Fundamentals Server (Continued) Provides connection point between Information Store and RPC Client Access Very few server-level properties relevant to HA −Server’s Database Availability Group −Server’s Activation Policy DB2 DB3 DB4 Mailbox Server 1 Mailbox Server 2 Mailbox Server 3 DB1 DB3 DB4 DB2 RCA*

24. Exchange Server 2010 HA Fundamentals Mailbox Database Unit of *over A database has 1 active copy – active copy can be mounted or dismounted Maximum # of passive copies == # servers in DAG – 1 active DB2 DB3 DB4 Mailbox Server 1 Mailbox Server 2 Mailbox Server 3 DB1 DB3 DB4 DB2 DB1

25. Exchange Server 2010 HA Fundamentals Mailbox Database (Continued) −~30 seconds database *overs −Server failover/switchover involves moving all active databases to one or more other servers −Database names are unique across a forest −Defines properties relevant at the database level −GUID: a Database’s unique ID −EdbFilePath: path at which copies are located −Servers: list of servers hosting copies

26. Exchange Server 2010 HA Fundamentals Active/Passive vs. Source/Target Availability Terms −Active: Selected to provide email services to clients −Passive: Available to provide email services to clients if active fails Replication Terms −Source: Provides data for copying to a separate location −Target: Receives data from the source

27. Defines properties applicable to an individual database copy −Copy status: Healthy, Initializing, Failed, Mounted, Dismounted, Disconnected, Suspended, FailedandSuspended, Resynchronizing, Seeding −CopyQueueLength −ReplayQueueLength ActiveCopy ActivationSuspended

28. Exchange-aware resource manager (high availability’s brain) −Runs on every server in the DAG −Manages which copies should be active and which should be passive −Definitive source of information on where a database is active or mounted −Provides this information to other Exchange components (e.g., RPC Client Access and Hub Transport) −Information stored in cluster database Exchange Server 2010 HA Fundamentals Active Manager

29. Active Directory is still primary source for configuration info Active Manager is primary source for changeable state information (such as active and mounted) Replication service monitors health of all mounted databases, and monitors ESE for IO errors or failure Exchange Server 2010 HA Fundamentals Active Manager

30. Primary Active Manager (PAM) −Runs on the node that owns the default cluster group (quorum resource) −Gets topology change notifications −Reacts to server failures −Selects the best database copy on *overs Standby Active Manager (SAM) −Runs on every other node in the DAG −Responds to queries from other Exchange components for which server hosts the active copy of the mailbox database

31. Exchange Server 2010 HA Fundamentals Continuous Replication Continuous replication has the following basic steps: −Database copy seeding of target −Log copying from source to target −Log inspection at target −Log replay into database copy

32. Exchange Server 2010 HA Fundamentals Database Seeding There are three ways to seed the target instance: −Automatic Seeding −Requires 1 st log file containing CreateDB record −Update-MailboxDatabaseCopy cmdlet −Can be performed from active or passive copies −Manually copy the database

33. Exchange Server 2010 HA Fundamentals Log Shipping Log shipping in Exchange Server 2010 leverages TCP sockets −Supports encryption and compression −Administrator can set TCP port to be used Replication service on target notifies the active instance the next log file it expects −Based on last log file which it inspected Replication service on source responds by sending the required log file(s) Copied log files are placed in the target’s Inspector directory

34. Exchange Server 2010 HA Fundamentals Log Inspection The following actions are performed to verify the log file before replay: −Physical integrity inspection −Header inspection −Move any Exx.log files to ExxOutofDate folder that exist on target if it was previously a source If inspection fails, the file will be recopied and inspected (up to 3 times) If the log file passes inspection it is moved into the database copy’s log directory

35. Exchange Server 2010 HA Fundamentals Log Replay Log replay has moved to Information Store The following validation tests are performed prior to log replay: −Recalculate the required log generations by inspecting the database header −Determine the highest generation that is present in the log directory to ensure that a log file exists −Compare the highest log generation that is present in the directory to the highest log file that is required −Make sure the logs form the correct sequence −Query the checkpoint file, if one exists Replay the log file using a special recovery mode (undo phase is skipped)

36. Exchange Server 2010 HA Fundamentals Lossy Failure Process In the event of failure, the following steps will occur for the failed database: −Active Manager will determine the best copy to activate −The Replication service on the target server will attempt to copy missing log files from the source - ACLL −If successful, then the database will mount with zero data loss −If unsuccessful (lossy failure), then the database will mount based on the AutoDatabaseMountDial setting −The mounted database will generate new log files (using the same log generation sequence) −Transport Dumpster requests will be initiated for the mounted database to recover lost messages −When original server or database recovers, it will run through divergence detection and perform an incremental reseed or require a full reseed

37. Exchange Server 2010 HA Fundamentals Active Manager Selection of Active Database Copy Active Manager selects the “best” copy to become active when existing active fails CatalogHealthy Copy statusHealthy, DisconnectedAndHealthy, DisconnectedAndResynchronizing, or SeedingSource CopyQueueLength< 10 ReplayQueueLength< 50 CatalogCrawling Copy statusHealthy, DisconnectedAndHealthy, DisconnectedAndResynchronizing, or SeedingSource CopyQueueLength< 10 ReplayQueueLength< 50 CatalogHealthy Copy statusHealthy, DisconnectedAndHealthy, DisconnectedAndResynchronizing, or SeedingSource ReplayQueueLength< 50 CatalogCrawling Copy statusHealthy, DisconnectedAndHealthy, DisconnectedAndResynchronizing, or SeedingSource ReplayQueueLength< 50 5 Copy statusHealthy, DisconnectedAndHealthy, DisconnectedAndResynchronizing, or SeedingSource ReplayQueueLength< 50 6 CatalogHealthy Copy statusHealthy, DisconnectedAndHealthy, DisconnectedAndResynchronizing, or SeedingSource CopyQueueLength< 10 7 CatalogCrawling Copy statusHealthy, DisconnectedAndHealthy, DisconnectedAndResynchronizing, or SeedingSource CopyQueueLength< 10 8 CatalogHealthy Copy statusHealthy, DisconnectedAndHealthy, DisconnectedAndResynchronizing, or SeedingSource 9 CatalogCrawling Copy statusHealthy, DisconnectedAndHealthy, DisconnectedAndResynchronizing, or SeedingSource 10 Copy statusHealthy, DisconnectedAndHealthy, DisconnectedAndResynchronizing, or SeedingSource

38. DB1 Exchange Server 2010 HA Fundamentals Incremental Resync Incremental reseed scenario −Active DB1 on server1 fails −Passive DB1 on server3 takes over service −Sometime later, failed DB1 on server1 comes back as passive – contains inconsistent data −Make DB1 on server1 consistent with new active Transaction logs of active and failed copy are compared to find divergence point Determines from logs the database pages that changed after divergent point Copies database pages from active to failed copy, then play new logs, until in-sync Replaces Exchange Server 2007’s Lost Log Resilience (LLR) −LLR is set to 1 DB1 Mailbox Server 1 Mailbox Server 2 Mailbox Server 3 DB1 X

39. Exchange Server 2010 HA Fundamentals Backups Streaming backup APIs for public use have been cut, must use Volume Shadow Copy Service (VSS) for backups −Backup from any copy of the database/logs −Always choose Passive (or Active) copy −Backup an entire server −Designate a dedicated backup server for a given database Restore from any of these backups scenarios DB2 DB3 DB2 DB3 DB1 DB3 DB1 VSS requestor DB2 Database Availability Group Mailbox Server 1 Mailbox Server 2 Mailbox Server 3

40. Multiple Database Copies Enable Backupless Configurations Exchange Server 2010 HA E-mail archive Extended/protected dumpster retention 7-14 day lag copy X Database Availability Group Mailbox Server 1 Mailbox Server 2 Mailbox Server 3 DB1 DB2 DB3 DB1 DB2 DB3 DB1 DB2 DB3 Site/server/disk failure Archiving/compliance Recover deleted items

41. Agenda  Review of Exchange Server 2007 Availability solutions  Overview of Exchange Server 2010 High Availability  Exchange Server 2010 High Availability fundamentals  Exchange Server 2010 High Availability architecture scenarios  Exchange Server 2010 site resilience

42. File Share High Availability architect scenarios CCR Design -> DAG Design

43. Single Site 3 HA Copies Database Availability Group (DAG) Mailbox Server 1 Mailbox Server 2 Mailbox Server 3 3 Nodes X X JBOD -> 3 physical Copies 2 servers out -> manual activation of server 3 In 3 server DAG, quorum is lost DAGs with more servers sustain more failures – greater resiliency High Availability architect scenarios Double Resilience – Maintenance + DB Failure

44. CAS/HUB/ MAILBOX 1 CAS/HUB/ MAILBOX 2 Member servers of DAG can host other server roles DB2 2 server DAGs, with server roles combined or not, should use RAID High Availability architect scenarios Branch Office or Smaller Deployment

45. Agenda Review of Exchange Server 2007 High Availability solutions Overview of Exchange Server 2010 High Availability Exchange Server 2010 High Availability fundamentals Exchange Server 2010 High Availability architecture scenarios Exchange Server 2010 site resilience

46. Exchange Server 2010 *over Cases Within a datacenter −Database *over −Server *over Between datacenters −Single database *over −Server *over Datacenter failover (which is really a switchover)

47. Single DB Cross- Datacenter *Over −Database mounted in another datacenter and another Active Directory site −Serviced by “new” Hub Transport servers −“Different OwningServer” – for routing −Transport dumpster re-delivery now from both Active Directory sites −Serviced by “new” CAS −“Different CAS URL” – for protocol access −Outlook Web Access (OWA) now re-directs connection to second CAS farm −Other protocols proxy or redirect (varies)

48. Cross-Site DB Failover (Direct Connect) RPCClientAccessServer = CAS-PRI

49. Cross-Site DB Failover (Redirect) RPCClientAccessServer = CAS- SEC Autodiscover detects profile change and requires restart of Client Outlook 2003 fails to connect due to CAS-PRI failure Connection fails due to CAS-PRI failure. Autodiscover detects profile change and requires restart of Client

50. Datacenter Failover −Customers can evolve to site resilience −Standalone  local redundancy  site resilience −Consider name space design at first deployment −Keep extending the DAG! −Monitoring and many other concepts/skills just re-applied −Normal administration remains unchanged −Disaster recovery not HA event

51. Split Brain Management Two datacenter *overs have a risk of split brain Primary datacenter power outage is classic example Exchange Server 2010 datacenter failovers maintain DAG membership but shrink cluster membership to create a new, “available topology” in the standby datacenter Exchange Server 2010 provides a safe answer with “datacenter activation coordination” (DAC) mode −Requires a DAG with three nodes −Requires activation in partial datacenter failure cases is “done right” −Mailbox servers must be “stopped” or powered off −Implements a “Mommy may I protocol” before active manager mounts databases

52. Split Brain Management (Cont’d) If DAC is not enabled, the DAG will not restart and mount databases until a majority of servers are restored If DAC is enabled, the “Mommy May I Protocol” is used to coordinate with Active Managers in DAG to determine state and recoverability There are several requirements that must be satisfied to prevent split brain between datacenters after datacenter failover

53. DAG1 DB1DB2 DB3 DB4 DB1DB2 DB3 DB4 DB1DB2 DB3 DB4 DB1DB2 DB3 DB4 DB1DB2 DB3 DB4 Failure Scenario: Database Failure 1.MBX-A-1 DB1 fails 2.Automatic failover to MBX-A-2 3.MBX-A-1 DB1 is fixed and becomes a copy DB1DB2 DB3 DB4 DB1DB2 DB3 DB4 Failure Scenario: Server Failure 1.MBX-A-1 fails 2.Automatic failover to MBX-A-2 3.MBX-A-1 is fixed DB1DB2 DB3 DB4 DB1DB2 DB3 DB4 DB1DB2 DB3 DB4 Failure Scenario: Data Center Failure 1.Primary data center fails 2.Adjust DNS records for SMTP and HTTPS access and adjust CAS configuration (if necessary) 3.Run Stop-DatabaseAvailabilityGroup DAG1 –ActiveDirectorySite Redmond –ConfigurationOnly (in both data centers) 4.Restore-DatabaseAvailabilityGroup DAG1 –ActiveDirectorySite “Bel Air” –AlternateFileShareWitnessShare \\ht- b\fsw 5.Databases mount (no activation block scenario) DB1DB2 DB3 DB4 DB1DB2 DB3 DB4 DB1DB2 DB3 DB4 LegendActive Database Database Copy Unhealthy? Database Contoso.com (MX Record) Autodiscover.contoso.com Mail.contoso.com Load Balance Array Records Contoso.com (MX Record) Autodiscover.contoso.com Mail.contoso.com Load Balance Array Records Outlook 2007/14 (MBX on DB1) Recovering Primary Data Center 1.Verify primary data center is capable of hosting service 2.Add primary data center servers back to DAG: Start-DatabaseAvailabilityGroup DAG1 –ActiveDirectorySite Redmond 3.Reconfigure DAG to use File Share Witness in primary data center: Set-DatabaseAvailabilityGroup DAG1 –FileShareWitnessShare \\ht-a\fsw 4.Reseed data or allow replication to occur and update copies in primary data center 5.Schedule downtime for the mailbox databases and dismount them 6.Change MX records and HTTP access back to primary data center 7.Move databases back to primary data center: Move-ActiveMailboxDatabase DB1 –ActivateOnServer MBX-A-1 8.Mount databases in primary data center DB1DB2 DB3 DB4

54. End of Exchange 2010 High Availability Module

55. For More Information Exchange Server Tech Center http://technet.microsoft.com/en-us/exchange/default.aspx http://technet.microsoft.com/en-us/exchange/default.aspx Planning services http://technet.microsoft.com/en-us/library/cc261834.aspx http://technet.microsoft.com/en-us/library/cc261834.aspx Microsoft IT Showcase Webcasts http://www.microsoft.com/howmicrosoftdoesitwebcasts http://www.microsoft.com/howmicrosoftdoesitwebcasts Microsoft TechNet http://www.microsoft.com/technet/itshowcase http://www.microsoft.com/technet/itshowcase

56. © 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.