1. Henk Den Baes Technology Advisor Microsoft BeLux
High availability and Disaster Recovery in a Multi-Site Virtual Environment using virtualization
Henk Den Baes
Technology Advisor
Microsoft BeLux

2. HA & DR with Multi-Site Clustering
Introduction
Networking
Storage
Quorum

3. Session Objectives Session Objective(s):
3/31/2017
Session Objectives
Session Objective(s):
Clustering is not too expensive and not that complex
Understanding the need and benefit of multi-site clusters
What to consider as you plan, design, and deploy your first multi-site cluster
Clustering your Hyper-V servers is a great solution for not only high availability, but also disaster recovery
© 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

4. Hyper-V Virtualization Scenarios
Microsoft Virtualization Deployment Summit
Hyper-V Virtualization Scenarios
3/31/2017 4:42 PM
Server Consolidation
Business Continuity
Test and Dev
Dynamic Datacenter
© 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

5. Keeping the Business Running
Business Continuity
Resumption of full operations combining People, Processes and Platforms
Disaster Recovery
Site-level crisis , data and IT operations resumption
Backup and Restore
Presumes infrastructure is whole
97% is file/small unit related
High Availability
Presumes that the rest of the environment is active 5

6. Business Continuity with Virtualization
Virtualization reduces BC costs and minimizes business downtime by:
increasing the availability of infrastructure
extending protection to more applications
simplifying backups, recovery and DR testing
High Availability
Disaster Recovery
Backup and Recovery
Disaster Recovery
VHD
Secondary Site
Primary Site
Storage Array
Clustering
Shared Storage
CSV
Quick/Live Migration
Backup/Recovery
Backup/Recovery
Backup/Recovery 6

7. Differences Between Single-site & Multi-site Clusters
Two node single-site cluster
Primary Site
Secondary Site
WAN Connectivity
VMs move between nodes on the same SAN and share common storage
VMs move between physical nodes on different SANs and without true shared storage between the sites
SAN
SAN
SAN
SAN Replication
Storage Array
Primary Storage Array
Secondary Storage Array
Single-site Cluster
Multi-site Cluster

8. Microsoft Stretch Clustering & Storage Continuity
Geographically distributed clusters are extended to different physical locations
Stretch clustering uses the same concept as local site clustering
Storage array or third party software provides SAN data replication
Stretch Clustering automatically fails VMs over to a geographically different site
Replicated data from site A
Primary site data is replicated to the secondary site
Storage Array
Storage Array
Primary Site
Secondary Site
Multi-site stretch configurations can provide automatic fail-over

9. Benefits of a Multi-Site Cluster
Protects against loss of an entire datacenter
Automates failover
Reduced downtime
Lower complexity disaster recovery plan
Reduces administrative overhead
Automatically synchronize application and cluster changes
Easier to keep consistent than standalone servers
The primary reason DR solutions fail is
dependence on people

10. DR: NMBS VDI use case - NOC
Windows 7 master
NOC is installed on 1 site
DRP is costly & has to be tested yearly
There is no automatic app. Sync
Dedicated master, manual upgrades
No persistent image, need for admin rights
Solution:
Remote Desktop Services
Hyper-V R2
Remote Desktop Connection

11. HA & DR with Multi-Site Clustering
Introduction
Networking
Storage
Quorum

12. Network Considerations
Network Deployment Options:
Stretch VLAN’s across sites
Cluster nodes can reside in different subnets
Site A
Public Network
Redundant Network
Site B

13. Stretching the Network
Longer distance traditionally means greater network latency
Missed inner-node health checks can cause false failover
Cluster inner-node heartbeating is fully configurable
SameSubnetDelay (default = 1 second)
Frequency heartbeats are sent
SameSubnetThreshold (default = 5 heartbeats)
Missed heartbeats before an interface is considered down
CrossSubnetDelay (default = 1 second)
Frequency heartbeats are sent to nodes on dissimilar subnets
CrossSubnetThreshold (default = 5 heartbeats)
Missed heartbeats before an interface is considered down to nodes on dissimilar subnets
Command Line: Cluster.exe /prop
PowerShell (R2): Get-Cluster | fl *

14. Updating VM’s IP on Subnet Failover
On cross-subnet failover, if guest is…
Best to use DHCP in guest OS for cross-subnet failover
DHCP
IP updated automatically
Static IP
Admin needs to configure new IP
Can be scripted

15. Client Reconnect Considerations
Nodes in dissimilar subnets
VM obtains new IP address
Clients need that new IP Address from DNS to reconnect
DNS Server 1
DNS Server 2
DNS Replication
Record Created
Record Updated
VM =
Site A
Site B
Record Updated

16. Solutions Solution #1: Prefer Local Failover
Scale up for local failover for higher availability
No change in IP addresses for HA
Means not going over the WAN and is still usually preferred
Cross-site failover for disaster recovery
Solution #2: Stretch VLAN’s
Deploying a VLAN minimizes client reconnection times
IP of the VM never changes
Solution #3: Abstraction in Network Device
Network device uses 3rd IP
3rd IP is the one registered in DNS & used by client

17. HA & DR with Multi-Site Clustering
Introduction
Networking
Storage
Quorum

18. Storage in Multi-Site Clusters
Different than local clusters:
Multiple storage arrays – independent per site
Nodes commonly access own site storage
No ‘true’ shared disk visible to all nodes
Site B
SAN
Site A

19. Storage Considerations
Site A
Site B
SAN
Replica
Changes are made on Site A and replicated to Site B
Requires data replication mechanism between sites

20. Synchronous Replication
Host receives “write complete” response from the storage after the data is successfully written on both storage devices
Primary Storage
Secondary Storage
Write Complete
Replication
Acknowled
gement
Write Request

21. Asynchronous Replication
Host receives “write complete” response from the storage after the data is successfully written to just the primary storage device, then replication
Secondary Storage
Write Complete
Write Request
Replication
Primary Storage

22. Synchronous vs. Asynchronous
No data loss
Potential data loss on hard failures
Requires high bandwidth/low latency connection
Enough bandwidth to keep up with data replication
Stretches over shorter distances
Stretches over longer distances
Write latencies impact application performance
No significant impact on application performance

23. Hardware Replication Partners
Hardware storage-based replication
EMC Cluster Enabler
SRDF /CE for DMX arrays
RecoverPoint /CE for Clariion arrays
HP Cluster Extension
HP StorageWorks CLX
HP LeftHand
NetApp
MetroCluster
IBM
IBM XIV Storage System
Compellent
LiveVolume
HDS
Hitachi Storage Cluster (HSC)

24. Software Replication Partners
Software host-based replication
Double-Take Availability
SteelEye DataKeeper Cluster Edition
Symantec Storage Foundation for Windows

25. Storage Virtualization Abstraction
Some replication solutions provide complete abstraction in storage array
Servers are unaware of accessible disk location
Fully compatible with Cluster Shared Volumes (CSV)
Site B
Site A
Virtualized storage presents logical LUN
Servers abstracted from storage

26. Focus on Double-Take for Hyper-V
Product Features
Host level filter driver replication
Simplified management
Auto discovery, guest level policies, & guest protection schema
Not a file level protection product (block based)
One click failover and failover management
WAN support (bandwidth throttling, compression…)
Integration with SCOM and SCVMM
All managed via one familiar console
Licensed per Hyper-V Host
Unlimited number of VMs

27. Basic Double-Take Configuration

28. Hyper-V’s File-Based Data
*.VHD – Files that hold all the data for a particular volume associated with a virtual machine
*.AVHD - When you take a snapshot of a virtual machine, its .VHD files are “frozen” and subsequent disk writes within the VM are instead stored in a .AVHD file
*.XML - Hyper-V stores a virtual machine’s configuration information in an industry-standard XML file
*.BIN and *.VSV - When you pause or take a snapshot of a running virtual machine, Hyper-V stores the contents of the virtual machine’s RAM in a .BIN file and information about the current state of the running virtual machine in a .VSV file

29. How Double-Take Replication Works
Operating
System
Hardware
Layer
File System
Applications
Double-Take Filter
Initial Mirror of Data
WAN Optimized
Three Levels of Data Compression and Scheduled Bandwidth Limiting Capabilities
Any IP Network

30. Host-Level Protection for Hyper-V
VHD
VHD
Hyper-V
Host
Hyper-V
Host

31. Double-Take GeoCluster
Integrates with Microsoft Failover Clustering
Uses Double-Take Patented Replication
Extends Clusters Across Geographical Distances
Eliminates Single Point of Disk Failure

32. How Double-Take GeoCluster Works
At failover, the new active node resumes with current, replicated data
Only the active node accesses its disks
Data is replicated to all passive nodes
Replication
GeoCluster nodes use separate disks, kept synchronized by real-time replication

33. GeoCluster for Hyper-V Workloads
Product Features
Provides redundancy of storage
Allows cluster nodes to be geographically distributed
Utilizes GeoCluster technology to extend Hyper-V clustering across virtual hosts without the use of shared disk
Replicates cluster data to a secondary node, eliminating single point of failure
Allows manual and automatic moves of cluster resources between virtual hosts

34. CSV with Replicated Storage
Traditional architectural assumptions may collide…
Traditional replication solutions typically assume only 1 array accessed at a time
Cluster Shared Volumes assumes all nodes can concurrently access a LUN
Talk to your storage vendor for their support story
Site B
Site A
VHD
Read/Only
Read/Write
VM attempts to access replica

35. HA & DR with Multi-Site Clustering
Introduction
Networking
Storage
Quorum

36. Quorum Overview Majority is greater than 50% Possible Voters:
Nodes (1 each) + 1 Witness (Disk or File Share)
4 Quorum Types
Disk only (not recommended)
Node and Disk majority
Node majority
Node and File Share majority
Vote
Vote
Vote
Vote
Vote

37. Replicated Disk Witness
A witness is a tie breaker when nodes lose network connectivity
When a witness is not a single decision maker, problems occur
Do not use in multi-site clusters unless directed by vendor
Vote
Vote
Vote ?
Replicated Storage

38. Node Majority 5 Node Cluster: Majority = 3 Site B Site A
Can I communicate with majority of the nodes in the cluster?
Yes, then Stay Up
5 Node Cluster:
Majority = 3
Can I communicate with majority of the nodes in the cluster?
No, drop out of Cluster Membership
Site B
Site A
Cross site network connectivity broken!
Majority in Primary Site

39. Node Majority 5 Node Cluster: Majority = 3 Site B Site A
We are down!
5 Node Cluster:
Majority = 3
Can I communicate with majority of the nodes in the cluster?
No, drop out of Cluster Membership
Site B
Site A
Need to force quorum manually
Disaster at Site 1
Majority in Primary Site

40. Forcing Quorum
Forcing quorum is a way to manually override and start a node even though it has not achieved quorum
Always understand why quorum was lost
Used to bring cluster online without quorum
Cluster starts in a special “forced” state
Once majority achieved, drops out of “forced” state
Command Line:
net start clussvc /fixquorum (or /fq)
PowerShell (R2):
Start-ClusterNode –FixQuorum (or –fq)

41. Multi-Site with File Share Witness
Site C (branch office)
SCENARIO:
Complete resiliency and automatic recovery from the loss of any 1 site
\\Foo\Share
WAN
Site A
Site B

42. Multi-Site with File Share Witness
Site C (branch office)
Can I communicate with majority of the nodes (+FSW) in the cluster?
Yes, then Stay Up
Can I communicate with majority of the nodes in the cluster?
No (lock failed), drop out of Cluster Membership
\\Foo\Share
WAN
SCENARIO:
Complete resiliency and automatic recovery from the loss of connection between sites
Site A
Site B

43. File Share Witness (FSW) Considerations
Simple Windows File Server
Single file server can serve as a witness
for multiple clusters
Each cluster requires it’s own share
Can be made highly available on a separate cluster
Recommended to be at 3rd separate site to enable automatic site failover
FSW cannot be on a node in the same cluster
FSW should not be in a VM running on the same cluster

44. Node and File Share Majority
Quorum Model Recap
Node and File Share Majority
Even number of nodes
Best availability solution – FSW in 3rd site
Node Majority
Odd number of nodes
More nodes in primary site
Node and Disk Majority
Use as directed by vendor
No Majority: Disk Only
Not Recommended

45. Datacenter Recovery Partners
Citrix Essentials for Hyper-V augments Hyper-V DR by automating disaster recovery configuration
StorageLink Site Recovery manages storage automation
Workflow orchestration for VM site failover
Non-disruptive testing & staging of VM prior to failover
Single click failback
Recovery plans
Integrates with SCVMM
Plus more…

46. Microsoft Site Recovery Solution Stack End to end Disaster Recovery
Storage and Data Availability
Server and Application Availability
Hyper-V
Clustering
Quick and Live Migration
Synchronous & Asynchronous Replication
Array state and application restart
Workflow automation
DR Run-book
Simplified configuration & testing
Management
Storage Partner Data Replication
Automation
Physical and Virtual
Performance and Resource Optimization

47. Microsoft Private Cloud – Server Platform
Simplify with integrated physical, virtual and cloud management
Improve agility with private cloud computing infrastructure
Optimize service delivery across datacenter infrastructure and business critical services
Lower costs through automation
IT Process Automation
Data Protection & Recovery
Design, Configure & Deploy
IT Service Management
Monitor & Manage Service End to End
Virtualize, Deploy & Manage
“We don’t have to manage our infrastructure with multiple tools…we have one central monitoring and management console from which we can care for every aspect of our environment”  - Doug Miller, Practice Architect, Microsoft Practice Group, CDW

48. 3/31/2017 4:42 PM
© 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
© 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.