Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.

Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.
Local Replication Module 3.3 Module 3.3 – Local Replication

Local Replication After completing this module you will be able to: Discuss local replication and the possible uses of local replicas Explain consistency considerations when replicating file systems and databases Discuss host and array based replication technologies Functionality Differences Considerations Selecting the appropriate technology The objectives for this module are shown here. Please take a moment to read them. Local Replication

Lesson: Local Replica and Data Consistency
Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved. Lesson: Local Replica and Data Consistency Upon completion of this lesson, you will be able to: Define local replication Discuss the possible uses of local replicas Explain replica considerations such as Recoverability and Consistency Describe how consistency is ensured in file system and database replication Explain Dependent write principle The objectives for this lesson are shown here. Please take a moment to read them. Local Replication

What is Replication Replica - An exact copy Replication - The process of reproducing data Local replication - Replicating data within the same array or the same data center Replication is the process of creating an exact copy of data. Creating one or more replicas of the production data is one of the ways to provide Business Continuity (BC). These replicas can be used for recovery and restart operations in the event of data loss. The primary purpose of replication is to enable users to have designated data at the right place, in a state appropriate to the recovery need. The replica should provide recoverability and re-startability. Recoverability enables restoration of data from the replicas to the production volumes in the event of data loss or data corruption. It must provide minimal RPO and RTO for resuming business operations on the production volumes, while re-startability must ensure consistency of data on the replica. This enables restarting business operations using the replicas. Local replication refers to replicating data within the same array or the same data center. REPLICATION Source Replica (Target) Local Replication

Possible Uses of Local Replicas
Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved. Possible Uses of Local Replicas Alternate source for backup Fast recovery Decision support Testing platform Data Migration One or more local replicas of the source data can be created for various purposes, including the following: Alternate source for backup: Under normal backup operations, data is read from the production volumes (LUNs) and written to the backup device. This places additional burden on the production infrastructure, as production LUNs are simultaneously involved in production work. As the local replica contains an exact point-in-time (PIT) copy of the source data, it can be used to perform backup operations. Another benefit of using local replicas for backup is that it reduces the backup window size. Fast recovery: In the event of a partial failure of the source, or data corruption, a local replica can be used to recover lost data. In the event of a complete failure of the source, the replica can be restored to a different set of source devices. In either case, this method provides faster recovery and minimal RTO. Decision-support activities such as reporting: Running the reports using the data on the replicas greatly reduces the I/O burden placed on the production device. Testing platform: A local replica can be used for testing critical business data or applications. For example, when planning an application upgrade, it can be tested using the local replica. If the test is successful, it can be restored to the source volumes. Data migration: Local replication can also be used for data migration. Data migration may be performed for various reasons, such as migrating from a small LUN to a larger LUN. Local Replication

Replication Considerations
Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved. Replication Considerations Types of Replica: choice of replica tie back into RPO Point-in-Time (PIT) non zero RPO Continuous near zero RPO What makes a replica good Recoverability/Re-startability Replica should be able to restore data on the source device Restart business operation from replica Consistency Ensuring consistency is primary requirement for all the replication technologies Key factors to consider with replicas: Replicas can either be Point-in-Time (PIT) or continuous: Point-in-Time (PIT) - the data on the replica is an identical image of the production at some specific timestamp For example, a replica of a file system is created at 4:00 PM on Monday. This replica would then be referred to as the Monday 4:00 PM Point-in-Time copy Note: The RPO will be a finite value with any PIT. The RPO will map to the time when the PIT was created to the time when any kind of failure on the production occurred. If there is a failure on the production at 8:00 PM and there is a 4:00 PM PIT available, the RPO would be 4 hours (8 – 4 = 4). To minimize RPO with PITs, take periodic PITs Continuous replica - the data on the replica is synchronized with the production data at all times The objective with any continuous replication is to reduce the RPO to zero What makes a replica good: Consistency: A consistent replica ensures that data buffered in the host is properly captured on the disk when the replica is created. Ensuring consistency is the primary requirement for all the replication technologies. Recoverability from a failure: The replication technology must allow for the restoration of data from the replicas to the production. Re-startability: Restart business operation from the replica after failure of source Local Replication

Understanding Consistency
Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved. Understanding Consistency Consistency is required to ensure the usability of replica Consistency can be achieved in various ways: For file Systems Offline: Un-mount file system Online: Flush host buffers For Databases Offline: Shutdown database Online: Database in hot backup mode Dependent Write I/O Principle By Holding I/Os Consistency is a primary requirement to ensure the usability of replica device. In case of file Systems consistency can be achieved either by taking FS offline i.e. by un-mounting FS or by keeping FS online and flushing host buffers before creating replica. Similarly in case of databases either database need to be shutdown for creating consistent replica or using hot backup mode for online database. Consistent replica of online database can be created by using dependent write I/O consistency or by Holding I/Os before creating replica. Local Replication

File System Consistency: Flushing Host Buffer
Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved. File System Consistency: Flushing Host Buffer Application File System Data Sync Daemon Memory Buffers Logical Volume Manager Physical Disk Driver File systems buffer data in host memory to improve application response time. The buffered information is periodically written to disk. In UNIX operating systems, the sync daemon is the process that flushes the buffers to disk at set intervals. In some cases, the replica may be created in between the set intervals. Hence, the host memory buffers must be flushed to ensure data consistency on the replica, prior to its creation. Figure on the slide illustrates flushing of the buffer to its source, which is then replicated. If the host memory buffers are not flushed, data on the replica will not contain the information that was buffered in the host. If the file system is un-mounted prior to the creation of the replica, the buffers would be automatically flushed and data would be consistent on the replica. If a mounted file system is replicated, some level of recovery such as fsck or log replay would be required on the replicated file system. When the file system replication process is completed, the replica file system can be mounted for operational use. Source Replica Local Replication

Database Consistency: Dependent write I/O Principle
Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved. Database Consistency: Dependent write I/O Principle Dependent Write: A write I/O that will not be issued by an application until a prior related write I/O has completed A logical dependency, not a time dependency Inherent in all Database Management Systems (DBMS) e.g. Page (data) write is dependent write I/O based on a successful log write Necessary for protection against local outages Power failures create a dependent write consistent image A Restart transforms the dependent write consistent to transitionally consistent i.e. Committed transactions will be recovered, in-flight transactions will be discarded A database may be spread over numerous files, file systems, and devices. All of these must be replicated consistently to ensure that the replica is restorable and re-startable. Replication can be performed with the database offline or online. If the database is offline, it is not available for I/O operations. Because no updates are occurring, the replica will be consistent. If the database is online, it is available for I/O operations. Transactions to the database will be updating data continuously. When a database is backed up while it is online, changes made to the database at this time must be applied to the backup copy to make it consistent. Performing an online backup requires additional procedures during backup and restore. Local Replication

Database Consistency: Dependent Write I/O
Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved. Database Consistency: Dependent Write I/O Source Replica Source Replica 1 1 1 2 2 2 3 3 3 3 Consistency of an online backup can be ensured using dependent write I/O principle inherent in any database management system (DBMS). In order for a transaction to be deemed complete, databases require that a series of writes have to occur in a particular order. These writes would be recorded on the various devices/file systems. Figure on the slide, illustrates the process of flushing the buffer from host to source; I/Os 1 to 4 must complete, in order for the transaction to be considered complete. I/O 4 is dependent on I/O 3 and will occur only if I/O 3 is complete. I/O 3 is dependent on I/O 2, which in turn depends on I/O 1. Each I/O completes only after completion of the previous I/O(s). Dependent write consistency is required for protection against power outages, loss of local channel connectivity, or storage devices. 4 4 4 4 C C D Consistent Inconsistent Local Replication

Database Consistency: Holding I/O
Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved. Database Consistency: Holding I/O Source Replica 1 1 5 5 2 2 3 3 An alternative way to ensure that an online replica is consistent is to: Hold I/O to all the devices at the same instant Create the replica Release the I/O Holding I/O is similar to a power failure and most databases have the ability to restart from a power failure. Note: While holding I/O simultaneously, one ensures that the data on the replica is identical to that on the source devices. The database application times out if I/O is held for too long. 4 4 Consistent Local Replication

Lesson Summary Key points covered in this lesson: Possible uses of local replicas Alternate source for backup Fast recovery Decision support Testing platform Data Migration Recoverability and Consistency File system and database replication consistency Dependent write I/O principle This concludes the lesson. Key points covered in this lesson are shown here. Please take a moment to review them. Local Replication

Lesson: Local Replication Technologies
Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved. Lesson: Local Replication Technologies Upon completion of this lesson, you will be able to: Discuss Host and Array based local replication technologies Options Operation Comparison The objectives for this lesson are shown here. Please take a moment to read them. Local Replication

Local Replication Technologies
Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved. Local Replication Technologies Host based Logical Volume Manager (LVM) based mirroring File System Snapshot Storage Array based Full volume mirroring Pointer based full volume replication Pointer based virtual replication Local Replication technologies can classified based on where the replication is performed. Host based - all the replication is performed by using the CPU resources of the host using software that is running on the host. Array based - all replication is performed on the storage array using CPU resources on the array via the array’s operating environment. Note: In the context of this discussion, local replication refers to replication that is performed within a data center if it is host based, and within a storage array if it is array based. Local Replication

Host Based Replication: LVM Based Mirroring
Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved. Host Based Replication: LVM Based Mirroring Physical Volume 1 Host Logical Volume Logical Volume Physical Volume 2 In LVM-based replication, logical volume manager is responsible for creating and controlling the host-level logical volume. An LVM has three components: physical volumes (physical disk), volume groups, and logical volumes. A volume group is created by grouping together one or more physical volumes. Logical volumes are created within a given volume group. A volume group can have multiple logical volumes. In LVM-based replication, each logical partition in a logical volume is mapped to two physical partitions on two different physical volumes. An application write to a logical partition is written to the two physical partitions by the LVM device driver. This is also known as LVM mirroring. Mirrors can be split and the data contained therein can be independently accessed. LVM mirrors can be added or removed dynamically. Local Replication

File System Snapshot Pointer-based replica Uses Copy on First Write principle Uses bitmap and block map Bitmap: Used to track blocks that have changed on the production/source FS after creation of snap – initially all zero Block map: Used to indicate block address from which data is to be read when the data is accessed from the Snap FS – initially points to production/source FS Requires a fraction of the space used by the original FS Implemented by either FS itself or by LVM File system (FS) snapshot is a pointer-based replica that requires a fraction of the space used by the original FS. This snapshot can be implemented by either FS itself or by LVM. It uses Copy on First Write (CoFW) principle. CoFW mechanism is discussed later in this module. When the snapshot is created, a bitmap and a block map are created in the metadata of the Snap FS. The bitmap is used to keep track of blocks that are changed on the production FS after creation of the snap. The block map is used to indicate the exact address from which data is to be read when the data is accessed from the Snap FS. Immediately after creation of the snapshot all reads from the snapshot will actually be served by reading the production FS. To read from the Snap FS, the bitmap is consulted. If the bit is 0, then the read is directed to the production FS. If the bit is 1, then the block address is obtained from the block map and data is read from that address. Reads from the production FS work as normal. Local Replication

File System Snapshots – How it Works
Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved. File System Snapshots – How it Works Write to Production FS Snap FS Metadata Prod FS BLK Bit 1-0 1-0 Metadata 2-0 2-0 1 Data a 3-0 3-2 3-0 3-1 2 Data b 4-1 4-0 4-0 4-1 3 Data C c New writes For example a update occurs to the 3rd block in the Prod FS which has a value of ‘Data c’. The new value should be ‘Data C’ The snapshot application will hold the IO to the Prod FS and will first copy the old data ‘Data c’ to an available data block on the Snap FS. The bitmap and block map of location 3 is changed in the snap metadata. The bitmap of 3 is changed to 1 indicating that this block has changed on the Prod FS. The block map of 3 is changed to indicate the block number where the data is actually written in Snap FS, (in this case block 2). Once this is done the IO to the Prod FS will be allowed to complete. Any new writes to the 3th block on the Prod FS will occur as normal. In a similar manner if an IO is issued the 4th block on the Prod FS to change the value from ‘Data d’ to ‘Data D’, the snapshot application hold the IO to the Prod FS and will copy the old data to an available data block on the Snap FS. Then it will change the bitmap of block 4 to 1 indicating that the data block has changed on the Prod FS. The block data for block 4 will indicate the actual data block the data can be found on, in this case data block 1 of the Snap FS. Once this is done the IO to the Prod FS will be allowed to complete. 4 Data D d 1 Nodata 1 Data d 1 no data 2 no data 2 Data c 3 no data N Data N 4 no data Local Replication

File System Snapshots – How it Works
Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved. File System Snapshots – How it Works Reads from snap FS Consult the bitmap If 0 then direct read to the production FS If 1 then go to the block map get the block address and read data from that address Snap FS Metadata Prod FS BLK Bit Metadata 1-0 1-0 1 Data a 2-0 2-0 2 Data b 3-2 3-1 3 Data C 4-1 4-1 4 Data D The reads from a Snap FS are illustrated in the graphic. In the same example the bitmap of the Snap FS indicates that the data in blocks 1 and 2 have not changed since the creation of the snap hence the data should actually be read from the Prod FS. The bitmap for blocks 3 and 4 indicate that the data on the Prod FS has changed since the creation of the snap and that the block map should be consulted. The block map for block 3 indicates that the data is located in the 2nd data block of the Snap FS, while the block map for block 4 indicates that the data is located in the 1st data block. 1 Data d 1 Nodata 2 Data c N Data N 3 no data 4 no data Local Replication

Host Based Replication: Limitations
Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved. Host Based Replication: Limitations LVM based replicas add overhead on host CPUs If host volumes are already storage array LUNs then the added redundancy provided by LVM mirroring is unnecessary The devices will have some RAID protection already Host based replicas can be usually presented back to the same server Keeping track of changes is a challenge after the replica has been created As every write generated by an application translates into two writes on the disk, an additional burden is placed on the host CPU. This can degrade application performance. Presenting an LVM-based local replica to a second host is usually not possible because the replica will still be part of the volume group, which is usually accessed by one host at any given time. Tracking changes to the mirrors and performing incremental synchronization operations is also a challenge as all LVMs do not support incremental resynchronization. If the devices are already protected by some level of RAID on the array, then the additional protection provided by mirroring is unnecessary. This solution does not scale to provide replicas of federated databases and applications. Both the replica and the source are stored within the same volume group. Therefore, the replica itself may become unavailable if there is an error in the volume group. If the server fails, both source and replica are unavailable until the server is brought back online. Local Replication

Storage Array Based Local Replication
Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved. Storage Array Based Local Replication Replication performed by the Array Operating Environment Replicas are on the same array Types of array based replication Full-volume mirroring Pointer-based full-volume replication Pointer-based virtual replication In storage array-based local replication, the array operating environment performs the local replication process. The host resources such as CPU and memory are not used in the replication process. Consequently, the host is not burdened by the replication operations. The replica can be accessed by an alternate host for any business operations. In this replication, the required number of replica devices should be selected on the same array and then data is replicated between source-replica pairs. A database could be laid out over multiple physical volumes: and in that case all the devices must be replicated for a consistent PIT copy of the database. Storage array-based local replication can be further categorized as full-volume mirroring, pointer-based full-volume replication, and pointer-based virtual replication. Replica devices are also referred as target devices, accessible by business continuity host. Array Source Replica Production Server BC Server Local Replication

Full Volume Mirroring: Attached
Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved. Full Volume Mirroring: Attached Target is a full physical copy of the source device Target is attached to the source and data from source is copied to the target Target is unavailable while it is attached Target device is as large as the source device Good for full backup, decision support, development, testing and restore to last PIT Attached In full-volume mirroring, the target is attached to the source and established as a mirror of the source. Existing data on the source is copied to the target. New updates to the source are also updated on the target. After all the data is copied and both the source and the target contain identical data, the target can be considered a mirror of the source. While the target is attached to the source and the synchronization is taking place, the target remains unavailable to any other host. However, the production host can access the source. Read/Write Not Ready Source Target Array Local Replication

Full Volume Mirroring: Detached
Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved. Full Volume Mirroring: Detached After synchronization, target can be detached from the source and made available for BC operations PIT is determined by the time of detachment After detachment, re-synchronization can be incremental Detached - PIT After the synchronization is complete, the target can be detached from the source and be made available for Business Continuity operations. The point-in-time (PIT) is determined by the time of detachment or separation of the Source and Target. For example, if the detachment time is 4:00 PM, the PIT of the replica is 4:00 PM. Note that the target device must be at least as large as the source device. After detachment, changes made to both source and replica can be tracked at some predefined granularity. This enables incremental resynchronization (source to target) or incremental restore (target to source). The granularity of the data change can range from 512 byte blocks to 64 KB blocks. Changes are typically tracked using bitmaps, with one bit assigned for each block. If any updates occur to a particular block, the whole block is marked as changed, regardless of the size of the actual update. However, for resynchronization (or restore), only the changed blocks have to be copied, eliminating the need for a full synchronization (or restore) operation. This method reduces the time required for these operations considerably. Read/Write Read/Write Source Target Array Local Replication

Full Volume Mirroring: Source and Target Relationship
Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved. Full Volume Mirroring: Source and Target Relationship Attached/ Synchronization Detached Resynchronization Source = Target Source ≠ Target Source = Target A host accessing data from one or more LUNs on the storage array is called a production host, and these LUNs are known as source LUNs (devices/volumes), production LUNs, or simply the source. A LUN (or LUNs) on which the data is replicated is called the target LUN or simply the target. Targets can also be accessed by hosts other than production hosts to perform operations such as backup or testing. Target data can be updated by the hosts accessing it without modifying the source. However, the source and the target are not an identical copy of each other anymore. The target can be incrementally resynchronized (copying of data that has changed since the previous synchronization) with the source to make both source and target identical. Local Replication

Pointer based Full Volume Replication
Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved. Pointer based Full Volume Replication Provide full copy of source data on the target Target device is made accessible for business operation as soon as the replication session is started Point-in-Time is determined by time of session activation Two modes Copy on First Access (deferred) Full Copy mode Target device is at least as large as the source device Like full-volume mirroring, this technology can provide full copies of the source data on the targets. Unlike full-volume mirroring, the target is made immediately available at the activation of the replication session. Hence, one need not wait for data synchronization to, and detachment of, the target in order to access it. The time of activation defines the PIT copy of source. Pointer-based, full-volume replication can be activated in either Copy on First Access (CoFA) mode or Full Copy mode. In either case, at the time of activation, a protection bitmap is created for all data on the source devices. Pointers are initialized to map the (currently) empty data blocks on the target to the corresponding original data blocks on the source. The granularity can range from 512 byte blocks to 64 KB blocks or higher. Data is then copied from the source to the target, based on the mode of activation. Local Replication

Copy on First Access Mode: Deferred Mode
Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved. Copy on First Access Mode: Deferred Mode Write to Source Read/Write Read/Write Source Target Write to Target Read/Write Read/Write Source Target In the Copy on First Access mode (or the deferred mode), data is copied from the source to the target only when: A write is issued for the first time after the PIT to a specific address on the source A read or write is issued for the first time after the PIT to a specific address on the target. When a write is issued to the source for the first time after session activation, original data at that address is copied to the target. After this operation, the new data is updated on the source. This ensures that original data at the point-in-time of activation is preserved on the target. When a read is issued to the target for the first time after session activation, the original data is copied from the source to the target and is made available to the host. When a write is issued to the target for the first time after session activation, the original data is copied from the source to the target. After this, the new data is updated on the target. In all cases, the protection bit for that block is reset to indicate that the original data has been copied over to the target. The pointer to the source data can now be discarded. Subsequent writes to the same data block on the source, and reads or writes to the same data blocks on the target, do not trigger a copy operation (and hence are termed Copy on First Access). Since data is only copied when required, if the replication session is terminated, the target device only has data that was copied (not the entire contents of the source at the PIT). In this scenario, the data on the Target cannot be used as it is incomplete. Read from Target Read/Write Read/Write Source Target Local Replication

Full Copy Mode On session start, the entire contents of the Source device is copied to the Target device in the background Most vendor implementations provide the ability to track changes: Made to the Source or Target Enables incremental re-synchronization In Full Copy mode, the target is made available immediately and all the data from the source is copied over to the target in the background. During this process, if a data block that has not yet been copied to the target is accessed, the replication process jumps ahead and moves the required data block first. When a full copy mode session is terminated (after full synchronization), the data on the Target is still usable as it is a full copy of the original data. Local Replication

Pointer Based Virtual Replication
Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved. Pointer Based Virtual Replication Targets do not hold actual data, but hold pointers to where the data is located Target requires a small fraction of the size of the source volumes A replication session is setup between source and target devices Target devices are accessible immediately when the session is started At the start of the session the target device holds pointers to data on source device Typically recommended if the changes to the source are less than 30% In pointer-based virtual replication, at the time of session activation, the target contains pointers to the location of data on the source. The target does not contain data, at any time. Hence, the target is known as a virtual replica. Similar to pointer-based full-volume replication, a protection bitmap is created for all data on the source device, and the target is immediately accessible. Granularity can range from 512 byte blocks to 64 KB blocks or greater. The primary advantage of pointer based copies is the reduction in storage requirement for the replicas. Local Replication

Virtual Replication: Copy on First Write Example
Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved. Virtual Replication: Copy on First Write Example Target Virtual Device Pointer-based virtual replication uses CoFW technology. When a write is issued to the source for the first time after session activation, original data at that address is copied to a predefined area in the array. This area is generally termed the save location. The pointer in the target is updated to point to this data address in the save location. After this, the new write is updated on the source. When a write is issued to the target for the first time after session activation, original data is copied from the source to the save location and similarly the pointer is updated to data in save location. Another copy of the original data is created in the save location before the new write is updated on the save location. Subsequent writes to the same data block on the source or the target do not trigger a copy operation. When reads are issued to the target, unchanged data blocks since session activation are read from the source. Original data blocks that have changed are read from the save location. Data on the target is a combined view of unchanged data on the source and data on the save location. Unavailability of the source device invalidates the data on the target. As the target only contains pointers to data, the physical capacity required for the target is a fraction of the source device. The capacity required for the save location depends on the amount of expected data change. Source Save Location Local Replication

Tracking Changes to Source and Target
Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved. Tracking Changes to Source and Target Changes will/can occur to the Source/Target devices after PIT has been created How and at what level of granularity should this be tracked Too expensive to track changes at a bit by bit level Would require an equivalent amount of storage to keep track Based on the vendor some level of granularity is chosen and a bit map is created (one for source and one for target) For example one could choose 32 KB as the granularity If any change is made to any bit on one 32KB chunk the whole chunk is flagged as changed in the bit map For 1GB device, map would only take up 32768/8/1024 = 4KB space Updates occur on the source device after the creation of point-in-time local replicas. If the primary purpose of local replication is to have a viable point in- time copy for data recovery or restore operations, then the target devices should not be modified. Changes can occur on the target device if it is used for non-BC operations. To enable incremental resynchronization or restore operations, changes to both the source and target devices after the point-in-time can be tracked. This is typically done using bitmaps, with one bit per block of data. The block sizes can range from 512 bytes to 64 KB or greater. For example, if the block size is 32 KB, then a 1 GB device would require 32,768 bits. The size of the bitmap would be 4 KB. If any or all of a 32 KB block is changed, the corresponding bit in the bitmap is flagged. Local Replication

Tracking Changes to Source and Target: Bitmap
Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved. Tracking Changes to Source and Target: Bitmap Source At PIT Target Source 1 1 1 After PIT… Target 1 1 1 The bits in the source and target bitmaps are all set to 0 (zero) when the replica is created. Any changes to the source or target are then flagged by setting the appropriate bits to 1 in the bitmap. When resynchronization or a restore is required, a logical OR operation between the source bitmap and the target bitmap is performed. The bitmap resulting from this operation references all blocks that have been modified in either the source or the target. This enables an optimized resynchronization or a restore operation, as it eliminates the need to copy all the blocks between the source and the target. The direction of data movement depends on whether a resynchronization or a restore operation is performed. If resynchronization is required, then changes to the target are overwritten with the corresponding blocks from the source If a restore is required, then changes to the source are overwritten with the corresponding blocks from the target. In either case, changes to both the source and the target cannot be simultaneously preserved. For resynchronization/restore Logical OR 1 1 1 1 1 = unchanged 1 = changed Local Replication

Restore/Restart Operation
Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved. Restore/Restart Operation Source has a failure Logical Corruption Physical failure of source devices Failure of Production server Solution Restore data from target to source The restore would typically be done incrementally Applications can be restarted even before synchronization is complete -----OR------ Start production on target Resolve issues with source while continuing operations on target After issue resolution restore latest data on target to source Local replicas can be used to restore data to production devices. Alternatively, applications can be restarted using the consistent point-in-time copy of the data on the replicas. A replica can be used to restore data to the production devices in the event of logical corruption of production devices — i.e., the devices are available but the data on them is invalid. Examples of logical corruption include accidental deletion of information (tables or entries in a database), incorrect data entry, and incorrect updating to existing information. Restore operations from a replica are incremental and provide a very small RTO. In some instances, applications can be resumed on the production devices prior to completion of the data copy. Production devices may also become unavailable due to physical failures, such as production server or physical drive failure. In this case, applications can be restarted using data on the latest replica. If the production server fails, once the issue has been resolved, the latest information from the replica devices can be restored back to the production devices. If the production device(s) fail, applications can continue to run on replica devices. A new PIT copy of the replica devices can be created or the latest information from the replica devices can be restored to a new set of production devices. Local Replication

Restore/Restart Considerations
Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved. Restore/Restart Considerations Before a Restore Stop all access to the Source and Target devices Identify target to be used for restore Based on RPO and Data Consistency Perform Restore Before starting production on target Identify Target to be used for restart Create a “Gold” copy of Target As a precaution against further failures Start production on Target Based on the type of failure, choose to either perform a restore to the production devices or shift production operations to the target devices. In either case, the recommendation would be to stop access to the production and target devices, then identify the target to be used for the restore or restart operations. The choice of target depends on the consistency of the data on the target and the desired RPO (e.g., a business may create PIT replicas every 2 hours; if a failure occurs, then at most only 2 hours of data would have been lost). If a target has been written (application testing for example) to after the creation of the PIT, then this target may not be a viable candidate for the restore or restart. Prior to restarting applications using the replica devices, access to the replica devices should be stopped. As a protection against further failures, a “Gold Copy” (another copy of replica device) of the replica device should be created to preserve a copy of data in the event of failure or corruption of the replica devices Local Replication

Restore/Restart Considerations (cont.)
Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved. Restore/Restart Considerations (cont.) Pointer based Full Volume Replicas Restores can be performed to either the original source device or to any other device of like size Restores to the original source could be incremental in nature Restore to a new device would involve a full synchronization Pointer Based virtual Replicas Restores can be performed to the original source or to any other device of like size as long as the original source device is healthy Target only has pointers Pointers to source for data that has not been written to after PIT Pointers to the “save” location for data was written after PIT Thus to perform a restore to an alternate volume the source must be healthy to access data that has not yet been copied over to the target With Full Volume replicas (both full-volume mirrors and pointer-based in Full Copy mode), all the data that was on the source device when the PIT was created is available on the target. Hence, can be used for restore to the original source devices or to a new set of source devices. Restores to the original source devices can be incremental, but restores to a new set of devices are a full-volume copy operation. With Pointer Based virtual replica and pointer based Full Volume replica in deferred mode (COFA), access to all the data on the target is dependent on the health (accessibility) of the original source volumes. If the original source volume is inaccessible for any reason, pointer based virtual or Full Volume (COFA) replicas are of no use in either a restore or a restart scenario. Local Replication

Array Replicas: Which Technology?
Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved. Array Replicas: Which Technology? Factor Full-volume mirroring Pointer-based full-volume replication Pointer-based virtual replication Performance impact on source No impact CoFA mode -some impact Full copy – no impact High impact Size of target At least same as the source Small fraction of the source Accessibility of source for restoration Not required CoFA mode – required Full copy – not required Required Accessibility to target Only after synchronization and detachment from the source Immediately accessible This table summarizes the differences between Full Volume mirroring and Pointer Based full volume, and Pointer based virtual replication technologies. Local Replication

Creating Multiple Replicas
Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved. Creating Multiple Replicas Target Devices 06:00 A.M. Source 12:00 P.M. Point-In-Time 06:00 P.M. Most array based replication technologies allow the Source devices to maintain replication relationships with multiple Targets. This can also reduce RTO because the restore can be a differential restore Each PIT could be used for a different BC activity and also as restore points In this example, a PIT is created every six hours from the same source. If any logical or physical corruption occurs on the Source, the data can be recovered from the latest PIT and at worst, the RPO will be 6 hours. Array local replication technologies also enable the creation of multiple concurrent PIT replicas. In this case, all replicas will contain identical data. One or more of the replicas can be set aside for restore or recovery operations. Decision support activities can be performed using the other replicas. 12:00 A.M. : 12 : 01 : 02 : 03 : 04 : 05 : 06 : 07 : 08 : 09 : 10 : 11 : 12 : 01 : 02 : 03 : 04 : 05 : 06 : 07 : 08 : 09 : 10 : 11 : A.M. P.M. Local Replication

Local Replication Management: Array Based
Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved. Local Replication Management: Array Based Replication management software residing on storage array Provides an interface for easy and reliable replication management Two types of interface: CLI GUI The replication management software residing on the storage array provides an interface for smooth and error-free replication. This management software interface provides options for synchronization, resynchronization, splitting, starting and stopping a replication session, and monitoring replication performance. In general, two types of interface are provided: CLI: An administrator can directly enter commands against a prompt in a CLI. A user can also perform some command operations based on assigned privileges. CLI scripts are developed to perform specific BC operations in a database or application environment. GUI: This type of interface includes the toolbar and menu bar options. A user can select an operation name and monitor performance in real time. Most of the tools have a browser-based GUI to transfer commands and are often integrated with other storage management suite products. Local Replication

Lesson Summary Key points covered in this lesson: Replication technologies Host based LVM based mirroring File system snapshot Array based Full volume mirroring Pointer-based full volume copy Pointer-based virtual replica This concludes the lesson. Key points covered in this lesson are shown here. Please take a moment to review them. Local Replication

Module Summary Key points covered in this module: Definition and possible use of local replicas Consistency considerations when replicating file systems and databases Host based replication LVM based mirroring, File System Snapshot Storage array based replication Full volume mirroring, Pointer based full volume and virtual replication Choice of technology This concludes the module. Key points covered in this module are shown here. Please take a moment to review them. Local Replication

Concept in Practice – EMC Local Replication Solutions
Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved. Concept in Practice – EMC Local Replication Solutions EMC Symmetrix Arrays EMC TimeFinder/Clone Full volume replication EMC TimeFinder/Mirror Full volume mirroring EMC TimeFinder/SNAP Pointer based replication EMC CLARiiON Arrays EMC SnapView Clone EMC SnapView Snapshot EMC has several array-based specialized local replication softwares for different storage arrays. For Symmetrix array, the “EMC TimeFinder” family of products are the solution for full volume and pointer-based local replication. EMC TimeFinder family of product consists of two base solutions and four add-on solutions. The base solutions are TimeFinder/Clone and TimeFinder/Snap. The add-on solutions are TimeFinder/Mirror, TimeFinder/Consistency Groups, TimeFinder/Exchange Integration Module, and TimeFinder/SQL Integration Module. EMC SnapView is a CLARiiON array-based local replication software that creates point-in-time views or point-in-time copies of logical volumes using SnapView snapshots and clones, respectively. Snapshots are pointer-based snaps that require only a fraction of the source disk space. Clones are full-volume copies that require disk space equal to the source. Local Replication

 Check Your Knowledge Describe the uses of a local replica in various business operations. How can consistency be ensured when replicating a database? Discuss one host based replication technology What are the differences among full volume mirroring and pointer based replicas? What is the key difference between full copy mode and deferred mode? What are the considerations when performing restore operations for each array replication technology? Local Replication

Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.

Similar presentations

Presentation on theme: "Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved.

Similar presentations

Presentation on theme: "Copyright © 2009 EMC Corporation. Do not Copy - All Rights Reserved."— Presentation transcript:

Similar presentations

About project

Feedback