How does SnapMirror Sync work ?

Guys, believe me it is really an interesting process and to understand SnapMirror Sync internals, let me take you through first the typical application environment (see below)

In most cases, these steps take only tiny fractions of a second to complete. If the storage system fails in such a way that all data on it is lost (such as a fire or flood that destroys all of the storage media), the impact to an individual transaction will vary based on when the failure occurs.

If the failure occurs before step 5, the storage system will not acknowledge receipt of the data. This results in the user receiving an error message from the application, indicating it failed to save the transaction.

If the failure occurs after step 5, the user sees client behavior that indicates correct operation (at least until the following transaction is attempted). Despite the indication by the client software (in step 8) that the write was successful, the data is lost.

The first case is obviously preferable to the second, because it provides the user or application with the knowledge of the failure and the opportunity to preserve the data until the transaction can be attempted again. In the second case, the data might be discarded based on the belief that the data is safely stored.

How SnapMirror Sync Works

After the baseline transfer is complete, SnapMirror can transition into synchronous mode:

1. A first asynchronous SnapMirror update occurs (Base snapshot creation and transfer to secondary filer)

2. Consistency point (CP) forwarding begins. This is a method to make sure that writes of data from memory to disk storage are transferred from the primary system to the secondary system. For more information, see the CP forwarding section later in this guide. A second asynchronous SnapMirror update is started.

3. After the second asynchronous SnapMirror update completes, NVLOG forwarding begins. This is a method to transfer updates as they occur. For more information, see the NVLOG forwarding section later in this guide.

New writes from clients or hosts to the primary file system begin to block until acknowledgment of those writes has been received from the secondary system. 

After SnapMirror has determined that all data acknowledged by the primary system has been safely stored on the secondary, the system is in synchronous mode. At this point the output of a SnapMirror status query will display the relationship as “In-Sync.”

Understanding NVLOG Forwarding

NVLOG forwarding is a critical component of how synchronous mode works. It is the method used to take write operations submitted from clients against the primary file systems to be replicated to the destination.

In order to completely understand NVLOG forwarding, a basic knowledge of how Data ONTAP performs local file system writes is required.

The RAID software writes the data blocks out to disk. When it is complete, it returns an acknowledgment to the WAFL software, and Data ONTAP considers the CP complete.

When NVLOG forwarding is active in synchronous mode, above steps 2 and 3 are modified as follows:
 

2. The request is journaled in NVRAM. It is also recorded in the cache memory and transferred over the network to the SnapMirror destination system, where it is journaled in NVRAM.
 

3. After the request is safely stored in NVRAM and cache memory on the primary system and in the NVRAM of the secondary system, Data ONTAP acknowledges the write to the client system. The application that requested the write request can continue to process user requests.
 

Therefore, to synchronously protect data, NVLOG forwarding is the primary mechanism.
 

It is important to know how the NVLOG data is stored on the secondary storage system. Since this system has its own storage as well as the mirrored data from the primary system (at the very least, every system has its own root volume), and because CPs need to be kept synchronized on any mirrored volumes, the NVLOG data cannot be stored in NVRAM on the secondary system in the same way as normal file system writes. Instead, the NVLOG data is treated as a stream of writes to a set of special files (replication log files) in the aggregate of the replicated volume on the secondary system. The write requests being sent to these files are logged in the secondary system’s NVRAM just like any other write request.

Understanding CP Forwarding

While NVLOG forwarding protects the data by synchronously replicating it from primary to secondary storage, it functions independently from file system writes to disk on each storage system. CP forwarding is the underlying mechanism used to make sure that the actual on-disk file system images, independent of NVRAM or NVLOG forwarding, are kept synchronized. CP forwarding (sometimes referred to as CP sync) modifies step 5 earlier (above diagram) as follows:
 

5. When a CP is triggered, Data ONTAP uses the transaction data in the cache memory to build a list of data block changes that need to be written to disk. It also computes parity information at this time. After the required disk modifications are ready, the local RAID software writes the data blocks to the disk, and the data blocks are also sent across the network to the secondary system, which initiates its own write to disk.
 

The secondary storage also writes the data blocks out to disk. When each system completes this process, it returns an acknowledgment to the WAFL software on the primary storage system, and Data ONTAP considers the CP complete.

The summary of high-level steps and a diagram depicting both NVLOG and CP forwarding are as follows:
 

1. The storage system receives a write request. This request could be a file-oriented request from an NFS, CIFS, or DAFS client, or it could be a block-oriented request using FCP or iSCSI.
 

2. The request is journaled in NVRAM. It is also recorded in cache memory and forwarded over the network to the SnapMirror destination system, where it is journaled in NVRAM.
 

3. After the request is safely stored in NVRAM and cache memory on the primary system and in NVRAM of the secondary systems, Data ONTAP acknowledges the write to the client system, and the application that requested the write is free to continue processing.
 

4. Under certain conditions, a consistency point (CP) is triggered. Typically this occurs when the NVRAM journal is one-half full, or when 10 seconds have passed since the most recent CP, whichever comes first.

When a CP is triggered, Data ONTAP uses the transaction data in cache memory to build a list of data block changes that need to be written to disk. It also computes parity information at this time. After the required disk modifications are ready, the local RAID software writes the data blocks to the disk, and the data blocks are also sent across the network to the secondary system, which initiates its own write to disk.

The secondary storage also writes the data blocks out to disk. When each system completes this process, it returns an acknowledgment to the WAFL software on the primary storage system, and Data ONTAP considers the CP complete.

Guys, don't forget to leave your comments ..!