IN THIS ARTICLE
Outlines how to utilize Continuous Replication within Qumulo Core
- Replication Page Overview
- Create a Replication Relationship
- Authorize a Replication Relationship
- Replication In Progress 2.7.10 and above
- Edit or Delete a Replication Relationship
- Additional Considerations
- Admin privileges required
- Source and target cluster running the same version of Qumulo Core 2.7.3 and above
We highly recommend upgrading to the latest version of Qumulo Core to fully utilize Replication with the additional improvements in functionality and performance we've implemented since the original release.
Note: You can replicate between any models of Qumulo hardware, including HPE.
Continuous Replication creates a copy of the data in a directory on your primary cluster and transfers it to a directory on a second target cluster. Continuous Replication is always running unless the relationship is configured to enter a blackout window during certain hours of the day(s) of the week.
While Continuous Replication is running, Qumulo Core will take your latest changes and replicate them without the worry of scheduling them into an existing job. With the help of snapshots, Continuous Replication generates a point-in-time consistent copy of the source directory on the target cluster.
There are some situations such as periods of peak cluster use where you may want to temporarily disable Replication. The Continuous Replication UI provides the ability to create one or more blackout windows where Replication will suspend operations until the blackout window expires, at which time Replication will automatically resume operations.
With Qumulo Core 2.9.2 and above, incremental replication jobs will no longer resend the entire file contents when only part of a file has changed. The replication engine will scan modified files for any changed regions and only send those changes over to the target so that job runtime is proportional to the amount of data changes made.
With this change, the throughput statistics in the Replication Job Details UI only include data actually transferred. Unmodified data that does not need to be transferred is reported as “Unchanged”. Note that the initial replication job will still send the entire file and the first job after upgrading to this release may fall back to resending the entire file if the relationship hasn’t completed a job since being upgraded to 2.8.8 or later.
Replication Page Overview
In the Web UI, navigate to the Cluster menu on the Source Cluster and select Replication under the Configurations section in the dropdown.
A list of Replication relationships will display with the following details:
Status (#): represents the current status of each relationship including Replicating, Blackout Window, Awaiting Authorization, etc. and displays the number of relationships
Source: outlines the file path of the source directory
Target: outlines the file path of the target directory
Role: indicates whether the cluster is serving in the role of a source or target directory for that particular relationship
Last Attempt: states whether the last Replication attempt completed or failed
Recovery Point: displays a timestamp from the last successful Replication on the source cluster and represents the time Qumulo Core took a snapshot of the source directory and started moving data across to the target
Continuous Replication: displays whether the relationship is enabled or disabled
Create a Relationship for Replication
- Click the Create Relationship button on the Replication page of the Source cluster
- Fill in the following fields:
- Source Directory Path: path of the existing directory you wish to replicate
- Target Directory Path: path of existing empty directory on the target cluster where you wish to replicate data
- Target Address: one of the IP (preferably floating IP) addresses from a node on the target cluster
- Port Number: select the default port option 3712 or create a custom port configuration
NOTE: Custom port configuration may be necessary to meet enterprise network firewall rules. To override, select the Custom radio button and enter the port number. For questions on custom port configuration, reach out to Qumulo Care for additional information.
- Verify that Continuous Replication Enabled is On
- Toggling it to Off suspends Replication for this relationship
- Select the day(s) or times for Blackout Windows to suspend Replication between the source and target directories
- To add additional blackout windows (up to 10), click the +Add Blackout Window button
- Click Save Relationship
The relationship will now be available on the Replication page where you can see the status and manage the Replication job. A Replication job will not start until the relationship has been authorized on the Target cluster. See additional details below for Authorizing a Replication Relationship.
NOTE: If you set up a blackout window, any existing Replication job will continue to run until completion. To stop the job immediately, click Manually abort keeping in mind that any progress since the last recovery point will be lost.
Authorize a Replication Relationship
- Log on to the Target cluster
- Click See Details on the authorization request notification banner
- Click Accept on the Replication page to authorize the relationship
Note that Replication will not start if you are currently in a blackout window or if Replication for that relationship is disabled.
Replication In Progress 2.7.10 and above
While a Replication job is running, a progress bar will display on the Replication page under the status column. Note that the percentage reflects both the number of files to be moved as well as the amount of data to be transferred and is not based solely on the amount of data being replicated.
Click on the Details option to review the Throughput, Run Time, and Data statistics of the current job in progress. Files that have not changed since the previous Replication job will be reflected in the Unchanged column and will not be transferred.
NOTE: Qumulo Core versions 2.7.9 and below will display "Replicating" in the status column until the Replication job completes. The Details page and Progress bar are not available in these previous releases.
Edit or Delete a Replication Relationship
- To Edit an existing Replication relationship, click the pencil icon next to the job on the Source cluster
Note that you cannot edit the source and target directory paths. To make these changes, you must create a new relationship.
- Delete a relationship by clicking the trashcan icon on either the Source or Target cluster and confirm by selecting Yes, Delete when prompted
- Upgrading Qumulo Core while a Replication job is running will cause the job in progress to fail. Once both clusters return to a state where Replication is possible (same version), Continuous Replication can resume. Please wait until after the Replication job completes to perform any upgrades.
- Utilizing Continuous Replication will increase load on the cluster and may cause latency delays depending on the applications in use while a job is running.
- At the beginning of a Replication job, a snapshot is created for the directory. The old snapshot is programmatically deleted when it is no longer needed. There will be one reference snapshot used for Replication on the cluster at all times. Admins can view the snapshot(s) used for replication along with other policy-based snapshots.
- A snapshot is taken of the directory on the target cluster at the conclusion of each successful job. The previous snapshot is then deleted so the directory always contains only the most recent snapshot.
- Target directory permissions will be updated from read/write to read-only when a Replication relationship is created. If the relationship is deleted, the permissions will revert to read/write automatically.
- A Replication relationship that includes the root directory of the target cluster is not permitted. Including the root directory makes the entire target cluster read-only and may prohibit administrative functions such as upgrades.
- Qumulo Core supports up to 100 Replication relationships.
You should now be able to successfully configure and manage Continuous Replication in Qumulo Core
Like what you see? Share this article with your network!