Print Email PDF

Qumulo in GCP: Configure IP Failover

IN THIS ARTICLE 

Outlines how to configure IP failover with Qumulo in GCP

REQUIREMENTS

  • Administrative access to a Qumulo cluster in Google Cloud
  • Administrative permissions in Google Cloud to reserve static IP addresses and to manage Qumulo's compute instances
  • The subnet in which Qumulo's compute instances reside must allow access to Google Cloud APIs. For information about how to configure this access, see Google Cloud Documentation on Configuring Private Google Access

NOTE: Modifying the type or size of the volumes in the Google Marketplace listing will render the software non-functional. Please use the default volume configuration available.

DETAILS

Qumulo Core supports IP failover in GCP using the same floating IP functionality that currently exists with Qumulo's on-prem clusters. Reserving a static internal IP in a VPC prevents new resources from being assigned the address and allows Qumulo to move the address between nodes when necessary.

To determine the number of floating IP addresses to use based on your cluster size, we recommend following the same guidelines outlined in the IP failover with Qumulo Core article. Keep in mind that only IPv4 addresses are supported and that the number of addresses per node cannot exceed 10 (per Google Cloud support).

Reserve IP Addresses via the GCP Console

  1. Log into the GCP Console.
  2. Use the Navigation menu in the left hand panel to select Networking, choose VPC Network, and click VPC networks.

    vpc_networking_close_up.png

  3. Select the name of the VPC network your instances will run on.

    vpc_network_list.png

    NOTE: You must navigate to the VPC Network, not the subnet. If you are using a shared VPC Network, it may be located within a different project.
  4. Click on the Static internal IP addresses tab for the selected network on the VPC network details page.
  5. Click Reserve static address to configure as many additional IPs (assigned automatically or manually) that you desire for that cluster. 

    vpc_network_static_ip_addresses.png

  6. Fill out the reservation form:
    • Provide a Name for the reservation and optional Description
    • The Subnet must match the subnet in which the Qumulo instances reside
    • The Static IP address can be assigned automatically or chosen manually
    • The Purpose must be Non-shared

      reserve_static_IP.png

  7. Click Reserve.

Repeat these steps until the number of addresses needed are reserved. Once Qumulo Core assigns the IP to a particular node in your cluster, the name of the instance will be displayed in the “In use by” column associated with that address.

Reserve IP Addresses via the gcloud CLI Tool

Addresses can also be reserved using the following command:

 $ gcloud compute addresses create \
address-name-1 address-name-2 \
--addresses 10.0.0.1,10.0.0.2 \
--subnet <subnet name>
--purpose GCE_ENDPOINT

All IP addresses included in --addresses must be within the subnet’s IP range. If none are specified, new addresses will be generated automatically and server-generated names will be assigned to the IP addresses.

Whether reserving addresses through the GCP console or using the gcloud CLI tool, manually specifying IP addresses requires that the addresses must be in the same subnet as the instances’ primary IPs.

NOTE: This will attempt to assign IPs in the configured project for the gcloud CLI. If you wish to create static IPs in a different project, you must change your project via gcloud init or use the Google Web UI.

Create and Assign a Service Account to the Instances

Before configuring IP failover, each Qumulo node needs to have permission to reassign static addresses. Each node must be granted the "Compute Instance Admin" (v1) role in the same project as the Qumulo instances. Roles are granted to instances with service accounts. For more information on creating a new service account with a specific set of roles, check out the following resources:

Obtain a service account that has been granted the "Compute Instance Admin" (v1) role. Note the name of this service account. Ensure all instances are in a stopped state and run the following command for each instance:

$ gcloud compute instances set-service-account <instance-name> \ --service-account <service-account-name> --scopes \ https://www.googleapis.com/auth/cloud-platform

Alternatively, you can edit each stopped instance by selecting the Service account option at the bottom of the Edit page.

A service account is usually available within 60 seconds after the instance is started, but sometimes can take up to 7 minutes to become available. Service accounts typically reflect an account-name@project-name.iam.gserviceaccount.com pattern as shown in the example below.

EXAMPLE: test-service-account@testproject-12345.iam.gserviceaccount.com

Assign Secondary IPs as Floating IPs in Qumulo Core

  1. Start the instances.
  2. Open a terminal window and SSH into your Qumulo cluster using the admin account.
  3. Assign the previously-reserved floating IPs to the Qumulo cluster using the following command. Append all of the reserved floating IP addresses to this command, separating individual IPs with a space:
    qq network_mod_network --network-id 1 --floating-ip-ranges <address-or-range> [<address-or-range> …]
    Using some example IP addresses, a complete command to assign four floating IP addresses would look like this:
    qq network_mod_network --network-id 1 --floating-ip-ranges 10.81.1.103 10.81.1.109 10.81.1.221 10.81.1.222
  4. After a short wait, check that the floating IPs have been assigned to nodes by running the following command:
    qq network_poll

Each node's DHCP network should have at least one floating IP assigned to it as outlined in the example below:

{
"interface_details": { ... },
"interface_status": { ... },
"gcp_status": {
"ip_aliases": [ "10.81.1.103/32" ],
"network": "projects/12341234/networks/yourvpc",
"private_ipv4_address": "10.81.5.64",
"subnet_mask": "255.255.0.0"
},
"network_statuses": [
{
"address": "10.81.5.64",
"assigned_by": "DHCP",
"dns_search_domains": [ ... ],
"dns_servers": [ "10.81.13.17" ],
"floating_addresses": [ "10.81.1.103" ],
"mtu": 1460,
"name": "Default",
"netmask": "255.255.240.0",
"vlan_id": 0
}
],
"node_id": 1,
"node_name": "...",
"update_status": "CHANGES_APPLIED"
}

The "ip_aliases" field under "gcp_status" indicates Google Cloud's perspective of the floating IPs assigned to each node. The "floating_addresses" field under "network_statuses'' indicates the floating IPs assigned to each node from Qumulo's perspective. On a given node, when IP failover is properly configured, the "ip_aliases" and "floating_addresses" fields should contain the same list of addresses (note that addresses will appear in CIDR format in the "ip_aliases" field).

Qumulo Core will distribute the floating addresses between the nodes and will manage the assignment of the floating address to each node for you. Keep in mind that if you un-assign one of the floating IPs from an instance, Qumulo Core will attempt to reassign it. Make sure to remove the floating IP from the Qumulo Core networking configuration in order to unassign the secondary static internal IP address from an instance.

You should now be able to mount the cluster via the floating IPs. If the node currently assigned a floating IP fails, the floating IP will be transferred to another node in the cluster and client operations will continue uninterrupted.

NOTE: Confirm that your clients are using the new floating IPs. If a client already has a mount point based on a previous static IP, you will need to unmount and remount the cluster, ensuring that the new mount picks up the floating IP set. The DNS record must be updated to list the floating IPs and not the static ones for the cluster before remounting.

RESOLUTION 

You should now be able to successfully configure IP failover with Qumulo in GCP

ADDITIONAL RESOURCES

Qumulo in GCP: Deploy a Cloud Cluster

IP failover with Qumulo Core

Set Explicit DNS Hostnames and IP Mappings

QQ CLI: Cluster Configuration

 

Like what you see? Share this article with your network!

Was this article helpful?
0 out of 0 found this helpful

Comments

0 comments

Please sign in to leave a comment.

Have more questions?
Open a Case
Share it, if you like it.