Qumulo in AWS: Build a Multi-Instance Cluster with CloudFormation

IN THIS ARTICLE

Outlines how to create a multi-instance cluster configuration using templates provided by CloudFormation in AWS

REQUIREMENTS

• Amazon Web Services (AWS) account
  • AWS account number and target region
  • IAM permissions for full access to EC2 and CloudFormation
  • SSH key-pair for accessing Qumulo instance
    
  • Virtual Private Cloud (VPC) with at least one subnet configured in the target region
    

• Amazon Machine Image (AMI) of Qumulo Core obtained from AWS Marketplace or Qumulo
  • Modifying the type or size of the EBS volumes in the Qumulo AMI will render the software not functional. Please use the block device layout provided in the original AMI.

NOTE: If you are deploying to special AWS regions or zones like AWS Outposts or GovCloud (US), you can proceed to build a multi-instance cluster using the steps below. There is no difference in deployment. For more info, check out the details provided in the Qumulo in AWS: Getting Started article on Qumulo Care. 

IAM PERMISSIONS

The table below lists the required IAM permissions for creating a Qumulo cloud cluster in AWS with CloudFormation.

If you are using EBS encryption, the following permissions are also required:

IMPORTANT! The s3:GetObject IAM permission will also be required if you are using a CloudFormation Template hosted in S3.

DETAILS

Deploying Qumulo in AWS involves creating unique EC2 instances backed by EBS storage and clustering them together into a distributed file system. Generally, the number of instances in your cluster relates to performance where the higher instance counts equate to higher throughput and IOPS. Before creating your Qumulo in AWS cluster, consider the amount of storage and number of instances that you need.

CloudFormation templates are especially helpful in that they can be customized to meet any size or type requirements and can be deployed in any region.

Build a Qumulo Cloud Cluster with CloudFormation

1. Log in to your AWS account and click the Services drop-down.
2. Click the CloudFormation link under the Management Tools section.
   

   

3. Click Create new stack to open the Create Stack page.
   

   

4. Depending on how your template has been provided, follow the appropriate step below:
   • If you have a URL for the template file, select Amazon S3 URL and enter the URL that was provided when you purchased your template.
     

     

   • If you have a downloaded template file, select Upload a template file and then Choose File to select your template file.
     

     

5. Click Next to specify details regarding your new stack.
6. Enter the required information for the fields displayed (stack name, network configuration, etc.). 
   

   

   Note: If you do not already have an SSH key pair, you can obtain one by clicking the Services menu and selecting Key Management Service. 
7. Enter the ARN of your own SNS topic into the Instance recovery alarm SNS topic text box (available in 3.2.0 or above) in the SNS configuration section if you would like to be notified when your instances stop or recover from a failed status check.
8. When finished, click Next.
9. Review the configuration details you have provided. If they are correct, click Create stack.

You may need to refresh the page in order to see progress updates. When it has finished, the Status column will display CREATE_COMPLETE.

Before proceeding, make sure that the creation process completes for the entire cluster, not just the individual nodes. In the screenshot above, you can see the process complete for four nodes, and then the cluster completion (sm-test) at the very top.

IMPORTANT! It is not recommended to run a Qumulo cluster in AWS for production data without a replication or backup strategy.

Launch Your New Qumulo Cloud Cluster

1. After the creation has finished, click the Outputs tab to locate your instance's details.
   

   

2. Click IP address link provided in the LinkToManagement field to open it in a new browser tab.
   
   • Note: Qumulo takes your security seriously. As a result, the template only creates a private IP address for each of the instances. To complete cluster creation you will need to access the instance in one of two ways: 
     • Ensure the browser you are using is running on a machine with access to the VPC, or
     • Update the firewall of the VPC to allow access to the instance using a public IP address. This is most commonly accomplished using the AWS Internet Gateway Service.
3. Enter admin for the user name.
4. Go back to the AWS browser tab and copy the value for TemporaryPassword from the table in the Outputs pane.
   
5. Enter this value for your password in the Qumulo instance.
   
6. Click Login to view your new Qumulo cluster.
7. Change your password after logging in for security purposes.

RESOLUTION

You should now be able to successfully create a multi-instance cloud cluster for Qumulo in AWS

ADDITIONAL RESOURCES

Qumulo in AWS: Getting Started

Qumulo in AWS: Add a Node to an Existing Cloud Cluster

QQ CLI: Cluster Configuration

Like what you see? Share this article with your network!