Print Email PDF

Qumulo in AWS: Qumulo Sidecar

IN THIS ARTICLE

Outlines how to use and deploy the Qumulo Sidecar for Qumulo cloud clusters in AWS

REQUIREMENTS

  • Cloud Cluster with Qumulo Core 3.1.1 or above
  • Privileges to create users and roles in your Qumulo cluster 
  • AWS Console access to use the following services:
    • Secrets Manager—Securely stores information and credentials about the cluster/input params
    • CloudWatch Metrics—Stores and allows viewing of cluster metrics
    • CloudWatch Events—Invokes the scripts at a regular interval
    • IAM—Gives the scripts permissions to access AWS resources
    • Lambda—Runs the scripts
    • EC2/EBS—Replaces EBS volumes
  • IAM permissions for full access to EC2, CloudFormation, Lambda, and Secrets Manager

IAM PERMISSIONS

The table below lists the required IAM permissions for deploying the Qumulo Sidecar with Qumulo cloud clusters in AWS.

cloudformation:CreateStack cloudformation:DeleteStack ec2:DescribeNetworkInterfaces
ec2:DescribeSecurityGroups
ec2:DescribeSubnets ec2:DescribeVpcs
events:DeleteRule events:DescribeRule events:PutRule
events:PutTargets events:RemoveTargets iam:AttachRolePolicy
iam:CreateRole iam:DeleteRole iam:DeleteRolePolicy
iam:DetachRolePolicy iam:GetRole iam:GetRolePolicy
iam:PassRole iam:PutRolePolicy lambda:AddPermission 
lambda:GetFunction lambda:CreateFunction lambda:DeleteFunction 
lambda:DeleteFunctionEventInvokeConfig lambda:GetFunctionConfiguration lambda:RemovePermission
lambda:PutFunctionEventInvokeConfig lambda:PutFunctionConcurrency s3:GetObject
secretsmanager:CreateSecret secretsmanager:DeleteSecret secretsmanager:TagResource

Sending cluster metrics to AWS CloudWatch with Qumulo Sidecar requires the following permissions:

cloudwatch:PutMetricData

secretsmanager:GetSecretValue sns:Publish

Detecting and repairing EBS volume failures with Qumulo Sidecar requires the following permissions:

ec2:AttachVolume

ec2:CreateTags ec2:CreateVolume

ec2:DescribeImages

ec2:DescribeInstances

ec2:DescribeVolumes
 ec2:DetachVolume

ec2:ModifyInstanceAttribute

 sns:Publish

DETAILS

The Qumulo Sidecar is a Qumulo tool that can deploy AWS services that are useful in monitoring and maintaining a Qumulo cloud cluster in AWS. The tool operates as an always-active service alongside the cluster and can be activated once your AWS cluster is up and running in order to perform the following:

Send Cluster Metrics to AWS CloudWatch

The Sidecar deploys an AWS Lambda Function that collects cluster metrics once every minute and then sends them to AWS CloudWatch. For more information on these metrics and how to find them in the CloudWatch console, check out the Qumulo in AWS: Monitoring with a CloudWatch Dashboard article.

Detect and Repair EBS Volume Failures

The Sidecar deploys an AWS Lambda Function that polls the Qumulo cluster for disk failures every 10 minutes. Once a disk failure is detected, the lambda automatically replaces the corresponding EBS volume. Check out the Qumulo in AWS: Automatic EBS Volume Replacement article for more details.

Create a Sidecar Local User Account

Several configurations must be made in the Qumulo Core Web UI prior to activating Qumulo Sidecar, including creating a user account and role for it to access Qumulo data.

  1. Login to the Qumulo Core Web UI.
  2. Hover over the Cluster menu and select Local Users & Groups.

    users_groups.png

  3. Click Create to create a new user.

    user_create.png

  4. Specify a name and password for the user. Leave the NFS UID field blank.
    NOTE: The user name can be anything; the examples below use 'SidecarUser'.

    sidecaruser_create.png

  5. Click the Groups tab and select the check box in the Primary column in the Guests group row. Leave all other fields unchecked.

    sidecar_groups.png

  6. Click Create to save the user account.

Now that the Sidecar has a local account, you can configure the appropriate permissions for it to access Qumulo data.

Configure a Sidecar Custom Role

  1. Login to the Qumulo Core Web UI.
  2. Hover over the Cluster menu and click Role Management.

    cluster_role_management.png

  3. Click Create Role.

    sidecar_custom_role.png

  4. Enter a name for the role and a description if desired.

    sidecar_role_name.png

  5. Select the following Privileges:
    • ANALYTICS_READ
    • CLUSTER_READ
    • FS_ATTRIBUTES_READ
    • NETWORK_READ
  6. Leave all other boxes unchecked and c lick Save.
  7. Click Add Member under the new role you created on the Role Management page.

    sidecar_assign_role.png

  8. Enter the name of the local user you created in the previous section.

    sidecar_user_assigned.png

  9. Click Yes, Add Member to assign the custom role.

Deploy Qumulo Sidecar

  1. Click the Qumulo Sidecar link provided in the product release notes for the version of Qumulo Core your cluster is running. A browser window will open to configure your Sidecar.
    NOTE: You can find Product Release Notes for all versions of Qumulo Core in the Product Releases section of Qumulo Care.
  2. Fill out the Stack details form with the information for your AWS cluster, including the details for the local user account you created for the Sidecar service under the Login Information section.
  3. Enter the details for the user account you created for the Sidecar service in Create a Sidecar Local User Account earlier in this document under the Login Information section.
    NOTE: See Setting Up Amazon SNS Notifications for details on creating SNS notifications.


    QSidecar_Stack.png

  4. Click Create Stack to deploy the Qumulo Sidecar.

Upgrade Qumulo Sidecar

In order to make sure you have access to the latest Qumulo Sidecar features, we recommend you use the Sidecar version that matches your cluster's version of Qumulo Core. For full details on upgrading CloudFormation templates, refer to Updating Stacks Directly.

  1. Select the Qumulo Sidecar stack you wish to upgrade from the CloudFormation Stacks list.
  2. Click Update.

    sidecar_update.png

  3. Select Replace current template in the Update stack box.
  4. Copy the Qumulo Sidecar Upgrade link provided in the product release notes for the version of Qumulo Core your cluster is running.
  5. Paste the link in the Amazon S3 URL field and click Next.
  6. Review the existing stack configuration details and make any changes needed (if any).
  7. Click Update stack.

Your Qumulo Sidecar stack is successfully upgraded when its Status lists UPDATE_COMPLETE.

RESOLUTION

You should now be able to successfully deploy the Qumulo Sidecar alongside your Qumulo cloud cluster in AWS

ADDITIONAL RESOURCES

Qumulo in AWS: Configure CloudWatch Alarms

Qumulo in AWS: Automatic EBS Volume Replacement

Role-Based Access Control (RBAC) with Qumulo Core

What is AWS Lambda?

Setting Up Amazon SNS Notifications

Updating Stacks Directly

 

Like what you see? Share this article with your network!

Was this article helpful?
0 out of 1 found this helpful

Comments

0 comments

Please sign in to leave a comment.

Have more questions?
Open a Case
Share it, if you like it.