Qumulo in AWS: Deploying and Upgrading Qumulo Sidecar for Your Qumulo Cloud Cluster in AWS

REQUIREMENTS

A cloud cluster with Qumulo Core 3.1.1 (or higher)
Permission to create users and roles on your Qumulo cluster
AWS Console access to the following services:
- CloudWatch Events: Invokes scripts at a regular interval
- CloudWatch Metrics: Stores, and permits viewing of, cluster metrics
- EC2 and EBS: Replaces EBS volumes
- IAM: Gives scripts permissions to access AWS resources
- Lambda: Runs scripts
- Secrets Manager: Securely stores information and credentials about the cluster and input parameters
Full IAM permissions for:
- CloudFormation
- EC2
- Lambda
- Secrets Manager

IAM PERMISSIONS

Deploying Qumulo Sidecar requires the following IAM permissions.

cloudformation:CreateStack
cloudformation:DeleteStack
ec2:DescribeNetworkInterfaces
ec2:DescribeSecurityGroups
ec2:DescribeSubnets
ec2:DescribeVpcs
events:DeleteRule
events:DescribeRule
events:PutRule
events:PutTargets
events:RemoveTargets
iam:AttachRolePolicy
iam:CreateRole
iam:DeleteRole
iam:DeleteRolePolicy
iam:DetachRolePolicy
iam:GetRole
iam:GetRolePolicy
iam:PassRole
iam:PutRolePolicy
lambda:AddPermission
lambda:GetFunction
lambda:CreateFunction
lambda:DeleteFunction
lambda:DeleteFunctionEventInvokeConfig
lambda:GetFunctionConfiguration
lambda:RemovePermission
lambda:PutFunctionEventInvokeConfig
lambda:PutFunctionConcurrency
s3:GetObject
secretsmanager:CreateSecret
secretsmanager:DeleteSecret
secretsmanager:TagResource

Sending cluster metrics to AWS CloudWatch by using Qumulo Sidecar requires the following permissions.

cloudwatch:PutMetricData
secretsmanager:GetSecretValue
sns:Publish

Detecting and repairing EBS volume failures by using Qumulo Sidecar requires the following permissions:

ec2:AttachVolume
ec2:CreateTags
ec2:CreateVolume
ec2:DescribeImages
ec2:DescribeInstances
ec2:DescribeVolumes
ec2:DetachVolume
ec2:ModifyInstanceAttribute
sns:Publish

DETAILS

Qumulo Sidecar is a tool that can deploy AWS services useful for monitoring and maintaining Qumulo cloud clusters in AWS. The tool operates as an always-active service alongside you cluster. You can activate Sidecar when your AWS cluster is operational and use it for the following operations.

Send Cluster Metrics to AWS CloudWatch: Sidecar deploys an AWS Lambda Function that collects cluster metrics every minute and then sends the metrics to AWS CloudWatch. For more information about these metrics and how to find them in the CloudWatch console, see Qumulo in AWS: Monitoring with a CloudWatch Dashboard.
Detect and Repair EBS Volume Failures: Sidecar deploys an AWS Lambda Function that polls your Qumulo cluster for disk failures every 10 minutes. When Sidecar detects a disk failure, the Lambda function replaces the affected EBS volume automatically. For more information, see Qumulo in AWS: Automatic EBS Volume Replacement.

Step 1: Create a Local Sidecar User Account

Before you activate Qumulo Sidecar, you must configure Qumulo Core.

Log in to the Qumulo Core Web UI.
Click Cluster > Local Users & Groups.
On the Users page, click Create.
In the Create user dialog box, on the Basic info tab, do the following:
1. Enter a User name, for example SidecarUser.
2. Leave the NFS UID blank.
3. Enter a Password.
On the Groups tab, click Primary for Guests and leave all other boxes unchecked.
Click Create.

Step 2: Configure a Custom Sidecar Role and Assign Your Local Sidecar User Account to It

Log in to the Qumulo Core Web UI.
Click Cluster > Role Management.
On the Role Management page, click Create Role.
On the Create Role page , do the following:
1. Enter a Name.
2. Enter a Description.
3. Click the following Privileges:
  - ANALYTICS_READ
  - CLUSTER_READ
  - FS_ATTRIBUTES_READ
  - NETWORK_READ
4. Click Save.
On the Role Management Page, under your new role, click Add Member.
In the Add Member to <Role> dialog box, for Trustee, enter the local Sidecar username that you have created.
Click Yes, Add Member.

Step 3: Deploy Qumulo Sidecar

In the release notes for the Qumulo Core version on your cluster, click the Qumulo Sidecar link.
The CloudFormation console opens with the JSON configuration file for Qumulo Sidecar.
Enter the details for your cluster in AWS.
- For the Login information section, enter your local Sidecar user account.
- For more information about configuring the Failure monitoring section, see Setting up Amazon SNS notifications in the Amazon CloudWatch User Guide.
Click Create Stack.

Step 4: Upgrade Qumulo Sidecar

We strongly recommend using the version of Qumulo Sidecar that matches the version of Qumulo Core on your cluster. For more information about upgrading CloudFormation templates, see Updating stacks directly in the AWS CloudFormation User Guide.

Log in to the CloudFormation console
On the Stacks page, click the Qumulo Sidecar stack to upgrade and then click Update.
In the Update stack dialog box, do the following:
1. Click Replace current template.
2. For Amazon S3 URL, enter the Qumulo Sidecar Upgrade link from the release notes for the Qumulo Core version on your cluster.
3. Click Next.
Review your stack configuration and then click Update stack.