Qumulo in AWS: Configure CloudWatch Alarms

REQUIREMENTS

AWS Cloud Cluster with Qumulo Core 3.1.0 or above
AWS Console access
SSH key-pair for accessing Qumulo instance
IAM permissions for full access to EC2
Command line (qq CLI) tools installed via API & Tools in the Web UI

IAM PERMISSIONS

The cloudwatch:PutMetricAlarm IAM permission is required to configure CloudWatch alarms for your Qumulo cloud cluster in AWS.

DETAILS

CloudWatch alarms allow you to use the Amazon Simple Notification Service (SNS) to receive notifications and trigger actions when AWS metrics indicate a cluster health issue. Any time the alarm is activated or recovers, you can trigger an SNS notification to be sent via the Amazon Resource Name (ARN) of the SNS topic to keep you in the loop regarding the status of your AWS cloud cluster.

When configuring this process from a clean install, you need to perform four major steps:

Create an SNS topic.
Add contacts for the SNS.
Add the topic to existing CloudWatch alarms.
Create an alert when Automated Instance Recovery is triggered.

Create an SNS Topic

NOTE: Skip this section if you already have an SNS topic that you created when you created the alarms.

Select Topics in the left-hand navigation pane on your AWS S3 SNS Dashboard.
Click Create Topic.
Enter a name for the topic in the Name field. If you're configuring an alert to trigger an SMS notification, provide a name in the Display Name field. The first 10 characters of this name will appear in your SMS.
Use the optional Tags section to add Tags as needed to help others in your organization understand the ownership and purpose of the SNS topic.
Leave the Encryption, Access policy, Delivery retry policy, and Delivery status logging sections unchanged.
Click Create Topic.

You'll now see the details of your new topic. Note the ARN of the topic for use later in this article and remain on this page for the next section.

Add Contact Details to your SNS Topic

Now that your topic is created in the AWS system, you’ll need to subscribe contacts for notifications. To add a new contact subscription:

Click Create Subscription from your topic's details.
Leave the Topic ARN field unchanged.
Select the type of subscription (SMS, email, etc.) in the Protocol field.
Specify the contact details for the subscription in the Endpoint field. For example, provide a phone number if configuring an SMS subscription. Note that the Endpoint field will be hidden until a Protocol is selected.
Leave the remaining fields unchanged and click Create Subscription.
NOTE: For email or SMS subscriptions, you will be sent a link to confirm.

Repeat these steps for each additional contact that needs to be subscribed. You now have an SNS topic that you can use for the CloudWatch Alarm and CloudWatch Events that will send notifications.

Add SNS Topic to CloudWatch Alarms

Click Alarms in the left-hand navigation pane on your AWS CloudWatch Dashboard.
Click the Alarm Name link to view details on an alarm that corresponds to one of your Qumulo nodes.
Click Edit to modify the alarm.
Leave everything on the first page unchanged and click Next to configure actions for the alarm.
Click Add notification to create a new notification for the alarm.
Select the existing SNS topic you created previously using the Send a notification to... dropdown menu.
Leave the remaining areas unchanged and click Update alarm at the bottom of the page to save the changes.

Repeat the steps above for every alarm that corresponds to a Qumulo node.

Create a Metric-Based Alarm

NOTE: Qumulo Metrics are only captured once the Qumulo Sidecar service is running. For more details see Qumulo in AWS: Qumulo Sidecar.

These steps create an alarm that triggers based on the value of a specific metric. While any metric can be used, the example below triggers an alarm when the FailedDriveCount metric is greater than 0. The alarm then sends an email to a specified recipient.

TIP! For additional details, see Create a CloudWatch Alarm Based on a Static Threshold.

Click Alarms in the left-hand navigation pane on your AWS CloudWatch Dashboard.
Click Create alarm.
Click Select metric.
Click Qumulo/Metrics under Custom Namespaces.
Click ClusterName to view metrics that are cluster-wide (e.g., total space or number of nodes).
Enter the name of the metric in the search box and click Graph search to display all clusters with the specified metric.
Check the box alongside the cluster you wish to configure the alarm and click Select metric.
Scroll to the Conditions section and specify the conditions of your alarm. Click Next to continue.

NOTE: For our example, the alarm will trigger if the Static value of FailedDriveCount is Greater than 0.
Set the Alarm state trigger to In alarm and select the existing SNS topic you created previously using the Send a notification to... dropdown menu.
Click Next at the bottom of the page.
Add an Alarm name and Alarm description (optional) and click Next.
Click Create alarm.