Capacity Trends

IN THIS ARTICLE

Outlines the analytics available in Qumulo Core's Capacity Trends

REQUIREMENTS

• Cluster running Qumulo Core

NOTE: This article reflects the new UI for Capacity Trends available in versions 2.12.3 and above. To get the full scope of this feature, ensure your cluster is running at least version 2.12.3 of Qumulo Core.

DETAILS

Capacity Trends gives you the details you need to monitor and manage your cluster’s capacity usage over time. By intelligently aggregating filesystem paths, you can quickly see significant changes by capacity in your data, verify when users are writing significant amounts of data, track large deletes in the form of negative capacity changes. From terabyte to petabyte scale, our data visualizations can help you manage your Qumulo cluster for scale while also drilling down into significant capacity changing events.

To access Capacity Trends:

1. Login to the Qumulo Core Web UI
2. Hover over the Analytics menu and click Capacity Trends. 
   
   
   

The summaries for each of your top-level capacity consumers (metadata, data, and snapshots) will be displayed via the overview widget (2.12.5 and above) at the top of the page, the Capacity History graph, and the Capacity Change graph.

Capacity History

The Capacity History graph is your top-level capacity over time for metadata, data, and snapshots on your cluster. It corresponds to your total used capacity and is the same data you’ll find on the Qumulo dashboard home page. To see how your storage is allocated for a specific point in time, hover over the graph to see more details.

Capacity Change

The Capacity Change graph uses “live data” to calculate the metadata and data changes (snapshots not included) over time on your cluster. Hover over any bar in the graph to see additional details on the changes and used capacity or click on the bar to see the corresponding significant capacity changes by path for that point in time via the chart featured below.

Filter by Time

To see your capacity changes and history over short, medium or long term time frames, use the time filter dropdown menu to select one of the following :

• Last 72 hours: hour-over-hour changes in capacity
• Last 30 days: day-over-day trends and changes in capacity
• Last 52 weeks: week-over-week trends and changes in capacity

TIP! If these timeframes and granularity aren’t sufficient, leverage our API to look at any hour’s data over the last year. This data visualization is a powerful view of significant capacity changes for a selected time period. By intelligently aggregating filesystem paths, you can quickly see significant changes by capacity in your data. You will be able to see when users are writing significant amounts of data to the clusters. You will also see large deletes in the form of negative capacity changes.

Get the time-series summary data by running the following:

/v1/analytics/capacity-history/?begin-time={epoch-timestamp}&end-time={epoch-timestamp}&interval=hourly

Get the path-level data for a particular timestamp using the call below:

/v1/analytics/capacity-history/{epoch-timestamp}/

Frequently Asked Questions

Why do I see a large spike in Snapshot usage and a dip in data usage?

• It’s likely that you or a cluster user deleted a bunch of data that is associated with a snapshotted directory. Until that snapshot expires or is manually deleted, the chart will show Snapshot usage corresponding to that data.

How do I manage capacity associated with Snapshots?

• You can leverage our API to calculate individual Snapshot consumption and/or directories. In general, it's effective to keep well-defined snapshot policies that don't go too far back in time.

Why don’t I see expected or deep paths in the capacity change graph?

• In the cases of very wide directories or lots of small changes, the only significant change reported could be root or a high level directory. The main threshold utilized in our system requires that aggregate capacity of the directory must be at least 1/1000th of your total used capacity. So, if you are using 1PB of capacity, the directories we track must be at least 1TB. If you need a higher granularity, we recommend polling our API directory and storing those directories and capacities in a database.

Why is the graph so flat?

• We now use a scale that properly reflects the high level capacity trends of your cluster over a specified period of time. The time filter on the Capacity Trends page will default to 72 hours, which is a relatively short-term view. To see the larger, long-term trends, switch to the 30 day or 52 week view (see Filter by Time section above), or scroll down to the capacity change graph to review changes on a relative scale.

RESOLUTION 

You should now be able to successfully use Capacity Trends to manage and monitor capacity on your cluster

ADDITIONAL RESOURCES

Snapshots: Capacity Consumption

QQ CLI: File System

 

Like what you see? Share this article with your network!