Auditing Qumulo on Azure using Azure Monitor

REQUIREMENTS

Microsoft Azure account

OVERVIEW OF QUMULO AUDIT LOGGING

Audit logging in Qumulo Core lets you track filesystem operations. When connected clients issue requests to the cluster, Qumulo generates a log message for each attempted operation (read, write, delete, and so on). These log messages include all of the detail about the event in the Syslog message body:

Client IP address
Username and user ID
File operation
Filename
Target file or directory ID

Qumulo sends the events over the network to a remote Syslog istance that the current audit configuration specifies in compliance with RFC 5424.

For more information about auditing Qumulo, see Qumulo Core Audit Logging.

OVERVIEW OF LOG FORWARDING TO AZURE MONITOR

Azure Monitor is Microsoft's platform for storing various machine log data and metrics. It provides a rich toolset for analyzing, querying, visualizing, and correlating different data types.

The following diagram gives an overview of the Azure Monitor architecture. For more information about Azure Monitoring, see Azure Monitor Overview.

Qumulo doesn't send audit events to Azure Monitor directly. Instead, we use a Syslog server to receive the messages and forward them to Azure Monitor using the Azure Monitor Agent.

The following diagram gives an overview of Qumulo auditing and Syslog forwarding using the Azure Monitor Agent.

When Qumulo forwards the logs, you can perform the following actions in an Azure Log Analytics Workspace:

Track filesystem operations.
Create advanced queries and correlate different data sources (for example, logging events from Active Directory or your firewall). This can help you detect suspicious activity more easily.
Detect unusual behavior automatically using Machine Learning techniques with Azure Sentinel.
Run automated responses using automation workbooks.
Detect a massive or unusual deletion.
Detect unexpected actions from certain users or machines.
Detect users and paths with unexpected permission errors.

FORWARDING QUMULO AUDIT DATA TO AZURE MONITOR

You can forward logs to Azure Monitor in a different ways. The following steps explain how to forward Qumulo audit data to Azure Monitor.

NOTE: Currently, Azure Monitor is in preview. However, Azure Monitor is designed to replace agents such as Log Analytics and Telegraph. For more information, see Overview of Azure Monitoring Agents and HTTP Data Collector.

Step 1: To Create a Log Analytics Workspace

A Log Analytics Workspace is an environment for interacting with Azure Monitor log data. Each unique workspace has its own data repository and configuration and its data sources and solutions are configured to store data in a particular workspace. To collect data from the following sources, you must create a Log Analytics workspace:

Azure resources under your subscription
On-premises machine monitored by System Center Operations Manager
Device collections from Configuration Manager
Diagnostics or log data from Azure storage

To query and correlate multiple data sources, you can create a new workspace for Qumulo Auditing or use an existing one.

Log in to the Azure Portal.
Search for Log Analytics Workspaces.
Click Add.
Enter the following information:
- Subscription ID
- Resource group
- Workspace name
- Deployment region
(Optional) Select the pricing tier.

For more information, see Create a Log Analytics Workspace in the Azure Portal.

Step 2: Installing a Syslog Server Using rsyslog

For information about configuring the Syslog server (included by default with most modern Linux distributions), see Qumulo Core Audit Logging. The configuration in the article is very simple: it writes data to a dedicated, configurable log file (this isn't necessary if you want to forward Syslog messages only to Azure Monitor).

Step 3: To Verify that Azure Arc Enabled Servers' Agent is Installed

NOTE: This step is required only if the Syslog server isn't running on Azure.

The Azure Arc enabled servers Connected Machine agent lets you manage your Windows and Linux machines hosted outside of Azure (on your corporate network or another cloud provider).

For information about installing the Azure Arc agent, see Connect Hybrid Machines to Azure from the Azure Portal. For information about the Azure Arc agent, see Overview of Azure Arc Enabled Servers Agent.

To verify that the agent is installed correctly:

Log in to Azure Portal.
Search for Azure Arc.
On the left-hand pane, click Servers.

Your configuration looks similar to the following.

Step 4: To Create a Data Collection Rule (DCR)

Data Collection Rules (DCR) define incoming data for Azure Monitor and specify where the data is sent and stored. The Azure Monitoring Agent uses the DCR to deploy on selected target machines automatically.

Log in to the Azure Portal.
On the left-hand pane, click Monitor > Data Collection Rules > New.
Enter the required information:
- Rule Name
- Subscription
- Resource Group
- Region
- Platform Type
Add your data sources (the machines to which you assign the DCR).
Select the destination (the name of your Log Analytics Workspace).
Click Create.
To check that your agent has deployed successfully, click Azure Arc > MySyslogVM > Extensions. The status changes from Creating to Succeeded.

Step 5: To Configure Qumulo to Forward Audit Events to the Syslog VM

After you finish configuring your infrastructure, you can configure your Qumulo cluster to send audit events to the syslog server

Log in to the Qumulo UI.
Click Cluster > Audit.
For Remote Syslog Address, enter the hostname or IP address of your Syslog server.
For Port Number, if your Syslog server is configured to listen on a different port (514 by default), click Custom and enter the port number.
Click Save.
The Connected status appears.

QUERYING YOUR AUDIT LOGS

You can query your audit logs in Azure Analytics by using the query language Kusto. For more information, see Getting Started with Kusto.

To Enter a Query into Your Log Analytics Workspace

Log in to the Azure portal.
Click Monitor.
Click your Log Analytics Workspace.
Click Logs.
Enter a query. For example, to show all messages, type syslog.
All incoming messages appear.
Customize your query. For example, to limit the events to those generated from hostnames that start with du6, you can use the following query:
```
Syslog

| where HostName startswith "du6"
```

Extracting Fields out of Syslog Messages

Syslog messages are stored in a raw format (for more information, see Qumulo Core Audit Logging). This format contains several values such as the following:

IPv4 or IPv6
User ID
Protocol
Computer
Operation
Target file or directory

The following is an example message:

192.168.1.10,"AD\alice",nfs,fs_read_data,ok,123,"/.snapshot/1_snapshot1225/dir/",""

For more complex queries, it might be helpful to extract the different values from the message body into several fields. For example:

Syslog

| extend CSVFields = split(SyslogMessage, ',')

| extend ClientIP  = tostring(CSVFields[0])

| extend UserID    = tostring(CSVFields[1])

| extend Protocol  = tostring(CSVFields[2])

| extend Operation = tostring(CSVFields[3])

| extend ResponseCode = tostring(CSVFields[4])

| extend MessageID = tostring(CSVFields[5])

| extend File      = tostring(CSVFields[6])

| extend Status    = tostring(CSVFields[7])

This query uses field names which you can use to customize your queries, or to save a query as a function for reuse or creating more complex queries. For example, you can store this query in a function called ExtractSyslogAugitMsg and then call the function for all audit events for text files within the selected scope.

ExtractSyslogAuditMsg
 
| project EventTime, ClientIP, File, Operation

| where File contains "txt"

RESOLUTION

You can now forward Qumulo events using Azure Monitoring Agent and then track your Qumulo system using Azure Monitor.

ADDITIONAL RESOURCES

Like what you see? Share this article with your network!

Was this article helpful?

0 out of 0 found this helpful

Comments

0 comments

Please sign in to leave a comment.