Qumulo File Permissions Overview

REQUIREMENTS

Cluster running Qumulo Core

DETAILS

Qumulo supports two types of formats:

NTFS Access Control Lists (SMB)
POSIX mode bits (NFS)

The Qumulo File System stores the permissions (NTFS ACL or POSIX mode bits) within the metadata of each file/directory. These permissions can be managed via NFS mounts or SMB share points on Windows and Qumulo API. Important notes:

Qumulo will synthesize NFS permissions for SMB and vice versa
Last Chmod wins determines which permissions will be stored meaning an NTFS permission setting operation will remove and replace POSIX mode bits with ACLs and vice versa.

PERMISSIONS BASICS

The object owner will always have the right to modify permissions of that object and will never be “locked out” of a file they own.
Qumulo Admin will always have FULL CONTROL of all objects in the Qumulo File System.
Objects are managed by their own permissions AND their parent directory’s permissions.
- The mode bits of a directory will impact the permissions of all the objects it contains

POSIX

Used by Linux, Unix, and Mac systems and include the three basic permissions of Read, Write, and Execute.

These permissions can then be assigned to Owner (UID), Group Owner (GID), and Others. Owner is determined by the creator’s UID and Group Owner is determined by creator’s primary GID. Note that UIDs and GIDs are the only means of identification available to NFSv3 systems.

UID: “User ID” a 16 or 32 bit number that is either automatically assigned by the system at account creation time or managed by an external directory service, such as LDAP or Active Directory
GID: “Group ID” similar to UID, it is a numerical ID for a group of users in a system or LDAP/AD Domain

Hosts can be restricted from accessing a NFSv3 export but NOT individual user beyond file permissions.
Usernames exist for the benefit of human users only and have no meaning in the local system
Duplicate UIDs/GIDs across systems are highly likely within network environments without directory service.

Umask: Permissions set at file/directory creation time is determined by each NFS user’s umask setting. The Umask is subtracted from 777 (rwx ) to determine the permissions.

Example: Umask is 0002 removed from 777 equals 0775 permissions at creation. Reference Octal Modes for additional information.

Umask value can be dynamically set by the user with the umask command at any time.
Umask values can be enforced by Qumulo on NFS exports via NTFS ACLs over SMB mounts

TIP! Umask values can be enforced for SMB users writing to shared NFS exports using:

qq smb_mod_share

Umask for SMB writers
Utilize the following commands:

qq smb_add_share

qq smb_mod_share

POSIX Mode Bits - Special Mode Bits
Qumulo supports 2 special mode bits:

Set GID (Directories only): changes ownership of all newly created files and directories to the same Group as the Group Owner of the parent directory.
Sticky Bit Restricted Deletion Bit: Only owner, directory owner or superuser can delete file or directory with sticky bit set, other users with proper permissions will be able to read & append files *sticky bit currently not respected via SMB

NTFS ACLS

Access Control Lists (ACL) composed of Access Control Entries that are used by Windows, Mac, and some Samba clients with Qumulo supporting 14 basic permissions. These permissions can be assigned to One Owner and Multiple Groups unlike POSIX where there is one group owner.
SID (security identifier): used to manage group and user identities. Each Qumulo cluster generates its own internal SIDs and performs SID to username lookup on behalf of clients.

Identifies a Trustee
Unique and Immutable
Many more SIDs are possible then UID/GID
Internally matched to a globally unique 128 bit GUID in AD
Orphan SIDs can happen when files are migrated from other sources OR between Qumulo Clusters.

Well Known SID: Identifies generic groups and users. Qumulo supports
- CREATOR_OWNER
- CREATOR_GROUP

ACCESS CONTROL LISTS: lists composed of one or more Access Control Entry (ACE) that is saved in each file/directory’s metadata. ACLs are:

Inheritable
- ACL Inheritance is enforced on NFS exports managed via NTFS ACLs
Composed of Allow or Deny entries
Can include over 1800 ACEs in one (not recommended)

ACL INHERITANCE: enforced on NFS exports managed via NTFS ACLs
Explicit Permissions: Permissions that are set by default when the object is created, or by user action.
Inherited Permissions: Permissions that are given to an object because it is a child of a parent object.

PERMISSION PRECEDENCE

Deny Permissions normally take precedence
Permissions applied directly to an object (explicit permissions) take precedence over permissions inherited from a parent (for example from a group).
Permissions inherited from near relatives take precedence over permissions inherited from distant predecessors. (I.e. Parent vs Grandparent)
Permissions from different user groups that a user belongs to and are at the same level (Explicit vs Inherited) are cumulative.

Enforcement Order Summarized:

Explicit Deny
Explicit Allow
Inherited Deny
Inherited Allow

OBJECT OWNERSHIP
User must be a Domain member and Windows Client PC must be connected to Domain to enumerate and assign another Domain member as the owner of an object. Note that changing owner does NOT change the ACL, although the object owner will gain some implicit rights. Groups can be made owners of objects but should be avoided in Multi-Mode environments!

QUMULO DEFAULT FILE PERMISSIONS

NFS DEFAULT PERMISSIONS
The NFS root directory is owned by root (UID 0) and group nfsnobody (GID 65534). The default permissions for the NFS root directory are rwxrwxrwx or 0777.

All users will be able to create files and directories in the current directory
All users will be able to delete files and directories in the current directory, including those owned by root.
File/Directory creation mode bits will be determined by the user’s umask

SMB DEFAULT PERMISSIONS
One User Account and Two Groups are given rights to the root share by default:

Qumulo\admin (User): All ACEs except Full Control and Delete for “This folder only”
Qumulo\users (Group): “Modify” ACL for “This folder only”
Everyone (Group): “Modify” ACL for “This folder only”

Note that Qumulo admin retains “Full Control” at all times, regardless of displayed permissions.

Bypass the Default Permissions by creating a directory in advance with the desired permissions and then making it into an NFS export or SMB share in Qumulo Core.

USER AUTHENTICATION
Qumulo supports two methods of user authentication:

Local Qumulo user accounts
Active Directory 2008 and above

Local Accounts: two users and two groups exist by default which include Users admin and guest and groups Guests and Users. Additional considerations include:

User accounts other than Admin do NOT have UIDs assigned by default (admin = UID 0)
Groups other than Guests do not have GIDs set by default (Guests = GID 65534)
Users and Groups can be created with UIDs and GIDs that match existing external NFS Users and Groups
Qumulo local Users and Groups can NOT be created that match the SID of existing Active Directory or local Windows Domain users.
Local accounts are ideal for mixed mode environments without Active Directory & RFC2307

Active Directory: Qumulo supports server 2008 Functional Level and above

Full Kerberos SSO support.
NTLMv2 Minimum Security Requirement
Multi-Domain Trust Support
No domain account is given implicit rights to Qumulo upon joining

Multi-Domain Trusts: Qumulo respects and fully supports cross-domain trusts over SMB.

Full Kerberos SSO support across Domains

RFC2307: the ability to store user and group information in an LDAP directory

Allows UID/GIDs to be mapped to Domain SIDs
Fully supported in Active Directory since version 2003 R2
Identity Management for UNIX/NIS Server roles are not required for RFC2307 support
Values can be added to AD accounts via ADUC or Powershell

RFC2307 is the key for multi-mode permissions harmony!

RFC2307 Values in ADUC

Required RFC2307 Values

gidNumber
loginShell (/bin/bash for example)*
uidNumber
unixHomeDirectory*

*Linux AD clients such as SSSD will not properly ID Domain users without these two values!

Organization Units (OUs) and Base Distinguished Names (DNs)
By default, Qumulo will look into the Users OU of an AD domain if no special DN is specified meaning that any user outside of that OU will NOT have the RFC2307 attributes looked up.

Specifying the TLD of the Domain as the BaseDN will force Qumulo to search the entire Domain for the UID/GID info.
In massive Domains, admins will likely benefit from restricting the lookups to a specific OU.

OUs in ADUC

BaseDN Setting

Format of BaseDN: Example Domain Name is AD.WEST.CLIENTCO.COM. BaseDN to search entire West Domain would be:

DC=AD,DC=WEST,DC=CLIENTCO,DC=COM

BASELINE PERMISSIONS

Setting Baseline Permissions: Ask yourself Who needs access to what? Careful planning will save a lot of unnecessary work later!

Qumulo\admin (SMB) or Root (NFS) access is required for setting the base permissions
All desired Domain User or Group access should be set at share roots before massive file migrations take place.
No Domain Groups or Users have automatic elevated privileges on Qumulo shares.
Traverse permissions or execute mode bit is required for every parent directory in the path to allow a user to reach a Share or Export

The Everyone Group: is the equivalent of the Others group in POSIX and present in all default ACLs.

The Everyone ACL is often deleted by client Admins leading to unintended behavior.
Every Group/User requiring access to an object will need to be added to the ACL before removing the Everyone entry.

Well Known SIDs

CREATOR_OWNER & CREATOR_GROUP
- Placeholder SID
- Gives the object creator or the object creator’s group special rights to objects created inside the parent directory that contains the Well Known SID
- Similar to SetGID and Sticky Bit
- Required for remote Windows Home Directories and Roaming Profiles

The Impact of hitting “Apply”
Changing permissions at the root level of a large directory in Windows can trigger a very time consuming operation as Permissions Inheritances are calculated for the entire directory structure. Windows Explorer will be tied up during this process and early abortion of operation can lead to corrupted permissions from the directory tree. Note that the process is generally faster via Powershell!

NFS Exports Restrictions

Access to NFS exports can be managed on a host by host basis.
The access level of IP ranges or single hosts can be managed.
Complex IP restrictions can be placed on NFS exports using the following command:

qq nfs_mod_share

Example: A single admin IP can be allowed root access while all other IPs are root-squashed.

MULTI-MODE PERMISSIONS

Qumulo has a very robust and versatile multi-mode permissions model that allows concurrent NFS and SMB user access. Again we must ask Who needs access to what? before attempting to set permissions. Keep in mind that life is much easier with RFC2307 and AD!

Client MUST decide on either POSIX or NTFS to set permissions and strictly use that method.
Last Chmod wins determines which permissions will be stored meaning an NTFS permission setting operation will remove and replace POSIX mode bits with ACLs and vice versa.
Most of the time clients will be better served using NTFS ACLs because you have
- Greater granularity of control
- The ability to allow the building of complex Group-based permissions that are impossible to do with POSIX mode bits alone
- Graphic Interface that is favorable for many users.

Multi-Mode Management via POSIX

Preferred for environments with populations of advanced NFS users who regularly set own permissions and is also good for environments where simpler Group access is needed.

Requires the use of SMB "umask equivalent" enforcement via qq
Generally much faster to batch apply permissions than via Windows Explorer

Generated versus Stored ACLs

Qumulo provides a method for determining if the permissions stored in an object is an actual NTFS ACL or an ACL interpreted from the equivalent POSIX mode bits using the commands:

qq fs_get_acl --path

qq fs_get_acl --id

Umask emulation via ACLs
Inheritable ACLs are followed and enforced over NFS exports as long as the ACL is not replaced by POSIX mode bits. Note that the NFS user’s umask will have no effect on the permissions of newly created files as the permissions will be set via ACL inheritance.

SID - Special SIDs

S-1-5-88-1-### : “NFS User”
S-1-5-88-2-### : “NFS Group”

These SIDs are generated when a POSIX client without a SID writes to a SMB/NFS common directory. This is more commonly seen in files imported from other NAS’es, or when LDAP or external AD Domain bound clients access Qumulo via NFS.

This directory was created by a NFS client (“Account Unknown”) that is not bound to the same AD Domain as the cluster, in this case UID 2078, GID 2000. You will also see this if the incorrect BaseDN is set in Qumulo’s AD options.

PERMISSIONS LIMITATIONS

“CP’ing” a file or directory via NFS will remove the ACL from the object (NFS performs SETATTR on cp)
Sticky Bit is not enforced over SMB
Non AD bound “rogue” NFS clients can wreak havoc on permissions if not closely managed.
Setting an NTFS Group as an Object owner will force the object to be mapped to UID & GID “nfsnobody” (65534) as the GID attribute will not be mapped to UID

NFS 16 Group Limit

Qumulo currently follows the auth_sys standard set in RFC5331 which limits a user to membership in 16 groups plus the primary group. Memberships after the 16th group will be ignored! AD with RFC2307 & Kerberos removes this restriction.