ACL_MASK, chmod Compatibility and Group Class in Linux Explained

 

ACL_MASK, chmod Compatibility and Group Class in Linux Explained

Part 4 of 8 —ACL_MASK, chmod Compatibility and Group Class in Linux Explained

🎭
chmod Compatibility
🔒
Permission Ceiling
🧮
AND Masking

The ACL_MASK entry is one of the trickiest parts of ACLs. Its purpose is to solve a backwards-compatibility problem: when old programs that don’t know about ACLs use chmod() to change permissions, the ACL_MASK ensures those changes don’t silently destroy the per-user/per-group access rules set by ACL-aware applications.

Think of ACL_MASK as a maximum permissions gate — even if an ACL entry says “rwx”, the actual effective permissions are limited by the ACL_MASK value. This only affects the “group class” entries (ACL_USER, ACL_GROUP_OBJ, ACL_GROUP) — the file owner (ACL_USER_OBJ) and ACL_OTHER are never masked.

Key Terms

ACL_MASKGroup classAND masking chmod() interactionstat() st_mode ACL-unaware appacl_calc_mask() #effective: commentsetfacl -n

What Is the Group Class?

The “group class” is a term for the set of ACL entries that ACL_MASK applies to:

ACL Entry In Group Class? Masked by ACL_MASK?
ACL_USER_OBJ (owner) ❌ No ❌ Never masked
ACL_USER (named users) ✅ Yes ✅ Always masked
ACL_GROUP_OBJ (file group) ✅ Yes ✅ Always masked
ACL_GROUP (named groups) ✅ Yes ✅ Always masked
ACL_OTHER ❌ No ❌ Never masked
Key rule: ACL_MASK is only required when ACL_USER or ACL_GROUP entries exist. It’s optional for a minimal ACL (no named users or groups).

The Problem ACL_MASK Solves

The chmod() Backward Compatibility Problem

Imagine you have an ACL set up for a file with per-user access. Now, an old tool (like a backup program or text editor) does a simple chmod(pathname, 0700) meaning “only the owner should have access”. Without ACL_MASK, there’s no clean way to implement this intent while preserving the ACL data.

Problem with approach 1: Just change ACL_GROUP_OBJ to ---. But the named ACL_USER and ACL_GROUP entries still grant permissions to specific users/groups. The intent is not fulfilled.

Problem with approach 2: Set all ACL_USER and ACL_GROUP entries to ---. This destroys the carefully set permissions, and a subsequent chmod(pathname, 0751) cannot restore them — you’d get wrong permissions back.

The ACL_MASK solution: Instead of touching any ACL_USER or ACL_GROUP entries, just update the ACL_MASK. When chmod() changes the group permission bits, it actually changes ACL_MASK (not ACL_GROUP_OBJ). The individual ACL_USER and ACL_GROUP entries remain untouched. When permissions are restored later, the mask is updated again and the full ACL semantics come back correctly.

# Before chmod: file has extended ACL
getfacl myfile
# user::rwx         ACL_USER_OBJ
# user:alice:r-x    ACL_USER for alice
# group::r-x        ACL_GROUP_OBJ
# group:devs:--x    ACL_GROUP for devs
# mask::r-x         ACL_MASK
# other::--x        ACL_OTHER

# Old program does: chmod(myfile, 0700)
# This changes group bits to '---'
# With ACL_MASK: chmod updates the MASK not the group entries

chmod 700 myfile
getfacl myfile
# user::rwx         unchanged
# user:alice:r-x    #effective:---  <-- still there, but masked to nothing
# group::r-x        #effective:---  <-- still there, but masked to nothing
# group:devs:--x    #effective:---  <-- still there, but masked to nothing
# mask::---         <-- ONLY THIS CHANGED (was r-x)
# other::--x        unchanged (not in group class)

# Now restore with: chmod 751
chmod 751 myfile
getfacl myfile
# user::rwx
# user:alice:r-x    #effective:r-x  <-- full alice permissions restored!
# group::r-x        #effective:r-x  <-- group permissions restored!
# group:devs:--x    #effective:--x  <-- dev permissions restored!
# mask::r-x         <-- mask restored to r-x (group bits of 751)
# other::--x

Mask, chmod(), and stat()

Two key rules when ACL_MASK is present
🔧 Rule 1: chmod() → updates ACL_MASK

When an extended ACL exists, changing the group permission bits with chmod() does NOT change ACL_GROUP_OBJ. It changes the ACL_MASK value instead. This preserves the individual group class entries.

chmod g+rw file
# Updates ACL_MASK to include rw
# Does NOT touch ACL_GROUP_OBJ entry
📊 Rule 2: stat() → reads ACL_MASK

When you call stat() on a file with an extended ACL, the group permission bits in st_mode reflect the ACL_MASK value, not ACL_GROUP_OBJ. This ensures ACL-unaware tools see a consistent picture.

ls -l file
# -rwxr-x--x+
#      ^^^
#  This shows ACL_MASK bits
#  NOT ACL_GROUP_OBJ bits
Gotcha: If you use chmod g+rw on a file with a restrictive ACL, it sets ACL_MASK to allow rw — but individual group entries may still deny write. For example if ACL_GROUP_OBJ is ---, the group still has no access despite the mask being rw-. The mask is a ceiling, not a floor. A workaround: also set ACL_GROUP_OBJ to rwx so it always gets whatever the mask allows.

The #effective: Comment in getfacl Output

When an ACL entry’s permissions are reduced by the mask, getfacl appends a #effective: comment to show what permissions actually apply after masking:

setfacl -m m::x myfile   # Set mask to only allow execute

getfacl myfile
# user::rwx
# user:alice:r-x   #effective:--x   <-- r-x AND x = --x
# group::r-x       #effective:--x   <-- r-x AND x = --x
# group:devs:--x                    <-- --x AND x = --x (same, no comment)
# mask::--x
# other::--x                        <-- other is not masked

# The #effective: comment tells you what really happens at runtime.
# Without it, you might think alice has r-x but she actually only has --x
The #effective: comment only appears when the actual effective permissions differ from what the entry says. If they’re the same, no comment is shown.

acl_calc_mask() — Auto-Calculate the Mask

The C function acl_calc_mask() automatically computes the correct ACL_MASK value by taking the union of all group class entry permissions. You should call this whenever you add or modify ACL_USER or ACL_GROUP entries in a C program.

#include <stdio.h>
#include <sys/acl.h>

int main(int argc, char *argv[]) {
    if (argc != 2) {
        fprintf(stderr, "Usage: %s filename\n", argv[0]);
        return 1;
    }

    /* Load existing ACL */
    acl_t acl = acl_get_file(argv[1], ACL_TYPE_ACCESS);
    if (!acl) { perror("acl_get_file"); return 1; }

    printf("Before acl_calc_mask:\n");
    char *t1 = acl_to_text(acl, NULL);
    printf("%s\n", t1);
    acl_free(t1);

    /* Add a new ACL_USER entry manually */
    acl_entry_t new_entry;
    acl_create_entry(&acl, &new_entry);

    acl_set_tag_type(new_entry, ACL_USER);

    uid_t uid = 1234;
    acl_set_qualifier(new_entry, &uid);

    acl_permset_t perms;
    acl_get_permset(new_entry, &perms);
    acl_add_perm(perms, ACL_READ);
    acl_add_perm(perms, ACL_EXECUTE);
    acl_set_permset(new_entry, perms);

    /* After adding entry, recalculate mask */
    /* acl_calc_mask sets mask = union of all group class perms */
    /* It also CREATES the mask entry if one doesn't exist yet */
    if (acl_calc_mask(&acl) == -1) {
        perror("acl_calc_mask");
    }

    printf("After acl_calc_mask:\n");
    char *t2 = acl_to_text(acl, NULL);
    printf("%s\n", t2);
    acl_free(t2);

    /* Validate and write back */
    if (acl_valid(acl) == 0) {
        acl_set_file(argv[1], ACL_TYPE_ACCESS, acl);
        printf("ACL written successfully.\n");
    } else {
        printf("ACL validation failed!\n");
    }

    acl_free(acl);
    return 0;
}
/* Compile: gcc -o calc_mask calc_mask.c -lacl */

Preventing Auto-Mask Update — setfacl -n

By default, when you remove ACL entries with setfacl -x, it automatically recalculates the ACL_MASK to be the union of remaining group class entries. To prevent this automatic adjustment, use the -n flag:

# Without -n: mask is auto-recalculated after removal
setfacl -x u:alice,g:devs myfile
# mask becomes union of remaining entries

# With -n: mask stays as-is, even after entry removal
setfacl -n -x u:alice myfile

# When is -n useful?
# When you've manually set a specific mask value and don't
# want it changed by entry additions/removals.

# Example: set a very restrictive mask manually
setfacl -n -m m::r-- myfile   # Force mask to read-only
# Now even if group entries say rwx, effective = r-- only

Coding Examples

Example 1: Demonstrate masking — compute effective permissions
#include <stdio.h>
#include <sys/acl.h>
#include <acl/libacl.h>

/* Compute effective permissions = entry_perms AND mask_perms */
void show_effective(acl_permset_t entry_ps, acl_permset_t mask_ps,
                    const char *entry_name) {
    int entry_r = acl_get_perm(entry_ps, ACL_READ);
    int entry_w = acl_get_perm(entry_ps, ACL_WRITE);
    int entry_x = acl_get_perm(entry_ps, ACL_EXECUTE);

    int mask_r = acl_get_perm(mask_ps, ACL_READ);
    int mask_w = acl_get_perm(mask_ps, ACL_WRITE);
    int mask_x = acl_get_perm(mask_ps, ACL_EXECUTE);

    int eff_r = entry_r && mask_r;
    int eff_w = entry_w && mask_w;
    int eff_x = entry_x && mask_x;

    printf("%-20s entry:%c%c%c  mask:see  effective:%c%c%c\n",
           entry_name,
           entry_r ? 'r' : '-', entry_w ? 'w' : '-', entry_x ? 'x' : '-',
           eff_r   ? 'r' : '-', eff_w   ? 'w' : '-', eff_x   ? 'x' : '-'
    );
}

int main(int argc, char *argv[]) {
    if (argc != 2) {
        fprintf(stderr, "Usage: %s filename\n", argv[0]);
        return 1;
    }

    acl_t acl = acl_get_file(argv[1], ACL_TYPE_ACCESS);
    if (!acl) { perror("acl_get_file"); return 1; }

    /* Find mask entry first */
    acl_permset_t mask_ps = NULL;
    acl_entry_t e; acl_tag_t t;
    for (int id = ACL_FIRST_ENTRY; acl_get_entry(acl, id, &e) == 1; id = ACL_NEXT_ENTRY) {
        acl_get_tag_type(e, &t);
        if (t == ACL_MASK) { acl_get_permset(e, &mask_ps); break; }
    }

    if (!mask_ps) {
        printf("No ACL_MASK entry — this is a minimal ACL\n");
        acl_free(acl);
        return 0;
    }

    printf("Effective Permissions for: %s\n\n", argv[1]);
    printf("%-20s %-20s %s\n", "Entry", "Raw", "Effective (after mask)");
    printf("%-20s %-20s %s\n", "-----", "---", "----------------------");

    /* Now loop and show effective for group class entries */
    for (int id = ACL_FIRST_ENTRY; acl_get_entry(acl, id, &e) == 1; id = ACL_NEXT_ENTRY) {
        acl_get_tag_type(e, &t);
        acl_permset_t ps;
        acl_get_permset(e, &ps);

        switch(t) {
            case ACL_USER_OBJ:
                /* Not masked */
                printf("%-20s (owner — not masked)\n", "ACL_USER_OBJ");
                break;
            case ACL_USER: {
                uid_t *uid = acl_get_qualifier(e);
                char name[32]; snprintf(name, 32, "ACL_USER uid=%d", (int)*uid);
                show_effective(ps, mask_ps, name);
                acl_free(uid);
                break;
            }
            case ACL_GROUP_OBJ:
                show_effective(ps, mask_ps, "ACL_GROUP_OBJ");
                break;
            case ACL_GROUP: {
                gid_t *gid = acl_get_qualifier(e);
                char name[32]; snprintf(name, 32, "ACL_GROUP gid=%d", (int)*gid);
                show_effective(ps, mask_ps, name);
                acl_free(gid);
                break;
            }
            case ACL_MASK: {
                int r = acl_get_perm(ps, ACL_READ);
                int w = acl_get_perm(ps, ACL_WRITE);
                int x = acl_get_perm(ps, ACL_EXECUTE);
                printf("%-20s mask value: %c%c%c\n", "ACL_MASK",
                       r?'r':'-', w?'w':'-', x?'x':'-');
                break;
            }
            case ACL_OTHER:
                printf("%-20s (not masked)\n", "ACL_OTHER");
                break;
        }
    }

    acl_free(acl);
    return 0;
}
/* Compile: gcc -o show_effective show_effective.c -lacl */
Example 2: Shell demo — mask behavior with chmod
#!/bin/bash
# mask_demo.sh — shows how chmod changes ACL_MASK

echo "=== Creating test file ==="
touch masktest.txt

echo "=== Setting up extended ACL ==="
setfacl -m u::rwx,u:bob:rwx,g::rwx,g:devs:rwx,m::rwx,o::--- masktest.txt
echo "Initial ACL:"
getfacl --omit-header masktest.txt

echo ""
echo "=== Running: chmod 700 masktest.txt ==="
chmod 700 masktest.txt
echo "After chmod 700 (group bits = ---):"
getfacl --omit-header masktest.txt
# Notice: mask becomes ---, all group class entries show #effective:---
# But the entries themselves are UNCHANGED

echo ""
echo "=== Running: chmod 750 masktest.txt ==="
chmod 750 masktest.txt
echo "After chmod 750 (group bits = r-x):"
getfacl --omit-header masktest.txt
# mask becomes r-x, entries show #effective:r-x
# All original rwx entries are STILL THERE, just masked

echo ""
echo "=== Running: chmod 770 masktest.txt ==="
chmod 770 masktest.txt
echo "After chmod 770 (group bits = rwx — but wait...):"
getfacl --omit-header masktest.txt
# mask becomes rw-, and the original entries are fully visible again
# THIS is why mask is important — it preserves the ACL data

Interview Questions

Q1. What is the purpose of the ACL_MASK entry?

ACL_MASK provides backward compatibility with ACL-unaware applications. It acts as an upper bound (ceiling) on permissions for all group class entries (ACL_USER, ACL_GROUP_OBJ, ACL_GROUP). When an ACL-unaware program changes group permission bits via chmod(), only ACL_MASK changes — the individual ACL_USER and ACL_GROUP entries are preserved. This means restoring permissions later brings back the full ACL semantics.

Q2. When chmod() is called on a file with an extended ACL, which entry changes?

The ACL_MASK entry changes, not the ACL_GROUP_OBJ entry. This is the key mechanism for backward compatibility. When stat() is subsequently called on the file, the group bits in st_mode reflect the ACL_MASK value (not ACL_GROUP_OBJ).

Q3. Does ACL_MASK apply to the file owner (ACL_USER_OBJ)?

No. ACL_MASK only applies to the “group class” entries: ACL_USER, ACL_GROUP_OBJ, and ACL_GROUP. The file owner (ACL_USER_OBJ) and ACL_OTHER are never masked. This ensures the owner’s permissions are always exactly what ACL_USER_OBJ says.

Q4. What does #effective: mean in getfacl output?

It shows the actual effective permissions for a group class entry after applying the ACL_MASK. For example, if an ACL_USER entry says rwx but ACL_MASK says r--, getfacl shows: user:alice:rwx #effective:r--. This helps you understand what permissions actually apply at runtime.

Q5. What does acl_calc_mask() do and when should you call it?

acl_calc_mask(&acl) computes the ACL_MASK value as the union (bitwise OR) of all group class entry permissions (ACL_USER, ACL_GROUP_OBJ, ACL_GROUP). It also creates the ACL_MASK entry if it doesn’t exist. Call it in C code whenever you add, modify, or remove ACL_USER or ACL_GROUP entries to ensure the mask is valid and consistent.

Q6. What is setfacl -n and when would you use it?

The -n flag tells setfacl not to automatically recalculate the ACL_MASK after modifications. Normally, adding or removing entries triggers an automatic mask recalculation. Use -n when you’ve manually set a specific mask value and want it preserved exactly, even as you make other changes to ACL entries.
Next: getfacl & setfacl Shell Commands →

Master all the command-line options for viewing and modifying ACLs

Continue → ← Back

Leave a Reply

Your email address will not be published. Required fields are marked *