ACL Permission-Checking Algorithm in Linux Explained

 

ACL Permission-Checking Algorithm in Linux Explained

Part 2 of 8 —ACL Permission-Checking Algorithm in Linux Explained

🔍
5-Step Check
First Match Wins
🎭
AND-Masking

When a process tries to access a file that has an ACL, the kernel checks permissions in a specific ordered sequence. The key rule is: the first matching step wins. The check stops as soon as one criteria matches — it does not continue to check further entries.

The kernel uses the process’s file-system user ID (usually same as effective UID) and file-system group IDs for these checks, not the real UID/GID.

Key Terms in This Topic

Effective UIDFile-system UID Supplementary GIDsAND-masking Group classPrivileged process EACCESFirst-match semantics

The 5-Step Algorithm — Flow Diagram

Linux checks ACL entries in this exact order. Stop at first match.

STEP 1: Is the process privileged (root / CAP_DAC_OVERRIDE)?
YES ↓
NO ↓
GRANT ALL
Exception: for execute, at least one ACL entry must grant it
Continue to Step 2 →

STEP 2: Does process UID == file owner UID?
YES ↓
NO ↓
✅ Use ACL_USER_OBJ permissions
(No masking with ACL_MASK)
Continue to Step 3 →

STEP 3: Does process UID match any ACL_USER entry qualifier?
YES ↓
NO ↓
Use ACL_USER entry permissions
AND-masked with ACL_MASK
✅ Grant if access satisfied, else ✘ Deny
Continue to Step 4 →

STEP 4: Does any process GID match file group (ACL_GROUP_OBJ) or any ACL_GROUP qualifier?
YES ↓
NO ↓

Sub-check 4a: If GID matches file group & ACL_GROUP_OBJ grants access → use it (AND ACL_MASK)

Sub-check 4b: If GID matches ACL_GROUP entry & it grants access → use it (AND ACL_MASK)

Sub-check 4c: If matched but no entry grants requested access → ✘ DENY

Continue to Step 5 →

STEP 5: Use ACL_OTHER permissions
Process matched nothing above → use ACL_OTHER entry
✅ Grant if ACL_OTHER permits, else ✘ DENY (EACCES)

Each Step Explained Clearly

Step 1: Privileged Process

A process with root privileges (UID 0) or the CAP_DAC_READ_SEARCH / CAP_DAC_OVERRIDE capability bypasses ACL checks for read and write. However, for execute permission, even root is only granted execute if at least one ACL entry (for any user/group) grants execute. This prevents accidentally running non-executable files.

# Root can always read/write, but for execute:
# If no ACL entry has 'x' bit, even root cannot execute the file

# Check this:
touch noexec.sh
# Even as root:
# access(noexec.sh, X_OK) will FAIL if no ACL entry has 'x'
Step 2: File Owner Match

If the process’s file-system UID equals the file’s owner UID, the ACL_USER_OBJ entry is used. The ACL_MASK does NOT apply here — owner permissions are always used directly.

# File owned by user 'ravi' (UID 1001)
# ACL_USER_OBJ = rw-
# ACL_MASK     = ---  (mask blocks everything)

# When ravi runs: access("file", R_OK)
# Result: ALLOWED! (Owner is NOT masked by ACL_MASK)
# Mask only affects group class entries
Step 3: Named User Match

If the process’s UID matches the qualifier in an ACL_USER entry, that entry’s permissions are used — AND-masked with ACL_MASK. Even if the user also belongs to the file’s group, step 3 takes precedence over step 4.

# ACL:  user:bob:rwx   mask::r--
# bob tries access(file, R_OK | W_OK | X_OK)

# ACL_USER for bob: rwx
# ACL_MASK:         r--
# Effective = rwx AND r-- = r--

# access(file, R_OK) → ALLOWED
# access(file, W_OK) → DENIED (masked out)
# access(file, X_OK) → DENIED (masked out)
Step 4: Group Matching (Most Complex)

This step is tricky. A process can have multiple group IDs (one effective GID + supplementary GIDs). The kernel checks if any of these GIDs match ACL_GROUP_OBJ or any ACL_GROUP entry.

Important rule: If the process’s GID matches (step 4 is entered), but none of the matching entries grant the requested permission → access is DENIED. It does NOT fall through to step 5 (ACL_OTHER).

# File group = GID 100
# ACL entries:
#   ACL_GROUP_OBJ (GID 100): rwx   mask: rw-
#   ACL_GROUP (GID 102):     r--
#   ACL_OTHER:               rwx   <-- very permissive

# Process with GID 100 asks for execute:
#   Step 4 matched (GID 100 == file group)
#   ACL_GROUP_OBJ says rwx, masked by rw- → effective rw-
#   No execute → DENIED!
#   (Does NOT fall through to ACL_OTHER even though other has rwx)

# Process with GID 999 (no match in step 4):
#   Falls through to Step 5 → ACL_OTHER → rwx → ALLOWED
Key insight: Once a process’s GID matches any group class entry, the decision is made within that step. ACL_OTHER is only used if NO group ID of the process matches ANY group class entry.
Step 4 — Multiple Group IDs Example

A process can have multiple groups. The kernel checks all of them against group class entries to find a match that grants the requested permission.

# File with ACL:
#   ACL_GROUP (GID 102): r--
#   ACL_GROUP (GID 103): -w-
#   ACL_MASK:            rwx

# Process GIDs: effective=102, supplementary=[103, 200]

# access(file, R_OK):
#   GID 102 matches ACL_GROUP → r-- AND rwx = r-- → READ OK → ALLOWED ✔

# access(file, W_OK):
#   GID 102: r-- (no write), GID 103: -w- (has write!) → ALLOWED ✔

# access(file, R_OK | W_OK):
#   No single ACL_GROUP entry grants BOTH r and w.
#   GID 102: only r. GID 103: only w.
#   Neither entry alone satisfies R+W → DENIED ✘

# This is the AND-masking + group matching subtlety!

Full Worked Examples

Example 1: Setup the ACL for all examples below
# Create a test file
touch testfile
chmod 640 testfile   # owner=rw, group=r, other=none

# Set up a rich ACL
setfacl -m u:alice:rwx,u:bob:r--,g:devs:rw-,g:qa:r--,m::rw-,o::--- testfile

# View the result
getfacl testfile
# user::rw-         ACL_USER_OBJ  (owner, say uid=1000)
# user:alice:rwx    ACL_USER for alice (uid=1001)
# user:bob:r--      ACL_USER for bob   (uid=1002)
# group::r--        ACL_GROUP_OBJ  (file group gid=50)
# group:devs:rw-    ACL_GROUP for devs (gid=60)
# group:qa:r--      ACL_GROUP for qa   (gid=70)
# mask::rw-         ACL_MASK
# other::---        ACL_OTHER
Example 2: Test access for different users
# Test as owner (uid=1000):
#   → Step 2 matches → ACL_USER_OBJ = rw- → rw- (NO MASKING)
#   access(R_OK|W_OK) = ALLOWED, X_OK = DENIED

# Test as alice (uid=1001):
#   → Step 3 matches (ACL_USER for alice = rwx)
#   → Effective = rwx AND mask(rw-) = rw-
#   → R_OK, W_OK = ALLOWED; X_OK = DENIED

# Test as bob (uid=1002):
#   → Step 3 matches (ACL_USER for bob = r--)
#   → Effective = r-- AND mask(rw-) = r--
#   → R_OK = ALLOWED; W_OK, X_OK = DENIED

# Test as user in 'devs' group (gid=60), not in any other listed group:
#   → Step 4: GID 60 matches ACL_GROUP for devs = rw-
#   → Effective = rw- AND mask(rw-) = rw-
#   → R_OK, W_OK = ALLOWED; X_OK = DENIED

# Test as user in file group (gid=50):
#   → Step 4: GID 50 matches ACL_GROUP_OBJ = r--
#   → Effective = r-- AND mask(rw-) = r--
#   → R_OK = ALLOWED; W_OK, X_OK = DENIED

# Test as completely unknown user (no group matches):
#   → Step 5: ACL_OTHER = ---
#   → ALL DENIED

Coding Examples

Example 3: C program — simulate ACL permission check manually
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/stat.h>
#include <sys/acl.h>
#include <acl/libacl.h>

/* Check if a permset satisfies requested access */
int check_perm(acl_permset_t permset, int requested) {
    int ok = 1;
    if (requested & R_OK) ok &= acl_get_perm(permset, ACL_READ);
    if (requested & W_OK) ok &= acl_get_perm(permset, ACL_WRITE);
    if (requested & X_OK) ok &= acl_get_perm(permset, ACL_EXECUTE);
    return ok;
}

/* Combine two permsets using AND (for masking) */
int check_masked(acl_permset_t permset, acl_permset_t mask, int requested) {
    int have_perm = 0;
    if (requested & R_OK)
        have_perm |= (acl_get_perm(permset, ACL_READ) &&
                      acl_get_perm(mask, ACL_READ)) ? R_OK : 0;
    if (requested & W_OK)
        have_perm |= (acl_get_perm(permset, ACL_WRITE) &&
                      acl_get_perm(mask, ACL_WRITE)) ? W_OK : 0;
    if (requested & X_OK)
        have_perm |= (acl_get_perm(permset, ACL_EXECUTE) &&
                      acl_get_perm(mask, ACL_EXECUTE)) ? X_OK : 0;
    return (have_perm == requested); /* All requested bits satisfied? */
}

int main(int argc, char *argv[]) {
    if (argc != 3) {
        fprintf(stderr, "Usage: %s filename access_mode\n", argv[0]);
        fprintf(stderr, "  access_mode: r=read, w=write, x=execute (combine: rw, rx, rwx)\n");
        return 1;
    }

    /* Parse requested access from argument */
    int requested = 0;
    for (char *p = argv[2]; *p; p++) {
        if (*p == 'r') requested |= R_OK;
        if (*p == 'w') requested |= W_OK;
        if (*p == 'x') requested |= X_OK;
    }

    /* Get current process IDs */
    uid_t my_uid = geteuid();
    gid_t my_gid = getegid();

    printf("Checking access to: %s\n", argv[1]);
    printf("Process UID: %d, GID: %d\n", (int)my_uid, (int)my_gid);
    printf("Requested: %s%s%s\n",
           requested & R_OK ? "read " : "",
           requested & W_OK ? "write " : "",
           requested & X_OK ? "execute " : "");
    printf("---\n");

    /* Step 1: root check */
    if (my_uid == 0) {
        printf("STEP 1: Root process → GRANT ALL (execute needs at least 1 x bit)\n");
        return 0;
    }

    /* Get file stat for owner info */
    struct stat st;
    if (stat(argv[1], &st) == -1) { perror("stat"); return 1; }

    /* Load ACL */
    acl_t acl = acl_get_file(argv[1], ACL_TYPE_ACCESS);
    if (!acl) { perror("acl_get_file"); return 1; }

    /* Find ACL_MASK permset first */
    acl_permset_t mask_permset = NULL;
    acl_entry_t e; acl_tag_t t;
    for (int id = ACL_FIRST_ENTRY; acl_get_entry(acl, id, &e) == 1; id = ACL_NEXT_ENTRY) {
        acl_get_tag_type(e, &t);
        if (t == ACL_MASK) { acl_get_permset(e, &mask_permset); break; }
    }

    /* Step 2: owner match */
    if (my_uid == st.st_uid) {
        printf("STEP 2: UID matches file owner\n");
        for (int id = ACL_FIRST_ENTRY; acl_get_entry(acl, id, &e) == 1; id = ACL_NEXT_ENTRY) {
            acl_get_tag_type(e, &t);
            if (t == ACL_USER_OBJ) {
                acl_permset_t ps; acl_get_permset(e, &ps);
                printf("Result: %s\n", check_perm(ps, requested) ? "GRANTED" : "DENIED");
                break;
            }
        }
        acl_free(acl);
        return 0;
    }

    printf("STEP 2: UID does not match owner, continuing...\n");

    /* Steps 3/4/5 left as exercise — shown conceptually above */
    printf("(Steps 3-5 check ACL_USER, group entries, and ACL_OTHER)\n");
    printf("Use 'access(\"%s\", %d)' for actual kernel check\n", argv[1], requested);

    acl_free(acl);
    return 0;
}
/* Compile: gcc -o acl_check acl_check.c -lacl */
Example 4: Shell script — test access for multiple users
#!/bin/bash
# Test file access for a list of users using 'su'
# Usage: ./test_access.sh filename

FILE="$1"
echo "=== ACL for $FILE ==="
getfacl "$FILE"
echo ""
echo "=== Access Test ==="

for USER in alice bob charlie ravi; do
    # Use runuser or su to check access as each user
    RESULT=$(su -s /bin/sh -c "test -r '$FILE' && echo YES || echo NO" "$USER" 2>/dev/null)
    printf "%-10s read: %s\n" "$USER" "$RESULT"
done
Example 5: Demonstrate the group-match-but-deny trap
# Create file and set ACL where group has limited access
# but 'other' has more

touch trap_demo.txt
chown ravi:devteam trap_demo.txt

# Group 'devteam' gets read-only, others get read+write
setfacl -m g:devteam:r--,o::rw- trap_demo.txt

getfacl trap_demo.txt
# group:devteam:r--
# other::rw-

# Now if a devteam member tries to WRITE:
# Step 4: GID matches devteam → ACL_GROUP entry = r--
# Write requested but only read granted → DENIED
# (Does NOT fall through to ACL_OTHER which would allow write!)
# This is the "group match trap" — once you're in step 4, you stay there

# Demonstration:
# su -c "echo hello >> trap_demo.txt" devteam_member
# -bash: trap_demo.txt: Permission denied

Interview Questions

Q1. What is the order of permission checking for ACLs in Linux?

The kernel checks in this order, stopping at the first match: (1) Privileged process → grant all (execute needs at least 1 x entry), (2) Process UID == file owner → use ACL_USER_OBJ, (3) Process UID matches ACL_USER entry qualifier → use that entry AND-masked with ACL_MASK, (4) Any process GID matches file group or ACL_GROUP qualifier → use matching entry AND-masked with ACL_MASK (deny if matched but access not granted), (5) Use ACL_OTHER.

Q2. Why is the ACL_MASK NOT applied to ACL_USER_OBJ (file owner) permissions?

ACL_USER_OBJ is not part of the “group class” (which consists of ACL_USER, ACL_GROUP_OBJ, ACL_GROUP). ACL_MASK only limits group class entries. The owner’s permissions are always applied directly without masking, ensuring the owner always has the access specified in ACL_USER_OBJ.

Q3. A user belongs to the file’s group AND the file has generous ACL_OTHER permissions. The group has limited ACL. Which wins?

The group permissions win. Once Step 4 is entered (because a group ID matched), the decision is made there. If the group class entry doesn’t grant the requested access, it’s DENIED — it does NOT fall through to Step 5 (ACL_OTHER). This is a common trap: group membership can actually reduce access compared to ACL_OTHER.

Q4. What is “AND-masking” in the context of ACL permission checking?

When a process is matched through a group class entry (ACL_USER, ACL_GROUP_OBJ, or ACL_GROUP), the effective permissions are computed as: entry_permissions AND ACL_MASK_permissions. This means even if an ACL_USER entry says rwx, if ACL_MASK says r--, the effective permission is only r--. ACL_MASK acts as a ceiling.

Q5. A process has GIDs 100 and 200. A file has ACL_GROUP(100)=r– and ACL_GROUP(200)=-w-. Can the process both read and write?

No. The access check looks for a single group class entry that grants ALL of the requested permissions. If the process requests R+W, it checks GID 100 (only read — insufficient) and GID 200 (only write — insufficient). No single entry satisfies R+W, so access is denied. The permissions from different group entries are NOT combined.

Q6. Which process IDs does Linux actually use during ACL permission checking?

Linux uses the process’s file-system user ID (fsuid) and file-system group IDs (fsgid), not the real UID/GID. In most cases these are the same as the effective UID/GID, but they can differ when using the setfsuid() / setfsgid() system calls (used mainly by NFS servers). The supplementary group IDs are also included in group checks.

Next: ACL Text Forms →

Learn how to read and write ACL entries in both long and short text format

Continue → ← Back: Overview

Leave a Reply

Your email address will not be published. Required fields are marked *