Memory Erasure, Core Dumps, and mlock()

 

Avoid Exposing Sensitive Information
Chapter 38.4 โ€” Memory Erasure, Core Dumps, and mlock()
๐Ÿ”’ Memory Erasure
๐Ÿ’พ Core Dumps
๐Ÿ” mlock()

Why Sensitive Data in Memory is Dangerous

When your program reads a password, an encryption key, or other sensitive information, it stores it in a memory buffer. Most programmers use that data and move on without thinking about what happens to the memory afterward. But the data doesn’t just disappear โ€” it stays in RAM until something else overwrites it. In the meantime, several paths can expose it to an attacker.

๐Ÿ” How Sensitive Data Leaks โ€” The Threat Model

Once sensitive data is in a process’s virtual memory, it can be exposed through these paths:

๐Ÿ’พ
Swap File Exposure
The OS can swap out a virtual memory page to disk to free RAM. Once in the swap area, the sensitive data is stored in a plain file on disk. A privileged attacker (or another root process) can read the swap partition directly and extract the password.
๐Ÿ’ฅ
Core Dump Files
When a process receives certain signals (SIGSEGV, SIGBUS, etc.), the kernel can write a core dump file containing a complete snapshot of the process’s memory. Any sensitive data in memory โ€” passwords, keys โ€” appears verbatim in this file.
๐Ÿ”ฌ
/proc/[pid]/mem
On Linux, a process’s memory can be read via /proc/[pid]/mem by processes with appropriate privilege (e.g., ptrace access). Sensitive data left in memory is accessible.

๐Ÿงน Rule 1: Erase Sensitive Data from Memory Immediately After Use

The moment you’re done using sensitive data (a password, a key), overwrite the buffer with zeros. Don’t rely on the program exiting to “clean up” โ€” the data may have been swapped out before exit.

There’s a subtle problem: a smart C compiler may optimize away a plain memset() if it analyzes that the buffer is never read again. The solution is to use memset() followed by a memory barrier, or use OS-specific “secure” erasure functions.

#include <string.h>
#include <stdlib.h>

/*
 * secure_erase(): Zero out a buffer in a way the compiler won't optimize away.
 *
 * A plain memset() on a buffer that is "dead" (never read again) may be
 * removed by an optimizing compiler. We use a volatile pointer trick.
 */
void secure_erase(void *buf, size_t len)
{
    /* Cast to volatile โ€” compiler must not optimize this away */
    volatile unsigned char *p = (volatile unsigned char *)buf;
    while (len--) {
        *p++ = 0;
    }
}

/* Alternative: on glibc, explicit_bzero() is guaranteed not to be optimized away */
/* void secure_erase(void *buf, size_t len) { explicit_bzero(buf, len); } */

int main(void)
{
    char password[256];

    /* Read password from user */
    printf("Enter password: ");
    if (fgets(password, sizeof(password), stdin) == NULL)
        return 1;

    /* ... use the password here (hash it, verify it, etc.) ... */
    printf("Password processed (length %zu)\n", strlen(password));

    /* IMPORTANT: Erase it from memory immediately after use */
    secure_erase(password, sizeof(password));

    printf("Password securely erased from memory.\n");

    /* At this point, even if the process core dumps, */
    /* the password won't be in the dump */

    return 0;
}

On modern Linux with glibc, explicit_bzero() is the preferred function. OpenSSL provides OPENSSL_cleanse(). On Windows, SecureZeroMemory(). All serve the same purpose: guaranteed memory erasure that the compiler cannot optimize away.

๐Ÿ’ฅ Rule 2: Prevent Core Dumps in Security-Sensitive Programs

A core dump is a complete snapshot of a process’s memory written to a file when the process crashes. For a program handling passwords or keys, this is a serious leak โ€” all sensitive data in memory lands in a file on disk.

A program can disable core dumps by setting the RLIMIT_CORE resource limit to 0 using setrlimit().

Note: Linux by default doesn’t permit a set-UID program to core dump (even after dropping privileges), but other UNIX systems may not have this protection. Explicitly disabling core dumps is the safe, portable approach.

#include <sys/resource.h>
#include <stdio.h>
#include <stdlib.h>

/*
 * disable_core_dumps(): Prevent the process from creating core dump files.
 *
 * This should be called early in main(), before any sensitive data is loaded.
 */
void disable_core_dumps(void)
{
    struct rlimit rl;

    /* Set core dump size limit to zero */
    rl.rlim_cur = 0;   /* soft limit */
    rl.rlim_max = 0;   /* hard limit */

    if (setrlimit(RLIMIT_CORE, &rl) == -1) {
        perror("setrlimit RLIMIT_CORE");
        exit(EXIT_FAILURE);
    }

    printf("[Security] Core dumps disabled.\n");
}

/*
 * verify_core_dumps_disabled(): Check that core dumps are actually off.
 */
void verify_core_dumps_disabled(void)
{
    struct rlimit rl;

    if (getrlimit(RLIMIT_CORE, &rl) == -1) {
        perror("getrlimit");
        return;
    }

    if (rl.rlim_cur == 0) {
        printf("[OK] Core dump limit: disabled (soft=0, hard=%llu)\n",
               (unsigned long long)rl.rlim_max);
    } else {
        fprintf(stderr, "[WARNING] Core dumps still enabled! soft=%llu hard=%llu\n",
                (unsigned long long)rl.rlim_cur,
                (unsigned long long)rl.rlim_max);
    }
}

int main(void)
{
    /* Step 1: Disable core dumps at program startup */
    disable_core_dumps();
    verify_core_dumps_disabled();

    /* Now it's safe to load sensitive data */
    char sensitive_key[] = "my-secret-key-12345";
    printf("Loaded sensitive key (will not appear in core dump).\n");

    /* Use the key... then erase it */
    volatile char *p = (volatile char *)sensitive_key;
    for (size_t i = 0; i < sizeof(sensitive_key); i++) p[i] = 0;

    return 0;
}

๐Ÿ” Rule 3: Lock Sensitive Memory to Prevent Swapping

Even with core dumps disabled, sensitive data can end up on disk via the swap partition. The mlock() system call locks memory pages in RAM, preventing the OS from swapping them out. mlockall() locks all current and future pages of the process.

This requires CAP_IPC_LOCK capability (or root privilege), or a sufficient RLIMIT_MEMLOCK resource limit. Use it only for the specific buffers that contain sensitive data โ€” locking all memory is wasteful and requires root.

#include <sys/mman.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#define KEY_SIZE 32

/*
 * Demonstrates locking a specific buffer in memory to prevent swapping.
 * This requires appropriate privileges (CAP_IPC_LOCK or root).
 */
int main(void)
{
    char *key_buffer;
    long page_size;

    /* mlock() works on page granularity โ€” allocate aligned */
    page_size = sysconf(_SC_PAGESIZE);
    printf("System page size: %ld bytes\n", page_size);

    /* Allocate a page-aligned buffer for the sensitive key */
    if (posix_memalign((void **)&key_buffer, page_size, page_size) != 0) {
        perror("posix_memalign");
        return 1;
    }

    /* Lock the page in memory โ€” kernel will NOT swap it out */
    if (mlock(key_buffer, page_size) == -1) {
        perror("mlock (may need CAP_IPC_LOCK or root)");
        /* Continue anyway for demo โ€” in production, this should be fatal */
    } else {
        printf("[OK] Memory page locked โ€” will not be swapped.\n");
    }

    /* Now use the buffer for sensitive data */
    memcpy(key_buffer, "my-super-secret-encryption-key!", KEY_SIZE);
    printf("Sensitive key stored in locked memory.\n");

    /* ... use the key ... */

    /* When done: erase the buffer */
    volatile char *p = (volatile char *)key_buffer;
    for (int i = 0; i < page_size; i++) p[i] = 0;
    printf("Key erased.\n");

    /* Unlock and free */
    munlock(key_buffer, page_size);
    free(key_buffer);

    return 0;
}

๐Ÿ“Œ Key Terms

Memory Erasure explicit_bzero() volatile Core Dump RLIMIT_CORE setrlimit() mlock() mlockall() Swap Exposure CAP_IPC_LOCK posix_memalign()
๐ŸŽฏ Interview Questions โ€” Sensitive Information
Q1. Why might a plain memset() not reliably erase sensitive data?An optimizing compiler may determine that the buffer is never read after memset() and remove the call entirely as a “dead store.” This leaves the sensitive data in memory. Use explicit_bzero(), volatile pointer writes, or OS-specific functions that the compiler is prohibited from removing.

Q2. What is a core dump and why is it a security risk?A core dump is a file the kernel creates when a process crashes, containing a complete copy of the process’s memory. Any sensitive data (passwords, keys) in memory at crash time appears verbatim in this file, which can be read by anyone with access to it.

Q3. How do you disable core dumps programmatically?Call setrlimit(RLIMIT_CORE, &rl) with both rlim_cur and rlim_max set to 0. This should be done at the very beginning of main(), before any sensitive data is loaded.

Q4. What does mlock() do and when should you use it?mlock(addr, len) locks specified memory pages in RAM, preventing the kernel from swapping them to disk. Use it for buffers holding passwords, encryption keys, or other sensitive data that must never appear in the swap partition.

Q5. Does Linux automatically prevent set-UID programs from creating core dumps?Yes โ€” by default, Linux doesn’t permit a set-UID program to core dump in response to a signal. However, other UNIX systems may not provide this protection, so explicitly disabling core dumps via setrlimit() is the portable and recommended practice.

Next: Confining the Process โ†’
Linux capabilities, chroot jails, virtual servers

Continue to Part 5 โ† Part 3

Leave a Reply

Your email address will not be published. Required fields are marked *