DoS Defense, Return Status Checking, and Safe Failure

 

Denial-of-Service Attacks & Fail Safely
Chapter 38.10โ€“38.11 โ€” DoS Defense, Return Status Checking, and Safe Failure
๐Ÿ›ก๏ธ DoS Defense
โœ… Return Checks
๐Ÿ”’ Fail Safely

Denial-of-Service โ€” Making a Service Unavailable

A denial-of-service (DoS) attack aims to make a service unavailable to legitimate users โ€” not by stealing data, but by crashing it, overwhelming it, or locking its resources. As Internet-based services have grown, DoS attacks have become increasingly common and damaging.

There are two broad categories: crash attacks (sending malformed data to crash the server) and overload attacks (flooding the server with so many requests that it can’t serve legitimate clients).

๐Ÿ’ฅ Crash Attacks โ€” Malformed Input

A crash attack sends specially crafted malformed data to exploit bugs in input handling โ€” particularly buffer overruns, integer overflows, or unexpected input formats that cause the server to crash or enter an undefined state.

Defense is the same as for buffer overrun attacks: rigorous input validation and bounds checking (as covered in the previous sections). A server that crashes under malformed input is giving an attacker an easy denial-of-service vector.

๐ŸŒŠ Overload Attacks โ€” Flooding the Server

An overload attack sends legitimate-looking requests faster than the server can handle them. The source IP can be spoofed; a distributed attack (DDoS) enlists thousands of compromised machines. The server cannot prevent receiving these requests, but it can be designed to degrade gracefully rather than collapse.

โš–๏ธ
Load Throttling
When the load exceeds a threshold, start dropping incoming requests. Legitimate requests may be lost too, but the server remains functional rather than collapsing. This is far better than a complete outage for all users.
โฑ๏ธ
Client Timeouts
A server must use timeouts for all client communication. If a client connects but doesn’t send data, or sends data slowly (a slowloris attack), the server should disconnect rather than wait indefinitely โ€” tying up a connection slot.
๐Ÿ“Š
Log and Throttle Logging
Log overload events to notify administrators, but throttle the logging itself. If each request generates a log entry and you’re receiving 1 million bogus requests per second, logging itself will DoS the system.
๐Ÿ”ข
Algorithmic Complexity Attacks
Carefully crafted inputs can cause O(nยฒ) or worse algorithms to run for a very long time. For example, a naive binary tree degrades to a linked list with sorted input. Use balanced data structures (red-black trees, hash tables with random seeds) that resist such attacks.
๐Ÿ“‰
Resource Limits and Disk Quotas
Use setrlimit() to cap CPU time, memory, and file sizes per process/user. Use disk quotas to prevent a single user from filling the filesystem.

โœ… Always Check Return Statuses

Every system call and library function that can fail has a way to report failure โ€” usually a return value of -1 or NULL, with errno set to indicate the cause. Privileged programs must always check these return values.

This isn’t just good practice โ€” for privileged programs it’s a security requirement. An unexpected failure often means the environment has changed in a way that could lead to a security compromise if the program blunders ahead.

Examples of failures that even root programs can encounter:

fork() โ€” system process limit reached
open() โ€” read-only filesystem
chdir() โ€” directory doesn’t exist
malloc() โ€” OOM killer active
setuid() โ€” capabilities manipulated

Special Case: open() and Standard File Descriptors

A successful open() call deserves an extra check: if the standard FDs (0, 1, 2) were closed before exec (as discussed earlier), the first call to open() would return fd 0 (stdin) or 1 (stdout). A privileged program should verify the returned FD is not one of the reserved standard ones when this matters.

๐Ÿ’ป Code Example 1: Simple Server with Timeout and Load Throttling
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <sys/time.h>
#include <signal.h>
#include <errno.h>

#define PORT 8080
#define MAX_CLIENTS 100       /* Maximum concurrent connections */
#define CLIENT_TIMEOUT_SEC 10  /* Disconnect idle clients after 10 sec */
#define MAX_REQUEST_SIZE 4096  /* Reject requests larger than this */

static volatile int active_clients = 0;

/*
 * set_socket_timeout(): Set recv/send timeouts on a client socket.
 * If the client doesn't send data within the timeout, recv() returns 0.
 */
void set_socket_timeout(int sockfd, int seconds)
{
    struct timeval tv;
    tv.tv_sec = seconds;
    tv.tv_usec = 0;

    /* SO_RCVTIMEO: receive timeout */
    if (setsockopt(sockfd, SOL_SOCKET, SO_RCVTIMEO, &tv, sizeof(tv)) == -1) {
        perror("setsockopt SO_RCVTIMEO");
    }

    /* SO_SNDTIMEO: send timeout */
    if (setsockopt(sockfd, SOL_SOCKET, SO_SNDTIMEO, &tv, sizeof(tv)) == -1) {
        perror("setsockopt SO_SNDTIMEO");
    }
}

/*
 * handle_client(): Process one client request with all DoS defenses applied.
 */
void handle_client(int client_fd)
{
    char buf[MAX_REQUEST_SIZE + 1];
    ssize_t nbytes;

    /* Enforce timeout โ€” slow/non-responsive clients get disconnected */
    set_socket_timeout(client_fd, CLIENT_TIMEOUT_SEC);

    nbytes = recv(client_fd, buf, MAX_REQUEST_SIZE, 0);

    if (nbytes == -1) {
        if (errno == EWOULDBLOCK || errno == EAGAIN) {
            /* Client timed out โ€” this is the expected DoS defense */
            fprintf(stderr, "Client timed out after %d seconds\n", CLIENT_TIMEOUT_SEC);
        } else {
            perror("recv");
        }
        close(client_fd);
        return;
    }

    if (nbytes == 0) {
        /* Client disconnected */
        close(client_fd);
        return;
    }

    /* Null-terminate and validate input length */
    buf[nbytes] = '\0';

    if (nbytes >= MAX_REQUEST_SIZE) {
        /* Request too large โ€” reject without processing */
        const char *err = "ERROR: Request too large\n";
        send(client_fd, err, strlen(err), 0);
        close(client_fd);
        fprintf(stderr, "Rejected oversized request (%zd bytes)\n", nbytes);
        return;
    }

    /* Process the request (simplified) */
    printf("Received %zd bytes: %.50s...\n", nbytes, buf);
    const char *response = "OK\n";
    send(client_fd, response, strlen(response), 0);
    close(client_fd);
}

/*
 * Demonstrates the throttling check at accept() time.
 * In a real server, this would be in the main accept loop.
 */
int throttle_check(void)
{
    if (active_clients >= MAX_CLIENTS) {
        fprintf(stderr, "THROTTLE: %d clients active, dropping new connection\n",
                active_clients);
        return 0;  /* Drop this connection */
    }
    return 1;  /* Accept */
}

int main(void)
{
    printf("DoS-resilient server concepts demonstrated\n");
    printf("  Max concurrent clients: %d\n", MAX_CLIENTS);
    printf("  Client timeout: %d seconds\n", CLIENT_TIMEOUT_SEC);
    printf("  Max request size: %d bytes\n", MAX_REQUEST_SIZE);
    printf("\nKey defenses:\n");
    printf("  1. Timeout on all client I/O\n");
    printf("  2. Load throttling (max active clients)\n");
    printf("  3. Request size limit\n");
    printf("  4. No blocking wait โ€” all operations have timeouts\n");

    return 0;
}

๐Ÿ’ป Code Example 2: Comprehensive Return Status Checking
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/stat.h>
#include <string.h>
#include <errno.h>

/*
 * Demonstrates comprehensive return status checking in a privileged program.
 *
 * For each operation:
 *  1. Check the return value
 *  2. Log the error with context
 *  3. Take the safe action (usually: terminate or reject the request)
 */

/* Global flag โ€” set if we're in a state where continuing is unsafe */
static int unsafe_state = 0;

/*
 * safe_chdir(): Change directory with error checking.
 * Terminates the program if chdir fails โ€” proceeding with wrong CWD is unsafe.
 */
void safe_chdir(const char *path)
{
    if (chdir(path) == -1) {
        fprintf(stderr, "FATAL: chdir('%s') failed: %s\n", path, strerror(errno));
        fprintf(stderr, "Cannot continue โ€” filesystem state unknown.\n");
        exit(EXIT_FAILURE);
    }
    printf("[OK] Working directory: %s\n", path);
}

/*
 * safe_open_log(): Open a log file with careful FD checking.
 * Returns -1 and sets unsafe_state if the returned FD is a standard FD.
 */
int safe_open_log(const char *path)
{
    int fd;

    fd = open(path, O_WRONLY | O_CREAT | O_APPEND, 0640);
    if (fd == -1) {
        fprintf(stderr, "ERROR: open('%s') failed: %s\n", path, strerror(errno));
        return -1;
    }

    /* CRITICAL CHECK: ensure open() didn't give us a standard FD */
    /* This can happen if stdin/stdout/stderr were closed before exec */
    if (fd == 0 || fd == 1 || fd == 2) {
        fprintf(stderr, "SECURITY ERROR: open() returned standard fd %d!\n", fd);
        fprintf(stderr, "Standard FDs may have been closed before exec โ€” aborting.\n");
        close(fd);
        unsafe_state = 1;
        return -1;
    }

    printf("[OK] Log file opened: fd=%d\n", fd);
    return fd;
}

/*
 * safe_fork(): Fork with resource limit awareness.
 */
pid_t safe_fork(void)
{
    pid_t pid = fork();

    if (pid == -1) {
        /* fork() can fail even as root โ€” e.g., system process limit */
        fprintf(stderr, "WARNING: fork() failed: %s\n", strerror(errno));
        fprintf(stderr, "System may be under resource exhaustion attack.\n");
        /* Don't terminate โ€” log and drop this request */
        return -1;
    }

    return pid;
}

/*
 * safe_setuid(): Drop privilege with verification.
 */
void safe_setuid(uid_t target_uid)
{
    /* Attempt to drop privilege */
    if (setuid(target_uid) == -1) {
        fprintf(stderr, "FATAL: setuid(%d) failed: %s\n", (int)target_uid, strerror(errno));
        exit(EXIT_FAILURE);  /* Cannot continue with wrong privilege */
    }

    /* VERIFY the drop actually worked */
    if (getuid() != target_uid || geteuid() != target_uid) {
        fprintf(stderr, "FATAL: setuid verification failed! uid=%d euid=%d expected=%d\n",
                (int)getuid(), (int)geteuid(), (int)target_uid);
        exit(EXIT_FAILURE);
    }

    printf("[OK] Privilege dropped to UID=%d\n", (int)target_uid);
}

int main(void)
{
    printf("=== Comprehensive Return Status Checking Demo ===\n\n");

    /* 1. Change to known-safe working directory */
    safe_chdir("/tmp");

    /* 2. Open log file with FD safety check */
    int log_fd = safe_open_log("/tmp/secure_prog_test.log");
    if (log_fd == -1 || unsafe_state) {
        fprintf(stderr, "Failed to open log safely โ€” aborting.\n");
        return EXIT_FAILURE;
    }

    /* 3. Test fork resilience */
    printf("\nTesting fork...\n");
    pid_t pid = safe_fork();
    if (pid == 0) {
        /* Child process */
        printf("Child running (pid=%d)\n", (int)getpid());
        _exit(0);
    } else if (pid > 0) {
        /* Parent */
        printf("Forked child pid=%d\n", (int)pid);
        wait(NULL);
    }

    /* 4. All operations done โ€” clean up */
    close(log_fd);
    printf("\nAll operations completed with verified return status checking.\n");

    return 0;
}

๐Ÿ”’ The Fail-Safe Principle

When a privileged program encounters an unexpected situation โ€” a file doesn’t exist when it should, a system call fails unexpectedly, data doesn’t match expectations โ€” the correct behavior is usually to terminate or, for a server, to drop the request.

The temptation is to “work around” unexpected problems and continue. But any workaround requires making assumptions that may not hold in all circumstances โ€” and wrong assumptions in privileged code create security holes. It is always safer to:

For privileged utilities:
Terminate immediately with a clear error message. A crashed program is less dangerous than a compromised one.
For servers:
Log the unexpected event and drop the client’s request. Don’t crash โ€” but don’t attempt to guess at the right behavior.
Always:
Log enough information for the administrator to diagnose what happened. Silent failures are dangerous.

๐Ÿ“‹ Chapter 38 Complete Summary โ€” The Secure Programming Checklist
# Rule Key Mechanism
1 Avoid privilege unless absolutely necessary Use non-root alternatives
2 Operate with least privilege seteuid(), setresuid()
3 Drop privilege permanently when done setresuid(uid,uid,uid)
4 Drop privilege before exec() setresuid() then execve()
5 Never exec a shell with privilege Avoid system(), popen()
6 Close privileged FDs before exec close() or FD_CLOEXEC
7 Erase sensitive data immediately explicit_bzero()
8 Prevent core dumps setrlimit(RLIMIT_CORE,0)
9 Use capabilities instead of full root libcap, cap_set_proc()
10 Use chroot jail for daemons chroot() + chdir(“/”)
11 Beware signal-based TOCTTOU attacks sigprocmask(), open+fstat
12 Set restrictive umask umask(0177)
13 Use O_EXCL for atomic file creation open(O_CREAT|O_EXCL)
14 Use mkstemp() for temp files mkstemp() not /tmp/name
15 Sanitize environment variables clearenv() + setenv()
16 Validate all external inputs Check length, charset, range
17 Check standard FDs are open fstat(0/1/2) at startup
18 Avoid gets(), use snprintf() fgets(), snprintf(), strncpy()
19 Use timeouts and throttle under load SO_RCVTIMEO, max_clients
20 Check all return statuses; fail safely Always check -1/NULL/errno

๐Ÿ“Œ Key Terms

Denial of Service DDoS Load Throttling Client Timeout SO_RCVTIMEO Algorithmic Complexity Attack Return Status errno Fail Safely Fork Bomb Resource Limits
๐ŸŽฏ Interview Questions โ€” DoS and Fail Safely
Q1. What are the two types of DoS attacks and how do you defend against each?Crash attacks: send malformed data to crash the server. Defense: rigorous input validation and bounds checking. Overload attacks: flood with requests. Defense: load throttling, client timeouts, resource limits, and graceful degradation.

Q2. What is an algorithmic complexity attack?Crafting input data that causes a data structure or algorithm to perform at its worst case (e.g., feeding sorted data to a naive binary tree, converting it to O(n) insertion). Defense: use balanced/hash data structures, randomize hash seeds, and test performance under adversarial inputs.

Q3. Why should logging itself be throttled during an overload?If every incoming request generates a log entry and the server receives millions of bogus requests per second, writing millions of log entries will consume all disk I/O and storage, effectively allowing the DoS attack to also destroy the logging system and fill the disk.

Q4. Why is checking the return value of open() for standard FDs important?If stdin/stdout/stderr were closed before exec, the first open() call returns fd 0, 1, or 2. Code that then calls write(1, …) thinking it’s writing to stdout is actually writing to the just-opened file. This can corrupt a sensitive file or expose sensitive data.

Q5. When a privileged program encounters an unexpected situation, what should it do?Usually terminate (for a utility) or drop the request and log (for a server). Attempting to “work around” the unexpected state requires making assumptions that may create security holes. A terminated program is safer than a compromised one.

Q6. Why can setuid() fail even for a process running as root?If the process has explicitly manipulated its capabilities (e.g., dropped CAP_SETUID), the setuid() call may fail or silently change only some of the UIDs. This is why return values must always be checked AND the actual UID state must be verified with getresuid() afterward.

Chapter 38 Complete!
You have covered all sections of Writing Secure Privileged Programs

โ† Start Over EmbeddedPathashala Home

Leave a Reply

Your email address will not be published. Required fields are marked *