What Will You Learn?
Writing a daemon correctly goes beyond the creation steps. Since a daemon is meant to run for months or years, any bugs in resource management, signal handling, or synchronization become serious problems over time. This tutorial covers the essential programming guidelines for writing robust, production-quality daemons.
Key Concepts
When the system shuts down, the init process (PID 1) sends SIGTERM to all of its children. By default, SIGTERM terminates a process immediately. If a daemon needs to do cleanup before it exits (like saving state, closing database connections, flushing buffers, or removing lock files), it must set up a SIGTERM handler.
This 5-second window sounds short but remember โ init signals ALL processes at the same time. They all compete for CPU during shutdown. Design your cleanup to be fast: no complex computations, no network calls, just: flush, close, unlink, exit.
| t=0 init sends SIGTERM to all children |
t=0 to t=5s Daemon runs SIGTERM handler (cleanup window) |
t=5s init sends SIGKILL โ force kill, no escape |
#include <signal.h>
#include <stdio.h>
#include <unistd.h>
#include <syslog.h>
static volatile sig_atomic_t terminate = 0;
/*
* SIGTERM handler โ set a flag for the main loop.
* This handler must be AS-safe (async-signal safe).
* Do NOT call printf, malloc, or other non-safe functions here.
*/
void sigterm_handler(int sig)
{
terminate = 1; /* Set atomic flag โ main loop will check this */
}
int main(void)
{
struct sigaction sa;
sa.sa_handler = sigterm_handler;
sigemptyset(&sa.sa_mask);
sa.sa_flags = 0;
if (sigaction(SIGTERM, &sa, NULL) == -1) {
perror("sigaction");
return 1;
}
openlog("my_daemon", LOG_PID, LOG_DAEMON);
syslog(LOG_INFO, "Daemon started");
/* Main work loop */
while (!terminate) {
/* ... do daemon work ... */
sleep(1);
}
/* Cleanup โ runs within the 5-second SIGTERM window */
syslog(LOG_INFO, "Received SIGTERM, cleaning up...");
/* Fast cleanup:
* - Close database connections
* - Flush write buffers
* - Remove PID/lock files
* - Release shared memory */
unlink("/var/run/my_daemon.pid");
syslog(LOG_INFO, "Daemon exited cleanly");
closelog();
return 0;
}
For a regular short-lived program, a small memory leak is barely noticeable. But a daemon can run for months without restart. Even a tiny 100-byte memory leak per second means 8.6 MB leaked per day, and 3.1 GB per year. This can crash the system or cause the OOM killer to kill the daemon.
| Resource Type | What Leaks Look Like | Prevention |
|---|---|---|
| Memory | malloc() without free(), growing linked lists | valgrind, AddressSanitizer |
| File Descriptors | open() without close(), accept() without close() | Check /proc/self/fd count |
| Threads | pthread_create() without join/detach | pthread_detach() or join |
| Temp Files | Creating /tmp files without cleanup | unlink() in cleanup code |
/* BAD: File descriptor leak in a daemon loop */
void bad_daemon_loop(void)
{
while (1) {
/* BUG: fd opened but never closed!
* After many iterations, the process hits the FD limit
* (typically 1024) and open() starts returning EMFILE */
int fd = open("/var/log/data.log", O_WRONLY | O_APPEND);
write(fd, "tick\n", 5);
/* Missing: close(fd); */
sleep(1);
}
}
/* GOOD: Proper resource management */
void good_daemon_loop(void)
{
while (1) {
int fd = open("/var/log/data.log", O_WRONLY | O_APPEND);
if (fd == -1) {
syslog(LOG_ERR, "open failed: %m");
} else {
write(fd, "tick\n", 5);
close(fd); /* Always close what you open */
}
sleep(1);
}
}
/* Monitor FD usage โ run this periodically to detect leaks */
void check_fd_count(void)
{
int count = 0;
DIR *dir = opendir("/proc/self/fd");
if (dir) {
struct dirent *entry;
while ((entry = readdir(dir)) != NULL) count++;
closedir(dir);
syslog(LOG_DEBUG, "Open file descriptors: %d", count - 2); /* -2 for . and .. */
}
}
Many daemons must have exactly one running instance. Imagine two copies of cron running โ both would try to execute the same scheduled job at the same time, causing duplicate work, data corruption, or race conditions.
The standard technique for enforcing a single daemon instance is the PID lock file:
| Step 1 Open/create PID file /var/run/daemon.pid |
โ | Step 2 Try F_SETLK (non-blocking write lock) |
โ | Step 3 Lock acquired? Write PID to file |
โ FAIL: | Lock FAILED Another instance is running โ EXIT |
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <fcntl.h>
#include <unistd.h>
#include <sys/stat.h>
#define PID_FILE "/var/run/example_daemon.pid"
static int pid_fd = -1; /* Keep FD open to hold the lock */
/*
* create_pid_file() - Creates a PID lock file.
*
* If the file can be exclusively locked, we are the only instance.
* If locking fails, another instance holds the lock.
*
* Returns 0 on success (we are the only instance),
* -1 if another instance is already running.
*/
int create_pid_file(const char *path)
{
char buf[32];
struct flock fl;
pid_fd = open(path, O_RDWR | O_CREAT, S_IRUSR | S_IWUSR);
if (pid_fd == -1) {
perror("open PID file");
return -1;
}
/* Attempt a non-blocking exclusive write lock on the entire file */
fl.l_type = F_WRLCK;
fl.l_whence = SEEK_SET;
fl.l_start = 0;
fl.l_len = 0;
if (fcntl(pid_fd, F_SETLK, &fl) == -1) {
/* Lock failed โ another instance owns it */
/* Read the PID from the file to report who is running */
ssize_t n = read(pid_fd, buf, sizeof(buf) - 1);
if (n > 0) {
buf[n] = '\0';
fprintf(stderr, "Another instance is running with PID: %s\n", buf);
} else {
fprintf(stderr, "Another instance is running\n");
}
close(pid_fd);
pid_fd = -1;
return -1;
}
/* We have the lock. Write our PID. */
ftruncate(pid_fd, 0);
snprintf(buf, sizeof(buf), "%d\n", getpid());
write(pid_fd, buf, strlen(buf));
/* Note: We do NOT close pid_fd here.
* The lock is released when pid_fd is closed (on process exit).
* This means if the daemon crashes, the lock is automatically released. */
return 0;
}
void remove_pid_file(const char *path)
{
if (pid_fd != -1) {
close(pid_fd);
pid_fd = -1;
}
unlink(path);
}
int main(void)
{
/* Check for existing instance before daemonizing */
if (create_pid_file(PID_FILE) == -1) {
fprintf(stderr, "Daemon already running. Exiting.\n");
return 1;
}
/* ... proceed with daemon creation steps ... */
printf("Daemon running with PID %d\n", getpid());
printf("PID file: %s\n", PID_FILE);
/* Main loop */
sleep(30);
remove_pid_file(PID_FILE);
return 0;
}
A daemon that tracks its own memory and FD usage and logs a warning if usage grows unexpectedly โ useful for catching leaks in production.
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <dirent.h>
#include <syslog.h>
/*
* get_rss_kb() - Returns the current Resident Set Size in kilobytes.
* Reads from /proc/self/status.
*/
long get_rss_kb(void)
{
FILE *f = fopen("/proc/self/status", "r");
if (!f) return -1;
char line[128];
long rss = -1;
while (fgets(line, sizeof(line), f)) {
if (strncmp(line, "VmRSS:", 6) == 0) {
sscanf(line + 6, "%ld", &rss);
break;
}
}
fclose(f);
return rss;
}
/*
* get_open_fds() - Count the number of open file descriptors.
* Reads /proc/self/fd directory entries.
*/
int get_open_fds(void)
{
int count = 0;
DIR *dir = opendir("/proc/self/fd");
if (!dir) return -1;
struct dirent *entry;
while ((entry = readdir(dir)) != NULL)
if (entry->d_name[0] != '.') count++;
closedir(dir);
return count;
}
/*
* Resource watchdog โ call this periodically.
* Logs a warning if usage exceeds thresholds.
*/
void check_resources(long max_rss_kb, int max_fds)
{
long rss = get_rss_kb();
int fds = get_open_fds();
if (rss > 0) {
if (rss > max_rss_kb) {
syslog(LOG_WARNING,
"MEMORY WARNING: RSS=%ld KB exceeds limit %ld KB โ possible leak!",
rss, max_rss_kb);
} else {
syslog(LOG_DEBUG, "Memory usage: %ld KB", rss);
}
}
if (fds > 0) {
if (fds > max_fds) {
syslog(LOG_WARNING,
"FD WARNING: %d file descriptors open, limit=%d โ possible leak!",
fds, max_fds);
} else {
syslog(LOG_DEBUG, "Open FDs: %d", fds);
}
}
}
int main(void)
{
openlog("resource_daemon", LOG_PID, LOG_DAEMON);
int cycle = 0;
while (1) {
/* Check resources every 60 seconds */
if (cycle % 60 == 0) {
check_resources(50 * 1024, /* Warn if RSS > 50 MB */
100); /* Warn if more than 100 FDs open */
}
/* Simulate some work */
cycle++;
sleep(1);
}
closelog();
return 0;
}
Answer: A daemon runs continuously for days, months, or even years without restarting. Even a small leak of a few bytes per iteration accumulates to gigabytes over time, eventually exhausting memory, causing the system to invoke the OOM killer, and possibly crashing the daemon or other processes.
Answer: The daemon installs a SIGTERM handler using sigaction(). In the handler (which must be async-signal-safe), it sets a volatile sig_atomic_t flag. The main loop checks this flag after each iteration and, when set, performs cleanup (flushing buffers, closing connections, removing PID files) before calling exit().
Answer: The init process follows SIGTERM with SIGKILL after 5 seconds. SIGKILL cannot be caught, blocked, or ignored. The daemon will be killed immediately regardless of its state. Any cleanup that hasn’t finished will be abandoned, potentially leaving the system in an inconsistent state (unsaved data, orphaned temp files, unreleased locks).
Answer: The daemon creates/opens a file (e.g., /var/run/daemon.pid), then tries to acquire a non-blocking exclusive write lock (F_SETLK) on it. If the lock succeeds, it writes its PID and keeps the FD open (holding the lock). If locking fails, another instance holds the lock. The key benefit: if the daemon crashes, the OS automatically releases the lock when the process exits, so the next instance can start without manual cleanup.
Answer: A file descriptor leak occurs when a process opens files, sockets, or pipes but never closes them. The leaked FDs accumulate until the process hits the system limit (EMFILE). In Linux, you can detect this by monitoring /proc/self/fd โ count the entries. A healthy daemon has a small, stable FD count. If it grows monotonically, there is a leak. Tools like lsof and /proc/PID/fd can show which files are open.
Chapter Summary
Robust daemons handle SIGTERM gracefully (within 5 seconds), guard against memory and FD leaks through careful resource management, and enforce single-instance behavior using PID lock files with fcntl() advisory locks.
