Routing Log Messages to Files, Consoles & Remote Hosts

 

/etc/syslog.conf
Chapter 37 – Part 7: Routing Log Messages to Files, Consoles & Remote Hosts
πŸ“„ syslog.conf Syntax
πŸ—‚οΈ Selector Rules
🌐 Remote Logging

What Will You Learn?

The /etc/syslog.conf file is the configuration file for the syslogd daemon. It controls exactly where each log message goes β€” which file, which console, which user’s terminal, or which remote server. Understanding this file lets you route your daemon’s messages to the right place and troubleshoot why certain messages are not appearing where you expect.

Key Concepts

/etc/syslog.conf Selector Rule facility.priority Wildcard (*) Action Field Remote Logging (@host) /dev/console rsyslog

1. File Structure β€” Lines, Selectors, and Actions

The /etc/syslog.conf file consists of lines, each containing two fields separated by whitespace (spaces or tabs):

facility.priority[;facility.priority;…]Β Β Β Β action
Selector Field (left)

Specifies which messages to match: facility.priority. Multiple selectors can be joined with semicolons.

Example: kern.err matches kernel error messages (and above).

Action Field (right)

Specifies where to send matched messages: a file path, a terminal, a username, or a remote host.

Example: /var/log/kern.log or @192.168.1.1

Lines beginning with # are comments. Blank lines are ignored.

2. Selector Syntax β€” facility.priority

The Facility Part

The facility name matches messages from that source category. The wildcard * matches all facilities.

Selector Example Matches
kern.* All kernel messages at any level
auth.info auth facility, INFO and above (INFO, NOTICE, WARNING, ERR, CRIT, ALERT, EMERG)
*.err ERR and above from ALL facilities
daemon.debug daemon facility at ALL levels (DEBUG is lowest, so all messages match)
local0.* All messages from local0 facility (your custom daemon)
*.* ALL messages from ALL facilities at ALL levels

The Priority Part β€” Important Subtlety

⚠️ Priority matching is “this level AND ABOVE”, not just “this level”.
daemon.err matches ERR, CRIT, ALERT, and EMERG β€” not just ERR.
This is because higher-severity messages are more important and should go everywhere that lower-severity messages go.

Special priority keywords:

* All priority levels (same as debug)
none No messages from this facility β€” used to exclude a facility from a wildcard selector
=priority Exactly this priority level only (BSD extension, supported by many syslogd)
!priority All levels EXCEPT this level (BSD extension)

3. Action Types β€” Where to Send Messages
Action Syntax Destination Notes
/var/log/syslog Regular file (path starts with /) Most common. Messages appended to file.
-/var/log/debug Regular file, no sync Leading – means do NOT fsync after each write. Faster but may lose messages on crash.
/dev/console System console Messages appear on the physical console screen
/dev/tty6 Named terminal Messages sent to that specific terminal
@192.168.1.10 Remote syslog server (UDP) Sends to remote host on UDP port 514
@@log.example.com Remote syslog server (TCP) rsyslog extension: TCP for reliable delivery
ravi,root Logged-in users Message written to terminal of each listed user (if logged in)
* All logged-in users Broadcast to all terminals with logged-in users

4. A Typical /etc/syslog.conf β€” Annotated
# /etc/syslog.conf β€” Complete annotated example
# Format: facility.priority [;facility.priority;...]   action
# Priority matching: "this level AND ABOVE"

# ──────────────────────────────────────────────────────────────────
# Kernel messages
# ──────────────────────────────────────────────────────────────────

# All kernel messages β†’ kernel log file
kern.*                          /var/log/kern.log

# Kernel errors and above β†’ also to console
kern.err                        /dev/console

# ──────────────────────────────────────────────────────────────────
# Authentication messages (login, sudo, sshd)
# ──────────────────────────────────────────────────────────────────

# All auth messages β†’ secure log (read by root only, mode 600)
auth,authpriv.*                 /var/log/auth.log

# ──────────────────────────────────────────────────────────────────
# System daemon messages
# ──────────────────────────────────────────────────────────────────

# Daemon info and above β†’ daemon log
daemon.*                        /var/log/daemon.log

# ──────────────────────────────────────────────────────────────────
# Mail system
# ──────────────────────────────────────────────────────────────────

# All mail logs β€” leading '-' disables fsync for performance
mail.*                          -/var/log/mail.log
mail.err                        /var/log/mail.err

# ──────────────────────────────────────────────────────────────────
# Cron daemon
# ──────────────────────────────────────────────────────────────────

cron.*                          /var/log/cron.log

# ──────────────────────────────────────────────────────────────────
# Debug messages (all facilities, debug level only)
# Exclude auth (security-sensitive) and mail (noisy)
# ──────────────────────────────────────────────────────────────────

# Multiple selectors: match daemon.debug BUT NOT auth or mail
*.debug;auth.none;mail.none     -/var/log/debug

# ──────────────────────────────────────────────────────────────────
# General syslog β€” catch-all for info and above
# Exclude kern (already in kern.log) and auth (already in auth.log)
# ──────────────────────────────────────────────────────────────────

*.info;kern.none;auth.none;mail.none    /var/log/syslog

# ──────────────────────────────────────────────────────────────────
# Emergency messages β€” broadcast to ALL logged-in users
# ──────────────────────────────────────────────────────────────────

*.emerg                         *

# ──────────────────────────────────────────────────────────────────
# Your custom daemon using LOG_LOCAL0
# ──────────────────────────────────────────────────────────────────

local0.*                        /var/log/myapp.log
local0.err                      /var/log/myapp.err

# ──────────────────────────────────────────────────────────────────
# Remote logging β€” forward all errors to central log server
# ──────────────────────────────────────────────────────────────────

*.err                           @192.168.1.100
# OR using TCP (rsyslog):
# *.err                         @@log.example.com

5. The “none” Keyword β€” Excluding Facilities

The none keyword is used to explicitly exclude a facility from a selector. This is essential when you have a catch-all selector like *.* but want to exclude certain facilities that go to their own dedicated files.

# Scenario: You want ALL messages in /var/log/everything.log
# but kernel messages should NOT go there (they're in kern.log)
# and auth messages should NOT go there (they're in auth.log, mode 600)

# WRONG β€” sends everything including sensitive auth messages:
*.*                             /var/log/everything.log

# CORRECT β€” exclude kern and auth using .none:
*.*;kern.none;auth.none         /var/log/everything.log

# Another example: debug-level messages from all except mail:
*.debug;mail.none               -/var/log/debug.log
Remember the semicolon: Multiple selectors are separated by semicolons on the same line. All must match (they are evaluated left to right, with exclusions). Example: *.info;kern.none;auth.none means “all info and above, but not kern, and not auth”.

πŸ’» Code Example 1: Custom Daemon with Dedicated Log File

Complete example: a daemon that uses LOG_LOCAL1, combined with syslog.conf configuration to route its messages to a dedicated file.

/* myservice_daemon.c
 * Uses LOG_LOCAL1 facility so syslog.conf can route it separately */
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <signal.h>
#include <syslog.h>
#include <fcntl.h>
#include <sys/stat.h>

static volatile sig_atomic_t running = 1;
static volatile sig_atomic_t reload  = 0;

void handle_sigterm(int s) { running = 0; }
void handle_sighup(int s)  { reload  = 1; }

void setup_signals(void)
{
    struct sigaction sa;
    sigemptyset(&sa.sa_mask);
    sa.sa_flags   = SA_RESTART;

    sa.sa_handler = handle_sigterm;
    sigaction(SIGTERM, &sa, NULL);

    sa.sa_handler = handle_sighup;
    sigaction(SIGHUP, &sa, NULL);

    /* Ignore SIGPIPE β€” common in network daemons */
    sa.sa_handler = SIG_IGN;
    sigaction(SIGPIPE, &sa, NULL);
}

int main(void)
{
    /* Open syslog BEFORE daemonizing (before close all fds) */
    openlog("myservice", LOG_PID | LOG_NDELAY, LOG_LOCAL1);
    syslog(LOG_NOTICE, "myservice starting");

    /* ---- Daemonize ---- */
    pid_t pid = fork();
    if (pid < 0) { syslog(LOG_ERR, "fork: %m"); return 1; }
    if (pid > 0) _exit(0);

    if (setsid() == -1) { syslog(LOG_ERR, "setsid: %m"); return 1; }

    pid = fork();
    if (pid < 0) { syslog(LOG_ERR, "fork2: %m"); return 1; }
    if (pid > 0) _exit(0);

    umask(0);
    chdir("/");

    int devnull = open("/dev/null", O_RDWR);
    dup2(devnull, 0); dup2(devnull, 1); dup2(devnull, 2);
    if (devnull > 2) close(devnull);
    /* ---- End Daemonize ---- */

    setup_signals();

    syslog(LOG_INFO, "myservice daemon running (PID=%d)", getpid());
    syslog(LOG_DEBUG, "Debug messages visible only if syslog.conf routes local1.debug");

    int cycle = 0;
    while (running) {
        if (reload) {
            reload = 0;
            syslog(LOG_INFO, "SIGHUP received β€” reloading configuration");
            /* load_config() here */
        }

        if (cycle % 10 == 0)
            syslog(LOG_INFO, "Service heartbeat, cycle=%d", cycle);

        if (cycle % 50 == 0)
            syslog(LOG_DEBUG, "Detailed debug: cycle=%d pid=%d", cycle, getpid());

        cycle++;
        sleep(1);
    }

    syslog(LOG_NOTICE, "myservice stopped (cycles=%d)", cycle);
    closelog();
    return 0;
}

Add to /etc/syslog.conf (then send SIGHUP to syslogd):

# Route all myservice messages to its own log file
local1.*                        /var/log/myservice.log

# Route only errors to the general error log as well
local1.err                      /var/log/error.log
# Compile and run
gcc -o myservice_daemon myservice_daemon.c
sudo ./myservice_daemon

# Apply syslog.conf changes (reload syslogd):
sudo kill -HUP $(pidof syslogd)
# OR on systemd:
sudo systemctl reload rsyslog

# Monitor the dedicated log:
sudo tail -f /var/log/myservice.log

# Send SIGHUP to the daemon (reload config):
sudo kill -HUP $(pidof myservice_daemon)

# Graceful stop:
sudo kill -TERM $(pidof myservice_daemon)

πŸ’» Code Example 2: Programmatically Testing syslog.conf Routing

A utility that sends one test message at every facility/level combination to help verify your syslog.conf routing is working correctly.

#include <stdio.h>
#include <syslog.h>
#include <unistd.h>

/* All facility names and their codes */
static struct { const char *name; int code; } facilities[] = {
    {"kern",    LOG_KERN},
    {"user",    LOG_USER},
    {"mail",    LOG_MAIL},
    {"daemon",  LOG_DAEMON},
    {"auth",    LOG_AUTH},
    {"syslog",  LOG_SYSLOG},
    {"lpr",     LOG_LPR},
    {"news",    LOG_NEWS},
    {"cron",    LOG_CRON},
    {"local0",  LOG_LOCAL0},
    {"local1",  LOG_LOCAL1},
    {"local2",  LOG_LOCAL2},
    {"local3",  LOG_LOCAL3},
    {"local4",  LOG_LOCAL4},
    {"local5",  LOG_LOCAL5},
    {"local6",  LOG_LOCAL6},
    {"local7",  LOG_LOCAL7},
    {NULL, 0}
};

/* All priority names and their codes */
static struct { const char *name; int code; } priorities[] = {
    {"emerg",   LOG_EMERG},
    {"alert",   LOG_ALERT},
    {"crit",    LOG_CRIT},
    {"err",     LOG_ERR},
    {"warning", LOG_WARNING},
    {"notice",  LOG_NOTICE},
    {"info",    LOG_INFO},
    {"debug",   LOG_DEBUG},
    {NULL, 0}
};

int main(int argc, char *argv[])
{
    printf("syslog routing test β€” sending to all facilities and priorities\n");
    printf("PID=%d\n\n", getpid());
    printf("Check /var/log/* to see where each message was routed.\n\n");

    for (int f = 0; facilities[f].name != NULL; f++) {
        openlog("syslog_test", LOG_PID | LOG_NDELAY, facilities[f].code);

        for (int p = 0; priorities[p].name != NULL; p++) {
            syslog(facilities[f].code | priorities[p].code,
                   "TEST: facility=%s priority=%s β†’ check routing",
                   facilities[f].name, priorities[p].name);
        }

        closelog();
        usleep(10000);  /* Small delay to avoid flooding */
    }

    printf("Done. Search each log file:\n");
    printf("  grep 'syslog_test' /var/log/syslog | head\n");
    printf("  grep 'syslog_test' /var/log/kern.log | head\n");
    printf("  sudo grep 'syslog_test' /var/log/auth.log | head\n");
    printf("  grep 'syslog_test' /var/log/daemon.log | head\n");

    return 0;
}
gcc -o syslog_routing_test syslog_routing_test.c
sudo ./syslog_routing_test

# See all facilities and levels that went to syslog:
grep "syslog_test" /var/log/syslog | awk '{print $5, $6}' | sort | uniq

# Count messages by destination to verify routing:
sudo grep -c "syslog_test" /var/log/*.log 2>/dev/null

🎯 Interview Questions β€” /etc/syslog.conf
Q1. In syslog.conf, does daemon.err match only ERR messages, or more?
Answer: It matches ERR and all higher-severity levels: CRIT, ALERT, and EMERG. The priority in syslog.conf is “this level and above”. Lower number = higher priority (EMERG=0 is highest). daemon.err captures all daemon messages with priority ≀ 3 (ERR level value). To match exactly one level, use =err (BSD extension).
Q2. What does the leading dash (-) before a filename action mean?
Answer: The leading dash (e.g., -/var/log/debug) tells syslogd to NOT call fsync() after writing each message to that file. Without the dash, syslogd calls fsync() after every write to ensure the message is on disk. The no-sync mode is much faster (especially for high-volume DEBUG messages) but risks losing the last few messages if the system crashes before the kernel writes the buffers to disk.
Q3. How do you route messages from your custom daemon to a dedicated log file?
Answer: Use one of the LOG_LOCAL0 to LOG_LOCAL7 facilities in your daemon’s openlog() call (e.g., LOG_LOCAL1). Then in /etc/syslog.conf, add a line like: local1.* /var/log/myapp.log. Send SIGHUP to syslogd to reload the configuration. Now all messages from your daemon go to /var/log/myapp.log.
Q4. What is the purpose of the “none” priority in syslog.conf?
Answer: “none” is used to exclude a facility from a multi-facility selector. For example, *.info;kern.none;auth.none matches all info-and-above messages except those from the kern and auth facilities. This prevents, say, sensitive auth messages from also being written to the general syslog file.
Q5. How do you configure syslogd to forward log messages to a remote host?
Answer: Use @hostname or @IP as the action. For example: *.err @192.168.1.100 forwards all error-and-above messages to that host’s syslogd on UDP port 514. For TCP (rsyslog), use @@hostname. The remote host must have syslogd configured to accept UDP/TCP connections (-r flag for classic syslogd, or the imudp/imtcp modules for rsyslog).
Q6. How do you apply changes to /etc/syslog.conf without restarting syslogd?
Answer: Send SIGHUP to syslogd: sudo kill -HUP $(pidof syslogd). syslogd follows the convention of using SIGHUP to reload its configuration file. On modern systemd-based systems running rsyslog, you can also use sudo systemctl reload rsyslog. The daemon re-reads syslog.conf and applies the new routing rules without losing any messages in transit.

πŸ“š Chapter 37 Complete β€” Quick Reference Summary
Topic Key Points
Daemon Definition Long-lived, no controlling terminal, parent=init
Creation Steps forkβ†’setsidβ†’forkβ†’umask(0)β†’chdir(“/”)β†’close FDsβ†’/dev/null
SIGTERM Handle for clean shutdown; 5-second window before SIGKILL
SIGHUP Convention: reload config; set flag in handler, process in main loop
syslog Facility LOG_DAEMON for system daemons; LOG_LOCAL0-7 for custom apps
syslog Priority EMERG(0) highest β†’ DEBUG(7) lowest
openlog() Always use LOG_PID; LOG_NDELAY before fork; LOG_PERROR for testing
setlogmask() LOG_UPTO() for production; expand to LOG_DEBUG for troubleshooting
syslog.conf facility.priority β†’ action; .none excludes; – prefix skips fsync

Chapter 37 Complete!

You have now mastered Linux Daemons: what they are, how to create them, how to write robust ones, and how to use the syslog system for logging. These are essential skills for any Linux system programmer.

← Back to Overview ← Review syslog API

Leave a Reply

Your email address will not be published. Required fields are marked *