dlopen API: Runtime Dynamic Loading Load shared libraries explicitly at runtime

 

Chapter 41 – Part 6 of 6
dlopen API: Runtime Dynamic Loading
Load shared libraries explicitly at runtime — the foundation of plugin architectures, codecs, and driver hot-loading
3
Code Examples
10
Interview Q&A
~25
min read

What Is dlopen?

So far, all shared library loading we have discussed is implicit — the dynamic linker loads libraries automatically at program startup because they are listed in the executable’s DT_NEEDED section.

dlopen gives programs the ability to load a shared library explicitly at any point during program execution. The program can then call dlsym() to get a pointer to any function or variable in the library, use it, and later call dlclose() to unload the library.

This is the foundation for: plugin architectures (like GStreamer codecs, VST audio plugins, Linux kernel modules via kmod), driver hot-loading, language interpreters loading native extensions, and embedded firmware with updateable modules.

Key Terms
dlopen() dlsym() dlclose() dlerror() RTLD_LAZY RTLD_NOW RTLD_GLOBAL RTLD_LOCAL RTLD_NEXT plugin interface -ldl

1. The dlopen API — Four Functions

All four functions are declared in <dlfcn.h>. Link with -ldl.

void *dlopen(const char *filename, int flags);

Opens the shared library named by filename. Returns an opaque handle on success, NULL on failure. If filename is NULL, returns a handle for the main executable itself. flags must include either RTLD_LAZY or RTLD_NOW, optionally OR’d with RTLD_GLOBAL or RTLD_LOCAL.

void *dlsym(void *handle, const char *symbol);

Returns the address of the symbol named symbol in the library identified by handle. Returns NULL if not found (but NULL can also be a valid symbol value — always use dlerror() to distinguish). Works for functions AND variables.

int dlclose(void *handle);

Decrements the reference count of the library. When the count reaches zero, the library is unloaded from the process (its destructor runs, its pages are unmapped). Returns 0 on success, non-zero on error. After dlclose, do NOT use any function pointers obtained from that library.

char *dlerror(void);

Returns a human-readable error message for the most recent failed dlopen/dlsym/dlclose call, or NULL if no error has occurred since the last call to dlerror(). Always call dlerror() once to clear it, then after each dl* call to check for errors.

2. dlopen Flags

Flag Meaning
RTLD_LAZY Resolve symbols lazily — only when actually called. Faster dlopen.
RTLD_NOW Resolve all symbols immediately. Fails if any undefined symbol. Safer for plugins.
RTLD_GLOBAL Symbols from this library are available for resolving symbols in subsequently loaded libraries.
RTLD_LOCAL Symbols from this library are NOT made available globally (default).
RTLD_NOLOAD Check if library is already loaded without actually loading it. Returns NULL if not loaded.
RTLD_NODELETE Do not unload on dlclose — useful for libraries with non-reversible side effects.

Code Example 1 — Basic dlopen/dlsym/dlclose

The simplest possible use case: load a shared library, get a function pointer, call it, then unload.

/* plugin_math.c — a simple plugin shared library */
#include <stdio.h>
#include <math.h>

/* Plugin entry point — every plugin exports this */
const char *plugin_name(void) {
    return "MathPlugin v1.0";
}

double plugin_compute(double x) {
    /* This plugin computes: sin²(x) + cos²(x) — always 1.0! */
    return sin(x) * sin(x) + cos(x) * cos(x);
}

void plugin_info(void) {
    printf("MathPlugin: computes sin²(x) + cos²(x)\n");
}
# Build the plugin as a shared library (no soname needed for explicit load)
gcc -g -shared -fPIC -o plugin_math.so plugin_math.c -lm
/* main_dlopen.c — application that loads the plugin at runtime */
#include <stdio.h>
#include <stdlib.h>
#include <dlfcn.h>   /* dlopen, dlsym, dlclose, dlerror */

/* Define function pointer types matching the plugin's exported functions */
typedef const char* (*pfn_plugin_name)(void);
typedef double      (*pfn_plugin_compute)(double);
typedef void        (*pfn_plugin_info)(void);

int main(int argc, char *argv[]) {
    void *handle;
    pfn_plugin_name   fn_name;
    pfn_plugin_compute fn_compute;
    pfn_plugin_info   fn_info;
    char *error;

    /* Step 1: Clear any existing error */
    dlerror();

    /* Step 2: Open the plugin */
    handle = dlopen("./plugin_math.so", RTLD_LAZY);
    if (!handle) {
        fprintf(stderr, "dlopen failed: %s\n", dlerror());
        exit(EXIT_FAILURE);
    }
    printf("Plugin loaded successfully\n");

    /* Step 3: Get symbol addresses */
    /* NOTE: cast via void* to avoid strict aliasing warnings */
    *(void **)(&fn_name)    = dlsym(handle, "plugin_name");
    *(void **)(&fn_compute) = dlsym(handle, "plugin_compute");
    *(void **)(&fn_info)    = dlsym(handle, "plugin_info");

    /* Step 4: Check for errors after dlsym */
    if ((error = dlerror()) != NULL) {
        fprintf(stderr, "dlsym failed: %s\n", error);
        dlclose(handle);
        exit(EXIT_FAILURE);
    }

    /* Step 5: Use the plugin */
    printf("Plugin name: %s\n", fn_name());
    fn_info();
    printf("compute(0.5) = %.6f\n", fn_compute(0.5));  /* should be ~1.0 */
    printf("compute(1.2) = %.6f\n", fn_compute(1.2));  /* should be ~1.0 */

    /* Step 6: Unload the plugin */
    dlclose(handle);
    printf("Plugin unloaded\n");

    return 0;
}
# Build the application — NOTE: -ldl is required
gcc -o main_dlopen main_dlopen.c -ldl

# Run
./main_dlopen
# Plugin loaded successfully
# Plugin name: MathPlugin v1.0
# MathPlugin: computes sin²(x) + cos²(x)
# compute(0.5) = 1.000000
# compute(1.2) = 1.000000
# Plugin unloaded
Important casting: The C standard technically makes casting between function pointers and void* undefined behavior. The POSIX workaround is to cast via *(void **)(&fn_ptr) = dlsym(...) which avoids the direct function-pointer-to-void* cast. GCC also accepts fn_ptr = (type)dlsym(...) with -Wpedantic suppressed.

Code Example 2 — Full Plugin Architecture

A real plugin system: a host application loads multiple plugins dynamically, each implementing a common interface.

/* plugin_interface.h — shared interface all plugins must implement */
#ifndef PLUGIN_INTERFACE_H
#define PLUGIN_INTERFACE_H

/* Every plugin must export this struct, named "plugin_ops" */
typedef struct {
    const char *name;
    const char *version;
    int  (*init)(void);                    /* called once on load */
    int  (*process)(const char *input, char *output, int outlen);
    void (*cleanup)(void);                 /* called on unload */
} PluginOps;

#endif
/* plugin_upper.c — plugin that converts string to uppercase */
#include "plugin_interface.h"
#include <ctype.h>
#include <string.h>
#include <stdio.h>

static int upper_init(void) {
    printf("[upper] init called\n");
    return 0;
}

static int upper_process(const char *input, char *output, int outlen) {
    int i;
    for (i = 0; i < outlen - 1 && input[i]; i++)
        output[i] = toupper((unsigned char)input[i]);
    output[i] = '\0';
    return i;
}

static void upper_cleanup(void) {
    printf("[upper] cleanup called\n");
}

/* THE EXPORTED SYMBOL — host will dlsym() for "plugin_ops" */
PluginOps plugin_ops = {
    .name    = "UppercasePlugin",
    .version = "1.0",
    .init    = upper_init,
    .process = upper_process,
    .cleanup = upper_cleanup,
};
/* plugin_reverse.c — plugin that reverses a string */
#include "plugin_interface.h"
#include <string.h>
#include <stdio.h>

static int rev_init(void) {
    printf("[reverse] init called\n");
    return 0;
}

static int rev_process(const char *input, char *output, int outlen) {
    int len = strlen(input);
    int i;
    if (len >= outlen) len = outlen - 1;
    for (i = 0; i < len; i++)
        output[i] = input[len - 1 - i];
    output[len] = '\0';
    return len;
}

static void rev_cleanup(void) {
    printf("[reverse] cleanup called\n");
}

PluginOps plugin_ops = {
    .name    = "ReversePlugin",
    .version = "2.0",
    .init    = rev_init,
    .process = rev_process,
    .cleanup = rev_cleanup,
};
/* host.c — host application that loads plugins */
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <dlfcn.h>
#include "plugin_interface.h"

#define MAX_PLUGINS 8

typedef struct {
    void      *handle;
    PluginOps *ops;
} LoadedPlugin;

LoadedPlugin plugins[MAX_PLUGINS];
int plugin_count = 0;

int load_plugin(const char *path) {
    if (plugin_count >= MAX_PLUGINS) {
        fprintf(stderr, "Too many plugins\n");
        return -1;
    }

    dlerror(); /* clear error state */

    void *handle = dlopen(path, RTLD_NOW | RTLD_LOCAL);
    if (!handle) {
        fprintf(stderr, "Cannot load plugin %s: %s\n", path, dlerror());
        return -1;
    }

    PluginOps *ops = (PluginOps *)dlsym(handle, "plugin_ops");
    char *err = dlerror();
    if (err) {
        fprintf(stderr, "No plugin_ops in %s: %s\n", path, err);
        dlclose(handle);
        return -1;
    }

    /* Call the plugin's init function */
    if (ops->init() != 0) {
        fprintf(stderr, "Plugin %s init failed\n", ops->name);
        dlclose(handle);
        return -1;
    }

    plugins[plugin_count].handle = handle;
    plugins[plugin_count].ops    = ops;
    plugin_count++;

    printf("Loaded plugin: %s v%s\n", ops->name, ops->version);
    return 0;
}

void run_all_plugins(const char *input) {
    char output[256];
    printf("\nInput: \"%s\"\n", input);
    for (int i = 0; i < plugin_count; i++) {
        plugins[i].ops->process(input, output, sizeof(output));
        printf("  [%s]: \"%s\"\n", plugins[i].ops->name, output);
    }
}

void unload_all_plugins(void) {
    for (int i = 0; i < plugin_count; i++) {
        plugins[i].ops->cleanup();
        dlclose(plugins[i].handle);
    }
    plugin_count = 0;
}

int main(void) {
    /* Load plugins at runtime — the host knows nothing about their internals */
    load_plugin("./plugin_upper.so");
    load_plugin("./plugin_reverse.so");

    /* Run all loaded plugins on some input */
    run_all_plugins("Hello World");
    run_all_plugins("Embedded Systems");

    /* Clean up */
    unload_all_plugins();
    return 0;
}
# Build all plugins and host
gcc -shared -fPIC -o plugin_upper.so   plugin_upper.c
gcc -shared -fPIC -o plugin_reverse.so plugin_reverse.c
gcc -o host host.c -ldl

# Run
./host
# [upper] init called
# [reverse] init called
# Loaded plugin: UppercasePlugin v1.0
# Loaded plugin: ReversePlugin v2.0
#
# Input: "Hello World"
#   [UppercasePlugin]: "HELLO WORLD"
#   [ReversePlugin]: "dlroW olleH"
#
# Input: "Embedded Systems"
#   [UppercasePlugin]: "EMBEDDED SYSTEMS"
#   [ReversePlugin]: "smetsyS deddebmE"
#
# [upper] cleanup called
# [reverse] cleanup called

Code Example 3 — Special dlopen Handles

/* special_handles.c — demonstrating NULL handle and RTLD_NOLOAD */
#include <stdio.h>
#include <dlfcn.h>
#include <stdlib.h>

int main(void) {
    void *handle;
    void *sym;

    /* === 1. dlopen(NULL) — handle for the main program itself ===
     * Lets you look up symbols defined in the executable.
     * Useful if the executable exports symbols for plugins to use.
     */
    handle = dlopen(NULL, RTLD_LAZY);
    if (!handle) {
        fprintf(stderr, "%s\n", dlerror());
        exit(1);
    }

    /* Look up the main() function in the executable itself */
    sym = dlsym(handle, "main");
    if (sym)
        printf("Address of main(): %p\n", sym);
    else
        printf("main not exported (add -rdynamic to gcc to export it)\n");

    dlclose(handle);

    /* === 2. RTLD_NOLOAD — check if a library is already loaded ===
     * Does NOT load the library — only checks if it's already in memory.
     * Returns non-NULL if already loaded, NULL if not.
     */
    dlerror();
    handle = dlopen("libc.so.6", RTLD_NOLOAD | RTLD_LAZY);
    if (handle) {
        printf("libc.so.6 is already loaded (as expected)\n");
        /* Don't dlclose — we didn't increase its ref count with RTLD_NOLOAD */
    }

    dlerror();
    handle = dlopen("libnonexistent.so.1", RTLD_NOLOAD | RTLD_LAZY);
    if (!handle)
        printf("libnonexistent.so.1 is NOT loaded (expected)\n");

    /* === 3. dlsym on already-loaded library ===
     * Access a symbol from libc without explicitly linking -lc
     */
    handle = dlopen("libc.so.6", RTLD_LAZY);
    if (handle) {
        /* Get the printf function pointer from libc */
        typedef int (*printf_fn)(const char *, ...);
        printf_fn my_printf;
        *(void **)(&my_printf) = dlsym(handle, "printf");
        if (my_printf)
            my_printf("printf via dlsym: works! %d\n", 42);
        dlclose(handle);
    }

    return 0;
}
# Compile: -rdynamic exports all symbols from the executable
# so that dlopen(NULL) can find them with dlsym
gcc -rdynamic -o special_handles special_handles.c -ldl

# Run
./special_handles
# Address of main(): 0x55f8... 
# libc.so.6 is already loaded (as expected)
# libnonexistent.so.1 is NOT loaded (expected)
# printf via dlsym: works! 42
Real-world dlopen uses: GStreamer loads codec plugins at runtime (gst-plugins-good, gst-plugins-bad are .so files dlopen’d). Python loads .so extension modules with dlopen when you do import numpy. BlueZ loads profile plugins. The ALSA audio library loads PCM and mixer plugins dynamically.

4. dlopen Reference Counting

Each call to dlopen() for the same library increments its reference count. The library is only unloaded when dlclose() has been called the same number of times. This means multiple parts of a program can independently load the same library without conflict — they each get the same handle, and the library stays loaded as long as anyone holds a reference.

/* ref_count_demo.c */
#include <stdio.h>
#include <dlfcn.h>

int main(void) {
    /* Load the same library twice */
    void *h1 = dlopen("./plugin_math.so", RTLD_LAZY);
    void *h2 = dlopen("./plugin_math.so", RTLD_LAZY);

    /* Both handles are non-NULL and equal — same library object */
    printf("h1 = %p\n", h1);
    printf("h2 = %p\n", h2);
    printf("h1 == h2: %s\n", (h1 == h2) ? "yes" : "no");
    /* Output: yes — same handle returned */

    /* First dlclose: ref count drops from 2 to 1 — NOT unloaded */
    dlclose(h1);
    printf("After first dlclose: library still loaded\n");

    /* Second dlclose: ref count drops from 1 to 0 — NOW unloaded */
    dlclose(h2);
    printf("After second dlclose: library unloaded\n");

    /* Using any function pointer obtained before h2 dlclose would now
     * be undefined behavior — the library code is unmapped */

    return 0;
}

Interview Questions & Answers

Q1. What are the four functions in the dlopen API and what does each do?
dlopen(path, flags): Loads the named shared library into the process, returns an opaque handle. dlsym(handle, symbol): Returns the address of the named symbol (function or variable) from the library. dlclose(handle): Decrements the library’s reference count; unloads when it reaches zero. dlerror(): Returns the human-readable error message for the most recent failed dl* operation, then clears the error. All four are declared in <dlfcn.h>, and programs must link with -ldl.
Q2. What is the difference between RTLD_LAZY and RTLD_NOW in dlopen?
RTLD_LAZY: Symbol resolution is deferred until the symbol is actually called. Faster dlopen, but an undefined symbol won’t be detected until the first call to that function, which could happen deep in execution. RTLD_NOW: All symbols are resolved immediately when dlopen is called. If any symbol is undefined, dlopen returns NULL with an error. Slower but safer — preferred for plugins where you want to fail fast at load time rather than crash during use.
Q3. Why should dlerror() be called before dlsym() as well as after?
Because dlsym() returns NULL both when the symbol is not found AND when the symbol’s value is legitimately NULL (e.g., a function pointer that is NULL, or a null global variable). Calling dlerror() before dlsym() clears any prior error. Then after calling dlsym(), calling dlerror() tells you whether NULL was returned due to an error. If dlerror() returns NULL after dlsym(), then the symbol was found and its value is legitimately NULL. If dlerror() returns a string, there was an error.
Q4. What does dlopen(NULL, flags) return and when is it useful?
dlopen(NULL, flags) returns a handle to the main executable itself (not a library). Combined with dlsym(), it lets you look up symbols defined in the executable. This is useful when: (1) A plugin needs to call back into the host application by symbol name. (2) You want to enumerate symbols dynamically. (3) Implementing hot-reload where new plugins resolve host functions by name. For dlsym to find symbols in the executable, the program must be compiled with -rdynamic (which adds all public symbols to the dynamic symbol table).
Q5. What is RTLD_GLOBAL vs RTLD_LOCAL and when does RTLD_GLOBAL matter?
RTLD_LOCAL (default): Symbols from the dlopen’d library are private to it — they cannot be used to resolve symbols in subsequently dlopen’d libraries. RTLD_GLOBAL: Symbols from this library are added to the global symbol table and are available for resolving symbols in any library loaded afterward. RTLD_GLOBAL is necessary when: a plugin A exports symbols that plugin B depends on. Without RTLD_GLOBAL for A, dlopen(B) would fail with “undefined symbol” because B’s symbols can’t find A’s exports. Python uses RTLD_GLOBAL when loading numpy so that other extensions can see numpy’s C API symbols.
Q6. Does calling dlclose() immediately unload the library? Explain reference counting.
Not necessarily. Each call to dlopen() for the same library path increments an internal reference count. Each call to dlclose() decrements it. The library is only actually unloaded (its destructor called, its pages unmapped) when the reference count reaches zero. So if a library was dlopen’d three times (by different parts of the program), it requires three dlclose calls to unload. Additionally, if the library was also listed as a DT_NEEDED dependency of the executable (loaded at startup), it is loaded with a permanent reference count from the startup and may never be unloaded.
Q7. What is -rdynamic and when is it needed?
-rdynamic (or equivalently -Wl,--export-dynamic) tells the linker to add all non-static symbols in the executable to the dynamic symbol table (DYNSYM section). Without it, the executable’s symbols are only in the static symbol table (SYMTAB) and are invisible to dlsym(). It is needed when: (1) plugins loaded with dlopen need to call functions in the host executable by name. (2) You use dlopen(NULL) to look up symbols from the executable. (3) Using backtrace_symbols() to get function names in stack traces.
Q8. How do you design a good plugin interface?
A good plugin interface: (1) Defines a single well-known symbol name (e.g., plugin_ops) that the host dlsym’s — avoids searching for many symbols. (2) Uses a vtable struct (like PluginOps) with function pointers — adding new optional functions at the end maintains backward compatibility. (3) Includes versioning in the struct (e.g., version field). (4) Has explicit init() and cleanup() functions so the host controls resource lifetime. (5) Uses stable primitive types in function signatures — avoid passing complex structs across the plugin boundary to reduce ABI issues.
Q9. What is the difference between implicit (startup) loading and explicit (dlopen) loading?
Implicit loading: Libraries listed in the ELF’s DT_NEEDED section are automatically loaded at program startup by the dynamic linker. The program calls functions directly (compiler resolves the symbol at link time). Explicit loading (dlopen): The program calls dlopen() at runtime to load a library, then dlsym() to get function pointers. The program has no knowledge of the library at compile time. Differences: explicit loading allows load-on-demand (save memory), unknown-at-compile-time library paths (plugins), versioned loading of multiple versions simultaneously, and error recovery (try to load an optional feature, gracefully skip if unavailable).
Q10. A plugin crashes the host application. How would you add isolation?
The dlopen API runs plugins in the same process, so a crashing plugin crashes the host. For isolation: (1) Subprocess isolation: run each plugin in a child process (fork + exec), communicate via pipes or shared memory. Crash in child doesn’t kill parent. (2) Signal recovery: install signal handlers (SIGSEGV, SIGFPE) that use setjmp/longjmp to recover — fragile but simple. (3) Sandboxing: use seccomp, namespaces, or capabilities to restrict what the plugin subprocess can do. (4) Strict interface: validate all data passed to/from plugins. (5) Watchdog thread: run the plugin function in a separate thread with a timeout; if it hangs, pthread_cancel or kill it. Real-world: Chrome and Firefox use separate processes for plugins for exactly this reason.

🎉 Chapter 41 Complete!

You’ve covered all 6 parts — static/shared libraries, ldconfig, compatibility rules, live upgrades, dynamic linking internals, and the dlopen API.

← Back to Chapter 41 Index EmbeddedPathashala Home

Leave a Reply

Your email address will not be published. Required fields are marked *