Using Shared Libraries at Run Time Dynamic Linker, ldconfig, LD_LIBRARY_PATH, ldd

 

41.5 — Using Shared Libraries at Run Time
Dynamic Linker, ldconfig, LD_LIBRARY_PATH, ldd | EmbeddedPathashala

Key Terms

Dynamic Linker (ld.so) ldconfig LD_LIBRARY_PATH ldd /etc/ld.so.conf /etc/ld.so.cache RPATH RUNPATH

What is the Dynamic Linker?

When you run a program that uses shared libraries, the kernel does not load all the code directly. Instead, it loads a special program called the dynamic linker (also called the runtime linker or dynamic loader). On Linux, this is the program /lib/ld-linux.so.2 (32-bit) or /lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 (64-bit).

The dynamic linker’s job is:

  1. Read the executable’s ELF header to find the list of required shared libraries (NEEDED entries).
  2. Find each required .so file on disk.
  3. Map each .so into the process’s virtual address space using mmap().
  4. Fill in the GOT and PLT entries with correct addresses (symbol resolution).
  5. Run any initialization code in the libraries (.init and .ctors sections).
  6. Transfer control to the program’s main() function.

All of this happens before your main() even starts. If any required library cannot be found, your program fails with an error like: “error while loading shared libraries: libfoo.so.1: cannot open shared object file: No such file or directory”.

How the Dynamic Linker Finds Libraries

The dynamic linker searches for shared libraries in this order:

Dynamic Linker Search Order
1
RPATH/RUNPATH embedded in the binary (-Wl,-rpath=/my/libs)
2
LD_LIBRARY_PATH environment variable (colon-separated list of directories)
3
/etc/ld.so.cache — a binary cache built by ldconfig from directories in /etc/ld.so.conf
4
Default paths: /lib, /usr/lib, /lib64, /usr/lib64

ldconfig — The Library Cache Manager

ldconfig is a system administration command that:

  • Reads the directory list from /etc/ld.so.conf and its includes
  • Scans those directories for shared library files
  • Creates and maintains the symlinks (soname → real name)
  • Writes a binary cache file /etc/ld.so.cache that lists all found libraries and their full paths

The dynamic linker reads /etc/ld.so.cache at run time for fast library lookups. You must run ldconfig (as root) whenever you install a new shared library into a system directory — otherwise programs won’t find it.

Methods to Make a Shared Library Available

Method How to Use Scope Use Case
LD_LIBRARY_PATH export LD_LIBRARY_PATH=/path/to/lib:$LD_LIBRARY_PATH Current shell session / process Testing/development; NOT for production
ldconfig + /etc/ld.so.conf Add dir to /etc/ld.so.conf.d/, run sudo ldconfig System-wide, persistent Production deployment; installed libraries
RPATH (in binary) gcc -Wl,-rpath,/path/to/lib That specific binary only Self-contained apps; embedded projects
Copy to /usr/local/lib sudo cp libfoo.so.1 /usr/local/lib/ + sudo ldconfig System-wide (local installs) User-installed libraries

Coding Example 1 — Library Search Path Demonstration

We’ll create a shared library and demonstrate different ways to make it available, including the effect of ldconfig and LD_LIBRARY_PATH.

/* sysinfo.h - system info library */
#ifndef SYSINFO_H
#define SYSINFO_H

typedef struct {
    int  cpu_count;
    long total_ram_mb;
    char hostname[64];
} SystemInfo;

void sysinfo_get(SystemInfo *info);
void sysinfo_print(const SystemInfo *info);

#endif
/* sysinfo.c - implementation */
#include "sysinfo.h"
#include <stdio.h>
#include <string.h>
#include <unistd.h>

/* Must be compiled with -fPIC for shared library use! */

void sysinfo_get(SystemInfo *info) {
    if (!info) return;

    /* Get number of CPUs */
    info->cpu_count = (int)sysconf(_SC_NPROCESSORS_ONLN);

    /* Get total RAM */
    long pages = sysconf(_SC_PHYS_PAGES);
    long page_size = sysconf(_SC_PAGE_SIZE);
    info->total_ram_mb = (pages * page_size) / (1024 * 1024);

    /* Get hostname */
    gethostname(info->hostname, sizeof(info->hostname));
}

void sysinfo_print(const SystemInfo *info) {
    if (!info) return;
    printf("=== System Information ===\n");
    printf("Hostname  : %s\n", info->hostname);
    printf("CPU Cores : %d\n", info->cpu_count);
    printf("Total RAM : %ld MB\n", info->total_ram_mb);
}
/* app_sysinfo.c - application using sysinfo shared library */
#include <stdio.h>
#include "sysinfo.h"

int main(void) {
    SystemInfo si;
    sysinfo_get(&si);
    sysinfo_print(&si);

    printf("\nThis app uses libsysinfo.so.1 shared library\n");
    return 0;
}
## ============================================
## BUILD AND INSTALL THE SHARED LIBRARY
## ============================================

## Step 1: Build the shared library
$ gcc -g -fPIC -c sysinfo.c -o sysinfo.o
$ gcc -g -shared -Wl,-soname,libsysinfo.so.1 \
    -o libsysinfo.so.1.0.0 sysinfo.o
$ ln -sf libsysinfo.so.1.0.0 libsysinfo.so.1
$ ln -sf libsysinfo.so.1     libsysinfo.so

## Step 2: Compile the application
$ gcc -g -o app_sysinfo app_sysinfo.c -L. -lsysinfo

## Step 3: Try running — will FAIL without telling linker where .so is
$ ./app_sysinfo
## Error: ./app_sysinfo: error while loading shared libraries:
##        libsysinfo.so.1: cannot open shared object file

## ---- SOLUTION A: LD_LIBRARY_PATH (for testing) ----
$ LD_LIBRARY_PATH=. ./app_sysinfo   # Works!

## ---- SOLUTION B: RPATH (embed path in binary) ----
## Recompile with rpath set to current directory
$ gcc -g -o app_sysinfo_rpath app_sysinfo.c \
    -L. -lsysinfo \
    -Wl,-rpath,'$ORIGIN'    # $ORIGIN = directory of the binary itself
$ ./app_sysinfo_rpath       # Works without LD_LIBRARY_PATH!

## Verify the rpath is embedded:
$ readelf -d app_sysinfo_rpath | grep -E "RPATH|RUNPATH"

## ---- SOLUTION C: Install to system (production) ----
## Copy to /usr/local/lib
$ sudo cp libsysinfo.so.1.0.0 /usr/local/lib/

## Create symlink
$ sudo ln -sf libsysinfo.so.1.0.0 /usr/local/lib/libsysinfo.so.1

## Update the ld.so.cache
$ sudo ldconfig

## Verify ldconfig sees it:
$ ldconfig -p | grep sysinfo
## libsysinfo.so.1 (libc6,x86-64) => /usr/local/lib/libsysinfo.so.1.0.0

## Now run without LD_LIBRARY_PATH:
$ ./app_sysinfo   # Works!

## ---- SOLUTION D: Add custom dir to ld.so.conf ----
$ echo "/home/ravi/mylibs" | sudo tee /etc/ld.so.conf.d/mylibs.conf
$ sudo ldconfig

The ldd Tool — Inspect Library Dependencies

ldd (List Dynamic Dependencies) shows which shared libraries a program or shared library needs, and where the dynamic linker will find them. This is the first tool to use when debugging “cannot open shared object file” errors.

Coding Example 2 — Using ldd and Debugging Library Issues

## ============================================
## USING ldd TO INSPECT LIBRARY DEPENDENCIES
## ============================================

## Show all shared libraries needed by a system program:
$ ldd /usr/bin/python3
## Sample output:
##     linux-vdso.so.1 (0x00007ffe...)
##     libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
##     libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2
##     libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1
##     libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6
##     libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6
##     /lib64/ld-linux-x86-64.so.2 (0x00007f...)

## Check our own app:
$ ldd app_sysinfo
##     libsysinfo.so.1 => not found     (← this is the problem!)
##     libc.so.6 => /lib/...

## "not found" means dynamic linker cannot locate libsysinfo.so.1

## After installing (Solution C above):
$ ldd app_sysinfo
##     libsysinfo.so.1 => /usr/local/lib/libsysinfo.so.1 (0x7f...)

## ============================================
## COMMON TROUBLESHOOTING COMMANDS
## ============================================

## 1. List all libraries in the ld.so.cache:
$ ldconfig -p | head -20

## 2. Search for a specific library in the cache:
$ ldconfig -p | grep libz
## libz.so.1 (libc6,x86-64) => /lib/x86_64-linux-gnu/libz.so.1

## 3. Run with verbose library loading (debug):
$ LD_DEBUG=libs ./app_sysinfo 2>&1 | head -30
## Shows every library the dynamic linker tries to open

## 4. Show library search details:
$ LD_DEBUG=libs,files ./app_sysinfo 2>&1

## 5. Check if a .so has missing dependencies:
$ ldd -r libsysinfo.so.1.0.0
## "undefined symbol" lines mean unresolved references

## ============================================
## BUILDING FOR EMBEDDED: Static vs Dynamic
## ============================================

## Compare binary sizes:
$ gcc -o app_dynamic app_sysinfo.c -L. -lsysinfo
$ gcc -static -o app_static app_sysinfo.c sysinfo.c
$ ls -lh app_dynamic app_static

## For embedded Linux (e.g., Buildroot, Yocto):
## Shared libraries save space when multiple apps share them
## Static linking is useful for initramfs or single-binary deployments

## Check if a binary is dynamically or statically linked:
$ file app_dynamic
## ELF 64-bit LSB pie executable, ..., dynamically linked

$ file app_static
## ELF 64-bit LSB executable, ..., statically linked
💡 $ORIGIN in RPATH:
When you embed -Wl,-rpath,'$ORIGIN' in a binary, the dynamic linker replaces $ORIGIN at run time with the directory where the binary itself lives. This is extremely useful for distributing self-contained applications where the .so file lives in the same directory as the executable — no system installation needed. Common pattern: -Wl,-rpath,'$ORIGIN/lib' to look in a lib/ subdirectory next to the binary.
⚠️ Never use LD_LIBRARY_PATH in Production:
LD_LIBRARY_PATH is handy for testing but should not be used in production for several reasons: it applies to ALL programs spawned from that shell (including system utilities), it can cause security issues (attackers could place malicious .so files in that path), and it breaks setuid programs (the kernel ignores LD_LIBRARY_PATH for setuid binaries as a security measure). Use ldconfig or RPATH instead.

Interview Questions — Section 41.5

Q1. What is the dynamic linker? What is it called on Linux?
The dynamic linker (also called the runtime linker or dynamic loader) is a special shared library that is loaded by the kernel when a dynamically-linked program is executed. Its job is to find all required shared libraries, load them into the process’s virtual memory, resolve symbol references, and then transfer control to main(). On Linux, it is /lib/ld-linux.so.2 (for 32-bit programs) or /lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 (for 64-bit programs). It is itself a shared library, stored in the ELF header of every dynamically-linked executable as the “interpreter” (PT_INTERP segment).
Q2. What does ldconfig do and when must you run it?
ldconfig scans the library directories listed in /etc/ld.so.conf (and /etc/ld.so.conf.d/ files), builds a binary cache file (/etc/ld.so.cache) of all found shared libraries, and creates/updates the soname symlinks. You must run ldconfig (as root) every time you install a new shared library into a system directory, or after changing /etc/ld.so.conf. Without running it, the dynamic linker won’t know about newly installed libraries.
Q3. What is LD_LIBRARY_PATH and why is it dangerous in production?
LD_LIBRARY_PATH is an environment variable containing a colon-separated list of directories that the dynamic linker searches for shared libraries before the standard paths. It is useful for testing a new library without installing it. However, it is dangerous in production because: it affects all programs run from that shell (including system utilities); an attacker who can set it could make programs load malicious .so files; and the Linux kernel ignores it for setuid binaries as a security measure. Better alternatives for production are ldconfig, RPATH embedded in the binary, or installing to standard paths.
Q4. What is the ldd command and what does “not found” in its output mean?
ldd (List Dynamic Dependencies) shows all shared libraries that a binary requires and the full path where the dynamic linker will find each one. When a library shows “not found” in ldd output, it means the dynamic linker cannot locate that .so file in any of its search paths. This will cause a runtime error when you try to execute the program. Fix: install the library, run ldconfig, or set LD_LIBRARY_PATH for testing.
Q5. What is RPATH? How is it different from LD_LIBRARY_PATH?
RPATH (or RUNPATH) is a list of library search directories embedded directly into the executable binary’s ELF header at link time using -Wl,-rpath,/path. It is used only for that specific binary. In contrast, LD_LIBRARY_PATH is an environment variable that applies to all programs in the current shell session. RPATH is preferred for distributing self-contained applications because it doesn’t require system configuration, uses $ORIGIN for relative paths, and is not affected by the user’s environment. RPATH is checked before LD_LIBRARY_PATH in the search order (unless RUNPATH is used, which is checked after LD_LIBRARY_PATH).
Q6. How does LD_DEBUG help troubleshoot shared library issues?
Setting LD_DEBUG=libs before running a program causes the dynamic linker to print detailed information about which libraries it is searching for and where it looks. LD_DEBUG=files shows every file the dynamic linker opens. These are sent to stderr. Common values: libs (library search), symbols (symbol resolution), bindings (function binding), all (everything). This is invaluable when a program fails to find a library at runtime.

Leave a Reply

Your email address will not be published. Required fields are marked *