Finding Shared Libraries at Run Time Dynamic Linker Search Rules

 

Finding Shared Libraries at Run Time
TLPI Chapter 41.11 — Dynamic Linker Search Rules, rpath, LD_LIBRARY_PATH & ldconfig
5
Search Steps
3+
Code Examples
10
Interview Q&As

Key Concepts
Dynamic Linker DT_RPATH DT_RUNPATH LD_LIBRARY_PATH /etc/ld.so.cache ldconfig -rpath linker flag $ORIGIN Turn-key application

What Is This About?

When your program starts, the operating system loads it into memory. But your program also depends on shared libraries (like libc.so, libpthread.so, etc.). Someone has to find those library files on disk and load them.

That “someone” is the dynamic linker (also called the dynamic loader), which on Linux is /lib/ld-linux.so.2 (or its 64-bit equivalent). It follows a strict set of rules — a search order — to find each required shared library. Understanding this search order is critical for building portable applications and debugging “library not found” errors.

Step 0 — Does the Dependency String Contain a Slash?

Before applying the search rules, the dynamic linker checks the dependency string stored inside the executable (put there at link time). If that string contains a slash (/), the linker treats it as a direct pathname and loads the library from that exact location (absolute or relative). No searching happens.

# Link with an EXPLICIT path — dependency string will contain a slash
gcc -o myprog main.c /opt/mylibs/libfoo.so

# At run time, dynamic linker reads /opt/mylibs/libfoo.so directly
# No search rules applied — slash found in dependency string
⚠️ Hardcoding absolute paths makes your binary non-portable. Use this only when you truly need to pin a specific library file.

The 5-Step Search Order (No Slash in Dependency String)

When the dependency string has no slash, the dynamic linker searches in this exact order:

1DT_RPATH (rpath) in the executable — if DT_RUNPATH is absent
2LD_LIBRARY_PATH environment variable directories
3DT_RUNPATH in the executable
4/etc/ld.so.cache (built by ldconfig)
5/lib and /usr/lib (in that order)
Step Mechanism How to Set Overridable?
1 DT_RPATH (if no DT_RUNPATH) -Wl,-rpath,/path at link time No (baked into binary)
2 LD_LIBRARY_PATH env var export LD_LIBRARY_PATH=/path Yes (runtime env)
3 DT_RUNPATH -Wl,--enable-new-dtags,-rpath,/path Partially (overridden by LD_LIBRARY_PATH)
4 /etc/ld.so.cache ldconfig command (root) Via ldconfig
5 /lib and /usr/lib System default, cannot be changed No

Step 1 in Detail — DT_RPATH (Run-Time Library Path)

rpath is a colon-separated list of directories baked into the ELF binary at link time. At run time, the dynamic linker checks these directories first (only if no DT_RUNPATH is present).

# Example 1: Embed rpath during linking
gcc -o myprog main.c -L./libs -lfoo -Wl,-rpath,/opt/myapp/libs

# Verify the rpath was embedded
readelf -d myprog | grep RPATH
# Output: 0x000000000000000f (RPATH) Library rpath: [/opt/myapp/libs]

# The binary will always look in /opt/myapp/libs first, regardless of
# what LD_LIBRARY_PATH says (since there's no DT_RUNPATH here)
# Example 2: Multiple directories in rpath
gcc -o myprog main.c \
    -Wl,-rpath,/opt/myapp/libs:/usr/local/lib \
    -L/opt/myapp/libs -lfoo

# Directories are searched left-to-right
# Example 3: Check all dynamic entries in a binary
readelf -d myprog
# Look for (NEEDED), (RPATH), (RUNPATH) entries

The Special $ORIGIN Token — Portable rpath

A major limitation of rpath is that hardcoding absolute paths breaks portability — if the user installs your app in a different directory, the path no longer works. Linux solves this with the special token $ORIGIN, which the dynamic linker expands at run time to the directory containing the executable itself.

# Example: Turn-key application with libraries in a 'lib' subdirectory
#
# Directory layout after installation:
#   /anywhere/myapp/          ← user can put this anywhere!
#   /anywhere/myapp/myprog   ← the executable
#   /anywhere/myapp/lib/     ← shared libraries directory
#       libfoo.so.1

# Build with $ORIGIN-relative rpath
gcc -o myprog main.c \
    -L./lib -lfoo \
    -Wl,-rpath,'$ORIGIN/lib'

# At run time, if myprog lives in /home/user/myapp/:
#   $ORIGIN expands to /home/user/myapp/
#   dynamic linker looks in /home/user/myapp/lib/
#
# If moved to /opt/myapp/:
#   $ORIGIN expands to /opt/myapp/
#   dynamic linker looks in /opt/myapp/lib/
# -> Fully portable!
💡 Use single quotes around $ORIGIN in shell commands to prevent the shell from expanding $ORIGIN as a shell variable (which is empty). The linker itself handles the expansion.
# Verify $ORIGIN rpath was embedded correctly
readelf -d myprog | grep RPATH
# Output: (RPATH) Library rpath: [$ORIGIN/lib]

# Another common pattern: same directory as executable
gcc -o myprog main.c -Wl,-rpath,'$ORIGIN' -L. -lfoo

# Going up one level
gcc -o myprog main.c -Wl,-rpath,'$ORIGIN/../lib' -L../lib -lfoo

Step 2 in Detail — LD_LIBRARY_PATH Environment Variable

LD_LIBRARY_PATH is a colon-separated list of directories that the dynamic linker searches at run time, before the default system paths. It is the easiest way to test a new version of a library without installing it system-wide.

# Example: Use a library from /tmp/testlibs without installing it
export LD_LIBRARY_PATH=/tmp/testlibs:$LD_LIBRARY_PATH
./myprog

# Or inline for a single run
LD_LIBRARY_PATH=/tmp/testlibs ./myprog

# Multiple directories
export LD_LIBRARY_PATH=/opt/lib:/home/user/mylibs:/tmp/testlibs
# Example: Debug library loading with LD_DEBUG
LD_DEBUG=libs ./myprog 2>&1 | head -30
# Shows every library the dynamic linker looks for and where it finds it

# More verbose: show all dynamic linker internals
LD_DEBUG=all ./myprog 2>&1 | less
⚠️ Security Note: If your executable is a set-user-ID (SUID) or set-group-ID (SGID) program, the dynamic linker completely ignores LD_LIBRARY_PATH. This is intentional — it prevents a normal user from tricking a privileged program into loading a malicious library with the same name as a legitimate one.

Step 3 in Detail — DT_RUNPATH

DT_RUNPATH is similar to DT_RPATH but is searched after LD_LIBRARY_PATH. This means LD_LIBRARY_PATH can override DT_RUNPATH, making it more flexible for testing and debugging.

# Enable DT_RUNPATH (instead of DT_RPATH) using --enable-new-dtags
gcc -o myprog main.c \
    -Wl,--enable-new-dtags,-rpath,/opt/myapp/libs \
    -L/opt/myapp/libs -lfoo

# Verify: RUNPATH appears instead of RPATH
readelf -d myprog | grep -E "RPATH|RUNPATH"
# Output: (RUNPATH) Library runpath: [/opt/myapp/libs]

# Now LD_LIBRARY_PATH can override the runpath:
LD_LIBRARY_PATH=/tmp/testlibs ./myprog
# -> Looks in /tmp/testlibs BEFORE /opt/myapp/libs

With DT_RPATH: the rpath takes priority over LD_LIBRARY_PATH.
With DT_RUNPATH: LD_LIBRARY_PATH takes priority over the runpath.

Step 4 in Detail — /etc/ld.so.cache and ldconfig

/etc/ld.so.cache is a binary cache file that maps library names (like libfoo.so.1) to their full paths on disk. It is generated by the ldconfig command, which scans the directories listed in /etc/ld.so.conf (and its include files). This allows fast lookups without scanning every directory.

# Install a library system-wide and update the cache
sudo cp libfoo.so.1 /usr/local/lib/
sudo ldconfig                  # rebuild the cache

# Verify the library is now in the cache
ldconfig -p | grep libfoo
# Output: libfoo.so.1 (libc6,x86-64) => /usr/local/lib/libfoo.so.1
# See which directories ldconfig scans
cat /etc/ld.so.conf
cat /etc/ld.so.conf.d/*.conf

# Add a custom directory to ldconfig (as root)
echo "/opt/myapp/libs" | sudo tee /etc/ld.so.conf.d/myapp.conf
sudo ldconfig

# Verify
ldconfig -p | grep -i myapp
# List ALL libraries currently in the cache
ldconfig -p | head -20
# Format: libname.so.version (arch) => /full/path/to/lib.so.version
💡 When you install a new shared library and your program can’t find it, the most common fix is: sudo ldconfig

Step 5 — Default System Directories: /lib and /usr/lib

These are the last-resort fallback locations. Core system libraries (glibc, libpthread, libm, etc.) live here. Searched in the order: /lib first, then /usr/lib.

# See what's in /lib and /usr/lib
ls /lib/*.so* | head -10
ls /usr/lib/*.so* | head -10

# On 64-bit systems, 64-bit libs are typically in:
ls /lib/x86_64-linux-gnu/
ls /usr/lib/x86_64-linux-gnu/

Complete Practical Example — Tracing Library Search
/* main.c — uses libgreet */
#include <stdio.h>
void greet(void);  /* declared in libgreet */

int main(void) {
    greet();
    return 0;
}
/* greet.c — the shared library source */
#include <stdio.h>

void greet(void) {
    printf("Hello from libgreet!\n");
}
# Build the shared library
gcc -fPIC -shared -o libgreet.so.1 greet.c
ln -s libgreet.so.1 libgreet.so

# Build the executable with rpath=$ORIGIN so it finds the lib next to itself
gcc -o hello main.c -L. -lgreet -Wl,-rpath,'$ORIGIN'

# Run — dynamic linker finds libgreet.so.1 in the same directory
./hello
# Output: Hello from libgreet!

# Check what libraries are needed and where they'll be found
ldd ./hello
# Output:
#   libgreet.so.1 => ./libgreet.so.1 (0x00007f...)
#   libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f...)

# Trace the full search with LD_DEBUG
LD_DEBUG=libs ./hello 2>&1 | grep greet

Interview Questions & Answers
Q1. What is the dynamic linker and what is its role?
The dynamic linker (ld-linux.so) is a special program that runs automatically when you execute a dynamically linked binary. Its job is to find all required shared libraries, load them into memory, resolve symbol references (function addresses, global variable addresses) between the executable and its libraries, and then transfer control to the program’s main() function.
Q2. In what order does the dynamic linker search for shared libraries?
1. If a slash is in the dependency string → use it as a direct path.
2. DT_RPATH directories (if DT_RUNPATH is absent)
3. LD_LIBRARY_PATH directories (ignored for SUID/SGID programs)
4. DT_RUNPATH directories
5. /etc/ld.so.cache
6. /lib, then /usr/lib
Q3. What is the difference between DT_RPATH and DT_RUNPATH?
Both are lists of directories embedded in the ELF binary at link time. The key difference is their position relative to LD_LIBRARY_PATH: DT_RPATH is searched before LD_LIBRARY_PATH (you cannot override it at run time with the env var), while DT_RUNPATH is searched after LD_LIBRARY_PATH (LD_LIBRARY_PATH can override it). Use --enable-new-dtags linker flag to create DT_RUNPATH instead of DT_RPATH.
Q4. What is $ORIGIN and why is it useful?
$ORIGIN is a special token in rpath that the dynamic linker expands at run time to the directory containing the executable. It allows you to write portable rpath values like $ORIGIN/lib that work regardless of where the application is installed. This is essential for “turn-key applications” where the user can place the package in any directory.
Q5. Why is LD_LIBRARY_PATH ignored for SUID programs?
Security. A set-user-ID program runs with elevated privileges (e.g., root). If LD_LIBRARY_PATH were honored, a normal user could create a malicious library (e.g., a fake libc.so) and set LD_LIBRARY_PATH to point to it. The privileged program would then load the attacker’s library and execute their code with root privileges. Ignoring LD_LIBRARY_PATH prevents this attack.
Q6. What is ldconfig and when do you need to run it?
ldconfig is a command that scans the standard library directories (from /etc/ld.so.conf) and builds a fast cache file /etc/ld.so.cache. You need to run it (as root: sudo ldconfig) whenever you: install a new shared library, remove a library, or change the directories listed in /etc/ld.so.conf. Without updating the cache, newly installed libraries won’t be found by programs.
Q7. How do you check which shared libraries a program requires?
Use the ldd command: ldd ./myprog. It shows each required library and the path where it will be loaded. You can also use readelf -d myprog | grep NEEDED to see the raw DT_NEEDED entries without invoking the dynamic linker.
Q8. How do you embed rpath into a binary at link time?
Pass the -rpath option to the linker via gcc’s -Wl flag: gcc -o myprog main.c -Wl,-rpath,/opt/myapp/libs -L/opt/myapp/libs -lmylib. You can verify with readelf -d myprog | grep RPATH.
Q9. Your program works with LD_LIBRARY_PATH set but fails without it. What are the proper fixes?
1. Install the library to a standard path (/usr/local/lib) and run sudo ldconfig.
2. Add the library directory to /etc/ld.so.conf.d/ and run sudo ldconfig.
3. Embed an rpath in the binary using -Wl,-rpath,/path/to/libs at link time.
4. Use $ORIGIN-based rpath for portable installations.
Q10. How do you debug shared library loading issues at run time?
Use the LD_DEBUG environment variable: LD_DEBUG=libs ./myprog shows the search process for each library. LD_DEBUG=all ./myprog shows all dynamic linker activity. Also useful: ldd ./myprog to see what will be loaded, and strace -e trace=openat ./myprog to see which file paths the linker actually tries to open.

Leave a Reply

Your email address will not be published. Required fields are marked *