Using a Shared Library Linux System Programming

 

Using a Shared Library
Linux System Programming ยท Chapter 41 ยท Section 41.4.3
๐Ÿ”—
Dynamic Linker
๐Ÿ“‚
LD_LIBRARY_PATH
โš™๏ธ
ELF & DT_NEEDED
๐Ÿ—๏ธ
Static vs Dynamic

What is a Shared Library?

A shared library (also called a dynamic library or shared object) is a compiled collection of object code that is not bundled inside your executable at compile time. Instead, the library is loaded into memory at runtime and shared across multiple programs simultaneously โ€” saving disk space and RAM.

When you link a program against a shared library, two critical things must happen that are NOT needed with static libraries:

  1. The name of the shared library must be embedded inside the executable at link time (recorded as a DT_NEEDED tag in the ELF binary).
  2. At runtime, a special program called the dynamic linker must find that library file on disk and load it into memory before the program can run.

Key Terms

Shared Library Dynamic Linker ld-linux.so.2 DT_NEEDED LD_LIBRARY_PATH ELF Dynamic Dependency List Static Linking Dynamic Linking Run-time Linker /lib /usr/lib

๐Ÿ“Œ Two Steps Required to Use a Shared Library

Unlike static libraries (where all object code is copied into the executable), shared libraries leave the code in a separate .so file. Two things must happen before your program can run:

  • Embed the library name at link time: When you compile and link your program with gcc -o prog prog.c libfoo.so, the linker records the library’s name inside the ELF executable as a DT_NEEDED tag. This is called the program’s dynamic dependency list.
  • Resolve & load the library at run time: The dynamic linker (/lib/ld-linux.so.2) reads the DT_NEEDED list, locates the .so file on disk, and maps it into the process’s address space before main() is called.

ELF Binary โ€” Where the dependency is stored:

ELF Executable (prog)
ELF Header Magic, architecture, entry point
Program Headers Segments to load into memory
Dynamic Section DT_NEEDED = libfoo.so โ† library name stored here
.text Section Program machine code
.data / .bss Initialized / uninitialized data

๐Ÿ› ๏ธ Coding Example 1 โ€” Creating and Linking a Shared Library

Let’s create a simple shared library with two modules and link a program against it.

Step 1: Write source files

/* mod1.c */
#include <stdio.h>

void mod1_func(void) {
    printf("Called mod1-x1\n");
}
/* mod2.c */
#include <stdio.h>

void mod2_func(void) {
    printf("Called mod2-x2\n");
}
/* prog.c */
void mod1_func(void);
void mod2_func(void);

int main(void) {
    mod1_func();
    mod2_func();
    return 0;
}

Step 2: Compile object files with Position Independent Code (-fPIC)

# -fPIC = Position Independent Code, required for shared libraries
# Code can be loaded at any address in memory
$ gcc -g -c -fPIC -Wall mod1.c mod2.c

Step 3: Create the shared library

# -shared  = create a shared library (not an executable)
# -o libfoo.so = output file name (must start with "lib", end with ".so")
$ gcc -g -shared -o libfoo.so mod1.o mod2.o

Step 4: Link the program against the shared library

# -L. = tell the linker to also look in current directory for libraries
# -lfoo = link with libfoo.so (linker strips "lib" prefix and ".so" suffix)
$ gcc -g -Wall -o prog prog.c -L. -lfoo

# OR directly name the .so file:
$ gcc -g -Wall -o prog prog.c libfoo.so

Step 5: Inspect embedded library name (DT_NEEDED)

# Use readelf to see what shared libraries the executable needs
$ readelf -d prog | grep NEEDED
 0x00000001 (NEEDED)  Shared library: [libfoo.so]
 0x00000001 (NEEDED)  Shared library: [libc.so.6]

# Alternatively with objdump:
$ objdump -p prog | grep NEEDED
  NEEDED               libfoo.so
  NEEDED               libc.so.6

Step 6: Try to run โ€” and see the error

$ ./prog
./prog: error in loading shared libraries: libfoo.so: cannot open
shared object file: No such file or directory
๐Ÿ’ก Why this error? The dynamic linker searches standard directories like /lib and /usr/lib. Our libfoo.so is in the current working directory, which is not in that standard list. We need to tell the linker where to find it.

โš™๏ธ The Dynamic Linker (ld-linux.so.2)

The dynamic linker is itself a shared library named /lib/ld-linux.so.2. It is automatically invoked by the Linux kernel every time an ELF executable that uses shared libraries is started.

Runtime loading flow:

Stage Who Acts What Happens
1. execve() Kernel Reads ELF header, finds PT_INTERP segment โ†’ path to dynamic linker
2. Load dynamic linker Kernel Maps /lib/ld-linux.so.2 into process address space
3. Read DT_NEEDED ld-linux.so.2 Reads the program’s dynamic section for list of required libraries
4. Locate libraries ld-linux.so.2 Searches LD_LIBRARY_PATH, /etc/ld.so.cache, /lib, /usr/lib
5. Map libraries ld-linux.so.2 mmap()s each library into process address space
6. Relocations ld-linux.so.2 Fixes up all symbol references (global variables, function pointers)
7. Run main() ld-linux.so.2 Transfers control to the program’s entry point

Check which dynamic linker an executable uses:

# See PT_INTERP โ€” the dynamic linker path
$ readelf -l prog | grep interpreter
      [Requesting program interpreter: /lib64/ld-linux-x86-64.so.2]

# On 32-bit x86:
#   /lib/ld-linux.so.2

# On x86-64 (64-bit):
#   /lib64/ld-linux-x86-64.so.2

# On ARM:
#   /lib/ld-linux-armhf.so.3  (or similar)

# On IA-64:
#   /lib/ld-linux-ia64.so.2
๐Ÿ“ Note: /lib/ld-linux.so.2 is usually a symbolic link pointing to the actual versioned file, e.g., ld-2.35.so. The actual name tracks the glibc version installed on your system.

Verify standard library search paths of the dynamic linker:

# /etc/ld.so.conf lists extra directories searched by the dynamic linker
$ cat /etc/ld.so.conf

# ldconfig regenerates the cache /etc/ld.so.cache from the config
$ sudo ldconfig

# Print standard library paths and cache contents
$ ldconfig -v 2>/dev/null | head -20

# See all libraries in the cache
$ ldconfig -p | head -20

Check which libraries a binary needs at runtime:

# ldd traces which shared libraries an executable depends on
$ ldd prog
    linux-vdso.so.1 (0x00007ffd5c5f9000)
    libfoo.so => not found              โ† our library is missing!
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f...)
    /lib64/ld-linux-x86-64.so.2 (0x00007f...)

# Once LD_LIBRARY_PATH is set, ldd shows it found:
$ LD_LIBRARY_PATH=. ldd prog
    ...
    libfoo.so => ./libfoo.so (0x00007f...)
    ...

๐Ÿ“‚ The LD_LIBRARY_PATH Environment Variable

LD_LIBRARY_PATH is a colon-separated list of directories that the dynamic linker will search before the standard directories. It is the quickest way to run a program that uses a shared library stored in a non-standard location (e.g., current directory, custom build folder).

Basic usage โ€” run program with library in current directory:

# Tell dynamic linker to also search current directory (.)
$ LD_LIBRARY_PATH=. ./prog
Called mod1-x1
Called mod2-x2

# Multiple directories (colon-separated):
$ LD_LIBRARY_PATH=/home/ravi/libs:/tmp/test ./prog

# Prepend to existing value:
$ LD_LIBRARY_PATH=/my/lib:$LD_LIBRARY_PATH ./prog

# Export for all subsequent commands in the shell session:
$ export LD_LIBRARY_PATH=/home/ravi/libs
$ ./prog

Dynamic linker search order:

Priority Location Notes
1st Directories in LD_LIBRARY_PATH Searched first, left to right
2nd RPATH / RUNPATH in ELF binary Embedded at link time with -Wl,-rpath,/path
3rd /etc/ld.so.cache Pre-built cache from ldconfig
4th /lib and /usr/lib Default system library directories
โš ๏ธ Never use LD_LIBRARY_PATH in production!
Using LD_LIBRARY_PATH in deployed applications is a security risk โ€” a malicious library in that path could be loaded instead of the real one. For production, install libraries in standard directories or use RPATH. Also, LD_LIBRARY_PATH is ignored for setuid/setgid programs for security reasons.

๐Ÿ› ๏ธ Coding Example 2 โ€” Using RPATH (Production Alternative to LD_LIBRARY_PATH)

The proper production way to embed a library search path is to bake it directly into the ELF binary using -rpath. This way the binary always knows where to find its libraries without any environment variable.

# Embed /home/ravi/libs as RPATH inside the executable
$ gcc -g -Wall -o prog prog.c -L. -lfoo \
      -Wl,-rpath,/home/ravi/libs

# The binary will now always search /home/ravi/libs at runtime
$ ./prog          # works without LD_LIBRARY_PATH!

# Inspect the embedded RPATH:
$ readelf -d prog | grep -i rpath
 0x0000000f (RPATH)   Library rpath: [/home/ravi/libs]

# Use $ORIGIN to make RPATH relative to the executable's own directory:
# Very useful for distributable applications
$ gcc -g -Wall -o prog prog.c -L. -lfoo \
      '-Wl,-rpath,$ORIGIN'

# Now prog always looks for .so files in its own directory:
$ readelf -d prog | grep -i rpath
 0x0000000f (RPATH)   Library rpath: [$ORIGIN]
๐Ÿ’ก $ORIGIN trick: Using $ORIGIN in RPATH makes the library path relative to the directory containing the executable. This is the recommended way to ship self-contained applications or SDK packages where the .so files ship alongside the binary.

๐Ÿ› ๏ธ Coding Example 3 โ€” Static Linking vs Dynamic Linking Side by Side

Every program goes through a static-linking phase (combining object files into an executable). Programs using shared libraries additionally undergo dynamic linking at runtime.

# โ”€โ”€โ”€ CREATE THE STATIC LIBRARY (archive) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
$ ar rcs libfoo_static.a mod1.o mod2.o
# ar = archive tool; rcs = replace, create, sort-index

# โ”€โ”€โ”€ LINK STATICALLY โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
# All code from libfoo_static.a is COPIED into prog_static
$ gcc -g -Wall -static -o prog_static prog.c -L. -lfoo_static
# OR:
$ gcc -g -Wall -o prog_static prog.c libfoo_static.a

# โ”€โ”€โ”€ LINK DYNAMICALLY โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
# prog_dynamic only REFERENCES libfoo.so โ€” code stays in the .so
$ gcc -g -Wall -o prog_dynamic prog.c -L. -lfoo

# โ”€โ”€โ”€ COMPARE SIZES โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
$ ls -lh prog_static prog_dynamic
-rwxr-xr-x 1 ravi ravi 820K  prog_static   โ† BIG (all of libc included!)
-rwxr-xr-x 1 ravi ravi  16K  prog_dynamic  โ† small (just references)

# โ”€โ”€โ”€ ldd COMPARISON โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
$ ldd prog_static
    not a dynamic executable      โ† no shared lib dependencies!

$ ldd prog_dynamic
    libfoo.so => ./libfoo.so (0x00007f...)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f...)
    /lib64/ld-linux-x86-64.so.2 (0x00007f...)

Key comparison table:

Feature Static Library (.a) Shared Library (.so)
Library code location Copied into executable Stays in separate .so file
Executable size Large Small
RAM usage (10 programs) 10 copies in RAM 1 shared copy in RAM
Library update Recompile all programs Replace .so, no recompile needed
Startup speed Slightly faster (no dynamic linking) Slightly slower (dynamic linking overhead)
Portability Self-contained binary Depends on .so being present
# โ”€โ”€โ”€ See shared libraries being loaded in real time (strace) โ”€โ”€โ”€โ”€
$ strace -e openat ./prog_dynamic 2>&1 | grep "\.so"
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libfoo.so", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
# You can see exactly which .so files are opened and when!

๐Ÿงฉ 32-bit vs 64-bit Library Directories

On architectures that support both 32-bit and 64-bit execution (like x86-64), the libraries are stored in separate directories to avoid conflicts:

Architecture 32-bit Libraries 64-bit Libraries
x86-64 (Ubuntu/Debian) /lib/i386-linux-gnu, /usr/lib/i386-linux-gnu /lib/x86_64-linux-gnu, /usr/lib/x86_64-linux-gnu
x86-64 (RHEL/Fedora) /lib, /usr/lib /lib64, /usr/lib64
PowerPC64 */lib */lib64
zSeries (IBM S390x) */lib */lib64
# Check your system's lib directories
$ ldconfig -v 2>/dev/null | grep "^/"
/usr/lib/x86_64-linux-gnu:
/usr/lib:
/lib/x86_64-linux-gnu:
/lib:

# Install a shared library system-wide (add to /usr/local/lib and run ldconfig)
$ sudo cp libfoo.so /usr/local/lib/
$ sudo ldconfig        # updates /etc/ld.so.cache
$ ./prog               # now works without LD_LIBRARY_PATH!

๐ŸŽฏ Interview Questions & Answers

Q1. What are the two steps required to use a shared library that are not needed for static libraries?
Step 1 (Link time): The name of the shared library must be embedded inside the executable. The linker records it as a DT_NEEDED tag in the ELF binary’s dynamic section. This list of all required shared libraries is called the dynamic dependency list.

Step 2 (Run time): The dynamic linker (/lib/ld-linux.so.2) must locate the library file on disk and load (mmap) it into the process’s address space before the program starts executing.

Q2. What is the dynamic linker? What is its name on Linux x86-64?
The dynamic linker (also called the run-time linker or dynamic linking loader) is a special program that runs before your program’s main(). It reads the ELF binary’s DT_NEEDED tags, finds each required .so file, and maps them into memory.

On x86 (32-bit): /lib/ld-linux.so.2
On x86-64 (64-bit): /lib64/ld-linux-x86-64.so.2
The dynamic linker is itself a shared library. Both paths are usually symbolic links pointing to the actual versioned file (e.g., ld-2.35.so).

Q3. What is LD_LIBRARY_PATH and when should you NOT use it?
LD_LIBRARY_PATH is an environment variable containing a colon-separated list of directories that the dynamic linker searches before the standard system directories (/lib, /usr/lib).

Do NOT use it in production because:
โ€ข It is a security risk โ€” a malicious .so placed in that path could hijack your program.
โ€ข It is completely ignored for setuid/setgid executables (security measure).
โ€ข It affects all child processes, potentially breaking other programs.

The correct production alternatives are: install the library in /usr/lib (and run ldconfig), or embed an RPATH using -Wl,-rpath,/path at link time.

Q4. What is the difference between static linking and dynamic linking?
Static linking: The linker (ld) copies all required object code from static libraries (.a archives) directly into the executable. No .so files are needed at runtime. Produces larger binaries but fully self-contained ones.

Dynamic linking: The executable only contains references (DT_NEEDED tags) to shared libraries. The dynamic linker resolves these references at runtime by finding and loading the .so files. Produces smaller binaries; multiple programs share one copy of the library in RAM. Every program using shared libraries undergoes both a static-linking phase (creating the executable) and a dynamic-linking phase (at runtime).

Q5. What does the -fPIC flag do and why is it required for shared libraries?
-fPIC stands for Position Independent Code. Normally, compiled code uses absolute memory addresses (the code assumes it will be loaded at a fixed address). But a shared library can be loaded at different addresses in different processes.

With -fPIC, the compiler generates code that uses relative addresses (accessed via a Global Offset Table, GOT, and Procedure Linkage Table, PLT). This allows the same .so file to be mapped at any address in any process while still working correctly. Without -fPIC, creating a shared library will either fail or produce a library that doesn’t work correctly on most architectures.

Q6. How do you inspect what shared libraries an executable needs? Name two tools.
1. ldd: The most common tool. It prints each required shared library and the path where it was found (or “not found”).
$ ldd ./prog

2. readelf -d: Reads the ELF dynamic section directly and shows DT_NEEDED tags (and other tags like RPATH, SONAME).
$ readelf -d prog | grep NEEDED

3. objdump -p: Similar to readelf, shows the dynamic section in a readable format.
$ objdump -p prog | grep NEEDED

Note: ldd actually runs the dynamic linker, so never run ldd on an untrusted binary. Use readelf or objdump for safe inspection.

Q7. What is $ORIGIN in RPATH and why is it useful?
$ORIGIN is a special token in RPATH that the dynamic linker expands to the directory containing the executable at runtime. For example, if your binary is at /opt/myapp/bin/prog and its RPATH is $ORIGIN/../lib, the dynamic linker looks in /opt/myapp/lib.

This is extremely useful for distributable, relocatable applications where the binary and its libraries ship together in a fixed relative directory structure. You don’t need to know the absolute install path at build time.

Q8. Why does the kernel not load the shared libraries directly? Why use a separate dynamic linker?
The kernel’s job is to load an ELF binary and start execution โ€” it keeps the kernel code simpler and smaller by not implementing the full library-loading logic there. Implementing dynamic linking in the kernel would make it more complex and harder to update.

Instead, the kernel only needs to load the dynamic linker (ld-linux.so.2) by reading the PT_INTERP segment of the ELF binary. The dynamic linker then handles all the complex work: parsing ELF files, searching for libraries, relocations, and executing init functions. This separation also allows the dynamic linker to be updated independently of the kernel.

Continue Learning

Next: Learn about Shared Library Sonames โ€” the versioning alias mechanism

Next โ†’ Shared Library Soname โ†ฉ Back to Index

Leave a Reply

Your email address will not be published. Required fields are marked *