Anatomy Of A Cross Toolchain
Open up the toolchain directory you just built and learn what every file inside it actually does.
free embedded linux course
free linux device drivers course
What You Will Learn
What the sysroot is and why it exists
The binutils command-line toolbox, one tool at a time
What actually makes up “the C library” today
Prerequisites
Basic familiarity with ELF binaries
Interrogating a Toolchain You Didn’t Build
You will very often work with a toolchain someone else configured: a vendor SDK, a coworker’s crosstool-NG output, or a Yocto-generated SDK installer. Before trusting it, ask it questions. Start with the version:
$ aarch64-linux-gnu-gcc --version
aarch64-linux-gnu-gcc (GCC) 16.1.0
Copyright (C) 2026 Free Software Foundation, Inc.Then ask how it was actually configured, which reveals far more than the version string alone:
$ aarch64-linux-gnu-gcc -v
Using built-in specs.
Target: aarch64-linux-gnu
Configured with: ../configure --target=aarch64-linux-gnu \
--prefix=/opt/toolchains/aarch64-linux-gnu \
--with-sysroot=/opt/toolchains/aarch64-linux-gnu/aarch64-linux-gnu/sysroot \
--enable-languages=c,c++ --with-arch=armv8-a --enable-threads=posix \
--disable-multilib
Thread model: posix
gcc version 16.1.0The two lines worth memorizing: --with-sysroot tells you where the target’s headers and libraries live, and --enable-languages tells you what this particular build can actually compile. Not every toolchain build includes C++, so check before you assume it does.
The Sysroot: Your Target’s Filesystem, in Miniature
The sysroot is a directory tree that mimics the layout of your target’s root filesystem, holding exactly the headers and libraries the compiler and linker need to build code for that target. You can always ask a compiler where its default sysroot lives:
$ aarch64-linux-gnu-gcc -print-sysroot
/opt/toolchains/aarch64-linux-gnu/aarch64-linux-gnu/sysroot| Path | Contains | Needed on |
|---|---|---|
| lib/ | Shared C library objects and the dynamic linker | Target, at runtime |
| usr/lib/ | Static library archives (.a files) | Host, at build time |
| usr/include/ | Header files for every library in the sysroot | Host, at build time |
| usr/bin/ | Target-side utilities, such as ldd | Target, at runtime |
| usr/share/ | Localization and internationalization data | Target, at runtime |
| sbin/ | The ldconfig utility for optimizing library search paths | Target, at runtime |
Notice the split: some of this directory is only useful on your development host while you’re compiling, and some of it must physically exist on the target board or nothing will run. Confusing the two is one of the most common early mistakes when hand-assembling a root filesystem, rather than letting Buildroot or Yocto do it for you.
Meet the Other Tools in Your Toolchain
GCC gets all the attention, but a GNU toolchain ships more than a dozen supporting utilities, most of them from binutils:
| Command | What it does |
|---|---|
| addr2line | Turns crash-report addresses back into source file names and line numbers |
| ar | Creates and manages static library archives |
| as | The GNU assembler |
| c++filt | Demangles C++ symbol names into readable form |
| cpp | The C preprocessor, expanding #define and #include directives |
| elfedit | Edits fields in an ELF file’s header |
| gcov | Code coverage analysis tool |
| gdb | The GNU debugger (or lldb, if you’re using an LLVM toolchain) |
| ld | The GNU linker (or lld, LLVM’s linker) |
| nm | Lists the symbols defined in an object file |
| objcopy | Copies and translates object files between formats |
| objdump | Dumps human-readable information from object files |
| ranlib | Builds a symbol index inside a static library for faster linking |
| readelf | Displays detailed information about ELF-format files |
| size | Reports section and total sizes of a binary |
| strings | Extracts printable character sequences from a file |
| strip | Removes debug symbols to shrink a binary before deploying it |
Two of these, readelf and objdump, are the ones you will reach for constantly once you start debugging why a binary refuses to run on target, so it’s worth practicing with them now rather than waiting for an emergency.
What “the C Library” Is Actually Made Of Today
Older material describes the C library as four separate pieces: libc for core POSIX functions, libm for math functions, libpthread for POSIX threads, and librt for real-time extensions like shared memory and async I/O. That description is now out of date for glibc. Starting with glibc 2.34, the pthread and most of the realtime and dl functions were merged directly into libc.so.6 itself. The separate libpthread.so.0 and librt.so.1 files still exist on current glibc systems only as thin, mostly empty compatibility stubs kept around so that old binaries linked against them don’t break.
| Function group | Pre-2.34 glibc | glibc 2.34 and later |
|---|---|---|
| Core POSIX (printf, open, read) | libc.so.6 | libc.so.6 |
| Math (cos, exp, log) | libm.so.6 | libm.so.6 (still separate) |
| POSIX threads (pthread_*) | libpthread.so.0 | Merged into libc.so.6; libpthread.so.0 is a compatibility stub |
| Real-time extensions (shm, aio) | librt.so.1 | Merged into libc.so.6; librt.so.1 is a compatibility stub |
Practically, this means you can still link with -lpthread for portability with older code, and it will keep working, but on a modern glibc target you no longer strictly need it: the functions are already in libc. musl took this unified approach from the very beginning, never splitting pthread and realtime support into separate libraries at all.
Try It: Inspect Your Own Binary
Reuse ep_hello.c from the first lecture and look at what it actually links against on a current glibc target:
$ aarch64-linux-gnu-gcc ep_hello.c -o ep_hello -lpthread
$ aarch64-linux-gnu-readelf -d ep_hello | grep NEEDED
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]Notice there is no separate libpthread.so.0 in the NEEDED list even though we explicitly linked with -lpthread: on modern glibc, the linker recognizes those symbols already live in libc and doesn’t add a redundant dependency.
Common Mistakes
| Mistake | Why it hurts |
|---|---|
| Copying only usr/include from a sysroot to a target board | Headers are a build-time artifact only; the target needs the runtime lib/ contents instead |
| Assuming libpthread.so.0 must ship on every target | On glibc 2.34+, it’s an empty stub; omitting it may still be fine, but always verify with ldd |
| Never running readelf or objdump until something breaks | These tools are diagnostic muscle memory you want built up before an emergency, not during one |
| Shipping unstripped binaries to a storage-constrained target | Debug symbols can multiply binary size several times over |
Best Practices
Learn readelf -d and objdump -T early, not during a debugging emergency
Keep debug symbols in a separate file with objcopy –only-keep-debug, then strip production binaries
Check your glibc version before assuming libpthread or librt are required at link time
Summary and Key Takeaways
A toolchain’s sysroot mirrors your target’s filesystem just enough to compile and link against it, cleanly separating host-only build artifacts from things that must physically exist on the target. Binutils supplies a whole toolbox beyond just the compiler, and readelf and objdump are worth learning well ahead of your first real debugging session. And the C library itself has quietly consolidated: since glibc 2.34, pthread and realtime functions live directly inside libc, a detail that trips up anyone still working from older references.
Next, we look at how your programs actually connect to these libraries at build time and runtime, through static and dynamic linking.
FAQ
What’s the difference between the toolchain’s own files and the sysroot?
The toolchain’s bin/ directory holds the host-side compiler and utilities you run on your development machine. The sysroot holds headers and libraries representing the target, some needed only at build time and some needed on the actual target device.
Do I still need to link with -lpthread on a modern glibc system?
You can, for compatibility with older build scripts, and it will still work. But since glibc 2.34, the pthread functions are already part of libc.so.6, so it is no longer strictly required.
What is the single most useful binutils command for debugging a broken binary?
readelf -d, which shows the dynamic section including NEEDED library dependencies, is usually the fastest way to spot a missing or unexpected shared library dependency.
Why does gcc -v show so much more than gcc –version?
–version just prints the compiler’s release string. -v shows the exact ./configure options the toolchain was built with, including the sysroot path and enabled languages, which is what you actually need to diagnose configuration issues.
Is librt.so.1 gone entirely on modern systems?
Not gone, but hollowed out. On glibc 2.34 and later it exists mainly as a compatibility stub so that binaries explicitly linked against it don’t fail to load, while the real implementation lives in libc.so.6.
Next: static vs dynamic linking, explained clearly