Debugging Leaks With Valgrind
Full memory-error detection without recompiling — free linux kernel development course
mtrace, from the last lecture, only tells you about leaks after your program exits and only catches unmatched malloc/free calls. Valgrind, the subject of this lecture, goes much further: it runs your unmodified binary inside an emulated CPU, catching leaks, use-after-free, and general memory misuse, all without a single source code change. This is one of the most valuable tools in any free linux kernel development course toolbox — right up until real-time constraints make its overhead unacceptable.
Keywords covered
What You Will Learn
- How Valgrind’s emulated-CPU approach differs from mtrace’s hook-based approach
- Valgrind’s diagnostic tools beyond memcheck: cachegrind, callgrind, helgrind, DRD, massif
- How to run memcheck with full leak checking against an original demo
- How to read a Valgrind leak report and map it back to source
- Where Valgrind is a poor fit on embedded targets, and why
Prerequisites
Comfort with the C toolchain and the VSS/RSS/USS/PSS concepts from earlier lectures. Valgrind itself needs to be available for your target architecture — ARM Cortex-A, x86, MIPS, and PPC are all supported, and it ships as a package option in both Yocto and Buildroot.
How Valgrind Differs From mtrace
Where mtrace relies on glibc hooking calls you must add yourself, Valgrind runs your program inside its own CPU emulator (the “synthetic CPU” underneath tools like memcheck), intercepting every memory access — reads, writes, and allocator calls — without any change to your source. You do not even need to recompile: Valgrind works on unmodified binaries, although compiling with -g gives it debug symbols so its reports point at real source lines.
That thoroughness has a real cost: because every instruction runs through emulation, programs under Valgrind typically run at a small fraction of native speed. For firmware or drivers with real-time deadlines, that overhead alone can rule Valgrind out for in-situ testing — you would instead run it against a desktop or CI build of the same logic where timing is not load-bearing.
Valgrind’s Tool Suite
Valgrind is a framework, not a single checker. Select a tool with --tool=:
| Tool | Purpose |
|---|---|
| memcheck | Default — detects leaks and general memory misuse (this lecture’s focus) |
| cachegrind | Calculates processor cache hit rate |
| callgrind | Calculates the cost of each function call |
| helgrind | Detects Pthread API misuse, potential deadlocks and race conditions |
| DRD | A second, independent Pthread analysis tool |
| massif | Profiles heap and stack usage over time |
Original Demo: Reusing The Leak Case
To make the comparison with mtrace concrete, this lecture reuses the same leak shape as before, but as an independent, original binary (ep_valdemo) — a small array-backed buffer pool where one buffer is never released:
/* ep_valdemo.c - deliberate leak for Valgrind's memcheck */
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#define POOL_SIZE 4
struct ep_buffer {
char *data;
size_t len;
};
int main(void)
{
struct ep_buffer pool[POOL_SIZE];
for (int i = 0; i < POOL_SIZE; i++) {
pool[i].len = 64;
pool[i].data = malloc(pool[i].len);
memset(pool[i].data, 0, pool[i].len);
}
/* Release only 3 of the 4 buffers - index 2 is leaked */
free(pool[0].data);
free(pool[1].data);
free(pool[3].data);
printf("ep_valdemo finished\n");
return 0;
}
Running memcheck With Full Leak Checking
Build with debug symbols, then run under Valgrind’s default tool with --leak-check=full to get a per-allocation report:
$ gcc -g -Wall -o ep_valdemo ep_valdemo.c
$ valgrind --leak-check=full ./ep_valdemo
==8421== Memcheck, a memory error detector
==8421== Command: ./ep_valdemo
==8421==
ep_valdemo finished
==8421==
==8421== HEAP SUMMARY:
==8421== in use at exit: 64 bytes in 1 blocks
==8421== total heap usage: 4 allocs, 3 frees, 256 bytes allocated
==8421==
==8421== 64 bytes in 1 blocks are definitely lost in loss record 1 of 1
==8421== at 0x4C2FB0F: malloc (vg_replace_malloc.c:381)
==8421== by 0x1091A2: main (ep_valdemo.c:19)
==8421==
==8421== LEAK SUMMARY:
==8421== definitely lost: 64 bytes in 1 blocks
==8421== indirectly lost: 0 bytes in 0 blocks
==8421== possibly lost: 0 bytes in 0 blocks
==8421== still reachable: 0 bytes in 0 blocks
==8421== ERROR SUMMARY: 1 errors from 1 contexts
Valgrind isolates the exact allocation (ep_valdemo.c:19) that was never freed, and — unlike mtrace — would have also caught related bugs like reading past the end of one of these buffers or freeing the same pointer twice, none of which mtrace is designed to detect.
Reading “Definitely Lost” vs Other Categories
| Category | Meaning |
|---|---|
| Definitely lost | No pointer to this block exists anywhere — a true leak |
| Indirectly lost | Lost because it was only reachable through a definitely-lost block |
| Possibly lost | A pointer exists but only to somewhere inside the block, not its start |
| Still reachable | Never freed but still pointed to at exit — often intentional (e.g. static caches) |
Common Mistakes
- Running Valgrind against real-time embedded code paths expecting normal timing — the emulation overhead invalidates any timing-sensitive test.
- Skipping
-g— Valgrind still works, but loss records show raw addresses instead of file:line. - Treating “still reachable” as an error — it usually is not; check whether the block is meant to live for the program’s lifetime before chasing it.
Best Practices
- Run Valgrind’s memcheck routinely in CI against host or QEMU builds of your embedded application logic, separate from real-time hardware testing.
- Combine memcheck for correctness with massif for heap growth profiling when you suspect gradual bloat rather than a hard leak.
- Use helgrind or DRD whenever you touch shared state across pthreads — a class of bug mtrace cannot see at all.
Performance Considerations
Expect roughly 10-50x slowdown under memcheck depending on workload, since every memory access is intercepted. Never use Valgrind as a substitute for real hardware timing validation; use it purely for correctness.
Summary And Key Takeaways
Valgrind’s memcheck catches leaks and a much broader class of memory bugs than mtrace, on unmodified binaries, at the cost of significant runtime slowdown from CPU emulation. It is the right tool once mtrace’s exit-only, malloc/free-only view is not enough — but it is not a fit for anything with real-time constraints.
Conclusion
Together with mtrace, Valgrind rounds out the memory-debugging toolkit every engineer following a free linux kernel development course should carry: mtrace for cheap, always-on leak checks; Valgrind for deep, occasional correctness sweeps.
FAQ
Do I need to recompile my program to use Valgrind?
No, Valgrind works on unmodified binaries; compiling with -g only adds source file:line information to its reports.
Which Valgrind tool finds memory leaks?
memcheck, the default tool, run with the –leak-check=full option for detailed per-allocation reports.
Why does my program run so slowly under Valgrind?
Valgrind emulates the CPU to intercept every memory access, which typically slows execution by an order of magnitude or more.
Is Valgrind suitable for real-time embedded testing?
No, its emulation overhead makes timing-sensitive testing unreliable; use it for correctness checks on host or QEMU builds instead.
What does “still reachable” mean in a Valgrind leak summary?
Memory that was never freed but is still pointed to when the program exits, often intentional such as static caches.
Does Valgrind run on ARM embedded targets?
Yes, Valgrind supports ARM Cortex-A, PPC, MIPS, and x86 in 32- and 64-bit variants, and is packaged for both Yocto and Buildroot.
Can Valgrind catch bugs mtrace cannot?
Yes, it also detects use-after-free, double-free, and out-of-bounds accesses, none of which mtrace is designed to catch.
Continue The Free Linux Kernel Development Course
Next up: memory overcommit and the OOM killer — what happens when a board truly runs out of RAM.
Next Lecture Course Index
2 Comments