Linux Kernel OOM Killer Explained: How and Why the Kernel Kills Processes
A simple, updated guide to the Linux kernel OOM killer — how memory reclaim works, how the kernel picks a victim process, and how to control or avoid it on modern systems.
Every Linux user has run into it at least once: the system grinds to a halt, and then suddenly a process is gone with no warning. That is the Linux kernel OOM killer in action — a controversial but essential kernel component designed to keep the system alive when memory runs out completely. In this lecture, part of our free Linux kernel development course, we break down exactly how the OOM killer works, how it scores and selects a victim, and how you can influence or avoid it on modern kernels, including cgroup v2 systems.
Topics covered in this free Linux kernel development course lecture:
kswapd
memory reclaim
oom_score_adj
cgroup v2 memory
watermark levels
embedded systems memory
What You Will Learn
- How the kernel reclaims memory before it ever considers killing anything
- What kswapd and watermark levels actually do
- How the OOM killer scores and selects a victim process
- How to read and tune oom_score_adj
- How cgroup v2 changes OOM behavior for containers
- How to deliberately trigger or observe OOM events safely
Prerequisites
Basic familiarity with Linux processes and the /proc filesystem is helpful. No kernel-internals experience is required — this lecture starts from the fundamentals.
Memory Reclaim: The Kernel’s First Line of Defense
Before the OOM killer ever gets involved, the kernel spends a lot of effort trying to avoid running out of memory in the first place. Linux tries to keep the “working set” of actively used memory pages as close to the CPU as possible for performance, spilling over into RAM and eventually swap only as pressure builds. A background kernel thread called kswapd continuously monitors memory usage per memory zone and proactively reclaims pages — shrinking caches first — long before things get critical.
Watermark Levels: How the Kernel Knows When to Act
The kernel tracks three watermark levels for each memory zone: min, low, and high, measured in pages. When free memory drops below the low watermark, kswapd wakes up and starts reclaiming in the background. If memory keeps falling toward the min watermark even under active reclaim, the kernel escalates to direct reclaim, and eventually to the OOM killer as a last resort. You can inspect live watermark values on any running system by reading /proc/zoneinfo.
cat /proc/zoneinfo | grep -A 3 "Node 0, zone Normal"
Visual Overview: From Memory Pressure to OOM Kill
How the OOM Killer Chooses a Victim Process
Once the kernel decides it truly has no other option, it does not kill processes at random. Every running process is assigned a “badness” score, primarily influenced by how much memory it is using, with adjustments for factors like how long it has been running and whether it is a privileged system process. The process with the highest resulting score is chosen because killing it is expected to free the most memory for the least overall damage to the system.
Controlling the OOM Killer with oom_score_adj
You are not powerless here. Every process exposes an oom_score_adj value under /proc that lets you bias the kernel’s decision, ranging from -1000 (never kill this process) to +1000 (kill this first). This is commonly used in embedded systems to protect critical processes, such as a watchdog daemon, from ever being selected as a victim.
cat /proc/<pid>/oom_score
echo -500 > /proc/<pid>/oom_score_adj
Use this power carefully: protecting one process always means another one becomes relatively more likely to be picked when memory truly runs out.
OOM Behavior in cgroup v2 and Containers
On modern systems using cgroup v2, which is now the default on most current Linux distributions, memory limits and OOM handling can be scoped per control group rather than system-wide. A container or service can be given a hard memory.max limit, and if it is exceeded, the OOM killer can act within that cgroup alone, optionally killing every process in the group together via memory.oom.group, instead of reaching for an unrelated process elsewhere on the system. This is a major usability improvement for container platforms compared to older whole-system-only OOM behavior.
Comparison Table: OOM Control Mechanisms
| Mechanism | Scope | Typical Use Case |
|---|---|---|
| oom_score_adj | Single process | Protect or sacrifice a specific process |
| cgroup v2 memory.max | Control group | Hard memory cap for a container/service |
| memory.oom.group | Control group | Kill an entire related group together |
| panic_on_oom sysctl | System-wide | Force a reboot instead of killing a process |
Real-World Use Case: Protecting an Embedded Watchdog Process
Consider an embedded Linux device running a critical watchdog daemon alongside a memory-hungry application. If the application leaks memory and triggers an OOM event, you do not want the watchdog killed along with it — that would leave the system with no safety net at all. Setting the watchdog’s oom_score_adj to a strongly negative value ensures the kernel will exhaust every other option before ever touching it, a pattern commonly used across embedded systems engineering.
Common Mistakes and Troubleshooting
- Assuming an OOM kill means a hardware fault, when it is usually a software memory leak
- Setting oom_score_adj to -1000 on too many processes, leaving the kernel no safe victim at all
- Ignoring dmesg logs, which record exactly why and which process was selected during an OOM event
- Confusing swap exhaustion with RAM exhaustion when diagnosing OOM events
Best Practices
- Always check dmesg or the kernel log after an unexpected process death
- Use cgroup v2 memory limits to contain memory-hungry services on shared systems
- Reserve oom_score_adj protection for genuinely critical processes only
- Monitor memory trends proactively instead of relying on the OOM killer as a strategy
Performance Considerations
An OOM event is inherently disruptive — reclaiming memory under pressure, and the process kill itself, both cost CPU time and can stall unrelated workloads momentarily. Systems tuned with sensible cgroup v2 limits tend to experience far more predictable performance than systems that rely on the global OOM killer to sort things out after the fact.
Security Considerations
A process that can deliberately exhaust system memory is effectively performing a denial-of-service attack, so memory limits and cgroup isolation are as much a security control as a stability one. On multi-tenant or embedded systems exposed to untrusted workloads, always cap memory per cgroup rather than trusting global OOM behavior to protect critical services.
Summary and Key Takeaways
- kswapd reclaims memory proactively long before the OOM killer is involved
- Watermark levels (min, low, high) drive when reclaim and OOM escalate
- The OOM killer scores processes and kills the one expected to free the most memory
- oom_score_adj lets you protect or sacrifice specific processes
- cgroup v2 enables per-container OOM handling on modern systems
Conclusion
The Linux kernel OOM killer often gets a bad reputation, but it exists purely to keep an out-of-memory system alive rather than letting it freeze completely. Once you understand the escalation path — from proactive reclaim, to direct reclaim, to a scored victim selection — the OOM killer stops feeling random and starts feeling like exactly what it is: a last-resort safety mechanism you can observe, understand, and even tune to protect the processes that matter most on your system.
Frequently Asked Questions
Q1: What triggers the Linux kernel OOM killer?
It activates only after background and direct memory reclaim both fail to free enough pages to satisfy a request.
Q2: How does the OOM killer pick which process to kill?
It calculates a badness score for every process, largely based on memory usage, and kills the process with the highest score.
Q3: Can I prevent a specific process from ever being killed?
You can strongly bias the decision using oom_score_adj set to -1000, though the kernel may still act if no other option exists.
Q4: What is the difference between kswapd and the OOM killer?
kswapd proactively reclaims memory in the background to avoid a crisis, while the OOM killer only acts as a last resort once reclaim has failed.
Q5: Does cgroup v2 change how the OOM killer behaves?
Yes — it allows memory limits and OOM handling to be scoped to a specific control group instead of always acting system-wide.
Q6: How can I see why a process was killed by the OOM killer?
Check the kernel log with dmesg, which records the OOM event along with the scores considered and the process selected.
Q7: Is the OOM killer a sign of a hardware problem?
No — it almost always indicates a software memory leak or genuinely insufficient RAM for the workload, not faulty hardware.
Q8: Is this explanation valid for the latest Linux kernels?
Yes — the reclaim and OOM escalation path described here reflects current mainline kernel behavior, including cgroup v2 systems in wide use today.
Continue Your Free Linux Kernel Development Course
This lecture is part of EmbeddedPathashala’s free Linux kernel programming and device driver course.