PREV_LEC | NEXT_LEC

What You Will Learn

• The three active release types of the Linux kernel: mainline, stable, and LTS
• How a patch travels from a subsystem maintainer to Linus Torvalds’s tree, and back down into stable branches
• What a release candidate is, and how the vX.Y and vX.Y.Z version numbers actually work
• The backporting rule that governs every fix that lands in a stable release
• How to inspect the kernel you are running right now, from both the shell and a small C program

Prerequisites

• Comfort with basic Linux command-line usage
• Basic C programming knowledge (for the version-check example)
• A Linux machine or VM running a reasonably recent kernel (6.x recommended) — this lesson is version-agnostic, so any modern distribution works

Why the Release Model Matters for Debugging

Every free embedded Linux course eventually collides with this reality: a driver that misbehaves on one machine works perfectly on another, and the only difference is the kernel version. Before touching a debugger or a tracer, you need a mental model of how kernel code moves from a developer’s patch to the kernel running on your board. That model has exactly three moving parts.

Mainline: Where Every Change Is Born

Subsystem maintainers collect patches from individual developers, review them inside their own subsystem trees, and eventually send pull requests to Linus Torvalds. Once merged into his tree — commonly called the mainline tree or the master repository — a change is considered “upstream.” This is the single point of origin for every fix that will ever reach a stable or LTS branch; nothing skips this step.

Each mainline development cycle opens with a merge window of about two weeks, during which new features are pulled in. After the merge window closes, Linus starts tagging weekly release candidates: -rc1, -rc2, and so on, typically up to -rc7 or -rc8. Release candidates exist purely for testing — distribution maintainers, driver authors, and CI systems run them to surface regressions before the final tag. When Linus is satisfied that a release candidate is solid, he tags the final version as vX.Y (for example, v6.9), and the cycle restarts. A full mainline cycle — merge window plus RC testing — usually takes nine to ten weeks, and a new mainline version ships roughly every two to three months.

Stable Releases and the Backporting Rule

The moment a mainline version is tagged, it also becomes the head of a new stable branch — for example, tagging v6.9 creates the 6.9.y stable branch. From that point forward, this branch only receives targeted bug and security fixes; no new features are ever added to a stable branch. Each fix that lands here is called a bugfix kernel release, and the branch is numbered vX.Y.1, vX.Y.2, vX.Y.3, and so on — commonly shortened to X.Y.y when referring to the branch as a whole rather than a specific point release.

There is one rule that overrides everything else in this process: a fix can only be backported to a stable branch after it has already been merged into Linus’s mainline tree. This is not a convention — it is a hard acceptance criterion enforced by the stable maintainers. The logic is simple: mainline is the single source of truth, so a fix must “go forward” into mainline before it is allowed to “come back” into an older, still-supported branch. If you ever see a patch series submitted directly to the stable mailing list without first being in Linus’s tree, it will be rejected unless it falls under a small set of documented exceptions (such as build fixes specific to an older branch).

Long-Term Support (LTS) Kernels

Most stable branches only receive a handful of bugfix releases before the next mainline version arrives and attention moves on. A small number of branches, however, are designated as Long-Term Support kernels and continue receiving backported fixes for several years. These LTS branches are what almost every production embedded Linux board, server distribution, and Android device is actually built on — nobody ships a bare mainline kernel to production. If you are following any free embedded Linux course with the goal of building real products, the LTS branch is the one you should default to for a board bring-up.

Release Type	Contains New Features	Typical Lifetime	Who Maintains It	Typical Use
Mainline	Yes	Until next -rc1	Linus Torvalds	Kernel development, testing new APIs
Stable	No (fixes only)	Until next mainline release supersedes it	Greg Kroah-Hartman + stable team	Short-term bugfix tracking
LTS	No (fixes only)	Multiple years	Stable team (designated branch)	Production embedded / server systems

Checking the Kernel You Are Actually Running

Before debugging anything, confirm exactly which kernel and branch you are on. The quickest checks are from the shell:

$ uname -r
6.9.7-arm64

$ cat /proc/version
Linux version 6.9.7 (builder@ep-buildhost) (gcc (GNU) 13.2.0) #1 SMP PREEMPT Wed Jun 5 10:12:03 UTC 2026

The three numbers in 6.9.7 map directly onto what we covered above: 6 is the major version, 9 is the mainline minor version (the vX.Y that Linus tagged), and 7 is the stable bugfix release number backported on top of it.

You can also confirm this programmatically from user space using the uname() system call, which is a good first exercise for anyone building intuition for later chapters on off-target and kernel-assisted debugging:

/* ep_kernel_version.c - original demo for this lecture */
#include <stdio.h>
#include <sys/utsname.h>

int main(void)
{
    struct utsname info;

    if (uname(&info) != 0) {
        perror("uname");
        return 1;
    }

    printf("Kernel release : %s\n", info.release);
    printf("Kernel version : %s\n", info.version);
    printf("Machine arch   : %s\n", info.machine);

    return 0;
}

Build and run it:

$ gcc -o ep_kernel_version ep_kernel_version.c
$ ./ep_kernel_version
Kernel release : 6.9.7-arm64
Kernel version : #1 SMP PREEMPT Wed Jun 5 10:12:03 UTC 2026
Machine arch   : aarch64

Common Mistakes

• Treating “stable” as a synonym for “safe to run in production.” A stable branch is only actively receiving fixes until the next mainline release ships; use an LTS branch if you need years of support.
• Assuming the newest mainline release is the most tested. It is the newest, not the most battle-tested — LTS branches have far more real-world hours behind them.
• Cherry-picking a fix directly into a product kernel without checking whether it is already upstream. This breaks the backport contract and makes future rebasing painful; always push the fix upstream first, then backport it downward.
• Confusing the bugfix number with a security-only counter. Bugfix releases (vX.Y.Z) bundle all accepted fixes, security and non-security alike — you cannot infer severity from the number alone.

Best Practices

• Pin production and embedded boards to an LTS branch, not a plain stable or mainline branch.
• Track the end-of-life date for your chosen LTS branch and plan your next migration well before it, since security backports stop entirely after end-of-life.
• When you find a bug, always try to reproduce it on the latest mainline release candidate first — if it is already fixed upstream, you just need the backport instead of writing a new patch.
• Security consideration: security fixes flow through exactly the same mainline-first backport path as ordinary bug fixes, so staying current on your LTS branch’s bugfix releases is your primary defense, not an optional extra.
• Performance consideration: avoid pulling isolated performance patches out of mainline into an old stable branch without their surrounding series — kernel performance fixes are frequently interdependent and can regress other subsystems if backported partially.

Summary and Key Takeaways

• The kernel has three active release types: mainline, stable, and LTS.
• All changes originate in Linus Torvalds’s mainline tree; nothing reaches a stable branch without first existing there.
• Release candidates (-rcN) exist purely for pre-release testing during each roughly nine-to-ten-week mainline cycle.
• Stable branches (X.Y.y) receive bugfix-only backports; LTS branches are the subset maintained for years and are the right target for production systems.
• Knowing exactly which branch and bugfix number you are running is the first real debugging step, and uname -r or a small uname() program gets you there instantly.

Understanding this release model is what makes the rest of this chapter — logging, tracing, and off-target debugging — make sense in context. It is also one of the most practical topics in any free Linux kernel development course, because nearly every real debugging session starts with “which kernel, and where did this fix come from?” The next lecture in this chapter moves from theory into practice, starting with the kernel’s built-in logging APIs.

FAQ

What is the difference between a stable kernel and an LTS kernel?

Every mainline release becomes a stable branch, but only a small number of those branches are additionally designated Long-Term Support and maintained for years. All LTS branches are stable branches; not all stable branches become LTS.

How often is a new mainline Linux kernel released?

There is no fixed schedule, but in practice a new mainline version ships roughly every two to three months, following a merge window plus a run of weekly release candidates.

Can a bug fix go straight into a stable kernel branch without going through mainline first?

No. The stable maintainers require that a fix already exist in Linus’s mainline tree before it can be backported, with only a few narrow documented exceptions.

Who maintains the Linux stable kernel tree?

Greg Kroah-Hartman maintains the stable tree, coordinating which fixes get backported into each supported branch.

What does the third number in a kernel version like 6.9.7 mean?

It is the bugfix release number for that branch — the count of backported fix releases applied on top of the 6.9 mainline version.

Why does this matter for a free embedded systems course?

Board bring-up and driver debugging both depend on knowing which kernel branch you’re on and whether a given fix is already backported — it is foundational knowledge before you touch a single debugging tool.

How do I check which kernel release candidate or version I’m running?

Run uname -r from the shell, or read /proc/version, or call the uname() system call from a small C program as shown in this lecture.

Should I always use the LTS kernel for embedded projects?

For production systems, yes — LTS branches give you years of backported security and bug fixes without needing to track fast-moving mainline development.

PREV_LEC | NEXT_LEC