How Does Linux Track Processes and Threads? – Free Linux Device Driver Course

Free Linux Kernel Development Course — EmbeddedPathashala

How the Linux Kernel Tracks Every Process and Thread: Task List Iteration Explained

Learn how the Linux kernel keeps track of every running process and thread using the task_struct data structure and how to walk the entire task list from inside a kernel module — with practical, modern code examples.

📚 Level
Intermediate
⏱ Read Time
~20 min
💻 Course
Free Linux Kernel Dev
🆕 Kernel
6.x (LTS)

Welcome to this lecture of the free Linux kernel development course on EmbeddedPathashala. In this article you will learn one of the most fundamental skills every kernel and device driver developer needs: understanding how the Linux kernel manages all running processes and threads internally, and how to traverse that list programmatically from a loadable kernel module.

If you have ever typed ps aux or top and wondered how the kernel supplies that information so quickly, this tutorial answers that question from the inside. This is not just theory — you will write a real kernel module that walks every thread on a live system.

What You Will Learn

  • What the task list is and why it is the heart of Linux process management
  • How the kernel represents every process and thread with a single task_struct
  • The difference between a PID and a TGID and why it matters in a multi-threaded program
  • How to tell a kernel thread apart from a user-space thread at the kernel level
  • How to use do_each_thread() and while_each_thread() to safely walk every live thread
  • How to read a thread’s name, stack address, and thread count from inside a module
  • Common pitfalls such as deadlocks when fetching thread names
  • How modern kernel versions (5.x / 6.x) affect these APIs

Prerequisites

  • Basic C programming knowledge
  • A Linux system with kernel headers installed (Ubuntu 22.04 / 24.04 recommended)
  • Familiarity with writing and loading a simple “Hello World” kernel module
  • Understanding of what a process and a thread are at the user-space level

If you are brand new to kernel modules, start with the Introduction to Kernel Modules lecture in this free Linux kernel development course first.

The Kernel Task List: How Linux Knows About Every Process

When the Linux kernel boots, one of the very first things it creates is a process — process number 1, known as init (or systemd on modern distros). From that moment on, every single process and thread that ever runs on the system gets registered in a global doubly-linked circular list maintained by the kernel. This is called the task list.

Each entry in this list is a structure of type task_struct. Think of task_struct as the kernel’s “file” on a running entity. It stores everything the kernel needs to know: the process ID, scheduling information, memory mappings, open file descriptors, signal handlers, the kernel-mode stack address, the thread’s name, and much more.

The important thing to understand early is that in Linux, both processes and threads are represented by a task_struct. There is no separate “thread descriptor.” A process with four threads will have four task_struct instances in the task list. The kernel distinguishes threads from standalone processes using the TGID field, explained below.

Linux Kernel Task List — Circular Doubly-Linked List
init_task
PID 0 (idle)
↔ systemd
PID 1
↔ kworker
kernel thread
↔ bash
PID N
↔ •••
Circular list — last entry points back to init_task

The Special Case: the CPU Idle Thread

Every CPU core has one very special thread that runs when there is absolutely nothing else to do: the idle thread, named swapper/N where N is the CPU core number. Core 0’s idle thread is called swapper/0, core 1’s is swapper/1, and so on.

The idle thread is anchored by a global variable called init_task which is always exported by the kernel. This is why, when you write a kernel module that walks the task list, you can always rely on init_task as a known starting point. The idle thread does not show up in ps or top in normal usage, but it absolutely exists and runs at the lowest scheduling priority.

PID vs TGID: Understanding Linux Threading at the Kernel Level

This is a concept that trips up many people learning Linux kernel internals for the first time, so let’s be very clear about it.

When you call pthread_create() from a C program to spawn a new thread, the kernel actually creates a new task_struct for that thread. From the kernel’s point of view, this new thread is almost like a new process — it gets its own unique PID (Process ID). However, all threads that belong to the same process share the same TGID (Thread Group ID).

PID vs TGID in Linux Threading
Field What It Means Same within a process? User-space equivalent
pid Unique ID for this specific thread/task No — every thread has its own PID gettid()
tgid Thread Group ID — shared by all threads in a process Yes — all threads share the same TGID getpid()

This is a very important insight: when you call getpid() from a user-space thread, the C library returns the TGID, not the kernel-level PID. This is by design — POSIX defines that all threads in a process should share the same process ID from the user’s perspective.

A Multi-threaded Process in the Kernel Task List
Process “myapp” (TGID = 1200)
Main Thread
PID=1200
TGID=1200
| Worker Thread 1
PID=1201
TGID=1200
| Worker Thread 2
PID=1202
TGID=1200
All three entries exist as separate task_struct nodes in the kernel task list. They share TGID but have unique PIDs.

Kernel Threads vs User-Space Threads: How to Tell the Difference

The kernel itself runs many background threads to handle work like flushing dirty pages to disk, managing software RAID, processing network packets, and running workqueues. These are called kernel threads. You can see them in ps aux output — they are the ones with names wrapped in square brackets like [kworker/0:0], [kswapd0], and [migration/0].

From inside a kernel module, distinguishing a kernel thread from a user-space thread is straightforward: look at the mm pointer inside task_struct. This pointer refers to the memory map of the process — all its virtual address space regions. For a user-space process, this will always be a valid (non-NULL) pointer. For a kernel thread, it will be NULL because kernel threads live entirely in kernel address space and do not have a separate user-space memory map.

💡 Key Insight: If task->mm == NULL, the task is a kernel thread. If task->mm != NULL, it is a user-space process or thread. This single check is how the kernel itself makes this distinction in many places.

Displaying Thread Names from a Kernel Module

Every task_struct has a field called comm which holds the short name of the thread (up to 15 characters plus a null terminator). For user-space tasks this is typically the executable name. For kernel threads it is the kernel thread function name.

You might wonder: “Why not use the get_task_comm() helper function?” The reason is a real-world pitfall — get_task_comm() acquires a task lock internally, and if you are already holding that lock (which you would be inside the thread-walking loop), calling it again will cause a deadlock. The safe approach inside the iteration loop is to read t->comm directly after acquiring the lock yourself with task_lock().

⚠ Warning — Deadlock Trap: Never call get_task_comm(t, buf) while you are already holding the task lock inside a do_each_thread loop. Read t->comm directly instead.

Walking Every Thread in the Linux Kernel: do_each_thread and while_each_thread

Now we reach the practical core of this lecture in our free Linux kernel development course. How do you actually iterate over every living thread from inside a kernel module?

The Linux kernel provides a pair of macros specifically for this purpose: do_each_thread(g, t) and while_each_thread(g, t). These form a nested loop construct. You declare two task_struct pointers — by convention called g (for “group leader” / process) and t (for thread) — and these macros drive the iteration.

The outer pointer g walks through the process group leaders (one per process). The inner pointer t walks through all the individual threads belonging to that process. Together, they enumerate every single thread on the entire system.

How do_each_thread / while_each_thread Iterates
↻ Outer loop — pointer g (process group leader)
↻ Inner loop — pointer t (individual thread)
→ Read t→pid, t→tgid
→ Read t→stack (kernel stack addr)
→ Check g→mm for kernel/user
→ Read t→comm for thread name
→ Call get_nr_threads(g) for count
→ Print / store result

Setting Up the Required Headers

Before writing the module code, you need to include the right headers. The kernel’s scheduling and task-struct definitions have been reorganized across kernel versions. Here is how to handle it correctly for both older and modern kernels:

#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/sched.h>       /* task_struct, current */
#include <linux/version.h>     /* LINUX_VERSION_CODE */

#if LINUX_VERSION_CODE > KERNEL_VERSION(4, 10, 0)
#include <linux/sched/signal.h>  /* do_each_thread, while_each_thread */
#endif

The conditional include is necessary because the kernel moved several scheduling-related declarations to linux/sched/signal.h in version 4.11. On kernels 4.10 and older, linux/sched.h alone was enough. On anything newer (which is almost everything you will work with today, given that 6.x is the current LTS series), you need both.

Writing the Thread-Walking Function

Here is a clean, well-commented function that walks the entire task list and prints information about every thread it finds. This is original code for learning purposes — it demonstrates the concepts clearly:

#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/sched.h>
#include <linux/version.h>
#if LINUX_VERSION_CODE > KERNEL_VERSION(4, 10, 0)
#include <linux/sched/signal.h>
#endif

MODULE_LICENSE("GPL");
MODULE_AUTHOR("EmbeddedPathashala");
MODULE_DESCRIPTION("Walk the kernel task list and display all threads");

#define BUFMAX 256
#define TMPMAX 64

static void display_idle_thread(void)
{
    /*
     * init_task is the task_struct of the idle thread on CPU core 0.
     * It is always exported by the kernel and is a safe starting point.
     * The idle thread (swapper/0) runs when no other work is available.
     */
    pr_info("[swapper/0] PID=0 TGID=0  [idle/CPU0]\n");
}

static int walk_all_threads(void)
{
    struct task_struct *g, *t;
    char buf[BUFMAX], tmp[TMPMAX];
    int total = 0;
    unsigned int nr_thrds;

    display_idle_thread();

    /*
     * do_each_thread / while_each_thread: the canonical kernel macro
     * pair for iterating every live thread on the system.
     *
     * 'g' = group leader (the "main" task_struct representing the process)
     * 't' = individual thread within that process
     *
     * For a single-threaded process, g == t for that one iteration.
     */
    do_each_thread(g, t) {
        /*
         * task_lock() acquires a per-task spinlock, making it safe
         * to read the task's fields without them changing under us.
         */
        task_lock(t);

        memset(buf, 0, sizeof(buf));
        memset(tmp, 0, sizeof(tmp));

        /* Print TGID (process ID seen from user space) and PID (thread ID) */
        snprintf(buf, BUFMAX - 1, "TGID=%6d  PID=%6d", g->tgid, t->pid);

        /* Print the kernel virtual address of this task_struct */
        snprintf(tmp, TMPMAX - 1, "  task=%px", t);
        strncat(buf, tmp, TMPMAX);

        /* Print the start address of this thread's kernel-mode stack */
        snprintf(tmp, TMPMAX - 1, "  kstack=%px", t->stack);
        strncat(buf, tmp, TMPMAX);

        /*
         * Determine if this is a kernel thread or a user-space thread.
         * Kernel threads have g->mm == NULL (no user virtual address space).
         * User-space threads have a valid mm_struct pointer.
         *
         * We read t->comm directly instead of using get_task_comm()
         * because get_task_comm() tries to acquire the task lock again
         * -- causing a deadlock since we already hold it.
         */
        if (!g->mm) {
            /* Kernel thread: display name in square brackets (like ps does) */
            snprintf(tmp, TMPMAX - 1, "  [%-15s]", t->comm);
        } else {
            /* User-space thread */
            snprintf(tmp, TMPMAX - 1, "   %-15s ", t->comm);
        }
        strncat(buf, tmp, TMPMAX);

        /*
         * If this is the main thread of a multi-threaded user-space process
         * (TGID == PID and thread count > 1), print the thread count.
         * get_nr_threads() returns the number of threads in the group.
         */
        nr_thrds = get_nr_threads(g);
        if (g->mm && (g->tgid == t->pid) && (nr_thrds > 1)) {
            snprintf(tmp, TMPMAX - 1, "  [%u threads]", nr_thrds);
            strncat(buf, tmp, TMPMAX);
        }

        pr_info("%s\n", buf);
        total++;

        task_unlock(t);

    } while_each_thread(g, t);

    return total;
}

static int __init tasklist_walk_init(void)
{
    int count;
    pr_info("=== EmbeddedPathashala: Walking the Kernel Task List ===\n");
    pr_info("%-10s  %-10s  %-18s  %-18s  %-17s\n",
            "TGID", "PID", "task_struct addr", "kernel stack addr", "name");
    count = walk_all_threads();
    pr_info("Total threads counted: %d\n", count);
    return 0;
}

static void __exit tasklist_walk_exit(void)
{
    pr_info("Module unloaded.\n");
}

module_init(tasklist_walk_init);
module_exit(tasklist_walk_exit);
✓ Tip: After inserting the module with sudo insmod ./tasklist_walk.ko, run sudo dmesg | tail -100 to see the output. You will see every thread on your system listed with its TGID, PID, kernel-mode stack address, and name.

Writing the Makefile

obj-m += tasklist_walk.o

KDIR := /lib/modules/$(shell uname -r)/build

all:
	make -C $(KDIR) M=$(PWD) modules

clean:
	make -C $(KDIR) M=$(PWD) clean

Build it with make and load it with sudo insmod tasklist_walk.ko.

Understanding the Key Fields: task_struct, mm, comm, and stack

Let’s take a closer look at each field we used in the walking function, because understanding these fields is foundational for Linux kernel and device driver development.

Key task_struct Fields Used in Task List Iteration
Field Type What It Stores Kernel Thread Value
pid pid_t Unique kernel-level thread ID Has a PID like any task
tgid pid_t Thread group ID — same as PID for group leader tgid == pid typically
mm struct mm_struct * Pointer to the process user virtual address space NULL
comm char[TASK_COMM_LEN] Short name of the task (max 15 chars) Kernel thread function name
stack void * Base address of the kernel-mode stack for this thread Valid kernel address

How This Connects to the /proc Filesystem

Everything we have discussed connects directly to what you see in the /proc filesystem. Each process gets a numbered directory under /proc — for example, /proc/1234/ for a process with TGID 1234. Inside that directory, there is a task/ subdirectory. Each thread in that process gets its own subdirectory inside task/ named by its kernel PID.

So if process 1234 has three threads with PIDs 1234, 1235, and 1236, you will find:

/proc/1234/task/1234/
/proc/1234/task/1235/
/proc/1234/task/1236/

This is exactly how tools like ps, top, htop, and pidstat get their thread information — they read from /proc, which is the kernel exposing its own task list through a virtual filesystem. When you write a kernel module that walks the task list directly, you are doing the same thing but without the /proc indirection layer.

Modern Kernel Considerations (Linux 5.x / 6.x)

If you are using a modern LTS kernel (5.15, 6.1, or 6.6 at the time of writing), there are a few things to be aware of beyond the header changes covered above.

RCU Locking and the Task List

The proper way to iterate the task list without races is to hold the RCU read lock. In modern kernels the recommended pattern is:

rcu_read_lock();
do_each_thread(g, t) {
    /* safe reads here */
} while_each_thread(g, t);
rcu_read_unlock();

The older tasklist_lock reader-writer spinlock exists but is not exported to kernel modules — you cannot use it from a module. RCU is the recommended alternative for read-only traversal.

The %px Printk Format Specifier

To print actual kernel virtual addresses from a module, use %px in pr_info() / snprintf(). The %pK format hashes the address based on /proc/sys/kernel/kptr_restrict and may show zeros on a hardened system. Use %px during development but understand that printing real kernel addresses is a security concern on production systems.

task_lock() and task_unlock()

Each task_struct has its own per-task spinlock. Always bracket your field reads with task_lock(t) and task_unlock(t) when accessing fields that can be modified concurrently, especially in production-grade modules. For a learning exercise reading comm and pid without the lock is unlikely to cause problems, but it is not the correct approach for production code.

Common Mistakes and How to Avoid Them

Common Pitfalls in Kernel Task List Iteration
Mistake What Goes Wrong Correct Approach
Calling get_task_comm() inside the loop Deadlock — tries to acquire task lock already held Read t->comm directly after calling task_lock(t)
Missing the linux/sched/signal.h include Compile error on kernels > 4.10 Use LINUX_VERSION_CODE conditional include
Using %pK in a hardened system Addresses print as zeros Use %px for development; remove address printing in production
Not checking g->mm before dereferencing it Kernel oops / null pointer dereference Always check if (!g->mm) before accessing mm fields
Holding spinlock across a pr_info() call with long strings Latency spike; may miss real-time deadlines on RT kernels Build the string while holding the lock; print after releasing it

Summary & Key Takeaways

Key Takeaways from This Lecture

✓ Linux represents every thread as a task_struct
✓ PID = per-thread; TGID = per-process (matches user-space getpid)
✓ Kernel threads have mm == NULL
✓ do_each_thread / while_each_thread walks all live threads
✓ init_task anchors the CPU idle thread
✓ Never call get_task_comm() while holding the task lock
✓ Use RCU read lock for race-free traversal
✓ /proc mirrors the task list through a virtual filesystem

In this lecture from the free Linux kernel development course at EmbeddedPathashala, you learned how the kernel maintains and iterates its internal task list. You now understand why task_struct is the single most important data structure in the Linux kernel, how PID and TGID relate to user-space thread IDs, how to detect kernel threads programmatically, and how to safely traverse every live thread from inside a loadable kernel module.

This knowledge is directly applicable to writing device drivers, debugging multi-threaded application issues at the kernel level, and building kernel tools and diagnostics. Understanding the task list is also the prerequisite for the next major topic: Linux kernel memory management internals.

Frequently Asked Questions

Q1: What is the difference between a process and a thread in the Linux kernel?

At the kernel level there is almost no difference. Both are represented by a task_struct. The distinction is that threads within the same process share the same mm_struct (memory map), file descriptor table, and signal handlers, while processes do not. The TGID groups threads belonging to the same process.

Q2: Why does init_task always point to the idle thread and not to PID 1 (systemd)?

init_task is a statically defined task_struct that was created at compile time for the CPU 0 idle thread. PID 1 (systemd/init) is created dynamically at boot. The naming is historical — in very early Linux, PID 0 was called “init” before the concept of the idle thread was formalized separately.

Q3: Is it safe to walk the task list from a kernel module on a production system?

Yes, with proper locking. Use rcu_read_lock() and rcu_read_unlock() around the do_each_thread loop for read-only traversal. Per-task fields that can change should be read while holding task_lock(t). Do not sleep while holding any of these locks.

Q4: How does the kernel know how many threads a process has?

Each process group leader’s task_struct contains a signal structure that tracks the thread count. The get_nr_threads() macro reads this counter. This is why it is fast — no traversal is needed to count threads.

Q5: What happens to the task_struct when a thread exits?

When a thread calls exit(), the kernel sets its state to TASK_DEAD and schedules it for removal. The task_struct is not freed immediately — it remains as a “zombie” until the parent process calls wait() to collect the exit status. After that, the kernel frees the task_struct and removes it from the task list.

Q6: Why can’t I use tasklist_lock in a kernel module?

The tasklist_lock reader-writer spinlock is an internal kernel variable and is not exported via EXPORT_SYMBOL. Only symbols explicitly exported by the kernel are accessible to loadable modules. Use RCU locking instead — it is designed for exactly this use case and is more scalable on multi-core systems anyway.

Q7: How do I find the task_struct of a specific process by PID from inside a module?

Use the find_get_pid() and pid_task() helpers. First get the pid struct with struct pid *p = find_get_pid(target_pid);, then get the task with struct task_struct *t = pid_task(p, PIDTYPE_PID);. Remember to call put_pid(p) when you are done with the pid struct.

Q8: Is do_each_thread available on ARM and other architectures?

Yes. The task list and the macros for iterating it are entirely architecture-independent. The same module code you write on x86_64 will compile and run on ARM64, RISC-V, or any other architecture supported by the Linux kernel.

Q9: What is TASK_COMM_LEN and why is the thread name limited to 15 characters?

TASK_COMM_LEN is defined as 16 in the kernel header (15 usable characters + one null terminator). This limit dates back to early Unix conventions. You can change a thread’s name from user space with pthread_setname_np(), but it will always be silently truncated to 15 characters by the kernel.

Q10: How is this knowledge useful for writing device drivers?

Device drivers often need to know the context in which they are executing — which process triggered an ioctl, whether they are running in interrupt context or process context, and what the current thread’s capabilities are. Using the current macro (which points to the currently running thread’s task_struct) and understanding the fields covered in this lecture directly enables these checks in real driver code.

Continue Your Free Linux Kernel Development Journey

EmbeddedPathashala offers completely free, in-depth courses on Linux kernel programming, Linux device drivers, and embedded systems development. No paywalls. No subscriptions.

Browse All Kernel Lectures Free Device Drivers Course

1 Comment

Leave a Reply

Your email address will not be published. Required fields are marked *