What is linux kernel – linux system programming course

 

Chapter 01 — The Linux Kernel
What the kernel is, what it does, kernel mode vs user mode, and how processes interact with the kernel through system calls.
Navigation: Home | Chapter 02 →

Keywords

Linux KernelOperating SystemSystem CallsKernel ModeUser ModeProcess SchedulingMemory ManagementVirtual MemoryvmlinuzPreemptive Multitasking

What Is an Operating System?

The term operating system has two meanings. In the broad sense it means the complete software package — the kernel plus all tools, shells, GUIs, editors, and utilities. In the narrow technical sense it means only the central software that manages hardware resources: the CPU, RAM, and devices. This central component is the kernel.

On Linux the kernel binary is stored as a compressed executable called vmlinuz — the “z” means it is compressed. Older Unix systems named this file “unix” or “vmunix”; Linux mirrored the convention using its own name. The bootloader loads vmlinuz into memory when the computer starts.

Key Point: Although programs could theoretically run without a kernel, the kernel greatly simplifies everything by providing a software layer that manages limited hardware on behalf of all programs simultaneously.

The Seven Tasks of the Kernel

1. Process Scheduling

Linux is a preemptive multitasking OS. Multiple processes reside in memory simultaneously and each gets CPU time. The kernel’s process scheduler decides which process runs and for how long — no process controls this itself. A hardware timer fires periodically and gives the kernel the chance to switch to another process.

2. Memory Management

Physical RAM is shared among all processes. Linux uses virtual memory which provides two key benefits: (1) processes are fully isolated — one process cannot read or write another’s memory; (2) only part of a process needs to be in RAM at any time, the rest lives in the swap area on disk, allowing more processes to run than would fit in physical memory.

3. Provision of a File System

The kernel provides a file system on disk allowing programs to create, read, update, and delete files. It abstracts physical disk sectors so programs work with named files through a uniform interface.

4. Creation and Termination of Processes

The kernel loads a new program into memory and provides it the resources it needs — CPU time, memory, file access. This running instance is called a process. When it finishes, the kernel reclaims all its resources.

5. Access to Devices

Keyboards, mice, disks, and network cards are all managed by device drivers inside the kernel. The kernel provides programs with a standardised interface to devices and arbitrates access when multiple processes want the same device.

6. Networking

The kernel transmits and receives network packets on behalf of user processes. It handles routing packets to the correct destination and delivering incoming packets to the correct process.

7. System Call API

Processes request kernel services through system calls — controlled entry points into the kernel. Examples: open(), read(), write(), fork(), kill(). The system call API is the primary interface between user programs and the kernel.

Kernel Mode vs User Mode

Two CPU Privilege Levels

Kernel Mode (Ring 0): The CPU can execute any instruction and access any memory address including hardware registers and kernel data structures. Only the kernel runs here.

User Mode (Ring 3): The CPU can only access memory marked as user space. Attempting to access kernel memory triggers a hardware exception. All normal programs run here.

This separation is the foundation of system security and stability. A bug in a user program cannot corrupt the kernel or another process. The only legal entry from user mode into the kernel is through a system call — the kernel validates all arguments before acting on them.

What Happens During a System Call

Step-by-Step System Call Flow
  1. The C library places the syscall number in a CPU register (e.g., rax on x86-64) and arguments in other registers.
  2. It executes the syscall instruction.
  3. The CPU switches to kernel mode and jumps to the kernel’s entry point.
  4. The kernel saves user-space registers and dispatches to the correct handler.
  5. The handler validates arguments and performs the operation. If it must wait, it sleeps and lets another process run.
  6. The return value is placed in rax (negative errno on error).
  7. The CPU returns to user mode. The C library checks for errors and sets errno.

Two mode transitions occur per syscall — this makes system calls ~50–100 ns on modern hardware, much more expensive than a regular function call.

Process View vs Kernel View

From a Process’s Point of View
  • Many things happen asynchronously — it does not know when it will be preempted or when it will run again.
  • It operates in isolation — cannot directly see other processes, cannot know where files are on disk, cannot tell which memory pages are in RAM vs swapped out.
  • It cannot create processes, end itself, or talk to hardware without system calls.
From the Kernel’s Point of View
  • The kernel knows and controls everything.
  • It decides which process runs, for how long, and on which CPU.
  • It maintains data structures for all running processes.
  • It maps filenames to disk locations and virtual memory to physical RAM.
  • All inter-process communication and all hardware I/O go through the kernel.
Remember: When we say “a process creates another process” or “a process writes to a file,” the kernel mediates all such actions. The more accurate statement is: “a process requests the kernel to create another process.”

Interview Questions

Q1: What is the kernel and how does it differ from the full operating system?

Answer: The kernel is the central privileged software that manages hardware resources — CPU, RAM, and devices. The full operating system includes the kernel plus all accompanying software: shells, editors, GUIs, package managers, and utilities. On Linux, the kernel is strictly the “linux” component; distributions like Ubuntu bundle it with many additional programs to make a complete OS.

Q2: Why must the kernel prevent user programs from accessing kernel memory?

Answer: If user programs could read kernel memory, they could steal passwords, encryption keys, or other processes’ private data. If they could write to it, a buggy or malicious program could corrupt kernel data structures, crash the system, or escalate its privileges. Hardware-enforced protection in user mode ensures the kernel’s integrity and guarantees that all processes are isolated from each other.

Q3: What is preemptive multitasking and why is it important?

Answer: In preemptive multitasking the kernel’s scheduler can forcibly remove a process from the CPU using a hardware timer interrupt — the process does not need to cooperate. The alternative, cooperative multitasking, required each program to voluntarily yield. Cooperative systems were fragile: one misbehaving program could freeze the entire computer. Preemption guarantees all processes receive fair CPU time regardless of how any individual process behaves.

Q4: Name and briefly explain the seven tasks of the Linux kernel.

Answer: (1) Process scheduling — deciding which process runs and for how long. (2) Memory management — allocating virtual address spaces, enforcing isolation, managing swap. (3) File system — creating, reading, updating, deleting files on disk. (4) Creating and terminating processes — loading programs and reclaiming resources on exit. (5) Device access — standardised driver interfaces for hardware. (6) Networking — routing packets to and from processes. (7) System call API — the controlled interface through which user programs request kernel services.

Q5: What is virtual memory and what two advantages does it provide?

Answer: Virtual memory is a technique where each process gets its own private address space that is not directly the physical RAM layout. Advantage 1: isolation — processes cannot access each other’s memory or the kernel’s memory, preventing both bugs and malicious access. Advantage 2: only the active portions of a process need to be in RAM; inactive pages can be stored in the swap area on disk, allowing more processes to run simultaneously than would physically fit in RAM and improving overall CPU utilisation.

Continue to Chapter 02

Next: The Shell, Users, Groups — /etc/passwd explained, UIDs, GIDs, and the root superuser.

Chapter 02 → ← Home

Leave a Reply

Your email address will not be published. Required fields are marked *