1. The Big Picture — Two Worlds in One Machine

Every modern operating system splits its world into two zones. Linux is no different. At the hardware level, the CPU itself enforces this split using privilege rings (on x86 hardware, these are Ring 0 through Ring 3). Linux uses only two of them in practice:

Think of it like a building with two security zones. The public area (user space) is where visitors roam freely. The restricted area (kernel space) is where the core systems live — power controls, security, plumbing. Visitors cannot walk in there directly; they must request a service from the front desk (system call interface).

2. User Space — Where Your Programs Live

User space is the environment in which all ordinary programs run — your text editor, web browser, database server, shell scripts, and the applications you write as a developer. Each process running in user space gets its own virtual address space, isolated from every other process.

This isolation is crucial. If your application crashes, it does not take down the entire system. The kernel can simply clean it up and move on. This is one of the biggest lessons from early computing — running everything in one space led to catastrophic failures when any one piece went wrong.

User space is also home to threads, shared libraries, and daemons (background services like systemd, sshd, cron). All of these run at the lower privilege level — they cannot directly touch hardware or kernel data structures.

3. Library APIs — The Building Blocks for User Space

When writing applications, developers rely heavily on Application Programming Interfaces (APIs). Rather than writing everything from scratch, you call pre-built, well-tested functions packaged in libraries.

On Linux, the most important library is glibc — the GNU C Standard Library. Every user-mode application on a Linux system is automatically linked against glibc. You do not choose this; it happens automatically during compilation. glibc provides the C standard library functions that are used in virtually every program.

Library APIs vs System Call APIs — What is the Difference?

This is a very common point of confusion. Here is a clear way to think about it:

Notice the man page section numbers. When you see printf(3), the (3) tells you it is a library function. When you see open(2), the (2) tells you it is a system call. This notation is used consistently across all Linux man pages and is worth memorising.

4. System Calls — The Only Legal Gateway to the Kernel

Here is the key question: if user space and kernel space are completely separate — different address spaces, different privilege levels — how does a user application ever do anything useful, like reading a file, creating a process, or sending data over a network?

The answer is system calls. A system call is a special, controlled mechanism that allows a user-space process to request a service from the kernel. It is the only legitimate way for a user process to enter kernel space. Everything else — direct memory access to kernel structures, directly calling kernel functions — is illegal and will cause a crash or a security violation.

Why is This Design Important?

This controlled gateway design is what makes Linux both stable and secure. Because every request to the kernel must go through a defined, validated interface:

• The kernel can validate parameters before acting on them, preventing privilege escalation attacks.
• A misbehaving user process cannot corrupt kernel data structures.
• The kernel can apply per-process limits, capabilities, and security policies at the system call boundary.
• Crashes in user space are contained — only that process dies, not the entire system.

How Many System Calls Does Linux Have?

In modern Linux (kernel 6.x), there are around 450+ system calls on x86-64 architecture. The number varies by architecture. You can look at the master syscall table yourself:

# On an x86-64 Linux system, count the syscalls:
grep -c "^[0-9]" /usr/share/syscalls/x86_64.tbl 2>/dev/null || \
  grep -c "common" /usr/src/linux/arch/x86/entry/syscalls/syscall_64.tbl

# Or simply look at the table:
cat /usr/src/linux/arch/x86/entry/syscalls/syscall_64.tbl | head -40

Some commonly used system calls you will encounter frequently as a kernel programmer:

5. Putting It All Together — The Full Architecture

Now that we understand each piece individually, let us look at how everything connects in a real running Linux system.

Every arrow crossing the user-kernel boundary is a system call. There is no other way. This is why kernel programming is both powerful and demanding — the kernel code you write runs in the privileged zone. A bug there can bring down the entire machine, not just one process.

6. Hands-On — Watching System Calls in Action

The best way to understand system calls is to watch them happen. Linux provides a fantastic tool called strace that intercepts and logs every system call a process makes.

# Watch every system call made by the 'ls' command
strace ls

# Count how many times each system call is used (-c flag)
strace -c ls /tmp

# Trace only specific system calls (e.g., only open and read)
strace -e trace=open,read,write ls

# Trace a running process by its PID
strace -p 1234

Try running strace -c ls on your system. You will see output like this (abbreviated):

% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 28.45    0.000312          10        30           mmap
 22.10    0.000242           8        29           read
 15.30    0.000168          14        12           openat
  8.90    0.000098           7        14           fstat
  6.20    0.000068           9         7     7      access
  ...
------ ----------- ----------- --------- --------- ----------------
100.00    0.001096                   142     7      total

Even a simple ls command makes over 100 system calls! Most of them are for loading shared libraries, reading directory entries, and writing output to the terminal. This gives you a real appreciation for how much work the kernel does on behalf of even simple programs.

📋 Interview Questions & Answers

These are real questions asked in Linux kernel engineer interviews at embedded companies, semiconductor firms, and OS development roles.