← Previous Lecture  |  Next Lecture →

What You Will Learn

• How the unlocked_ioctl file operation works on modern kernels
• How to safely validate an incoming ioctl command in a driver
• How to move data between user space and kernel space safely with copy_to_user and copy_from_user
• A complete, working example character driver ioctl handler
• Security and performance considerations specific to ioctl handlers

Prerequisites

• Part 1 of this course (ioctl fundamentals and command encoding)
• Basic understanding of the file_operations structure in a character driver
• Familiarity with kernel modules and building against your kernel headers

Registering the ioctl Handler in file_operations

On every actively maintained Linux kernel today, a character driver registers its ioctl callback using the unlocked_ioctl field of struct file_operations. The older four-argument ioctl field you may see in legacy code was removed from the kernel more than a decade ago, so there is no need to write compatibility code for it on any kernel you’ll actually build against today.

static const struct file_operations epdrv_fops = {
    .owner          = THIS_MODULE,
    .open           = epdrv_open,
    .release        = epdrv_release,
    .read           = epdrv_read,
    .write          = epdrv_write,
    .unlocked_ioctl = epdrv_ioctl,
};

Complete Example: A Modern ioctl Handler

Below is a complete, original example ioctl handler for a simple character driver, matching the shared header we defined in Part 1 of this free Linux kernel programming course:

#include <linux/fs.h>
#include <linux/uaccess.h>
#include "epdrv_ioctl.h"

static int epdrv_mode = 0;

static long epdrv_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
{
    int tmp;

    /* Step 1: validate the magic number */
    if (_IOC_TYPE(cmd) != EPDRV_MAGIC)
        return -ENOTTY;

    /* Step 2: validate the command number is within our known range */
    if (_IOC_NR(cmd) > EPDRV_IOC_MAXNR)
        return -ENOTTY;

    switch (cmd) {
    case EPDRV_IOC_RESET:
        epdrv_mode = 0;
        break;

    case EPDRV_IOC_GET_MODE:
        if (copy_to_user((int __user *)arg, &epdrv_mode, sizeof(int)))
            return -EFAULT;
        break;

    case EPDRV_IOC_SET_MODE:
        if (copy_from_user(&tmp, (int __user *)arg, sizeof(int)))
            return -EFAULT;
        epdrv_mode = tmp;
        break;

    default:
        return -ENOTTY;
    }

    return 0;
}

A few important details worth calling out for beginners:

• Never dereference the arg pointer directly. It is a user-space address. Always go through copy_to_user() / copy_from_user(), which validate the address and safely handle page faults.
• Return -ENOTTY for unknown commands. This is the historical, POSIX-consistent errno the kernel expects an unrecognized ioctl command to return.
• Return -EFAULT when a copy fails. This tells user space the pointer it passed was invalid.

Why Locking Still Matters

Removing the old kernel-wide lock did not remove the need for locking altogether — it just moved that responsibility to each driver author. If your ioctl handler reads or modifies shared driver state (like our epdrv_mode variable above), and your device can be opened from multiple processes or threads at once, you must protect that state yourself, typically with a mutex. We cover kernel synchronization primitives like mutexes and spinlocks in detail later in this free Linux kernel development course.

static DEFINE_MUTEX(epdrv_lock);

static long epdrv_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
{
    int tmp, ret = 0;

    if (_IOC_TYPE(cmd) != EPDRV_MAGIC || _IOC_NR(cmd) > EPDRV_IOC_MAXNR)
        return -ENOTTY;

    mutex_lock(&epdrv_lock);

    switch (cmd) {
    case EPDRV_IOC_RESET:
        epdrv_mode = 0;
        break;
    case EPDRV_IOC_GET_MODE:
        if (copy_to_user((int __user *)arg, &epdrv_mode, sizeof(int)))
            ret = -EFAULT;
        break;
    case EPDRV_IOC_SET_MODE:
        if (copy_from_user(&tmp, (int __user *)arg, sizeof(int)))
            ret = -EFAULT;
        else
            epdrv_mode = tmp;
        break;
    default:
        ret = -ENOTTY;
    }

    mutex_unlock(&epdrv_lock);
    return ret;
}

Security Considerations

• Validate every input value coming from user space, not just the pointer — a malicious or buggy application can pass any integer value for “mode.”
• Never trust the size implied by the command blindly for variable-length data; use the size encoded in the command via _IOC_SIZE(cmd) for defensive checks.
• Restrict privileged operations. If a command can affect hardware in a dangerous way (for example, resetting a shared peripheral), check capabilities with something like capable(CAP_SYS_ADMIN) before allowing it.

Performance Considerations

ioctl() calls are synchronous system calls, so a handler that blocks for a long time (waiting on hardware, for example) will block the calling thread. For commands that may take time, consider whether the operation truly needs to be synchronous, or whether it should kick off work and let the application poll or wait on a separate mechanism. Keep the actual data copied via copy_to_user/copy_from_user as small as possible — for large data transfers, other mechanisms like mmap() are usually a better fit than ioctl().

ioctl() vs Other Kernel Interfaces

Common Mistakes and Troubleshooting

Best Practices Summary

• Always validate the magic number and command range before the switch statement
• Always use copy_to_user/copy_from_user, never touch the raw pointer
• Protect shared state with a mutex when the device can be accessed concurrently
• Return standard errno values (-ENOTTY, -EFAULT, -EINVAL) so user-space error handling stays predictable
• Keep one shared header between the driver and every application that talks to it

Key Takeaways

• Modern kernels use the three-argument unlocked_ioctl callback exclusively
• Validation happens in two steps: magic number, then command range
• copy_to_user/copy_from_user are mandatory for any data crossing the user/kernel boundary
• Locking is now the driver author’s responsibility, not the kernel’s

Conclusion

You’ve now seen both halves of the ioctl() picture in this free Linux kernel programming course: the user-space call in Part 1, and the kernel-space driver implementation here in Part 2, using the modern, currently-supported unlocked_ioctl approach with proper validation, safe data copying, and locking. This pattern — magic number check, command range check, switch on command, copy safely, protect shared state — is the same pattern you’ll find in real, production Linux drivers today.

FAQ

Q1. What replaced the old four-argument ioctl callback?
The three-argument unlocked_ioctl callback, which every actively maintained driver uses today.

Q2. Why can’t I just cast and dereference the arg pointer directly in the kernel?
Because it’s a user-space address; dereferencing it directly can crash the kernel or be exploited. Always use copy_to_user()/copy_from_user().

Q3. What does -ENOTTY mean in this context?
It’s the standard error returned for an ioctl command the driver doesn’t recognize.

Q4. Do I need locking in every ioctl handler?
Only if the handler reads or modifies state that can be accessed concurrently, for example from multiple open file descriptors or threads.

Q5. Is ioctl() deprecated in modern Linux kernel development?
No, it’s still widely used, though sysfs, netlink, or mmap may be more appropriate for certain use cases as shown in the comparison table above.

Q6. How do I check what capability an ioctl command should require?
Consider the real-world impact of the command; anything that can affect other users or hardware state broadly should typically require CAP_SYS_ADMIN or a similarly narrow capability, checked with capable().

Q7. What is the difference between copy_to_user and copy_from_user?
copy_to_user() moves data from the kernel to a user-space buffer; copy_from_user() moves data from a user-space buffer into the kernel.

← Previous Lecture  |  Next Lecture →