Why Unsupported File Operations Need Deliberate Handling

Every device driver author eventually faces the same question: what should happen when a user-space process calls a system call your driver never implemented? It is tempting to assume that leaving a function pointer unset in file_operations is harmless, since your driver “just doesn’t do that.” In reality, the kernel’s Virtual Filesystem Switch layer still has to respond to that system call somehow, and the response it gives is not always the clearest possible signal to the calling application.

Getting this right matters for real users of your driver. A poorly chosen or accidental default error code can send application developers down the wrong debugging path entirely, making them believe their arguments were invalid when the real issue is that your driver simply does not support that operation at all.

What Actually Happens When an Operation Is Unimplemented

When a file_operations pointer is left NULL and a matching system call is issued, the VFS layer detects the missing pointer and returns a negative error value to the calling C library function. The C library negates this value and stores it in the process’s errno variable, which is what tools like perror() and strerror() report back to the user.

The trouble is that this default error code is not always intuitive. For example, if your driver leaves the read pointer unset, the default response can lead an application developer to believe their function arguments were invalid, when the true reason is simply that reading is not supported at all on this device.

The Correct Way to Reject Unsupported Operations

The clearest, most professional approach is to implement the callback anyway and explicitly return -ENOSYS. This causes the calling process to see the message “Function not implemented,” which tells the application developer immediately and unambiguously what is going on, rather than leaving them to guess between several possible causes.

static ssize_t ep_write_unsupported(struct file *filp,
                                     const char __user *ubuf,
                                     size_t count, loff_t *off)
{
    /* This device is intentionally read-only. */
    return -ENOSYS;
}

static const struct file_operations ep_fops = {
    .owner = THIS_MODULE,
    .read  = ep_read,
    .write = ep_write_unsupported,
};

This pattern is especially valuable for drivers that are read-only or write-only by design. Rather than leaving the unused pointer NULL and letting user space receive a generic, sometimes misleading, default response, you take direct control of the message reported back to the calling application.

When to Simply Leave a Pointer NULL

Not every unimplemented operation needs an explicit stub function. For operations where the kernel’s default fallback error code is already clear and appropriate for your device, such as an unimplemented ioctl returning a well-understood “not a typewriter” style error, leaving the pointer NULL is perfectly acceptable and is standard practice across the kernel tree.

Handling the Seek Operation on Non-Seekable Devices

Seeking deserves special attention because its failure mode is unusually easy to get wrong. Most hardware devices, sensors, and simple control interfaces have no meaningful concept of a byte offset, so seeking should be explicitly rejected rather than left to an ambiguous default. If you leave the llseek pointer completely unset, some code paths can return a value that looks like success to a careless caller, even though no meaningful seek took place.

static int ep_open(struct inode *inode, struct file *filp)
{
    return nonseekable_open(inode, filp);
}

static const struct file_operations ep_fops = {
    .owner  = THIS_MODULE,
    .open   = ep_open,
    .read   = ep_read,
    .llseek = no_llseek,
};

Calling the non-seekable open helper in your open() callback, combined with explicitly assigning the no-seek llseek handler, guarantees that any later seek attempt on your device fails predictably with -ESPIPE, meaning “illegal seek.” This is a clear, well-understood error that immediately tells the calling application that your device simply does not support positioning.

Comparing Common Unsupported Operation Outcomes

Operation Left Unimplemented	Typical Default Behaviour	Recommended Fix
read	Misleading “invalid argument” style error	Implement and return -ENOSYS explicitly
write	Same misleading default	Implement and return -ENOSYS explicitly
llseek	Can appear to succeed with a meaningless offset	Use no_llseek plus nonseekable_open()
unlocked_ioctl	Clear “inappropriate device” style error	Leaving it NULL is generally acceptable

Real-World Use Cases

This pattern shows up constantly in production kernel code. Read-only sensor drivers explicitly reject write attempts, write-only logging or control interfaces explicitly reject read attempts, and almost every simple control device explicitly rejects seeking. Getting this right is a small detail that has an outsized impact on how professional and predictable your driver feels to the developers who build on top of it.

Common Mistakes and Troubleshooting

• Assuming NULL is always the safest choice. It often produces a confusing default error rather than a clear one.
• Forgetting nonseekable_open() in the open() callback. Setting llseek alone without this call can leave gaps in seek rejection behaviour.
• Returning the wrong error code from a stub function. Use -ENOSYS specifically for “not implemented,” not generic codes like -EINVAL.
• Not documenting unsupported file operations in code comments. Future maintainers, including yourself, will thank you for a one-line comment explaining why an operation is rejected.

Best Practices

• Explicitly implement and return -ENOSYS for operations you intentionally do not support when the default error would be misleading.
• Always pair a non-seekable device with both no_llseek and nonseekable_open().
• Leave truly self-explanatory defaults, like unimplemented ioctl, as NULL rather than adding unnecessary boilerplate.
• Comment every intentionally unsupported operation so the reasoning is clear to future readers of your code.

Performance Considerations

Explicitly returning an error code from a stub function has effectively zero performance cost compared to leaving a pointer NULL; both paths are extremely fast. The real performance consideration in driver design is always in the operations you do implement, not in the handful of stub functions you write for clarity.

Security Considerations

Explicit rejection of unsupported file operations is also a security best practice. A driver that clearly and predictably rejects operations it does not support is far easier to audit and reason about than one that relies on implicit kernel fallback behaviour, which can vary subtly across kernel versions and configurations.

Summary and Key Takeaways

• Leaving a file_operations pointer NULL is not the same as “no behaviour”; it still produces a real error code back to user space.
• Explicitly returning -ENOSYS gives application developers the clearest possible signal for genuinely unsupported file operations.
• Non-seekable devices should always pair no_llseek with nonseekable_open() in their open() callback.
• Predictable, well-documented error handling is both a usability and a security best practice.

Conclusion

Handling unsupported file operations correctly is a small but important skill that reflects the difference between a driver written by a beginner and one written with real production discipline. By deliberately choosing clear error codes and explicitly marking devices as non-seekable where appropriate, you make your driver’s behaviour predictable and easy to debug for everyone who uses it. This wraps up the foundational file_operations series in this free Linux kernel development course; from here, you are ready to explore more advanced driver topics such as ioctl design and blocking I/O.

Frequently Asked Questions