PREV_LEC  |  NEXT_LEC

What You Will Learn

Prerequisites

Why Video Drivers Need Videobuf2

A camera sensor produces frames continuously. Somebody has to own the memory those frames land in, track which frames belong to the kernel and which belong to user space at any given moment, and hand that memory back and forth without ever letting the wrong side touch it. Before videobuf2 existed, every V4L2 driver wrote this logic by hand, and every driver wrote it slightly differently — different bugs, different race conditions, different DMA-mapping mistakes.

Videobuf2, usually shortened to vb2, is the kernel’s shared answer to that problem. It is a queue manager sitting between your capture hardware and the V4L2 core. Your driver tells vb2 how many buffers to allocate, how big each one is, and what to do at a handful of well-defined points in the buffer’s life. Vb2 takes care of everything else: the streaming ioctls, the queue/dequeue bookkeeping, and the memory-mapping mechanics for whichever memory model you choose.

Because vb2 is shared code, it also standardizes the vocabulary every V4L2 driver uses. Once you understand vb2 buffer management here, the same mental model applies whether you’re reading the UVC webcam driver, a CSI camera driver, or an HDMI capture bridge driver.

The Concept of a Buffer in Videobuf2

In V4L2 videobuf2 buffer management, a “buffer” is the unit of exchange between kernel and user space — typically one complete video frame, or one plane of a multi-planar frame. Every buffer vb2 manages is represented internally by struct vb2_buffer, defined in include/media/videobuf2-core.h. You don’t normally allocate this structure yourself; vb2 allocates and owns an array of them for you when your driver calls vb2_queue_init().

The fields that matter most to a driver author are:

• vb2_queue — back-pointer to the owning queue
• index — this buffer’s slot number in the queue
• type — capture, output, or their multi-planar variants
• memory — which VB2_MEMORY_* model this buffer uses
• num_planes — how many memory planes this buffer has
• planes[] — per-plane size, offset, and dma_addr information
• state — the current point in the buffer state machine
• timestamp — capture time, filled in when the buffer completes

The exact field list has grown over kernel releases — recent kernels add fields such as synced and prepared to support DMA-BUF cache synchronization — so always check your target kernel’s videobuf2-core.h rather than assuming a fixed layout. What stays stable across versions is the model: a buffer moves through a fixed set of states, and your driver only ever reacts to state transitions through callbacks, never by touching state directly.

VB2_MEMORY_* Models: Where Buffer Memory Lives

When user space calls VIDIOC_REQBUFS, it picks one of three memory models. Vb2 stores the choice in vb2_buffer.memory for the lifetime of the queue.

Model	User-space equivalent	Who allocates memory	Typical use case
VB2_MEMORY_MMAP	V4L2_MEMORY_MMAP	The kernel driver	Most webcams and embedded capture drivers
VB2_MEMORY_USERPTR	V4L2_MEMORY_USERPTR	User space (malloc/static buffer)	Zero-copy into an existing user buffer
VB2_MEMORY_DMABUF	V4L2_MEMORY_DMABUF	Another driver, exported as a DMA-BUF fd	Zero-copy camera-to-GPU or camera-to-encoder pipelines

MMAP remains the default choice for most drivers you’ll write in this free embedded Linux course: the kernel allocates the frame buffers, and user space maps them into its address space with mmap(). DMABUF has become the preferred model for modern SoC camera pipelines because it lets the capture driver, an image signal processor, and a video encoder share the same physical memory with zero copies. USERPTR is still present in the vb2 core, but many current drivers choose not to advertise support for it, since it complicates DMA-mapping guarantees; check a given driver’s capabilities with VIDIOC_QUERYCAP before relying on it.

The VB2_BUF_STATE_* Buffer State Machine

Every buffer vb2 manages sits in exactly one state at a time. Understanding this state machine is the real key to understanding V4L2 videobuf2 buffer management, because almost every vb2_ops callback exists purely to move a buffer from one state to the next.

• VB2_BUF_STATE_DEQUEUED — buffer belongs to user space
• VB2_BUF_STATE_PREPARING — vb2 core is running your buf_prepare()
• VB2_BUF_STATE_QUEUED — queued, waiting for the driver to start filling it
• VB2_BUF_STATE_ACTIVE — the driver’s DMA is actively writing into it
• VB2_BUF_STATE_DONE — capture finished successfully, ready to dequeue
• VB2_BUF_STATE_ERROR — capture failed, still ready to dequeue

Your driver moves a buffer out of ACTIVE by calling vb2_buffer_done(vb, state) with either VB2_BUF_STATE_DONE or VB2_BUF_STATE_ERROR. If streaming fails to start at all, the driver is responsible for returning every buffer it was handed back to VB2_BUF_STATE_QUEUED, which is effectively giving the buffers back to videobuf2 without ever having used them.

Implementing vb2_ops in an Original Driver

Let’s extend the ep_vcap driver from the previous lecture with a working vb2_queue and a minimal but complete set of vb2_ops callbacks. The five callbacks below are the ones almost every capture driver needs to implement.

#include <media/videobuf2-core.h>
#include <media/videobuf2-vmalloc.h>

struct ep_vcap_buffer {
    struct vb2_v4l2_buffer vb;
    struct list_head list;
};

struct ep_vcap_dev {
    struct video_device vdev;
    struct vb2_queue queue;
    struct mutex queue_lock;
    spinlock_t buf_lock;
    struct list_head buf_list;
    unsigned int frame_width;
    unsigned int frame_height;
};

static int ep_vcap_queue_setup(struct vb2_queue *vq,
                                unsigned int *nbuffers,
                                unsigned int *nplanes,
                                unsigned int sizes[],
                                struct device *alloc_devs[])
{
    struct ep_vcap_dev *dev = vb2_get_drv_priv(vq);
    unsigned int size = dev->frame_width * dev->frame_height * 2; /* YUYV */

    if (*nplanes)
        return sizes[0] < size ? -EINVAL : 0;

    *nplanes = 1;
    sizes[0] = size;

    if (*nbuffers < 3)
        *nbuffers = 3;

    dev_info(dev->vdev.dev, "ep_vcap: queue_setup nbuffers=%u size=%u\n",
             *nbuffers, size);
    return 0;
}

static int ep_vcap_buf_prepare(struct vb2_buffer *vb)
{
    struct ep_vcap_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
    unsigned long size = dev->frame_width * dev->frame_height * 2;

    if (vb2_plane_size(vb, 0) < size) {
        dev_err(dev->vdev.dev, "ep_vcap: buffer too small\n");
        return -EINVAL;
    }

    vb2_set_plane_payload(vb, 0, size);
    return 0;
}

static void ep_vcap_buf_queue(struct vb2_buffer *vb)
{
    struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
    struct ep_vcap_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
    struct ep_vcap_buffer *buf = container_of(vbuf, struct ep_vcap_buffer, vb);
    unsigned long flags;

    spin_lock_irqsave(&dev->buf_lock, flags);
    list_add_tail(&buf->list, &dev->buf_list);
    spin_unlock_irqrestore(&dev->buf_lock, flags);
}

static int ep_vcap_start_streaming(struct vb2_queue *vq, unsigned int count)
{
    struct ep_vcap_dev *dev = vb2_get_drv_priv(vq);

    dev_info(dev->vdev.dev, "ep_vcap: start_streaming, %u buffers queued\n", count);
    /* Program the capture hardware / start the DMA engine here. */
    return 0;
}

static void ep_vcap_stop_streaming(struct vb2_queue *vq)
{
    struct ep_vcap_dev *dev = vb2_get_drv_priv(vq);
    struct ep_vcap_buffer *buf, *tmp;
    unsigned long flags;

    /* Stop DMA/hardware here, then flush every outstanding buffer. */
    spin_lock_irqsave(&dev->buf_lock, flags);
    list_for_each_entry_safe(buf, tmp, &dev->buf_list, list) {
        list_del(&buf->list);
        vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
    }
    spin_unlock_irqrestore(&dev->buf_lock, flags);

    dev_info(dev->vdev.dev, "ep_vcap: stop_streaming\n");
}

static const struct vb2_ops ep_vcap_vb2_ops = {
    .queue_setup     = ep_vcap_queue_setup,
    .buf_prepare     = ep_vcap_buf_prepare,
    .buf_queue       = ep_vcap_buf_queue,
    .start_streaming = ep_vcap_start_streaming,
    .stop_streaming  = ep_vcap_stop_streaming,
};

Wiring this queue into the device during probe looks like this:

static int ep_vcap_init_queue(struct ep_vcap_dev *dev)
{
    struct vb2_queue *q = &dev->queue;

    q->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
    q->io_modes = VB2_MMAP | VB2_DMABUF;
    q->drv_priv = dev;
    q->buf_struct_size = sizeof(struct ep_vcap_buffer);
    q->ops = &ep_vcap_vb2_ops;
    q->mem_ops = &vb2_vmalloc_memops;
    q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
    q->lock = &dev->queue_lock;
    q->min_buffers_needed = 2;

    return vb2_queue_init(q);
}

Notice io_modes only advertises VB2_MMAP | VB2_DMABUF. USERPTR support is intentionally left out here — a good example of a driver choosing not to expose a memory model it doesn’t want to guarantee correctness for.

Selectively Disabling Ioctls with v4l2_disable_ioctl()

Your video_device exposes a single, large v4l2_ioctl_ops table shared by every driver of that type, but not every piece of capture hardware supports every ioctl in that table. Rather than allocating a whole new ops struct just to drop one callback, call v4l2_disable_ioctl() before video_register_device() to tell the core to ignore a specific ioctl on a specific device.

static int ep_vcap_probe(struct platform_device *pdev)
{
    struct ep_vcap_dev *dev;
    int ret;

    dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
    if (!dev)
        return -ENOMEM;

    /* ... v4l2_device_register(), video_device setup from earlier lectures ... */

    ret = ep_vcap_init_queue(dev);
    if (ret)
        return ret;

    dev->vdev.queue = &dev->queue;

    /* This hardware has no hardware frequency-seek capability. */
    v4l2_disable_ioctl(&dev->vdev, VIDIOC_S_HW_FREQ_SEEK);
    v4l2_disable_ioctl(&dev->vdev, VIDIOC_G_HW_FREQ_SEEK);

    return video_register_device(&dev->vdev, VFL_TYPE_VIDEO, -1);
}

Building and Testing ep_vcap on a Real Kernel

Build the module against your running kernel’s headers:

$ make -C /lib/modules/$(uname -r)/build M=$(pwd) modules
$ sudo insmod ep_vcap.ko

Expected dmesg output on load and during a capture session with a tool such as v4l2-ctl:

$ dmesg | tail -n 10
[ 102.441823] ep_vcap: probe: registered as /dev/video3
[ 102.441901] ep_vcap: queue_setup nbuffers=3 size=614400
[ 103.118762] ep_vcap: start_streaming, 3 buffers queued
[ 108.902314] ep_vcap: stop_streaming

Then drive it with the standard V4L2 command-line tool:

$ v4l2-ctl --device=/dev/video3 --stream-mmap --stream-count=5
<< VIDIOC_REQBUFS, VIDIOC_QBUF x3, VIDIOC_STREAMON >>
<< 5x VIDIOC_DQBUF / VIDIOC_QBUF pairs >>
<< VIDIOC_STREAMOFF >>

If queue_setup never appears in dmesg, double check that vdev.queue was assigned before video_register_device() and that vb2_queue_init() returned zero.

Videobuf2 vs the Legacy Videobuf (v1) API

Aspect	videobuf (v1)	videobuf2 (vb2)
Function prefix	videobuf_*	vb2_*
Source file	drivers/media/v4l2-core/videobuf-core.c	drivers/media/common/videobuf2/
Multi-planar support	Limited / bolted on	Native, first-class
DMA-BUF import/export	Not supported	Supported via VB2_MEMORY_DMABUF
Status in current kernels	Deprecated, drivers migrated away	The only supported buffer framework for new drivers

Any driver you write today for this free Linux device drivers course should target vb2 directly — the legacy API exists in tree only for a shrinking set of old drivers still being migrated.

Common Mistakes and Troubleshooting

Best Practices for V4L2 Videobuf2 Buffer Management

Always let stop_streaming() be the single place that clears your internal buffer list and calls vb2_buffer_done() on every outstanding buffer — this keeps error handling consistent whether streaming is stopped by the application or forced to stop after a hardware fault. Keep buf_queue() short; it typically runs with the queue lock held, so heavy work belongs in a workqueue or the interrupt handler that later calls vb2_buffer_done(). For performance, prefer VB2_MEMORY_DMABUF on SoC pipelines where the sensor, ISP, and encoder are separate drivers, since it avoids memory copies entirely. From a security standpoint, always validate buffer sizes in buf_prepare() rather than trusting values reported earlier by user space in queue_setup().

Summary and Key Takeaways

Videobuf2 is the shared buffer-management core underneath every modern V4L2 driver. A buffer is represented by struct vb2_buffer, tagged with a memory model from VB2_MEMORY_*, and moved through a predictable state machine using VB2_BUF_STATE_* values. Your driver’s only real job is implementing a handful of vb2_ops callbacks — queue_setup, buf_prepare, buf_queue, start_streaming, and stop_streaming — and letting the vb2 core handle the rest of the streaming ioctls. Combined with v4l2_disable_ioctl() for hardware that doesn’t support every optional feature, this gives you a clean, standards-compliant capture driver without reinventing buffer plumbing by hand. That’s the core of V4L2 videobuf2 buffer management, and it’s a pattern you’ll reuse in every capture driver you write going forward in this free Linux kernel development course.

Frequently Asked Questions

What is videobuf2 (vb2) in V4L2?

Videobuf2 is the Linux kernel’s shared framework for managing video buffer allocation, queuing, and state tracking between a V4L2 driver and user space, replacing the older videobuf (v1) API.

What is struct vb2_buffer used for?

It’s the internal data structure vb2 uses to represent a single video buffer, tracking its index, memory model, plane information, and current state in the buffer lifecycle.

What are the VB2_MEMORY_* models in videobuf2?

VB2_MEMORY_MMAP, VB2_MEMORY_USERPTR, and VB2_MEMORY_DMABUF, corresponding respectively to kernel-allocated mmap buffers, user-space allocated buffers, and DMA-BUF-based zero-copy buffers.

What does vb2_buffer_done() do?

It transitions a buffer out of the ACTIVE state into either VB2_BUF_STATE_DONE or VB2_BUF_STATE_ERROR, signaling that the driver has finished with the buffer and it’s ready to be dequeued.

Which vb2_ops callbacks does a minimal capture driver need?

At minimum: queue_setup, buf_prepare, buf_queue, start_streaming, and stop_streaming. More advanced drivers may also implement buf_init, buf_cleanup, and buf_finish.

What does v4l2_disable_ioctl() do?

It marks a specific V4L2 ioctl as unsupported on a given video_device, called before video_register_device(), so the core rejects that ioctl without requiring a separate ioctl_ops table.

Is VB2_MEMORY_USERPTR still used in modern drivers?

The vb2 core still supports it, but many current drivers choose not to advertise it because of the DMA-mapping guarantees it requires; DMABUF has become the preferred zero-copy option.

Where can I learn more about videobuf2 for free?

The official kernel documentation at docs.kernel.org and the videobuf2-core.h header in the Linux kernel source are the best authoritative references, alongside this free Linux kernel development course.

PREV_LEC  |  NEXT_LEC

<!– ================================================================