What You Will Learn

• The remaining fields of struct vb2_queue — io_modes, mem_ops, lock, min_buffers_needed, gfp_flags, and drv_priv
• The three videobuf2 memory allocators — vmalloc, contiguous DMA, and scatter-gather DMA — and when to choose each one
• The complete struct vb2_ops callback table used by modern V4L2 bridge drivers
• The exact call order of these callbacks during buffer allocation, queueing, and streaming
• Why wait_prepare/wait_finish have disappeared from current kernels, and what replaced them
• How to wire all of this into a real, working capture driver skeleton

Prerequisites

• Completion of the earlier lectures in this free linux kernel development course V4L2 series (v4l2_device, video_device, and the vb2 buffer/state basics)
• Comfort with kernel module basics, mutexes, and pointer-to-function fields in C structures
• A Linux kernel build environment (kernel 6.x headers) for the hands-on section

Quick Recap: What vb2_queue Represents

struct vb2_queue is the object through which a bridge driver introduces itself to the videobuf2 core. Every V4L2 capture device has exactly one queue per stream, and that queue is what videobuf2 uses to track buffer allocation, ownership, and state transitions. We already looked at type, ops, mem_ops, and buf_ops in the previous lecture. Let’s finish the picture.

The io_modes Bitmask

io_modes tells videobuf2 which buffer-access methods your device is willing to support. It’s set once, at queue-init time, as an OR of the following flags:

Flag	Meaning
VB2_MMAP	Buffers live in kernel memory and userspace accesses them via mmap(). This is the mode almost every capture driver supports first.
VB2_USERPTR	Userspace supplies its own pre-allocated buffer pointers. Requires scatter-gather DMA capable hardware in most real drivers.
VB2_DMABUF	Buffers are shared as DMA-BUF file descriptors, letting your capture device and, say, a GPU or encoder share memory with zero copy.
VB2_READ / VB2_WRITE	Legacy buffers driven through the classic read()/write() system calls, mainly used for simple or backward-compatible devices.

For a modern capture driver skeleton, VB2_MMAP alone is usually enough to get started; you add VB2_DMABUF once you need zero-copy sharing with another subsystem.

Choosing a Memory Allocator: mem_ops

mem_ops is where the driver tells videobuf2 how its buffers are physically backed. There are three ready-made allocators shipped with the kernel — you almost never need to write your own.

Allocator	Header	Best for
vb2_vmalloc_memops	media/videobuf2-vmalloc.h	Software-only or emulated capture paths; memory is virtually contiguous, not physically contiguous
vb2_dma_contig_memops	media/videobuf2-dma-contig.h	Hardware that can only DMA into one physically contiguous block per buffer
vb2_dma_sg_memops	media/videobuf2-dma-sg.h	Hardware with a real scatter-gather DMA engine, scattered physical pages are fine

Rule of thumb: start with vb2_vmalloc_memops while you bring up and test a new driver, then switch to vb2_dma_contig_memops or vb2_dma_sg_memops once you wire up real hardware DMA.

min_buffers_needed and the Streaming Gate

If min_buffers_needed is non-zero, videobuf2 will not call your start_streaming callback until userspace has queued at least that many buffers. This protects DMA-heavy hardware that genuinely needs, say, 2 or 3 buffers in flight before the engine can be kicked off safely — it lets the core do that bookkeeping instead of every driver re-implementing the same check.

struct vb2_ops — Your Driver’s Contract With videobuf2

While mem_ops handles memory allocation, ops (of type struct vb2_ops) is where your driver plugs in the logic that actually runs your hardware. This is the single most important structure in a V4L2 bridge driver. Here is the current callback set, verified against the mainline kernel source:

Callback	When it’s called
queue_setup	Before buffer allocation, to negotiate buffer count, plane count, and per-plane sizes
buf_init	Once, right after a buffer is allocated, for one-time per-buffer setup
buf_prepare	Every time userspace queues (or prepares) that buffer, before it’s handed to the driver
buf_finish	Right before a completed buffer is handed back to userspace
buf_cleanup	Once, right before a buffer is freed
prepare_streaming	Once, before streaming starts, for validation that doesn’t touch hardware yet
start_streaming	Once, to actually arm the DMA engine and enter streaming state
stop_streaming	Once, to halt the DMA engine and return all owned buffers
buf_queue	Every time a buffer is handed to the driver for hardware processing

The Callback Lifecycle, Visually

queue_setup — Negotiating Buffers and Planes

This is the very first callback videobuf2 invokes, in response to VIDIOC_REQBUFS or VIDIOC_CREATE_BUFS. Your driver is handed a requested buffer count and must fill in the actual per-plane sizes:

static int ep_vcap_queue_setup(struct vb2_queue *vq,
                                unsigned int *nbuffers,
                                unsigned int *nplanes,
                                unsigned int sizes[],
                                struct device *alloc_devs[])
{
    struct ep_vcap_dev *dev = vb2_get_drv_priv(vq);

    if (*nplanes)
        return sizes[0] frame_size ? -EINVAL : 0;

    *nplanes = 1;
    sizes[0] = dev->frame_size;

    if (*nbuffers < 2)
        *nbuffers = 2;

    return 0;
}

buf_prepare and buf_queue — Handing a Buffer to Hardware

static int ep_vcap_buf_prepare(struct vb2_buffer *vb)
{
    struct ep_vcap_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
    unsigned long size = dev->frame_size;

    if (vb2_plane_size(vb, 0) dev, "buffer too small (%lu vb2_queue);
    struct ep_vcap_buffer *buf =
        container_of(vb, struct ep_vcap_buffer, vb.vb2_buf);
    unsigned long flags;

    spin_lock_irqsave(&dev->qlock, flags);
    list_add_tail(&buf->list, &dev->buf_list);
    spin_unlock_irqrestore(&dev->qlock, flags);
}

start_streaming and stop_streaming — Arming and Disarming the Engine

In real hardware these two callbacks would program DMA registers. In our teaching driver we simulate the “DMA engine” with a kernel timer that periodically completes the oldest queued buffer — the same original ep_vcap pattern we’ve been building across this series:

static int ep_vcap_start_streaming(struct vb2_queue *vq, unsigned int count)
{
    struct ep_vcap_dev *dev = vb2_get_drv_priv(vq);

    dev->sequence = 0;
    mod_timer(&dev->fake_dma_timer, jiffies + msecs_to_jiffies(33));
    return 0;
}

static void ep_vcap_stop_streaming(struct vb2_queue *vq)
{
    struct ep_vcap_dev *dev = vb2_get_drv_priv(vq);
    struct ep_vcap_buffer *buf, *tmp;
    unsigned long flags;

    del_timer_sync(&dev->fake_dma_timer);

    spin_lock_irqsave(&dev->qlock, flags);
    list_for_each_entry_safe(buf, tmp, &dev->buf_list, list) {
        list_del(&buf->list);
        vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
    }
    spin_unlock_irqrestore(&dev->qlock, flags);
}

Whatever Happened to wait_prepare / wait_finish?

If you look up struct vb2_ops in an older book or an outdated blog post, you’ll almost certainly see two more callbacks: wait_prepare and wait_finish. They used to run around the point where videobuf2 blocks waiting for a buffer to arrive, releasing and re-taking the driver’s lock. In current mainline kernels these two callbacks have been removed entirely — videobuf2 now just uses vb2_queue->lock directly if it’s set, which is simpler and removes an entire class of driver bugs where the lock/unlock pairing was implemented incorrectly. If you’re reading old sample code that still sets .wait_prepare/.wait_finish, you can safely drop those lines on a modern kernel as long as your queue’s lock field is populated.

prepare_streaming / unprepare_streaming — Newer Additions

Recent kernels also added an optional pair, prepare_streaming and unprepare_streaming, that run before/after start_streaming/stop_streaming. They exist for drivers that need to reserve shared resources (like a media pipeline link or a shared clock) before actually touching hardware registers, keeping that concern separate from the DMA-arming logic in start_streaming itself. Most simple capture drivers can skip these two and rely on start_streaming/stop_streaming alone.

Wiring It All Together: the ep_vcap vb2_ops Table

static const struct vb2_ops ep_vcap_qops = {
    .queue_setup     = ep_vcap_queue_setup,
    .buf_prepare     = ep_vcap_buf_prepare,
    .buf_queue       = ep_vcap_buf_queue,
    .start_streaming = ep_vcap_start_streaming,
    .stop_streaming  = ep_vcap_stop_streaming,
};

static int ep_vcap_queue_init(struct ep_vcap_dev *dev)
{
    struct vb2_queue *q = &dev->queue;

    q->type            = V4L2_BUF_TYPE_VIDEO_CAPTURE;
    q->io_modes         = VB2_MMAP | VB2_DMABUF;
    q->drv_priv          = dev;
    q->buf_struct_size   = sizeof(struct ep_vcap_buffer);
    q->ops               = &ep_vcap_qops;
    q->mem_ops           = &vb2_vmalloc_memops;
    q->min_buffers_needed = 2;
    q->lock              = &dev->lock;
    q->dev               = dev->dev;

    return vb2_queue_init(q);
}

Build and Run

Build the module against your kernel headers and load it, then attach a userspace capture tool such as v4l2-ctl to exercise the streaming path:

$ make -C /lib/modules/$(uname -r)/build M=$PWD modules
$ sudo insmod ep_vcap.ko
$ v4l2-ctl --device=/dev/video0 --stream-mmap --stream-count=5

Expected kernel log:

$ dmesg | tail
[ 1201.552011] ep_vcap: queue_setup: nbuffers=4 plane size=614400
[ 1201.601233] ep_vcap: start_streaming: fake DMA timer armed
[ 1201.634509] ep_vcap: buf_queue: buffer 0 queued to driver
[ 1201.667788] ep_vcap: buf_queue: buffer 1 queued to driver
[ 1201.701122] ep_vcap: fake_dma_tick: buffer 0 marked VB2_BUF_STATE_DONE
[ 1201.734501] ep_vcap: fake_dma_tick: buffer 1 marked VB2_BUF_STATE_DONE
[ 1201.900112] ep_vcap: stop_streaming: fake DMA timer disarmed

Common Mistakes and Troubleshooting

• Forgetting to set q->lock: without it, videobuf2 can’t serialize DQBUF waits safely on modern kernels since wait_prepare/wait_finish no longer exist as a fallback.
• Returning wrong plane sizes from queue_setup: if *nplanes is already non-zero, you’re being asked to validate an existing request, not overwrite it — check the incoming sizes[] instead of blindly resetting it.
• Not returning all owned buffers in stop_streaming: every buffer previously handed to your driver via buf_queue must come back through vb2_buffer_done(), or DQBUF will hang forever.
• Mixing allocators: don’t set mem_ops to vb2_dma_contig_memops if your hardware actually needs scatter-gather DMA — you’ll get allocation failures under memory pressure.

Best Practices

• Start development with vb2_vmalloc_memops, switch to a DMA-backed allocator once real hardware is involved.
• Keep buf_prepare lightweight — it runs on every QBUF, not just once per buffer.
• Use min_buffers_needed instead of hand-rolled buffer-count checks inside start_streaming.
• Always pair every buf_queue hand-off with a matching vb2_buffer_done() call, even on error paths.

Summary

In this lecture of our free linux kernel development course we completed the picture of struct vb2_queue, compared the three videobuf2 memory allocators, and walked field-by-field through the current struct vb2_ops callback table — including catching a real, practical difference between the old book’s description and today’s mainline kernel (the removal of wait_prepare/wait_finish). With queue_setup, buf_prepare, buf_queue, start_streaming, and stop_streaming now wired into our ep_vcap driver, our capture skeleton can genuinely allocate, queue, and stream buffers end to end. Next lecture, we’ll extend this further with V4L2 controls and format negotiation.

Frequently Asked Questions

What is the difference between buf_prepare and buf_queue?

buf_prepare validates and prepares a buffer every time it’s queued from userspace, while buf_queue is what actually hands that buffer to the driver so hardware can start filling it.

Do I always need to implement queue_setup?

Yes — it’s the only mandatory callback in struct vb2_ops. Videobuf2 calls it to learn how many buffers and planes your device needs and how big each plane should be.

Why did wait_prepare and wait_finish get removed?

They existed to release/re-acquire the driver’s lock while videobuf2 waited for a buffer. Modern kernels do this automatically using vb2_queue->lock, so the callbacks became redundant and were dropped.

Which memory allocator should a beginner use?

vb2_vmalloc_memops is the easiest to start with since it needs no real DMA-capable hardware — perfect for learning and for this course’s demo drivers.

What happens if start_streaming fails?

Any buffers already handed to the driver via buf_queue must be returned with vb2_buffer_done() using VB2_BUF_STATE_QUEUED or VB2_BUF_STATE_ERROR before returning the error code.

Is min_buffers_needed mandatory?

No, it defaults to zero. Set it only if your hardware genuinely cannot begin DMA until a minimum number of buffers are already queued.

Can one driver support both VB2_MMAP and VB2_USERPTR?

Yes, as long as the chosen memory allocator supports both modes and the hardware can actually DMA into userspace-supplied memory for the USERPTR case.