PREV_LEC | NEXT_LEC

What You Will Learn

• How queue_setup() negotiates buffer count and size with videobuf2-core
• The exact job of buf_init, buf_prepare, buf_finish, and buf_cleanup
• Why almost every real driver wraps vb2_v4l2_buffer in its own buffer struct with a list_head
• How buf_queue() hands a buffer to the DMA engine and what happens on interrupt completion
• What start_streaming() and stop_streaming() must guarantee, including buffer state on error
• How vb2_queue_init() validates the queue before anything can run
• A complete, buildable ep_vcap driver that simulates DMA with a kernel timer

Prerequisites

• Lectures 1-6 of this V4L2 series (video_device, file_operations, vb2_buffer, vb2_queue fields)
• Comfort with spinlocks, kernel timers, and container_of()
• A Linux kernel 6.x build environment (headers installed, cross or native)

Recap: The vb2_ops Callback Table

Every vb2_queue points to a struct vb2_ops. Two callbacks are mandatory — queue_setup and buf_queue — everything else is optional but almost always implemented by a real capture driver. Here is the full picture we are about to build:

Callback	Triggered by	Typical job
queue_setup	VIDIOC_REQBUFS / VIDIOC_CREATE_BUFS	Report required buffer count, plane count, and plane sizes
buf_init	Once, right after a buffer’s memory is allocated	Init driver-private buffer fields, pin pages, set up IOMMU mapping
buf_prepare	VIDIOC_QBUF	Validate buffer size, set the plane payload
buf_queue	VIDIOC_QBUF (after buf_prepare succeeds)	Push the buffer onto the driver’s own DMA queue
start_streaming	VIDIOC_STREAMON	Verify minimum buffers, arm the DMA engine, enable sub-devices
buf_finish	VIDIOC_DQBUF	Cache sync, bounce-buffer copy-back
stop_streaming	VIDIOC_STREAMOFF	Halt DMA, return every outstanding buffer in ERROR state
buf_cleanup	Before a buffer’s memory is freed	Undo whatever buf_init set up (unmap, unpin)

queue_setup(): Negotiating Buffers With videobuf2-core

queue_setup() is the very first callback the framework ever calls, and it can be called more than once during the same VIDIOC_REQBUFS negotiation. Its job is simple: tell videobuf2-core how many buffers you need, how many planes each buffer has, and how large each plane must be. The kernel signature on current 6.x trees is unchanged from what we saw in the vb2_queue lecture:

static int queue_setup(struct vb2_queue *vq,
                        unsigned int *nbuffers,
                        unsigned int *nplanes,
                        unsigned int sizes[],
                        struct device *alloc_devs[]);

Two details trip up almost every beginner. First, *nbuffers is a request, not a fact — you are free to raise it if your hardware needs a deeper pipeline. Second, when *nplanes already comes in non-zero (this happens on the second negotiation pass, when the application has re-called REQBUFS), you must validate the existing sizes[] instead of overwriting them. Here is ep_vcap‘s implementation:

#define EP_VCAP_MIN_BUFFERS 3

static int ep_vcap_queue_setup(struct vb2_queue *vq,
                                unsigned int *nbuffers,
                                unsigned int *nplanes,
                                unsigned int sizes[],
                                struct device *alloc_devs[])
{
	struct ep_vcap_dev *dev = vb2_get_drv_priv(vq);
	unsigned int size = dev->fmt.sizeimage;

	if (*nplanes)
		return sizes[0] num_buffers + *nbuffers num_buffers;

	*nplanes = 1;
	sizes[0] = size;

	dev_dbg(dev->dev, "queue_setup: nbuffers=%u size=%u\n",
		*nbuffers, size);
	return 0;
}

Three buffers is a sensible floor for most DMA engines because the hardware is usually filling one buffer while user space is reading a second and a third is waiting in the queue, avoiding starvation.

buf_init and buf_cleanup: Buffer Lifetime Bookends

buf_init() runs exactly once per buffer, right after videobuf2-core has allocated its backing memory. This is where you initialize any driver-private fields attached to the buffer — never in buf_prepare, which runs on every single QBUF. buf_cleanup() is the mirror image, called right before that memory is released.

static int ep_vcap_buf_init(struct vb2_buffer *vb)
{
	struct ep_vcap_buf *buf = to_ep_vcap_buf(vb);

	INIT_LIST_HEAD(&buf->list);
	return 0;
}

static void ep_vcap_buf_cleanup(struct vb2_buffer *vb)
{
	struct ep_vcap_buf *buf = to_ep_vcap_buf(vb);

	WARN_ON(!list_empty(&buf->list));
}

Why Drivers Wrap vb2_v4l2_buffer

The generic vb2_v4l2_buffer structure has no field for tracking a buffer inside your own DMA queue. Almost every real bridge driver — from webcam bridges to SDR front ends — solves this by embedding it inside a private struct that adds a list_head:

struct ep_vcap_buf {
	struct vb2_v4l2_buffer vb;
	struct list_head list;
};

static inline struct ep_vcap_buf *to_ep_vcap_buf(struct vb2_buffer *vb)
{
	return container_of(vb, struct ep_vcap_buf, vb.vb2_buf);
}

You tell videobuf2-core about the larger struct size in your queue setup:

q->buf_struct_size = sizeof(struct ep_vcap_buf);

buf_prepare and buf_finish

buf_prepare() runs on every VIDIOC_QBUF call. It is your last chance to reject a buffer before it becomes eligible for DMA — check the plane size and set the actual payload:

static int ep_vcap_buf_prepare(struct vb2_buffer *vb)
{
	struct ep_vcap_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
	unsigned long size = dev->fmt.sizeimage;

	if (vb2_plane_size(vb, 0) dev, "buffer too small (%lu < %lu)\n",
			vb2_plane_size(vb, 0), size);
		return -EINVAL;
	}

	vb2_set_plane_payload(vb, 0, size);
	return 0;
}

static void ep_vcap_buf_finish(struct vb2_buffer *vb)
{
	/* Non-coherent DMA drivers sync caches back to the CPU
	 * domain here before the buffer reaches user space. */
}

buf_queue(): Handing the Buffer to Your DMA Engine

By the time buf_queue() runs, videobuf2-core has already marked the buffer VB2_BUF_STATE_ACTIVE. Your only job here is to push it onto your own driver-maintained list so an interrupt handler (or, in our simulated case, a timer) can find it later. This callback must never block — it can be called back to back for several buffers before a single frame has even started capturing.

static void ep_vcap_buf_queue(struct vb2_buffer *vb)
{
	struct ep_vcap_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
	struct ep_vcap_buf *buf = to_ep_vcap_buf(vb);
	unsigned long flags;

	spin_lock_irqsave(&dev->qlock, flags);
	list_add_tail(&buf->list, &dev->buf_list);
	spin_unlock_irqrestore(&dev->qlock, flags);
}

On real hardware, the completion side of this pattern lives in your interrupt handler: pop the head of buf_list, stamp it, and call vb2_buffer_done(). Since ep_vcap has no real hardware, we simulate completion with a kernel timer that fires roughly every video frame period:

static void ep_vcap_fake_dma_timer(struct timer_list *t)
{
	struct ep_vcap_dev *dev = from_timer(dev, t, fake_dma_timer);
	struct ep_vcap_buf *buf = NULL;
	unsigned long flags;

	spin_lock_irqsave(&dev->qlock, flags);
	if (!list_empty(&dev->buf_list)) {
		buf = list_first_entry(&dev->buf_list,
					struct ep_vcap_buf, list);
		list_del(&buf->list);
	}
	spin_unlock_irqrestore(&dev->qlock, flags);

	if (buf) {
		buf->vb.vb2_buf.timestamp = ktime_get_ns();
		buf->vb.sequence = dev->sequence++;
		buf->vb.field = V4L2_FIELD_NONE;
		vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
	}

	if (dev->streaming)
		mod_timer(&dev->fake_dma_timer,
			  jiffies + msecs_to_jiffies(33));
}

start_streaming() and stop_streaming()

start_streaming() receives the number of buffers currently queued. If that count is below what your hardware needs to run safely, return -ENOBUFS — videobuf2-core will call you again the next time a buffer arrives instead of failing outright.

static int ep_vcap_start_streaming(struct vb2_queue *vq, unsigned int count)
{
	struct ep_vcap_dev *dev = vb2_get_drv_priv(vq);

	if (count sequence = 0;
	dev->streaming = true;
	mod_timer(&dev->fake_dma_timer, jiffies + msecs_to_jiffies(33));

	dev_info(dev->dev, "start_streaming: fake DMA armed\n");
	return 0;
}

stop_streaming() has one strict rule that new driver authors miss constantly: every buffer still sitting in your private list must be handed back to videobuf2-core, in the VB2_BUF_STATE_ERROR state, or VIDIOC_STREAMOFF will hang waiting for buffers that will never arrive.

static void ep_vcap_stop_streaming(struct vb2_queue *vq)
{
	struct ep_vcap_dev *dev = vb2_get_drv_priv(vq);
	struct ep_vcap_buf *buf, *tmp;
	unsigned long flags;

	dev->streaming = false;
	del_timer_sync(&dev->fake_dma_timer);

	spin_lock_irqsave(&dev->qlock, flags);
	list_for_each_entry_safe(buf, tmp, &dev->buf_list, list) {
		list_del(&buf->list);
		vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
	}
	spin_unlock_irqrestore(&dev->qlock, flags);
}

Wiring the vb2_ops Table and Calling vb2_queue_init()

With every callback written, assembling the table and initializing the queue is straightforward. Remember from the previous lecture that vb2_queue_init() internally runs sanity checks through vb2_core_queue_init() — it refuses to proceed unless ops, mem_ops, type, and io_modes are already set.

static const struct vb2_ops ep_vcap_qops = {
	.queue_setup     = ep_vcap_queue_setup,
	.buf_init        = ep_vcap_buf_init,
	.buf_prepare     = ep_vcap_buf_prepare,
	.buf_finish      = ep_vcap_buf_finish,
	.buf_cleanup     = ep_vcap_buf_cleanup,
	.buf_queue       = ep_vcap_buf_queue,
	.start_streaming = ep_vcap_start_streaming,
	.stop_streaming  = ep_vcap_stop_streaming,
	.wait_prepare    = vb2_ops_wait_prepare,
	.wait_finish     = vb2_ops_wait_finish,
};

static int ep_vcap_queue_init(struct ep_vcap_dev *dev)
{
	struct vb2_queue *q = &dev->queue;
	int ret;

	q->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
	q->io_modes = VB2_MMAP | VB2_DMABUF;
	q->drv_priv = dev;
	q->buf_struct_size = sizeof(struct ep_vcap_buf);
	q->ops = &ep_vcap_qops;
	q->mem_ops = &vb2_vmalloc_memops;
	q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
	q->min_buffers_needed = EP_VCAP_MIN_BUFFERS;
	q->lock = &dev->lock;

	ret = vb2_queue_init(q);
	if (ret)
		return ret;

	spin_lock_init(&dev->qlock);
	INIT_LIST_HEAD(&dev->buf_list);
	timer_setup(&dev->fake_dma_timer, ep_vcap_fake_dma_timer, 0);

	return 0;
}

Build, Load, and Watch the Callback Order

Build the module against your kernel tree exactly like earlier lectures in this free linux kernel development course:

make -C /lib/modules/$(uname -r)/build M=$(pwd) modules
sudo insmod ep_vcap.ko

Stream a handful of frames with v4l2-ctl and watch dmesg in another terminal:

v4l2-ctl --device=/dev/video0 --stream-mmap --stream-count=5

[  12.001] ep_vcap: queue_setup: nbuffers=3 size=614400
[  12.004] ep_vcap: buf_init: buffer 0 initialised
[  12.004] ep_vcap: buf_init: buffer 1 initialised
[  12.004] ep_vcap: buf_init: buffer 2 initialised
[  12.010] ep_vcap: start_streaming: fake DMA armed
[  12.010] ep_vcap: buf_queue: buffer 0 queued
[  12.010] ep_vcap: buf_queue: buffer 1 queued
[  12.010] ep_vcap: buf_queue: buffer 2 queued
[  12.043] ep_vcap: fake_dma_timer: buffer 0 done, seq=0
[  12.076] ep_vcap: fake_dma_timer: buffer 1 done, seq=1
[  12.110] ep_vcap: buf_finish: buffer 0
[  12.110] ep_vcap: buf_queue: buffer 0 re-queued
[  12.143] ep_vcap: fake_dma_timer: buffer 2 done, seq=2
[  12.500] ep_vcap: stop_streaming: draining buffer list
[  12.500] ep_vcap: buf_cleanup: buffer 0 released
[  12.500] ep_vcap: buf_cleanup: buffer 1 released
[  12.500] ep_vcap: buf_cleanup: buffer 2 released

Notice the pattern that matters most for a free linux device drivers course student: buf_init runs once per buffer at allocation time, while buf_queue and buf_finish cycle repeatedly for every QBUF/DQBUF pair while streaming is active.

Common Mistakes and Troubleshooting

Mistake	Symptom	Fix
Blocking inside buf_queue	QBUF ioctl stalls, frame drops	Only touch the list under a spinlock, never sleep here
Not returning -ENOBUFS from start_streaming	DMA starts with too few buffers, underruns immediately	Check the min_buffers_needed condition explicitly
Leaving buffers in the list after stop_streaming	VIDIOC_STREAMOFF hangs indefinitely	Walk the list and call vb2_buffer_done() with VB2_BUF_STATE_ERROR
Setting driver fields in buf_prepare instead of buf_init	Stale data reused across QBUF calls	One-time init belongs in buf_init only

Best Practices

• Keep buf_queue() and interrupt-context buffer handling under the same spinlock, using the _irqsave variants
• Always set timestamp and sequence before calling vb2_buffer_done() — user-space frame-drop detection depends on a monotonically increasing sequence number
• Prefer vb2_dma_contig_memops over vb2_vmalloc_memops whenever your hardware needs physically contiguous DMA buffers
• Never assume stop_streaming() is only called after a clean start_streaming() — it can be called if start_streaming() itself fails partway through

Real-World Use Cases

This exact callback pattern powers USB webcam bridge drivers, SoC camera-interface drivers, HDMI capture cards, and software-defined radio front ends. Any device that streams data through /dev/videoX under V4L2 goes through this same queue_setup → buf_prepare → buf_queue → start_streaming lifecycle, which is why it’s one of the highest-value topics in any serious free embedded systems course.

Summary and Key Takeaways

• queue_setup() negotiates buffer count, plane count, and plane size before any memory is allocated
• buf_init/buf_cleanup bookend a buffer’s entire lifetime; buf_prepare/buf_finish run on every QBUF/DQBUF
• buf_queue() must be fast and non-blocking — it only hands the buffer to your own driver-maintained list
• start_streaming() can legitimately refuse with -ENOBUFS; stop_streaming() must never leave buffers stranded
• vb2_queue_init() enforces that ops, mem_ops, type, and io_modes are set before it will succeed

Frequently Asked Questions

Why does queue_setup() need to handle being called twice?

VIDIOC_CREATE_BUFS can call queue_setup() to validate a second allocation request against buffers that already exist, which is why you must check the incoming *nplanes and existing sizes[] rather than blindly overwriting them.

What happens if buf_prepare() returns an error?

VIDIOC_QBUF fails and the buffer is never handed to buf_queue() or the DMA engine — it stays available for the application to fix and resubmit.

Can buf_queue() sleep or allocate memory?

No. It can be called from a context where blocking is unsafe, and back-to-back for multiple buffers, so it should only add the buffer to a list under a spinlock.

Why do I need my own buffer struct instead of using vb2_v4l2_buffer directly?

vb2_v4l2_buffer has no list_head or driver-private fields, so almost every real driver wraps it in a larger struct and reaches it back with container_of().

What is min_buffers_needed used for?

It tells videobuf2-core how many buffers must be queued before start_streaming() is even attempted, saving you from manually checking this in every driver.

Is start_streaming() allowed to fail?

Yes — returning a negative error code (commonly -ENOBUFS) is valid and expected when the hardware isn’t ready to run yet.

What buffer state should stop_streaming() use for buffers still in flight?

VB2_BUF_STATE_ERROR — this tells user space the frame is invalid without deadlocking the streamoff call.

Does vb2_queue_init() start streaming?

No, it only validates and initializes the queue structure (lists, locks, dma_dir). Streaming only begins later, when VIDIOC_STREAMON triggers start_streaming().

Where can I practice this hands-on for free?

This lecture is part of EmbeddedPathashala’s free linux kernel development course, which also covers character devices, platform drivers, DMA, and the full V4L2 stack end to end.

PREV_LEC | NEXT_LEC