Recap: Where Completion Fits in the DMA Flow

The DMA engine slave API follows five steps: allocate a channel, configure slave parameters, get a transaction descriptor, submit the transaction, and finally wait for callback notification. We covered the first three steps in earlier lectures. This lecture focuses entirely on the last two steps, which is exactly where the Linux DMA completion callback comes into play.

Submitting the DMA Transaction

Once you have a descriptor returned by a prep call such as dmaengine_prep_dma_memcpy() or dmaengine_prep_slave_sg(), it is not yet active. You must attach a callback and hand it to the DMA engine core with dmaengine_submit():

dma_cookie_t dmaengine_submit(struct dma_async_tx_descriptor *desc);

This function returns a dma_cookie_t, a small integer handle that can later be used with dma_async_is_tx_complete() to poll transaction status. Always check the cookie with dma_submit_error() before proceeding:

dma_cookie_t cookie = dmaengine_submit(desc);

if (dma_submit_error(cookie)) {
    dev_err(dev, "DMA submit failed\n");
    return -EIO;
}

Submitting a descriptor only places it on the pending queue — it does not start the hardware transfer yet. This two-step design (submit, then issue) lets a driver queue several descriptors together before triggering the DMA engine, which is efficient for scatter-gather workloads.

Issuing Pending DMA Requests

To actually start the hardware, call:

void dma_async_issue_pending(struct dma_chan *chan);

If the channel is idle, the first queued transaction begins immediately, and any additional queued descriptors run one after another as each finishes. Every time a transaction completes, the DMA engine core invokes the completion callback attached to that specific descriptor, which is the heart of the Linux DMA completion callback design.

Understanding the Linux DMA Completion Callback

The struct dma_async_tx_descriptor exposes two ways to be notified when a transfer finishes:

Field	Purpose	Status in Kernel 6.x
callback	Simple function pointer, called with callback_param, no status information	Still works, considered legacy
callback_result	Called with a struct dmaengine_result * describing success or failure and residue	Recommended for new drivers

A minimal callback using the legacy field looks like this:

static void ep_dma_callback(void *param)
{
    struct ep_dma_demo *demo = param;

    complete(&demo->dma_done);
}

desc->callback = ep_dma_callback;
desc->callback_param = demo;

The modern equivalent using callback_result also reports the transfer result:

static void ep_dma_callback_result(void *param,
                                    const struct dmaengine_result *result)
{
    struct ep_dma_demo *demo = param;

    if (result->result != DMA_TRANS_NOERROR)
        dev_warn(demo->dev, "EP DMA: transfer error %d\n", result->result);

    complete(&demo->dma_done);
}

desc->callback_result = ep_dma_callback_result;
desc->callback_param = demo;

Important: the callback always runs from the DMA engine’s tasklet context, never from hardware interrupt context. This means your callback can safely call complete(), but it should not sleep or perform blocking work.

struct completion: Waiting for the Callback

The kernel’s struct completion is a lightweight synchronization primitive purpose-built for exactly this “wait until some other context says I’m done” pattern, which makes it a natural partner for the DMA completion callback.

Declare and initialize a completion either statically or dynamically:

/* Static */
DECLARE_COMPLETION(my_comp);

/* Dynamic, inside a struct */
struct completion my_comp;
init_completion(&my_comp);

The waiting side blocks with:

void wait_for_completion(struct completion *comp);

/* Bounded wait, recommended for hardware operations */
long wait_for_completion_timeout(struct completion *comp,
                                  unsigned long timeout);

The signaling side wakes waiters with one of:

void complete(struct completion *comp);
void complete_all(struct completion *comp);

complete() wakes exactly one waiter, while complete_all() wakes every waiter blocked on that completion. Completions are implemented so that calling complete() before wait_for_completion() is perfectly safe — the waiter will simply return immediately instead of blocking, which avoids the classic race condition of “did the event happen before I started waiting?”

If you plan to reuse the same completion object for a second transfer, reset it first with reinit_completion() rather than calling init_completion() again.

Original Example: DMA Completion Demo Driver

The driver below is an original teaching example (not copied from any book) that ties together descriptor preparation, submission, issuing, and a Linux DMA completion callback backed by struct completion. It performs a simple memory-to-memory DMA copy and blocks until the hardware confirms completion or a timeout occurs.

// ep_dma_completion_demo.c
#include <linux/module.h>
#include <linux/platform_device.h>
#include <linux/dmaengine.h>
#include <linux/dma-mapping.h>
#include <linux/completion.h>
#include <linux/slab.h>

#define EP_BUF_SIZE 4096

struct ep_dma_demo {
    struct device *dev;
    struct dma_chan *chan;
    struct completion dma_done;
    void *src_buf;
    void *dst_buf;
    dma_addr_t src_dma;
    dma_addr_t dst_dma;
};

static void ep_dma_callback(void *param)
{
    struct ep_dma_demo *demo = param;

    dev_info(demo->dev, "EP DMA: transfer finished, waking waiter\n");
    complete(&demo->dma_done);
}

static int ep_dma_run_transfer(struct ep_dma_demo *demo)
{
    struct dma_async_tx_descriptor *desc;
    dma_cookie_t cookie;

    desc = dmaengine_prep_dma_memcpy(demo->chan, demo->dst_dma,
                                      demo->src_dma, EP_BUF_SIZE,
                                      DMA_CTRL_ACK);
    if (!desc) {
        dev_err(demo->dev, "EP DMA: failed to get descriptor\n");
        return -ENOMEM;
    }

    desc->callback = ep_dma_callback;
    desc->callback_param = demo;

    reinit_completion(&demo->dma_done);

    cookie = dmaengine_submit(desc);
    if (dma_submit_error(cookie)) {
        dev_err(demo->dev, "EP DMA: submit failed\n");
        return -EIO;
    }

    dma_async_issue_pending(demo->chan);

    if (!wait_for_completion_timeout(&demo->dma_done,
                                      msecs_to_jiffies(2000))) {
        dev_err(demo->dev, "EP DMA: transfer timed out\n");
        return -ETIMEDOUT;
    }

    dev_info(demo->dev, "EP DMA: transfer completed successfully\n");
    return 0;
}

static int ep_dma_probe(struct platform_device *pdev)
{
    struct ep_dma_demo *demo;
    dma_cap_mask_t mask;
    int ret;

    demo = devm_kzalloc(&pdev->dev, sizeof(*demo), GFP_KERNEL);
    if (!demo)
        return -ENOMEM;

    demo->dev = &pdev->dev;
    init_completion(&demo->dma_done);

    dma_cap_zero(mask);
    dma_cap_set(DMA_MEMCPY, mask);

    demo->chan = dma_request_channel(mask, NULL, NULL);
    if (!demo->chan) {
        dev_err(&pdev->dev, "EP DMA: no memcpy-capable channel found\n");
        return -ENODEV;
    }

    demo->src_buf = dma_alloc_coherent(&pdev->dev, EP_BUF_SIZE,
                                        &demo->src_dma, GFP_KERNEL);
    demo->dst_buf = dma_alloc_coherent(&pdev->dev, EP_BUF_SIZE,
                                        &demo->dst_dma, GFP_KERNEL);
    if (!demo->src_buf || !demo->dst_buf) {
        dma_release_channel(demo->chan);
        return -ENOMEM;
    }

    memset(demo->src_buf, 0xAB, EP_BUF_SIZE);
    platform_set_drvdata(pdev, demo);

    ret = ep_dma_run_transfer(demo);
    if (ret)
        dev_warn(&pdev->dev, "EP DMA: demo transfer reported %d\n", ret);

    return 0;
}

static void ep_dma_remove(struct platform_device *pdev)
{
    struct ep_dma_demo *demo = platform_get_drvdata(pdev);

    dma_free_coherent(&pdev->dev, EP_BUF_SIZE, demo->src_buf, demo->src_dma);
    dma_free_coherent(&pdev->dev, EP_BUF_SIZE, demo->dst_buf, demo->dst_dma);
    dma_release_channel(demo->chan);
}

static struct platform_driver ep_dma_driver = {
    .probe  = ep_dma_probe,
    .remove = ep_dma_remove,
    .driver = {
        .name = "ep_dma_completion_demo",
    },
};
module_platform_driver(ep_dma_driver);

MODULE_LICENSE("GPL");
MODULE_AUTHOR("EmbeddedPathashala");
MODULE_DESCRIPTION("EP DMA completion callback demo driver");

Building and Running the Demo

Create a matching Makefile:

obj-m += ep_dma_completion_demo.o

all:
	make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules

clean:
	make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean

Build and load the module, then register a matching platform device (or bind it via your board’s device tree if a DMA controller is present):

$ make
$ sudo insmod ep_dma_completion_demo.ko
$ dmesg | tail

Expected dmesg output on a system with an available memcpy-capable DMA channel:

[  102.481203] platform ep_dma_completion_demo.0: EP DMA: transfer finished, waking waiter
[  102.481219] platform ep_dma_completion_demo.0: EP DMA: transfer completed successfully

If no DMA channel is available on your system, dmaengine will report EP DMA: no memcpy-capable channel found, which is expected on virtualized or minimal setups without a DMA controller driver loaded.

Common Mistakes with DMA Completion Callbacks

• Calling wait_for_completion() before dma_async_issue_pending()
• Sleeping or blocking inside the callback function
• Forgetting reinit_completion() before reusing a completion object
• Not checking dma_submit_error() after dmaengine_submit()
• Using wait_for_completion() without a timeout in production drivers
• Freeing DMA buffers before the callback has actually fired

Best Practices for DMA Completion Handling

• Prefer callback_result over callback for new drivers
• Always use a bounded wait_for_completion_timeout() for hardware-dependent waits
• Keep callback bodies short: just complete() and minimal bookkeeping
• Release the DMA channel with dma_release_channel() in your remove/error paths
• Log transfer errors from dmaengine_result for easier field debugging

Performance and Security Considerations

From a performance angle, batching several descriptors and calling dma_async_issue_pending() once amortizes the cost of kicking the hardware, which matters on high-throughput peripherals. From a security angle, never let userspace directly control the DMA source or destination physical addresses passed into descriptor preparation; always validate buffer ownership and size in kernel space before submitting a transaction, since a stray DMA write can corrupt unrelated kernel memory.

Summary and Key Takeaways

• dmaengine_submit() queues a descriptor and returns a dma_cookie_t
• dma_async_issue_pending() actually starts the hardware transfer
• The Linux DMA completion callback fires from tasklet context, never IRQ context
• callback_result is the modern, status-aware replacement for callback
• struct completion pairs naturally with DMA callbacks for safe blocking waits

This wraps up the DMA engine slave API series in this free linux kernel development course. You now understand the complete lifecycle: channel allocation, slave configuration, descriptor preparation, submission, issuing, and the Linux DMA completion callback that ties it all together with struct completion. This same pattern — submit, issue, wait on completion — is reused throughout real-world Linux storage, networking, and crypto DMA drivers, so mastering it here pays off across the kernel.

Frequently Asked Questions

What is a Linux DMA completion callback?

It is a function pointer attached to a DMA descriptor that the DMA engine core invokes once the hardware transfer finishes, letting the driver know the operation is done.

What is the difference between callback and callback_result?

callback only passes a private parameter with no status, while callback_result also passes a struct dmaengine_result describing success, error, or residue, and is the recommended choice on modern kernels.

Why do I need dma_async_issue_pending() after dmaengine_submit()?

dmaengine_submit() only queues the descriptor. dma_async_issue_pending() is the call that actually tells the DMA hardware to start processing the queued transactions.

Can I call complete() before wait_for_completion()?

Yes. struct completion is designed so that if complete() runs first, the later wait_for_completion() call returns immediately instead of blocking forever.

Is it safe to sleep inside a DMA callback function?

No. DMA callbacks run from tasklet context, which cannot sleep. Keep the callback minimal and use complete() to hand off work to a process context waiter.

What does wait_for_completion_timeout() return?

It returns 0 if the timeout elapsed before completion, or the remaining jiffies (a positive value) if the completion happened in time.

Why use complete_all() instead of complete()?

complete() wakes only one waiter, while complete_all() wakes every task blocked on that completion object, which is useful when multiple threads wait on the same DMA event.

Do I need reinit_completion() for every new DMA transfer?

Yes, if you reuse the same completion object across multiple transfers, call reinit_completion() before each new submission to reset its internal state.