Simple Camera Pipeline Configuration Example

A minimal, IPU-free camera pipeline — sensor, mux, CSI capture — configured end to end with media-ctl on an original demo SoC.

What You Will Learn

• Why some SoCs use a bare mux + CSI chain instead of a full image processing unit
• How a video mux entity selects between multiple physical camera inputs
• How a CSI capture interface hands frame data to a video device node
• Configuring this simple pipeline completely with media-ctl commands

Prerequisites

• Comfortable with media-ctl link and format syntax from the previous lecture
• Basic understanding of MIPI CSI-2 as a physical camera interface

The Entities in This Simple Camera Pipeline

Our original demo SoC exposes three media entities for its camera subsystem:

• ep_mipi_rx — the MIPI CSI-2 receiver. One sink pad accepts pixel data from an external camera sensor; one source pad forwards it to virtual channel 0.
• ep_vmux — a video multiplexer with two sink pads (one for a parallel camera input, one for the MIPI receiver’s output) and a single source pad that always routes to the CSI block. Only one sink can be active at a time.
• ep_csi — the CSI capture interface. It has an internal FIFO and DMA engine, one sink pad fed by ep_vmux, and a source pad that routes directly to a capture video device node.

Simple Mux + CSI Camera Pipeline

[MIPI Camera Sensor] → [ep_mipi_rx] → [ep_vmux] → [ep_csi] → /dev/video0

Because ep_vmux also has an unused second sink pad reserved for a parallel camera input, the topology always contains one enabled link and one disabled link on its input side — the disabled one simply represents a hardware possibility that is not currently selected.

Configuring the Pipeline With media-ctl

Assume a sensor identified on the media bus as ep_camsensor 2-0036 is wired to the MIPI receiver. First, link every stage of the chain and activate the MIPI input on the mux:

# Setup links
$ media-ctl --reset
$ media-ctl -l "'ep_camsensor 2-0036':0 -> 'ep_mipi_rx':0[1]"
$ media-ctl -l "'ep_mipi_rx':1 -> 'ep_vmux':1[1]"
$ media-ctl -l "'ep_vmux':2 -> 'ep_csi':0[1]"
$ media-ctl -l "'ep_csi':1 -> 'ep_csi capture':0[1]"

The same four commands merged into one, using -r to reset first:

$ media-ctl -r -l \
   '"ep_camsensor 2-0036":0->"ep_mipi_rx":0[1], \
    "ep_mipi_rx":1->"ep_vmux":1[1], \
    "ep_vmux":2->"ep_csi":0[1], \
    "ep_csi":1->"ep_csi capture":0[1]'

Next, push a matching pad format down the whole chain. This demo sensor outputs a raw Bayer format at 800×600:

# Configure pad formats
$ media-ctl -V "'ep_camsensor 2-0036':0 [fmt:SBGGR10_1X10/800x600]"
$ media-ctl -V "'ep_mipi_rx':0 [fmt:SBGGR10_1X10/800x600]"
$ media-ctl -V "'ep_vmux':1 [fmt:SBGGR10_1X10/800x600]"
$ media-ctl -V "'ep_vmux':2 [fmt:SBGGR10_1X10/800x600]"
$ media-ctl -V "'ep_csi':0 [fmt:SBGGR10_1X10/800x600]"

Merged into a single -f call for a bring-up script:

$ media-ctl -f \
   '"ep_camsensor 2-0036":0 [SBGGR10 800x600], \
    "ep_mipi_rx":0 [SBGGR10 800x600], \
    "ep_vmux":1 [SBGGR10 800x600], \
    "ep_vmux":2 [SBGGR10 800x600], \
    "ep_csi":0 [SBGGR10 800x600]'

Why the Mux Needs Two Format Calls

Unlike a simple pass-through sub-device, ep_vmux has two relevant pads for this data path: the sink pad receiving from the MIPI receiver (pad 1) and the source pad feeding the CSI block (pad 2). Both must carry the same format, because the mux itself performs no scaling or conversion — it is purely a routing switch. Any mismatch between pad 1 and pad 2 on the mux usually means the driver rejected one of the two --set-v4l2 calls, and is the single most common misconfiguration on this style of pipeline.

Starting the Capture

With links and formats configured, streaming is driven through the capture video node, not through media-ctl itself:

$ v4l2-ctl --device /dev/video0 \
    --set-fmt-video=width=800,height=600,pixelformat=BG10 \
    --stream-mmap --stream-count=1 --stream-to=frame.raw

Expected terminal output on success:

<<< 1 >>>

A single < or > per frame confirms buffers were queued and dequeued correctly; frame.raw now contains one raw Bayer frame you can inspect with any RAW image viewer configured for the SBGGR10 pattern.

Common Mistakes and Troubleshooting

• Leaving the parallel sink active: if both mux sink links were left enabled from a previous session, the mux may still be routing the wrong source — always reset before reconfiguring.
• Mismatched mux pad formats: pad 1 and pad 2 on ep_vmux must always match; check both with --get-v4l2 if capture fails.
• Wrong virtual channel: MIPI receivers with multiple virtual channels route each channel to a different source pad — make sure the link targets channel 0 unless your sensor is configured otherwise.
• Format code typos: pixel format tokens like SBGGR10_1X10 are case- and spelling-sensitive; use media-ctl --known-mbus-fmts to list valid tokens.

Best Practices

• Always configure the mux’s sink and source pads together, never independently
• Verify with --print-topology before attempting to stream
• Keep bring-up scripts idempotent by always starting with --reset
• Match the pixel format between the physical sensor’s actual output and every downstream pad exactly

Real-World Use Case

This exact mux + CSI shape appears on many low-power SoCs aimed at battery-powered or cost-sensitive embedded products — doorbell cameras, industrial vision modules, and single-board computers without a dedicated ISP. Because there is no image processing block in the chain, all color conversion and scaling must happen either in the sensor itself or in a userspace/GPU pipeline downstream of capture, which is an important design constraint to keep in mind when picking a sensor for this class of hardware.

Summary and Key Takeaways

• Not every SoC needs a full image processing unit; a mux plus CSI capture interface is a common minimal pipeline
• A video mux entity has multiple possible sinks but routes only one at a time
• Both pads of a routing-only entity must always carry matching formats
• Streaming itself happens on the capture video node, after links and formats are configured through media-ctl

Conclusion

This simple camera pipeline configuration example strips the media controller framework down to its bare essentials — one routing decision and one capture interface — which makes it an ideal reference pipeline while you are still learning. The next lecture in this free linux kernel development course shows how these exact entities are declared in the device tree in the first place, so you can trace a pipeline all the way from the .dts source file to the running system.

Frequently Asked Questions

Why doesn’t this SoC have an image processing unit?

Many low-cost or power-constrained SoCs omit a dedicated ISP block entirely, relying on either sensor-side processing or a downstream software/GPU pipeline instead.

Can the video mux select the parallel input instead of MIPI?

Yes — activate the link from the parallel entity to ep_vmux pad 0 instead of the MIPI link to pad 1, keeping only one sink link enabled at a time.

Why must both mux pads carry the same format?

A pure routing entity performs no color conversion or scaling, so whatever format enters on the sink pad must exit unchanged on the source pad.

What does SBGGR10_1X10 mean?

It describes a raw Bayer pattern starting with a blue-green row, 10 bits per sample, packed one sample per media bus clock cycle.

How do I list all valid mbus format tokens?

Run media-ctl –known-mbus-fmts to print every format code the tool recognizes, along with its numeric value.

Do I need v4l2-ctl in addition to media-ctl?

Yes — media-ctl configures routing and pad formats, but actually starting and stopping the video stream happens through the capture node using v4l2-ctl or an application using the V4L2 streaming ioctls.