← Previous Lecture
Next Lecture →

Prerequisites

Complete Part 1 of this cgroups series first, or at least know what the unified hierarchy and controllers are. You need any Linux system with a pure cgroup v2 layout, which means practically any current distribution: verify with stat -fc %T /sys/fs/cgroup/ and confirm the answer is cgroup2fs. Root access is required, because we write into kernel interface files. A virtual machine is ideal for this kind of experiment. Students following our free embedded systems course can run the same lab on a Raspberry Pi or any embedded Linux board with a recent kernel.

How the cgroups v2 CPU Controller Works

The cgroups v2 CPU controller regulates how CPU cycles are distributed among groups of processes. It offers two independent models, and you can use either or both:

• Weight model (cpu.weight): proportional sharing. Groups receive CPU time in proportion to their weights, but only when there is competition. An idle system lets any group use everything. Think of it as “who wins when everyone wants CPU at once”.
• Bandwidth model (cpu.max): a hard ceiling. The group may consume at most a fixed amount of CPU time within each accounting period, competition or not. This is what Docker’s --cpus flag and systemd’s CPUQuota= configure.

These knobs apply to normal (fair-class) scheduling. Since kernel 6.6 the fair scheduler is EEVDF, the successor of CFS, but the cgroup interface files did not change: cpu.max and cpu.weight work exactly the same on top of it. One long-standing limitation is worth knowing: the cpu controller can only be enabled when all realtime-class processes sit in the root cgroup, and cgroup v2 bandwidth control does not govern realtime tasks. On desktop systems, systemd occasionally places an RT process in a non-root group; move it to the root cgroup first if enabling the controller fails.

Understanding cpu.max

The cpu.max file holds two values:

$MAX $PERIOD

Both are in microseconds. The meaning: all processes in the group together may run for at most MAX microseconds out of every PERIOD microseconds. The default is max 100000, meaning no limit over a 100 ms period. Some worked examples:

Value written to cpu.max	Effective CPU limit	Reading
max 100000	Unlimited	Default
50000 100000	50% of one CPU	50 ms of run time per 100 ms window
300000 1000000	30% of one CPU	0.3 s of run time per 1 s window
200000 100000	2 full CPUs	MAX can exceed PERIOD on multi-core systems

The last row surprises many learners: the quota is a group-wide budget across all cores, so on an 8-core machine a value of 200000 over 100000 lets the group keep two cores fully busy in aggregate.

Hands-On Lab: Cap a Workload at 30% CPU

Time to build it. Six steps, all as root (sudo -i gives you a root shell; plain sudo does not survive the shell redirections we use below).

Step 1: Confirm your setup

# stat -fc %T /sys/fs/cgroup/
cgroup2fs

# cat /sys/fs/cgroup/cgroup.controllers
cpuset cpu io memory hugetlb pids rdma misc

Both checks pass on any current distribution with no boot-time changes. If you see tmpfs instead, you are on an old hybrid layout; add systemd.unified_cgroup_hierarchy=1 to the kernel command line and reboot, then continue.

Step 2: Enable the cpu controller for child groups

Remember the top-down rule from Part 1: a controller must be enabled in the parent’s cgroup.subtree_control before children can use it. On systemd systems the root usually already has cpu enabled, but making it explicit never hurts:

# echo "+cpu" > /sys/fs/cgroup/cgroup.subtree_control

# cat /sys/fs/cgroup/cgroup.subtree_control
cpuset cpu io memory pids

If this write fails with “Invalid argument”, a realtime-class process is sitting outside the root cgroup, or your kernel was built with CONFIG_RT_GROUP_SCHED and RT tasks need moving to the root group first. This is the single most common stumbling block with the cgroups v2 CPU controller.

Step 3: Create your cgroup

# mkdir /sys/fs/cgroup/study_group

# ls /sys/fs/cgroup/study_group/ | head
cgroup.controllers
cgroup.events
cgroup.freeze
cgroup.kill
cgroup.max.depth
cgroup.max.descendants
cgroup.procs
cpu.idle
cpu.max
cpu.max.burst

One mkdir, and the kernel materialised the whole interface, including the cpu. files, because we enabled the controller one level up.

Step 4: Set the CPU bandwidth cap

Allow the group 300 ms of CPU time out of every 1 second, i.e. 30% of one core:

# echo "300000 1000000" > /sys/fs/cgroup/study_group/cpu.max

# cat /sys/fs/cgroup/study_group/cpu.max
300000 1000000

Step 5: Launch a CPU-hungry workload and move it in

We need something that burns CPU flat out. A shell arithmetic loop that counts as fast as it can is perfect, because the final count tells us how much CPU time it really received. Save this as burn.sh:

#!/bin/bash
# burn.sh - count as fast as possible for a fixed number of seconds,
# then report how far we got. The count is our CPU-time meter.
SECONDS_TO_RUN=${1:-10}
count=0
end=$(( $(date +%s) + SECONDS_TO_RUN ))
while [ "$(date +%s)" -lt "$end" ]; do
    count=$((count + 1))
done
echo "Iterations completed in ${SECONDS_TO_RUN}s: $count"

First, a baseline run with no cgroup restriction:

# chmod +x burn.sh
# ./burn.sh 10
Iterations completed in 10s: 2181647

Now run it inside the capped group. Launch it in the background, write its PID into cgroup.procs, and wait:

# ./burn.sh 10 &
[1] 4172
# echo 4172 > /sys/fs/cgroup/study_group/cgroup.procs

# cat /proc/4172/cgroup
0::/study_group

# wait
Iterations completed in 10s: 653210

The membership check via /proc/4172/cgroup confirms the move: the line 0::/study_group means the process now lives in our group. And the result speaks for itself: roughly 650 thousand iterations against a 2.18 million baseline, which is almost exactly 30%. Your absolute numbers will differ with hardware, but the ratio will track whatever you wrote into cpu.max. Try 100000, then 800000, and watch the iteration count follow.

Step 6: Inspect throttling statistics and clean up

The kernel keeps live accounting in cpu.stat:

# cat /sys/fs/cgroup/study_group/cpu.stat
usage_usec 3011842
user_usec 2570114
system_usec 441728
nr_periods 10
nr_throttled 10
throttled_usec 6987205

Read it like a story: over 10 accounting periods the group was throttled in all 10 (nr_throttled), spent about 3 seconds actually running (usage_usec), and sat runnable-but-forbidden for about 7 seconds (throttled_usec). Three plus seven equals our ten-second run, split precisely 30/70. When you monitor production containers, a steadily climbing nr_throttled is the classic signature of an undersized CPU limit.

Cleanup is one command once the group is empty of processes and children:

# rmdir /sys/fs/cgroup/study_group

Beyond cpu.max: The Other CPU Interface Files

Modern kernels expose more CPU knobs than older books cover. A quick tour:

File	What it does	Since
cpu.weight	Proportional share, range 1–10000, default 100. A group with weight 200 gets twice the CPU of a sibling with 100 under contention.	4.15
cpu.weight.nice	Same knob expressed in familiar nice values (-20 to 19)	4.15
cpu.max.burst	Lets a group bank unused quota and burst above cpu.max briefly; smooths latency for spiky workloads	5.14
cpu.pressure	PSI (Pressure Stall Information): how much time tasks stalled waiting for CPU; superb for detecting starvation	4.20
cpu.uclamp.min / max	Utilization clamping: hints the scheduler and cpufreq about how “big” tasks should appear; heavily used on Android for energy-aware scheduling	5.3
cpu.idle	Marks the group as idle-priority: it runs only when nothing else wants the CPU	6.0

Real-World Use Cases

• Container runtimes: when you run a container with a CPU limit of 0.5, the runtime writes 50000 100000 into that container’s cpu.max. You have now done by hand what Docker and Kubernetes automate.
• systemd services: CPUQuota=30% in a unit file is exactly our lab, managed declaratively. Try systemd-run --scope -p CPUQuota=30% ./burn.sh 10 and compare.
• Embedded systems: on a set-top box or automotive head unit, capping the media-scanning or telemetry tasks guarantees the UI thread stays responsive on a weak CPU.
• Build servers: place CI jobs in a weighted group so interactive users never fight compile farms for cycles.

Best Practices

• Prefer cpu.weight over cpu.max when your goal is fairness. Hard caps waste idle CPU; weights only bite under contention.
• Keep the default 100 ms period unless you have a latency reason to change it. Very short periods add scheduling overhead; very long ones create visible stutters while throttled.
• Watch nr_throttled and cpu.pressure before and after setting limits. Data beats guessing.
• In production, manage groups through systemd (units, slices, or delegation) rather than raw mkdir, so your settings survive reboots and do not fight the service manager.
• Set limits on your own sub-groups, never by editing groups that systemd owns, such as those inside system.slice.

Common Mistakes and Troubleshooting

• “No such file or directory” on cpu.max: the cpu controller is not enabled in the parent. Fix the parent’s subtree_control first (Step 2).
• “Device or resource busy” on write to subtree_control: you are hitting the no-internal-process rule: the cgroup already contains processes, so it cannot also delegate controllers to children. Move the processes into a leaf child first.
• echo fails under plain sudo: sudo echo x > file redirects as your normal user. Use a root shell, or echo x | sudo tee file.
• rmdir says busy: the group still has member processes or child directories. Move the PIDs out (write them into the parent’s cgroup.procs) or wait for them to exit.
• Limit seems ignored: confirm membership with cat /proc/<PID>/cgroup. Children created after the move inherit membership; processes started before the move and never migrated do not.

Performance and Security Considerations

Bandwidth throttling introduces latency by design: a throttled task waits for the next period even if cores sit idle. For latency-sensitive services, combine a generous cpu.max with cpu.max.burst, or use weights instead. On the security side, cgroups are a resource-containment tool, not a sandbox: a capped process can still read files and open sockets as its credentials allow. Combine cgroups with namespaces, seccomp, and proper delegation (so unprivileged users can only manage subtrees explicitly handed to them) for real isolation. The pids controller deserves a place in any hardening checklist, since it neutralises fork-bomb style denial of service with a single number.

Key Takeaways

• The cgroups v2 CPU controller offers weights (proportional, work-conserving) and bandwidth caps (hard ceiling via cpu.max).
• cpu.max takes “MAX PERIOD” in microseconds; the group runs at most MAX per PERIOD across all cores combined.
• The full manual workflow is: enable +cpu in the parent, mkdir the group, write cpu.max, write PIDs into cgroup.procs, verify via /proc/PID/cgroup, and read cpu.stat.
• Newer files worth knowing: cpu.weight, cpu.max.burst, cpu.pressure, cpu.uclamp, and cpu.idle.
• Everything Docker, Kubernetes, and systemd do with CPU limits reduces to these interface files.

Conclusion

You have now controlled the Linux scheduler with nothing but mkdir and echo. That is the elegance of the cgroups v2 CPU controller: a clean filesystem interface over a powerful kernel mechanism, and you measured its enforcement down to the percentage point. From here, experiment: give the group two members and watch them share the 30% budget, add cpu.max.burst, or reproduce the lab with systemd-run and compare.

This lab belongs to our free Linux kernel development course, which sits alongside the free Linux device drivers course and the free embedded systems course here on EmbeddedPathashala. In upcoming lessons we look at the memory controller and at how the scheduler that enforces these budgets actually works inside the kernel.

Frequently Asked Questions

How do I limit CPU usage of a process in Linux using cgroups v2?

Enable the cpu controller in the parent’s cgroup.subtree_control, create a group with mkdir under /sys/fs/cgroup, write a quota such as “300000 1000000” into the group’s cpu.max, and write the process PID into the group’s cgroup.procs. The kernel enforces the cap immediately.

What do the two numbers in cpu.max mean?

MAX and PERIOD, both in microseconds. Processes in the group may collectively run for MAX microseconds within each PERIOD. “300000 1000000” means 0.3 seconds of CPU per 1 second, i.e. a 30% cap of one core.

Can a cgroup use more than 100% CPU?

Yes, on multi-core systems. The quota is a group-wide budget, so MAX larger than PERIOD is valid: “200000 100000” allows two cores’ worth of CPU time in aggregate.

What is the difference between cpu.max and cpu.weight?

cpu.max is a hard ceiling that applies even when the system is idle. cpu.weight is proportional sharing that matters only under contention; an uncontended group can use all available CPU regardless of weight.

How can I tell whether my process is being throttled?

Read cpu.stat inside the group. nr_throttled counts periods in which the group hit its quota, and throttled_usec accumulates time spent runnable but not scheduled. cpu.pressure gives a complementary stall-based view.

Why does echo into a cgroup file fail with sudo?

The redirection is performed by your unprivileged shell, not by sudo. Open a root shell with sudo -i, or pipe through tee: echo value | sudo tee /sys/fs/cgroup/group/cpu.max.

Does cpu.max limit realtime (SCHED_FIFO/SCHED_RR) tasks?

No. cgroup v2 bandwidth control applies to fair-class scheduling. Realtime processes must reside in the root cgroup for the cpu controller to be enabled at all, and their bandwidth is governed by separate global sysctls.

Is manually creating cgroups safe on a systemd system?

For learning and experiments, yes: create your own top-level group and remove it afterwards. For production, use systemd units, systemd-run, or proper delegation so your configuration persists and does not conflict with the service manager.

References

• Linux kernel documentation: Control Group v2, CPU interface files (docs.kernel.org/admin-guide/cgroup-v2.html)
• man 7 cgroups (man7.org)
• Linux kernel documentation: PSI – Pressure Stall Information
• systemd.resource-control(5) manual page