CPU Bandwidth Control with cgroups v2: cpu.max, cpu.weight and systemd
Linux Kernel Programming Course | CPU Scheduler Series – Part 2 | Lecture 4
Affinity decides where a thread runs and policy decides how urgently. The third lever — the subject of this lecture in our free Linux kernel programming course — is how much: CPU bandwidth control with cgroups. Control groups (cgroups) let you cap, weight and account the CPU consumption of entire groups of processes. This is the mechanism underneath containers, Kubernetes CPU limits, systemd resource properties, and Android’s app throttling. We use cgroup v2 exclusively, since it has been the default on all major distributions for years; older tutorials showing cpu.cfs_quota_us describe the legacy v1 interface.
What You Will Learn
- The cgroup v2 unified hierarchy and how the CPU controller plugs into the scheduler
- Hard capping with
cpu.max(quota and period) - Proportional sharing with
cpu.weight - A hands-on demo: throttling a CPU hog and verifying with
cpu.stat - Doing the same the clean way with systemd (
CPUQuota=,CPUWeight=) - How container CPU limits map onto these exact files
Prerequisites
Lectures 1–3 of this series, root access on a modern distro (Ubuntu 22.04+, Fedora, Debian 12+), and basic shell skills. Verify you are on cgroup v2:
stat -fc %T /sys/fs/cgroup
# cgroup2fs <-- good; "tmpfs" would indicate legacy v1 mounts
How the CPU Controller Works
cgroup v2 exposes one unified tree under /sys/fs/cgroup. Every directory is a group; processes listed in a group’s cgroup.procs belong to it. When the cpu controller is enabled for a subtree, the scheduler’s fair class treats each group as a schedulable entity: bandwidth is first divided between groups according to weights and caps, then within each group among its tasks as usual.
| /sys/fs/cgroup (root) 100% of CPU capacity |
||
| ↓ split by cpu.weight ↓ | ||
| app/ weight 200 → ~2 shares |
batch/ weight 100 → ~1 share |
capped/ cpu.max = 50ms/100ms → hard 50% ceiling |
Weights matter only under contention; cpu.max is enforced always
The two knobs behave very differently:
| File | Type | Meaning |
|---|---|---|
cpu.weight |
Soft / proportional | 1–10000 (default 100). Relative share when CPUs are contended; unused capacity flows freely. |
cpu.max |
Hard cap | “QUOTA PERIOD” in microseconds. The group gets at most QUOTA µs of CPU per PERIOD, even on an idle machine. |
cpu.stat |
Read-only | Usage plus nr_throttled / throttled_usec — the proof your cap is biting. |
Hands-On: Throttling a CPU Hog
Step 1 – Create a group and enable the controller
cd /sys/fs/cgroup
sudo mkdir demo
# Make sure the cpu controller is delegated to children of root
cat cgroup.subtree_control # should list: cpu ...
echo "+cpu" | sudo tee cgroup.subtree_control # if it doesn't
Step 2 – Apply a 25% cap
# 25 ms of CPU per 100 ms period = 25% of one CPU
echo "25000 100000" | sudo tee demo/cpu.max
Step 3 – Put a spinner inside and watch
# A deliberate 100% CPU burner
yes > /dev/null &
HOG=$!
# Move it into the group
echo $HOG | sudo tee demo/cgroup.procs
# Observe: top now shows ~25% CPU for 'yes'
top -p $HOG
Step 4 – Verify throttling statistics
cat demo/cpu.stat
# usage_usec 2513xxx
# nr_periods 101
# nr_throttled 98 <-- throttled in 98 of 101 periods
# throttled_usec 733xxxx
A rising nr_throttled is the definitive signal that the group wanted more CPU than its quota allows. Clean up with:
kill $HOG
sudo rmdir demo
Multi-CPU note: quota can exceed the period. 200000 100000 means “up to 2 full CPUs worth” — exactly how a container gets a 2-CPU limit.
The Clean Way on Real Systems: systemd
On systemd distributions, systemd owns the cgroup tree; creating raw directories under its slices can conflict with it. The supported approach is resource properties, which systemd translates into the same cpu.max/cpu.weight files:
# One-off command capped at 25% of one CPU
sudo systemd-run --scope -p CPUQuota=25% -- yes > /dev/null
# Persistent limit on a service
sudo systemctl set-property myapp.service CPUQuota=150% CPUWeight=200
# Inspect where it landed
systemctl show myapp.service -p CPUQuota
cat /sys/fs/cgroup/system.slice/myapp.service/cpu.max
Container runtimes do the same translation: docker run --cpus=1.5 and a Kubernetes CPU limit both end up writing cpu.max for the container’s cgroup. Once you understand this lecture, container CPU limits stop being magic.
Real-World Use Cases
- Embedded infotainment/HMI: cap logging and telemetry groups so the UI group always has headroom.
- Multi-tenant servers: weight production traffic above batch analytics without hard-starving either.
- CI runners: cap each job so one runaway build cannot slow the fleet.
- Thermal management on fanless devices: a hard cpu.max cap is a crude but effective thermal ceiling.
Common Mistakes and Troubleshooting
- “No such file” for cpu.max: the cpu controller isn’t enabled in the parent’s
cgroup.subtree_control. Enable it top-down. - Writing to cgroup.procs fails with EBUSY/ENOTSUPP: cgroup v2’s no-internal-process rule — a group with child groups holding controllers cannot also hold processes directly. Use leaf groups.
- Mixing v1 habits:
cpu.cfs_quota_us,cpu.sharesandcpuacctare legacy v1 names. Their v2 equivalents arecpu.max,cpu.weight, andcpu.stat. - Latency spikes under quota: a group that exhausts its quota mid-period sleeps until the next period. Latency-sensitive groups should get weight-based sharing, not hard caps — or a shorter period.
- Fighting systemd: manual groups inside system.slice may be cleaned up. Use systemd properties or a delegated subtree.
Performance Considerations and Best Practices
- Prefer
cpu.weightfor general prioritization — it wastes nothing on an idle machine. Reservecpu.maxfor true ceilings (billing, thermal, containment). - Keep hierarchies shallow; every level adds scheduling bookkeeping.
- Monitor
cpu.statin production — persistent throttling with idle CPUs means a misconfigured cap. - Combine the three levers deliberately: cpuset (where) + policy (how urgently) + bandwidth (how much) is the complete scheduler-control toolkit of this free Linux kernel development course.
Key Takeaways
- cgroup v2’s CPU controller schedules groups first, tasks second.
cpu.weight= proportional under contention;cpu.max= absolute ceiling always.cpu.stat‘s throttling counters verify your configuration objectively.- systemd properties and container CPU limits are thin wrappers over these same files.
Conclusion
You can now bound the CPU consumption of any group of processes with surgical precision — by hand for learning, and via systemd for production. This completes the resource-control triangle in our free Linux kernel programming course. If you’re also following the free linux device drivers course or free embedded systems course tracks, note how often shipped devices rely on exactly these knobs to keep critical subsystems responsive. Next lecture, we tackle the big one: turning mainline Linux into a real-time operating system with PREEMPT_RT.
Frequently Asked Questions (FAQ)
1. How do I know whether my system uses cgroup v1 or v2?
Run stat -fc %T /sys/fs/cgroup. cgroup2fs means the unified v2 hierarchy; all current mainstream distros default to it.
2. What is the difference between cpu.weight and cpu.max?
cpu.weight divides CPU proportionally only when there is contention; cpu.max is a hard ceiling enforced even when the machine is otherwise idle.
3. Can a cgroup use more than one CPU under cpu.max?
Yes — set quota larger than period. “300000 100000” allows up to three CPUs’ worth of runtime per period.
4. Why is my capped service seeing latency spikes?
Quota exhaustion: the group ran out mid-period and slept until the refill. Check nr_throttled in cpu.stat; consider weights instead of caps, or reduce the period.
5. Do these limits apply to real-time (SCHED_FIFO) threads?
cpu.max governs the fair class. RT tasks are constrained separately by the global RT throttling limits discussed in Lecture 3 (and RT group scheduling where enabled).
6. How do Docker/Kubernetes CPU limits relate to this?
Directly: –cpus and Kubernetes limits are translated by the runtime into the container cgroup’s cpu.max; requests map to cpu.weight.
7. Is it safe to create cgroups manually on a systemd system?
For experiments, yes — at the top level. For anything persistent, use systemd’s CPUQuota=/CPUWeight= properties or request a delegated subtree.
Continue Your Free Linux Kernel Development Course
Next up: converting mainline Linux into an RTOS with PREEMPT_RT.

2 Comments