In the previous lecture of this free linux kernel development course we covered timeshare scheduling, where fairness is the goal. Some workloads — a motor control loop, an audio pipeline, a protocol stack with a hard deadline — cannot tolerate “fair” scheduling; they need deterministic scheduling instead. That’s what Linux’s real-time scheduling policies are for, and that’s the subject of this lecture in our free embedded linux course.
Choosing a real-time policy is a statement: “I understand this thread’s timing needs better than the general-purpose scheduler does, and I want it to preempt everything else.” That power comes with responsibility, which we’ll cover along with the mechanics.
What You Will Learn
- How SCHED_FIFO and SCHED_RR differ from each other and from timeshare policies
- The 1–99 real-time priority range and how preemption between real-time threads works
- How Linux protects the system from a runaway real-time thread
- How to create a real-time thread from C using pthreads
- Practical guidance for deciding whether a thread actually needs a real-time policy
Prerequisites
- The previous lecture on timeshare scheduling and nice values
- Root access or CAP_SYS_NICE on your test machine (required to run real-time threads)
- Working pthreads knowledge (thread creation, attributes)
SCHED_FIFO and SCHED_RR
Linux offers two real-time policies. Both always preempt any timeshare thread, and both use a static priority from 1 to 99, where 99 is highest.
| Policy | Behaviour | Best Fit |
|---|---|---|
| SCHED_FIFO | Run-to-completion. Once running, a thread keeps the CPU until it blocks, terminates, or a higher-priority real-time thread becomes ready. | A single dominant time-critical thread with no peers at the same priority. |
| SCHED_RR | Same as SCHED_FIFO, but threads at equal priority round-robin once each exceeds its time slice (default 100 ms, tunable via /proc/sys/kernel/sched_rr_timeslice_ms since Linux 3.9). | Several equally important real-time threads that must all get a turn. |
Getting Permission: CAP_SYS_NICE
Assigning a real-time policy to a thread requires the CAP_SYS_NICE capability, which by default only root holds. In production you would grant this capability to your specific binary rather than running it as root:
$ sudo setcap cap_sys_nice+ep ./ep_rt_thread
Guarding Against a Runaway Real-Time Thread
A real-time thread stuck in an infinite loop (usually a bug) can starve every lower-priority real-time thread and every timeshare thread, potentially locking up the system. Since Linux 2.6.25 the kernel guards against this by reserving a slice of CPU time for non-real-time threads, controlled by two knobs:
$ cat /proc/sys/kernel/sched_rt_period_us
1000000
$ cat /proc/sys/kernel/sched_rt_runtime_us
950000
With the defaults shown above, out of every 1-second period, real-time threads may consume at most 950 ms — the remaining 50 ms is always available to non-real-time threads. If your application genuinely needs real-time threads to take 100% of the CPU, you can disable the guard by setting sched_rt_runtime_us to -1, but do this deliberately and only after you trust your real-time code.
sched_rt_runtime_us during development “to make things run smoother,” then forgetting to re-enable it. A single bug in a real-time thread can then hang the entire board with no safety net.
Original Demo: Creating a SCHED_FIFO Thread
Let’s write ep_rt_thread, a small original program that spawns one SCHED_FIFO worker thread at priority 50 and prints confirmation of the policy actually applied.
/* ep_rt_thread.c
* Creates a SCHED_FIFO worker thread and confirms its policy/priority.
* Build: gcc -O2 -pthread -o ep_rt_thread ep_rt_thread.c
* Run (needs CAP_SYS_NICE or root): sudo ./ep_rt_thread
*/
#include <stdio.h>
#include <pthread.h>
#include <sched.h>
#include <string.h>
static void *ep_worker(void *arg)
{
int policy;
struct sched_param param;
pthread_getschedparam(pthread_self(), &policy, ¶m);
printf("ep_worker: running with policy=%s priority=%d\n",
policy == SCHED_FIFO ? "SCHED_FIFO" : "other",
param.sched_priority);
return NULL;
}
int main(void)
{
pthread_t tid;
pthread_attr_t attr;
struct sched_param param;
pthread_attr_init(&attr);
pthread_attr_setinheritsched(&attr, PTHREAD_EXPLICIT_SCHED);
pthread_attr_setschedpolicy(&attr, SCHED_FIFO);
memset(¶m, 0, sizeof(param));
param.sched_priority = 50;
pthread_attr_setschedparam(&attr, ¶m);
if (pthread_create(&tid, &attr, ep_worker, NULL) != 0) {
perror("pthread_create");
return 1;
}
pthread_join(tid, NULL);
pthread_attr_destroy(&attr);
return 0;
}
Build and run it as root (or with the capability granted):
$ gcc -O2 -pthread -o ep_rt_thread ep_rt_thread.c
$ sudo ./ep_rt_thread
ep_worker: running with policy=SCHED_FIFO priority=50
If you run it without the required capability, pthread_create returns EPERM and the worker never starts — a good sanity check that the guard is actually working.
Choosing a Policy
In practice, most workloads should stay on timeshare policies. I/O-bound threads accumulate spare entitlement while blocked and get scheduled almost immediately once they unblock; CPU-bound threads naturally absorb whatever cycles are left over. Reach for a real-time policy only when a thread genuinely meets all four of these:
- It has a hard deadline by which it must produce output
- Missing that deadline breaks the system’s usefulness, not just its performance
- It is event-driven, not continuously busy
- It is not compute-bound (it does a bounded amount of work per event)
Classic real-time examples include robot arm servo control, low-latency audio/multimedia pipelines, and time-sensitive communication processing.
Best Practices
- Default to SCHED_NORMAL. Only promote a thread to SCHED_FIFO/SCHED_RR after you can articulate its actual deadline.
- Keep real-time thread bodies short and bounded — a real-time thread that blocks on an unbounded operation defeats the point of using a real-time policy.
- Never leave
sched_rt_runtime_usat-1in production unless you have thoroughly tested every real-time code path. - Grant CAP_SYS_NICE to the specific binary via
setcapinstead of running whole services as root.
Summary
SCHED_FIFO and SCHED_RR give you deterministic, priority-based scheduling that always preempts timeshare threads, with SCHED_RR adding round-robin fairness among equal-priority peers. The kernel protects the rest of the system from a runaway real-time thread by reserving CPU time for non-real-time work, tunable via sched_rt_period_us and sched_rt_runtime_us. Use real-time policies sparingly, and only for genuinely deadline-driven threads. Next in this free linux device drivers course: how to actually pick real-time priorities using Rate Monotonic Analysis.
FAQ
What is the difference between SCHED_FIFO and SCHED_RR?
Both are run-to-completion real-time policies that preempt timeshare threads. SCHED_RR additionally round-robins between threads that share the same priority once each exceeds its time slice; SCHED_FIFO does not — an equal-priority thread runs until it blocks or terminates.
What is the real-time priority range in Linux?
1 to 99, with 99 being the highest priority. This range is separate from and unrelated to the -20 to 19 nice value used by timeshare threads.
Do real-time threads always preempt normal threads?
Yes. Any ready real-time thread preempts every timeshare (SCHED_NORMAL/BATCH/IDLE) thread, regardless of nice value.
Can a real-time thread hang the whole system?
It can, if it becomes compute-bound (for example, an infinite loop bug). Since Linux 2.6.25 the kernel reserves a default 5% of CPU time for non-real-time threads to prevent a full lockup, configurable via sched_rt_period_us and sched_rt_runtime_us.
What privilege is needed to use a real-time policy?
CAP_SYS_NICE, held by root by default. You can grant it to a specific binary with setcap instead of running the whole program as root.
Should I use a real-time policy for my embedded application?
Only if the thread has a hard deadline, missing it breaks correctness, it’s event-driven, and it is not compute-bound. Most application threads are better served by the default SCHED_NORMAL policy.
What happens if sched_rt_runtime_us is set to -1?
It disables the CPU-time reservation for non-real-time threads, letting real-time threads consume up to 100% of the CPU. This removes an important safety net and should only be done deliberately, after thorough testing.
Continue the Free Linux Kernel Development Course
Next up: assigning real-time priorities correctly using Rate Monotonic Analysis (RMA).
Next Lecture Course Index
2 Comments