View source for How To Use Posix Thread Priority Scheduling under Linux

== What is this about? ==

The linux scheduler supports the 
SCHED_FIFO scheduling policy defined by POSIX.1-2001. Threads scheduled with this "real-time" policy can be assigned a priority (under linux) in the range 1..99 with 99 representing the highest priority. Since ordinary, "non-real time" processes execute at priority 0 (<tt>nice(1)</tt> modifies a "dynamic priority" which only affects processes with real-time priority 0 and implements fair timesharing among such processes) SCHED_FIFO threads are always capable to preempt "ordinary processes" (these use the policy SCHED_OTHER).

Note, however, that unless the linux kernel is built with the RT_PREEMPT patch
and measures are taken to lock the EPICS process in memory etc.,
no strictly deterministic latencies can be expected even under the
SCHED_FIFO policy which thus is to be considered ''soft-real time''.

== How can I let epicsThreads use the SCHED_FIFO policy? ==

In order to let EPICS use the SCHED_FIFO real-time policy you first need to check that the following option is set to "YES" in <tt>epics-base/configure/CONFIG_SITE</tt>

<pre><nowiki>
# Use POSIX thread priority scheduling (YES or NO)
USE_POSIX_THREAD_PRIORITY_SCHEDULING = YES
</nowiki></pre>

If you find that the current setting is "NO" then you need to ''rebuild'' EPICS base (<tt>make clean uninstall; make</tt>) after changing to 'YES'.

Since engaging the SCHED_FIFO policy gives any thread created under that policy
a higher priority than any normal process'es using SCHED_FIFO requires special
''privileges''. Under a reasonably recent linux equipped with the
<tt>pam_security</tt> module it is not necessary to execute EPICS as ''root''.
The system administrator can define the maximum priority that may be used
by specific users and/or groups in a file under <tt>/etc/security/limits.d/</tt>, see the manpage for <tt>limits.conf(5)</tt>.

E.g., if a file <tt>/etc/security.limits.d/epics.conf</tt> is created with 
the following contents (it may be necessary for a user to log out
and log back on in order to obtain the new privileges):

<pre><nowiki>
someuser hard rtprio 20
someuser soft rtprio  0
</nowiki></pre>

then processes created by 'someuser' still have priority 0 by default (this is
the 'soft' value) but 'someuser' may increase the resource limit up to 20 
either from a running program (using the system call <tt>setrlimit(2)</tt>)
or within a shell e.g., with <tt>bash</tt>'s <tt>ulimit -r</tt> utility.

Note that a user process may set and/or lower its hard limit but ''never increase'' it. RTM.

=== Summary ===
You need to do three things:
* Build EPICS base with <tt>USE_POSIX_THREAD_PRIORITY_SCHEDULING=YES</tt>
* Make sure your EPICS application process has sufficient privileges to use SCHED_FIFO and the desired priority range.
* Set the soft and hard RTPRIO limits for your EPICS application process to enable it to actually make use of the privilege. This step is required because it is usually a bad idea to run ''all'' processes created by a given user or group at real-time priorities.

== End of the story? ==
Unfortunately, this is not the end of the story. Linux' <tt>sched_get_priority_max(2)</tt> system call which is used
by EPICS to map EPICS priorities from/to linux/pthread priorities
always reports the maximal supported priority (99) without
considering any limits that the system enforces (as discussed above).
This has the consequence [https://bugs.launchpad.net/epics-base/+bug/835138 (see bug #835138)] that threads which are created with
a desired priority that is greater than the resource limit
eventually end up with the ''lowest'' priority 0.

=== Work-around ===
* Either apply the [https://bugs.launchpad.net/epics-base/+bug/835138 bugfix] (AFAIK the problem is still present in 3.14.12.1, may be fixed thereafter)
* or make sure the RTPRIO limits are set to the maximally possible value (99)