Difference between revisions of "How To Use Posix Thread Priority Scheduling under Linux"
(9 intermediate revisions by the same user not shown) | |||
Line 2: | Line 2: | ||
The linux scheduler supports the | The linux scheduler supports the | ||
SCHED_FIFO scheduling policy defined by POSIX.1-2001. Threads scheduled with this "real-time" policy can be assigned a priority (under linux) in the range 1..99 with 99 representing the highest priority. Since ordinary, "non-real time" processes execute at priority 0 ( | SCHED_FIFO scheduling policy defined by POSIX.1-2001. Threads scheduled with this "real-time" policy can be assigned a priority (under linux) in the range 1..99 with 99 representing the highest priority. Since ordinary, "non-real time" processes execute at priority 0 (<tt>nice(1)</tt> modifies a "dynamic priority" which only affects processes with real-time priority 0 and implements fair timesharing among such processes) SCHED_FIFO threads are always capable to preempt "ordinary processes" (these use the policy SCHED_OTHER). | ||
Note, however, that unless the linux kernel is built with the RT_PREEMPT patch | Note, however, that unless the linux kernel is built with the RT_PREEMPT patch | ||
Line 25: | Line 25: | ||
<tt>pam_security</tt> module it is not necessary to execute EPICS as ''root''. | <tt>pam_security</tt> module it is not necessary to execute EPICS as ''root''. | ||
The system administrator can define the maximum priority that may be used | The system administrator can define the maximum priority that may be used | ||
by specific users and/or groups in a file under <tt>/etc/security/limits.d/</tt>, see | by specific users and/or groups in a file under <tt>/etc/security/limits.d/</tt>, see the manpage for <tt>limits.conf(5)</tt>. | ||
E.g., if a file <tt>/etc/security.limits.d/epics.conf</tt> is created with | E.g., if a file <tt>/etc/security.limits.d/epics.conf</tt> is created with | ||
the following contents: | the following contents (it may be necessary for a user to log out | ||
and log back on in order to obtain the new privileges): | |||
<pre><nowiki> | <pre><nowiki> | ||
Line 35: | Line 36: | ||
</nowiki></pre> | </nowiki></pre> | ||
then processes created by 'someuser' still have priority 0 by default (this is | |||
the 'soft' value) but 'someuser' may increase the resource limit up to 20 | the 'soft' value) but 'someuser' may increase the resource limit up to 20 | ||
either from a running program (using the system call <tt>setrlimit(2)</tt>) | either from a running program (using the system call <tt>setrlimit(2)</tt>) | ||
or | or within a shell e.g., with <tt>bash</tt>'s <tt>ulimit -r</tt> utility. | ||
Note that | Note that a user process may set and/or lower its hard limit but ''never increase'' it. RTM. | ||
=== Summary === | |||
You need to do three things: | |||
* Build EPICS base with <tt>USE_POSIX_THREAD_PRIORITY_SCHEDULING=YES</tt> | |||
* Make sure your EPICS application process has sufficient privileges to use SCHED_FIFO and the desired priority range. | |||
* Set the soft and hard RTPRIO limits for your EPICS application process to enable it to actually make use of the privilege. This step is required because it is usually a bad idea to run ''all'' processes created by a given user or group at real-time priorities. | |||
== End of the story? == | |||
Unfortunately, this is not the end of the story. Linux' <tt>sched_get_priority_max(2)</tt> system call which is used | |||
by EPICS to map EPICS priorities from/to linux/pthread priorities | |||
always reports the maximal supported priority (99) without | |||
considering any limits that the system enforces (as discussed above). | |||
This has the consequence [https://bugs.launchpad.net/epics-base/+bug/835138 (see bug #835138)] that threads which are created with | |||
a desired priority that is greater than the resource limit | |||
eventually end up with the ''lowest'' priority 0. | |||
=== Work-around === | |||
* Either apply the [https://bugs.launchpad.net/epics-base/+bug/835138 bugfix] (AFAIK the problem is still present in 3.14.12.1, may be fixed thereafter) | |||
* or make sure the RTPRIO limits are set to the maximally possible value (99) |
Latest revision as of 03:35, 31 August 2011
What is this about?
The linux scheduler supports the SCHED_FIFO scheduling policy defined by POSIX.1-2001. Threads scheduled with this "real-time" policy can be assigned a priority (under linux) in the range 1..99 with 99 representing the highest priority. Since ordinary, "non-real time" processes execute at priority 0 (nice(1) modifies a "dynamic priority" which only affects processes with real-time priority 0 and implements fair timesharing among such processes) SCHED_FIFO threads are always capable to preempt "ordinary processes" (these use the policy SCHED_OTHER).
Note, however, that unless the linux kernel is built with the RT_PREEMPT patch and measures are taken to lock the EPICS process in memory etc., no strictly deterministic latencies can be expected even under the SCHED_FIFO policy which thus is to be considered soft-real time.
How can I let epicsThreads use the SCHED_FIFO policy?
In order to let EPICS use the SCHED_FIFO real-time policy you first need to check that the following option is set to "YES" in epics-base/configure/CONFIG_SITE
# Use POSIX thread priority scheduling (YES or NO) USE_POSIX_THREAD_PRIORITY_SCHEDULING = YES
If you find that the current setting is "NO" then you need to rebuild EPICS base (make clean uninstall; make) after changing to 'YES'.
Since engaging the SCHED_FIFO policy gives any thread created under that policy a higher priority than any normal process'es using SCHED_FIFO requires special privileges. Under a reasonably recent linux equipped with the pam_security module it is not necessary to execute EPICS as root. The system administrator can define the maximum priority that may be used by specific users and/or groups in a file under /etc/security/limits.d/, see the manpage for limits.conf(5).
E.g., if a file /etc/security.limits.d/epics.conf is created with the following contents (it may be necessary for a user to log out and log back on in order to obtain the new privileges):
someuser hard rtprio 20 someuser soft rtprio 0
then processes created by 'someuser' still have priority 0 by default (this is the 'soft' value) but 'someuser' may increase the resource limit up to 20 either from a running program (using the system call setrlimit(2)) or within a shell e.g., with bash's ulimit -r utility.
Note that a user process may set and/or lower its hard limit but never increase it. RTM.
Summary
You need to do three things:
- Build EPICS base with USE_POSIX_THREAD_PRIORITY_SCHEDULING=YES
- Make sure your EPICS application process has sufficient privileges to use SCHED_FIFO and the desired priority range.
- Set the soft and hard RTPRIO limits for your EPICS application process to enable it to actually make use of the privilege. This step is required because it is usually a bad idea to run all processes created by a given user or group at real-time priorities.
End of the story?
Unfortunately, this is not the end of the story. Linux' sched_get_priority_max(2) system call which is used by EPICS to map EPICS priorities from/to linux/pthread priorities always reports the maximal supported priority (99) without considering any limits that the system enforces (as discussed above). This has the consequence (see bug #835138) that threads which are created with a desired priority that is greater than the resource limit eventually end up with the lowest priority 0.
Work-around
- Either apply the bugfix (AFAIK the problem is still present in 3.14.12.1, may be fixed thereafter)
- or make sure the RTPRIO limits are set to the maximally possible value (99)