EPICS for Dummies

What is EPICS?

EPICS (Experimental Physics and Industrial Control System) is a distributed control system, which means that the IOC (software) can run on a different computer as the user interfaces.

The next 5 sections are dedicated to the definition of basic EPICS components (IOC, PV, record, database, add devices?). To create your own EPICS support, you can skip directly to #How does one create an IOC?.

What is an IOC?

IOC (Input/Output Controller) is a piece of software running on a computer:

• loads databases
• serves something called process variables (PVs) that you can read and write to
• used to run on VMEs, but nowadays it runs on anything.
• usually talks to hardware, but that isn’t a requirement.

IOCs are started with startup scripts. Those scripts tell the IOC what database files to load and where those files are. When the IOC is told to load a database, what it is doing is parsing that database file, identifying the records that are contained within, and then generating the actual epics PVs that a user will interact with.

Records and databases (lists of records) are configuration files describing the initial setup for PVs, the IOC is the software that manages the behavior of PVs and communication to get/set their values.

What is a PV?

PV (Process Variable) is a the field from a record:

PV = record_name + . + field_name

Most of the interactive parts of user displays have PVs associated with them.

What is a record?

Records are the building blocks of EPICS:

• There is a lot of different types of record, and each type is designed to perform a specific function
• Records are customizable via fields that allow users to configure how they behave and monitor their state (see #More about record fields to know how to learn about record fields)
• Records do nothing until they are processed (see #SCAN, PROC & FLNK fields)
• Records vary significantly in complexity (see #More about records)
• It is rare for new records to be created, now that asyn device support exists add link to asyn?).

Where do records come from?

Typically, records at the APS comes from either:

• EPICS base: main core of EPICS
• synApps: a collection of EPICS modules that are commonly used at the APS

A module is similar to a python package: it is an already written set of databases and associated behavior you can load into your IOC to do certain things, for example talk to specific hardware. An IOC can link to several modules.

Records in synApps modules

Module	Records	Module	Records
alive	alive	motor	motor
asyn	asyn	optics	table
busy	busy	scaler	scaler
calc	acalcout, scalcout, sseq, swait, transform	sscan	sscan
camac	camac	std	epid, timestamp, throttle
lua	luascript	vac	digitel, vs
mca	mca	vme	vme
allenBradley	ab1771IFE, ab1771N, ab1771IX, ...	allenBradley	..., ab1791, abDcm

If a record isn't listed here, it probably comes from EPICS base. The advantage to using synApps over a collection of modules one assembles on their own is that some amount of BCDA testing has been done.

What is a record field?

Record fields are controllable or informative properties of a record.

Noteworthy fields

• VAL - the default field (used if no field is specified in the vast majority of cases)
• SCAN - determines when a record processes
• NAME - the record name (helpful if records are aliased)
• DESC - record description
• RTYP - the record type
• DTYP - the device support type (not meaningful for all record types)
• DISP - disable puts (ca = channel access & pva = pv access) from outside the IOC (when non-zero); e.g. disable a motor from the motorx_all screen
• DISV/DISA/SDIS - disable record processing internally
  • Record processing is disabled when DISA equals DISV (e.g. uses SDIS field to read in a value/mode, which will disable some control if a certain mode is active)
  • DISA is set to the value of the PV in the SDIS field, if it isn't empty
• TPRO - trace processing: when set to 1 prints out all the records that get processed after the record is processed; used only for troubleshooting

Note: some of the fields of a record are actually record attributes (they return the same value for all instances of the record type; see https://epics.anl.gov/)

SCAN, PROC & FLNK fields

Records do nothing until they are processed. Records can be processed multiple ways:

• Manually:
  • Write 1 (or any non-zero value) to a record’s PROC field
  • Forward link (FLNK) from another record
• Periodically by setting the SCAN field to something other than Passive

SCAN field options

Index	Option	Index	Option
0	Passive	5	2 second
1	Event	6	1 second
2	I/O Intr	7	0.5 second
3	10 second	8	0.2 second
4	5 second	9	0.1 second

Note:

• If a record’s SCAN field is not specified in a database, it defaults to Passive.
• The SCAN field is common to all record types and the options can be selected by index instead of by string.
• I/O Intr processes a record when the driver gets a new value, but is device/driver dependent and isn’t guaranteed to work

Record link types

Much of the power of EPICS comes from the ability to link EPICS records together. There are two link implementations: standard and dynamic.

• Dynamic links (pink) are found on userCalc (swait records) and scan (sscan records) screens; this type of link accept only a PV name. THe role of the process command (see below) is taken in a separate field.
• Standard links (purple) are found everywhere else.

inlinkHelp and outlinkHelp are accessible from userTransform screens and scaler screens (for the later only).

Input Links

Input links read a value from another PV. It accepts a PV name followed by a process command (e.g. NPP) and an alarm command (e.g. NMS, ignore this). Here are the possible process commands for an input link:

• NPP - Just read the value; don't process the target record (i.e. read a potentially stale value); this is a default command.
• PP - Process the target record and then read a value from it.
• CA - Read a value without causing processing to occur.
• CP - Process linking record every time the target record processes.
• CPP - Same as CP, but linking record process only if its SCAN field has the value Passive.

Note: inlinkHelp is accessible from userTransform screens.

Output Links

Output links write a value to another PV. It accepts a PV name followed by a process command (e.g. NPP) and an alarm command (e.g. NMS, ignore this). Here are the possible process commands for an output link:

• NPP - Just write a value; don't process the target record; this is a default command.
• PP - Write a value and then process the target record. To be use if you want EPICS to trace processing (= wait for completion?)
• CA - Write a value; let the target record decide whether or not to process (as it would if you typed the value by hand); if the processing does occur, EPICS will not trace it.

Note:

• The default link process command, NPP, is almost never the desired behavior for output links. Change NPP to PP immediately and revert if problems are discovered.
• If the target PV is itself a link field, you must specify the command CA; if you want to wait for completion in this case, you must use either the sseq or the sCalcout record, select CA and set the link's WAIT field to Wait
• outlinkHelp is accessible from userTransform and scaler screens.

Forward Links

Forward link look like output links; they are purple on user displays, but they don't auto-populate with process & alarm commands. Forward link causes the linked record to process when the record containing the forward link is processed

• Linked record is in the same IOC (PROC field for linked record is optional):
  • record_name.FLNK = linked_record_name
  • record_name.FLNK = linked_record_name.PROC
• Linked record is in a different IOC (PROC field for linked record is required):
  • record_name.FLNK = linked_record_name.PROC

Note:

• Forwarding-linking records in the same IOC by only specifying the record name is an exception to the "VAL is the default field" rule.
• When omitting the PROC field of a linked record in a different IOC, nothing will happen but you won't get any error message.
• Always specify the PROC field to avoid problems.

More about records

• Records vary significantly in complexity; one way to classify them would be (*):
  • Simple records are associated with individual data types

  bi, bo, mbbi, mbbo, longin, longout, ai, ao, stringin, stringout, waveform

  • Intermediate records often perform calculations or link multiple records

  calcout, scalcout, acalcout, swait, transform, fanout, dfanout

  • Complex records are associated with devices or higher-level functionality

  motor, scaler, mca, sscan, table, digitel, vs

  • Flexible records allow custom functionality to be implemented

  sub, aSub, luascript

(*) this is not an official classification, nor an exhaustive list of records.

Note:

• areaDetector databases are composed of mostly simple record.

More about record fields

To find documentation for a specific field you need to know/find answers to the following questions:

• Is the field common to all or input/output record types?
• If not, does the record come from EPICS base or a different EPICS module (synApps)?
  • The EPICS base record reference documentation is the best place to find information about record types that come from EPICS base.
    Field documentation appears in multiple locations:
    • Fields common to all record types
    • Fields common to input record types
    • Fields common to output record types
    • Record-specific documentation
  • synApps record reference documentation
    • In each module's documentation directory
      • https://github.com/epics-modules
      • /APSshare/epics/synApps_X_Y/support

What is a databases?

A database is just a text file with a .db (or .template) extension containing a collection of records that are related.

Databases are generalized with macros to allow multiple instances to be loaded (e.g. $(P)):

• Macros are variables containing strings to be substituted when the database is loaded.
• Macros without default values are required to load the database; there is no easy way to figure out what those are for a given database other than looking at the file itself.
• Macros required by the database should match the macros required by associated screens & autosave (.req) files.
• The P (prefix) macro usually includes a colon (:), which is the standard separator used in PV names.

How are databases loaded?

There are 2 ways to load databases which are strictly equivalent.

dbLoadRecords

dbLoadRecords("$(IP)/db/ADAM_4018.db","P=kmp:,R=adam1,PORT=serial1,A=01")
dbLoadRecords("$(IP)/db/ADAM_4018.db","P=kmp:,R=adam2,PORT=serial1,A=02")

dbLoadTemplate

dbLoadTemplate("substitutions/ADAM_4018.substitutions","P=kmp:")

where ADAM_4018.substitutions:

file "$(IP)/db/ADAM_4018.db"
{
pattern
{R,PORT,A}
{adam1, serial1,01}
{adam2, serial1,02}
}

What is a device?

What is a device?

How does one create an IOC?

There are multiple ways to create an IOC, but no one does it from scratch:

The EPICS base approach

It creates a bare bones IOC but requires more knowledge of the EPICS build system to get started:

makeBaseApp.pl

• Must be run multiple times
• Creates IOC contents in the current directory
• synApps support needs to be added manually

The synApps approach

It is based on the xxx module and designed to give you a usable IOC with as little effort as possible:

mkioc

• A single command leaves you with an IOC you can build:

/APSshare/bin/mkioc -n -f -s 6-2-1 kmp

• Creates a top-level IOC directory in the current directory
• Common synApps support is included by default
• mkioc will ask you questions if you run it without options

Note: The BCDA convention is to name development IOCs using the developer's initials (here kmp).

IOC Layout

Overview

The IOC layout can be overwhelming and confusing to those who aren’t yet familiar with it. The EPICS build system creates many directories, making it harder to find the important directories. Important files are often buried multiple directories down from the top-level. It is helpful to look at subsets of the IOC directory.

kmp
	bin	rhel8-x86_64
	configure	O.Common
		O.rhel8-x86_64
	db
	dbd
	iocBoot	iockmp	autosave
			iocsh
			softioc	commands
			substitutions
	kmpApp	Db	O.Common
			O.rhel8-x86_64
		op	adl
			ui	autoconvert
		src	O.Common
			O.rhel8-x86_64
	lib	rhel8-x86_64

Build configuration

Orange folders:

• kmp/configure:

contains the files that specify which versions of EPICS base and EPICS modules should be should be used when building the IOC.

• kmp/kmpApp/Db:

contains local databases (and associated autosave files) and protocol files used by the IOC.

• kmp/kmpApp/src:

contains the Makefile that determines all of the software that gets built into the IOC binary. ; local sequence programs and other EPICS support that needs to be compiled go here.

Intermediate build directories

Grey folders:
The EPICS Build system will generate O. files (Common, EPICS HOST and TARGET architectures) in the configure, Db, and src directory. These directories are not needed after the IOC is built and can be removed with the make clean command. If they are not removed, they can be safely ignored. It is highly unlikely that you will ever need to look at any of the files in them.

Build products

The EPICS build system creates top level bin, db, dbd, and lib directories (red folders):

• kmp/bin/rehl8-x86_64:

contains the architecture-specific IOC binary (or munch file for VxWorks IOCs).

• kmp/db:

contains installed database files which may or may not be used, since many IOCs reference the databases in the Db directory instead.

• kmp/dbd:

contains a single database definition file to be loaded by the IOC.

• kmp/lib/rehl8-x86_64:

contains architecture-specific files for EPICS modules, but should be empty for the IOC.

The build system creates an envPaths file in the startup directory, which contains the locations of the modules defined in the RELEASE file.

Run-time configuration

Green folders:

The iockmp directory is referred to as the startup directory. It contains the st.cmd file, which is the runtime configuration for the IOC what means runtime? The IOC’s config is often broken into separate .iocsh (or .cmd) files so that support can be more easily commented out and to improve readability. Substitutions and iocsh files usually reside in subdirectories to reduce clutter in the startup directory.

Start scripts

Blue folders:
The top-level IOC directory contains scripts that start medm/caQtDM. The softioc subdirectory of the startup dir contains a bash script which simplifies managing the IOC on Linux. Windows IOCs: the startup directory contains batch files to start the IOC.

Note: The startup directory can reside elsewhere.

How does support get into an IOC?

Quick overview

• configure/RELEASE:
  • Defines the locations of EPICS base and EPICS modules
  • Often includes other RELEASE files
• kmpApp/src/Makefile:
  • Specifies which database definitions (dbd) to include
  • Specifies which libraries should be included in the IOC binary
  • Specifies which local code should be included in the IOC binary
  • Libraries will not be included unless an associated dbd file is included
  • The order of the libraries is important (single-pass linker)
• iocBoot/iockmp/st.cmd:
  • The primary IOC config file that is sourced when the IOC is started
  • Name can vary slightly based on how the IOC was created

Startup script st.cmd

Legacy IOCs often have monolithic st.cmd files. Newer IOCs source more .iocsh files from EPICS modules, which makes it easier to keep an IOC's config up-to-date.

< envPaths

dbLoadDatabase("../../dbd/iockmpLinux.dbd")
iockmpLinux_registerRecordDeviceDriver(pdbbase)

### Databases are loaded
### Drivers are initialized/configured here

iocInit

### Sequence programs & autosave are started here

dbl > dbl-all.txt

date

We can break it up into 3 functional zones:

• Zone 1: before dbLoadDatabase. Usually this is where env vars and paths are defined.
• Zone 2: after dbLoadDatabase & before iocInit. Databases can only be loaded here.
• Zone 3: after iocInit. This zone is mostly empty thanks to doAfterIocInit from the std module (sequence programs and autosave get initialized here, but the calls that do the initializing appear much earlier in the startup)

Note: PVs are available during iocInit, even though the IOC is still starting up.

Autosave

autosave is a synApps module that provides seamless reboot functionality. It is mostly automatic when mkioc is used. PVs are saved while the IOC is running and loaded during iocInit. Autosaved values overwrite database defaults.

Databases with associated req files (_settings.req) are automatically added to built_*.req by autosaveBuild Multiple .sav files can be created:

• Default:
  • .sav & .savB
  • actually restored
• Periodic:
  • .sav0, .sav1, .sav2, …
  • overwritten cyclically
• Dated:
  • .sav-230109-134502
  • written once at boot time

The 1st half of common.iocsh sets up autosave: https://github.com/epics-modules/xxx/blob/master/iocBoot/iocxxx/common.iocsh#L1-L46

How to build & clean a Linux IOC

• Confirm the RHEL version
  • uname -r → 3.10.0-1160.62.1.el7.x86_64
  • cat /etc/redhat-release → Red Hat Enterprise Linux Server release 7.9
• Set EPICS_HOST_ARCH environment variable
  • bash: export EPICS_HOST_ARCH=rhel7-x86_64
  • tcsh: setenv EPICS_HOST_ARCH rhel7-x86_64
• Initiate the build in the top-level IOC directory
  • make

• Remove intermediate build dirs (for the current EPICS_HOST_ARCH )
  • make clean
• Remove all intermediate build dirs and top-level build directories
  • make distclean
• Show all make options
  • make help

Running an IOC

Use the script

Create an alias

Create an alias for the script (assuming PWD = IOC’s top-level dir)

$ alias kmp=${PWD}/iocBoot/iockmp/softioc/kmp.sh

$ kmp

Usage: kmp.sh {console|restart|run|start|caqtdm|medm|status|stop|usage}

Noteworthy arguments:

• run: Runs the IOC in the current terminal (doesn't return the command prompt). Useful for troubleshooting. Problematic for normal operation.
• start: Runs the IOC in the background (returns the command prompt) using screen or procServ
• status: Tells you if the IOC is running and refuses to start if it is

Notes:

• Use the full path to the IOC’s start script so the alias can be run from any directory (bash syntax shown).
• EPICS_HOST_ARCH needs to still be set otherwise the IOC will not start (unless the EPICS_HOST_ARCH is hard-coded in the start script)
• The ampersand is needed for synApps_6_2_1 IOCs, but it won’t be needed in the future because it was added to the start_caQtDM_xxx script.
• Bugs in the deployed copy of xxx.sh may prevent it from detecting an IOC is running.

Start caQtDM

kmp caqtdm &

Start the IOC

Starts the IOC in the background (using screen or procServ):

kmp start

IOCs are usually run in screen or procServ so that:

• the window in which they’re started can be closed
• multiple people can connect to them
• [procServ] allows other user accounts to restart an IOC
• [procServ] allows users to connect to Windows IOCs

Confirm the IOC is running

kmp status

kmp is running (pid=1281733) in a screen session (pid=1281732)

Connecting to an IOC’s shell

There are many ways to do it:

• Manually connect to an IOC running in screen from the IOC’s host

$ screen -x kmp

The -x option will connect even if someone is already connected to the screen session.

The -r option only succeeds if no one is connected to the session.

• Connect to an IOC running in screen or procServ from the IOC’s host

$ kmp console

• Connect to an IOC running in screen or procServ with logging from any host

$ /APSshare/bin/iocConsole.py iockmp

Note: iocConsole.py requires ssh access which beamline accounts don't have by default

• Manually connect to an IOC running in procServ from the IOC’s subnet

$ telnet s100bcda 53127

Disconnecting from an IOC’s shell

• Close the terminal that is connected to the screen or procServ session
  • Disconnects from both screen & procServ
  • Kills an IOC running outside of screen & procServ
• Ctrl+a, Ctrl+d
  • Disconnects from both screen & procServ
  • Kills an IOC running outside of screen & procServ
• Ctrl+a, d
  • Only disconnects from screen

IOC shell commands

help [command] - shows available commands or syntax of specified command

dbpr record_name [0-9] - displays fields and values for a given record

dbl - lists all records in an IOC

dbl record_type - lists all records of a specific type in an IOC

dbl record_type "field_list" - lists specified fields (space-separated list) of records of a specific type in an IOC

dbpf pv_name value - change a PV’s value – analogous to caput

dbgf pv_name - read a PV’s value – analogous to caget

epicsEnvShow - prints environment variables

seqShow - shows running sequence programs

seqStop - stops a running sequence program

External links

• Help formatting
• https://epics.anl.gov/
• EPICS base record reference documentation