EPICS networking - Revision history

Tguruswamy: /* Multiple paths between client and server */ adjust note about pva localhost broadcast

2024-02-27T17:42:03Z

Multiple paths between client and server: adjust note about pva localhost broadcast

← Older revision		Revision as of 17:42, 27 February 2024
Line 72:		Line 72:

	EPICS_CA_ADDR_LIST=127.255.255.255		EPICS_CA_ADDR_LIST=127.255.255.255
	EPICS_PVA_ADDR_LIST=127.~~255~~.~~255~~.~~255~~		EPICS_PVA_ADDR_LIST=127.0.0.1
	EPICS_CA_AUTO_ADDR_LIST=no		EPICS_CA_AUTO_ADDR_LIST=no
	EPICS_PVA_AUTO_ADDR_LIST=no		EPICS_PVA_AUTO_ADDR_LIST=no
Line 78:		Line 78:
	The difference between using only the loopback address (127.0.0.1) and using the loopback broadcast address (127.255.255.255) is that the broadcast address supports communicating with multiple IOCs on the same host on Linux. See the [https://docs.epics-controls.org/en/latest/sys-admin/channel-access-reach-multiple-soft-iocs-linux.html EPICS documentation] for more details and alternative solutions.		The difference between using only the loopback address (127.0.0.1) and using the loopback broadcast address (127.255.255.255) is that the broadcast address supports communicating with multiple IOCs on the same host on Linux. See the [https://docs.epics-controls.org/en/latest/sys-admin/channel-access-reach-multiple-soft-iocs-linux.html EPICS documentation] for more details and alternative solutions.

	Note: ~~Limiting traffic~~ to ~~127.0.0.1/~~127.255.255.255 ~~may~~ not work for the EPICSv7 built-in pvaServer due to an issue in the implementation~~, in which case try a different local IP~~. See TechTalk: [https://epics.anl.gov/tech-talk/2021/msg00806.php] [https://epics.anl.gov/tech-talk/2021/msg00807.php]		Note: PVA is different to CA in this example because using the broadcast address 127.255.255.255 does not work for the EPICSv7 built-in pvaServer due to an issue in the implementation. See TechTalk: [https://epics.anl.gov/tech-talk/2021/msg00806.php] [https://epics.anl.gov/tech-talk/2021/msg00807.php]

	== References ==		== References ==

Tguruswamy: /* Potential issues */ fix techtalk link

2024-02-26T21:58:00Z

Potential issues: fix techtalk link

← Older revision		Revision as of 21:58, 26 February 2024
Line 78:		Line 78:
	The difference between using only the loopback address (127.0.0.1) and using the loopback broadcast address (127.255.255.255) is that the broadcast address supports communicating with multiple IOCs on the same host on Linux. See the [https://docs.epics-controls.org/en/latest/sys-admin/channel-access-reach-multiple-soft-iocs-linux.html EPICS documentation] for more details and alternative solutions.		The difference between using only the loopback address (127.0.0.1) and using the loopback broadcast address (127.255.255.255) is that the broadcast address supports communicating with multiple IOCs on the same host on Linux. See the [https://docs.epics-controls.org/en/latest/sys-admin/channel-access-reach-multiple-soft-iocs-linux.html EPICS documentation] for more details and alternative solutions.

	Note: Limiting traffic to 127.0.0.1/127.255.255.255 may not work for the EPICSv7 built-in pvaServer due to an issue in the implementation, in which case try a different local IP. See TechTalk: [https://epics.anl.gov/tech-talk/2021/msg00806.php] [https://epics.anl.gov/tech-talk/2021/~~msg00806~~.php]		Note: Limiting traffic to 127.0.0.1/127.255.255.255 may not work for the EPICSv7 built-in pvaServer due to an issue in the implementation, in which case try a different local IP. See TechTalk: [https://epics.anl.gov/tech-talk/2021/msg00806.php] [https://epics.anl.gov/tech-talk/2021/msg00807.php]

	== References ==		== References ==

Tguruswamy: /* The search request */ reword

2024-02-21T21:47:49Z

The search request: reword

← Older revision		Revision as of 21:47, 21 February 2024
Line 26:		Line 26:
	# If the environment variable <code>EPICS_{CA,PVA}_AUTO_ADDR_LIST=yes</code>, then add all broadcast IP addresses for all available network interfaces to the list.		# If the environment variable <code>EPICS_{CA,PVA}_AUTO_ADDR_LIST=yes</code>, then add all broadcast IP addresses for all available network interfaces to the list.
	#* "yes" is the default value if unset.		#* "yes" is the default value if unset.
	#* ~~To skip this step, set~~ <code>EPICS_{CA,PVA}_AUTO_ADDR_LIST=no</code>.		#* If <code>EPICS_{CA,PVA}_AUTO_ADDR_LIST=no</code>, skip this step.
	# Add the contents of the environment variable <code>EPICS_{CA,PVA}_ADDR_LIST</code> to the list.		# Add the contents of the environment variable <code>EPICS_{CA,PVA}_ADDR_LIST</code> to the list.
	#* This should be a space-separated list of IP addresses or hostnames.		#* This should be a space-separated list of IP addresses or hostnames.

Tguruswamy: /* Connection Flow */ mention ip search list

2024-02-21T21:37:12Z

Connection Flow: mention ip search list

← Older revision		Revision as of 21:37, 21 February 2024
Line 9:		Line 9:
	== Connection Flow ==		== Connection Flow ==

	* A client emits a request for particular PV names, e.g. ''13SIM1:cam1:Image''.		* A client emits a request for particular PV names, e.g. ''13SIM1:cam1:Image'' to a specific "search list" of IP addresses (which may be unicast or broadcast addresses).
	* Any IOC which receives the request and provides any PV name requested will respond to the client.		* Any IOC which receives the request and provides any PV name requested will respond to the client.
	* The client and Server now know each other's IP addresses.		* The client and Server now know each other's IP addresses.

Tguruswamy: /* Potential issues */ add additional techtalk link

2024-01-09T23:38:42Z

Potential issues: add additional techtalk link

← Older revision		Revision as of 23:38, 9 January 2024
Line 78:		Line 78:
	The difference between using only the loopback address (127.0.0.1) and using the loopback broadcast address (127.255.255.255) is that the broadcast address supports communicating with multiple IOCs on the same host on Linux. See the [https://docs.epics-controls.org/en/latest/sys-admin/channel-access-reach-multiple-soft-iocs-linux.html EPICS documentation] for more details and alternative solutions.		The difference between using only the loopback address (127.0.0.1) and using the loopback broadcast address (127.255.255.255) is that the broadcast address supports communicating with multiple IOCs on the same host on Linux. See the [https://docs.epics-controls.org/en/latest/sys-admin/channel-access-reach-multiple-soft-iocs-linux.html EPICS documentation] for more details and alternative solutions.

	Note: Limiting traffic to 127.0.0.1/127.255.255.255 may not work for the EPICSv7 built-in pvaServer due to an issue in the implementation, in which case try a different local IP. https://epics.anl.gov/tech-talk/2021/msg00806.php		Note: Limiting traffic to 127.0.0.1/127.255.255.255 may not work for the EPICSv7 built-in pvaServer due to an issue in the implementation, in which case try a different local IP. See TechTalk: [https://epics.anl.gov/tech-talk/2021/msg00806.php] [https://epics.anl.gov/tech-talk/2021/msg00806.php]

	== References ==		== References ==

Tguruswamy: /* Potential issues */ issue with pvaServer and localhost

2024-01-09T23:37:03Z

Potential issues: issue with pvaServer and localhost

← Older revision		Revision as of 23:37, 9 January 2024
Line 77:		Line 77:

	The difference between using only the loopback address (127.0.0.1) and using the loopback broadcast address (127.255.255.255) is that the broadcast address supports communicating with multiple IOCs on the same host on Linux. See the [https://docs.epics-controls.org/en/latest/sys-admin/channel-access-reach-multiple-soft-iocs-linux.html EPICS documentation] for more details and alternative solutions.		The difference between using only the loopback address (127.0.0.1) and using the loopback broadcast address (127.255.255.255) is that the broadcast address supports communicating with multiple IOCs on the same host on Linux. See the [https://docs.epics-controls.org/en/latest/sys-admin/channel-access-reach-multiple-soft-iocs-linux.html EPICS documentation] for more details and alternative solutions.

			Note: Limiting traffic to 127.0.0.1/127.255.255.255 may not work for the EPICSv7 built-in pvaServer due to an issue in the implementation, in which case try a different local IP. https://epics.anl.gov/tech-talk/2021/msg00806.php

	== References ==		== References ==

Tguruswamy: Writeup of BDP presentation

2023-12-23T02:40:52Z

Writeup of BDP presentation

New page

= EPICS networking =

For basic terminology, see [[EPICS 101]].

An EPICS IOC is an application which serves EPICS clients (command-line programs like caget; applications using p4p, pyepics, pvapy, or other libraries; GUI display managers like MEDM or caQtDM) process variables over the network. They do so using either the Channel Access (CA) or PVAccess (PVA) network protocols.

By default, an EPICS IOC with CA and/or PVA support enabled will listen for clients on '''all available network interfaces.'''

== Connection Flow ==

* A client emits a request for particular PV names, e.g. ''13SIM1:cam1:Image''.
* Any IOC which receives the request and provides any PV name requested will respond to the client.
* The client and Server now know each other's IP addresses.
* A one-to-one communications channel between client and server is established.
* The opened Channel is used for either a one-off READ or WRITE, or a SUBSCRIPTION (monitor) to receive an update every time the PV value changes.

This flow is more or less the same for the CA or PVA protocols, and involves a mix of TCP/IP and UDP/IP transport with varying port numbers. Many client libraries (e.g. pyepics) will cache connections, or convert repeated READs into SUBSCRIPTIONS behind-the-scenes to save resources.

This system ensures that a newly-started IOC is immediately accessible to all clients without needing to register with a central server, and the only information you need to connect to a PV is its full name.

== The search request ==

EPICS clients follow a simple two-step process to construct the destination of a PV search request. This is performed separately for CA and PVA protocols (substitute ''either'' <code>CA</code> or <code>PVA</code> into the names of the environment variables described below).

# Start with an empty list.
# If the environment variable <code>EPICS_{CA,PVA}_AUTO_ADDR_LIST=yes</code>, then add all broadcast IP addresses for all available network interfaces to the list.
#* "yes" is the default value if unset.
#* To skip this step, set <code>EPICS_{CA,PVA}_AUTO_ADDR_LIST=no</code>.
# Add the contents of the environment variable <code>EPICS_{CA,PVA}_ADDR_LIST</code> to the list.
#* This should be a space-separated list of IP addresses or hostnames.
# Send the PV search request to every IP or host in the list.

Environment variables should be set prior to starting the client program to be effective, and may need to be "exported" to persist (depending on your shell). Example in bash:

export EPICS_CA_AUTO_ADDR_LIST=no EPICS_CA_ADDR_LIST=s1pvgate.xray.aps.anl.gov
caget 1BM:ExampleVariable

== Potential issues ==
=== Crossing subnet boundaries ===

Since by default only broadcast addresses are in the search request destination list, and broadcast packets are not routed between subnets, without extra configuration you will only be able to find PVs on the same subnet as your client.

;Solution 1: add the IP address of the IOC server directly to <code>EPICS_{CA,PVA}_ADDR_LIST</code>.
:This works across subnet boundaries as long as the required ports are available (no firewalls), but doesn't scale well if there are many IOCs you want to connect to.

;Solution 2: set up a PV Gateway server in the IOC subnet, and add the PV gateway IP address
:A [https://github.com/epics-extensions/ca-gateway PV gateway] is a dedicated server program which relays incoming requests on to the rest of the subnet. It can also apply security rules to the traffic it is forwarding, e.g. it can force a PV to be read-only. The PV gateway hostname or IP address must be added to the relevant <code>ADDR_LIST</code> for all clients.

; Solution 3: set up a PV nameserver.
:Similar to DNS, the [https://github.com/epics-extensions/ca-nameserver nameserver] is a server program that takes over responding to all client name search requests. In this setup, the nameserver must be added to the <code>ADDR_LIST</code> for all clients.

=== Multiple paths between client and server ===

An IOC will reply to '''all''' requests for PVs it servers, even multiple ones from the same client.

Therefore if there are multiple network paths from client to server, for example because the client and/or server have multiple network interfaces, the server may see multiple requests from the client and the client will get multiple responses, each with a different IP.

The same situation can occur if multiple IOCs are serving a PV with the exact same name. The client will usually pick the first response it receives and print a warning to the user, for example:

12-12-2023 01:36:01 Channel Access Exception Channel: "bdpSim0:cam1:Model_RBV", Connecting to: beryl-gpm.xray.aps.anl.gov:5064, Ignored: 10.0.0.1:5064 on ? (op=5 data_type=DBR_invalid count=0)

Which server IP is selected is essentially random each time a search request completes and a connection is created, and there is no guarantee it is the optimal or fastest path from client to server.

;Solution: limit EPICS traffic to only one interface and one network path.
:On the client, set <code>EPICS_{CA,PVA}_AUTO_ADDR_LIST=no</code> and manually set <code>EPICS_{CA,PVA}_ADDR_LIST</code> to the single broadcast address on the network interface you want to use.

It is also possible to set <code>EPICS_{CA,PVA}S_INTF_ADDR_LIST</code> on the server to limit the addresses the IOC listens on, but this must be done ''before'' starting the IOC.

The most common variant of this situation is when the client and server are on the same computer. In this case, all the available network interfaces (including the loopback interface ''lo'') present a valid path from client to server.

To limit all traffic to only travel over the loopback interface (which will be the fastest), before running your client set:

EPICS_CA_ADDR_LIST=127.255.255.255
EPICS_PVA_ADDR_LIST=127.255.255.255
EPICS_CA_AUTO_ADDR_LIST=no
EPICS_PVA_AUTO_ADDR_LIST=no

The difference between using only the loopback address (127.0.0.1) and using the loopback broadcast address (127.255.255.255) is that the broadcast address supports communicating with multiple IOCs on the same host on Linux. See the [https://docs.epics-controls.org/en/latest/sys-admin/channel-access-reach-multiple-soft-iocs-linux.html EPICS documentation] for more details and alternative solutions.

== References ==

* [https://epics.anl.gov/base/R3-14/12-docs/CAref.html EPICS Channel Access Reference Manual (v3.14)]
* [https://mdavidsaver.github.io/pvxs/netconfig.html PVXS (PVA) Network Configuration Information]