V4 Design: Record Processing
EPICS V4: Record Processing
June 22 2005
Overview
NOTE: Preliminary thoughts.
This document describes how records are processed.
The record processing model is different than the V3 model.
The model consists of the following features:
- provides synchronous and asynchronous record processing
- provides callbacks when an asynchronous record completes processing
- provides alarms
- provides posting of modifications
The asynchronous processing model is designed so that both control and data acquisition database applications can be implemented. Data acquisition applications need a way to delay record processing until linked records complete processing. In V3 several special record types, in particular the wait record, were created to do the waiting. The V4 asynchronous processing model supports data acquisition applications without requiring special record types.
Synchronous vs Asynchronous processing
Each record instance is either synchronous, i.e. it will not block during
record processing, or it is asynchronous, i.e. it can block.
- COMMENT: Permission to block could hugely simplify the sscan record code, for example, but I think there must be some sort of back door. All the records I know of that currently wait for completion (excepting those that do it via asyn device support calls) must do it in a way that permits them still to respond to writes to their fields, so their processing can be aborted -- or, in some cases, redirected -- by a client. - TimMooney
It is the responsibility of record support and associated link or device support to declare if a record instance is asynchronous. A field in dbCommon states if the record is asynchronous.
If a record is synchronous than the record is processed via normal scan threads. For example if the record is periodically scanned than it is processed by a periodic scan thread.
- COMMENT: Several of the existing records that wait for completion also process periodically while they are waiting, so the user can get in-progress data while acquisition (which may last for many minutes) is ongoing. The scaler record does this by having device support schedule wakeups which process the record; the motor and MCA records rely on an external periodically scanned record to tell them to read and display current data. - TimMooney
A pool of threads are provided to process asynchronous records. The maximum number of threads in the pool is configurable.
A pool has an associated queue of records to process. When a process request is received it is assigned to a thread with the shortest queue, ideally one with an empty queue.
An asynchronous record is allowed to block.
The rest of this section describes additional features related to asynchronous processing.
Notification of Process Completion
The request to process an asynchronous record can specify a callback that is called when the record completes processing. It is also possible to make a process callback request over the network.
- COMMENT: The requesting record doesn't know if the record it's calling is synchronous or asynchronous, so I guess this implies the request to process any record is permitted to specify a completion callback. What happens if the called record is synchronous, and a callback routine is specified? - TimMooney
Wait for intermediate results
A record instance has an associated list of epicsEvents. Instead of blocking, link and or device support can add an epicsEvent to the list. Record support can make a call to wait for all events on the list before proceding.
As an example the calcOut record could do the following:
- call the link support for each input link. Any link support that makes an asynchrous request to obtain data adds an entry to the event list.
- after all the input link support has been called, record support waits for all the events to be signaled.
- record support does the calculation and puts the result in the value field.
- record support calls the support for the output link. If the output link makes an asynchronous call it adds a entry to the event list.
- record support waits for the event before completing record processing.
COMMENT: This is cool! The ability of an input link to trigger processing, wait for completion, and know that the data it receives are the results of completed processing would really simplify database development and open new ground for run-time programming. In V3, to achieve this effect, the record whose completion is required must drive all listeners who need completion-qualified data, and this means that it must know who those listeners are and be reconfigured when those listeners change. V3 is very awkward in this respect. Worse, V3 requires clients that want a completion-qualified value to know what record must be processed to generate that value. This new ability of V4 would allow clients merely to ask for the value, and let the record they're asking figure out how to acquire it. - TimMooney
flavors of link support
Link support for database and/or channel access links should support the following:
- get
- get value
- process first then get data
- put
- put only
- put and request processing
- put, request processing, and wait for completion
COMMENT: Two possibilities for CP links:
- In V3, a record with a CP link, or any other CA client that puts a monitor on a PV, has no way of determining whether the value it receives is the result of completed processing, or merely an intermediate posting of work in progress. Also, if a monitored record processes and the result is the same value it had before processing started, the monitoring client typically doesn't get any indication that something has happened. This is a problem if the reason the client put a monitor on the PV was to discover that the record had processed, or to trigger some processing that should only be performed on completion-qualified values. It would be nice if CA clients, including records, could say "send me all values that are the results of completed processing (whether or not they differ from previously posted values) and send me only such values."
- In V3, processing that results from a CP link (or any CA client's monitor) is outside of the putNotify execution trace. A workable kludge (the 'busy' record) permits clients (typically, SNL programs) to participate by allowing them to control the calling of recGblFwdLink() with a caput(). Whatever new completion-detection scheme is used in V4, some form of this capability will still be needed.
ENDCOMMENT - TimMooney
Alarm Processing
Are changes needed from the V3 model?
Posting Modifications
Whenever code modifies a field that is acessable outside the record it must call ????
When record support wants the modifications to be made available to code that has set monitors on fields in the record it calls ???
