Future Development Ideas

These are my notes for what I think we can/should add to V3 through an evolutionary process. The ordering is probably vaguely right, but not necessarily that accurate.

Tasks which could be done now

These items can be worked on immediately as they do not depend on any other work to have been completed, and could even be included in the next 3.14 release if appropriate.

Documentation (3.14/3.15)

Lots we can do here:

Record Reference Manual

• Copy the 3.13 Wiki version to make a 3.14 version (delete the synApps record types?)
• Update to 3.14 and beyond (generate tables automatically from DBD files, include text too?)

Application Developers Guide

• Convert into an OpenDocument Text file (or some other non-proprietary format). Machine-translating index markers is the most important part of this conversion, we don't want to lose the index.
• Write a "Building and Porting EPICS" chapter
• Restructure the AppDevGuide to make it more accessible
  • Combine the libCom and libCom/osi chapters
  • Maybe have a separate section for all callable APIs

Other Documents

• Look at all the documentation we have and see what's missing
• Data Access, other new CA-related documentation...

PV Aliases (3.14/3.15)

The ability to add aliases to existing records has already been demonstrated by Larry Hoff and is a desirable feature, although the implementation that goes into Base might be somewhat different.

• Do we need the ability to add aliases after iocInit()? There are circumstances where it would be useful, but it might not be thread-safe to allow it. Need to review database locking to answer this.
• Review any Access Security issues.
• Decide how to specify the aliases to the IOC. I would prefer to extend the DB file syntax rather than use something like Larry's addPvAlias(name, alias) command, but adding alias support to dbLoadRecords() might preclude runtime aliasing.

Priority: Medium
Libs affected: dbStatic, db.

Longer strings using char arrays (3.14/3.15)

Change dbPutField() to allow an array of DBF_CHARs to be combined in order to put a long string value into a fixed-length field that holds more than 40 characters. Reading a long string back should also be possible somehow, but may be harder to do. MEDM and EDM already support this (maybe only when the channel's native type is array of DBF_CHAR though), but it will probably require changes in most other CA clients.

Priority: Medium
Libs affected: db, dbStatic?

Named Soft Events (3.14/3.15)

Add the ability to use an event name instead of a number in the EVNT field. EVNT is currently a short, but we only allow soft event numbers to be 8 bits, so this allows us space to add a new set of event numbers for named events. The event names should not have to be declared in advance, although that does leave the possibility of typos so maybe the IOC should be told first. Will probably need changes to the Event record type and/or device support.

Priority: Low
Libs affected: dbStatic, db, rec, softDev.

Reload subroutines at runtime (3.14/3.15)

The genSub can already do this: Allow the subroutine record subroutine name to be looked up after initialization, thus permitting subroutines to be reloaded (on vxWorks anyway).

Priority: Low
Libs affected: rec.

ANSIfication of Base (3.14/3.15)

Many of the record types and probably a lot of other code (device support) inside base still uses K&R-style function declarations and definitions which need to be converted to ANSI standard C.

Priority: Low
Libs affected: rec, softDev, ...

Timestamp on Value Change (3.14/3.15)

Be able to select whether a record's timestamp should indicate when a record last processed (current behavior), or when its value last changed. Since the timestamp gets set by the record's process() routine, this is best done by adding a menu field to each record type, making the recGblGetTimeStamp() call conditional on that field and add another call to it inside the monitor() routine.

Priority: Low
Libs affected: db, rec.

Rework the devLib Interface

The devLib API is not very well implemented: Some calls are redirected through a function table while others are not; the RTEMS and vxWorks implementations assume that a VMEbus is always available; PCI and its relatives are not handled at all. The API to devLib's clients must not change, but the API to the support code in devLibOSD.c may.

Note that there is an out-of-tree implementation of the devLibOSD.c file for Linux that the CLS and Australian Synchrotron use; any replacement must be able to support a plug-in library in a similar manner. Ideally obtain and modify this implementation at the same time as changing Base.

Priority: Medium
Libs affected: libCom/osi.

Database diagnostics

When an IOC is too busy to service CA the engineers need a way to find out what records are causing the problem. Something like a dbSpy command might be helpful, which would keep a count of the records processing and give periodic reports. Would need to be designed carefully so it doesn't eat RAM or slow down the machine too much.

Other diagnostics could also be added: dbProcess rate, ...

Priority: Medium
Libs affected: db, dbStatic.

Extended Device Support

Asyn's device support layers needs to be converted to Extended support, so their hardware addresses can be changed at runtime. The same applies to many other device support, but Asyn is probably the largest low-hanging fruit here.

Priority: Medium
Libs affected: (asyn).

Data Access and JSON

Replace Jeff's one-directional PropertyCatalogToXML implementation with an equivalent that is bidirectional and uses JSON.

Priority: Low
Libs affected: (dataAccess).

Automated Testing (3.14/3.15)

Convert mrkSoftTest so tests run automatically. If it's easy to run tests they will be run more often.

Priority: Low
Libs affected: (mrkSoftTest).

Tasks which are already under way

These tasks are already being worked on but may have stalled. They might be hard for someone else to take over at this stage.

Merge Redundancy hooks

Develop standard collaborative thread-management APIs for epicsThread, and insert these and other hooks that DESY needs for redundant control of remote I/O, allowing hot-spare backup IOCs. Patches for this have been posted to core-talk and are being discussed and revised.

Priority: R3.14.10
Libs affected: libCom, db, rsrv

Time Infrastructure Upgrade

Peter Dennison (Diamond) and Babak Kalantari (PSI) have been merging the SNS General Time with the Event System code. When finished the result needs to be reviewed and merged into Base.

Priority: R3.14.10
Libs affected: libCom/osi.

3.15 Cleanup

3.15 already includes the Up change to the build system; the ramifications of this change have not been followed up completely though:

• Provide a perl script for converting 3.14 apps to 3.15
• Bring over latest templates from 3.14
• Implications for building extensions not worked out
• makeBaseExt.pl is probably no longer needed
• Install local RULES_BUILD files into a top level directory
  • Use this dir instead of the configure/O.arch directories so that a make clean doesn't remove them...
• Change script installation rules (see ctlsys)

Compiling DBD Files (3.15)

A Perl implementation of the DBD file parser exists but is incomplete, and I'm hoping to revive this development effort to convert all DBD file information into C tables that get compiled into the IOC executable.

With this parser being written in Perl, it will become easier to extend the new facilities that we implement, such as adding new field types (see Variable Length Strings below).

Libs affected: dbStatic, +dbHost, db.

CA Event Rate Limiting (3.15)

A CA client should be able to specify a maximum event rate for each channel subscription it creates (and possibly be able to change that rate without re-subscribing).

Libs affected: ca, cas (rsrv?).

CA Event Filtering (3.15++)

LANSCE's Time and Flavored Data requirements. It might be wise to do the Unbundling task before merging these changes.

Jeff's plan is to be able to attach an expression to a channel subscription, which is evaluated by the client's server thread to check each monitor event to determine whether to send it to the client or not. This could use an enhanced CALC expression parser and evaluation routine to do the expression, but we need a new parser since the expression can include PV names, and the calcPerform() API may have to be enhanced to match.

Libs affected: libCom/calc, ca, cas, dbStatic?, +da, db.

EPICS IOC IDE

Janet is tasked with taking Ken Evans' Eclipse IDE for EPICS, making it easy to add new record types and/or device support to an IOC from externally built support modules. This will probably require adding an XML file to all support modules that describes what they provide and how to integrate them into the IOC application.

Things which can't be done yet

These are tasks are waiting on other people or for other tasks to be completed before they can be worked on.

IOCSH Command Registration (3.15)

Add DBD file syntax to convert DBD file information into a C header file which declares the command functions (similar to how we do record types in the current code-base). This header would then be included by the C file that defines those functions, thus the C compiler would do the work of ensuring that the routine implemented actually matches the command that will call it.

The command information in the DBD file will also be useful for future versions of the IRMIS parser for st.cmd scripts, where we need to be able to tie the device support configuration commands into IRMIS' need to map hardware devices to record addresses.

Libs affected: dbHost, all libs Register code.

Array Field Support (3.16++)

Make it much easier to add array fields to record types without having to do as much coding in the record support. See Ben Franksen email

Libs affected: dbHost, dbStatic, db, rec.

Unbundling (3.15/++)

Split Base into three or more independent modules. Each module would have its own version number, and the modules can therefore each have an independent existance, although there would obviously be specific dependencies associated with particular versions.

If we adopt this, we should bundle working subsets of modules together and release the complete bundle as a tarfile.

I see the unbundled modules as follows:

Core

Depends on: Nothing

These are the common parts used by everything:

• Build system
• libCom & OSI layer
  • Basic registry routines
  • IOC shell
• Antelope
• Flex
• makeApp tool

Channel Access Client

Depends on: Core

Jeff Hill's responsibility.

• CA-specific build rules (probably none)
• CA client library
• CA Tools
• CA application makeApp templates

Channel Access Server

Depends on: Core, IOC

Jeff Hill's responsibility.

• CAS-specific build rules (probably none)
• GDD
• The CAS server tool
• RSRV (the old CA server code)
• CAS application makeApp templates

IOC

Depends on: Core, CAC, CAS

The rest is code that runs on the IOC.

• IOC-specific build rules
• Static and runtime database access
• Runtime database
• Standard record types
• Soft device support
• The registry (device, driver, record, function)
• IOC application makeApp templates

Variable Length Strings (3.15++)

3.15 already includes the BaseString C++ classes in libCom, and I intend to create a set of C wrapper routines for the base class operations (started, currently incomplete). This can form the basis of a new variable length string type for use in the database in addition to DBF_STRING.

Libs affected: dbHost, dbStatic, db, ...

Alternate DB File Formats (3.15++)

There are calls to replace the existing DB file format with an XML-based format. Currently the DB file parser is too highly integrated into the process of record creation to be able to add a second format, but it would be feasible to separate these two and hence allow an expat-based or similar parser to be written. This would probably require a close analysis and upgrade to the locking arrangements of the database.

Need to coordinate with VDCT folks, etc.

Libs affected: dbHost?, dbStatic.

New Record API

These requests all need a significant change to the record support API. They are also fairly closely related, so should be used as input into any new API that gets devised.

Asynchronous Gets (3.16++)

From Tim Mooney: It would be great if an input link could initiate processing, and get a value that is known to be the result of that processing, even if records to be processed are asynchronous. Currently, a record that needs completion-qualified data must know how to trigger the processing that generates that data.

From Ben Franksen: Implemented an asyncronous dbGetLinkCallback() routine. To do this properly though needs an upgrade to the record support API, but maybe could be shoe-horned into the the existing API by adding a dbGetLinkCallback() with a private callback pointer, and we'd require new record types to use this. What does this do to the device support interface though? Implications need study.

Blocking Puts (3.16++)

From Tim Mooney: It would be nice to have a link attribute that could put, request processing, and wait for completion. We have this capability now, in dbCaPutCallback(), but it's selectable only at DCT time.

The idea is for a version of the SEQ record where rather than a fixed delay, the second link would wait for the completion callback from the first link. This should be possible today with a special record type, but Tim suggests it should be possible with any link field of any record type.

Multiple Master problem (3.16++)

From Jeff: Changes to output values in a PLC or similar device initiated outside of the output record. This could be resolved adding a new routine to the RSET that the device support calls back when it has a new raw ouptut value to back-propagate to the VAL field. Currently done by some devices that set PACT and call the record's process() routine to use its second-half processing only.

Records Can't Block (3.16++)

From Tim Mooney: I think almost half the code in the sscan record is devoted to bookmarking. In the middle of some algorithm, the record needs to wait for external events; the only way back in is process(), and so the record has to have a complicated arrangement of switches that get it back into the algorithm it was executing, with enough state information to decide what to do next. At the same time, the record must check to see if it's now processing because of some out-of-band request from the user. Normal and exceptional processing must, therefore, be mixed together in the switch arrangement, and adding new behaviors to the record becomes a very hairy undertaking.

But what the record would really like to do is just wait with its context intact for events, and allow special() to release it from the wait if the user has requested something that requires this. I guess this means the record should include author-defined event types in the wait mask, and special() should be able to generate author-defined events.

Online Record Creation (3.16++)

With good database locking, it might be feasible to look at the ability to add new record instances at runtime, and maybe delete as well. This would probably need a change to the record interface to initialize the new instances.

Dynamically Loaded Modules (3.16++)

After converting the DBD load process into table registration, it should become easier to allow support modules (containing record types, device support and other drivers) to be discovered and loaded dynamically. Unloading is a very different matter though, we don't have the shutdown facilities to allow that.