V4 Design: epicsTypes

EPICS: epicsTypes

May 31 2005

Overview

This document describes the C++ definitions for storing data that can be accessed without pre-complied code, i.e. it has self describing features.

epicsTypes can be used by any code that stores data accessable via V4 Channel Access. Some examples are:

1. IOC records - Most data accessable from outside record support is stored as an epicsType.
2. Channel Access Gateway - Can be used to store and transfer data.
3. Channel Access Clients - Can be used to get/receive data from CA server.

epicsTypes is not a replacement for dataAccess. Standard support can be provided to access, via dataAccess, epicsType data. For example standard support can be provided to move IOC record data between record instances and a Channel Access server.

epicsType is an enum that defines the following:

• epicsUnknownT - Type is unknown
• epicsBooleanT, ..., epicsFloat64T - C++ primitive types
• epicsStringT - EpicsString which contains a UTF-8 Encoded Character String
• epicsArrayT - EpicsArray which describes type and storage for a one dim array. The element type can be any epicsType.
• epicsStructT - EpicsStruct which describes and provides storage for a set of fields each of some epicsType
• epicsMDArrayT - EpicsMDArray which describes type and provides storage for a multidimensional array. The element type can be a primitive type or epicsStringT.

The actual types associated with the epicsTypes are:

    typedef bool               epicsBoolean;
    typedef char               epicsOctet;
    typedef short              epicsInt16;
    typedef unsigned short     epicsUInt16;
    typedef int                epicsInt32;
    typedef unsigned int       epicsUInt32;
    typedef long long          epicsInt64;
    typedef unsigned long long epicsUInt64;
    typedef float              epicsFloat32;
    typedef double             epicsFloat64;

    /*EpicsString holds UTF-8 characters*/
    class EpicsString  {
    public:
        EpicsUTF_8Buffer    *pbuffer;
    };
    class EpicsArray {
    public:
        epicsType        type;
        EpicsArrayBuffer *pbuffer;
    };
    class EpicsStruct{
    public:
        EpicsStructDef *pstructDef;
        void    *pstorage;
    };
    class EpicsMDArray {
    public:
       epicsType type;
       EpicsMDArrayDescription *pdescription;
       EpicsMDArrayBuffer *pbuffer;
    };

The types are :

• epicsBoolean, ... epicsFloat64 each has an associated C++ fundamental type. An instance of a primitive type has only storage associated with it.
• epicsOctet is an 8 bit byte that just contains data. It is not used as an integer type.
• EpicsString is a UTF-8 encoded character string. Multiple string buffer management implementation are supported. Both contiguous and segmented buffers are available.
• EpicsArray is an array of any type except unknown. Multiple array buffer management implementations are supported. Both contiguous and segmented implementations are available.
• EpicsStruct is a container with fields each of which is any epicsType
• EpicsMDArray describes a multi-dimensional array of a primitive or string type.

epicsTypes

epicsTypes.h contains the following:

    /* The following may require OSD definitions*/
    typedef bool               epicsBoolean;
    typedef char               epicsOctet;
    typedef short              epicsInt16;
    typedef unsigned short     epicsUInt16;
    typedef int                epicsInt32;
    typedef unsigned int       epicsUInt32;
    typedef long long          epicsInt64;
    typedef unsigned long long epicsUInt64;
    typedef float              epicsFloat32;
    typedef double             epicsFloat64;
   
    enum epicsType {
        epicsUnknownT,
        epicsBooleanT,     // epicsBoolean
        epicsOctetT,       // epicsOctet
        epicsInt16T,       // epicsInt16
        epicsUInt16T,      // epicsUInt16
        epicsInt32T,       // epicsInt32
        epicsUInt32T,      // epicsUInt32
        epicsInt64T,       // epicsInt64
        epicsUInt64T,      // epicsUInt64
        epicsFloat32T,     // epicsFloat32
        epicsFloat64T,     // epicsFloat64
        epicsStringT,      // EpicsString
        epicsArrayT,       // EpicsArray
        epicsStructT,      // EpicsStruct
        epicsMDArrayT      // EpicsMDArray
    };

epicsTypes provides classes for describing data that can be introspected and can be passed between different platforms. All data that is sent to or received from EPICS records will be composed of epicsTypes.

The types epicsBooleanT, ..., epicsFloat64T all map to a C++ standard type. It may be necessary to provide operating system dependent definitions for some of the types. For example on some architectures a epicsInt64 may have to be defined as a long rather than a long long.

The unsigned integer types should only be used for data that represents masks. If they are used for representing large numbers, e.g. an epicsUInt16 that is greater than 32767, then Java clients/servers will not work properly.

Java Notes

• epicsOctet is a Java byte
• Java does not support unsigned integers. It should, however, accept each unsigned type as the corresponding signed type.

epicsUnknownT is provided in case something expected to produce an epicsType fails.

The types epicsStringT, epicsArrayT, epicsStructT, and epicsMDArrayT are designed so that the data they contain can be described, introspected, and passed over a network. Each is described in the following sections.

Global comments about non-primitive types

Method Expose

The class definitions for non-primitive types, i.e. epicsStringT, epicsArrayT, epicsStructT, and epicsMDArrayT, all provide a method expose which returns the address of data. In order to make expose safe, some rules must be established. The application that uses epicsTypes must set the rules since a generic solution may not be desirable.

As an example of rules, a lock can be associated with each object that supports expose. Code can only call expose and access the exposed data while it holds the lock.

The rules are not specified by epicsTypes since they are application dependent,

Buffer Implementations

The class definitions for epicsStringT, epicsArrayT, and epicsMDArrayT, all manage storage for the associated type. Multiple implementations are provided. Implementations are provided for contiguous and segmented storage.

The contiguous implementation is appropriate for a string that is allocated at initialization and is only rarely modified. At least two contiguous implementations are provided: mutable and non-mutable. The mutable implementation allows the storage to be modified. The non-mutable implementation does not allow the storage to be changed after it has been given an initial value.

The segmented implementation is appropriate for storage that is frequently modified or is transient, e.g. data being transfered over the network.

Multiple segmented implementation may be necessary: An implementation is required that is appropriate for fields of ioc databases. Other implementations may be required for transfering data to/from network buffers.

epicsString

epicsString.h contains the following:

    class EpicsUTF_8Buffer;
    
    /*EpicsString holds UTF-8 characters*/
    class EpicsString  {
    public:
        EpicsUTF_8Buffer    *pbuffer;
    };

    class EpicsUTF_8Buffer {
    public:
        virtual epicsInt32 allocate(epicsInt32 capacity) = 0;
        virtual void release(bool onlyStorage) = 0;
        virtual epicsInt32 capacity() = 0;
        virtual epicsInt32 limit() = 0;
        virtual void limit(epicsInt32 newLimit) = 0;
        virtual epicsInt32 get(epicsOctet *pto,
                               epicsInt32 offset, epicsInt32 limit) = 0;
        virtual epicsInt32 put(const epicsOctet *pfrom,
                               epicsInt32 offset, epicsInt32 limit) = 0;
        virtual bool isEqual(const EpicsUTF_8Buffer *pbuffer) = 0;
        virtual bool isEqual(const epicsOctet *pstring, epicsInt32 len) = 0;
        virtual void expose(epicsInt32 offset, epicsInt32 limitRequest,
                            epicsOctet *pdata, epicsInt32 *limit);
        virtual epicsUint32 hash(epicsInt16 nBitsHashIndex) = 0;
    };

    typedef EpicsUTF_8Buffer *(EpicsUTF_8BufferAllocate)();
    class EpicsUTF_8BufferFactory {
    public:
        static epicsUint16 typeToTypeID(const char *type);
        static EpicsUTF_8Buffer *allocate(epicsUint16 typeId);
        static void register(const char *type,
                         EpicsUTF_8BufferAllocate allocater);
    };

An EpicsString is an instance of an epicsStringT. It contains a UTF-8 encoded string, i.e. multiple bytes may be needed to store a single character. As long as it does not assume that each character is a single byte, most code can just ignore the fact that the string is UTF-8 encoded. If the local computer has been internationalized for the particular UTF-8 encoding then the string can be printed via the printf family of methods.

EpicsString has the following fields:

pbuffer

The address of a EpicsUTF_8Buffer, which is a class that manages the string storage.

EpicsUTF_8Buffer

A string buffer is a container for a UTF_8 encoded character string.

Multiple string buffer implementations are available. At least the following are available:

• contiguous - The string is stored in a contiguous set of bytes.
  • mutable - The string value can be modified after an initial value is assigned.
  • non-mutable - After the original assignment, the string becomes read only.
• segmented - The string is stored in chunks. This form should be used for strings that are constantly modified.

A string buffer can only be accessed via interface EpicsUTF_8Buffer. In addition to holding storage for a string, a string buffer keeps the following information.

capacity

The number of octets allocated. Note that this is NOT the number of characters since a UTF-8 character may require several bytes.

limit

The current size, i.e. the index of the first octet that can not hold data. Data can not be read from or written into a buffer beyond limit. When data is being written to a buffer, limit is normally equal to capacity. When data is being read from a buffer, limit is normally less than capacity and indicates the end of valid data.

Implementation Note Code must always set limit so that it is at the end of a complete character.

EpicsUTF_8Buffer has the following methods:

allocate

This allocates space for up to capacity octets. The number of octets allocated is returned. An implementation attempts to allocate the requested capacity but some implementations, e.g. network buffers, may impose a maximum size. If capacity is not zero when this is called and new storage is allocated then the old storage is freed or reused and the octets spanned by position, limit appear in the newly allocated storage.

release

Storage for the string is released. If onlyStorage is true then storage for the buffer is also released.

capacity

returns the capacity

limit

Two methods are available, one to get the current limit and one to set the limit.

get

copies characters to pto and returns the number of octets transfered.

put

copies characters from pfrom, puts them into the buffer, and returns the number of octets transfered. limit is set equal to the number of characters written.

isEqual(EpicsUTF_8Buffer *)

Compares the string stored in the buffer with a string stored in a different buffer. This is normally called by code that uses an EpicsUTF_8Buffer.

isEqual(epicsOctet *pstring, epicsInt32 len)

Compares the string stored in the buffer with a string supplied by the caller. This is normally called by EpicsUTF_8Buffer itself to compare it's string with the string stored in another buffer.

expose

A request to return the address of actual bytes of storage. Since a buffer implementation may used segmented memory the number of bytes exposed may be less than the amount requested.

hash

implement a hash on the octets stored in the buffer.

EpicsUTF_8BufferFactory

This is a class for allocating an EpicsUTF_8Buffer and also for registering EpicsUTF_8Buffer implementations. Note that allocating a buffer is different than allocating storage for the data that is stored in the buffer.

epicsArray

epicsArray.h contains the following:

    class EpicsArrayBuffer;

    class EpicsArray {
    public:
        epicsType        type; 
        EpicsArrayBuffer *pbuffer;
    };

    class EpicsArrayBuffer {
    public:
        virtual epicsUInt32 allocate(
                    epicsUInt32 capacity,epicsUint16 elementSize) = 0;
        virtual void release(bool onlyStorage) = 0;
        virtual epicsUInt32 capacity() = 0;
        virtual epicsUInt32 elementSize() = 0;
        virtual epicsUInt32 limit() = 0;
        virtual void limit(epicsUInt32 newLimit) = 0;
        virtual epicsUInt32 position() = 0;
        virtual void position(epicsUInt32 newPosition) = 0;
        virtual epicsInt32 get(void *pto,
                               epicsInt32 offset, epicsInt32 limit) = 0;
        virtual epicsInt32 put(const void *pfrom,
                               epicsInt32 offset, epicsInt32 limit) = 0;
        virtual void expose(epicsUInt32 offset, epicsUInt32 limitRequest,
                                   void *pdata, epicsUInt32 *limit);
    }

    typedef EpicsArrayBuffer *(EpicsArrayBufferAllocate)();
    class EpicsArrayBufferFactory {
    public:
        static epicsUint16 typeToTypeID(const char *type);
        static EpicsArrayBuffer *allocate(epicsUint16 typeId);
        static void register(const char *type,
                         EpicsArrayBufferAllocate allocater);
    };
    // type : At least "Contiguous" and "Segmented" are implemented

An EpicsArray can contain data of any epicsType excerpt unknown. It has the following fields:

type

Any epicsType except unknown

pbuffer

The address of an EpicsArrayBuffer that provides access to the array.

EpicsArrayBuffer

Multiple array buffer implementations are available. An implementation can provide the following semantics:

• contiguous - The array is stored in a contiguous set of bytes.
  • mutable - The array can be modified after an initial value is assigned.
  • non-mutable - After the original assignment, the array becomes read only.
• segmented - The array is stored in chunks. This form should be used for strings that are constantly modified.

Since the fields can be of type epicsArrayT arrays of arrays are supported. This is analogous to multidimensional arrays in Java.

An array buffer can only be accessed via interface EpicsArrayBuffer. In addition to holding storage for the array an array buffer keeps the following information.

capacity

The number of elements for which storage is allocated.

element size

the number of bytes for each element

limit

The current size, i.e. the index of the first element that can not hold data. Data can not be read from or written into a array beyond limit. When data is being written to a buffer limit is normally equal to capacity. When data is being read from a buffer limit is normally less than capacity and indicates the end of valid data.

position

The index of the first element that has data.

EpicsArrayBuffer has the following methods:

allocate

This allocates space for up to capacity elements. The number of elements allocated is returned. An implementation attempts to allocate the requested capacity but some implemenations, e.g. network buffers, may impose a maximum size. If capacity is not zero when this is called and new storage is allocated then the old storage is freed or reused and the elements spanned by position, limit appear in the newly allocated storage.

release

Storage for the array is released. If onlyStorage is true then the storage for the buffer itself is also released.

capacity

return the capacity

limit

Two methods are available, one to get the current limit and one to set the limit.

position

Two methods are available, one to get the current position and one to set the position.

get

copies characters to pto and returns the number of octets transfered.

put

copies characters from pfrom, puts them into the buffer, and returns the number of octets transfered.

expose

A request to return the address of actual storage. Since a buffer implementation may used segmented memory the amount of storage exposed may be less than the amount requested.

EpicsArrayBufferFactory

This is a class for allocating an EpicsArrayBuffer and also for registering EpicsArrayBuffer implementations. Note that allocating a buffer is different than allocating storage for the data that is stored in the buffer.

epicsStruct

epicsStruct.h contains the following:

    class EpicsStructDef;
    class EpicsStructLifetime;

    class EpicsStruct{
    public:
        EpicsStructDef *pstructDef;
        void    *pstorage;
    };

    class EpicsStructField {
    public:
        EpicsString name;
        epicsType   type;
    };
   
    class EpicsStructDef{
    public:
        EpicsString  name;
        EpicsStructLifetime *plifetime;
        epicsInt16     nfields;
        EpicsStructField *pfield[]; // ptr to array of ptr to EpicsStructField
    };

    class EpicsStructLifetime {
    public:
        virtual void allocate(EpicsStruct *pstruct) = 0;
        virtual void destroy(EpicsStruct *pstruct) = 0;
        virtual void *exposeField(EpicsStruct *pstruct, epicsInt16 index) = 0;
    };

An epicsStruct is a container with fields each of which is any epicsType including unknown.

The following classes are involved with an epicsStruct:

• EpicsStruct - An instance of an epicsStruct
• EpicsStructDef - Describes an epicsStruct
• EpicsStructField - Describes a field of an epicsStruct
• EpicsStructLifetime - An interface that manages storage for an epicsStruct.

It is expected that each of these classes are extended. In fact if any field of an epicsStruct has type epicsUnknownT then it is likely that most will be overridden.

epicsTypes does not specify how to locate an EpicsStructLifetime or an EpicsStructDef. This is application dependent.

An EpicsStruct contains two fields:

pstructDef

Address of a EpicsStructDef that describes the structure.

pstorage

Address of storage for the data contained in the structure.

EpicsStructDef

EpicsStructDef has the fields:

name

The structure name.

plifetime

Address of a EpicsStructLifetime interface. See below.

nfields

The number of fields in the structure.

pfield

An array of pointers to EpicsStructField, one for each field.

EpicsStructField

EpicsStructField has the fields:

name

The field name.

type

The field type, which can be any epicsType.

EpicsStructLifetime

EpicsStructLifetime is an interface that has three methods:

allocate

This sets pstructDef to the definition for the associated structure and sets pstorage to the address of storage for the fields in the structure.

destroy

releases the storage for the structure

exposeField

This returns the address of the storage for the data associated with the field.

epicsMDArray

An EpicsMDArray is an instance of an epicsMDArrayT It specifies the element type, has the address of the array description, type and has an associated buffer that holds the storage for the array.

epicsMDArrayT, i.e. multidimensional array data, is a supported type, because collection and display of two and three dimensional images is a common requirement.

An EpicsMDArray can only hold primitive or string data. Thus it can be any of the types epicsBooleanT, ..., epicsStringT but it can not be type epicsUnknownT, epicsArrayT, epicsStructT, or epicsMDArrayT.

epicsMDArray.h contains the following:

    class EpicsMDArrayDescription;
    class EpicsMDArrayBuffer;

    class EpicsMDArray {
    public:
       EpicsMDArrayDescription *pdescription;
       EpicsMDArrayBuffer *pbuffer;
    };

    class EpicsMDArrayBounds {
    public
        epicsUInt32 low;
        epicsUInt32 high;
    };
    class EpicsMDArrayDescription {
    public:
        epicsType   type; 
        epicsInt16  ndim;      // number of dimensions
        EpicsMDArrayBounds bounds[]; // bounds[ndim]
    };

    class EpicsMDArrayBuffer {
    public:
        virtual epicsUInt32 allocate(
                    epicsUInt32 capacity,epicsUint16 elementSize) = 0;
        virtual void release(bool onlyStorage) = 0;
        virtual epicsUInt32 capacity() = 0;
        virtual epicsUInt32 elementSize() = 0;
        virtual epicsUInt32 limit() = 0;
        virtual void limit(epicsUInt32 newLimit) = 0;
        virtual void expose(epicsUInt32 offset, epicsUInt32 limitRequest,
                                   void *pdata, epicsUInt32 *limit);
    };

    typedef EpicsMDArrayBuffer *(EpicsMDArrayBufferAllocate)();
    class EpicsMDArrayBufferFactory {
    public:
        static epicsUint16 typeToTypeID(const char *type);
        static EpicsMDArrayBuffer *allocate(epicsUint16 typeId);
        static void register(const char *type,
                         EpicsMDArrayBufferAllocate allocater);
    };
    // type : At least "Contiguous" and "Segmented" are implemented

EpicsMDArray has the following fields:

pdescription

The address of an EpicsMDArrayDescription that describes the array

pbuffer

The address of an EpicsMDArrayBuffer that provides access to the array.

EpicsMDArrayDescription

EpicsMDArrayDescription has the following fields:

type

The element type which must be one of bool,...,string

ndim

The number of dimensions

bounds

An array(ndim) of EpicsMDArrayBounds

EpicsMDArrayBounds has the fields:

low

high

The definitions describe a "slice" of a multi-dimensional array.

EpicsMDArrayBuffer

Multiple buffer implementations are available. An implementation can provide the following semantics:

• contiguous - The array is stored in a contiguous set of bytes.
  • mutable - The array can be modified after an initial value is assigned.
  • non-mutable - After the original assignment, the array becomes read only.
• segmented - The array is stored in chunks. This form should be used for strings that are constantly modified.

A buffer can only be accessed via interface EpicsMDArrayBuffer. In additions to holding storage it keeps the following information:

capacity

The number of elements for which storage is allocated.

element size

the number of bytes for each element

limit

The current size, i.e. the index of the first element that can not hold data. Data can not be read from or written into a array beyond limit. When data is being written to a buffer limit is normally equal to capacity. When data is being read from a buffer limit is normally less than capacity and indicates the end of valid data.

EpicsMDArrayBuffer has the following methods:

allocate

This allocates space for up to capacity elements. The number of elements allocated is returned. An implementation attempts to allocate the requested capacity but some implemenations, e.g. network buffers, may impose a maximum size. If capacity is not zero when this is called and new storage is allocated then the old storage is freed or reused and the elements spanned by position, limit appear in the newly allocated storage.

release

Storage for the array is released. If onlyStorage is true then the storage for the buffer itself is also released.

capacity

return the capacity

limit

Two methods are available, one to get the current limit and one to set the limit.

expose

A request to return the address of actual storage. Since a buffer implementation may used segmented memory the amount of storage exposed may be less than the amount requested.

EpicsMDArrayBufferFactory

This is a class for allocating an EpicsMDArrayBuffer and also for registering EpicsMDArrayBuffer implementations. Note that allocating a buffer is different than allocating storage for the data that is stored in the buffer.