V4 Design: epicsTypes

Marty Kraimer and Andrew Johnson

June 13, 2005

Overview

This document describes the fundamental C++ data types which will be used internally by iocCore and hence must be supported in EPICS Version 4.0. We do not cover composite (struct) types here, as a generic interface to such types requires introspection information which is only provided at a higher level.

All data that is sent to or received from any EPICS record will be composed out of the fundamental data types described here.

The data types are:

• Primitive Types
  • epicsBoolean
  • epicsOctet
  • epicsInt16
  • epicsInt32
  • epicsInt64
  • epicsFloat32
  • epicsFloat64
• Aggregate Types
  • EpicsString
  • EpicsArray
  • EpicsMDArray
  • EpicsEnum

Locking Issues

These fundamental EPICS datatypes do not provide facilities for preventing simultaneous access by multiple threads, which is especially important for the aggregate types. For example the class definitions for EpicsString, EpicsArray and EpicsMDArray all have methods which return pointers into their internal data buffers, for efficiency reasons. In order to make the use of these types thread-safe, suitable access rules and mutual exclusion protection must be established at some higher level. The particular locking scheme used must be appropriate to the application, so cannot be defined in this document.

---

Primitive Types

epicsTypes.h contains the following typedefs, which are called the primitive types:

    typedef bool               epicsBoolean;
    typedef char               epicsOctet;
    typedef short              epicsInt16;
    typedef int                epicsInt32;
    typedef long long          epicsInt64;
    typedef float              epicsFloat32;
    typedef double             epicsFloat64;

It may be necessary to provide operating system dependent definitions for some of the types. For example on some architectures an epicsInt64 may have to be defined as a long rather than a long long.

---

Aggregate Types

The aggregate types must have a fixed size but still be able to manage memory in various ways which will be different for different purposes. Therefor they are all based on a standard interface to a memory buffer which can have different implementations for different memory management requirements.

EpicsBuffer

epicsBuffer.h defines a generic interface EpicsBuffer to a buffer of octet data, for which there may be several different implementations.

A buffer factory EpicsBufferFactory provides a central registry of EpicsBufferCreator objects and uses these to create instances of a particular buffer type on demand.

    class EpicsBuffer { // interface
    protected:
        virtual ~EpicsBuffer() = 0;
    public:
        virtual EpicsBufferCreator *creator() const = 0;
        virtual void reserve(epicsInt32 capacity) = 0;
        virtual epicsInt32 capacity() const = 0;
        virtual void resize(epicsInt32 newsize) = 0;
        virtual epicsInt32 size() const = 0;
        virtual epicsInt32 maxSize() const = 0;
        virtual bool mutable() const = 0;
        virtual bool isEqual(const EpicsBuffer &cmp) const = 0;
        virtual bool isEqual(epicsInt32 offset, epicsInt32 len,
                             const epicsOctet *pdata) const = 0;
        virtual void expose(epicsInt32 offset, epicsInt32 &len,
                            epicsOctet *&pdata) = 0;
        virtual void expose(epicsInt32 offset, epicsInt32 &len,
                            const epicsOctet *&pdata) const = 0;
    
    friend class EpicsBufferCreator;
    };

An EpicsBuffer is a container for epicsOctet data values, and may be used to store things such as character strings containing UTF-8 characters or arrays of some other data type.

Multiple implementations of EpicsBuffer are needed with different characteristics, but all buffers will be accessed using the same EpicsBuffer interface. At least the following buffer types will be provided:

• readonly - A contiguous array that already contains the desired data is provided at buffer initialization time, and cannot be modified. This buffer is for use with literal character strings, and probably won't be registered with the EpicsBufferFactory.
• contiguous - The data is stored in a contiguous array of octets.
• segmented - The data is stored in fixed-size chunks. This form should be used for buffers that are frequently created and released, or which often need to change in size.

Additional implementations may be provided for other purposes such as managing network buffers. There might be a need for a contiguous zero-terminated buffer type for code that regularly talks to C string handling routines.

In addition to storing the data, the buffer keeps the following information:

capacity

The number of octets of storage allocated for the buffer.

size

The number of octets of data currently held in the buffer.

Implementations of EpicsBuffer must provide the following methods, the names of which have been designed (where possible) to match the names of the equivalent methods in the C++ standard template library's container classes:

~EpicsBuffer()

All data storage allocated for the buffer is reclaimed by the destructor.

EpicsBufferCreator *creator() const

An EpicsBuffer must know its own creator.

void reserve(epicsInt32 capacity)

Allocate space to store capacity octets. Some implementations may impose a max_size() smaller than the limits of memory. If either limit is exceeded by the request, an exception will be thrown. This method is used to request an increase or decrease in the amount of storage alloted for the buffer and may cause the data to move in memory, but does not change the data stored or its size. In particular it cannot reduce the capacity below the amount of data currently stored in the buffer, so to release all the data storage an application would call resize(0) then reserve(0).

epicsInt32 capacity() const

Returns the allocated capacity of the buffer.

void resize(epicsInt32 newsize)

Sets the number of octets currently stored, up to the current capacity.

epicsInt32 size() const

Returns the number of octets currently stored in the buffer.

epicsInt32 maxSize() const

Returns the size of the largest buffer that may be allocated. An attempt to allocate more than this amount of space will fail for certain, but there is no guarantee that this amount of memory is available. This method provides a way for application code to discover any limits that may be imposed by the particular buffer type.

bool mutable() const

Returns true if the buffer data can be modified, i.e. are puts allowed.

isEqual(const EpicsBuffer &cmp)

Compares the contents of this buffer with the contents of the buffer cmp, and returns true if the data is identical.

isEqual(epicsInt32 offset, epicsInt32 len, const epicsOctet *pdata)

Compares len octets of data starting offset octets into this buffer with the octet array pdata supplied by the caller. This method is mainly provided to simplify the implementation of the other isEqual method.

void expose(epicsInt32 offset, epicsInt32 &len, epicsOctet *&pdata)

A request for the address of len octets of buffer data starting offset octets into the buffer. If the buffer implementation uses segmented memory the maximum number of contiguous octets exposed is the segment size, which may be less than the length requested. In this case the value of len will be reduced before the method returns. The caller must process the data provided and call expose again with an increased offset to retrieve or modify subsequent segments.

Two versions of the expose method are provided, one for use when modifying the buffer and one where the buffer (and the returned data pointer) is const.

• The const expose() method will only present data up to the current limit as reported by the size() method.
• The non-const expose() method will present data up to the current capacity of the buffer, so application code that is reading as well as modifying the buffer's contents must keep track of the current value of size() and stop reading when this is reached (or use the both methods, the const one for reading and the other for writing).
• The non-const version will throw an exception if called for a buffer that is immutable.

The design of the expose method is intended to increase the efficiency of data access. The caller must follow these conventions:

• Must call resize(n) if the data length is to be changed.
• Must not access storage outside the length returned by expose. The caller may have to make multiple expose calls to read or write a complete string.

Class methods that allocate or deallocate memory must be thread-safe; buffers could be resized, created or destroyed at any time from any thread. However synchronizing access to the data in the buffer is not the responsibility of this class, thus a buffer should probably not contain a mutex/lock, but its creator might.

EpicsBufferCreator

    class EpicsBufferCreator { // interface
    public:
        virtual const char *bufferType() const = 0;
        virtual EpicsBuffer *create(epicsInt32 elementSize = 1) = 0;
        virtual void destroy(EpicsBuffer *&pbuffer) = 0;
    };

An EpicsBufferCreator is an interface implemented by something that knows how to create a particular type of EpicsBuffer. How the creator manages the memory involved is up to the implementation - buffers and their data segments may be stored on a freelist by some implementations, and obtained from system memory by others. Three methods must be implemented:

const char *bufferType() const

Returns some well-known name of the buffer type created.

EpicsBuffer *create(epicsInt32 elementSize = 1)

Instanciates an EpicsBuffer which is appropriate for storing arrays of size elementSize. Array elements can never cross a segment boundary, thus a buffer segment can never be smaller than an individual element. This method may throw an exception if it is unable to create the buffer.

void destroy(EpicsBuffer *&pbuffer)

Executes pbuffer's destructor to reclaim the buffer storage, reclaims the pbuffer object itself, and sets pbuffer to NULL.

The way to deallocate an EpicsBuffer *pbuffer is like this:

    if (pbuffer) pbuffer->creator()->destroy(pbuffer);

or use the convenience function provided by EpicsBufferFactory which does exactly the same thing:

    EpicsBufferFactory::destroy(pbuffer);

Implementations of this class must be thread-safe; buffers could be created or destroyed at any time from any thread.

EpicsBufferFactory

    class EpicsBufferFactory {
    public:
        static void register(EpicsBufferCreator *creator);
        static EpicsBufferCreator *creator(const char *type);
        static EpicsBuffer *create(const char *type,
                                   epicsInt32 elementSize = 1) {
            return EpicsBufferFactory::creator(type)->create(elementSize);
        }
        static void destroy(EpicsBuffer *&pbuffer) {
            if (pbuffer) pbuffer->creator()->destroy(pbuffer);
        }
    };

The EpicsBufferFactory provides a registry of EpicsBuffer implementations, each defined by an EpicsBufferCreator. While each EpicsBufferCreator is ultimately responsible for creating all EpicsBuffers of its particular type, the EpicsBufferFactory provides a convenience function that looks up an EpicsBufferCreator by name and calls its create() function directly. Use of the factory to create buffers is optional, since the registry can also be queried to find an EpicsBufferCreator by name.

void register(EpicsBufferCreator *creator)

Adds creator to the list of known buffer types. It will throw an exception if the same name is already registered at a different address.

EpicsBufferCreator *creator(const char *type)

Looks up the EpicsBufferCreator associated with type and returns a pointer to it. Return NULL, or throw if not found?

EpicsBuffer *create(const char *type, epicsInt32 elementSize = 1)

Looks up the EpicsBufferCreator associated with type, passes elementSize to its create() method and returns the resulting EpicsBuffer pointer. Return NULL, or throw if not found?

void destroy(EpicsBuffer *&pbuffer)

Asks pbuffer's creator destroy the buffer, reclaiming its associated memory and set pbuffer to NULL.

It should be clear that creating a buffer is not the same as allocating storage for the data held in that buffer. The former is an EpicsBufferCreator's job, while the latter is the responsibility of the EpicsBuffer itself. However an EpicsBufferCreator may provide additional services to its EpicsBuffer instances such as free lists.

---

epicsString

epicsString.h defines a string class that uses an EpicsBuffer to hold its data. Within the core software the intention is that all strings will be encoded in Unicode/UTF-8, but there is nothing specific to that encoding in the EpicsString interface.

    class EpicsString {
    public:
        EpicsString();
        EpicsString(const char *literal);
        EpicsString(const char *bufferType, epicsInt32 capacity);
        EpicsString(EpicsBufferCreator *creator, epicsInt32 capacity);
        virtual ~EpicsString();
        EpicsString& operator=(const EpicsString &rhs);
        void createBuffer(const char *bufferType, epicsInt32 capacity = 0)
        void createBuffer(EpicsBufferCreator *creator, epicsInt32 capacity = 0);
        EpicsBufferCreator *bufferCreator() const;
        void destroyBuffer();
        epicsInt32 get(epicsInt32 offset, epicsInt32 len,
                       epicsOctet *pto) const;
        epicsInt32 put(epicsInt32 offset, epicsInt32 len,
                       const epicsOctet *pfrom);
        epicsInt32 hash(epicsInt16 nBitsHashIndex) const;
        
        // These routines are as described for EpicsBuffer
        void reserve(epicsInt32 capacity);
        epicsInt32 capacity() const;
        void resize(epicsInt32 newsize);
        epicsInt32 size() const;
        epicsInt32 maxSize() const;
        bool mutable() const;
        bool isEqual(const EpicsBuffer &cmp) const;
        bool isEqual(epicsInt32 offset, epicsInt32 len,
                     const epicsOctet *pdata) const;
        bool expose(epicsInt32 offset, epicsInt32 &len,
                    epicsOctet *&pdata);
        bool expose(epicsInt32 offset, epicsInt32 &len,
                    const epicsOctet *&pdata) const;
    protected:
        EpicsBuffer *pbuffer;
    private:
        EpicsString(const EpicsString &str);   // No copy constructor
    };
    
    epicsBoolean operator==(const EpicsString &lhs, const EpicsString &rhs);
    epicsBoolean operator!=(const EpicsString &lhs, const EpicsString &rhs);

In Unicode/UTF-8 encoded strings, multiple octets may be needed to store a single character. However most code can probably ignore the character encoding as long as it does not assume that a single character can be stored in one octet. As long as the final output device (be it a terminal window, a printer or some other software package) is expecting Unicode/UTF-8 characters, a string can be output using the printf family of methods.

The EpicsString data is not guaranteed to be null-terminated and in most cases will not be, so applications using the string must take precautions when interfacing with routines that expect the string to end with a zero byte. We may provide a type of contiguous EpicsBuffer that does guarantee a null terminator.

EpicsString provides the following methods:

EpicsString()

If a string is default constructed, one of the createBuffer() methods must be called to set the underlying buffer type before any data can be stored.

EpicsString(const char *literal)

Use of this constructor causes the string to have a readonly buffer that holds just the literal string given, without copying. This is intended to be efficient as it is likely to be commonly used.

EpicsString(const char *bufferType, epicsInt32 capacity)

EpicsString(picsInt16 bufferTypeId, epicsInt32 capacity)

Here a buffer type is specified, either by name or by identifier, so the selected buffer type will be created using the EpicsBufferFactory, and if capacity is non-zero the space will be reserved for at least that number of octets of data.

~EpicsString()

The destructor will destroy the EpicsBuffer if one has been created.

void createBuffer(const char *bufferType, epicsInt32 capacity)

void createBuffer(EpicsBufferCreator *creator, epicsInt32 capacity)

These methods create an EpicsBuffer to hold the string data, and if capacity is non-zero the space will be reserved for at least that number of octets of data.

If a buffer type has already been selected for the string, this method will throw an exception. (However this definition may change if it is found to be too onerous; the alternative would be to create a new buffer, copy the data to it and then destroy the old buffer)

EpicsBufferCreator *bufferCreator() const

Returns the EpicsBufferCreator used for the underlying buffer storage, or NULL if none has been set yet.

void destroyBuffer()

Causes the underlying buffer storage to be destroyed, after which one of the createBuffer() methods must be used before data can be stored in the string again.

epicsInt32 get(epicsInt32 offset, epicsInt32 len, epicsOctet *pto)

Copies up to len octets starting at offset from the string buffer to pto, and returns the number of octets transfered. The return value will be less than len if offset+len > size().

epicsInt32 put(epicsInt32 offset, epicsInt32 len, const epicsOctet *pfrom)

Copies up to len octets from pfrom into the buffer starting offset octets from the beginning, and returns the number of octets copied. The return value will be less than len if offset+len > capacity(). The string size will be updated if this call extends the string beyond its original size. However on entry offset must not be greater than size() or an exception will be thrown.

epicsInt32 hash(epicsInt16 nBitsHashIndex) const

Calculates an n-bit hash of the octets stored in the string buffer.

EpicsString also implements all of the routines in the EpicsBuffer interface. If an EpicsBuffer has been created these calls are forwarded to the underlying EpicsBuffer method. If an EpicsBuffer has not been created however, all methods except for createBuffer() will throw an exception.

---

epicsArray

epicsArray.h defines an array class that uses an EpicsBuffer to hold its data, which can be of any datatype that has a fixed size.

    class EpicsArray {
    public:
        EpicsArray(epicsInt32 elementSize = 0);
        EpicsArray(const char *bufferType, epicsInt32 capacity,
                   epicsInt32 elementSize = 0);
        EpicsArray(EpicsBufferCreator *creator, epicsInt32 capacity,
                   epicsInt32 elementSize = 0);
        virtual ~EpicsArray();
        void createBuffer(const char *bufferType, epicsInt32 capacity = 0);
        void createBuffer(EpicsBufferCreator *creator, epicsInt32 capacity = 0);
        EpicsBufferCreator *bufferCreator() const;
        void destroyBuffer();
        epicsOctet * element(epicsInt32 index);
        const epicsOctet * element(epicsInt32 index) const;
        epicsInt32 get(epicsInt32 offset, epicsInt32 len,
                       epicsOctet *pto) const;
        epicsInt32 put(epicsInt32 offset, epicsInt32 len,
                       const epicsOctet *pfrom);
        
        // Like EpicsBuffer, but units are elements not octets
        void reserve(epicsInt32 capacity);
        epicsInt32 capacity() const;
        void resize(epicsInt32 newsize);
        epicsInt32 size() const;
        epicsInt32 maxSize() const;
        bool mutable() const;
        void expose(epicsInt32 offset, epicsInt32 &len,
                    epicsOctet *&pdata);
        void expose(epicsInt32 offset, epicsInt32 &len,
                    const epicsOctet *&pdata) const;
    protected:
        void setElementSize(epicsInt32 elementSize);
        epicsInt32 getElementSize() const;
    protected:
        epicsInt32 elementSize;
        EpicsBuffer *pbuffer;
    private:
        EpicsArray(const EpicsArray &);            // No copy constructor
        EpicsArray& operator=(const EpicsArray &); // No assignment operator
    };

An EpicsArray holds an array of elements of constant size.

EpicsArray has the following methods:

EpicsArray(epicsInt32 elementSize = 0)

If an array is default constructed, one of the createBuffer() methods must be called to set the underlying buffer type before any storage can be reserved for the array elements. The array must also be told its element size before buffer space can be reserved.

EpicsArray(const char *bufferType, epicsInt32 capacity, epicsInt32 elementSize = 0)

EpicsArray(EpicsBufferCreator *creator, epicsInt32 capacity, epicsInt32 elementSize = 0)

Here a buffer type is specified, either by name or by identifier, so the selected butter type will be created using the EpicsBufferFactory, and if both capacity and elementSize are non-zero the space will be reserved for at least that number of array elements.

~EpicsArray

The destructor will destroy the EpicsBuffer if one has been created.

void createBuffer

These methods create an EpicsBuffer to hold the array data, and if capacity and elementSize are non-zero the space will be reserved for at least that number of array elements.

If a buffer type has already been selected for the array, this method will throw an exception. (However this definition may change if it is found to be too onerous; the alternative would be to create a new buffer, copy the data to it and then destroy the old buffer)

EpicsBufferCreator *bufferCreator() const

Returns the EpicsBufferCreator used for the underlying buffer storage, or NULL if none has been set yet.

destroyBuffer()

Causes the underlying buffer storage to be destroyed, after which one of the createBuffer() methods must be used before data can be stored in the array again.

epicsOctet * element(epicsInt32 index)

Returns a pointer giving direct access to the index'th element of the array, allowing this element to be modified. The pointer returned will have to be cast to an appropriate element type before use. Access to either adjacent array element must not be made by adjusting the pointer since the underlying buffer storage may be segmented - instead call element() again with the appropriately modified index.

const epicsOctet * element(epicsInt32 index) const

Returns a pointer giving const (read-only) access to the index'th element of the array, allowing this element to be read but not modified. If the underlying buffer type is not mutable, this method will throw an exception. The pointer returned will have to be cast to an appropriate element type before use. Access to either adjacent array element must not be made by adjusting the pointer since the underlying buffer storage may be segmented - instead call element() again with the appropriately modified index.

epicsInt32 get(epicsInt32 offset, epicsInt32 len, epicsOctet *pto)

Copies up to len elements starting at offset from the array buffer to pto, and returns the number of elements transfered. The return value will be less than len if offset+len > size().

epicsInt32 put(epicsInt32 offset, epicsInt32 len, const epicsOctet *pfrom)

Copies up to len elements from pfrom into the buffer starting offset elements from the beginning, and returns the number of elements copied. The return value will be less than len if offset+len > capacity(). The array size will be updated if this call extends the array beyond its original size. However on entry offset must not be greater than size() or an exception will be thrown.

void setElementSize(epicsInt32 elementSize)

This sets the size (in octets) of the element type to be stored in the array. If a buffer has already been used for previous array data, this will be released and the buffer capacity set to zero.

epicsInt32 getElementSize()

This returns the element size currently stored.

EpicsArray also implements the access routines from the EpicsBuffer interface, translating capacity, size and offset values from element counts into octets and passing them to the underlying EpicsBuffer. If an EpicsBuffer has not been created however, all these methods will throw an exception.

---

epicsMDArray

An EpicsMDArray is a multi-dimensional array of elements of constant size, using an EpicsBuffer to hold its data.

epicsMDArray.h contains the following:

    struct EpicsMDArrayDimensions {
        epicsInt32 elementSize;     // octets per element
        epicsInt16 nDimensions;     // number of dimensions
        epicsInt32 *dimensionSizes; // array containing size in each dimension
    };
    
    class EpicsMDArray {
    public:
        EpicsMDArray();
        EpicsMDArray(const char *bufferType);
        EpicsMDArray(EpicsBufferCreator *creator);
        virtual ~EpicsMDArray();
        void createBuffer(const char *bufferType);
        void createBuffer(EpicsBufferCreator *creator);
        void destroyBuffer();
        void setDimensions(const EpicsMDArrayDimensions *pdim) ????
    protected:
        EpicsMDArrayDimensions *pdims;
        EpicsBuffer *pbuffer;
    private:
        EpicsMDArray(const EpicsMDArray &);            // No copy constructor
        EpicsMDArray& operator=(const EpicsMDArray &); // No assignment
    };

Class definition and description are not yet complete as the dimensionality structure shown above isn't obviously the best way to implement this. We have to allow the number of dimensions to change at runtime, which means the storage needed for dimensionSizes will vary, although it's unlikely that we'll actually need to do this very often. It seems unwise to use a fixed length array (although that would be the simplest solution), but unnecessary overhead to use an EpicsArray to hold the dimensions (which would be the most logical solution).

Comments on this are welcome...

---

EpicsEnum

An EpicsEnum is a 16-bit index value, which uses a standard interface to convert between choice strings and their index values. There may be more than one implementation of that interface.

class EpicsEnumChoices {
public:
    virtual EpicsEnumChoices *duplicate() const = 0;
    virtual void release() = 0;
    virtual epicsInt16 nChoices() const = 0;
    virtual epicsInt16 index(const EpicsString &choice) const = 0;
    virtual void choice(epicsInt16 index, EpicsString &choice) const = 0;
};

class EpicsEnum {
public:
    enum {invalid = -1};
    
    EpicsEnum();
    EpicsEnum(const EpicsEnum &e);
    EpicsEnum(EpicsEnumChoices *choices);
    EpicsEnum(EpicsEnumChoices *choices, epicsInt16 index);
    virtual ~EpicsEnum();
    EpicsEnum& operator=(const EpicsEnum &rhs);
    
    virtual void choices(EpicsEnumChoices *pchoices);
    EpicsEnumChoices *choices() const { return pchoices; }
    epicsInt16 nChoices() const { return pchoices ? pchoices->nChoices() : 0; }
    epicsInt16 get() const { return index; }
    virtual void get(EpicsString &choice) const;
    virtual void put(epicsInt16 index);
    virtual void put(const EpicsString &choice);
protected:
    EpicsEnumChoices *pchoices;
    epicsInt16 index;
};