V4 Design: epicsTypes
EPICS: epicsTypes - Storing Network Accessable Data
May 25 2005
Overview
This document describes the C++ definitions for storing data that can be accessed without pre-complied code, i.e. it has self describing features.
Some examples are:
- IOC records - All data accessable from outside record support
- Channel Access Gateway - Usefull for storing and transfering data.
- Channel Access Clients - Can be used to get/receive data from CA server.
epicsTypes is not a replacement for dataAccess. Standard support can be provided to access, via dataAccess, epicsType data. For example standard support can be provided to move IOC record data between record instances and a Channel Access server.
epicsType is an enum that defines the following:
- epicsUnknownT - Type is unknown
- epicsBooleanT,...,epicsFloat64T - C++ primitive types
- epicsStringT - EpicsString which contains a UTF-8 Encoded Character String
- epicsArrayT - EpicsArray which describes type and storage for a one dim array. The array can be any epicsType.
- epicsStructT - EpicsStruct which describes and provides storage for a set of fields each of some epicsType
- epicsMDArrayT - EpicsMDArray which describes type and storage for a multidimensional array.
The actual types associated with the epicsTypes are:
typedef bool epicsBoolean;
typedef char epicsOctet;
typedef short epicsInt16;
typedef unsigned short epicsUInt16;
typedef int epicsInt32;
typedef unsigned int epicsUInt32;
typedef long long epicsInt64;
typedef unsigned long long epicsUInt64;
typedef float epicsFloat32;
typedef double epicsFloat64;
/*EpicsString holds UTF-8 characters*/
class EpicsString {
public:
EpicsUTF_8Buffer *pbuffer;
};
class EpicsArray {
public:
epicsType type;
EpicsArrayBuffer *pbuffer;
};
class EpicsStruct{
public:
EpicsStructDef *pstructDef;
void *pstorage;
};
class EpicsMDArray {
public:
epicsType type;
EpicsMDArrayDescription *pdescription;
EpicsMDArrayBuffer *pbuffer;
};
Primitive Types
The types epicsBoolean, ... epicsFloat64 each has an associated C++ fundamental type. An instance of a primitive type has only storage associated with it.
epicsOctet is an 8 bit byte that just contains data. It is not use as an integer type.
The unsigned integer types should only be used for data that represents masks. If they are used for representing large numbers, e.g. an epicsUInt16 that is greater than 32767, then Java clients/servers will not work properly.
Java Notes
- epicsOctet is a Java byte
- Java does not support unsigned integers. It should, however, accept each unsigned type as the corresponding signed type.
String
An EpicsString is instance of an epicsStringT. It is a UTF-8 encoded string, i.e. multiple bytes may be needed to store a single character. If code does not need to manipulate the characters in a string then, as long as it does not assume that each character is a byte, it just ignore the fact that the string is UTF-8 encoded. If the local computer has been internationalized for the particular UTF-8 encoding then the string can be printed via the printf family of methods.
An EpicsString has an associated buffer that holds the storage for the string. The buffer is accessed via an interface.
Multiple interface implementations are available. At least the following are available:
- contiguous - The string is stored in a contiguous set of bytes.
- mutable - The string value can be modified after an initial value is assigned.
- non-mutable - After the original assignment, the string becomes read only.
- segmented - The string is stored in chunks. This form should be used for strings that are constantly modified.
Array
An EpicsArray is instance of an EpicsArrayT.
It specifies the element type and has an associated buffer that holds the storage for the array. The buffer is accessed via an interface.
Multiple interface implementations are available. At least the following are available:
- contiguous - The array is stored in a contiguous set of bytes.
- mutable - The array can be modified after an initial value is assigned.
- non-mutable - After the original assignment, the array becomes read only.
- segmented - The array is stored in chunks. This form should be used for strings that are constantly modified.
Since an EpicsArray can contain data of any epicsType it can also be an array of arrays. This is analogous to multidimensional arrays in Java. The type can be epicsUnknownT, e.g., unknown type. In this case the code that creates the arrays must provide additional code. Class EpicsArray can be extended so that it contains additional informartion about the data that is stored in the array. The only requirement is that each array element must have the same fixed size.
Struct
An EpicsStruct contains fields each of which is an epicsType. It has the address of the structure description and the address of storage for the fields of the structure.
Questions
- Can a field have type epicsUnknownT? Probably NOT.
- Can a field have type epicsStructT? May not be usefull.
- Can the Class definitions associated with EpicsStruct be extended?
- Perhaps EpicsStruct should be defined such that the field types are restricted such that it is just a container of primitive types including EpicsString and an array of primitive types.
- What is the namespace for structure names? That is how do we prevent struct names from becoming global?
Multi-Dimensional Arrays
An EpicsMDArray is an instance of an epicsMDArrayT It specifies the element type, has the address of the array description, type and has an associated buffer that holds the storage for the array.
epicsMDArrayT, i.e. multidimensional array data, is a supported type, because collection and display of two and three dimensional images is a common requirement.
Question
- Should epicsMDArrayT be a valid type? Probably not.
Global Comment about method expose
The class definitions for non-primitive types all provide a method expose which returns the address of data. This is done for efficiency and convenience. In order to make expose safe, some rules must be established. The application that uses epicsTypes must set the rules since a generic solution may not be desirable.
One possibility is that a lock is associated with each object that supports expose. Code can only call expose and can only access the data returned by expose if it holds the lock.
epicsTypes
epicsTypes.h contains the following:
/* The following may require OSD definitions*/
typedef bool epicsBoolean;
typedef char epicsOctet;
typedef short epicsInt16;
typedef unsigned short epicsUInt16;
typedef int epicsInt32;
typedef unsigned int epicsUInt32;
typedef long long epicsInt64;
typedef unsigned long long epicsUInt64;
typedef float epicsFloat32;
typedef double epicsFloat64;
enum epicsType {
epicsUnknownT,
epicsBooleanT, // epicsBoolean
epicsOctetT, // epicsOctet
epicsInt16T, // epicsInt16
epicsUInt16T, // epicsUInt16
epicsInt32T, // epicsInt32
epicsUInt32T, // epicsUInt32
epicsInt64T, // epicsInt64
epicsUInt64T, // epicsUInt64
epicsFloat32T, // epicsFloat32
epicsFloat64T, // epicsFloat64
epicsStringT, // EpicsString
epicsArrayT, // EpicsArray
epicsStruct, // EpicsStruct
epicsMDArrayT // EpicsMDArray ???? Should this be a type
};
Discussion of epicsTypes
epicsTypes provides classes for describing data that can be introspected and can be passed between different platforms. All data that is sent to or received from EPICS records will be composed of epicsTypes.
The types epicsBooleanT, ..., epicsFloat64T all map to a C++ standard type. It may be necessary to provide operating system dependent definitions for some of the types. For example on some architectures a epicsInt64 may have to be defined as a long rather than a long long.
The types epicsStringT, epicsArrayT, epicsStructT, and epicsMDArrayT are designed so that the data they contain can be described, introspected, and passed over a network.
An EpicsArray can hold data of any epicsType including epicsUnknownT. This is usefull because the storage for an array can be managed by generic code.
An epicsStruct can contain fields of any epicsType except epicsUnknownT.
epicsUnknownT is provided in case something expected to produce an epicsType fails.
epicsString
epicsString.h contains the following:
class EpicsUTF_8Buffer;
/*EpicsString holds UTF-8 characters*/
class EpicsString {
public:
EpicsUTF_8Buffer *pbuffer;
};
class EpicsUTF_8Buffer {
public:
virtual epicsInt32 allocate(epicsInt32 capacity) = 0;
virtual void release(bool onlyStorage) = 0;
virtual epicsInt32 capacity() = 0;
virtual epicsInt32 limit() = 0;
virtual void limit(epicsInt32 newLimit) = 0;
virtual epicsInt32 get(epicsOctet *pto,
epicsInt32 offset, epicsInt32 limit) = 0;
virtual epicsInt32 put(const epicsOctet *pfrom,
epicsInt32 offset, epicsInt32 limit) = 0;
virtual bool isEqual(const EpicsUTF_8Buffer *pbuffer) = 0;
virtual bool isEqual(const epicsOctet *pstring, epicsInt32 len) = 0;
virtual void expose(epicsInt32 offset, epicsInt32 limitRequest,
epicsOctet *pdata, epicsInt32 *limit);
virtual epicsUint32 hash(epicsInt16 nBitsHashIndex) = 0;
};
typedef EpicsUTF_8Buffer *(EpicsUTF_8BufferAllocate)();
class EpicsUTF_8BufferFactory {
public:
static epicsUint16 typeToTypeID(const char *type);
static EpicsUTF_8Buffer *allocate(epicsUint16 typeId);
static void register(const char *type,
EpicsUTF_8BufferAllocate allocater);
};
// type : At least "Contiguous" and "Segmented" are implemented
Discussion of epicsString
An EpicsString contains UTF-8 encoded character strings. It has the following fields:
- pbuffer
- The address of a EpicsUTF_8Buffer, which is a class that manages the string storage.
EpicsUTF_8Buffer is a pure abstract base class because multiple implementations are provided.
EpicsUTF_8Buffer
An EpicsUTF_8Buffer contains storage for a UTF-8 encoded string. In addition to holding storage for a string, a string buffer keeps the following information.
- capacity
- The number of octets allocated, i.e. the number of UTF-8 characters the buffer can hold.
- limit
- The current size, i.e. the index of the first octet that can not hold data. Data can not be read from or written into a buffer beyond limit. When data is being written to a buffer limit is normally equal to capacity. When data is being read from a buffer limit is normally less than capacity and indicates the end of valid data.
EpicsUTF_8Buffer has the following methods:
- allocate
- This allocates space for up to capacity octets. The number of octets allocated is returned. An implementation attempts to allocate the requested capacity but some implemenations, e.g. network buffers, may impose a maximum size. If capacity is not zero when this is called and new storage is allocated then the old storage is freed or reused and the octets spanned by position, limit appear in the newly allocated storage.
- release
- Releases storage. If onlyStorage is true then the storage for the string is freed and capacity is set to zero; otherwise the string storage and the storage for EpicsUTF_8Buffer itself is freed.
- capacity
- returns the capacity
- limit
- Two methods are available, one to get the current limit and one to set the limit.
- get
- copies characters to pto and returns the number of octets transfered.
- put
- copies characters from pfrom, puts them into the buffer, and returns the number of octets transfered.
- isEqual(EpicsUTF_8Buffer *)
- Compares the string stored in the buffer with a string stored in a different buffer. This is normally called by code that uses an EpicsUTF_8Buffer.
- isEqual(epicsOctet *pstring, epicsInt32 len)
- Compares the string stored in the buffer with a string supplied by the caller. This is normally called by EpicsUTF_8Buffer itself to compare it's string with the string stored in another buffer.
- expose
- A request to return the address of actual bytes of storage. Since a buffer implementation may used segmented memory the number of bytes exposed may be less than the amount requested.
- hash
- implement a hash on the octets stored in the buffer.
EpicsUTF_8BufferFactory
This is a class for allocating an EpicsUTF_8Buffer and also for registering EpicsUTF_8Buffer implementations.
Multiple implementations are provided. Implementations are provided for contiguous and segmented strings.
The contiguous implementation is appropriate for a string that is allocated at initialization and is only rarely modified. At least two contiguous implementations are provided: mutable and non-mutable. The mutable implementation allows the string to be modified. The non-mutable implementation does not allow the string to be changed after it has been given an initial value.
The segmented implementation is appropriate for strings that are frequently modified or are transient, e.g. strings being transfered over the network.
epicsArray
epicsArray.h contains the following:
class EpicsArrayBuffer;
class EpicsArray {
public:
epicsType type;
EpicsArrayBuffer *pbuffer;
};
class EpicsArrayBuffer {
public:
virtual epicsUInt32 allocate(
epicsUInt32 capacity,epicsUint16 elementSize) = 0;
virtual void release(bool onlyStorage) = 0;
virtual epicsUInt32 capacity() = 0;
virtual epicsUInt32 elementSize() = 0;
virtual epicsUInt32 limit() = 0;
virtual void limit(epicsUInt32 newLimit) = 0;
virtual epicsUInt32 position() = 0;
virtual void position(epicsUInt32 newPosition) = 0;
virtual epicsInt32 get(void *pto,
epicsInt32 offset, epicsInt32 limit) = 0;
virtual epicsInt32 put(const void *pfrom,
epicsInt32 offset, epicsInt32 limit) = 0;
virtual void expose(epicsUInt32 offset, epicsUInt32 limitRequest,
void *pdata, epicsUInt32 *limit);
}
typedef EpicsArrayBuffer *(EpicsArrayBufferAllocate)();
class EpicsArrayBufferFactory {
public:
static epicsUint16 typeToTypeID(const char *type);
static EpicsArrayBuffer *allocate(epicsUint16 typeId);
static void register(const char *type,
EpicsArrayBufferAllocate allocater);
};
// type : At least "Contiguous" and "Segmented" are implemented
Discussion of epicsArray
An EpicsArray describes and array with elements of any epicsType. It has the following fields:
- <type>
- Any epicsType
- pbuffer
- The address of an EpicsArrayBuffer that provides access to the array.
EpicsArrayBuffer
EpicsArrayBuffer has the following methods:
- allocate
- This allocates space for up to capacity elements. The number of elements allocated is returned. An implementation attempts to allocate the requested capacity but some implemenations, e.g. network buffers, may impose a maximum size. If capacity is not zero when this is called and new storage is allocated then the old storage is freed or reused and the elements spanned by position, limit appear in the newly allocated storage.
- release
- Releases storage. If onlyStorage is true then the storage for the string is freed and capacity is set to zero; otherwise the string storage and the storage for EpicsUTF_8Buffer itself is freed.
- capacity
- return the capacity
- limit
- Two methods are available, one to get the current limit and one to set the limit.
- position
- Two methods are available, one to get the current position and one to set the position.
- get
- copies characters to pto and returns the number of octets transfered.
- put
- copies characters from pfrom, puts them into the buffer, and returns the number of octets transfered.
- expose
- A request to return the address of actual storage. Since a buffer implementation may used segmented memory the amount of storage exposed may be less than the amount requested.
EpicsArrayBufferFactory
This is a class for allocating an EpicsArrayBuffer and also for registering EpicsArrayBuffer implementations.
At least two implementations will be provided: contiguous and segmented. The contiguous implementation is appropriate for a string that is allocated at initialization and is only rarely modified. The segmented implementation is appropriate for strings that are frequently modified or are transient, e.g. strings being transfered over the network.
An EpicsArrayBuffer can be used to support arrays that have data that is not an epicsType. The only requirement is that the each element has the same fixed size. An EpicsArrayBuffer can also be used to manage storage for an EpicsMDArray.
epicsStruct
epicsStruct.h contains the following:
class EpicsStructDef;
class EpicsStructLifetime;
class EpicsStruct{
public:
EpicsStructDef *pstructDef;
void *pstorage;
};
class EpicsStructField {
public:
EpicsString name;
epicsType type;
};
class EpicsStructDef{
public:
EpicsString name;
EpicsStructLifetime *plifetime;
epicsInt16 nfields;
EpicsStructField *pfield[]; // ptr to array of ptr to EpicsStructField
};
class EpicsStructLifetime {
public:
virtual void allocate(EpicsStruct *pstruct) = 0;
virtual void destroy(EpicsStruct *pstruct) = 0;
virtual void *exposeField(EpicsStruct *pstruct, epicsInt16 index) = 0;
};
class EpicsStructFactory {
public:
static EpicsStructLifetime *find(const char *structName);
static void register(const char *structName,
EpicsStructDef *pdef);
};
Discussion of epicsStruct
epicsStruct</pp> contains two fields:
- pstructDef
- Address of a EpicsStructDef that describes the structure.
- pstorage
- Address of storage for the data contained in the structure.
EpicsStructDef
EpicsStructDef has the fields:
- name
- The structure name.
- plifetime
- Address of a EpicsStructLifetime interface. See below.
- nfields
- The number of fields in the structure.
- pfield
- An array of pointers to EpicsStructField, one for each field.
EpicsStructField
EpicsStructField has the fields:
- name
- The field name.
- type
- The field type, which can be any epicsType.
EpicsStructLifetime
EpicsStructLifetime is an interface that has three methods:
- allocate
- This sets pstructDef to the definition for the associated structure and sets pstorage to the address of storage for the fields in the structure.
- exposeField
- This returns the address of the storage for the data associated with the field.
EpicsStructFactory
EpicsStructFactory is a factory for registering and finding EpicsStructLifetime interfaces. It has the methods:
- find
- Given a name find the associated interface.
- register
- Register a EpicsStructDef for the name.
If register is called with EpicsStructDef.plifetime = null then the factory will provide a default implementation. The default implementation will be an array of pointers for each field. Each pointer will have the address of storage for the corresponding field type.
epicsMDArray
epicsMDArray.h contains the following:
class EpicsMDArrayDescription;
class EpicsMDArrayBuffer;
class EpicsMDArray {
public:
epicsType type;
EpicsMDArrayDescription *pdescription;
EpicsMDArrayBuffer *pbuffer;
};
class EpicsMDArrayBuffer {
public:
virtual epicsUInt32 allocate(
epicsUInt32 capacity,epicsUint16 elementSize) = 0;
virtual void release(bool onlyStorage) = 0;
virtual epicsUInt32 capacity() = 0;
virtual epicsUInt32 elementSize() = 0;
virtual epicsUInt32 limit() = 0;
virtual void limit(epicsUInt32 newLimit) = 0;
virtual void expose(epicsUInt32 offset, epicsUInt32 limitRequest,
void *pdata, epicsUInt32 *limit);
};
class EpicsMDArrayBounds {
epicsUInt32 low;
epicsUInt32 high;
};
class EpicsMDArrayDescription {
public:
epicsUInt32 capacity; //capacity in number of elements
epicsInt16 ndim; // number of dimensions
epicsType type;
void *pstorage; // storage for capacity elements of type
EpicsMDArrayBounds bounds[]; // bounds[ndim]
};
typedef EpicsMDArrayBuffer *(EpicsMDArrayBufferAllocate)();
class EpicsMDArrayBufferFactory {
public:
static epicsUint16 typeToTypeID(const char *type);
static EpicsMDArrayBuffer *allocate(epicsUint16 typeId);
static void register(const char *type,
EpicsMDArrayBufferAllocate allocater);
};
// type : At least "Contiguous" and "Segmented" are implemented
Discussion of epicsMDArray
MARTY DISCUSS
