This tutorial describes how to create custom property types.

Introduction

DevBot provides a flexible mechanism for supplementing the built-in data types with third-party extensions. The type system is designed to be lightweight but powerful enough to support DevBot's requirements for reflection and network transparency.

When (not) to use a custom type

Custom types should be avoided wherever possible because it:

Increases maintenance burden
Limits interoperability with third-party components
Can introduce difficult to identify bugs

Before implementing a custom type you should ensure that none of the built-in array types could meet your requirements. So, for example, there is no need to invent a new data type to represent LIDAR output -- a real vector is sufficient.

You should also consider whether the value must be treated atomically. If not, it might be better to use a set of properties. For example, a GPS receiver will typically provide both position and time information as an atomic "measurement". However for most purposes they may be separated.

Type overview

A custom type is represented by an instance of the BotPropertyType structure summarized below:

struct BotPropertyType {
    char *name;
    uint32_t size;
    char *(*serialize)(void *storage, uint64_t num_members);
    bool (*deserialize)(void *storage, uint64_t num_members, const char *text);
    bool (*copy)(void *src_raw, void *dest_raw, uint32_t length);
    void (*free)(void *storage, uint32_t length);
};

Field	Required?	Description
name	Yes	Every type must have a (globally) unique name.
size	Yes	The size is used to determine the size of type instances that will be managed by the core library. For example, a single 'int' is 4 bytes.
serialize	Yes	A function used to convert a property's 'storage' into a network-safe string. For DevBot built-in types, this string comprises humanly readable, comma separated values, and can represent either a 'unit' or 'array' of storage values. However, this serialization method may not always be appropriate. This function cannot fail.
deserialize	Yes	A function that reverses the transformation performed by the the serialize function. This function is permitted to fail, as indicated by the boolean return code. Failure may occur if the input text is mismatched or malformed.
copy	No	Copies a property's storage from one place to another. This may be done in a type-specific way. If not provided, the operation is performed via a direct memory copy. This may be useful for validation purposes -- say a type can represent a value, but that value is not ever contextually legal. Boolean return value indicates success/failure.
free	No	Free's property storage in a type-specific way. If not provided, the operation is performed via a C library 'free()'. This function may be useful if a type allocates additional private data in its copy routine, which is preferably freed on property destruction. This function cannot fail.

Via carefully written copy/free functions, types can refer to memory that is not handled by the DevBot property storage system. This allows for dynamically sized types, or those which refer to a common data source.

Consequently, serialization functions should take into consideration the untangling of external memory references, and not ever send pointers over network channels (even at the expense of memory duplication). These functions should also consider issues such as endian safety.

DevBot types should refer to universally fixed-size structures for maximum cross architecture and cross platform support.

Serialization functions may also need to consider compactness of the resulting serialized data. Ideally, properties that receive frequent updates should fit into a single 512-byte UDP packet, so that DevBot does not have to switch to TCP for transmitting updates. (Of course, should this not be possible, DevBot supports updates of any size over TCP channels.)

Implementing a fixed-width type

This section covers developing a fixed width type. Arrays of any DevBot type can be arbitrarily large, but cannot change in size after property creation time (there are ways around this, explained later).

DevBot already provides built-in primitive types for a variety of types, but a 16-bit wide integer type (short), due to its limited usefulness, is not provided by the core.

The final solution implements both signed and unsigned short types, but only the "short" (signed) type is documented here. Conventional accessors are also implemented, in a similar way to the core types.

DevBot provides the following "raw" accessors for all types, which lack type safety and should be used with care:

bot_property_get (obtains all units from an array property)
bot_property_get_slice (obtains a range of units from an array property)
bot_property_set (sets all units in an array property)
bot_property_set_slice (sets a range of units in an array property)

Declaring your type

All types should define a constant BotPropertyType structure. This should be static to the type's module:

static const BotPropertyType bot_type_short = {
    .name = BOT_TYPE_SHORT_NAME,
    .size = sizeof(BOT_TYPE_SHORT_TYPE),
    .serialize = short_serialize,
    .deserialize = short_deserialize,
};

An accessor should also be provided:

const BotPropertyType *bot_short_get_type(void)
{
    return &bot_type_short;
}

The function can be used with bot_type_register to register a type with DevBot, and enable its use internally, providing network transparency. It is not recommended to use bot_type_short extshort.c directly, as this is not binding-friendly and will make your type harder to use from non-C programming languages.

#define BOT_TYPE_SHORT_NAME     "short"
#define BOT_TYPE_SHORT_TYPE     int16_t

Conventionally, types break out constant expressions as macros. This allows your type to be used unambiguously. Ordinarily, these fragments are put into the type's header file.

Implementing type (de)serializers

Next, the short_serialize and short_deserialize functions must be implemented. These functions can be static to the containing file. Both functions must be capable of handling arrays.

static char *short_serialize(void *storage, uint32_t length)
{
    return bot_type_csv_serialize(storage, length, short_write);
}

static bool short_deserialize(void *storage, uint32_t length,
                              const char *source)
{
    return bot_type_csv_deserialize(storage, length, source, short_read);
}

Since short is a very simple type to serialize, similar to the other core primitive types, DevBot provides CSV (comma separated values) convenience functions to insert commas and handle variable length string allocation. Many types may not be able to make use of these functions and will need to handle arrays explicitly.

static char *short_write(const uint32_t i, void *storage)
{
    char buf[10], *copy;
    snprintf(buf, 10, "%hd", (short)((int16_t *)storage)[i]);
    copy = malloc(strlen(buf) + 1);
    strcpy(copy, buf);
    return copy;
}

Briefly, the short_write function converts a single "atom" from the array storage into a humanly readable string, and returns it. The bot_type_csv_serialize function appends a comma and produces a complete string serialization. Notice here that the %hd printf modifier is used here to indicate the short data type.

static bool short_read(const uint32_t i, const char *token, void *storage)
{
    char *error;
    ((int16_t *)storage)[i] = (int16_t)strtol(token, &error, 10);
    return (error[0]) ? FALSE : TRUE;
}

The short_read function converts a string back into an array storage atom, and returns bool to indicate deserialization failure. The function bot_type_csv_deserialize propagates this back to the caller. Notice here that the strtol function is employed to deserialize the short.

Providing a public API

Finally, all that is left is to provide a type-safe API for users of your type.

void bot_property_get_short_all(BotProperty *p, BOT_TYPE_SHORT_TYPE *value)
{
    bot_property_get(p, &bot_type_short, value);
}

This accessor obtains all shorts in an array property.

void bot_property_get_short_slice(BotProperty *p, BOT_TYPE_SHORT_TYPE *value,
                                  uint32_t offset, uint32_t length)
{
    bot_property_get_slice(p, &bot_type_short, value, offset, length);
}

This accessor obtains a number of shorts from an offset inside an array property.

static inline BOT_TYPE_SHORT_TYPE bot_property_get_short_at(BotProperty *p,
                                                            uint32_t offset)
{
    BOT_TYPE_SHORT_TYPE value;
    bot_property_get_short_slice(p, &value, offset, 1);
    return value;
}

This accessor obtains a single short from an offset inside an array property.

BOT_TYPE_SHORT_TYPE bot_property_get_short(BotProperty *p)
{
    BOT_TYPE_SHORT_TYPE value;
    bot_property_get(p, &bot_type_short, &value);
    return value;
}

Finally, this accessor obtains a single short from a unit property.

Accessors need not take this form, and these type-safe versions of the core accessors are not strictly necessary.

Pulling it all together

/* Implements the short and ushort types.
 *
 * Copyright 2006 Edinburgh Robotics Ltd.
 */
#include "extshort.h"

#include <stdlib.h>
#include <string.h>
#include <stdio.h>

/* SHORT SERIALIZE/DESERIALIZE ***********************************************/

static char *short_write(const uint32_t i, void *storage)
{
    char buf[10], *copy;
    snprintf(buf, 10, "%hd", (short)((int16_t *)storage)[i]);
    copy = malloc(strlen(buf) + 1);
    strcpy(copy, buf);
    return copy;
}

static char *short_serialize(void *storage, uint32_t length)
{
    return bot_type_csv_serialize(storage, length, short_write);
}

static bool short_read(const uint32_t i, const char *token, void *storage)
{
    char *error;
    ((int16_t *)storage)[i] = (int16_t)strtol(token, &error, 10);
    return (error[0]) ? FALSE : TRUE;
}

static bool short_deserialize(void *storage, uint32_t length,
                              const char *source)
{
    return bot_type_csv_deserialize(storage, length, source, short_read);
}

/* BUILT-IN SHORT TYPE DECLARATION *******************************************/

static const BotPropertyType bot_type_short = {
    .name = BOT_TYPE_SHORT_NAME,
    .size = sizeof(BOT_TYPE_SHORT_TYPE),
    .serialize = short_serialize,
    .deserialize = short_deserialize,
};

/* USHORT SERIALIZE/DESERIALIZE **********************************************/

static char *ushort_write(const uint32_t i, void *storage)
{
    char buf[10], *copy;
    snprintf(buf, 10, "%hu", (unsigned short)((uint16_t *)storage)[i]);
    copy = malloc(strlen(buf) + 1);
    strcpy(copy, buf);
    return copy;
}

static char *ushort_serialize(void *storage, uint32_t length)
{
    return bot_type_csv_serialize(storage, length, ushort_write);
}

static bool ushort_read(const uint32_t i, const char *token, void *storage)
{
    char *error;
    ((uint16_t *)storage)[i] = (uint16_t)strtoul(token, &error, 10);
    return (error[0]) ? FALSE : TRUE;
}

static bool ushort_deserialize(void *storage, uint32_t length,
                               const char *source)
{
    return bot_type_csv_deserialize(storage, length, source, ushort_read);
}

/* BUILT-IN USHORT TYPE DECLARATION ******************************************/

const BotPropertyType bot_type_ushort = {
    .name = BOT_TYPE_USHORT_NAME,
    .size = sizeof(BOT_TYPE_USHORT_TYPE),
    .serialize = ushort_serialize,
    .deserialize = ushort_deserialize,
};

/* PUBLIC SHORT API **********************************************************/

const BotPropertyType *bot_short_get_type(void)
{
    return &bot_type_short;
}

void bot_property_get_short_all(BotProperty *p, BOT_TYPE_SHORT_TYPE *value)
{
    bot_property_get(p, &bot_type_short, value);
}

void bot_property_get_short_slice(BotProperty *p, BOT_TYPE_SHORT_TYPE *value,
                                  uint32_t offset, uint32_t length)
{
    bot_property_get_slice(p, &bot_type_short, value, offset, length);
}

BOT_TYPE_SHORT_TYPE bot_property_get_short(BotProperty *p)
{
    BOT_TYPE_SHORT_TYPE value;
    bot_property_get(p, &bot_type_short, &value);
    return value;
}

void bot_property_set_short_all(BotProperty *p, BOT_TYPE_SHORT_TYPE *value)
{
    bot_property_set(p, &bot_type_short, value);
}

void bot_property_set_short_slice(BotProperty *p, BOT_TYPE_SHORT_TYPE *value,
                                  uint32_t offset, uint32_t length)
{
    bot_property_set_slice(p, &bot_type_short, value, offset, length);
}

void bot_property_set_short(BotProperty *p, BOT_TYPE_SHORT_TYPE value)
{
    bot_property_set(p, &bot_type_short, &value);
}

/* PUBLIC USHORT API *********************************************************/

const BotPropertyType *bot_ushort_get_type(void)
{
    return &bot_type_ushort;
}

void bot_property_get_ushort_all(BotProperty *p, BOT_TYPE_USHORT_TYPE *value)
{
    bot_property_get(p, &bot_type_ushort, value);
}

void bot_property_get_ushort_slice(BotProperty *p, BOT_TYPE_USHORT_TYPE *value,
                                   uint32_t offset, uint32_t length)
{
    bot_property_get_slice(p, &bot_type_ushort, value, offset, length);
}

BOT_TYPE_USHORT_TYPE bot_property_get_ushort(BotProperty *p)
{
    BOT_TYPE_USHORT_TYPE value;
    bot_property_get(p, &bot_type_ushort, &value);
    return value;
}

void bot_property_set_ushort_all(BotProperty *p, BOT_TYPE_USHORT_TYPE *value)
{
    bot_property_set(p, &bot_type_ushort, value);
}

void bot_property_set_ushort_slice(BotProperty *p, BOT_TYPE_USHORT_TYPE *value,
                                   uint32_t offset, uint32_t length)
{
    bot_property_set_slice(p, &bot_type_ushort, value, offset, length);
}

void bot_property_set_ushort(BotProperty *p, BOT_TYPE_USHORT_TYPE value)
{
    bot_property_set(p, &bot_type_ushort, &value);
}

Previous tutorial