Runtime Value Storage

Sometimes you need to persist state across device start and stop operations. For example, a database polling device may need to remember which record was last read from the database. Or a file ingestion device may need to remember ignored files or the date/time of the last directory poll. Connexion/Remote Agent expose the IKeyValueStorageProvider interface, which can be used to persist state.

The KeyValueStorageProvider implementation stores data in the Connexion database, which is accessible to both Active and Passive nodes (if running a cluster). If using a different mechanism to store state (file system, discrete database, etc.), you should ensure your data source is accessible to both Active and Passive Connexion nodes.

In versions prior to 16.1, the IKeyValueStorageProvider only provides two methods:

string GetConfiguration(Guid configurationKey, string nameSpace);
DateTime SaveConfiguration(Guid configurationKey, string nameSpace, string configuration);

The configurationKey parameter (guid) and nameSpace parameter (string) are the primary key of your configuration record. This combination must be unique within your Connexion instance. If you are storing state for a specific device, you could use the device key (the DeviceKey property is exposed in the BaseDevice) and the Type name of your device as the namespace ("MyCompany.MyDeviceTypeName").

In this implementation, you must provide your own serialization/deserialization to/from a string.

In 16.1, four additional methods have been added:

    GetUserConfigurationResponse<T> GetConfiguration<T>(GetUserConfigurationRequest request);
    SaveUserConfigurationResponse SaveConfiguration<T>(SaveUserConfigurationRequest<T> request);
    Task<GetUserConfigurationResponse<T>> GetConfigurationAsync<T>(GetUserConfigurationRequest request);
    Task<SaveUserConfigurationResponse> SaveConfigurationAsync<T>(SaveUserConfigurationRequest<T> request);

These helper methods take care of the serialization and deserialization for you.

These helper methods will only work if the object you’re storing is serializable. Typically this requires your class to have the [DataContract] attribute, and any properties to have the [DataMember] attribute. Your properties may also need to have setters (which can be private set). In some cases, a parameterless class constructor may also be required.

An example class which can be used to store state might look as follows:

[DataContract]
public class MyDeviceState
{
    [DataMember]
    public string State { get; set; }

    [DataMember]
    public int StateId { get; set; }

    [DataMember]
    public DateTime LastPoll { get; set; }
}

The KeyValueStorageProvider is registered as a singleton via the ServiceProvider. Typically, you would get a handle to the provider with the following (which would typically happen in the device constructor or the device load method):

Typically, devices that store state are source devices. These devices are not passed messages for processing, but instead read data from an external source and post messages on a channel. These types of devices either start their own polling loop within a separate Task, or, use the built-in scheduling call-back to perform work. In these cases it is easy to read the state data:

In the above case, we’re using the async GetConfigurationAsync method. This is the appropriate method for polling devices where work is being performed in an async processing loop or scheduling callback.

If your device is not a source device and processes messages, then you’ll typically read your state in the device’s Start() method. In this case, we recommend using the non-async GetConfiguration(...) method.

You should not be doing any significant blocking in the device Start() or Stop() methods. The non-async Get/SaveConfiguration() methods are blocking calls to the database, and as such you should not be reading or storing large payloads using the KeyValueStorageProvider. It is possible to use the Get/SaveConfigurationAsync methods by awaiting within the ProcessMessagesAsync method, however, this is an advanced pattern which requires careful exception handling. Please get in touch if you need to use this pattern.

Storing configuration can happen when the device is stopping:

Or, you may want to store the configuration more frequently (within a polling loop, for example). Keep in mind that if the process crashes, you may have stale state and end up processing the same data multiple times.