ThingsBoard - Watmonitor HTTP integration

The popular IoT platform ThingsBoard has many options for integrating data from other systems. On the blog, we have already described the methods of the Pull model using a custom HTML widget that periodically retrieves the latest data from Watmonitor, as well as a form of direct writing of device telemetry to ThingsBoard from the Node-RED tool based on the created block flow via MQTT.

As we mentioned then, ThingsBoard is a type of Push platform. This means that the platform is designed primarily for data acquisition, when the request is made by a device that pushes data to the platform. Today we will deal with integrations. Integrations were developed on the ThingsBoard platform mainly for the purpose of mass data collection from various devices that use a specific data format. It was mainly aimed at LPWAN networks such as LoRaWAN, Sigfox, NB-IoT. That is, data entry could be direct or indirect from the device's point of view. This means that if the device could not send data directly to the platform, it was done through an intermediary, most often an application server or cloud backbone depending on the technology.

We will use this model today, although it will not be a mass collection of data from multiple IoT sensors, but from a single platform. The integration itself will be practically portable for importing any Watmonitor instance, as long as the user operates only one interface, so as not to mix data using the same integration, which expects the same data format. For multiple Watmonitor instances, it is necessary to use separate integrations, devices, but it is possible to use one comprehensive dashboard even with different sources of time series data.

1. Creating an integration

Log in to the ThingsBoard platform and select Integrations in the Integrations Center section. In the right corner of the interface, select Add dashboard via the + icon. In the displayed menu, it is necessary to select the type of integration. There are many pre-prepared profiles for various platforms (AWS, The Things Stack Industries / Community, ChirpStack, Sigfox, KPN Things, Azure, Apache Pulsar, Kafka, Loriot, RabbitMQ, iotcreators.com (T-Mobile IoT), Tuya and application protocols CoAP, HTTP, OPC-UA, MQTT..., there is also the possibility of directly using the "low-level" TCP and UDP protocols, i.e. below the application layer of the OSI model.

Watmonitor is a web-oriented application, so we will use HTTP integration. We will set a name for the selected integration and leave the functionalities enabled by default - the integration's permission to create new devices and also to launch the integration, meaning that it will be able to receive requests and perform actions.

The next step will be to set up the Uplink data converter. This is a decoder function that allows you to read the payload and can then work with it and perform actions. The payload is in JSON format by default. It can be a primitive JSON structure, or a JSON field with different levels of data depth. For this purpose, a custom decoder will be created that will know this universal data format and can work with it. In our case, we will send data from Watmonitor with one parameter in JSON - water, to which we will assign a value.

On the Watmonitor side, the value can be differential, i.e. the distance from the water level to the well lid, or total, i.e. from the bottom to the surface, depending on what we want to visualize. Based on this, we can convert the visualization directly, or perform postprocessing to calculate the real water level on the ThingsBoard side, even if we only have the difference.

Nested code under the decoder function function decoder(payload, metadata) { ... } :

/** Decoder **/

// decode payload to JSON

var data = decodeToJson(payload);

var deviceName = 'integration-node';

// Result object

var result = {

deviceName: deviceName,

telemetry: {

level: data.water

}

};

return result;

When creating the code for the decoder, you can also test it directly with sample data, where you can also see the result, whether the telemetry conversion was successful, and whether a specific number appears there that belonged to the one in the sample payload. As you can see, the decoder also defines the integration-node device to which the given telemetry will be attributed.

Since we authorized the integration for creating devices as well, the device will be created even if it does not yet exist under Devices. The decoder itself that we created is called the Uplink data converter. After saving it, the converter will be created under the Data Converters tab in the Integrations Center. The Uplink data converter itself will be assigned as the data converter for the given HTTP integration.

We will skip the next step for creating a downlink data converter by clicking Skip, as we will not be sending data to the device and continue to the last step Connection. Here we see the created HTTP endpoint that ThingsBoard uses for data collection and the actual launch of the integration with its uplink converter. We can also enable secondary verification of the request from an external source by verifying a definable HTTP header and its value. For the sake of demonstration, we will leave it by default without verifying additional HTTP headers. We will create the integration by clicking Add at this point.

2. CURL call from Watmonitor

In order to achieve periodic integration startup and at the same time keep the Push model when we send data to the platform, we will use the CURL call part in the Watmonitor source file data.php, which can send data to the ThingsBoard platform. The CURL call for an HTTP POST request with JSON data is pre-prepared, the only parameter that needs to be changed is the target URL of the ThingsBoard platform to the endpoint on which your integration is running, and you can also use which value you want to send (differential or total).

In this setting, the ThingsBoard is written whenever Watmonitor receives new data from the sensor node (ESP32, ESP8266, Arduino). After writing the data, ThingsBoard returns the HTTP 204 return code, since the endpoint API does not send any payload (No-Content) by default. The API sends only one HTTP header, namely date with the date and time of the request in GMT.

You can see whether the data has been written to ThingsBoard by opening the Devices tab, where one of the indicators may be the Active status of the integration-node device, and also by opening the device menu and clicking on Latest telemetry, you will also see the current data if it has arrived for the level key.

3. Data visualization (telemetry) on ThingsBoard

You can visualize the given data as standard telemetry through Dashboards in any graph that provides the latest telemetry, or even time series data. The integration always saves the data with the current UNIX timestamp, so you can also visualize the data historically for the entire period of operation of the integration.

Widgets working with time series data also allow for aggregation or averaging of data, which can be advantageous for visualizations where the data changes abruptly and its smoothing between transitions is necessary.

If Watmonitor interests you, you can find more information about this web application on the project page: https://your-iot.github.io/Watmonitor/, you can try the project with your hardware (ESP32, Arduino, ESP8266, along with an ultrasonic or laser ToF distance sensor. It is also possible to use more sophisticated solutions based on industrial sensors and solutions that can be visualized directly in Watmonitor with a connection to ThingsBoard.

Uplink data converter used for Watmonitor interface:
https://github.com/martinius96/hladinomer-studna-scripty/blob/master/examples/Tools_Integrations/ThingsBoard/uplink_converter.json