Integrating Watmonitor into Node-RED via JSON endpoint

Node-RED is a tool that enables data acquisition, transformation and visualization in real time. It is designed as a low-code platform for creating event-driven applications. The basic principle is to compose blocks (nodes) into flows that provide the required automation. It is used not only in small projects, but also in large scalable industrial solutions. Node-RED has an extensive community and libraries that contain pre-prepared flows, as well as separate blocks with functions. You can easily add, customize or use them as a basis if you do not want to build the entire logic from scratch.

Today we will therefore show how to integrate the water level meter - Watmonitor project through JSON enpoint into Node-RED along with subsequent automation. First of all, it is necessary to install Node-RED into the operating system. This is usually done in one of three ways:

• npm - under node.js (the most common option)
• Docker
• Snap

The most used is node.js with its npm command (package manager) we can install Node-RED into the operating system as a global module. The command for installing node-red: npm install -g --unsafe-perm node-red. (If you are using Linux, do not forget to sudo before the command). You can then run the installed Node-RED (version 4.1.0) via the command line (CMD) by entering the node-red command. The server will start on the localhost callback (127.0.0.1) and port 1880. The Node-RED environment is available through a browser.

The Node-RED interface is intuitive. On the left side, you have blocks that you drag and drop onto your desktop. After clicking on each block, you can set its parameters, if any. You can view the documentation for each block directly in the interface.

The blocks are divided by category:

• common
• function
• network
• sequence
• parser 
• storage

We will create a basic flow that will consist of a 5-minute routine, an HTTP(S) GET request, a JSON parser. This flow will periodically retrieve data from Watmonitor and its JSON endpoint json_output.php, which by default carries the last known data.

Basic flow
From the common category, we will select the Inject node. This node is intended for manual or automated message sending, thus functioning as a "TICK" generator for launching the following block. It can send messages of type msg.payload or msg.topic to another node. The payload carries the information itself, for example an object, value, text.

The topic essentially carries the context that defines where the data belongs / what it belongs to. This is especially useful in large automations, where messages have to be filtered because they come via different channels from various IoT devices, etc. It is best to imagine this with an example: msg.payload: 21.4, msg.topic: "bedroom_temperature".

We will leave the Inject node in a predefined state, where it sends the number of milliseconds since the epoch (Unix timestamp) in msg.payload. At the bottom, we will adjust the interval to 5 minutes and we can also activate inject once after 0.1 seconds. This setting will allow inject to be run even after the application is deployed and we do not have to wait 5 minutes for the first iteration to run.

The second block will be the network node http request. After clicking on it, we will enter the HTTP method GET, the URL of the Watmonitor instance and its JSON endpoint. The request will be run every time a message arrives from the Inject node, i.e. once every 5 minutes. The request will be executed, the server response (payload) will be obtained and forwarded.

The http request node itself does nothing other than request. That is, it does not even parse data in JSON format, in which Watmonitor distributes it. For this, it is necessary to use a node from the parser category, specifically json. The json node has a preset action "Convert between JSON String & Object", which is the best solution, since based on the input data format it knows whether to make a String or an Object. We connect the individual flow nodes to each other to chain their functionality

To know what each node is doing, and how it affects the payload, or rather what it sends to the next node, you can also add debug nodes to the flow. These allow us to get a better view of what is happening. In the right part, we can select the Debug section, which will list what is happening in the given flow. A flow built in this way gives us the basic functionality of periodic running, obtaining data and parsing it into a JSON object, which we can work with further.

In the desired automation, we can add input evaluation to perform an action. If the water level is too low, we can create a notification or run a subroutine. From the function category, we select the function node. After clicking on it, we can see that this node requires the creation of a subroutine and we get to low-level programming, which is commonly used in Node-RED. The subroutine can be entered for various events (On Start, On Message, On Stop). It also has a setup where you can set the timeout and other parameters.

We can also use the function node for debugging, where we can print out the input data if we are not using the debug node. It can also make the development phase easier, where you can first print out the data and then you know how to work with it based on whether it came as a payload or topic. If the code is not executable, it will print out information about it with the given error.

Using a function, we can evaluate a specific value that belongs to a key and perform an action. For example, to reformat the output and print out whether the level is sufficient, lacking, or excessive. The format can be a text (string) or a separate JSON object depending on what the other object expects. Scripting and working with data in functions is very simple. The paradigm is regular Javascript.

But how about sending data from Node-RED to ThingsBoard via the MQTT protocol? First of all, we need to know the information that the MQTT connection will require. Specifically, it will be the host (MQTT Broker), topic v1/devices/me/telemetry and also authentication via token. The token is used to identify the user (username) in the MQTT connection.

In Node-RED, we will use a node named mqtt out of the network category and connect it to the output of the function node. This node allows us to send (Publish) data to the MQTT Broker. In the node settings, we need to create a new connection. In the host, we will enter thingsboard.cloud and the standard unencrypted MQTT port 1883, but we can also choose the encrypted 8883. Then we will also enter the topic v1/devices/me/telemetry.

In the Security tab, under the username, we will enter the token belonging to the created device on ThingsBoard. The node sends msg.payload automatically, so there is no need to set anything. It needs to be in a compatible format from the function node output.

The mqtt out node itself does not have an output for connecting a debug node, but it has its own status bar that shows whether it is connected to the MQTT broker on ThingsBoard and it can also be used for a certain form of debugging. The data sent as payload is written to ThingsBoard in the telemetry of the relevant device and can be further worked with, included in visualizations, etc. At the same time, it is time-series data, so it is always saved with a time stamp on ThingsBoard, which means that it can also be used for a graph in the dashboard.

When outputting data to the ThingsBoard dashboard, you can use variables directly, or you can also use postprocessing of the variable, where you can adjust the value with a separate script. In this case, for the well filling level, the measured value / well depth (x100) is calculated, which gives us a percentage expression of the well filling.

In a similar way, it is possible to use the http request node, which can write data to ThingsBoard via REST API, but it can also be used for other actions such as launching webhooks (IFTTT, Zapier, n8n and the like), which can bring an interesting and fast way of informing the user about actions that have occurred in Watmonitor. You can easily extend Node-RED like an octopus.

Export of nodes and flows from Node-RED can be found at: 
https://github.com/martinius96/hladinomer-studna-scripty/blob/master/examples/Tools_Integrations/ThingsBoard/Node-RED.json

You can try the Watmonitor project for free with your hardware at:
https://your-iot.github.io/Watmonitor/