Introduction
Node-RED, being a flow-based, low-code development tool for visual programming, offers us the advantage of effortlessly crafting projects with expansive capabilities. Our approach involves extracting raw values from the Data Layer at a high frequency, aggregating them within Node-RED, and transmitting them to InfluxDB for storage and visualization.
Requirements
- ctrlX CORE - X3 or X7 or a ctrlX OS installation
- Node-RED
- InfluxDB or Nexeed IAS
InfluxDB
Step 1: Install and configure Node-Red and InfluxDB
The initial step involves installing the required apps and configuring them. As the primary focus of the article is not on this setup process, the correct configuration can be ensured by referring to the following link: Use ctrlX CORE as a monitoring platform using InfluxDB and Grafana
Step 2: The Flow
To illustrate the process, we will utilize 10 specific data points: a single PLC counter function and nine consistently present system values. The PLC counter has been set to increment every milisecond.
Within our workflow, we opted for Node-RED Data Layer Subscribe nodes over Data Layer Request nodes due to their demonstrated efficiency and lower impact on the system's CPU. Since the Subscribe node transmits a value solely when a change occurs, we must manually dispatch the same value during periods of inactivity, when no changes are detected. To accomplish this, we employ both a "change" node and an "inject" node set at a 100ms interval. Additionally, all Subscribe nodes are configured to publish at the same 100ms interval.
Following the inject node, the values are held in the "batch" node until 10 values accumulate. They are then transmitted to the first join node, where they are organized into an array. After this initial join, all the individual arrays containing data points are merged within the second "join" node, resulting in a consolidated large array.
The provided image illustrates a segment of the flow featuring all the requisite configurations.
The CORE CPU is significantly impacted during these processes, specifically when retrieving values from the Data Layer and transmitting them to InfluxDB. While we have optimized the former using Subscribe nodes, a parallel optimization is now necessary for InfluxDB. Rather than using separate "influxdb out" nodes for each data point, the approach is to consolidate them into a single node. However, before implementing this consolidation, it is essential to include a "function" node to configure the line protocol. This part is explained in Step 2.
In the end, all data flows into InfluxDB over the "influxdb out" node and looks like this:
Step 3: The Function
After collecting and combining the data points, we need to modify the output to align with the format accepted by InfluxDB, known as the Line Protocol.
The correct Line Protocol for InfluxDB is exemplified in the following link.
To accommodate the changes, a "function" node is employed, and the output is configured accordingly. An illustrative example can be observed in the picture below.
Nexeed IAS
The motivation for this project stemmed from the exploration of sending multi-dimensional arrays to a Nexeed IAS server, showcasing the significant possibilities. Utilizing the same process, a different function node was implemented to adapt to the Line Protocol specifications employed by Nexeed IAS.
In this demonstration, we have 80 data points with a 100ms sampling rate. Utilizing our approach to minimize the impact on CPU, we successfully transmitted the data to the server with negligible loss. However, it's worth noting that at this point, we were approaching the limits of Node-RED.
80 data points in Node-RED
One part of the flowFollowing the function node, where the arrays are formatted into the correct line protocol, the data is transmitted to Nexeed, resulting in the following representation.
To give it a try, please copy the content from the attachments and paste it into Node-RED.
