I've been doing a node-red application for tracking material data at work (PCB facility).
Currently, I poll data from different machinery at different intervals:
- 3s intervals from 26 machines (for now) that run on Siemens S7 PLCs.
- 15s intervals from 3 databases running Microsoft SQL EXPRESS to get the material stored in smart storage units.
The data gathered is written to a MySQL database that holds a table for each machine type, tracking the timestamp and characteristics of the material at the moment it passed by.
In the future, I'll add several more machines that run on Omron NX PLCs, and possibly gather information from some additional databases from process machinery.
When I started the trials a few weeks ago, I was using the node-red-contrib-ui-led to display the machine status, but the impact in performance was enormous. I was also scanning much more often (500 ms) to catch the machine lights' actual status (they blink on a 1 Hz clock).
If I tried accessing the dashboard from a Raspberry Pi or one of the connected Smart TVs we have in the plant, they would display the "Connection lost" quite often, and miss the refresh. Changing from one tab to the other took a lot of time, even when most tabs only had about 6 to 8 machines on them.
In order to improve performance, I removed all the node-red-contrib-ui-led nodes and changed them to text nodes that used fontawesome icons to simulate lights. I also increased the polling time to 3000ms, and instead of getting the status lights, I scanned the general status and some critical errors I needed. This increased the number of variables I had to read (a few more booleans), but reduced the polling frequency.
Instead of using many test fields on the dashboard, I just used a few and sent them strings with formatted HTML code that used tables to display the information. This allowed me to increase the information density on each group, and only use about 4-5 text nodes per machine instead of 15-16 nodes. This is how it looks currently (the data has been obscured for obvious reasons, but you get an idea):
Each row is a different text node formatted as a table. I used css to apply the card-like borders.
I also started using subflows to standardise things. Basically, for regular automation from the same maker, I have a subflow that looks like this:
The input is connected to each S7 node that polls the PLCs and returns the data I need, the outputs 1-5 are connected to the text nodes in each group, and exits 6 and 7 are used for diagnostics.
You'll notice that I split the Data Tracking nodes (essentially, it's the same database), because the polling is almost syncronous for all machines, and the DB couldn't handle so many requests made at the same time in the same connection. Splitting it like this seemed to fix that particular problem.
On a PC, RPi or Smart TV browser, the performance now it's very similar: the dashboard looks responsive and refreshes often, but from time to time, the page looks as if it missed an update: most data disappears, only to be refreshed in the next poll cycle (about 3s later).
This didn't happen before I included the database part and was only displaying the data in real time. This reminds me of what happened before with the Pis or the Smart TV browsers, except that I don't get the "Connection lost" message
Even though it's not a big problem, it is a bit annoying, specially considering I will have to add more information in the next few months, and was even thinking about using the node-red to handle the Smart Storage units and AGVs in the plant instead of the suppliers' proprietary software (which is terribly buggy, has no documentation (and is only sparsely commented), it is extremely demanding with system resources, and performs poorly).
This would put more stress on node-red. If it can't handle so much data effectively, I could have one node-red server to gather information and another one to handle smart storage and AGV solutions, or even two. I could also split the data-gathering tasks in two (or more) servers for different parts of the factory, with all of them writing the tracking information to the same centralised database.
Since I only have a few weeks' experience with node-red, I would appreciate any input/advice regarding performance improvements.