Another migration thread, I guess

I'm triying to migrate my whole project to Dasboard 2.0 and I am facing several challenges. Many of them I suppose I will manage to solve eventually but one that worries me is a function node that is in charge of managing a queue of ModBus requests. For some strange reason, what worked perfectly in 1.0 in 2.0 starts working fine but gets progressively stuck until it stops and I don't know where to attack it to find the reason. Can anyone tell me what has changed from 1.0 to 2.0 to cause this behaviour? If I restart nodered.service it works again for a while.

    {
        "id": "36d2ea0f0eaf0c33",
        "type": "function",
        "z": "4c2326a48b5b1cea",
        "g": "7f2ab856c5b1c9d6",
        "name": "Modbus Queue",
        "func": "let resendifnoresponse = true; // Reenviar el mensaje si no se recibe respuesta\nlet resendinterval = 10; // Reenviar Ășltimo mensaje cada x segundos\nlet online_threshold = 10; // Segundos entre actualizaciones por debajo de las que el dispositivo se considera online\nlet offline_threshold = 300; // Segundos entre actualizaciones por encima de las que el dispositivo se considera offline\n\nlet notifmsg = null;\n\n// Comprobar que el mensaje recibido tiene un topic\nif ((msg.topic === \"\") || (msg.topic === null) || (msg.topic === undefined)) {\n    node.status({ fill: \"red\", shape: \"dot\", text: \"Topic missing\" });\n    return;\n}\n\nlet lastupdate = context.get(\"lastupdate\");\nlet state = context.get(\"state\") | 0;\nlet queue = context.get(\"queue\");\nlet queuecount = 0;\nif (queue === undefined) {\n    queue = [];\n} else {\n    if (Array.isArray(queue)) {\n        queuecount = queue.length;\n    } else {\n        queue = [];\n    }\n}\nlet current = new Date().getTime();\nlet send = false;\n\nswitch (msg.topic.toLowerCase()) {\n    case \"update\":\n        // Actualizar timer y estadĂ­sticas\n\n        if (lastupdate !== undefined) {\n            notifmsg = { \"topic\": \"Information\", \"payload\": {} };\n            current = current - lastupdate;\n            current = Math.floor(current / 1000);\n            notifmsg.payload.secondsincelastupdate = current;\n            var minute = Math.floor(current / 60);\n            var hour = Math.floor(minute / 60);\n            var day = Math.floor(hour / 24);\n            if (current > 24 * 60 * 60) {\n                notifmsg.payload.updatetext = \"Last update \" + day + \" days, \" + hour % 24 + \" hours, \" + minute % 60 + \" minutes, \" + current % 60 + \" seconds ago\";\n            } else if (current > 60 * 60) {\n                notifmsg.payload.updatetext = \"Last update \" + hour % 24 + \" hours, \" + minute % 60 + \" minutes, \" + current % 60 + \" seconds ago\";\n            } else if (current > 60) {\n                notifmsg.payload.updatetext = \"Last update \" + minute % 60 + \" minutes, \" + current % 60 + \" seconds ago\";\n            } else {\n                notifmsg.payload.updatetext = \"Last update \" + current % 60 + \" seconds ago\";\n            }\n\n            // Reenviar el Ășltimo mensaje si no hay respuesta del servidor\n            if (resendifnoresponse) {\n                if ((current > 0) && (current % resendinterval === 0)) {\n                    let lastmsg = context.get(\"lastmsg\");\n                    if ((lastmsg !== undefined) && (context.get(\"sent\"))) {\n                        notifmsg.payload.resend = true;\n                        if ((lastmsg.payload.fc === 1) || (lastmsg.payload.fc === 2) || (lastmsg.payload.fc === 3) || (lastmsg.payload.fc === 4)) {\n                            // Es una solicitud de lectura modbus\n                            node.status({ fill: \"green\", shape: \"dot\", text: \"Read re-sent!\" });\n                            return [lastmsg, null, notifmsg];\n                        } else {\n                            // Es una solicitud de escritura modbus\n                            node.status({ fill: \"green\", shape: \"dot\", text: \"Write re-sent!\" });\n                            return [null, lastmsg, notifmsg];\n                        }\n                    }\n                }\n            }\n\n            // Comprobar si el estado es online\n            if (state !== 1) {\n                if (current < online_threshold) {\n                    notifmsg.topic = \"Warning\";\n                    notifmsg.payload.text = \"Device is now online\";\n                    notifmsg.payload.statuschange = true;\n                    state = 1;\n                    context.set(\"state\", state);\n                }\n            } else {\n                if (current > offline_threshold) {\n                    notifmsg.topic = \"Error\";\n                    notifmsg.payload.text = \"Device is not transmitting\";\n                    notifmsg.payload.statuschange = true;\n                    state = 99;\n                    context.set(\"state\", state);\n                }\n            }\n            notifmsg.payload.state = state;\n            if (state === 1) {\n                node.status({ fill: \"blue\", shape: \"ring\", text: queuecount + \" | \" + notifmsg.payload.updatetext });\n            } else {\n                node.status({ fill: \"red\", shape: \"ring\", text: queuecount + \" | \" + notifmsg.payload.updatetext });\n            }\n            return [null, null, notifmsg];\n\n        } else {\n            node.status({ fill: \"grey\", shape: \"ring\", text: \"No data\" });\n        }\n        break;\n    case \"next\":\n        // Actualizar el contador lastupdate\n        context.set(\"lastupdate\", current);\n        context.set(\"sent\", false);\n        send = true;\n        break;\n    case \"reset\":\n        context.set(\"queue\", []);\n        context.set(\"sent\", false);\n        context.set(\"lastmsg\", undefined);\n        break;\n    default:\n        // El mensaje recibido es una solicitud modbus\n\n        // borramos el Ășltimo msg para evitar el reenvĂ­o\n        // context.set(\"lastmsg\", undefined);\n\n        // Comprobamos si hay un mensaje en la cola con el mismo topic. \n        // Si lo hubiera lo borramos, sĂłlo nos quedamos con el Ășltimo.\n        for (let i = queue.length - 1; i >= 0; i--) {\n            if (queue[i].topic === msg.topic) {\n                queue.splice(i, 1);\n            }\n        }\n\n        // Añadimos el mensaje al final de la cola\n        queue.push(msg);\n        context.set(\"queue\", queue);\n\n        if (!context.get(\"sent\")) {\n            send = true;\n        }\n        node.status({ fill: \"green\", shape: \"dot\", text: queue.length });\n\n}\n\n// Enviamos un nuevo mensaje\nif (send) {\n\n    if (queue.length > 0) {\n        // Cogemos el mensaje mĂĄs antiguo de la cola\n        let newmsg = queue[0];\n        // Lo borramos\n        queue.splice(0, 1);\n        context.set(\"queue\", queue);\n        context.set(\"sent\", true);\n        context.set(\"lastmsg\", newmsg);\n\n        if (((newmsg.payload.fc === 1) || (newmsg.payload.fc === 2) || (newmsg.payload.fc === 3) || (newmsg.payload.fc === 4)) && newmsg.payload.unitid === 1) {\n            // Esta es una solicitud de lectura modbus de la tarjeta 1\n            node.status({ fill: \"green\", shape: \"dot\", text: \"Read L1 sent!\" });\n            return [newmsg, null, null, null, null];\n        } else {\n            if (((newmsg.payload.fc === 1) || (newmsg.payload.fc === 2) || (newmsg.payload.fc === 3) || (newmsg.payload.fc === 4)) && newmsg.payload.unitid === 2) {\n                // Esta es una solicitud de lectura modbus de la tarjeta 2\n                node.status({ fill: \"green\", shape: \"dot\", text: \"Read L2 sent!\" });\n                return [null, null, null, newmsg, null];\n            } else {\n                if (((newmsg.payload.fc === 1) || (newmsg.payload.fc === 2) || (newmsg.payload.fc === 3) || (newmsg.payload.fc === 4)) && newmsg.payload.unitid === 3) {\n                    // Esta es una solicitud de lectura modbus de la tarjeta 3\n                    node.status({ fill: \"green\", shape: \"dot\", text: \"Read L3 sent!\" });\n                    return [null, null, null, null, newmsg];\n                } else {\n                    // Esta es una solicitud de escritura modbus\n                    node.status({ fill: \"green\", shape: \"dot\", text: \"Write sent!\" });\n                    return [null, newmsg, null, null, null];\n                }\n            }\n\n        }\n\n    }\n}",
        "outputs": 5,
        "noerr": 0,
        "initialize": "",
        "finalize": "",
        "libs": [],
        "x": 5920,
        "y": 1940,
        "wires": [
            [
                "8096c34f09c80413",
                "4ce649f482e59fd5"
            ],
            [
                "bac66e961248e3ef",
                "b64fafb95a8e2ffb"
            ],
            [],
            [
                "8096c34f09c80413",
                "af2fa96517c8f77d"
            ],
            [
                "8096c34f09c80413",
                "c5999de5501876ef"
            ]
        ],
        "info": "# Modbus Queue\r\n\r\nThis node queueing read and write messages for modbus. Use this node if you are reading and writing the same device with many different requests. E.g. reading different coil/register intervals continously and also writing to the device at the same time.\r\n\r\nIt does a few things:\r\n- queues all messages arrive on the input port\r\n- based on the msg.topic, older messages of the same topci is ignored\r\n- sends out the oldest message and waits for the \r\n- monitors the time since last message and send out report on the output\r\n- handles online/offline status\r\n- resend the last message is response is not received in time\r\n\r\n## Input Data\r\n\r\n### payload\r\n\r\nThe payload should contain the data that gets sent to the flex-getter or flex-write node.\r\nTypical modbus read payload:\r\n`{\"value\":0,\"fc\":3,\"unitid\":1,\"address\":1000,\"quantity\":20}`\r\nTypical modbus write payload:\r\n`{\"value\":false,\"fc\":5,\"unitid\":1,\"address\":0,\"quantity\":1}`\r\n\r\n### topic\r\n\r\nEach message must contain a topic (any text), and this topic is used to identify the different read/write requests and delete any earlier request with the same topic if it still in the queueing\r\n\r\nThere are a few reserved topic for special function (for these payload is ignored):\r\n- reset: resets the queue and deleted any data collected so far\r\n- next: this is the message fed back from the flex getter/write node to indicate to this node that a new message can be sent out\r\n- update: this should be coming from a 1 second time to display the current queue count, time since the last update and online/offline status\r\n\r\n## Output ports\r\n\r\n### Port 1: flex getter\r\n\r\nThis output should be connected to a modbos-flex-getter and all the read requests will be sent out through this port\r\n\r\n### Port 2: flex write\r\n\r\nThis output should be connected to a modbos-flex-write and all the write requests will be sent out through this port\r\n\r\n### Port 3: status messages\r\n\r\nThis port outputs a status message for every update message (msg.topic=\"update\").\r\n\r\n- topic: \"Information\" for regular updates, \"Warning\": offline device is now back online, \"Error\": device is offline\r\n- payload.text: message like when the device gone offline, or back online\r\n- payload.updatetext: time passed since the last update (human readable format)\r\n- payload.secondsincelastupdate: number of seconds since the last update from the device\r\n- payload.statuschange: true if status is changed (gone offline, back online)\r\n- payload.state: 0: initial state, no data yet, 1: device online, 99: device offline\r\n\r\n## Node Settings\r\n\r\nChange the settings in the first 4 lines of the code to influence the behaviour. Explanation is in the code as comment."
    }
]```

A function by itself has nothing to do with dashboard (function runs server side, dashboard runs client side).

What does that mean? Screenshots might help you explain.

That is my assumption, it has nothing to do with migration, but it has to be related in some way. The expected behaviour is that, most of the time, it will send read requests to three different slaves and I see that under node status. This status should show the executed order (Read L1 sent, Read L2 sent, Read L3 sent or Write sent) or the size of the queue. On a machine running Dasboard 1.0 I can see the three consecutive "reads" and no queue at all (worst case scenario 1). On the machine running Dasboard 2.0, as I said, initially works great but after some minutes it starts to lose "reads" (at this moment I cannot see "Read L3 sent") and the message queue ranges from 4 to 10 entries.

Finally, while I was writing this post, it hangs, passing the readings erratically. From sending 9 readings per second, it changed to sending 1 every 3 or 4 seconds.
As I have used the migration script and I have only adapted gauges, text and chart nodes, without touching the logic or other nodes that the script automatically converts, I have no idea where to start.

In my testing last night, I realised that it's not so much that node as the general functioning of node--RED. I am still investigating.
PS: CPU usage is much higher in Dasboard 2.0 than in 1.0. 160% vs. 20%. After restarting nodered.service, I can see a normal usage (little higher than 1.0) but it increases quickly. The only difference I see that increases the size of the flow is that Dashboard 2.0 needs function nodes before gauges and text nodes
P.S2: and indeed, if I disable the visualisation part of the flow, it lowers the CPU usage to less than 10%. I will need to work on this part, any suggestions?

In the end, the partial solution was to reduce the number of function nodes attacking the display nodes by joining them into one with several outputs. By doing this, the CPU usage has been reduced a bit (very little) but enough to make it somewhat usable node-RED. In my case Dasboard 2.0 is not useful on a small Raspberry and I will stick with Dashboard 1.0 for the time being.

What pi are you running on? Are you running the browser on the pi?

Rapsberry Pi 4 with Raspberry Pi OS 64-bit. And, yes, it has to display locally some data from the machine to which it is connected.
In fact, I had to create an empty home screen (it only shows an image) to reduce the CPU usage of node-RED to around 20% at idle. A tab with 6 gauges makes it go up to 80%, with Dashboard 1.0, of course (the exposed problem, with Dashboard 2.0, already raised the CPU consumption without even having the browser open).

I run a pi4 (4GB) with dozens of dashboard 2.0 nodes, including a tab with 6 classic-gauges and node-red consumes typically about 1% CPU. Most dashboard nodes are updated only once or twice a minute though.

Are you building charts on the dashboard?

Yes, I have 6 graphs showing data from, at most, a couple of hours. Anyway, I emphasise that Dashboard 1.0 has the same charts.

It is known that the chart has performance issues, which are going to addressed at some point, I believe, but it is not trivial. If you temporarily disable the chart nodes does it make a noticeable difference?

I,ve just tried and yes, there is a big difference. Certainly charts are to blame for this situation. I will continue without them for the time being

How many lines are there on each chart and how often are they updated?

1 line per chart. They are updated every second

Four hours at one/second is 7200 samples across the chart, which is a little excessive since I guess your chart is only hundreds of pixels wide, so you would probably improve matters a lot if you rate limited the chart inputs to 1 every 10 seconds.

Edit: It isn't 7200, it is 14,400, which is definitely excessive. With 6 charts that is 86,400 points being shuffled around every second!

1 Like