Delay node - rate limit not working for all messages?

Tried using delay node to limit how many messages are passed through. I want to discard all intermediate messages.

Action: Rate Limit
All Messages
Rate: 1 msg per 1 second
Drop intermediate messages

Ì then go on to send 10 messages at the same time, but it builds up queue! Why?

Was able to solve it by using "For each msg.stopic", but then needed to add a constant dummy topic. Is topic required to drop intermediate messages?

How do you send 10 messages at the same time?
It doesn't behave like that for me, no messages are queued.

What version of Node-red do you have?

Also please create a small flow that exhibits the problem. Export it and paste it here so we can test it.

Just make a list of objects in function node and send that. Using node red 4.0.5.

[
    {
        "id": "5704b41a79567472",
        "type": "tab",
        "label": "Flow 2",
        "disabled": false,
        "info": "",
        "env": []
    },
    {
        "id": "e8d25d6757046128",
        "type": "inject",
        "z": "5704b41a79567472",
        "name": "",
        "props": [
            {
                "p": "timestamp",
                "v": "",
                "vt": "date"
            }
        ],
        "repeat": "",
        "crontab": "",
        "once": false,
        "onceDelay": 0.1,
        "topic": "",
        "x": 190,
        "y": 120,
        "wires": [
            [
                "ec9f31558ba34498",
                "1ce0a7c7d16f529c"
            ]
        ]
    },
    {
        "id": "1ce0a7c7d16f529c",
        "type": "function",
        "z": "5704b41a79567472",
        "name": "function 22",
        "func": "const messages = [];\nfor(let i = 0; i < 10; i++){\n    messages.push({\n        nr: i+1\n    });\n}\nreturn [messages];",
        "outputs": 1,
        "timeout": 0,
        "noerr": 0,
        "initialize": "",
        "finalize": "",
        "libs": [],
        "x": 590,
        "y": 120,
        "wires": [
            [
                "492b126bca0438ae"
            ]
        ]
    },
    {
        "id": "60c6aa58e443d024",
        "type": "debug",
        "z": "5704b41a79567472",
        "name": "debug",
        "active": true,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "true",
        "targetType": "full",
        "statusVal": "",
        "statusType": "auto",
        "x": 910,
        "y": 120,
        "wires": []
    },
    {
        "id": "492b126bca0438ae",
        "type": "delay",
        "z": "5704b41a79567472",
        "name": "",
        "pauseType": "rate",
        "timeout": "5",
        "timeoutUnits": "seconds",
        "rate": "1",
        "nbRateUnits": "1",
        "rateUnits": "second",
        "randomFirst": "1",
        "randomLast": "5",
        "randomUnits": "seconds",
        "drop": true,
        "allowrate": false,
        "outputs": 1,
        "x": 770,
        "y": 120,
        "wires": [
            [
                "60c6aa58e443d024"
            ]
        ]
    },
    {
        "id": "ec9f31558ba34498",
        "type": "delay",
        "z": "5704b41a79567472",
        "name": "",
        "pauseType": "delay",
        "timeout": "500",
        "timeoutUnits": "milliseconds",
        "rate": "1",
        "nbRateUnits": "1",
        "rateUnits": "second",
        "randomFirst": "1",
        "randomLast": "5",
        "randomUnits": "seconds",
        "drop": false,
        "allowrate": false,
        "outputs": 1,
        "x": 390,
        "y": 160,
        "wires": [
            [
                "1ce0a7c7d16f529c"
            ]
        ]
    }
]

That works for me (I see just one message in the debug node).
As @jbudd asked, please tell us more about your system.
In fact please post the startup log that you get if you start node-red in a command window. It should start with Welcome to node red. That will tell us a lot.

Here is startup logs:

2024-11-25 12:21:17 Welcome to Node-RED
2024-11-25 12:21:17 ===================
2024-11-25 12:21:17
2024-11-25 12:21:17 25 Nov 11:21:17 - [info] Node-RED version: v4.0.5
2024-11-25 12:21:17 25 Nov 11:21:17 - [info] Node.js version: v20.18.0
2024-11-25 12:21:17 25 Nov 11:21:17 - [info] Linux 5.15.146.1-microsoft-standard-WSL2 x64 LE
2024-11-25 12:21:17 25 Nov 11:21:17 - [info] Loading palette nodes
2024-11-25 12:21:29 25 Nov 11:21:29 - [info] Dashboard version 3.6.5 started at /ui
2024-11-25 12:21:30 25 Nov 11:21:30 - [info] Settings file : /data/settings.js
2024-11-25 12:21:30 25 Nov 11:21:30 - [info] Context store : 'default' [module=memory]
2024-11-25 12:21:30 25 Nov 11:21:30 - [info] User directory : /data
2024-11-25 12:21:30 25 Nov 11:21:30 - [warn] Projects disabled : editorTheme.projects.enabled=false
2024-11-25 12:21:30 25 Nov 11:21:30 - [info] Flows file : /data/flows.json
2024-11-25 12:21:30 25 Nov 11:21:30 - [info] Server now running at http://127.0.0.1:1880/
2024-11-25 12:21:30 25 Nov 11:21:30 - [warn] Using unencrypted credentials
2024-11-25 12:21:30 25 Nov 11:21:30 - [info] Starting flows
2024-11-25 12:21:34 25 Nov 11:21:34 - [info] Started flows

I'm running node red from custom docker image (mostly based on the official custom docker image).

Here you can see queue built up to 20, then go down, sending 1 msg per second:

I have no idea why you are seeing that. It works fine for me with the same version of node-red.

If you re-import the flow you posted, telling it to make copies of the duplicate nodes, does the import also behave the way you describe?

I already reimported the flow into another container so I could get the startup logs. Will try to clone messages instead if that's what you meant?

Edit: Still getting same msg._msgid after clone. Also don't seem to be able to modify that value.

I only get one message output from your flow too.

Is there some reason why you are neither injecting msg.payload nor creating a payload in the messages you send from the function?

Does it make any difference if you change your function to this?

for(let i = 0; i < 10; i++){
    msg = {"nr": i+1}
    node.send (msg)
}

Can try that. Reason I don't do payload is because payload is irrelevant.

Fair enough. Why is msg.timestamp relevant?

It isn't. It's just a dummy variable to trigger the function node. I have a habit of renaming default payload from inject node because in many situations I will require that name for something else downstream in the flow. And for inject I just need to trigger and not take up variable names like payload to be reserved for later use.

Edit: Tried with modified loop to use send() to dispatch messages immediately in loop. Still same behaviour.

What I get is basically "delay each message 1 second".

I don't think it's relevant to this issue, but you don't actually need anything in the inject node to trigger the flow:

1 Like

Thanks, learned something new!

So onto the topic, I get different result than everyone else... that's scary!

Perhaps flip it around, you make an example that sends a handful of messages, but only passes through 1 message (per time interval). Then let me import your example and see how it behaves?

Tried to play around with "Send intermediate messages on 2nd output", and perhaps as expected, it doesn't send anything on 2nd output. All messages go through output 1 regardless (queued with 1 sec delay between each message).

I'm gonna try with stock nr container like this:

 docker run -it -p 1880:1880 -v node_red_data:/data --name mynodered nodered/node-red

And see if it changes anything.

Edit: And now it works! So I've done something to nr / nr container to make delay node behave differently :open_mouth:

You had it failing on two containers. Does it still fail on one of them?

It fails on all my custom containers. It works on all stock containers. I'm going to try to apply varies bits and bobs one by one and see if/when it changes everything.

If you track down what actually changes the behaviour, please let us know.
It does seem rather undesirable if core nodes work differently!

1 Like

I just had to enable settings.js to trigger the behavior. So now I need to toggle each change in that file to find the source.

And there it is:

nodeMessageBufferMaxLength: 100,

Enabling this setting makes delay node not drop intermediate messages. I want a set limit to avoid memory filling up from bugs or tests. But I don't want this limit to prevent discarding intermediate messages.

To summarize, enabling this setting makes "Drop intermediate messages" not work. It also makes "Send intermediate messages to 2nd output" also not work. However, it still works when using grouping on msg.topic (ie. drop intermediate messages per topic).

Can anyone enable this setting and verify delay node changes behavior?