Node.send not always works

Developing Nodes
1

I am developing a node that receives asynchronous messages from an external system through AMQP and sends the data to the developed node output.

My external message function is triggered perfectly fine every time, but when I call node.send with the created node-red message. It is not always sent (I am testing it with a Debug node, or with another custom node).

The more nodes of this type I have, the percentage of undelivered output messages from all of the is getting higher.

What should be the way to debug it?
Thanks

2

In your function add a node.warn() statement immediately before the node.send() That will output a message in the node red log. Also configure the debug node to output to the console, so that will also appear in the log. Then run your flow and look in the log to see if you see node-warn outputs without the corresponding debug. Note that the warn and the debug may be separated as, if the function loops with multiple sends then they will all appear before the debug output, so they will have to have an identifiable content in order to link them up.

Also tell us what version of node-red and of nodejs you are using and what hardware/OS you are running on.

3

The function does not loop the sends (just one send per externally received message) every 5 seconds. The size of the message is pretty big - ~5kb.

I did this scenario precisely (printing node.warn with a message id after node.send).
I do see node.warn log for 100% of the cases but the debug log for less than 50%.
I use node-red 2.2.2 with node v14.19.0, on Ubuntu 20.04 development machine

Thanks!

4

Is it possible that the object you are attempting to send is null? Or perhaps you are repeatedly sending the same object. If you do that and the object gets sets to null then the previous one sent will also become null.

Otherwise are you able to construct a test flow we can run to see the problem?
If not then can you post the code of the function?

5

Or in fact if you did

node.send(msg)
// some code
msg = null

then by the time the debug node receives it then it may be null. You must not do anything with msg after you send it.

6

In this instance, setting msg to null will not modify anything downstream as it is reassigning what the local variable msg points at, rather than modifying the contents of what msg pointed at.

7

That is true. The contents of msg could be corrupted, but not msg itself.
Perhaps msg is null before the node.send() call then.

8

I am defining the msg with var msg = []; and adding content with msg.push. node.send is the last instruction in the function except node.warn printout. The printout prints the size of the message (${JSON.stringify(msg).length}) so it's certainly not null.

The problem is not that debug node receives a null message, the problem is that it receives every second or third message only. I double tested it with another custom node by manually printing the input into the console.

9

If you send an array to node.send, then it attempts to send each entry in the array to each output pin of your node. The fact your array length is > 0 is irrelevant.

Console log the full msg array and show us the actual content.

10

Please explain what your intention is when you send an array.

11

If you send an array to node.send , then it attempts to send each entry in the array to each output pin of your node. The fact your array length is > 0 is irrelevant.

Console log the full msg array and show us the actual content.

This is the message as it appears after node.send call:

[
  {
    topic: 'combined_sample',
    msg_id: 67829,
    payload: {
      sensor1: -25,
      sensor2: -7.5,
      sensor3: -12,
      ... (160 sensors)
    },
    timestamp: '2022-04-04 16:25:25.922000+00:00',
    _msgid: 'c1db56467e7a9218'
  }
]

It is not always delivered.

_msgid: 'c1db56467e7a9218' is generated by node.send even if the message is not delivered.

12

Sometimes I need to send more than a single message during the same call. Messages are not delivered regardless if it's one or more messages sent together.

13

I am using amqp-connection-manager package which is based on amqplib:

My AMQP message subscription code is based on this example.

The onMessage function is where the node.send called from.

14

It's not (to me) 100% clear where you are losing messages. I can say with absolute certainty that node.send is reliable and will not be discarding messages from your array unless they are empty/Null.

where is this array of messages being sent from? From your custom node? Or from a function node in your flow?

Do you have a repository on line where I can view your code & ora flow that demonstrates this issue?

15

node.send is called from my custom node. I added debug prints to Node.prototype.send = function(msg) within node_modules/@node-red/runtime/lib/nodes/Node.js and it always finishes (it's not returning on error at the beginning).

So the message is not delivered somewhere else (on another node receive maybe?).

Any idea where to look?

I also suspect it is happening when more than a single node of this custom type is used.

(the code is not public so cannot publish)

16

That's going to make it tricky to help you

Unless your prepared to pay someone commercial rates and sign an NDA of course

17

The behavior is even weirder:

The 1st time I am starting the node-red (from command line with nodemon) - all the messages arrive. If I restart the flow (from the web menu) - every second message arrives. Restart once again - every 3rd message arrives and so on....

Any clue?

18

you are not cleaning up whatever listeners you have added when the node is closed (called on deploy) ?

19

you are not cleaning up whatever listeners you have added when the node is closed (called on deploy) ?

What exactly should I clean on node-red side? I do receive all the external messages after every restart. If I enable "metrics" options in settings.json I do see that message is sent from one node (my custom node), but not always received on another (debug) node.

20

you are not cleaning up whatever listeners you have added when the node is closed (called on deploy) ?

It indeed looks like some kind of a cleanup problem - after a restart all messages are sent twice, after 2 restarts 3 times, and so on..... (I discovered it by enabling preRoute hook)