Message processing sequence

Hi All,

Let's suppose I have a flow with a switch having two conditions:

msg.payload>flow.a
msg.payload<=flow.a

where flow.a in the beginning is 0. If msg.payload>flow.a then the value of msg.payload gets assigned to flow.a in a function node after the switch.

There are two incoming messages which have a payload that equals to 1.

The question: is it possible that the first message passes the switch (msg.payload>flow.a) but the function node does not assign the value of msg.payload to flow.a by the time the second message hits the switch and gets evaluated?

Thanks&regards,
r0ller

Yes, that is entirely possible.

Thanks for the quick reply! I guess I need to build a queue from the messages to force sequential processing.

One more thing came into my mind: what if I merge the switch and the function node i.e. I'd implement the conditional in the function? Would it be still possible that the second message gets to the evaluation of the conditions in the code earlier than the value assignment of the first message's payload to flow.a happens?

If you are having problems like that with context it may be an indication that you should not be using context for that data, but should be passing the value directly to the nodes that need it.
If you can't see how to do that for your use case show us a bit more of the flow so we can suggest better solutions.

I also thought about that but the incoming messages are created by a status node so they do not contain the info I store in the flow variable used for the comparison. If you happen to know the solution for that I'm pretty open for any ideas :slight_smile: Showing the flow currently does not make sense as it's pretty complex and haven't yet made a minimal example of it.

Hi @r0ller

if you combine the logic of the Switch and Function into a single Function node then, as long as the Function doesn't do any asynchronous work (which it doesn't sound like from your description), then you will be fine. If a Function node's code is entirely synchronous then only one message will be passing through it at any one time.

Already implementing that, as I got way too curious about it :slight_smile: Thanks again!

Thinking about it, if you want the contents of one message to affect what happens to the next message then the simplest solution is to do it all in one node. The only other way, I think, is to force one message to be fully processed before the next one gets started, using a semaphore or gated queue of some sort, which would definitely be overkill for this application.

I must admit, I got caught out by this in the Change node recently.

I had a stream of messages coming in and wanted to append a value from each message into a context array.

The Change node had a rule configured to set flow.list to the result of $append(msg.payload, $flowContext("list"))

And I would always end up with a 1-element array in flow.list. This is because when the first message arrives at the node, it would evaluate $flowContext("list") which is an asynchronous operation. This would allow the second message to start to be processed, which would again evaluate $flowContext("list") at that point in time - which is before the first message had got to the part of updating flow.list. The end result - each message got flow.list before it had been updated by the previous message, so they all got an empty array, then they appended their value and saved it back.

It certainly caught me out for a while - and I'm meant to know how this stuff works.

If we were to make the Change node strictly sequential it would not be good for performance. But I do wonder if we should add an option to the node to enforce sequential processing - something that can be turned on for the occasional scenarios that need it.

That would not have helped in the original issue in this topic - where the work to update Context is done in a subsequent node.

Yepp, an option for forcing sequential processing would help in many cases I think.

I hadn't realised that. Is that because it is jsonata or because it is context?
Are flow.get/set in a function asynchronous?

It depends if the underlying store is synchronous or asynchronous - https://nodered.org/docs/user-guide/writing-functions#asynchronous-context-access

OK, thanks, so normally in a function node it is not asynchronous. What about in the example you gave, is that because it is jsonata?

JSONata functions are always evaluated asynchronously. To try to keep the syntax as simple as possible, the $flowContext() function will complete asynchronously, but you don't have to worry about adding callbacks or the like in the JSONata expression - you can write it as if it were synchronous.

Understood, thanks. I didn't know that.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.