Nodes connectors creation order influencing Typing: Bug?

General
1

Hi there,

I know the order of creation of connectors impact the order of execution of the connected nodes. Until now, I assumed it was the only impact on execution of the order of creation of the connectors.
But now, I'm seeing strange stuffs happening, here is the story:

It wrote a quick and dirty attempt at playing with JSON & CBOR.
CBOR is a compact binary representation of JSON.
I found a JS library to handle the conversions. It uses ArrayBuffers to store its representations of CBOR.
So I created two different function nodes: one converting JSON->CBOR and another doing CBOR->JSON.
My quick and dirty implementation is to copy/paste the code of the library in a function node and replace msg.payload by its converted value.
So, piping JSON->CBOR and CBOR->JSON function nodes should return the original payload in JSON.

Try the attached flow and you will notice:

  • pushing the 1st inject button causes no error and the output is correct
  • pushing the 2nd inject button causes no error and the output is correct
  • pushing the 3rd inject button causes the following error: "TypeError: First argument to DataView constructor must be an ArrayBuffer"

The only difference between the 2nd and the 3rd line of my flow is the order of creation of the connectors:

On the 2nd line:

  • I first piped the output of JSON->CBOR to CBOR->JSON
  • Then I piped the output of JSON->CBOR to the intermediate debug node
    On the 3rd line:
  • I first piped the output of JSON->CBOR to the intermediate debug node
  • Then I piped the output of JSON->CBOR to CBOR->JSON

So it seems to me that the debug node somehow "back modifies" the payload that will then fed to the CBOR->JSON functions. From my understanding, both these nodes should have got the same immutable message to process as their input is fed from the same node. It seems it is not the case.

Did I miss something about Node-RED principles or did I just found an bug ?

Best regards

DoNcK

nodered_typing_flow_ordering.json (79.3 KB)

2

I can't explain it fully yet, but there are a few things at play here. Whether it's a bug or limitation, I'm not sure yet

Part one
When a node has multiple wires connected to a port, the runtime clones the message passed to each wire to ensure one branch cannot modify the message object passed to a different branch.

For the sake of efficiency, the original, un-cloned, message instance is passed to the first wire - only the second (and subsequent) wires get clones.

This is a tangible difference between your second and third flows; whether the original or cloned message is passed to the next node.

Part two

The Function nodes run their code in a sandbox. Within this sandbox it gets its own copy of the core javascript types (such as Boolean, ArrayBuffer, DataView etc). The instanceof operator works by comparing the type of an object with the reference types.

If you create one of these types in one Function node, then use instanceof to check its type in another Function node, the check will fail - even though it will have all the properties and behaviour of that type.

You can see that in action with this flow that does just that:

[{"id":"2240dc5d.4886a4","type":"inject","z":"69352e8a.e486b","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":140,"y":660,"wires":[["386c67c2.a0cf88"]]},{"id":"386c67c2.a0cf88","type":"function","z":"69352e8a.e486b","name":"","func":"msg.payload = new ArrayBuffer();\nnode.warn(\"msg.payload instanceof ArrayBuffer=\"+(msg.payload instanceof ArrayBuffer));\nreturn msg;","outputs":1,"noerr":0,"x":290,"y":660,"wires":[["76cdb30.8ef964c"]]},{"id":"76cdb30.8ef964c","type":"function","z":"69352e8a.e486b","name":"","func":"node.warn(\"msg.payload instanceof ArrayBuffer=\"+(msg.payload instanceof ArrayBuffer));\n\nreturn msg;","outputs":1,"noerr":0,"x":430,"y":660,"wires":[["38eedcd3.11fac4"]]},{"id":"38eedcd3.11fac4","type":"debug","z":"69352e8a.e486b","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":590,"y":660,"wires":[]}]

The first function node reports msg.payload is an ArrayBuffer, the second function node says it isn't, the final Debug node says it is (because Debug doesn't use instanceof to check types).

However

I've added the following code to all of your CBOR to JSON nodes:

node.warn("msg.payload instanceof ArrayBuffer="+(msg.payload instanceof ArrayBuffer));
node.warn("msg.payload.toString="+msg.payload);

In all three cases, instanceof ArrayBuffer returns false and the toString returns [object ArrayBuffer]. This is consistent with the Part two comments. But it doesn't explain why the third flow, where the cloned message is passed in, causes the error.

So....

Not sure what conclusion to draw yet - needs more digging into to understand what's different about the payload in the third case that the DataView constructor doesn't like.

3

Thanks a lot for these enlightening explanations !

I wrote a very simplified test case that also show the strange behaviour that might help you:
I create a new Uint8Array([1,2,3]), then branch and print in two debug nodes.
Only the 1st debug node is able to print, the other that uses the cloned message displays "[Type not printable]".

So something seems to be wrong at branching&cloning time, right ?

[{"id":"de9af788.3a57e8","type":"inject","z":"80cf6b57.a42bd","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":160,"y":1640,"wires":[["65aba342.86c764"]]},{"id":"65aba342.86c764","type":"function","z":"80cf6b57.a42bd","name":"Uint8Array([1,2,3])","func":"msg.payload=new Uint8Array([1,2,3]);\nreturn msg;","outputs":1,"noerr":0,"x":350,"y":1640,"wires":[["c9e47b02.a5abe8","4b9f319e.5182d"]]},{"id":"c9e47b02.a5abe8","type":"debug","z":"80cf6b57.a42bd","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":570,"y":1640,"wires":[]},{"id":"4b9f319e.5182d","type":"debug","z":"80cf6b57.a42bd","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":570,"y":1680,"wires":[]}]

Best regards,

DoNcK

PS: Just to be sure: do you agree the situation looks like something should be fixed within Node-RED, or does using ArrayBuffer/TypedBuffer payloads should be considered as a bad practice ?

4

If you reverse the wires to the debug nodes (delete both wires and attach the bottom one first) the top one will diplay the [Type not printable]

However if you change both debug nodes to display the complete mag object, they both display the same result:

1/10/2019, 6:08:51 AMnode: 6e87c7b3.447bb
msg : Object
  object
  _msgid: "ea4fb55b.9e5708"
  topic: 
  payload: object
    0: 1
    1: 2
    2: 3