Counting upstream wires

Developing Nodes
1

Hello,

I’m trying to create some custom parallelize/serialize nodes to handle task parallelization in my flow.

parallel-flow
parallel-flow1066×212 11.9 KB

Imagine there is a dozen Task A, Task B … and that each task is some async code that waits a couple of minutes for some data to be available for download. I don’t want to run all these tasks in series, as it will take 30 minutes before processing can continue. This could be done under 2 minutes when parallelized.

The parallelize node just adds a unique run id to its output message. The serialize node waits for N messages with the same run id, before passing the results/payload downstream. With N being the number of wires connected to the input port of the serialize node.

I can't seem to programmatically find this number N. In the run time, the serialize node has a wires property, but with a single identifier, and _wireCount = 1 ... when there are clearly two wires in this example.

SerializeNode {
  id: '0fc27faab5b0ce67',
  type: 'Serialize',
  z: '7caca3e75f9a5056',
  _closeCallbacks: [],
  _inputCallback: null,
  _inputCallbacks: null,
  name: 'Serialize',
  wires: [ [ '9d0fe589326a8da9' ] ],
  _wireCount: 1,
  send: [Function (anonymous)],
  _wire: '9d0fe589326a8da9',
}

I guess I don't understand how wires are represented and may be accessed from the run time. Any help/pointers will be greatly appreciated.

2

There is no easy way for a node to determine what is connected to it's inputs. In general, nodes don't care - they operate on the messages they are given, regardless of where they have come from.

3

I believe that the wires and wirecount are the output not the input.

So in theory, your paralleize node would know how many wires were connected. The wires property is an array of arrays. The outer array having 1 inner array per output port.

0|Node-RED  | >> THIS >> nodeInstance {
0|Node-RED  |   id: 'b375fe7010b32cce',
0|Node-RED  |   type: 'uibuilder',
0|Node-RED  |   z: 'b086fef38d54f545',
0|Node-RED  |   g: undefined,
0|Node-RED  |   _closeCallbacks: [],
0|Node-RED  |   _inputCallback: null,
0|Node-RED  |   _inputCallbacks: null,
0|Node-RED  |   wires: [
0|Node-RED  |     [ 'afdfb81a45da51ec' ],
0|Node-RED  |     [ '1a0dc389f6fd6779', '3371dd2a83590132' ]
0|Node-RED  |   ],
0|Node-RED  |   _wireCount: 3,
0|Node-RED  |   send: [Function (anonymous)],
0|Node-RED  |   credentials: { jwtSecret: 'Replace This With A Real Secret' }
0|Node-RED  | }

So you could get the number of wires and add the number to all of the messages (or you could pass things one of several other ways but that would be the most "node-red style"). Of course, that assumes that nobody adds another node to the output such as a debug node.

Or, for a more simple and robust method, since you can see and count the number of parallel flows for yourself, you could simply have an input in the serialize node. Much simpler.

4

Thank you very much for your quick replies!

@knolleary If the output connections are what is stored in the wires property. Could it be easier to iterate on each node in a flow, and check if it is connected to the Serialize node? This could be done just once, when the Serialize node is initialized.

Otherwise, I may end up adding N, as a parameter to the Serialize node configuration, as suggested by @TotallyInformation.

5

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.