Node-red performance benchmark

[baoyicai]

I'm wondering how many concurrent flows can node-red handle? is there any load test?

My use case:
1 generate alarm on iot data. like temperature exceed limit, around 10,000 alarms
2. calculate new values based on input data, and stores back to database. around 2,000 calculations.

both will be between 1-10 seconds interval.

input and output data source will be influxdb.

Anyone can share any info is appreciated! Thanks.

[Colin]

That will entirely depend on the performance of the hardware you are running on. The bottleneck may be Influx, or it may not be, there is no way to know in advance.

By the way, node-red does not do anything concurrently, it is single threaded.

It should not be too difficult to create a test flow to simulate your requirement and see how it performs.

If the input source is influx, how do you know when to run the checks?

[baoyicai]

Thanks Colin.

By single threaded, does it mean it will only perform one flow at a time?
So if I have 10,000 alarms, my first thought is to create 10,000 flows. Does it mean the 10,000 flows will be run one after another?

[Colin]

No, only one individual node can be run at a time but each time you send a message on or do something outside of node red, such as initiate an influx read or write then other nodes will get a chance to run. Influx is a separate process so it can run at the same time as node red.

@baoyicai

my first thought is to create 10,000 flows.

That sounds like a very bad idea. Instead can you send them all through one flow and use, for example, the topic to identify which alarm you are dealing with? Remember any time a message is held up in that flow waiting for influx, for example, then other nodes in the same flow will get on with processing the other alarms.

[krambriw]

@Colin

Remember any time a message is held up in that flow waiting for influx, for example, then other nodes in the same flow will get on with processing the other alarms.

I might be wrong, just wondering, would it instead make sense to have multiple number of influx nodes? To get some "waiting in parallel" so to say, allowing NR to be offloaded by pushing processed alarms to multiple nodes instead of "waiting in serial" for the single node to be ready for the next message?

[Colin]

@krambriw

would it instead make sense to have multiple number of influx nodes

That is an interesting question and the answer is that I have no idea whether the throughput would be greater with mulitple influx nodes. Also there is the question as to whether it would be best with multiple influx nodes but one underlying database config node or whether one should have mutliple config nodes too. It would depend, I think, on how the nodes were written.

But overriding all that is the fact that initially one should always implement using the simplest strategy. After getting it going then if performance is an issue then work out where the bottleneck is and refactor that section. Ninety nine times out of a hundred performance won't be an issue, and in the few cases where it is, I can virtually guarantee that the bottleneck will not be where you expect it to be on starting out. Premature optimisation is almost always a waste of time and can lead to more complex and difficult to maintain systems.