How to avoid use same flow multiple times with differents parameters and running in parallel

General
1

I've created a flow to read data from device trought external API request. The data is sent by MQTT to my own API. This flow is inside infinity loop.

Exists multiple devices, then, I copy & paste the flow multiple times, each one with differents parameters of device (API URL, IP , MQTT topic, etc...).

All flows have to be executed at same time, not in a queue.

At init, there is a query on database to read all device's parameters. The paramaters are stored as global variables with a numeration (IP1, IP2, IP3...).

In the flow, I set "msg.id=1" (or 2,3,4...depends on device), and It's reads global variables like "global.get('IP${msg.id}')" to use in flow.

The problem is that I want avoid copy & paste the same flow every time, since when I change something, It's very difficult to mantein.

Is there any way to have only one flow, with multiple ins(one per device) and running in parallel.

Screenshot_20220724_122351
Screenshot_20220724_1223511080Ă—1502 252 KB

Screenshot_20220724_114353
Screenshot_20220724_1143531731Ă—468 193 KB

2

firstly, why do you have a loop. Surely an inject set to repeat every 5 seconds would do the job?

Secondly, there are 2 optimisations I can see.

  1. Instead of storing this${msg.id} and that${msg.id} and so_on_and_on${msg.id} you could store the "recipe" in an object. then you only need to do
    msg.payload = global.get("recipe"+msg.id) || {}
return msg
  2. Use link-call node to collect the setup "recipe"
    • set msg.id = 1
    • execute link-call
    • get back parameters in msg.payload (or msg.whatever_you_want)
    • continue your flow & use msg.payload.runaMQTT and msg.payload.username etc.
3

Okey. Good optimizations. They will be applied.

The use of loop is because I want to do actions sequenty, when one ends start the next, all time. If I use lapsed timestamp, i wouldn't wait to ends nodes befote init others. (If the http request take a long time, higher than lapsed time of timestamp). Correct me if i'm wrong.

In regard to use one flow many times, someone knows any optimization? It is very important emit mĂşltiple responses (one per device) at same time by MQTT .

Thanks!

4

Nodejs is single threaded so only one thing will happen at a time. Also each MQTT msg will be a separate TCP packet on the wire to the broker , so again they will arrive in sequence and not at the same time.

5

So you mean that even if I have 1000 flows copied and pasted each with its start timestamp, the execution will be sequential, and until the 999th is finished the 1000th will not be executed?

6

No, only one thing can be happening at any instant, but (simplifying somewhat) the 'thing' that is happening is the execution of a single node, not a flow. So if you start many flows running at approximately the same time the flows will run in parallel, even though only one node is being executed at one time. Also for what is known as async nodes, which is nodes that have to wait for some external operation to complete, such as IO, network access, and so on, the node doing that will pause while it is waiting, and allow other nodes to run.

7

Okay. So the only solution to run multiple flows in parallel (which are the same but with different parameters) is to copy and paste as many flows as you want? Or is there another better solution?

8

I don't think you even need multiple flows. You should be able to feed all the messages into the same flow as suggested by @Steve-Mcl. You can have multiple messages passing through one flow at the same time. In fact node-red does not actually have the concept of a 'flow', there are just lots of nodes all connected together. The editor shows them in multiple tabs, but mostly that is just a convenience for the user to make it easier to organise the total system.

9

I may have explained myself wrong. When I talk about flow, I am talking about a set of nodes with input and output. In this case, in the dashboard image of the topic question, there would be two flows. I don't distinguish flows by tabs.

11

So why can you not merge most of the contents of the two flows into one? Exactly what are the differences between the two flows that prevent you using just one flow?

12

The general technique to use is to make sure that all the information you need is contained within the message. Then when you feed the message back to the start from the end it will restart the required operation, even though other operations are currently making their way through. You can use message properties that are not normally used to pass information along the flow. If you set something like `msg.myProperty' then it should be passed through nodes along the way so you can use that to know what you are supposed to be doing with this message.

13

Because I used each stream for one device, since I wanted multiple devices to run in parallel.

The goal of the topic is to know if with a single stream I can run multiple devices running in parallel. From your answers I understand yes.

The solution would be to use a join node with multiple timestamps (one for each device)?

14

I don't know why you need to use a Join node. Put a device identifier in msg.device and use that in each node where it is necessary to perform device dependent operations.

Or do you need the synchronise the start of a complete scan of all devices and wait till they have all completed before you start a new scan? If so then are you sure you need to do that?

1 Like
15

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.