Shared subflow context


#1

Hello,

That'd be useful if there was a context that would be shared by every instances of a subflow. It's like having private static variables in a class. Every instances of the class have access to theses static variables.

I can see many uses of this.

For exemple: I want get config from a database that is needed by every instances of a subflow. Now I don't have choice to let the config in the global context. If I had a shared subflow context, the request to the datable could be done at the first execution of the subflow, then it would be shared by any other instances of the subflow.

Sorry for my english

Thank you


#2

I totally agree with @adesjardins.

A subflow is by essence a flow made to be reusable in another flow. It makes sense that the subflow can access the flow context of the flow using it, thus minimizing integration efforts between a flow and its subflow.

Currently the only way I found to share information between a flow and a subflow is to either use the global context or the msg object, neither of which are ideal if I only need to get or set information at flow level only.


#3

What if I have two flow tabs A and B and they each use a flow variable flow.mydata and they both call sub flow 'sub flow-foo.

So A sets flow.myflow to 'data is Aand then enters the sub flow. At the moment everything is fine, but blowBis also running and it setsflow.myflowto 'data is B.

Now the sub flow from A goes and uses the flow.myflow but it contains 'data is B`.

you have a mess....


#4

OK, I admit a little misconception on my part on the scope of flows in node-red. :slight_smile:

However, I can totally see something like that happening in a function node in a subflow:

var namespace = get_flow_namespace(); // returns string eg: 'ns1-myflow'
var contextData = flow.get(namespace);

// happily do something with the shared context data...

Albeit it is a little bit more work to manage flow context key namespacing, it is totally manageable and would bring the afore mentioned benefits. On the other hand, short of copying - read duplicating - a subflow and editing its contents, I have found no way of "injecting" usable context data. This has unfortunately stopped me from being able to compose my flows together.


#5

So why don't you - just before calling the sub flow - retreive the flow.variable and add it to msg.payload that will be sent to the sub flow?


#6

Yes, it is a solution.

I just like it less because I am trying to work in a more functional programming paradigm and I like to consider my msg object immutable. This is because my messages pass through many complex flows and I cannot keep track of the message state easily.

So, yes, for now I will inject my data in msg and then cleanup after. :slight_smile:


#7

Trying to operate this way in a node-red flow is probably self-defeating -- the msg object is expected to have its contents changed as it flows through each node... and relying on side-effects like adding vars to global context should generally be avoided, if possible.

I agree that with large flows it can be confusing to keep track of all the bits of data that the msg can accumulate. And since most nodes want to work with the payload property of the msg, each step needs to have a change node before it to move the right data into the payload. I find it most straight-forward to use change nodes to accumulate the results into another field (e.g. msg.output), and at the end to Move: msg.output to msg.payload


#8

It is also a bit of an anti-pattern to store per-message data in context. As soon as you have two messages being handled by your flow at the same time you then have to make sure one message's use of context doesn't conflict with the other.


#9

I think I was not clear at first.

  • I don't ask for a flow and a subflow to share the flow context.
  • I don't ask for a subflow to share his flow context with other subflows.

It would be a new context called... static_subflow.

It's like static variables of a class X. If you instaciate multiple time the class X, they will all have their own private properties, but they also share static variables. I'd like to see to same for subflows.

All instances of a subflow would have their own flow context, not shared. But they would have a static_subflow context that would be shared for all intances of the same subflow.

I'd like my subflow to stores configs that are requested from the db. Now I have to store these configs into global or to repeat the same request in each subflows. It's not optimal. With a static_subflow context, I'd do a request to the database once, eg: pulling configs, then I would pass them to any instance of the subflow into msg.config. Then the sublfow detect it and store it into static_sublow. Each instances of the same subflow now have access to the same config. No need to repeat SQL request, no need to pass the same msg.config each time. No need to store anything into global. Cleaner, faster and less confusing !

Thanks

Sorry for my english


#10

Hi @adesjardins - my apologies for not responding to your original request.

Yes, the concept of a subflow have its own context is one we're aware of and we will address it at some point. What we cannot practically do is introduce another context object in addition to flow and global - as nodes would need to know if they were inside a subflow in order to expose that as an option.

So we need to design a way to have a subflow-scoped context that seamlessly fits with the apis we have. I imagine it would be a special namespace under the flow context - such as flow.$subflow.foo or some other syntax that is clear and unlikely to clash with existing usage.


#11

Nice,

I understand the difficulty of this. I think that sub-context shloud do it.

Thanks


#12

Hi @knolleary, @shrickus and @zenofmud ,

using for example the made up output attribute of the msg object is a great idea for a per message data: 1-to-1 correspondence, same scope and lifetime.

[Disclaimer :wink: ] Despite realizing a little late that a flow was a tab in node-red and that one flow/tab can contain many "sub-processes", I still think there is substance to giving a subflow access to its containing flow context, specifically if we are dealing with per-flow logic.

My use-case in more detail: I have grouped many "sub-processes" into one flow because I have made a flow a collection of message processing actions around a same "concept". So for example in one of my flows I have three "sub-processes" starting with an http-in node that point to the same http endpoint, with a different verb (GET, POST, DELETE). In each of these "sub-processes" I have nodes that read configuration values from the flow context. I love subflows because they are great for contracting nodes together and in essence prototyping new nodes and their behaviors. Encapsulating the subflow context stops me from sharing configuration between my flow and my subflow, which would be brilliant for behavior composition.

The reason why I made this request in the first place is because the "sub-processes" in the same flow should share the same configuration as we are in a per-flow logic, however the only way to implement this behavior would be to turn to the global context or to create some pseudo private attribute at the msg level. Both alternatives could work but seem slightly inadequate to me.

@adesjardins, sorry, I did not mean to highjack your thread! I guess I actually misunderstood your question in the first place too :wink: