How does the flow context work in a subflow?

General
1

I found a couple of discussions about this, but the info is usually pretty unclear. I found this text: The nodes in a subflow can use flow context, bit it is scoped to the nodes in the subflow, not the flow the instance of the subflow is on.

To me, this says that all the nodes in a subflow have a shared flow context, that is separate from any other flow or subflow. It doesn't make clear however if this context is then relative to the subflow itself, or to each respecitve subflow node.

I'm trying to use the flow context in a subflow to build a log message .Basically it calls the subflow a couple of times, each time add a log message to a certain flow variable, and when a message contains print_log, print the content of that flow variable and then clear it. When I replace all the flow's with global's, the code works just fine, so what's causing the issue when using the flow context in a subflow?

Can somebody clarify how the flow context actually works inside a subflow?

2

What you are describing sound like it should work. Can't say more unless you share your flow.

3

I quickly set up an example that should make it pretty clear what's going on:

[{"id":"1ae4d5e11a65ec7b","type":"subflow","name":"Logging (global instead of flow context)","info":"### Inputs\n - **log_type_prefix**:\n    - `msg.log_type_prefix` or `env.log_type_prefix`\n    - [**Prefix** (uID) Log message]\n - **uID**:\n    - `msg.uID`\n    - [Prefix (**uID**) Log message]\n - **log_msg**:\n    - `msg.log_message`\n    - [Prefix (uID) **Log message**]\n - **print_log**: boolean","category":"","in":[{"x":80,"y":60,"wires":[{"id":"9a60922deec717f8"}]}],"out":[],"env":[{"name":"log_type_prefix","type":"str","value":"globalPrefix"}],"meta":{},"color":"#DDAA99"},{"id":"9a60922deec717f8","type":"function","z":"1ae4d5e11a65ec7b","name":"Create log function","func":"node.warn(msg)\nif (!msg.log_type_prefix) {\n    msg.log_type_prefix = env.get(\"log_type_prefix\");\n}\n\nlet buffer = \"\";\nif (global.keys().includes(msg.uID)) {\n    buffer = global.get(msg.uID);\n} else {\n    buffer = \"--------------------------------------------------\\n\"\n}\nbuffer += `[${msg.log_type_prefix} (${msg.uID})] ${msg.log_msg}\\n`\nbuffer += `${JSON.stringify(msg)}\\n`\n\nglobal.set(msg.uID, buffer)\nif (msg.print_log) {\n    global.set(msg.uID, undefined)\n    return { log_msg: `${buffer}\\n` }\n}","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":210,"y":60,"wires":[["0acf4665bec5abf7"]]},{"id":"0acf4665bec5abf7","type":"flogger","z":"1ae4d5e11a65ec7b","name":"Log","logfile":"bedroom_lights_automation.log","inputchoice":"object","inputobject":"log_msg","inputobjectType":"msg","inputmoustache":"[{{log_type_prefix}} ({{uID}})] {{log_msg}}","loglevel":"INFO","logconfig":"2d86649a1f8356dd","sendpane":true,"x":370,"y":60,"wires":[[]]},{"id":"23e49dfd85a84a57","type":"subflow","name":"Logging","info":"### Inputs\n - **log_type_prefix**:\n    - `msg.log_type_prefix` or `env.log_type_prefix`\n    - [**Prefix** (uID) Log message]\n - **uID**:\n    - `msg.uID`\n    - [Prefix (**uID**) Log message]\n - **log_msg**:\n    - `msg.log_message`\n    - [Prefix (uID) **Log message**]\n - **print_log**: boolean","category":"","in":[{"x":80,"y":60,"wires":[{"id":"5c75ed62173fca58"}]}],"out":[],"env":[{"name":"log_type_prefix","type":"str","value":""}],"meta":{},"color":"#DDAA99"},{"id":"5c75ed62173fca58","type":"function","z":"23e49dfd85a84a57","name":"Create log function","func":"node.warn(msg)\nif (!msg.log_type_prefix) {\n    msg.log_type_prefix = env.get(\"log_type_prefix\");\n}\n\nlet buffer = \"\";\nif (flow.keys().includes(msg.uID)) {\n    buffer = flow.get(msg.uID);\n} else {\n    buffer = \"--------------------------------------------------\\n\"\n}\nbuffer += `[${msg.log_type_prefix} (${msg.uID})] ${msg.log_msg}\\n`\nbuffer += `${JSON.stringify(msg)}\\n`\n\nflow.set(msg.uID, buffer)\nif (msg.print_log) {\n    flow.set(msg.uID, undefined)\n    return { log_msg: `${buffer}\\n` }\n}","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":210,"y":60,"wires":[["c3a82a246d514d57"]]},{"id":"c3a82a246d514d57","type":"flogger","z":"23e49dfd85a84a57","name":"Log","logfile":"bedroom_lights_automation.log","inputchoice":"object","inputobject":"log_msg","inputobjectType":"msg","inputmoustache":"[{{log_type_prefix}} ({{uID}})] {{log_msg}}","loglevel":"INFO","logconfig":"2d86649a1f8356dd","sendpane":true,"x":370,"y":60,"wires":[[]]},{"id":"2d86649a1f8356dd","type":"config-log","logname":"Basic logging","logdir":"/data/logs","stamp":"local","logstyle":"plain","logrotate":false,"logcompress":false,"logrotatecount":"5","logsize":"1000","logtopic":false,"logsource":false},{"id":"92a034d42eaf462c","type":"subflow","name":"test logging flow","info":"","category":"","in":[{"x":80,"y":100,"wires":[{"id":"3b95e60fce02e0f3"},{"id":"663e8730cc2d106e"},{"id":"a8041a8e88a39498"}]}],"out":[],"env":[],"meta":{},"color":"#DDAA99"},{"id":"3b95e60fce02e0f3","type":"subflow:23e49dfd85a84a57","z":"92a034d42eaf462c","name":"","env":[{"name":"log_type_prefix","value":"flowPrefix","type":"str"}],"x":200,"y":80,"wires":[]},{"id":"663e8730cc2d106e","type":"delay","z":"92a034d42eaf462c","name":"","pauseType":"delay","timeout":"1","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"outputs":1,"x":200,"y":120,"wires":[["7afdc468e9843e49","621ea2cbdfcc4ca6","338f0eb90fcb2182"]]},{"id":"7afdc468e9843e49","type":"subflow:23e49dfd85a84a57","z":"92a034d42eaf462c","name":"","env":[{"name":"log_type_prefix","value":"flowPrefix","type":"str"}],"x":380,"y":80,"wires":[]},{"id":"621ea2cbdfcc4ca6","type":"delay","z":"92a034d42eaf462c","name":"","pauseType":"delay","timeout":"1","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"outputs":1,"x":380,"y":120,"wires":[["0cc41f6eed291603"]]},{"id":"66a0cc845ce779f3","type":"subflow:23e49dfd85a84a57","z":"92a034d42eaf462c","name":"","env":[{"name":"log_type_prefix","value":"flowPrefix","type":"str"}],"x":760,"y":80,"wires":[]},{"id":"0cc41f6eed291603","type":"change","z":"92a034d42eaf462c","name":"","rules":[{"t":"set","p":"print_log","pt":"msg","to":"true","tot":"bool"}],"action":"","property":"","from":"","to":"","reg":false,"x":550,"y":120,"wires":[["66a0cc845ce779f3","3d81df2cf71b9eb7"]]},{"id":"a8041a8e88a39498","type":"subflow:1ae4d5e11a65ec7b","z":"92a034d42eaf462c","name":"","x":290,"y":200,"wires":[]},{"id":"338f0eb90fcb2182","type":"subflow:1ae4d5e11a65ec7b","z":"92a034d42eaf462c","name":"","x":470,"y":160,"wires":[]},{"id":"3d81df2cf71b9eb7","type":"subflow:1ae4d5e11a65ec7b","z":"92a034d42eaf462c","name":"","x":850,"y":120,"wires":[]},{"id":"2585ba6af801602c","type":"inject","z":"f6f2187d.f17ca8","name":"","props":[{"p":"uID","v":"testUID","vt":"str"},{"p":"log_msg","v":"bla","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":320,"y":1980,"wires":[["84e27cdcdb6cd09f"]]},{"id":"84e27cdcdb6cd09f","type":"subflow:92a034d42eaf462c","z":"f6f2187d.f17ca8","name":"","x":480,"y":1980,"wires":[]}]

Be aware that this uses node-red-contrib-flogger-now. Replacing the flogger nodes with debug nodes should also work, but the log file shows it a bit more clearly than the debug output. If you'd prefer a version with simple debug nodes, let me know.

If I hit the inject button, after the set delays, this is what I see in my log:

2022/01/27 11:13:13 INFO --------------------------------------------------
[flowPrefix (testUID)] bla
{"_msgid":"8a09643d6feb263d","uID":"testUID","log_msg":"bla","print_log":true,"log_type_prefix":"flowPrefix"}

2022/01/27 11:13:13 INFO --------------------------------------------------
[globalPrefix (testUID)] bla
{"_msgid":"8a09643d6feb263d","uID":"testUID","log_msg":"bla","log_type_prefix":"globalPrefix"}
[globalPrefix (testUID)] bla
{"_msgid":"8a09643d6feb263d","uID":"testUID","log_msg":"bla","log_type_prefix":"globalPrefix"}
[globalPrefix (testUID)] bla
{"_msgid":"8a09643d6feb263d","uID":"testUID","log_msg":"bla","print_log":true,"log_type_prefix":"globalPrefix"}

If you look at the logging subflows, the code in the function node is exactly the same, other than one using the flow context, the other using the global context.
The one that uses the flow context uses "flowPrefix", the one with the global context uses "globalPrefix". As you can see, the flow context logging subflow only prints one message, the last one. The logging subflow that uses the global context prints the message 3 times (because it was sent to 3 different logging nodes sequentially, that's what it should do).

I also tried a version where the logging node is triggered by separate messages, instead of one message reaching multiple logging nodes sequentially. This way it does seem to work as expected. I guess this means that the flow context is only set/updated when a message has completed its route, while the global context is set/updated immediately?

Also, when I'm in a subflow, on the context data tab in the sidebar, it says "none selected" when I refresh the flow context. I assume this is to be expected, but I'm not sure why, and what it means with "none selected".

4

ok, so I see your issue. Flow context is per instance. that includes instances of a nested subflow

So as you have subflow instances embedded in a subflow, the flow context is different for each instance and each nested instance.

I could re-write this to show you a better practice but you'd miss out on the learning part.

Options

  1. Use a pre built logging node - i can recommend node-red-contrib-flogger
  2. Re-write this avoiding nesting. TBF - the nested subflow only contains 1 function anyway - just put that in your top level subflow
  3. Search the forum for "$parent flow context" (e.g. flow.get('$parent.xxxx')) for clues if you really want to proceed down this path.
5

So if I have place the logging subflow twice, those both have a different flow context? That kinda makes sense, but also doesn't.

Err, I am using flogger :sweat_smile:. The thing I'm trying to achieve is to gather all messages related to a certain UID, and then print them as one when that UID is finished. Since some of my flows have delays of a few minutes in them, there's an option that multiple things are happening simultaneously. I think it's nice for a log to have things bundled in a logical way.

This was just an example, there's more nesting above this. Also, it's "only" one function node, but it's vital that that function node is the same everywhere, plus the logging subflow gets reused a LOT. This is exactly what a subflow is for, just like a function is in regular code: reusing the same section, so if it requires a change, you only have to change it once, not tens or hundreds of times. Taking the function node out of this subflow would be a very bad design choice IMO.

I feel like this isn't going to work for me, but that's not that big of a deal. I'll just use a different tactic using the global context.

Thanks for looking at it and clarifying.

6

Hi @Timmiej93 I'd love to spend some time on this and show you some ways of making this more maintainable and less complicated but unfortunately I dont have the time right now.

I would suggest for reusable parts you look into making use of the link call - this avoids many of the pitfalls of subflows.

And one other point i often try to convey is - if anyone else is maintaining this, subflows introduce the concept of instancing - something many "low coders" / plc programmers etc just dont fully grasp (IMO).

Ok, then that makes sense.

Cool - its a good node.

Instead of getting complicated with sub-sub-sub flows, you could build the running log in flow context and pass that object into a subflow (or link call) via a msg property (e.g. msg.log) then operate on the msg.log - this will avoid trying to access $parent stuff.

Any how, happy coding and good luck.

7

In my opinion it works perfectly - that each flow variable in a subflow has its own context which is not shared with other instances of the subflow node. Otherwise the subflow would be not usable as each instance of an subflow could influence each other.
The reason of using a subflow is to use independent instances of this node (including a separate flow context). If you want to share information then global context should be used.

Otherwise it would be much more confusing if a subflow uses a flow context - the subflow node is placed on.

The current behaviour is perfect as I can define variables within the context of a subflow.

8

I have to disagree on this point. Using global context is a akin using global variables in an application. It's gonna bite you.

My preferred approach here is similar to the pure function principal

Pure Functions
A pure function is a function which: Given the same inputs, always returns the same output. Has no side-effects

This is my personal interpretation...
Input all necessary variables required to do computation inside the subflow.
Output the final values then on the main flow push the result to its final destination (file / context etc)

Of course as things evolve in your flows and difficulties arise you may stray from this but if you aim to stay pure you can avoid many gremlins.

9

No worries, I've only recently started with NR, so trying things out is good for me, it'll teach me what works and what doesn't.

I was actually using that before I moved to subflows. I've never really understood why these options exist side by side. I assume the difference is that subflows are meant to be used in multiple flows, and link calls only in the flow they are in? Maybe when my flow is complete I'll remove some subflows with link calls again. One thing I really don't like is that I can't use an inject node in a subflow for debugging.

Thankfully, I'm the only one to ever look at this. It also depends on which programming language you're used to IMO. If you were taught OOP then it wouldn't be too hard. The thing is that (IMO) the documentation of NR lacks a lot of detail in some parts, so some things don't work as you expect them to, and you have to resort to the forum to figure out why.

That makes sense. It would be nice to be able to choose though, for some things a scope specific to the subflow object instead of the separate instances could be very handy.

Agreed, that's why I use objects in my global variables. If my flow is named X, I have a global variable called X, which then contains an object containing all variables that for whatever reason need to be global, but still only belong to that specific flow. It's a bit risky because you can loose a lot of data easily, but that's a matter of implementation.

10

That in itself is a good enough reason to use a link call.

It is subtle but with a link call node there are no instances & in a way promote pure functions. And as alluded to before, you can plug an inject into a link call

No, link call can call to other flow tabs. A bit more experience and a bit more play you and it will click :slight_smile:

The reason I keep referring to pure functions is in some way indicate I'm asking you to avoid using context unless absolutely necessary. You will find situations arise where you call into a link call or subflow twice in quick succession and the context is messed up. In simpler terms try to pass everything you can in the message, have it processed in your subflow/link call, then return the computed result. This not only avoids situations where context is overwritten mid-flight it also makes your code much more portable.

11

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.