Maintaining separate flow files for multiple user logins

Hello there,

From last 2 days, I am trying to look out for possibilities of having multiple users login to editor and not disturb each other by having separate flow files.

My current approach -
Maintain session IDs for each user login externally by using Redis and at the same time assign/map a flow file dedicated to a user session ID. For example if Gaurav logs in assign a session ID along with file name - "flows_gaurav.json". And finally inside the RED.js instead of picking flowFile as a global var, picking it from session object, so as to have separate flows getting used by the Editor for separate sessions.

But I am not yet familiar with Node-RED code and obviously I don't know the main vision behind the way it has been developed. So I assume that for some very important reason, the editor has been kept a single user instance. But please do guide me if we can achieve things in this way within Node-RED.

I am not sure if I am making sense here, but I am sure you guys will suggest and guide better!

For Info - As of now, we are able to spin multiple instances/pods of Node-RED (inside k8s cluster) for separate users. When I use the term "we", I mean, me and my team who are working on Node-RED solutions!

But still, I think if we can have multiple users in a single instance, the kind of value and dynamics we can achieve from NR will be manifolds.

The runtime is a single-tenant runtime. It runs one set of flows and there is nothing in there to provide any isolation between the flows. So if one flow uses global context, all other flows can see it. If one flow is heavily loaded and uses a lot of CPU time, there's nothing to 'protect' the other flows from being starved of resources.

This is why we recommend running separate runtime instances per user - when you want to have the sort of user experience you describe.

All of that said, there is some work in the roadmap to provide a more standard way to hook into certain key processing flows within the runtime - such as when a deploy happens, or when an editor requests a copy of the flows to display. That would allow custom code to be added that filters the view of the flows passed to the editor - effectively hiding some parts of the flows from the user. There are lots of possible use cases for that, including trying to provide some sense of multi-tenancy. But that would not solve the lack of flow-isolation within the runtime itself.

Hello Nick,

Thanks for this reponse.

If I understand you correctly, your vision is that suppose we have 3 users X, Y and Z, so for all of them there will be a single flows.json, where all the flows will be associated to their users by using sessionID (or some ID) as the link between them. And the "custom code" you are talking about will be responsible to filter them, meaning user X will be able to see and work upon only his/her set of flows.

My vision/idea is having separate flow files like flows_X.json, flows_Y.json andd flows_Z.json and the "custom code" to decide which file to be assigned to the settings object before rendering it or passing it to the editor.

So as per your thoughts above, I understand that isolating the separate flows inside runtime will be a challenge, still is there any possibility to solve it in a way that the runtime can take flow file from session object ?

Hope I am able to explain !

The flow file is loaded when Node-RED starts. Changes to it are only picked up if Node-RED restarts or a Deploy is done. This is no different to any other microservice, especially a Node.js one. If the code changes, you need to restart the code. Node-RED can be a bit more efficient by only restarting parts of the flow that have changed but the same general principle applies.

The clean way to handle multiple users each with their own flow is to run multiple instances of Node-RED. Put them behind a reverse proxy to make things nicer for the users and to manage the authentication/authorisation if you want to. Or even run Node-RED in a VDI (remote desktop) if you prefer.

Not without a lot of changes to the code. The entire storage layer is completely unaware of individual users.

As others have pointed out node.js is single threaded... so if one of your "users" does something that blocks or kills the thread then it would kill everyone. I can't think of a single use case where that may be a good idea....
(let alone the single shared (insecure) global context, etc).

2 Likes

What advantage does that have over running multiple instances of node-red?

Hello Colin,
Thanks for the question !

We are using Node-RED for micro services orchestration in a highly sophisticated product which invloves multiple micro services, lambdas/functions and many other third party tools too! We are trying to understand that when we present Node-RED as a tool/solution to our first set of users, who will be using our core nodes and subflows to achieve their own use cases, how easy will it be for them to adapt to it!

In the process, when we already knew that Node-RED is not meant to be used in actual multi user concept, we figured that out by spinning dedicated instances for each user interested in using Node-RED. And thanks to the community as we have been asking a lot of questions till date and you all have supported in a great sense.

So coming to the point, in a particular team there can be 4 to 5 devs/QA trying to use NR and there will be N number of teams. So if we could have achieved the ability to have all users in a team use the same instance of NR at the same time, we think our users would have felt more agile. :sunglasses:

But I see there are numerous challenges at this point of time in achieving this, so we are good with the multiple instances solution for the time being.

Your thoughts are most welcome :smiley:

I am not sure that answers the question of what is the advantage of using multiple instances of node-red running one flow each versus one instance running multiple flows and switching between them at run time. The former might use less physical resources (RAM for example) and the latter less CPU due to not switching. The latter might have dramatically better performance due to using a thread for each user rather than all users on the same thread.

The former might use less physical resources (RAM for example) and the latter less CPU due to not switching. The latter might have dramatically better performance due to using a thread for each user rather than all users on the same thread.

@Colin, totally agree with this. As I said, it's just a point of having more usability or more agility. Nothing to do with performance as such in my case. I can easily achieve that by having the former approach as being discussed (having multiple instances).

But in cases where I know how much RAM I might need for a group of users, I can easily allot that amount of memory and CPU to a particular instance and easily have everyone play on the same instance and then also have some way by which all of them can collaborate on a single flow by merge request approach (not the git merge request literally).

But anyways this is just a thought and not a necessity, we are not blocked due to this! :v:

That is what I don't understand, in what way does it give more usability or agility?

@Colin if Node-RED offered it out of the box, then it would remove the need to figure out all the complications of building a proper multi-instance, multi-tenant system with all of the administration and orchestration needed.

In short, it makes all of that hard work someone else's problem.

Ah, perhaps I have misunderstood the comment, I thought that @gauravdasgupta was talking about his end user experience, but I think you may be right, it is from the system developer's viewpoint that it would be better, in which case I agree fully.