Run multiple instances of node-red within a jupyterhub

Hi,

my actual architecture is this: picture

Therefore a single user node-red instance is started (spawned) by the jupyterhub. To secure the routes the node-red instance has to do oauth2 against the jupyterhub.
This can be done by adding a passport-oauth2 strategy in the settings file.

Unfortunately to make this work I need to fix two files.

  1. @node-red/editor-api/lib/auth/index.js in function genericStrategy
    const mystrategy = new strategy.strategy(options, verify);
  
    mystrategy.userProfile = function (accesstoken, done) {
	    log.audit('depp')
	    log.audit(accesstoken)
      this._oauth2._request("GET", process.env.JUPYTERHUB_API_URL+"/user", { Authorization: "Bearer "+ accesstoken }, null, accesstoken, (err, data) => {
	      log.audit(data)
      if (err) { return done(err); }
      try {
        data = JSON.parse( data );
      }
      catch(e) {
        return done(e);
      }
      done(null, data);
     });
    };

    passport.use(mystrategy);


This is an override that is specific to the jupyterhub rest api to get the user profile.
? Could this also be done in the settings.js?

  1. in passport-oauth2/lib/strategy.js in function OAuth2Strategy.prototype.authenticate
    I have to force that the callback url is relative and will not be resolved to the full host path. Thats solved by comment out this section
if (callbackURL) {
    var parsed = url.parse(callbackURL);
    if (!parsed.protocol) {
      // The callback URL is relative, resolve a fully qualified URL from the
      // URL of the originating request.
      callbackURL = url.resolve(utils.originalURL(req, { proxy: this._trustProxy }), callbackURL);
    }
  }

The reason is that the callback url is checked by the oauth2 provider (= jupyterhub) and shall be accessable within the namespace in the kubernetes cluster.
See this commit here which requests for a merge in passport-oauth2 https://github.com/jaredhanson/passport-oauth2/pull/52/files

Is this of topic for node-red?
Are there any hints?

Thanks
Andreas

Question: Why do you need to run Node-RED from the hub? Why not run the instances directly and secure them with Azure Active Directory?

Hi,

that's a valid question.
So my thoughts or better the requirements are:

  • Multiple instance of NR needed because I want to offer every user in our Active Directory an on demand instance of NR
  • There shall be a platform / hub where I can monitor / see the running instances
  • After Idle operation the instances shall be stop automatically
  • The user data eg. the flows / projects shall be persistent
  • Enable SSO against our AzureAD
  • Enable Admins / instructors access to the user instances

So because I already use jupyter notebooks manged by jupyterhub this was my first approach to use also jupyterhub running in a kubernetes cluster. I've searched the internet and here the forum for something like a "node-red hub" but there was - as far as I - see no simliar approach.
There is a commercial approach from Siemens. They call it "flow editor" (= node-red) and have it integrated in their mindsphere cloud product portfolio.

BR
Andreas

Maybe look at flowforge.com by the main creator of Node-RED ?

Yes, that's reasonable and I agree.

Also reasonable but there are certainly many ways to do that.

Not so sure how this would work. Define "idle" for example? I can see some way to be able to turn on/off the instances manually and to be able to shut them down on a timer (e.g. not running after 7pm or at weekends).

Well that's just a matter of making sure you have some Azure storage to work with. You'd need that anyway of course.

That could be done either with a suitable Passport recipe in each instance or maybe via an external reverse proxy. The latter needing an extra piece of persistent architecture of course but would certainly be easier to manage. If you set up node-red to run as an Azure web app, it uses a plugin for IIS and you get AAD SSO automatically without any changes to Node-RED. Not sure Azure web apps would be suitable here but it gives you an idea of how it could work.

That would be part of the SSO config of course.

It is because I don't really know how that works that I asked. And I'm interested as we are finally beginning to integrate Node-RED into our enterprise architecture.

To me, the idea of being able to easily stand up Node-RED instances for people, secured with Azure SSO, is really interesting. But we don't use Jupyterhub.

You've raised a possibility in my mind. Azure is pretty good at being able to automate provisioning and other activities so maybe this is possible without jupyterhub?

I think that I will reach out to my Microsoft contacts next week to see if they can come up with some suggestions. Unfortunately, the one that I know used Node-RED has retired but it may interest them enough to try and find a workable and low-cost answer.

I can certainly foresee a workflow where users request a Node-RED instance via a Microsoft form which is processed in Azure Pipeline that automatically creates the instance with default settings via an Azure Template. It might create it for a defined amount of time and perhaps include schedules to turn it on/off out of hours. An admin dashboard could be created by an admin instance of Node-RED or, if you are already using things like Logstash, maybe Grafana. That admin dashboard could include buttons that trigger a webhook for turning on/off instances and even automatically deprovisioning them. A webhook for provisioning would also be possible.


Update: Check out App Service — Build & Host Web Apps | Microsoft Azure. I suspect that is the answer. Coupled with Azure Pipeline devops automation.

Hi,

thanks for the valuable input and comments. I've already built up a productive environment for students (many users which already "live" in the AzureAD of the university). I'm really interested in flowforge. But it seems that a professional version is not yet available.
The weak point is that I need to slightly adjust some files which are part of the npm installation (node-red and passport-oauth2). This is ok but the maintenance gets less robust. (see my initial post/question how to improve this)

To demonstrate what is possible so far I've created a github repo where the necessary files and rough documentation are available. https://github.com/aschiffler/nrhub
You can test it by self-sign-up service provided for a demo hub https://nrhub.vlab.education/. You can use your Microsoft or Gmail account. Users with the role of group-admin in the AzureAD can then assign the users to groups.

@TotallyInformation : "Idle" means eg. if there is no traffic against the admin api for one hour turn off the instance. --> This is a feature from the jupyterlab notebooks to save energy/money

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.