Embedded Node-red vs Middelware

im trying to convert nodejs express web application to node-red .
My understanding till now is it that node-red in this case work juste like any other middelware in the nodejs host but with some differences :
. if i have the same http url that start with the node-red httpNodeRoot in both nodejs and Node-red editor the request (express nodejs server) will ignore the nodejs app.use(url, middelware) and go directelly to Node-red... and if i remove the http node that use the that same url then the request will consider this middeware in nodejs like any other nodejs middelware even if it start with the HttpNodeRoot this prevent me from using any middelware (router, controller...) or pass data through req from nodejs To node-red ....
. There is some kind of redirection and i guess it start a new and different request so all the data in the request that reach nodejs become a msg..paylod.... and does not go through other nodejs middeware chain... node-red middelware work in isolation....
.So Node-red is a middelware but does not belong to nodejs express chain of middelwares or scope so no data can be exchanged i cant use next() also...

the good thing is that i can require all the modules installed with npm in Node-red
but i cant require the javascripts files with module.require because there no files to require...so as a work around i used a function node and i used instant invoqued function to have the same result and store the returned function in req.property...

My question is how can i use nodejs midellwares like router js files like controllers files in node-Red without having to convert all my nodejs code to IIF functions if only i can use any nodejs middeware or pass data through req......

if we can require modules from Nodejs i should be able to require other files.js because they are considered as modules in nodejs....

i was able to use the envirenement variable without adding them to settings.js
there is a npm module called dotEnv that convert the config.env files to envirement variable and with process.env.VARIABLE it work because global.env didnt work for me ... this why i want to have access to nodejs because he can do a lot for Node-red

Yes, sort of. Node-RED is obviously very complex itself so bear that in mind.

With middleware, everything is dependent on the order of loading. If you use two different folders to a static serve for example, both folders WILL actually get served. But the one will always take preference if there are clashing names.

Similarly with other middleware. The order of loading matters.

??? No, Express is simply processing things in the order you told it to. If one use function doesn't match your URL, it will fall through to the next and so on - provided that your use functions end with the next() function call.

Not really, Express has just been told to handle a particular url by Node-RED's use functions and that is what is happening. There is nothing left to 'fall through' because node-red expects to handle the whole request which is logical.

If you load a middleware onto the same url pattern before you load node-red and make sure you call next() at the end, it will work fine. Try adding an extra header for example.

No, it absolutely does. But as I say, node-red is handling the full url, it does not expect to leave anything behind for something else to handle - why should it? An http-in node for example is given a URL pattern and tells node-red to deal with it. If you want to do some processing on that URL pattern, do it in Node-RED not in your Express app.

If you have some processing already written in your Express app that you want to use on a node-red handled path, move the code into a module and create a simple custom node that uses the module.

And you can. But they have to make sense to node-red and not everything does because you are trying to mix two different paradigms. You will need to think through your code separation a little more so that the processing is in its own module. Then you can require that into a function node or, for better efficiency into a custom node.

If you want to see what can be done with node-red and express, have a look at the code for uibuilder. That currently uses the node-red Express server and adds new middleware to it and even allows you to add your own middleware. That is doing everything within node-red of course and with node-red's Express web server but when you embed node-red to an existing Express server, you are really only shifting things around a bit. Express doesn't really care.

Urm, you don't really need dotEnv. node-red is built on node.js which natively lets you access environment variables. Personally I tend to use a global in settings.js called _env and I simply set it to process.env:

    functionGlobalContext: {
        _pid: process.pid,
        _env: process.env,  // Pass ALL environment vars to Node-RED
        _userid: process.env.user || process.env.username || process.env.USER,
        _userHome: process.env.home || process.env.userhome || process.env.HOME,
        _hostName: require('os').hostname(),
    },

I don't really recommend doing that in a production environment though as it can easily expose passwords - for some reason, some admins think that env variables are suitable places to put passwords for running processes - go figure.

I also make extensive use of env variables in the settings file so that more things can easily be overwritten on startup.

You can, in most nodes, also use env variables as configuration inputs.