Node-Red in conjunction with pm2 in cluster mode context issue

[msupra]

Hi all

I embedded node-red in an express app as to allow me to start the app with pm2 in cluster mode.

I start two different express apps with node-red embedded. My architecture is based on a microservice architecture, thus I have a node-red instance running on different ports offering different functionality.
I use mqtt as the "inter node-red instance" communication layer.

In my current scenario my first node-red instance is an http service. It receives http requests then via mqtt sends it to my second node-red instance that performs business logic. The second instance will then respond back to the http service via mqtt and the http service will then respond back to the client.

I have spoken about this sort of architecture before and the group managed to assist me in the issues I had with keeping track of the http req and res, so as to allow me to always respond back to the correct client. I use this by storing the http req and res objects in a global context and then cleaning it up after a response was successful.

I have now gone one step further, where I use pm2 to start more than one instance of my express apps. With pm2 you can configure the startup process to use cluster mode and it will then start as many instances of your nodejs app as you have CPU's available. This actually works very well and there is no problem with how it operates.

I have actually run into a problem with my global context now. Let's say that I have two CPU's then two instances of the embedded node-red application will start. Under the hood, this uses the Node.js cluster module such that the scaled application’s child processes can automatically share server ports.

If I save my http req and res in a global context then there is no guarantee that the same instance of node red will process the response, thus there will be no context set for the request/response. This is a hit and miss scenario.

I was thinking to use Redis as an in memory data store where I can push and pop the required data, but this unfortunately does not work, as I am unable to store a http res in Redis.

When trying to store it as an object, it always stores it as [object Object], which when you pop it from Redis, it is just a string which means nothing. If I try to stringify it, get TypeError: Converting circular structure to JSON and it is not pushed into Redis.

If it is obvious mistake I am making, I would appreciate your assistance or if there is another way of doing this, I would really appreciate your help.

Thanks
Morne

[msupra]

Hi all

I think I answered my own question.

So far I have been able to RPUSH the http res into REDIS by using the flatted npm module. It is now placed in REDIS as a flat text string.

My next step will now be to BRPOP the value and concerting it back into an object and see if I can respond to the client.

Regards
Morne

[msupra]

Hi all

Flatted is not going to work. When trying to parse the msg string back into an object again, it does not parse it correctly for the http response node to understand.

I must say, I am a bit stumped here.

Regards
Morne

[msupra]

Hi all

Ok, I figured it out. When pm2 starts more than one instance of the http node-red app, it starts them all with different process Id's (PID). So, when I receive a req, the first thing I do is get the PID of the instance that received the req and I pass that value on via mqtt to the other nodes that will do the rule management.

When the rule nodes are done with its work it will then again via mqtt pass the result back to the http nodes. I then get the PID again and compare it with the PID in the message received via mqtt. If it is the same then set complete=true and a join node will then take the original http res plus the new rule data returned and respond back to the correct client.

If the PID's are not the same, then the message is dropped and the process stops. All http node-red instances listen on the same topic, so all instances will process the same message, so one and only one instance will process the transaction successfully.

This works fine, but it does add overhead as all http node-red instances will process each response from the rule instances.

This can be overcome if I can pragmatically set the mqtt in node to subscribe to a topic that has it's own PID in it, but I still have to figure that out, as it looks like the topic can only be set via the mqtt in node properties.

Regards
Morne

[msupra]

Hi all

Resolved. Add env variable to settings.js:
process.env.PID = "authentication/resp/"+process.pid;

And then use $(PID) as the topic.

Regards
Morne