I have had a single server in production for more than a year, but I'm reaching the point where I need additional server instances to handle the load. The problem I'm trying to solve is my use of context. I have a database with a list of devices. New devices may be added or removed, or their properties may be updated. Since the admin web page that makes such changes is hosted on the sole server instance, I can trigger the device list to be loaded after changes are applied.
With multiple instances I can't do this any longer because the device list on one server will get updated, but the other server(s) will not receive the update event. I suppose I could call out to the other servers with a web service to force an update, but then I would need awareness of other servers, which seems like a brittle solution.
Or, I could poll the database for changes in the device list, but this has a cost and there will still be some lag time before the servers are in sync.
An easy solution would be to use the contextStorage property to have context stored in a database. Unfortunately, there is no contextStorage support for databases (I use MongoDB).
What are others doing to solve this problem?
context storage is now pluggable so you can extend it.
For example see this as yet un-merged redis example that is work in progress (not yet complete) - https://github.com/node-red/node-red-context-redis/pull/1
1 Like
This appears to be non-trivial. Is there any other documentation that you can refer me to? Any alternative approaches you would recommend? I'm sure that others must have solved this problem.
Hi. Without understanding the whole topic (I e. There are likely other ways to skin the cat you are currently skinning). However if your aim is to synchronise context between node-red instances - and it is important that are all synchronised at the same time I would probably do something like this...
- Maintain (in context) a live object of "servers"
- A edit object of servers.
When editing, edit the edit object and upon completion, send the object via MQTT.
Any other instances (including the one you're on) subscribed to the topic should then receive the edit object and store it in their own context.
Then you could send a "save" topic on MQTT to have each node copy the edit object to live
There are a few other ways around this and you may want to consider interlocking and feedback but I believe MQTT will solve your problem.
Hi Steve, Thanks for the suggestion. If I'm going to be aware of the servers that are live, couldn't I simply send all servers that are not me (the one that caught the UI update event) a web service request to load context? Using MQTT opens up the new challenge of actually running an MQTT server.
MQTT is a pub sub architecture (and extremely easy to setup and run by the way)
I suggested it as your flow wouldn't need to be aware of other node-red to participate.
I.e. pub topic "servers" payload [{IP:""},{IP:""},etc]
Your node red machine(s) would subscribe to topic "servers".
Dead simple.
Not sure how you would achieve this using rest endpoints without for knowledge or coordination.
OK, cool. Sounds promising so I'll give it a whirl.
Thanks!
I suspect that the usual way to approach this would be to make your database servers synchronise over a cluster rather than trying to get Node-RED to do it. They are much more likely to have the technology already available.
For example, I seem to remember that CouchDB is a database that ensures that all instances of the same database across multiple servers are guaranteed to "eventually" be in sync. Of course, you really want a database that will notify listening clients that there is an update.
Assuming you can work that out, all you need of the multiple Node-RED instances is that they have the same flow.
There are many ways to solve this problem, but I tried the MMQT approach and I'm very happy with it. In the flow that updates the entity in question, I publish the name of the updated MongoDB collection on the "updated" topic. There are subscribers to this topic on all servers. They are notified almost instantaneously, at which point they refresh their context. Very straightforward to implement and reasonably efficient.
A couple of drawbacks: 1) I had to spin up (and start paying for) another instance to host MMQT, and 2) I have another single point of failure, i.e. if the MMQT instance goes down, the other servers won't receive updates. But I can work that out.
1 Like
Couldn't you just install mqtt on one of the node red?
There is also an MQTT broker node for node-red
Also you can cluster MQTT.
I used HAProxy as a load balancer and EMQX cluster for MQTT.
Either way, well done. I said MQTT was easy 
It was surprisingly easy to install and to code. The node-red MMQT nodes were easy to configure. So thanks!
Next I have to get this all into a Docker swarm. I'll be back
