Clustered multi instance node-red question

General
1

I have created a 3-node cluster. I use keepalived to publish a single ip address for the cluster. I have 3 instances of node-red running, one on each node, all sharing the same set of flow files and settings.js.

When I go to update a flow either using the keepalived ip address or a single node ip address, it seems as if the synchronization is getting confused. I do have multiplayer enabled.

Is what I am doing even doable? Am I missing something?

2

Typically keepalived is used to share an IP address between different hosts. This is useful for incoming connections. Is that what your flows are doing and only this ?
When you deploy the flows, is there a chance that affects what is used by keepalived to assess that everything is normal ?
So that the IP address is moved to another host... Hence the potential confusion.

3

I agree with your take on keepalived. My thought that with multiplayer turned on and the node-red files being shared, that node-red would take care of changes no matter where they came from. Maybe I am misunderstanding what multiplayer does. I thought it was to synchronize node-red when multiple people were making changes. So I am thinking that it doesn't matter which node-red that keepalived sends me to to do my edits. That the rest would just sync up so when a query came in it too would not matter which node-red was queried.

4

Multiplayer mode lets multiple browsers see what other browsers are doing to a single back end server. IE many clients to one server. I think what you are describing is several servers trying to stay in sync by sharing a flows (and settings) file.

That is not the same, and is not supported.

One reason being that when you hit deploy, only the instance that currently has the ip address will get the restart request. It will rewrite the updated flows file, but it can’t tell the other servers to restart.

5

A solution is to write a storage plugin that uses a central shared storage, e.g. a database

And when ever the flow is updated the storage plugin can trigger all 3 instances to reload the flows from the database.

1 Like
6

I can easily create a manual trigger as well when I update a flow. But how do I trigger the instances to reload the flows? Do a ssh based restart?

7

The Admin API has an option to reload flows

https://nodered.org/docs/api/admin/methods/post/flows/

8

So I make a change on node1 (I turned authentication off for testing) and created a http POST node with the correct headers to reload the flows for node2. I am getting a v1 204 success but the flows are not getting reloaded on node2.

Thoughts?

image
image524Ă—573 11.9 KB

9

What version of Node-RED are you using?

But the following is working as expected for me:

  1. Start Node-RED with empty flow
  2. create inject -> debug with msg.payload set to Hello and inject on start ticked and debug set to print to debug sidebar and console
  3. Deploy
  4. See output in both sidebar and console
  5. edit the flow.json file to change Hello to HelloWorld
  6. run following curl command
    curl http://localhost:1880/flows -H "Node-RED-Deployment-Type: reload" -v -X POST
  7. see new output with HelloWorld in both sidebar and console (also see warning about unecrypted creds...)
10

Thanks you for that example. It does seem to be working now.

Is there a way to turn off all of the merge notifications? I don't need them.

And what should I be setting the multiplayer value to?

11

You can't turn off merge notifications

Multiplayer shouldn't be in play here, because you should ONLY be interacting with one instance (not thought the load balancer) when editing

12

Understood. Thanks.

But, why not the load balancer? Does it really matter which instance is making the changes?