I am trying to build a scalable rpa platform with Node-Red. Nod red is great for the orchestration for this. But I run into a small bump, where I can not find any solution yet so I want to ask you for inspiration/feedback.
I am using the docker image of node red in docker swarm so I can easy scale up and down.
With scaling down I run into a problem. When a container is shut down the flows that are still busy are lost and so is the data in transition. Within RPA we try to interact with systems where the two way commit solution will not help us. So the best way would be a graceful shutdown of the node red instance.
The consuming/start nodes that I have in scope are amqp and http.
Does anyone have an idea how to shutdown a node red container gracefully so all the flows will be done before shutting down?
My fallback scenario is to shutdown consumption on the mq layer for example, wait for max time of processing and then scale and start consuming again. But this is far from ideal.