What is the best way to keep data synced, from say an Redis, Timeseries, MQTT, NoSQL or SQL database in Node Red?
For example, lets say I have something that triggers a flow, and a function node does something with the properties attached to msg. One of the properties is the payload, and another one called timeout. Lets also say that payload is always sent to this function node, but timeout is optional, and if it's not set, then the data should default to some value that's stored in a Redis key or an MQTT path.
In the Unreal Engine they handle this by adding multiple input nodes (you can see a Vector being injected into the AddActorWorldOffset node when SpaceBar is pressed):
This is a good solution, but not sure how well it fits into NodeRed.
Currently I have nodes that monitor Redis and MQTT, watching/polling for updates, and then pushing any changes to the global context, but I was wondering if there was a more elegant way to achieve this? It may cause a node to become an "await" node as it pulls the data from the external source, which would add a slight delay. Prefetching possible data for flows at flow start time would be useful here, or maybe caching and setting a cache stale timeout of cached data and refetching may also be a good solution.
I usually solve this by using a Join node to join together the source date messages into one message which is then passed to the node that requires it.
I think I get what you mean. Just have MQTT or Redis node connected to the join node. I guess that works, fine.
The reason I'm doing this is because I run NodeRed in a K8s cluster with one "master" instance (for updating flows etc), and some cloned instances. In order to avoid Node Red firing off multiple events (one per instance) if a new MQTT message comes in, I use Redis as a type of token store for the first instance that writes to NodeRed for that message ID. It's complex, but it works.
If you need to know which input has changed you can check the topic on message out of the Join node, and it will be the topic of incoming message. So if you only want to act when a new message is received from a particular source you can use a Switch on the output to only pass messages with that topic.
Yep, problem is when there are several instances of the same Node Red running across several K8s Nodes, they'd all fire off the same event since they'd all receive the same MQTT payload. Using Redis to hold which NodeRed instance got in first seems to prevent it.
Using the join node, I can now simplify a lot of flows when reading from Redis, thank you!
