Use of NodeRed for international weather, climate, ocean, hydrology data exchange

Hello,
I wanted to share with the community what we have implemented using NodeRed (Thanks to all for the great tool) in the World Meteorological Organisation (WMO).
WMO is a United Nation entity in charge of weather, climate, ocean, hydrology, see https://www.wmo.int
One key requirement is the exchange of data in "real time" (not miliiseconds but rapidly - less that a minute -).
For this need we have designed a new method called WIS2 (WMO Information System version 2).
An introduction is available here: WIS 2.0 | World Meteorological Organization

The system is build around the use of MQTT. For redundancy purposes we will have multiple brokers (RabbitMQ, VerneMQ,...) operated by different National Weather Centers. Those brokers are creating a mesh so that all messages are seen by all brokers. As part of this (and to avoid an infinite exchange of messages) we need what we have called an anti loop feature.

This is implemented using Node Red! We are taking benefits and some great nodes: MQTT, redis, prometheus... The main flow itself is fairly basic:

We also have made a docker container out of it see GitHub - golfvert/WIS2-GlobalBroker-NodeRed: Basic reference of a Global Broker for WIS2

At the moment, we have one docker container per data source. It may not be the most efficient way but is very flexible as all sources are independent and you may want to add/remove them without affecting the others.

So far, NodeRed has proven to be extremely reliable and has allowed us to tweak quickly the flows as the architecture was not fully finalised.

Once again, great stuff from all devs and contributors on the forum.

Best wishes for 2023.

11 Likes

Nice! All the best with that project.

The best ones always are :-)... simple, repeatable, reliable, debuggable. Gets my vote.