Each weather API returns data in it's own format.
For example, hourly forecast data from OpenWeatherMap's is msg.payload.hourly[]
UK Met Office is msg.payload.features[0].properties.timeSeries[]
Met.no is msg.payload.properties.timeseries[]
Of course the individual data items don't have consistent names between platforms.
I wonder if anyone has produced a uniform format and mappings from the various APIs?
This would help in building a dashboard independent of the data source; whichever is currently best, "load balancing" etc.
I'm affraid this won't work for (very) long, API's are always changing/evolving. Perhaps not for the core properties, but everyone wants different data.
Yes they are always changing.
But if OpenWeatherMap change some fieldnames, then all that the data consumers need is a change in the mapping to a standard format, which might be on Github.
No need for changes to each dashboard code.
And if met.no suddenly decide to make their API free for Norwegians only, we could all switch painlessly to tomorrow.io.
For anything like this I feed the data from the API through MQTT, with meaningful topics. Anything I want to do with the data, display on dashboard or feed it into Influxdb or anything else subscribes to the topics to get the data.
So if I want to use a different API I just need to write the flows to pick the data up from the API and squirt it into MQTT and the rest of the system keeps working.
In fact I do this for all sensors and hardware outputs etc too, so that If I change, for example, a room temperature sensor from 1-wire to Zibgee then all I have to change is the bit between the sensor and MQTT.
It has eased the task of moving my dashboard to the new Dashboard 2.0 as the dashboard picks the data up from MQTT and does not care about where it comes from.
Another advantage is that I can pick up the data on any machine on my network.
The general concept is to split the design into three sections, hardware interface, control or business logic, and UI, with an MQTT interface providing the centralised data source for all three.
Yes MQTT also makes it easy to have the data input and dashboard on different machines, which might be an advantage.
If there was a standard format, that would be what you pass over MQTT.
On some of the original weather nodes that we had under the node-red-web-nodes repository I did try to keep as much of the msg.payload as similar as possible and use other properties for custom/raw data. It would be good to go back and redo these based on latest APIs etc, for exactly the reasons you suggest to try to make them more interchangeable at at least one level.
It would be useful for sure, but not necessarily an easy task.
I have been playing around with MET Office and OWM recently, and got a bit frustrated with the content of the MET APIs. It doesn't provide any sunset times etc. nor a text summary. Also missing is the Air Quality and pollen info that can be seen on the web page.
Queue HTML node and a function using "Cheerio" to get everything from one web request
No API, credentials, or limits.
{"day":"Tue 4 Jun","summary":"Overcast changing to light rain by lunchtime.","sunRise":"04:39","sunSet":"21:30","high":"16.4","low":"7.3","uv":"M","aq":"L","pollen":"L"}
I have been trying initially to standardise the output from OpenWeatherMap and Met.No.
OpenWeatherMap provides "current" conditions. Met.no only has conditions for the start and end of the current hour.
If I want the standardised output to include current, which hourly met.no output should I use - the nearest to now? Some sort of average?
OpenWeatherMap provides "daily" conditions. Met.no only has hourly (which are 6-hourly beyond midnight the day after tomorrow) Which hour should be used for the "daily" forecast?
I guess it's completely meaningless to try and represent next wednesday's weather with a single icon, a single temperature/pressure. It might start sunny and dry but a cold front might bring torrential rain by lunchtime. So how does OpenWeatherMap use their fixed time forecasts to populate daily values?
Personally, I'd probably use the end of the hour since the start of the hour should be the same as either the end of the previous hour or the current conditions. After all, you are looking for a forecast.
Maybe use a different service to get the daily summary?
No, I don't think it is meaningless. The icon should represent the most important weather of the day, all weather forecasts have daily summaries after all.
Yes, that is meaningless, you probably want a max and min values for those.
Think about what your forecasts are important for. In that specific case, what is important is that you are likely to get wet and cold if you don't take a coat when going out that day.
All of the API's that provide daily summaries will have some algorithm that will highlight the important weather feature(s). This is why daily summaries (and particularly longer term forecasts generally have a text description as well.
True, which is why we want to standardise a bit, right?
Weather forecasts are as much art as they are science. There is no such thing as a truly Accurate (capital A) forecast, this is especially true in places like the UK - I won't bore people with the reasons, most people probably already know. A level of interpretation is always needed. That's why I also always look out the window!
So I wouldn't sweat it too much, take the information that is critical for your use.
