Has anyone managed to download map images from the Met Office?
Is there something peculiar about the format?
I have httpStatic setup correctly, so I can display some downloaded PNG images on the dashboard.
The Met Office image downloads, and the first few bytes in the file include "PNG" but I can't display it in Node-red or in any image editor applications.
Is there another step needed in my flow?
Hard to see how you are saving it from info you supplied. I can not remember my metoffice api username and i would have to email or call to find it so can not test download.
Are you using set from msg.encoding or default or binary? when saving to httpstatic.
What API are you trying to use?
The images seem to return GIF's
http://datapoint.metoffice.gov.uk/public/data/image/wxfcs/surfacepressure/gif?timestep=0&key=<your-api-key>
Sorry for the delay answering.
I was using default. I have also tried utf8.
The HTTP node I have tried returning a utf8 string and a buffer. No ide what the right combination is.
I wish that was a simple question to answer!
The API doucumentation is at https://metoffice.apiconnect.ibmcloud.com/metoffice/production/map-images-api-documentation#/MapImagesDataRESTAPI_111/overview
The list of services I am subscribed to is at https://gridded-data-ui.cda.api.metoffice.gov.uk/subscriptions
Here is the message coming from the HTTP node. I think content type image/png confirms it's sending a PNG file not a GIF
Oh, the payload:
The URL I shared certainly returned a GIF. There is a URL that returns all the possible image URL's. That's how I got the URL where I got the image.
But yes, that does appear to be a PNG.
What I'd suggest is to dump the file to a file and look at it using an image viewer.
The available image viewers (Irfanview, jpegview and Photofiltre) can't open it.
That's after ftping it to a PC. Not explicitly set as binary mode but copying other images works.
Let me have the URL (minus your API key) and I'll try it tomorrow.
Your http request node should return a binary buffer.
You should use msg.encoding to save to file, default and binary should work also. Do not check add newline to payload
You should recieve a buffer from the http request like below
Try this png as a test
https://pluspng.com/img-png/images-owls-png-hd-owl-free-download-png-png-image-485.png
OK Thanks Julian. I'm not sure how much use it will be because it requires an "order id" for this map well as the client-id and client-secret, all redacted below.
This is the "Weather Data Hub"
[{"id":"5631f7912941c322","type":"group","z":"c91127da6284d27f","name":"Get met office image","style":{"label":true},"nodes":["f702f183af723971","5d9e0666bcc209cf","42481690ebe765d5","eac3d3c6a4ac4346","085eaef8065fde64"],"x":14,"y":79,"w":832,"h":122},{"id":"f702f183af723971","type":"inject","z":"c91127da6284d27f","g":"5631f7912941c322","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":120,"y":120,"wires":[["5d9e0666bcc209cf"]]},{"id":"5d9e0666bcc209cf","type":"function","z":"c91127da6284d27f","g":"5631f7912941c322","name":"set payload and headers","func":"const clientid = flow.get(\"MetClientID\") || \"32hex\"\nconst clientsecret = flow.get(\"MetClientSecret\") || \"32hex\"\nconst orderid = flow.get(\"MetOrderNo\") || \"o12digits\"\n\nmsg.url = 'https://api-metoffice.apiconnect.ibmcloud.com/metoffice/production/map-images/1.0.0/orders/' + orderid + '/latest/land_cover_ts0_+00/data'\nmsg.headers = {\n 'x-ibm-client-id': clientid,\n 'x-ibm-client-secret': clientsecret\n}\nreturn msg;\n","outputs":1,"timeout":"","noerr":0,"initialize":"","finalize":"","libs":[],"x":330,"y":120,"wires":[["42481690ebe765d5"]]},{"id":"42481690ebe765d5","type":"http request","z":"c91127da6284d27f","g":"5631f7912941c322","name":"","method":"GET","ret":"txt","paytoqs":"ignore","url":"","tls":"","persist":false,"proxy":"","insecureHTTPParser":false,"authType":"","senderr":false,"headers":[],"x":550,"y":120,"wires":[["085eaef8065fde64","eac3d3c6a4ac4346"]]},{"id":"eac3d3c6a4ac4346","type":"file","z":"c91127da6284d27f","g":"5631f7912941c322","name":"","filename":"/home/pi/.node-red/node-red-static/foo.png","filenameType":"str","appendNewline":false,"createDir":false,"overwriteFile":"true","encoding":"utf8","x":650,"y":160,"wires":[[]]},{"id":"085eaef8065fde64","type":"debug","z":"c91127da6284d27f","g":"5631f7912941c322","name":"debug 402","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":730,"y":120,"wires":[]}]
OK, try @E1cid's suggestion first then. I've a basic MetOffice login but think I'd need to get an additional subscription (though still a free tier it seems). That perhaps also explains why the image I got was a GIF.
Also, can you try the request I shared to see if you get a valid GIF?
Some progress!
I can download @E1cid's sample PNG and display it on the dashboard. Also of course, at IPAddress:1880/imagename.png
Now I have learned some basic stuff, back to the met office image...
Edit: @TotallyInformation's GIF downloads and displays too. This is the older Datapoint API but I managed to find an API key for this too.
Finally, success! And good grief what an attractive and informative map it isn't.
For anyone else struggling to do the same, here is the flow
[{"id":"5631f7912941c322","type":"group","z":"c91127da6284d27f","name":"Get met office image","style":{"label":true},"nodes":["f702f183af723971","5d9e0666bcc209cf","42481690ebe765d5","eac3d3c6a4ac4346","085eaef8065fde64","86f5347dd5e757b5","56c52ebeef0d8c7c"],"x":14,"y":79,"w":832,"h":122},{"id":"f702f183af723971","type":"inject","z":"c91127da6284d27f","g":"5631f7912941c322","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":120,"y":120,"wires":[["5d9e0666bcc209cf"]]},{"id":"5d9e0666bcc209cf","type":"function","z":"c91127da6284d27f","g":"5631f7912941c322","name":"set payload and headers","func":"const clientid = flow.get(\"MetClientID\") || \"32hex\"\nconst clientsecret = flow.get(\"MetClientSecret\") || \"32hex\"\nconst orderid = flow.get(\"MetOrderNo\") || \"o12digits\"\n\nmsg.url = 'https://api-metoffice.apiconnect.ibmcloud.com/metoffice/production/map-images/1.0.0/orders/' + orderid + '/latest/land_cover_ts0_+00/data'\nmsg.headers = {\n 'x-ibm-client-id': clientid,\n 'x-ibm-client-secret': clientsecret\n}\nreturn msg;\n","outputs":1,"timeout":"","noerr":0,"initialize":"","finalize":"","libs":[],"x":330,"y":120,"wires":[["42481690ebe765d5"]]},{"id":"42481690ebe765d5","type":"http request","z":"c91127da6284d27f","g":"5631f7912941c322","name":"","method":"GET","ret":"bin","paytoqs":"ignore","url":"","tls":"","persist":false,"proxy":"","insecureHTTPParser":false,"authType":"","senderr":false,"headers":[],"x":550,"y":120,"wires":[["085eaef8065fde64","eac3d3c6a4ac4346"]]},{"id":"eac3d3c6a4ac4346","type":"file","z":"c91127da6284d27f","g":"5631f7912941c322","name":"","filename":"/home/pi/.node-red/node-red-static/foo.png","filenameType":"str","appendNewline":false,"createDir":false,"overwriteFile":"true","encoding":"setbymsg","x":650,"y":160,"wires":[[]]},{"id":"085eaef8065fde64","type":"debug","z":"c91127da6284d27f","g":"5631f7912941c322","name":"debug 402","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":730,"y":120,"wires":[]},{"id":"86f5347dd5e757b5","type":"inject","z":"c91127da6284d27f","g":"5631f7912941c322","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":120,"y":160,"wires":[["56c52ebeef0d8c7c"]]},{"id":"56c52ebeef0d8c7c","type":"ui_template","z":"c91127da6284d27f","g":"5631f7912941c322","group":"f149abdd741ca58e","name":"","order":3,"width":0,"height":0,"format":"<div>\n <img src = \"/foo.png\"></img>\n</div>","storeOutMessages":true,"fwdInMessages":true,"resendOnRefresh":true,"templateScope":"local","className":"","x":280,"y":160,"wires":[[]]},{"id":"f149abdd741ca58e","type":"ui_group","name":"Demo","tab":"1caa8458.b17814","order":1,"disp":true,"width":"12","collapse":false,"className":"test"},{"id":"1caa8458.b17814","type":"ui_tab","name":"Demo","icon":"dashboard","order":3,"disabled":false,"hidden":false}]
Glad the effort was worth it!
Thanks for your help @E1cid and @TotallyInformation
This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.
Copyright OpenJS Foundation and Node-RED contributors. All rights reserved. The OpenJS Foundation has registered trademarks and uses trademarks. For a list of trademarks of the OpenJS Foundation, please see our Trademark Policy and Trademark List. Trademarks and logos not indicated on the list of OpenJS Foundation trademarks are trademarks™ or registered® trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them.
The OpenJS Foundation | Terms of Use | Privacy Policy | OpenJS Foundation Bylaws | Trademark Policy | Trademark List | Cookie Policy