# Working FTP node ? or direct XML input
**URL:** https://discourse.nodered.org/t/working-ftp-node-or-direct-xml-input/3232
**Category:** General
**Created:** [18 September 2018 00:10 UTC](https://discourse.nodered.org/t/working-ftp-node-or-direct-xml-input/3232 "2018-09-18T00:10:26Z")
**Posts on this page:** 19
**Page:** 1
### Author:  [@craigcurtin](https://discourse.nodered.org/u/craigcurtin)
#### Post date: [18 September 2018 00:10 UTC](https://discourse.nodered.org/t/working-ftp-node-or-direct-xml-input/3232/1 "2018-09-18T00:10:26Z")
Guys,
Wanting to grab some FTP files - see here ftp://ftp2.bom.gov.au/anon/gen/fwo/IDN10064.xml
And put them into node red, so i can try and parse some Sydney weather info from our BOM.
I have looked at each of the FTP nodes and also the forum discussions and it does not seem there is a consensus on a working FTP anonymous node - does anyone have a reliable one they would recommend.
Alternatively a way to pull that XML stream directly into NR and push to a JSON stream so i can drill down and extract what i want ?
regards
Craig
---
### Author:  [@Andrei](https://discourse.nodered.org/u/Andrei)
#### Post date: [18 September 2018 00:51 UTC](https://discourse.nodered.org/t/working-ftp-node-or-direct-xml-input/3232/2 "2018-09-18T00:51:06Z")
> [@craigcurtin](#):
>
> ftp://ftp2.bom.gov.au/anon/gen/fwo/IDN10064.xml
Someone already did this in the past. There is a working solution in the link::
[https://groups.google.com/d/topic/node-red/JLiYwcEiJj0/discussion](https://groups.google.com/d/topic/node-red/JLiYwcEiJj0/discussion)
---
### Author:  [@craigcurtin](https://discourse.nodered.org/u/craigcurtin)
#### Post date: [18 September 2018 00:52 UTC](https://discourse.nodered.org/t/working-ftp-node-or-direct-xml-input/3232/3 "2018-09-18T00:52:27Z")
Thanks Andrei - my google Fu does not seem to be strong at the moment !
Craig
---
### Author:  [@craigcurtin](https://discourse.nodered.org/u/craigcurtin)
#### Post date: [18 September 2018 03:03 UTC](https://discourse.nodered.org/t/working-ftp-node-or-direct-xml-input/3232/4 "2018-09-18T03:03:57Z")
Maybe not so fast - so going through that - they end up giving up on the FTP node as it is broken and instead head down the curl path.
The last message posted though has a flow in it - talking about jsonata - when i paste that and run it i get an error
And here is the flow that produces it.
```auto
[{"id":"270fb26c.d03b6e","type":"inject","z":"4c1ba2a9.c1050c","name":"Additional Info","topic":"","payload":"ftp://ftp.bom.gov.au/anon/gen/fwo/IDV10753.xml","payloadType":"str","repeat":"","crontab":"","once":false,"onceDelay":"","x":410,"y":600,"wires":[["fe0a2149.3aa49"]]},{"id":"fe0a2149.3aa49","type":"exec","z":"4c1ba2a9.c1050c","command":"curl","addpay":true,"append":"","useSpawn":"false","timer":"","name":"","x":540,"y":600,"wires":[["54df56f4.4b8218"],[],[]]},{"id":"54df56f4.4b8218","type":"xml","z":"4c1ba2a9.c1050c","name":"","attr":"","chr":"","x":670,"y":580,"wires":[["8fe62ffa.54b33"]]},{"id":"de2110e4.c03fa","type":"debug","z":"4c1ba2a9.c1050c","name":"","active":true,"console":"false","complete":"payload","x":1010,"y":580,"wires":[]},{"id":"8fe62ffa.54b33","type":"change","z":"4c1ba2a9.c1050c","name":"Get Forcast","rules":[{"t":"move","p":"payload.product.forecast[0].area[52].forecast-period","pt":"msg","to":"weather","tot":"msg"},{"t":"set","p":"payload","pt":"msg","to":"{ \t\"today\":{\t \"date\":msg.weather[0].\"$\".\"start-time-local\" ,\t msg.weather[0].text[1].\"$\".type:product.forecast[0].area[52].\"forecast-period\"[0].text[1].\"_\"\t },\t\"tomorrow\": {\t \"date\": msg.weather[1].\"$\".\"start-time-local\",\t msg.weather[1].element[1].\"$\".type: msg.weather[1].element[1].\"_\",\t msg.weather[1].element[2].\"$\".type: msg.weather[1].element[2].\"_\" & \" degrees\",\t msg.weather[1].element[3].\"$\".type: msg.weather[1].element[3].\"_\" & \" degrees\",\t msg.weather[1].text[1].\"$\".type: msg.weather[1].text[1].\"_\" & \" chance of rain\"\t },\t\"two\": {\t \"date\": msg.weather[2].\"$\".\"start-time-local\",\t msg.weather[2].element[1].\"$\".type: msg.weather[2].element[1].\"_\",\t msg.weather[2].element[2].\"$\".type: msg.weather[2].element[2].\"_\" & \" degrees\",\t msg.weather[2].element[3].\"$\".type: msg.weather[2].element[3].\"_\" & \" degrees\",\t msg.weather[2].text[1].\"$\".type: msg.weather[2].text[1].\"_\" & \" chance of rain\"\t },\t\"three\": {\t \"date\": msg.weather[3].\"$\".\"start-time-local\",\t msg.weather[3].element[1].\"$\".type: msg.weather[3].element[1].\"_\",\t msg.weather[3].element[2].\"$\".type: msg.weather[3].element[2].\"_\" & \" degrees\",\t msg.weather[3].element[3].\"$\".type: msg.weather[3].element[3].\"_\" & \" degrees\",\t msg.weather[3].text[1].\"$\".type: msg.weather[3].text[1].\"_\" & \" chance of rain\"\t },\t\"four\": {\t \"date\": msg.weather[4].\"$\".\"start-time-local\",\t msg.weather[4].element[1].\"$\".type: msg.weather[4].element[1].\"_\",\t msg.weather[4].element[2].\"$\".type: msg.weather[4].element[2].\"_\" & \" degrees\",\t msg.weather[4].element[3].\"$\".type: msg.weather[4].element[3].\"_\" & \" degrees\",\t msg.weather[4].text[1].\"$\".type: msg.weather[4].text[1].\"_\" & \" chance of rain\"\t},\t\"five\": {\t \"date\": msg.weather[5].\"$\".\"start-time-local\",\t msg.weather[5].element[1].\"$\".type: msg.weather[5].element[1].\"_\",\t msg.weather[5].element[2].\"$\".type: msg.weather[5].element[2].\"_\" & \" degrees\",\t msg.weather[5].element[3].\"$\".type: msg.weather[5].element[3].\"_\" & \" degrees\",\t msg.weather[5].text[1].\"$\".type: msg.weather[5].text[1].\"_\" & \" chance of rain\"\t },\t\"six\": {\t \"date\": msg.weather[6].\"$\".\"start-time-local\",\t msg.weather[6].element[1].\"$\".type: msg.weather[6].element[1].\"_\",\t msg.weather[6].element[2].\"$\".type: msg.weather[6].element[2].\"_\" & \" degrees\",\t msg.weather[6].element[3].\"$\".type: msg.weather[6].element[3].\"_\" & \" degrees\",\t msg.weather[6].text[1].\"$\".type: msg.weather[6].text[1].\"_\" & \" chance of rain\"\t}\t}","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":830,"y":580,"wires":[["de2110e4.c03fa"]]}]
```
Guess i will rip it apart and see if i can see anything
I know nothing abut jsonata - is there an easy expression builder etc ??
Craig
---
### Author:  [@Andrei](https://discourse.nodered.org/u/Andrei)
#### Post date: [18 September 2018 04:05 UTC](https://discourse.nodered.org/t/working-ftp-node-or-direct-xml-input/3232/5 "2018-09-18T04:05:41Z")
> [@craigcurtin](#):
>
> Guess i will rip it apart and see if i can see anything
This is exactly what you should do. I see that the JSONata expression is trying to read some elements that does not exist in the input JSON data. I removed a few ones from the change rule and it worked. You will need to compare that data received from the API against with the JSONata expression to see what fix is necessary.
```auto
[{"id":"2e7939ce.b5aec6","type":"tab","label":"Flow 2","disabled":false,"info":""},{"id":"d57fbd39.268b4","type":"inject","z":"2e7939ce.b5aec6","name":"Additional Info","topic":"","payload":"ftp://ftp.bom.gov.au/anon/gen/fwo/IDV10753.xml","payloadType":"str","repeat":"","crontab":"","once":false,"onceDelay":"","x":170,"y":160,"wires":[["68616787.fc9de8"]]},{"id":"68616787.fc9de8","type":"exec","z":"2e7939ce.b5aec6","command":"curl","addpay":true,"append":"","useSpawn":"false","timer":"","name":"","x":300,"y":160,"wires":[["809d8a68.26d8e8"],[],[]]},{"id":"809d8a68.26d8e8","type":"xml","z":"2e7939ce.b5aec6","name":"","attr":"","chr":"","x":430,"y":140,"wires":[["2a45783b.a03f68","518ff716.915678"]]},{"id":"a7fe50ee.4daa8","type":"debug","z":"2e7939ce.b5aec6","name":"","active":true,"console":"false","complete":"payload","x":770,"y":140,"wires":[]},{"id":"2a45783b.a03f68","type":"change","z":"2e7939ce.b5aec6","name":"Get Forcast","rules":[{"t":"move","p":"payload.product.forecast[0].area[52].forecast-period","pt":"msg","to":"weather","tot":"msg"},{"t":"set","p":"payload","pt":"msg","to":"{\t \"today\":{\t \"date\":msg.weather[0].\"$\".\"start-time-local\" ,\t msg.weather[0].text[1].\"$\".type:product.forecast[0].area[52].\"forecast-period\"[0].text[1].\"_\"\t \t },\t \"tomorrow\": {\t \"date\": msg.weather[1].\"$\".\"start-time-local\",\t msg.weather[1].element[1].\"$\".type: msg.weather[1].element[1].\"_\",\t msg.weather[1].element[2].\"$\".type: msg.weather[1].element[2].\"_\" & \" degrees\",\t msg.weather[1].element[3].\"$\".type: msg.weather[1].element[3].\"_\" & \" degrees\",\t msg.weather[1].text[1].\"$\".type: msg.weather[1].text[1].\"_\" & \" chance of rain\"\t \t },\t \"two\": {\t \"date\": msg.weather[2].\"$\".\"start-time-local\",\t msg.weather[2].element[1].\"$\".type: msg.weather[2].element[1].\"_\",\t msg.weather[2].element[2].\"$\".type: msg.weather[2].element[2].\"_\" & \" degrees\",\t msg.weather[2].element[3].\"$\".type: msg.weather[2].element[3].\"_\" & \" degrees\",\t msg.weather[2].text[1].\"$\".type: msg.weather[2].text[1].\"_\" & \" chance of rain\"\t \t }\t \t }","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":590,"y":140,"wires":[["a7fe50ee.4daa8"]]},{"id":"518ff716.915678","type":"debug","z":"2e7939ce.b5aec6","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":600,"y":200,"wires":[]}]
```
---
### Author:  [@craigcurtin](https://discourse.nodered.org/u/craigcurtin)
#### Post date: [18 September 2018 04:57 UTC](https://discourse.nodered.org/t/working-ftp-node-or-direct-xml-input/3232/6 "2018-09-18T04:57:09Z")
Thanks for that - i will put in some time tonight and try to work out what this is all doing ! (and more importantly how it does it !)
Craig
---
### Author:  [@shrickus](https://discourse.nodered.org/u/shrickus)
#### Post date: [18 September 2018 05:02 UTC](https://discourse.nodered.org/t/working-ftp-node-or-direct-xml-input/3232/7 "2018-09-18T05:02:02Z")
That JSONata expression is much more complicated than it needs to be, just to provide some named fields ("today", "tomorrow", "two", etc). The only difference between each day's object (other than today) is the index number into the array.
Instead, I suggest mapping the 7-element `forecast-period` array, using this expression:
```auto
(
$weather := payload.product.forecast[0].area[52]."forecast-period";
$date := $weather.{
"date": $."$"."start-time-local"
};
$elem := $weather.[element].{
$."$".type: $."_" & ($."$".units ? "°" & $."$".units.$substring(0,1) : "")
};
$text := $weather.[text].{
$."$".type: $."_"
};
$zip($date, $elem, $text).$merge($)
)
```
This produces a 7 day array of forecast objects -- which looks like this:
```auto
[
{
"date": "2018-09-18T05:00:00+10:00",
"forecast_icon_code": "11",
"precipitation_range": "1 to 2 mm",
"air_temperature_maximum": "20°C",
"precis": "Shower or two. Windy.",
"probability_of_precipitation": "70%"
},
{
"date": "2018-09-19T00:00:00+10:00",
"forecast_icon_code": "11",
"precipitation_range": "1 to 2 mm",
"air_temperature_minimum": "7°C",
"air_temperature_maximum": "15°C",
"precis": "Showers.",
"probability_of_precipitation": "80%"
},
{
"date": "2018-09-20T00:00:00+10:00",
"forecast_icon_code": "3",
"precipitation_range": "0 to 0.2 mm",
"air_temperature_minimum": "6°C",
"air_temperature_maximum": "16°C",
"precis": "Partly cloudy.",
"probability_of_precipitation": "30%"
},
{
"date": "2018-09-21T00:00:00+10:00",
"forecast_icon_code": "3",
"air_temperature_minimum": "8°C",
"air_temperature_maximum": "18°C",
"precis": "Partly cloudy.",
"probability_of_precipitation": "5%"
},
{
"date": "2018-09-22T00:00:00+10:00",
"forecast_icon_code": "17",
"precipitation_range": "0 to 1 mm",
"air_temperature_minimum": "9°C",
"air_temperature_maximum": "20°C",
"precis": "Possible late shower.",
"probability_of_precipitation": "40%"
},
{
"date": "2018-09-23T00:00:00+10:00",
"forecast_icon_code": "11",
"precipitation_range": "0 to 1 mm",
"air_temperature_minimum": "8°C",
"air_temperature_maximum": "14°C",
"precis": "Shower or two.",
"probability_of_precipitation": "50%"
},
{
"date": "2018-09-24T00:00:00+10:00",
"forecast_icon_code": "3",
"air_temperature_minimum": "6°C",
"air_temperature_maximum": "16°C",
"precis": "Partly cloudy.",
"probability_of_precipitation": "5%"
}
]
```
So the `msg.payload[0]` object represents today, `msg.payload[1]` is one day from now or tomorrow, and so on. The UI can add the human-readable day offset labels, or extra wording.
---
### Author:  [@craigcurtin](https://discourse.nodered.org/u/craigcurtin)
#### Post date: [18 September 2018 05:12 UTC](https://discourse.nodered.org/t/working-ftp-node-or-direct-xml-input/3232/8 "2018-09-18T05:12:19Z")
Thank you very much - that will give me more to compare with - is there a simple discussion/document on using jsonata like this - obviously i can understand the result of what you are doing - but the mechanics of geting there is another matter.
I assume that we are essentially specifing variable objects such as $weather and then trying to populate those based on looking inside the XML - which it appears we are treating as Arrays (such as area[52].
But then looking at something like the $elem is just blurring before my eyes !!
Looks like lots of reading to do here
Craig
---
### Author:  [@TotallyInformation](https://discourse.nodered.org/u/TotallyInformation)
#### Post date: [18 September 2018 06:00 UTC](https://discourse.nodered.org/t/working-ftp-node-or-direct-xml-input/3232/9 "2018-09-18T06:00:21Z")
The cookbook should have some ideas. I know there were some suggestions in the cookbook WIKI, looks like Steve's example would be a great one to add to that (with some annotation maybe 🙂)
I suspect that, outside of IBM, Steve may be the best expert on JSONata!
Actually, I nearly forgot, I put all of my examples that were in the WIKI into a knowledgebase post on my blog:
> **[Using JSONata to transform msg data](https://it.knightnet.org.uk/kb/nr-qa/jsonata/)**
>
> JSONata is a syntax designed by IBM to help reformat and restructure JSON data in a similar vein to the way that XLST is used to transform XML data.
I might nick this one too!
---
### Author:  [@craigcurtin](https://discourse.nodered.org/u/craigcurtin)
#### Post date: [18 September 2018 06:24 UTC](https://discourse.nodered.org/t/working-ftp-node-or-direct-xml-input/3232/10 "2018-09-18T06:24:57Z")
Thanks for that update. I went to the [jsonata.org](http://jsonata.org) website and played around with their stuff and have a bit of an idea now.
Stilll struggling to understand how the stuff comes from the xml and then gets transformed - will continue reading and playing with the data that i have until i crack it
Craig
---
### Author:  [@craigcurtin](https://discourse.nodered.org/u/craigcurtin)
#### Post date: [18 September 2018 06:57 UTC](https://discourse.nodered.org/t/working-ftp-node-or-direct-xml-input/3232/11 "2018-09-18T06:57:07Z")
OK obviously i am missing something basic here. I assume even as good as Steve is - he can not look at that XLM output and know where the various arrays are going to be etc.
There must be a way that someone takes that XML string and then plays with it interactively to identify the areas and values of interest ?
Is that just through the debug node and then grabbing the path in the debug window or is there some other fancy tool that helps with this and hence then makes it easier to build the jsonata ?
Craig
---
### Author:  [@dceejay](https://discourse.nodered.org/u/dceejay)
#### Post date: [18 September 2018 07:45 UTC](https://discourse.nodered.org/t/working-ftp-node-or-direct-xml-input/3232/12 "2018-09-18T07:45:22Z")
If you play withe change node - jsonata option - you can hit the three dots on the right to open the jsonata editor and tester
This allows you to paste in your expression and then some test data and see the result as you play.
---
### Author:  [@shrickus](https://discourse.nodered.org/u/shrickus)
#### Post date: [18 September 2018 12:56 UTC](https://discourse.nodered.org/t/working-ftp-node-or-direct-xml-input/3232/13 "2018-09-18T12:56:34Z")
As Dave mentioned, I rely on the JSONata tester found in the `change` node pretty heavily. First, use the debug sidebar buttons to copy the value of the payload to the clipboard, then paste it into the tester...
However, your data was so big that it was easier to use the online tester at [try.jsonata.org](http://try.jsonata.org)
And you are right -- there's no way to just look at your data and know how to access the fields. For that, I rely on the debug sidebar again -- when you hover the mouse on the data you want, the "copy path" button appears on the right. Your data structure is made extra complicated because it was converted to a JS object from XML, so I would NOT suggest trying to learn JSONata with this data!
---
### Author:  [@craigcurtin](https://discourse.nodered.org/u/craigcurtin)
#### Post date: [18 September 2018 20:15 UTC](https://discourse.nodered.org/t/working-ftp-node-or-direct-xml-input/3232/14 "2018-09-18T20:15:33Z")
Aaah - i thought there was a magic step i was missing !!!
Thanks for the explanation - i will try it with the real data for the actual area i am interested in and see if i can get any further
regards
Craig
---
### Author:  [@craigcurtin](https://discourse.nodered.org/u/craigcurtin)
#### Post date: [18 September 2018 20:30 UTC](https://discourse.nodered.org/t/working-ftp-node-or-direct-xml-input/3232/15 "2018-09-18T20:30:24Z")
Would it be easier to leave it as XML or will the Jsonata tool not accept that ?
Craig
---
### Author:  [@shrickus](https://discourse.nodered.org/u/shrickus)
#### Post date: [18 September 2018 20:33 UTC](https://discourse.nodered.org/t/working-ftp-node-or-direct-xml-input/3232/16 "2018-09-18T20:33:30Z")
Uh, that would be the XMLonata tool ;\*) ... if there was one
But if you really want to process XML instead, skip the `xml` and `change` nodes, install `node-red-contrib-xsltransform` and write some xslt rules...
---
### Author:  [@craigcurtin](https://discourse.nodered.org/u/craigcurtin)
#### Post date: [18 September 2018 20:42 UTC](https://discourse.nodered.org/t/working-ftp-node-or-direct-xml-input/3232/17 "2018-09-18T20:42:04Z")
OH right - so no is the answer !!! Already opened up one can of worms - dont need a second one.
I will knuckle down over the next couple of days and see what i can nut out
thanks for all your help Steve
Craig
---
### Author:  [@TotallyInformation](https://discourse.nodered.org/u/TotallyInformation)
#### Post date: [18 September 2018 23:12 UTC](https://discourse.nodered.org/t/working-ftp-node-or-direct-xml-input/3232/18 "2018-09-18T23:12:34Z")
> [@craigcurtin](#):
>
> OH right - so no is the answer !!! Already opened up one can of worms - dont need a second one.
Maybe one day we will get a XSLT node (the XML equivalent of JSONata). But most of us are using JSON over XML these days. We also have the XML node that makes a valiant attempt at turning XML into JSON.
---
### Author:  [@craigcurtin](https://discourse.nodered.org/u/craigcurtin)
#### Post date: [19 September 2018 00:06 UTC](https://discourse.nodered.org/t/working-ftp-node-or-direct-xml-input/3232/19 "2018-09-19T00:06:44Z")
I hope not Julian - it is enough with jsonata !!
The XML node appears to be swallowing this stuff and spitting out something that jsonata can work with so i am slowly pushing my way through !
Craig