Creating multiple files with one msg

[Deuceyd]

Hey All!

Very new here and I'm 100% certain I'm missing something simple so apologies for any stupid questions. I have a fairly simple flow running to check a web folder for a bunch of files and create a html file for each of the files found.

The issue I'm having at the moment is I'm really struggling to find a way to send each msg.payload to my file create node as a separate command, triggering the file node once per payload rather than sending all payloads to the node at once and creating 1 file with say 8 links.

My flow is essentially, check folder & collect the file details in the msg object => set variables for the html start tags, end tags etc => basic function node to combine the details:

msg.fileObj = msg.htmlStart + msg.fileObj.permalink + msg.htmlCloseTag + msg.file + msg.htmlEnd

msg.payload = msg.fileObj
return msg;

I then clear up the msg so only the payload returns and fire it into a file create node. Lets say I have 8 msg.payloads, is there a simple way or a handy piece of javascript I could use to essentially say - send the first payload, then wait... then the second and so on?

Thanks in advance!

[Steve-Mcl]

@Deuceyd

Lets say I have 8 msg.payloads, is there a simple way or a handy piece of javascript I could use to essentially say - send the first payload, then wait... then the second and so on?

The split node is designed for this. Not certain why you say it should "wait" - are you using global/flow context? If no, then you can run this at full speed with no waiting.

If you share your flow (or a simplified, minimal example) with demo data we can show you how.

[Colin]

Or you could node.send() multiple times in a Function node to send multiple messages, if that fits your requirement better. See Writing Functions : Node-RED

[Deuceyd]

Thanks a lot Steve, you're absolutely right that was just bad wording on my part. I dont actually need it to wait as long as each payload is sent to the file create as a separate msg. Here's the key part of my flow at the moment:

[{"id":"03084b6a8123d65d","type":"function","z":"bde6fd980bf6e6c4","name":"","func":"msg.fileObj = msg.htmlStart + msg.fileObj.permalink + msg.htmlCloseTag + msg.file + msg.htmlEnd\n\n//if (msg.fileObj) {\n //node.warn(msg.fileObj[0]); // first match\n\n//}\n\nmsg.payload = msg.fileObj\nreturn msg;\n\n\n\n","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":200,"y":220,"wires":[["3012734f8c47c192"]]},{"id":"a08a6de5f6bfc2f1","type":"change","z":"bde6fd980bf6e6c4","name":"Set Variables","rules":[{"t":"set","p":"htmlStart","pt":"msg","to":"<a href=\"","tot":"str"},{"t":"set","p":"htmlCloseTag","pt":"msg","to":"\">","tot":"str"},{"t":"set","p":"htmlEnd","pt":"msg","to":"</a></p>","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":590,"y":160,"wires":[["03084b6a8123d65d"]]},{"id":"3012734f8c47c192","type":"function","z":"bde6fd980bf6e6c4","name":"","func":"\nvar fileLink = { payload: msg.fileObj};\nreturn fileLink;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":420,"y":220,"wires":[["28837590b8845d38"]]},{"id":"28837590b8845d38","type":"file","z":"bde6fd980bf6e6c4","name":"Create file","filename":"C:\\Users\\ben.readings\\Documents\\Demos\\Client A\\inboundfiles.html","appendNewline":true,"createDir":true,"overwriteFile":"false","encoding":"none","x":670,"y":220,"wires":[["c83081990f44c6db"]]}]

[Deuceyd]

Thanks a lot Steve! You're right no real need to wait, I just want to make sure each of my msg's comes through separately to the file create node. I've added a snippet of my flow below

[{"id":"03084b6a8123d65d","type":"function","z":"bde6fd980bf6e6c4","name":"","func":"msg.fileObj = msg.htmlStart + msg.fileObj.permalink + msg.htmlCloseTag + msg.file + msg.htmlEnd\n\n//if (msg.fileObj) {\n //node.warn(msg.fileObj[0]); // first match\n\n//}\n\nmsg.payload = msg.fileObj\nreturn msg;\n\n\n\n","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":200,"y":220,"wires":[["3012734f8c47c192"]]},{"id":"a08a6de5f6bfc2f1","type":"change","z":"bde6fd980bf6e6c4","name":"Set Variables","rules":[{"t":"set","p":"htmlStart","pt":"msg","to":"<a href=\"","tot":"str"},{"t":"set","p":"htmlCloseTag","pt":"msg","to":"\">","tot":"str"},{"t":"set","p":"htmlEnd","pt":"msg","to":"</a></p>","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":590,"y":160,"wires":[["03084b6a8123d65d"]]},{"id":"3012734f8c47c192","type":"function","z":"bde6fd980bf6e6c4","name":"","func":"\nvar fileLink = { payload: msg.fileObj};\nreturn fileLink;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":420,"y":220,"wires":[["28837590b8845d38"]]},{"id":"28837590b8845d38","type":"file","z":"bde6fd980bf6e6c4","name":"Create file","filename":"C:\\Users\\ben.readings\\Documents\\Demos\\Client A\\inboundfiles.html","appendNewline":true,"createDir":true,"overwriteFile":"false","encoding":"none","x":670,"y":220,"wires":[["c83081990f44c6db"]]}]

[Deuceyd]

Thanks for the help Colin! This is an interesting one, I guess my main concern with using a node.send in the function node was I have an unpredictable number of files. Maybe I could stick the node.send in a for loop or something? Thats something I'm struggling a bit with.

[Steve-Mcl]

looking at the partial flow you have posted, I am confused.

First you say...

@Deuceyd

check a web folder for a bunch of files and create a html file for each of the files found

Are you creating 1 file or multiple files?

"create a html file for each of the files found" implies creating multiple files. however, you have a hard coded (single) file name set/fixed in the write file node!

---

If I were to guess, you want...

1. read a directory listing
2. Generate an a anchor (hyperlink) that points to each file in the diretory
3. Write a file containing a list of hyperlinks

Is that the goal? 1 file to be generated?

[Deuceyd]

Yeah I think you just made me realise my embarassing mistake... on reflection not sure how I expected that to work hardcoded. I'll grab the filename and fire it as an input with each msg object and give that a shot to create each file individually. Thanks a lot for your help mate.

[Colin]

@Deuceyd

check folder & collect the file details

How are you doing that, and in what form do you get the file details?

[Steve-Mcl]

While I was waiting for your repsonse, I knocked this up to generate a single file of hyperlinks.

While it may not be what you want, it may give you some clues...

[{"id":"f6e764337c455bef","type":"fs-ops-dir","z":"ccdc4f1f78201a08","name":"List PDFs in c:\\temp","path":"path","pathType":"msg","filter":"filter","filterType":"msg","dir":"payload","dirType":"msg","x":1620,"y":140,"wires":[["5bb5ae22a5d81a5c"]]},{"id":"3c7cb4afe87b57ec","type":"inject","z":"ccdc4f1f78201a08","name":"","props":[],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":1250,"y":140,"wires":[["34ce3cdda5961df8"]]},{"id":"ef8535503bf95ce8","type":"debug","z":"ccdc4f1f78201a08","name":"debug 97","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":1640,"y":200,"wires":[]},{"id":"5bb5ae22a5d81a5c","type":"function","z":"ccdc4f1f78201a08","name":"make document","func":"const links = []\nconst files = msg.payload\nconst html = []\n\nhtml.push(\"<html>\")\nhtml.push(\"  <head>\")\nhtml.push(\"  <title>File List</title>\")\nhtml.push(\"  </head>\")\nhtml.push(\"  <body>\")\nhtml.push(\"    <p>\")\nfor (let index = 0; index < files.length; index++) {\n    const fullpath = path.join(msg.path, files[index])\n    const text = files[index]\n    const anchor = `      <a href=\"${fullpath}\">${text}</a>`\n    links.push(anchor)\n}\nhtml.push(...links)\nhtml.push(\"    </p>\")\nhtml.push(\"  </body>\")\nhtml.push(\"</html>\")\n\n//make final HTML\nmsg.payload = html.join('\\n')\n\n//generate the save as filename for the \"write file\" node\nmsg.filename = path.join(msg.path, msg.outputFilename)\n\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[{"var":"path","module":"path"}],"x":1440,"y":200,"wires":[["ef8535503bf95ce8"]]},{"id":"34ce3cdda5961df8","type":"change","z":"ccdc4f1f78201a08","name":"Set Variables","rules":[{"t":"set","p":"path","pt":"msg","to":"c:/temp","tot":"str"},{"t":"set","p":"filter","pt":"msg","to":"*.pdf","tot":"str"},{"t":"set","p":"outputFilename","pt":"msg","to":"result.html","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":1410,"y":140,"wires":[["f6e764337c455bef"]]},{"id":"7b56130a0c6bab9b","type":"comment","z":"ccdc4f1f78201a08","name":"Make single file full of hyperlinks","info":"","x":1330,"y":100,"wires":[]}]

NOTE: uses the most excellent node-red-contrib-fs-ops for listing files in a directory (and much ore)

[Deuceyd]

Thanks all very much for your help! There was a fairly major failing in my flow design which was hardcoding the filename (many thanks for picking up on that Steve). Soon as I grabbed the filename from elsewhere and threw the path on the front then fired it to the create file it's all tickedy boo.

Again, a little embarrassing but thanks all for your help!