Keep the filename of an attachment when downloading it with the email node

General
1

Hello everyone, I have been discovering node-red on a Pi for a few months without any notion of programming.
By helping me with the different forum posts, I managed to create a small process that allows me:

  1. to retrieve attachments from my mailbox
  2. to store them in a directory on my Pi.
    However, I am unable to keep the name of the downloaded file. I think the problem is in the second change node where I unsuccessfully attempt to set msg.filename.
    I have tried and retried a lot of solutions but nothing helps.
    Could someone give me some directions?
    Thank you in advance.
[{"id":"c128c243.3d8ca","type":"change","z":"a20d7f28.025a2","name":"","rules":[{"t":"set","p":"payload","pt":"msg","to":"attachments","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":200,"y":180,"wires":[["6ef8cc1c.4b88e4","73cba29c.810c5c"]]},{"id":"6ef8cc1c.4b88e4","type":"split","z":"a20d7f28.025a2","name":"","splt":"\\n","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":370,"y":180,"wires":[["2a8396a0.bf301a","da5f185e.b7a3c8"]]},{"id":"d59830f4.bb3a8","type":"file","z":"a20d7f28.025a2","name":"Save picture to disk","filename":"","appendNewline":true,"createDir":true,"overwriteFile":"true","encoding":"none","x":1010,"y":180,"wires":[[]]},{"id":"83af9472.1aba98","type":"inject","z":"a20d7f28.025a2","name":"trigger email fetch","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":true,"onceDelay":0.1,"x":230,"y":80,"wires":[["16bcdf7c.2563f1"]]},{"id":"16bcdf7c.2563f1","type":"e-mail in","z":"a20d7f28.025a2","name":"Fetch Emails","protocol":"IMAP","server":"imap.gmail.com","useSSL":true,"port":"993","box":"","disposition":"Read","criteria":"UNSEEN","repeat":"300","fetch":"trigger","inputs":1,"x":510,"y":80,"wires":[["c128c243.3d8ca"]]},{"id":"73cba29c.810c5c","type":"debug","z":"a20d7f28.025a2","name":"set","active":true,"tosidebar":true,"console":false,"tostatus":true,"complete":"payload","targetType":"msg","x":150,"y":240,"wires":[]},{"id":"2a8396a0.bf301a","type":"debug","z":"a20d7f28.025a2","name":"split","active":true,"tosidebar":true,"console":false,"tostatus":true,"complete":"payload","targetType":"msg","x":350,"y":240,"wires":[]},{"id":"df95f50d.dc4548","type":"debug","z":"a20d7f28.025a2","name":"compose","active":true,"tosidebar":true,"console":false,"tostatus":true,"complete":"payload","targetType":"msg","x":610,"y":240,"wires":[]},{"id":"da5f185e.b7a3c8","type":"change","z":"a20d7f28.025a2","name":"compose filename and load content to payload","rules":[{"t":"set","p":"filename","pt":"msg","to":"/home/pi/.local/share/kiosque/ref/photos/{{filename}}","tot":"json"},{"t":"set","p":"payload","pt":"msg","to":"payload.content","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":660,"y":180,"wires":[["d59830f4.bb3a8","df95f50d.dc4548"]]}]
2

At least one issue is that in the change node "compose filename and load content to payload", your setting the file name as an object ({} icon) instead of string.

3

Thank you for your help !
I tried in "string" but what I got in my target directory is a file named "{{filename}}".

4

here is what I would do. in the change node set msg.filepath to /home/pi/.local/share/kiosque/ref/photos/thefilename
Add a second rule using 'change' and msg.filepath. In the options use 'Search for' as thefilename and the replace as msg.filename
add a third rule to move msg.filepath to msg.filename and you should be all set

5

Thanks !
i'll try il right now

6

I tried but now my file (which goes well in the correct directory) is called: "Undefined".
I recheck my entries several times. No doubt an error in my drafting of the 3 rules?
Below is the change node.

[{"id":"8304bf73.c7f3f","type":"change","z":"8424aab0.5f9b18","name":"compose filename and load content to payload","rules":[{"t":"set","p":"filepath","pt":"msg","to":"/home/pi/.local/share/kiosque/ref/photos/thefilename","tot":"str"},{"t":"change","p":"filepath","pt":"msg","from":"thefilename","fromt":"str","to":"filename","tot":"msg"},{"t":"move","p":"filepath","pt":"msg","to":"filename","tot":"msg"},{"t":"set","p":"payload","pt":"msg","to":"payload.content","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":1840,"y":180,"wires":[["814b9bab.36a1b8","6d429d40.c4c574"]]}]

Debug node of the split node just before the change node :

18/09/2020 à 18:13:24[node: split]
(http://localhost:1880/admin/#)test image : msg.payload : Object
Object
type: "attachment"
content: buffer[18972]
contentType: "image/jpeg"
partId: "2"
release: null
contentDisposition: "attachment"
filename: "11.jpg"
contentId: "<f_kf8e3wy40>"
cid: "f_kf8e3wy40"
headers: object
checksum: "b3e283a5fbc82cfd0d494eec42a1a926"
size: 18972

7

I just did a test with a template node at the exit of the email node.
In the template node I only entered "{{filename"}}. The debug node at the exit of this template node gives:
09/18/2020 at 7:07:11 pmnode: filenametemplate
test image: msg.payload: string [0]
""

This answer ("") is undoubtedly at the origin of "Undefined". But then, that would mean that msg.filename cannot be used?

[{"id":"c5d0246d.ce12a8","type":"inject","z":"8424aab0.5f9b18","name":"","topic":"","payload":"true","payloadType":"bool","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":210,"y":220,"wires":[["6fb1065d.1b9f98"]]},{"id":"6fb1065d.1b9f98","type":"e-mail in","z":"8424aab0.5f9b18","name":"email","protocol":"IMAP","server":"imap.gmail.com","useSSL":true,"port":"993","box":"","disposition":"Read","criteria":"UNSEEN","repeat":"300","fetch":"trigger","inputs":1,"x":388,"y":220,"wires":[["9745087a.b1b9b8","adce327a.d7d9f","6dc3999e.7c2768","e5a8d04a.aa171","c5d24933.68e078","e1df7245.17b84","56e9bdfa.b82714","26276ca2.32e274"]]},{"id":"26276ca2.32e274","type":"template","z":"8424aab0.5f9b18","name":"","field":"payload","fieldType":"msg","format":"handlebars","syntax":"mustache","template":"{{filename}}","output":"str","x":520,"y":100,"wires":[["ab71e81f.36a3e8"]]},{"id":"ab71e81f.36a3e8","type":"debug","z":"8424aab0.5f9b18","name":"filenametemplate","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","x":720,"y":100,"wires":[]}]
8

Hi ..

If according to your split node debug output your file name is in msg.payload.filename then
using a function node after the split node with :

msg.filename = `/home/pi/.local/share/kiosque/ref/photos/${msg.payload.filename}`
return msg;

it should in theory give you the path you need .. but i havent really tested it cause i dont have the mail node setup.

9

So in your debug, the name of the file isn't in msg.filename, it is in msg.payload.filename so you have to use that instead so the replace woutl be

Screen Shot 2020-09-18 at 7.30.34 PM
Screen Shot 2020-09-18 at 7.30.34 PM678×520 28.9 KB

10

Thank you both very much. Thanks to your help I now know how to download files to my pi which were sent to my mailbox! However, there is still a very big problem: when I try to open the file I get the following error message: "Error interpreting the JPEG image file (Not a JPEG file: starts with 0x7b 0x22)" . What have I (still) done wrong?

[{"id":"335234f9.38f4bc","type":"change","z":"8424aab0.5f9b18","name":"","rules":[{"t":"set","p":"payload","pt":"msg","to":"attachments","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":1860,"y":940,"wires":[["7cafe500.611fdc","d9e34674.a87788"]]},{"id":"7cafe500.611fdc","type":"split","z":"8424aab0.5f9b18","name":"","splt":"\\n","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":2030,"y":940,"wires":[["966674ab.4fc608","3614a107.a8aede"]]},{"id":"f9545fe9.66c2a","type":"inject","z":"8424aab0.5f9b18","name":"trigger email fetch","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":true,"onceDelay":0.1,"x":1710,"y":880,"wires":[["1af64526.b8dabb"]]},{"id":"1af64526.b8dabb","type":"e-mail in","z":"8424aab0.5f9b18","name":"Fetch Emails","protocol":"IMAP","server":"imap.gmail.com","useSSL":true,"port":"993","box":"","disposition":"None","criteria":"UNSEEN","repeat":"300","fetch":"trigger","inputs":1,"x":1690,"y":940,"wires":[["335234f9.38f4bc"]]},{"id":"d9e34674.a87788","type":"debug","z":"8424aab0.5f9b18","name":"set","active":true,"tosidebar":true,"console":false,"tostatus":true,"complete":"payload","targetType":"msg","x":1810,"y":1000,"wires":[]},{"id":"966674ab.4fc608","type":"debug","z":"8424aab0.5f9b18","name":"split","active":true,"tosidebar":true,"console":false,"tostatus":true,"complete":"payload","targetType":"msg","x":2010,"y":1000,"wires":[]},{"id":"3614a107.a8aede","type":"function","z":"8424aab0.5f9b18","name":"","func":"msg.filename = `/home/pi/.local/share/kiosque/ref/photos/${msg.payload.filename}`\nreturn msg;\n","outputs":1,"noerr":0,"x":2190,"y":940,"wires":[["67225ee4.ef15","33447ec5.b20ec2"]]},{"id":"67225ee4.ef15","type":"file","z":"8424aab0.5f9b18","name":"Save picture to disk","filename":"","appendNewline":false,"createDir":false,"overwriteFile":"true","encoding":"none","x":2410,"y":940,"wires":[[]]},{"id":"33447ec5.b20ec2","type":"debug","z":"8424aab0.5f9b18","name":"function","active":true,"tosidebar":true,"console":false,"tostatus":true,"complete":"payload","targetType":"msg","x":2220,"y":1000,"wires":[]}]

Laurent

11

you have the answer in your very first post .. that apparently was working :wink:

image

compare it with what you have now

12

0x7b 0x22 is a {" in the ascii table - looks like you are writing JSON

Out of interest, open it with a text editor.

EDIT...

on closer look, you create a file name like this...

msg.filename = `/home/pi/.local/share/kiosque/ref/photos/${msg.payload.filename}`

which suggests msg.payload is an object.

you do realise the file node writes whatever is in msg.payload to the file right?

So, inspect the msg, find the image data, and then move it to msg.payload before the file node. Something like adding the line msg.payload = msg.payload.attachment; just before returning the msg in your function (I dont know the real property name - have a look in the debug output for the debug labelled "function")

13

I have tried putting in the function node the expression "msg.payload = msg.payload.attachment" before "msg.filename =/home/pi/.local/share/kiosque/ref/photos / $ { msg.payload.filename}" then after. I then redid the same double test with "attachments" instead of "attachment". No success in either case.
I think I have to go back to Zenofund's solution which I also tested without success despite the last fix (replacement of msg.filename by msg.payload.filename).
I must say that I am a little lost ...
Laurent

14

This is from email node help file

Any attachments supplied in the incoming email can be found in the msg.attachments property. This will be an array of objects where each object represents a specific attachments
content: // The actual content of the data contained in a Node.js Buffer object
// We can turn this into a base64 data string with content.toString('base64')

Did you have it working at the beginning ?

[EDIT]

After investigating a little bit more and according to the mail node help file
the actual attachment buffer data are in msg.attachments[x].content

so a slight modification to your function node should give you the result you want :

msg.filename = `/home/pi/.local/share/kiosque/ref/photos/${msg.payload.filename}`
msg.payload = msg.payload.content
return msg;

please test with multiple attachments also to see if it works for those as well

15

It actually works!
Thank you so much !
I took the opportunity to add the shipping date at the start of the file name, which gives:
msg.filename = /home/pi/.local/share/kiosque/ref/photos/ $ {msg.date} $ {msg.payload.filename}
msg.payload = msg.payload.content
return msg;

To complete this project, I just have to find out how to change the format of this date:
What I'm getting now: Sat Sep 19 2020 19:11:32 GMT + 0200 (Central European Daylight Time) 14.jpg
What I would like: 20 09 19 - 14.jpg
(I'm not a programmer at all, so if you have any idea ...)

Thank you again for your help,
Laurent

16

I'm glad :wink:

The only thing that was a concern for me was maybe the rate of the attachment messages going to the file out node. If you put a debug node after the function you'll see that all messages are sent instantly and i don't know how the file out node would handle lets say 10 new emails with 5 large attachments. Maybe someone with more experience can advice on this. but you could always use a delay node to give the chance to file node to complete the writing of the files.

Regarding the date formatting, before your function node you could use @TotallyInformation 's Moment node that is a life saver for manipulating dates.

image

17

I tested with 4 mails containing 0, 1, 2 and 4 files of different format ... it works well. I will therefore now test on larger volumes and if necessary I will insert a delay node. As for the date, I should have thought about it because I have used this node before. thank you for all this info, I'll let you know the result. Thanks again for your help, Laurent

18

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.