Generate Email Draft

[trialerra]

Hi all, newbie here trying to find my way in Node Red.

I see there's no node able to not send an email, but actually simply generate it and save it in Drafts. Seems something quite easy to do once the IMAP connection is sorted out.

Any hint on what could be the best way to achieve it? Developing a new node from scratch when we have already the well estabished email one seems pointless, but I'm not sure what's the alternative.

Any opinion?

[gregorius]

Who's your email provided? Definitely using IMAP - I ask because mine does not support IMAP ... long story.

The NodeRED email node is separate but can easily be imported using the palette manager, hamburger icon and then:

The email node does seem to support IMAP (I've never used it).

[trialerra]

Hi

I have several providers but I need it mostly for business and so they all support IMAP and it's the main protocol I'd target.

I do have and use already the mail node but it doesn't support creation of Drafts, the output module only sends emails, while the creation of a Draft means creating an email and store it in the IMAP folder called usually 'Draft'.

The 'connection' (auth etc.) part is already there with the mail node, but there's nothing allowing to make an email a Draft instead of actually sending it.

It's a pretty important part of automating emails, so I'm wondering if anyone has a suggestion on what would be the best way to proceed for implementing this. Try and modify/extending the existing module? Creating a new one? Requesting the authors of the original node-email to implement it?

[gregorius]

I had a quick search for imap but all nodes seem to focus on sending emails. So perhaps there isn't a "generalistic" IMAP node that support the IMAP protocol as a protocol (if that makes sense).

[gregorius]

Btw what happens if you "send" an email without a TO/CC/BCC email? Just wondering whether that would implicitly create a draft - just thinking out loud.

Also the email node uses nodemailer which might be worth having a look at and checking whether that can store a draft.

[trialerra]

Hi, thanks all good points. I actually explored all of them, without a solution.

The email out module just uses SMTP to send emails, it's not linked to Draft creation: the way to create a Draft is to literally create an email in drafts, typically using APPEND (part of Imap commands).

I explored nodemailer and there's nothing on there about creating Drafts.

Once you go up into more Enterprise level, email drafts become an important part of using emails as IMAP is made to share mailboxes across different teams, and Drafts become a fundamental aspect.

Talking of automating emails, that's even more important, but is a pattern I noticed on other cloud services. For example it's easy to create Drafts on Make.com or Zapier, but there's no ready/easy way on n8n (which is more targeted at 'tech' or 'developers').

I come from backend more than JS, so I was wondering if there was an easy way possibly extending the existing node instead of creating a new one, because a new one would need to re-implement all the connection/auth of IMAP that's quite tricky and already dealt with in the existing node.

[gregorius]

@trialerra

I come from backend more than JS,

Interesting way of putting it because JS is very much also a backend language, in the guise of NodeJS of course. Personally I've been won over (coming from a Ruby background) mainly because it's so much simpler to share code between client (browser) and backend built in NodeJS (coming from a web development using Ruby background - not talking about app development).

@trialerra

if there was an easy way possibly extending the existing node instead

I would suggest my practice of developing Node-RED nodes in Node-RED which, in turn, makes extending existing nodes fairly simple but it's a complex ramp up involving using my collection of NodeDev nodes. Importing existing packages can be easily done and these can be reinstalled into Node-RED from within Node-RED.

[trialerra]

Hi thanks a lot for your suggestion, I checked it out but it seems quite a steep learning from someone not used to node red nodes development at all.

I'm more keen for now to duplicate and modify the existing imap node, inverting its functionality, as that might prove simpler. Once I have a module that has me into the IMAP, should be relatively simple to build in a functionality to accept a msg.payload and topic and then perform the append commands to create an email in the Drafts folder.

I already got your website screencast to understand how to start with nodes, so I'll hopefully have some time to do that soon.

I'm pretty sure it might help a lot other similar cases of using Node Red to automate internal processes of bigger organisations needing a more teams approach to emails than a personal one.

[gregorius]

@trialerra

thanks a lot for your suggestion

No worries, any time

@trialerra

I checked it out but it seems quite a steep learning from someone not used to node red nodes development at all.

Can definitely understand. Took me a good year to get that far, so definitely not something for beginners. It would be nice to make it simpler though. As I keep saying, if I want to extend my Emacs, I don't open vi{m} to write the extension, hence why should I use my textual editor to extend my visual editor!

With Node-RED this is even more on-point since people using Node-RED might never have touched a code editor before nor want to.

That was the inspiration/thinking behind/idea for me to create the NodeDev package.

@trialerra

I already got your website screencast to understand how to start with nodes, so I'll hopefully have some time to do that soon.

DM me if you have questions, happy to help.

@trialerra

I'm pretty sure it might help a lot other similar cases of using Node Red to automate internal processes of bigger organisations

Totally agree and would also be interested in doing the work but I don't have a use-case for it! In addition, other things take priority at the moment.

[TotallyInformation]

Hi, I've often found similar issues to you in the past. Few developers bother to actually learn the complexities of IMAP so you end up with greatly simplified options.

Recently, I had a need to auto-process some inbound emails from a mailing list and because I also wanted to mark the email as read once processed AND move it out of the inbox to another folder, I needed to use something a bit more comprehensive.

I ended up adding these node.js packages to Node-RED via the function node:

IMAPFLOW does all the IMAP stuff and mailparser is needed to read and make sense of an email, cheerio extracts data from the HTML body of an email. Sanitize-html is for safety. url is a node.js native library.

You would likely need imapflow and something to create a well formed email from data.

Here is the code - obviously not what you are needing but gives you an idea about the possible complexities. Much of it deals with the HTML content which you probably don't need.

const outdoorActive = global.get('outdoorActive', 'file') ?? {}

// https://www.npmjs.com/package/imapflow
// https://imapflow.com/module-imapflow-ImapFlow.html
const { ImapFlow } = imapflow

// https://nodemailer.com/extras/mailparser/
const { simpleParser } = mailparser

// https://www.npmjs.com/package/sanitize-html
const sanitizeOpts = {
    allowedTags: [
        "address", "article", "aside", "footer", "header", "h1", "h2", "h3", "h4",
        "h5", "h6", "hgroup", "main", "nav", "section", "blockquote", "dd", "div",
        "dl", "dt", "figcaption", "figure", "hr", "li", "main", "ol", "p", "pre",
        "ul", "a", "abbr", "b", "bdi", "bdo", "br", "cite", "code", "data", "dfn",
        "em", "i", "kbd", "mark", "q", "rb", "rp", "rt", "rtc", "ruby", "s", "samp",
        "small", "span", "strong", "sub", "sup", "time", "u", "var", "wbr", "caption",
        "col", "colgroup", "table", "tbody", "td", "tfoot", "th", "thead", "tr",
        "img", "html", "head", "body",
        // "link", "meta",
    ],
    allowedAttributes: {
        '*': [
            'id',
            // 'class',
        ],
        a: [ 'href', 'name', 'target' ],
        img: [ 'src', 'srcset', 'alt', 'title', 'width', 'height', 'loading' ],
        // meta: [ '*' ],
        // link: [ 'href', 'type' ],
    },
}

const client = new ImapFlow({
    host: msg.host,
    port: 993,
    secure: true,
    auth: {
        user: msg.user,
        pass: msg.pw,
    },
    logger: false,
    emitLogs: true,
})

const main = async () => {
    // Wait until client connects and authorizes
    try {
        await client.connect()
        node.status({fill:"green",shape:"dot",text:"Connected"})
    } catch (e) {
        node.status({fill:"red",shape:"ring",text:"Failed to connect"})
        msg.payload = e
        node.error(`Could not connect to mail server. ${e.message}`, msg)
        node.send(msg)
        return
    }

    // These are all optional
    // msg.serverInfo = client.serverInfo
    // msg.capabilities = client.capabilities
    // msg.mailboxes = await client.listTree()
    // msg.inboxStatus = await client.status('INBOX', { recent: true, unseen: true, messages: true, })

    // What properties to return for each email
    const fetchOptions = {
        // bodyStructure: true,
        // envelope: true,
        // flags: true,
        // headers: true,
        // headers: [ 'date', 'subject', 'to', 'from', 'X-Spam-Score', 'X-CampaignID', ],
        // labels: true,
        // size: true,
        source: true,
        // threadId: true,
        // uid: true,
        // bodyParts: [
        //     'text',
        // ],
    }
    // What to search for
    const search = {
        // seen: false,
        // flagged: true,
        from: 'for-you@news.outdooractive.com',
    }

    // Select and lock a mailbox. Throws if mailbox does not exist
    let mb = await client.getMailboxLock(msg.mailbox)
    node.status({fill:"blue",shape:"ring",text:"INBOX open"})
    // client.mailbox includes information about currently selected mailbox
    msg.inbox = client.mailbox

    try {
        const srch = client.fetch(search, fetchOptions, { uid: true })
        node.send({topic: 'Messages found', payload: srch})
        const foundIds = []

        // Process all weekly routes emails in the inbox
        for await ( let message of srch ) {
            // Note the id so we can process them
            foundIds.push(message.uid)

            // msg.message = message
            const mail = await simpleParser(message.source)
            // msg.mail = mail
            const received = mail.date

            const $ = cheerio.load(sanitizeHtml(mail.html, sanitizeOpts))
            const entries = $('body > div:nth-child(2) > div:nth-child(1) > table > tbody > tr > td > div:nth-child(5) > table > tbody > tr > td > div').find('div')

            // Process each route in the current email - also updates the outdoorActive var
            entries.each( (i, entry) => {
                const d = $(entry).find('table > tbody > tr > td > table > tbody')

                // Title/Who: body > div:nth-child(1) > table > tbody > tr > td > table > tbody > tr:nth-child(1) > td > table > tbody > tr:nth-child(1) > td > a
                const d1 = $(d).find('tr:nth-child(1) > td > table > tbody > tr:nth-child(1) > td > a')
                let u = new url.URL(d1[0].attribs.href)
                let routeId = u.pathname.split('/').slice(-1)[0]
                const title = {
                    href: `${u.origin}${u.pathname}`,
                    text: d1[0].firstChild.data,
                }
                const d2 = $(d).find('tr:nth-child(1) > td > table > tbody > tr:nth-child(2) > td > a')
                u = new url.URL(d2[0].attribs.href)
                const who = {
                    href: `${u.origin}${u.pathname}`,
                    text: d2[0].firstChild.data,
                }

                // Image: body > div:nth-child(1) > table > tbody > tr > td > table > tbody > tr:nth-child(2) > td > table > tbody > tr > td > a
                const d3 = $(d).find('tr:nth-child(2) > td > table > tbody > tr > td > a > img')
                const image = {
                    src: d3[0].attribs.src,
                    width: d3[0].attribs.width,
                    height: d3[0].attribs.height,
                }

                // Meta: body > div:nth-child(1) > table > tbody > tr > td > table > tbody > tr:nth-child(3) > td > table > tbody > tr:nth-child(2)
                const d4 = $(d).find('tr:nth-child(3) > td > table > tbody > tr:nth-child(2) > td')
                const details = {
                    distance: d4[0].firstChild.data,
                    time: d4[1].firstChild.data,
                    up: d4[2].firstChild.data,
                    down: d4[3].firstChild.data,
                }

                // console.log({i, title, who, image, details, d1, d2, d3, d4})

                outdoorActive[routeId] = {title, who, image, details, received}
                // node.send({payload: entry.html()})
            })
        }

        // Clean up
        if (foundIds.length > 0) {
            // Mark emails as read by setting the \Seen flag
            const r1 = await client.messageFlagsAdd({ uid: foundIds }, ['\\Seen'])
            node.send({topic: 'Seen flag set?', payload: r1})

            // Move emails to the target folder
            const r2 = await client.messageMove({ uid: foundIds }, 'INBOX/2-Info/Sheffield/Walks-Cycling')
            node.send({ topic: 'Messages moved?', payload: r2 })
        }

        // Mark the found emails as read & move to INBOX/2-Info/Sheffield/Walks-Cycling
        // let r1 = await client.messageFlagsSet(srch, ['\Seen'])
        // console.log(r1)
        // let result = await client.messageMove('1:*', 'INBOX/2-Info/Sheffield/Walks-Cycling')
        // console.log('Moved %s messages', result.uidMap.size)
    } catch (e) {
        node.error(`Something went wrong. ${e.message}`)
    } finally {
        // Make sure lock is released, otherwise next `getMailboxLock()` never returns
        mb.release()
        node.status({fill:"grey",shape:"dot",text:"INBOX Lock Released"})
    }

    // log out and close connection
    await client.logout()
    node.status({fill:"grey",shape:"ring",text:"Logged out"})
}

main().catch(err => node.error(err))

global.set('outdoorActive', outdoorActive, 'file')

// return msg

[trialerra]

Wow...thanks for this, makes it all even easier, I could create a custom Function Node like that simply importing 'on the fly' the modules I need and running it all like this.

I found this node and planned to explore it node-red-contrib-mail-actions (node) - Node-RED

The function node seems the next best and fastest solution if that doesn't work...I wasn't aware of this level of integration, very powerful and convenient.

Thanks for sharing your code, much appreciated, seems quite clear and a good starting point...if I find a solution and it works, I'll post it here.