I am using the Email Node to receive emails via IMAP.
It works great but notice's it keeps crashing on a few emails that have large attachments 7-15mb.
Does anyone have any ideas to work around this? Thanks.
Welcome to the forum @nred
Please stop node red and restart it in a terminal and copy/paste the startup log here. Then make it crash and post that log here too.
Thanks @Colin. I believe this is the crash log:
2021-04-18T04:40:16.788503+00:00 app[web.1]: <--- Last few GCs --->
2021-04-18T04:40:16.788504+00:00 app[web.1]:
2021-04-18T04:40:16.788509+00:00 app[web.1]: [4:0x533a170] 552165 ms: Scavenge (reduce) 272.8 (289.7) -> 272.7 (290.9) MB, 6.2 / 0.0 ms (average mu = 0.988, current mu = 0.955) allocation failure
2021-04-18T04:40:16.788509+00:00 app[web.1]: [4:0x533a170] 552222 ms: Scavenge 273.9 (285.2) -> 273.8 (288.4) MB, 13.4 / 0.0 ms (average mu = 0.988, current mu = 0.955) allocation failure
2021-04-18T04:40:16.788510+00:00 app[web.1]: [4:0x533a170] 552250 ms: Scavenge 274.8 (291.9) -> 274.8 (293.2) MB, 7.5 / 0.0 ms (average mu = 0.988, current mu = 0.955) allocation failure
2021-04-18T04:40:16.788510+00:00 app[web.1]:
2021-04-18T04:40:16.788511+00:00 app[web.1]:
2021-04-18T04:40:16.788511+00:00 app[web.1]: <--- JS stacktrace --->
2021-04-18T04:40:16.788511+00:00 app[web.1]:
2021-04-18T04:40:16.788520+00:00 app[web.1]: FATAL ERROR: MarkCompactCollector: young object promotion failed Allocation failed - JavaScript heap out of memory
2021-04-18T04:40:16.789415+00:00 app[web.1]: 1: 0xa04200 node::Abort() [node-red]
2021-04-18T04:40:16.790041+00:00 app[web.1]: 2: 0x94e4e9 node::FatalError(char const*, char const*) [node-red]
2021-04-18T04:40:16.790660+00:00 app[web.1]: 3: 0xb7978e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node-red]
2021-04-18T04:40:16.791301+00:00 app[web.1]: 4: 0xb79b07 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node-red]
2021-04-18T04:40:16.792025+00:00 app[web.1]: 5: 0xd34395 [node-red]
2021-04-18T04:40:16.792768+00:00 app[web.1]: 6: 0xd64f2e v8::internal::EvacuateNewSpaceVisitor::Visit(v8::internal::HeapObject, int) [node-red]
2021-04-18T04:40:16.793526+00:00 app[web.1]: 7: 0xd70f66 v8::internal::FullEvacuator::RawEvacuatePage(v8::internal::MemoryChunk*, long*) [node-red]
2021-04-18T04:40:16.794279+00:00 app[web.1]: 8: 0xd5d14f v8::internal::Evacuator::EvacuatePage(v8::internal::MemoryChunk*) [node-red]
2021-04-18T04:40:16.795042+00:00 app[web.1]: 9: 0xd5d3c8 v8::internal::PageEvacuationTask::RunInParallel(v8::internal::ItemParallelJob::Task::Runner) [node-red]
2021-04-18T04:40:16.795810+00:00 app[web.1]: 10: 0xd4fca9 v8::internal::ItemParallelJob::Run() [node-red]
2021-04-18T04:40:16.796542+00:00 app[web.1]: 11: 0xd72ec0 void v8::internal::MarkCompactCollectorBase::CreateAndExecuteEvacuationTasks<v8::internal::FullEvacuator, v8::internal::MarkCompactCollector>(v8::internal::MarkCompactCollector*, v8::internal::ItemParallelJob*, v8::internal::MigrationObserver*, long) [node-red]
2021-04-18T04:40:16.797266+00:00 app[web.1]: 12: 0xd7375c v8::internal::MarkCompactCollector::EvacuatePagesInParallel() [node-red]
2021-04-18T04:40:16.798005+00:00 app[web.1]: 13: 0xd73925 v8::internal::MarkCompactCollector::Evacuate() [node-red]
2021-04-18T04:40:16.798740+00:00 app[web.1]: 14: 0xd85911 v8::internal::MarkCompactCollector::CollectGarbage() [node-red]
2021-04-18T04:40:16.799462+00:00 app[web.1]: 15: 0xd41c38 v8::internal::Heap::MarkCompact() [node-red]
2021-04-18T04:40:16.800251+00:00 app[web.1]: 16: 0xd43728 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node-red]
2021-04-18T04:40:16.801012+00:00 app[web.1]: 17: 0xd46754 v8::internal::Heap::AllocateExternalBackingStore(std::function<void* (unsigned long)> const&, unsigned long) [node-red]
2021-04-18T04:40:16.801805+00:00 app[web.1]: 18: 0xe5a5a2 v8::internal::BackingStore::Allocate(v8::internal::Isolate*, unsigned long, v8::internal::SharedFlag, v8::internal::InitializedFlag) [node-red]
2021-04-18T04:40:16.802482+00:00 app[web.1]: 19: 0xb96290 v8::ArrayBuffer::NewBackingStore(v8::Isolate*, unsigned long) [node-red]
2021-04-18T04:40:16.803133+00:00 app[web.1]: 20: 0xad2e33 node::EmitToJSStreamListener::OnStreamAlloc(unsigned long) [node-red]
2021-04-18T04:40:16.803789+00:00 app[web.1]: 21: 0xadb5dc [node-red]
2021-04-18T04:40:16.804665+00:00 app[web.1]: 22: 0x138708a [node-red]
2021-04-18T04:40:16.805538+00:00 app[web.1]: 23: 0x1387bf0 [node-red]
2021-04-18T04:40:16.806397+00:00 app[web.1]: 24: 0x138e5e5 [node-red]
2021-04-18T04:40:16.807245+00:00 app[web.1]: 25: 0x137c438 uv_run [node-red]
2021-04-18T04:40:16.807854+00:00 app[web.1]: 26: 0xa44974 node::NodeMainInstance::Run() [node-red]
2021-04-18T04:40:16.808463+00:00 app[web.1]: 27: 0x9d1e15 node::Start(int, char**) [node-red]
and this is the startup log:
2021-04-18T04:54:11.667449+00:00 app[web.1]: Welcome to Node-RED
2021-04-18T04:54:11.667449+00:00 app[web.1]: ===================
2021-04-18T04:54:11.667450+00:00 app[web.1]:
2021-04-18T04:54:11.670810+00:00 app[web.1]: 18 Apr 04:54:11 - [info] Node-RED version: v1.3.2
2021-04-18T04:54:11.672729+00:00 app[web.1]: 18 Apr 04:54:11 - [info] Node.js version: v14.16.1
2021-04-18T04:54:11.672730+00:00 app[web.1]: 18 Apr 04:54:11 - [info] Linux 4.4.0-1088-aws x64 LE
2021-04-18T04:54:12.042506+00:00 app[web.1]: 18 Apr 04:54:12 - [info] Loading palette nodes
2021-04-18T04:54:12.862662+00:00 app[web.1]: 18 Apr 04:54:12 - [warn] Use of autoInstallModules is deprecated. Use externalModules.autoInstall instead
2021-04-18T04:54:12.864609+00:00 app[web.1]: 18 Apr 04:54:12 - [warn] Missing node modules:
2021-04-18T04:54:12.864932+00:00 app[web.1]: 18 Apr 04:54:12 - [warn] - node-red-node-email (1.11.0): e-mail, e-mail in, e-mail mta
2021-04-18T04:54:12.865121+00:00 app[web.1]: 18 Apr 04:54:12 - [warn] - node-red-contrib-send-multipart-formdata (1.1.0): http-send-multipart-formdata
2021-04-18T04:54:12.865312+00:00 app[web.1]: 18 Apr 04:54:12 - [warn] - node-red-node-dropbox (1.0.2): dropbox-config, dropbox in, dropbox, dropbox out
2021-04-18T04:54:12.865493+00:00 app[web.1]: 18 Apr 04:54:12 - [warn] - node-red-node-base64 (0.3.0): base64
2021-04-18T04:54:12.866366+00:00 app[web.1]: 18 Apr 04:54:12 - [info] Settings file : /app/settings.js
2021-04-18T04:54:12.866645+00:00 app[web.1]: 18 Apr 04:54:12 - [info] HTTP Static : /app/public
2021-04-18T04:54:12.867497+00:00 app[web.1]: 18 Apr 04:54:12 - [info] Context store : 'default' [module=memory]
2021-04-18T04:54:12.868456+00:00 app[web.1]: 18 Apr 04:54:12 - [info] Installing module: node-red-node-email, version: 1.11.0
2021-04-18T04:54:12.877519+00:00 app[web.1]: getFlows
2021-04-18T04:54:12.906631+00:00 app[web.1]: getCredentials
2021-04-18T04:54:12.951542+00:00 app[web.1]: 18 Apr 04:54:12 - [info] Waiting for missing types to be registered:
2021-04-18T04:54:12.951804+00:00 app[web.1]: 18 Apr 04:54:12 - [info] - dropbox-config
2021-04-18T04:54:12.952121+00:00 app[web.1]: 18 Apr 04:54:12 - [info] - e-mail in (provided by npm module node-red-node-email)
2021-04-18T04:54:12.952225+00:00 app[web.1]: 18 Apr 04:54:12 - [info] - dropbox out
2021-04-18T04:54:12.952404+00:00 app[web.1]: 18 Apr 04:54:12 - [info] To install any of these missing modules, run:
2021-04-18T04:54:12.952497+00:00 app[web.1]: 18 Apr 04:54:12 - [info] npm install <module name>
2021-04-18T04:54:12.952651+00:00 app[web.1]: 18 Apr 04:54:12 - [info] in the directory:
2021-04-18T04:54:12.952761+00:00 app[web.1]: 18 Apr 04:54:12 - [info] undefined
2021-04-18T04:54:12.959794+00:00 app[web.1]: 18 Apr 04:54:12 - [info] Server now running at http:///red/
2021-04-18T04:54:13.686915+00:00 heroku[web.1]: State changed from starting to up
2021-04-18T04:54:20.761222+00:00 app[web.1]: 18 Apr 04:54:20 - [info] Installed module: node-red-node-email
2021-04-18T04:54:21.001704+00:00 app[web.1]: 18 Apr 04:54:21 - [info] Missing type registered: e-mail in
2021-04-18T04:54:21.002911+00:00 app[web.1]: 18 Apr 04:54:21 - [info] Added node types:
2021-04-18T04:54:21.002986+00:00 app[web.1]: 18 Apr 04:54:21 - [info] - node-red-node-email:e-mail
2021-04-18T04:54:21.003065+00:00 app[web.1]: 18 Apr 04:54:21 - [info] - node-red-node-email:e-mail in
2021-04-18T04:54:21.003140+00:00 app[web.1]: 18 Apr 04:54:21 - [info] - node-red-node-email:e-mail mta
2021-04-18T04:54:21.003434+00:00 app[web.1]: 18 Apr 04:54:21 - [info] Installing module: node-red-contrib-send-multipart-formdata, version: 1.1.0
2021-04-18T04:54:26.324622+00:00 app[web.1]: 18 Apr 04:54:26 - [info] Installed module: node-red-contrib-send-multipart-formdata
2021-04-18T04:54:26.346656+00:00 app[web.1]: 18 Apr 04:54:26 - [info] Added node types:
2021-04-18T04:54:26.346854+00:00 app[web.1]: 18 Apr 04:54:26 - [info] - node-red-contrib-send-multipart-formdata:http-send-multipart-formdata
2021-04-18T04:54:26.347143+00:00 app[web.1]: 18 Apr 04:54:26 - [info] Installing module: node-red-node-dropbox, version: 1.0.2
2021-04-18T04:54:34.530651+00:00 app[web.1]: 18 Apr 04:54:34 - [info] Installed module: node-red-node-dropbox
2021-04-18T04:54:34.544252+00:00 app[web.1]: 18 Apr 04:54:34 - [info] Missing type registered: dropbox-config
2021-04-18T04:54:34.544519+00:00 app[web.1]: 18 Apr 04:54:34 - [info] Missing type registered: dropbox out
2021-04-18T04:54:34.548581+00:00 app[web.1]: 18 Apr 04:54:34 - [info] Added node types:
2021-04-18T04:54:34.548656+00:00 app[web.1]: 18 Apr 04:54:34 - [info] - node-red-node-dropbox:dropbox-config
2021-04-18T04:54:34.548775+00:00 app[web.1]: 18 Apr 04:54:34 - [info] - node-red-node-dropbox:dropbox in
2021-04-18T04:54:34.548838+00:00 app[web.1]: 18 Apr 04:54:34 - [info] - node-red-node-dropbox:dropbox
2021-04-18T04:54:34.548881+00:00 app[web.1]: 18 Apr 04:54:34 - [info] - node-red-node-dropbox:dropbox out
2021-04-18T04:54:34.549098+00:00 app[web.1]: 18 Apr 04:54:34 - [info] Installing module: node-red-node-base64, version: 0.3.0
2021-04-18T04:54:34.555114+00:00 app[web.1]: 18 Apr 04:54:34 - [info] Starting flows
2021-04-18T04:54:34.588827+00:00 app[web.1]: 18 Apr 04:54:34 - [info] Started flows
2021-04-18T04:54:40.053789+00:00 app[web.1]: 18 Apr 04:54:40 - [info] Installed module: node-red-node-base64
2021-04-18T04:54:40.063987+00:00 app[web.1]: 18 Apr 04:54:40 - [info] Added node types:
2021-04-18T04:54:40.064059+00:00 app[web.1]: 18 Apr 04:54:40 - [info] - node-red-node-base64:base64
Based on the crash log showing out of memory I managed to adjust my flow to not use the attachment if it is over 5mb and it seems to work. It's not ideal but better then crashing. If you have any other ideas, feel free to mention them. Thanks.
Frustrating - At the same till sending emails with 5 mb attachments is not great either,
A simple node that would upload to a dropbox/google drive/sharepoint etc and retrieve secure link would be ideal , but also to my knowledge does not exist.
You can hand craft if but bit of work.
You didn't mention that you are running on Heroku. I don't know how much stack memory you get allocated there, but you are running out, as you noticed. I also notice that it seems to be using some sort of auto-install of additional modules and that it isn't installing the latest version of the email node, so probably worth sorting that, though I don't expect that to be the issue.
The fact that it doesn't crash if you don't use the attachment means that it isn't receiving the attachment that is the problem, but whatever you are doing with it after that, so possibly nothing to do with the email node at all. You could try breaking the flow further down the line to find where it is crashing.
Thanks @Colin and @chrisn-au.
I'll mark it as solved as I now agree that receiving the attachment doesn't seem to be the problem and likely it is because the attachments are carried through too many flows.
I should be able to debug further myself so thanks for your help.
1 Like
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.
