I have been a busy little person with a couple of things I want to get done in NR.
(Remote machine monitoring/health ping and condition for instance)
This got me thinking:
What are the pros and cons of writing a lot of code in a Function Node as opposed to using a bucket load of standard nodes to get something done?
My solution involves about 7 Function nodes and a lot of code and a few "standard" nodes.
I am interested if writing custom code costs more processor time, or is it quicker to use the "standard" nodes.
The function node has a small performance tax as it runs in a pseudo vm environment, in my case I notice it only when sending a lot of data (several times a second)
You can use the node-red-contrib-unsafe function node to avoid this tax.
Lately I've been replacing a lot of function nodes with change nodes and JSONata. What I like about JSONata is there is usually a single way of doing something as opposed to JavaScript meaning it can be easier to reread code after. It always depends on the complexity of your functions...
Colin has done some interesting performance tests in the past. You can find them in the old node-red forum as "Overheads of function node on pi Zero". For some reason, I cannot copy/paste the link here on my smartphone...
Hi Walter,
Every blockly node runs in its own sandbox (similar to the function nodes). This means it has only a little subset of the Node-RED API available. In fact, my server side code is a copy of the function node. Which is again a problem for me, since I have to keep it in sync with every change in the function node. But the blockly node can be installed on older Node-RED version, which means in run a new function node API in an older Node-RED version ...
7 function nodes isn't a lot & the number people use may often depend on how well they know JavaScript. A function node can sometimes greatly simplify a complex flow making it easier to understand - but equally, it can sometimes do the reverse.
When you say "a lot", what are we talking about here? 100's of lines of code? 1000's?
When you get to very high levels of complexity, you need to be thinking about other issues too of course such as memory and CPU utilisation. Also you need to think about supporting all that code. Is it documented? Will you understand it in 6 months or more? Is there any common code that you could refactor into a library of it's own? Has some of the code reached a critical mass where it may no longer make sense in Node-RED and might be better in its own microservice?
In truth, unless your function code is 1000's or 10's of 1000's of lines long, you've probably not really hit any critical complexity as yet. However, once you hit a few 100 lines per node, probably time to start to think ahead about how to best organise it.
I have a very complex flow with many Function nodes and flow tabs with lots of stuff. It is running rock-solid on a Pi3 since long and never restarts or crashes. Memory consumption is stable, CPU utilization as well. Maybe the size of your flow file can give an indication? Mine is 348 kB, has 142 Function nodes (some might be possible to replace with std nodes) and the response is still very fast, triggering a light on/off from the dashboard turns it on/off "immediately"
Haha, well there's all the inputs from various hardware that gets standardised and send to MQTT. Then there are secondary flows that listen for the hardware MQTT topics and break the data out for specific purposes whether informational or controlling stuff. There's a lot of processing for the Telegram conversational and command interfaces, some web interface stuff, some warnings (processing sensor inputs and outputing warnings).
It is somewhat more complex than absolutely required because I decided early on that I wanted processing disaggregated so that it was much easier to swap out hardware. That has proved very useful over time. It has also allowed me to easily add new hardware, sensors, controls and new interfaces.
And then there are some test flows too. I use my live system to test out ideas over time. Though I'm gradually moving over to a new Pi3 (from my live Pi2).
And, of course, as this has happened over several years, there is some history in there that needs tidying which will happen as I migrate.
Because I submitted the pull request to use "function node without vm module" in the node-red-nodegen project, I'm interested in the performance of the default function node. Today, I conducted simple experiments to measure the performance in each approach.
Compared to the default function node, "nodegen function without vm module" and unsafe function can dramatically reduce the execution time. I hope that "nodegen function without vm module" will be one of the general approaches in Node-RED community.
Approaches
No filter
Default function node
Default switch node
nodegen function (original node generated from function node using node-red-nodegen v0.0.2)
nodegen function without vm module (original node which was generated from function node using new node template submitted in the pull request)
unsafe function (node-red-contrib-unsafe-function)
Flow to evaluate
(1) Kick the flow using inject node and then output timestamp to debug node.
(2) Create the array which contains 0 and 1 values (Lengths of the array are 16K, 32K, 64K and 128K)
(3) Split the array using split node
(4) Execute the filter code in each approach
My main takeaway from that is the Switch node needs some serious optimisation work. I'm surprised to see it perform so badly. I wonder if the persistence context work, which adds async behaviour, is causing this overhead.
Sure! I pasted my test flow as follows. I'm interested in the results on other devices. Especially, Raspberry Pi Zero.
As @knolleary mentioned, the current switch node has low performance. The result is different from the results which @Colin investigated in January.
[Test flow]
[{"id":"22b6f824.1d3d28","type":"debug","z":"96cb18ac.349558","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":1150,"y":160,"wires":[]},{"id":"b6391290.93b68","type":"split","z":"96cb18ac.349558","name":"","splt":"\\n","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":510,"y":160,"wires":[["c2795061.c68ec"]]},{"id":"c2795061.c68ec","type":"join","z":"96cb18ac.349558","name":"","mode":"auto","build":"string","property":"payload","propertyType":"msg","key":"topic","joiner":"\\n","joinerType":"str","accumulate":"false","timeout":"","count":"","reduceRight":false,"x":870,"y":160,"wires":[["1f02fec1.10f9a1"]]},{"id":"e589003.1b546","type":"debug","z":"96cb18ac.349558","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":1150,"y":320,"wires":[]},{"id":"ee961a3f.b3c8d8","type":"function","z":"96cb18ac.349558","name":"function","func":"if (0 < msg.payload) {\n return [msg, null];\n} else {\n return [null, msg];\n}","outputs":2,"noerr":0,"x":700,"y":320,"wires":[["c0a23080.d5156"],["c0a23080.d5156"]]},{"id":"f3351b90.325488","type":"split","z":"96cb18ac.349558","name":"","splt":"\\n","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":510,"y":320,"wires":[["ee961a3f.b3c8d8"]]},{"id":"c0a23080.d5156","type":"join","z":"96cb18ac.349558","name":"","mode":"auto","build":"string","property":"payload","propertyType":"msg","key":"topic","joiner":"\\n","joinerType":"str","accumulate":"false","timeout":"","count":"","reduceRight":false,"x":870,"y":320,"wires":[["1c8ec1a3.fd437e"]]},{"id":"9cfc6267.658ff","type":"switch","z":"96cb18ac.349558","name":"","property":"payload","propertyType":"msg","rules":[{"t":"gt","v":"0","vt":"num"},{"t":"else"}],"checkall":"true","repair":false,"outputs":2,"x":690,"y":480,"wires":[["9ef5b0e9.e0794"],["9ef5b0e9.e0794"]]},{"id":"6f7ea158.72daa","type":"debug","z":"96cb18ac.349558","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":1150,"y":480,"wires":[]},{"id":"9306e26b.bc3e3","type":"split","z":"96cb18ac.349558","name":"","splt":"\\n","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":510,"y":480,"wires":[["9cfc6267.658ff"]]},{"id":"9ef5b0e9.e0794","type":"join","z":"96cb18ac.349558","name":"","mode":"auto","build":"string","property":"payload","propertyType":"msg","key":"topic","joiner":"\\n","joinerType":"str","accumulate":"false","timeout":"","count":"","reduceRight":false,"x":870,"y":480,"wires":[["795ab8fb.a76418"]]},{"id":"e0f7fccf.d3d1f","type":"debug","z":"96cb18ac.349558","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":1150,"y":960,"wires":[]},{"id":"324eb40b.8ac1dc","type":"split","z":"96cb18ac.349558","name":"","splt":"\\n","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":510,"y":960,"wires":[["840c08dd.30c768"]]},{"id":"b6e70644.bf8818","type":"join","z":"96cb18ac.349558","name":"","mode":"auto","build":"string","property":"payload","propertyType":"msg","key":"topic","joiner":"\\n","joinerType":"str","accumulate":"false","timeout":"","count":"","reduceRight":false,"x":870,"y":960,"wires":[["8bf2d19f.a3a7e"]]},{"id":"b751a475.ce1018","type":"debug","z":"96cb18ac.349558","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":370,"y":200,"wires":[]},{"id":"1f02fec1.10f9a1","type":"function","z":"96cb18ac.349558","name":"get time","func":"msg.payload = new Date().getTime();\nreturn msg;","outputs":1,"noerr":0,"x":1000,"y":160,"wires":[["22b6f824.1d3d28"]]},{"id":"1c8ec1a3.fd437e","type":"function","z":"96cb18ac.349558","name":"get time","func":"msg.payload = new Date().getTime();\nreturn msg;","outputs":1,"noerr":0,"x":1000,"y":320,"wires":[["e589003.1b546"]]},{"id":"795ab8fb.a76418","type":"function","z":"96cb18ac.349558","name":"get time","func":"msg.payload = new Date().getTime();\nreturn msg;","outputs":1,"noerr":0,"x":1000,"y":480,"wires":[["6f7ea158.72daa"]]},{"id":"8bf2d19f.a3a7e","type":"function","z":"96cb18ac.349558","name":"get time","func":"msg.payload = new Date().getTime();\nreturn msg;","outputs":1,"noerr":0,"x":1000,"y":960,"wires":[["e0f7fccf.d3d1f"]]},{"id":"843c40e8.02945","type":"function","z":"96cb18ac.349558","name":"create array","func":"msg.payload = [];\nfor (var i = 0; i < msg.topic; i++) {\n msg.payload.push(0);\n msg.payload.push(1);\n}\nreturn msg;","outputs":1,"noerr":0,"x":370,"y":160,"wires":[["b6391290.93b68"]]},{"id":"840c08dd.30c768","type":"unsafe-function","z":"96cb18ac.349558","name":"unsafe function","func":"if (0 < msg.payload) {\n return [msg, null];\n} else {\n return [null, msg];\n}","outputs":2,"noerr":0,"x":700,"y":960,"wires":[["b6e70644.bf8818"],["b6e70644.bf8818"]]},{"id":"acd21a99.d22b08","type":"debug","z":"96cb18ac.349558","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":1150,"y":640,"wires":[]},{"id":"fd1f5b96.955378","type":"split","z":"96cb18ac.349558","name":"","splt":"\\n","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":510,"y":640,"wires":[["2739ed02.031772"]]},{"id":"308209d.70953f6","type":"join","z":"96cb18ac.349558","name":"","mode":"auto","build":"string","property":"payload","propertyType":"msg","key":"topic","joiner":"\\n","joinerType":"str","accumulate":"false","timeout":"","count":"","reduceRight":false,"x":870,"y":640,"wires":[["2f576f8f.49ed9"]]},{"id":"2f576f8f.49ed9","type":"function","z":"96cb18ac.349558","name":"get time","func":"msg.payload = new Date().getTime();\nreturn msg;","outputs":1,"noerr":0,"x":1000,"y":640,"wires":[["acd21a99.d22b08"]]},{"id":"7253166b.0a6fb8","type":"debug","z":"96cb18ac.349558","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":1150,"y":800,"wires":[]},{"id":"caf294f8.7b0a58","type":"split","z":"96cb18ac.349558","name":"","splt":"\\n","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":510,"y":800,"wires":[["c7be1c7b.1db83"]]},{"id":"fb197dc6.94184","type":"join","z":"96cb18ac.349558","name":"","mode":"auto","build":"string","property":"payload","propertyType":"msg","key":"topic","joiner":"\\n","joinerType":"str","accumulate":"false","timeout":"","count":"","reduceRight":false,"x":870,"y":800,"wires":[["a0178366.7c2df"]]},{"id":"a0178366.7c2df","type":"function","z":"96cb18ac.349558","name":"get time","func":"msg.payload = new Date().getTime();\nreturn msg;","outputs":1,"noerr":0,"x":1000,"y":800,"wires":[["7253166b.0a6fb8"]]},{"id":"2739ed02.031772","type":"nodegen-function","z":"96cb18ac.349558","name":"","x":690,"y":640,"wires":[["308209d.70953f6"],["308209d.70953f6"]]},{"id":"c7be1c7b.1db83","type":"nodegen-function-wovm","z":"96cb18ac.349558","name":"","x":690,"y":800,"wires":[["fb197dc6.94184"],["fb197dc6.94184"]]},{"id":"a0b93f5.871afc","type":"inject","z":"96cb18ac.349558","name":"","topic":"16384","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":180,"y":120,"wires":[["843c40e8.02945","b751a475.ce1018"]]},{"id":"9d6db1fc.bc468","type":"inject","z":"96cb18ac.349558","name":"","topic":"32768","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":180,"y":160,"wires":[["843c40e8.02945","b751a475.ce1018"]]},{"id":"a26945de.fe00a8","type":"inject","z":"96cb18ac.349558","name":"","topic":"65536","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":180,"y":200,"wires":[["843c40e8.02945","b751a475.ce1018"]]},{"id":"e1d0f2c2.74f4","type":"inject","z":"96cb18ac.349558","name":"","topic":"131072","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":170,"y":240,"wires":[["843c40e8.02945","b751a475.ce1018"]]},{"id":"df436b3f.88da08","type":"debug","z":"96cb18ac.349558","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":370,"y":360,"wires":[]},{"id":"935172d4.12d58","type":"function","z":"96cb18ac.349558","name":"create array","func":"msg.payload = [];\nfor (var i = 0; i < msg.topic; i++) {\n msg.payload.push(0);\n msg.payload.push(1);\n}\nreturn msg;","outputs":1,"noerr":0,"x":370,"y":320,"wires":[["f3351b90.325488"]]},{"id":"ff205763.2fdce8","type":"inject","z":"96cb18ac.349558","name":"","topic":"16384","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":180,"y":280,"wires":[["935172d4.12d58","df436b3f.88da08"]]},{"id":"cbcd61bd.1f60e","type":"inject","z":"96cb18ac.349558","name":"","topic":"32768","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":180,"y":320,"wires":[["935172d4.12d58","df436b3f.88da08"]]},{"id":"af555047.2279","type":"inject","z":"96cb18ac.349558","name":"","topic":"65536","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":180,"y":360,"wires":[["935172d4.12d58","df436b3f.88da08"]]},{"id":"dc506e50.0dada","type":"inject","z":"96cb18ac.349558","name":"","topic":"131072","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":170,"y":400,"wires":[["935172d4.12d58","df436b3f.88da08"]]},{"id":"5d5f9d9e.58cec4","type":"debug","z":"96cb18ac.349558","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":370,"y":520,"wires":[]},{"id":"e717f529.a293f8","type":"function","z":"96cb18ac.349558","name":"create array","func":"msg.payload = [];\nfor (var i = 0; i < msg.topic; i++) {\n msg.payload.push(0);\n msg.payload.push(1);\n}\nreturn msg;","outputs":1,"noerr":0,"x":370,"y":480,"wires":[["9306e26b.bc3e3"]]},{"id":"27fc8cb8.b3af14","type":"inject","z":"96cb18ac.349558","name":"","topic":"16384","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":180,"y":440,"wires":[["e717f529.a293f8","5d5f9d9e.58cec4"]]},{"id":"17c5c232.a506fe","type":"inject","z":"96cb18ac.349558","name":"","topic":"32768","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":180,"y":480,"wires":[["e717f529.a293f8","5d5f9d9e.58cec4"]]},{"id":"564ac31b.dc2fcc","type":"inject","z":"96cb18ac.349558","name":"","topic":"65536","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":180,"y":520,"wires":[["e717f529.a293f8","5d5f9d9e.58cec4"]]},{"id":"53e70f40.30328","type":"inject","z":"96cb18ac.349558","name":"","topic":"131072","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":170,"y":560,"wires":[["e717f529.a293f8","5d5f9d9e.58cec4"]]},{"id":"64aaf49c.ddb4dc","type":"debug","z":"96cb18ac.349558","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":370,"y":680,"wires":[]},{"id":"3b44c559.57b6ba","type":"function","z":"96cb18ac.349558","name":"create array","func":"msg.payload = [];\nfor (var i = 0; i < msg.topic; i++) {\n msg.payload.push(0);\n msg.payload.push(1);\n}\nreturn msg;","outputs":1,"noerr":0,"x":370,"y":640,"wires":[["fd1f5b96.955378"]]},{"id":"70dfc3ca.c841ec","type":"inject","z":"96cb18ac.349558","name":"","topic":"16384","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":180,"y":600,"wires":[["3b44c559.57b6ba","64aaf49c.ddb4dc"]]},{"id":"c16f793c.9b4058","type":"inject","z":"96cb18ac.349558","name":"","topic":"32768","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":180,"y":640,"wires":[["3b44c559.57b6ba","64aaf49c.ddb4dc"]]},{"id":"93705687.4237f8","type":"inject","z":"96cb18ac.349558","name":"","topic":"65536","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":180,"y":680,"wires":[["3b44c559.57b6ba","64aaf49c.ddb4dc"]]},{"id":"5354db08.e19524","type":"inject","z":"96cb18ac.349558","name":"","topic":"131072","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":170,"y":720,"wires":[["3b44c559.57b6ba","64aaf49c.ddb4dc"]]},{"id":"14e2fcad.603a13","type":"debug","z":"96cb18ac.349558","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":370,"y":840,"wires":[]},{"id":"eee3a9ea.1f4d98","type":"function","z":"96cb18ac.349558","name":"create array","func":"msg.payload = [];\nfor (var i = 0; i < msg.topic; i++) {\n msg.payload.push(0);\n msg.payload.push(1);\n}\nreturn msg;","outputs":1,"noerr":0,"x":370,"y":800,"wires":[["caf294f8.7b0a58"]]},{"id":"8b523560.689b68","type":"inject","z":"96cb18ac.349558","name":"","topic":"16384","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":180,"y":760,"wires":[["eee3a9ea.1f4d98","14e2fcad.603a13"]]},{"id":"ade36a1f.feaa68","type":"inject","z":"96cb18ac.349558","name":"","topic":"32768","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":180,"y":800,"wires":[["eee3a9ea.1f4d98","14e2fcad.603a13"]]},{"id":"7f7814c7.96c62c","type":"inject","z":"96cb18ac.349558","name":"","topic":"65536","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":180,"y":840,"wires":[["eee3a9ea.1f4d98","14e2fcad.603a13"]]},{"id":"7a1522ac.4db17c","type":"inject","z":"96cb18ac.349558","name":"","topic":"131072","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":170,"y":880,"wires":[["eee3a9ea.1f4d98","14e2fcad.603a13"]]},{"id":"23cd67c5.5ac488","type":"debug","z":"96cb18ac.349558","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":370,"y":1000,"wires":[]},{"id":"fa3e34e0.336788","type":"function","z":"96cb18ac.349558","name":"create array","func":"msg.payload = [];\nfor (var i = 0; i < msg.topic; i++) {\n msg.payload.push(0);\n msg.payload.push(1);\n}\nreturn msg;","outputs":1,"noerr":0,"x":370,"y":960,"wires":[["324eb40b.8ac1dc"]]},{"id":"e6d8a9e5.b6e088","type":"inject","z":"96cb18ac.349558","name":"","topic":"16384","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":180,"y":920,"wires":[["fa3e34e0.336788","23cd67c5.5ac488"]]},{"id":"7749a10b.35a49","type":"inject","z":"96cb18ac.349558","name":"","topic":"32768","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":180,"y":960,"wires":[["fa3e34e0.336788","23cd67c5.5ac488"]]},{"id":"ac7649b0.5dc7d8","type":"inject","z":"96cb18ac.349558","name":"","topic":"65536","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":180,"y":1000,"wires":[["fa3e34e0.336788","23cd67c5.5ac488"]]},{"id":"1691b9c4.38e086","type":"inject","z":"96cb18ac.349558","name":"","topic":"131072","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":170,"y":1040,"wires":[["fa3e34e0.336788","23cd67c5.5ac488"]]},{"id":"a95d4b79.b5e398","type":"comment","z":"96cb18ac.349558","name":"No filter","info":"","x":690,"y":140,"wires":[]},{"id":"b1c00742.17d608","type":"comment","z":"96cb18ac.349558","name":"Filter using default function","info":"","x":710,"y":280,"wires":[]},{"id":"97874710.ff8d88","type":"comment","z":"96cb18ac.349558","name":"Filter using nodegen function","info":"","x":700,"y":600,"wires":[]},{"id":"ce9093d2.6435a","type":"comment","z":"96cb18ac.349558","name":"Filter using nodegen function without vm","info":"","x":700,"y":760,"wires":[]},{"id":"48f00ef5.70c19","type":"comment","z":"96cb18ac.349558","name":"Filter using unsafe function","info":"","x":690,"y":920,"wires":[]},{"id":"590a90b.45ff97","type":"comment","z":"96cb18ac.349558","name":"Filter using default switch","info":"","x":690,"y":440,"wires":[]}]
[Screenshot of test flow] *Red rectangle means the filters.
