A NodeRed Challenge: Avoid the logging is filled with node results

[WJ4IoT]

I hope someone is out there who has more knowledge then me (to be honest there is a 1000% change on that).

What is the challenge?
I want to utilize from the Node-Red plugin @meshtastic/node-red-contrib-meshtastic the DECODE node. I got it working but it either fills all disc space (170Gb free) in 3 days to zero bytes available or if I limit the size of the logging file most likely burns a hole in my drive. Why?

Regardless of the logging console level (including "off") in settings,js it floods the log files with DECODEd data which you normally only expect to see if you activate the DEBUG node. I think I can not control this output because it does not contain a logging console level.

So, easy question for you: How can this be avoided? Is there a simple setting for this (maybe one to be omitted) ? Thanks for your attention and perhaps solution in advance.

For testing you need a MQTT bridge to the Meshtastic MQTT server, but I hope the problem itself is generic enough to tackle. Be aware if you are using already decoding of protobufs that there is node naming overlap with the comparable plugin node-red-contrib-protobuf (node) - Node-RED but due to (probably) out of date meshtastic protobufs I do not get this one to work, also have no other protobuf examples to verify if it has the same (unwanted) behavior.

System details:

• Ubuntu 24.04.1 LTS (GNU/Linux 6.8.0-51-generic x86_64)
• Nodered: v4.0.8 (Docker)
• @meshtastic/node-red-contrib-meshtastic: 2.2.18-1

Example of the first decode record which is pushed into the log file:

nodered  | Serializing ServiceEnvelope to JSON for output
nodered  | Decoder was not null and not a TextDecoder. Assuming Protobuf decoder and decoding payload
nodered  | Decoded payload to JSON:
nodered  | {
nodered  |   "packet": {
nodered  |     "from": 573288178,
nodered  |     "to": 4294967295,
nodered  |     "channel": 0,
nodered  |     "decoded": {
nodered  |       "portnum": 3,
nodered  |       "payload": {
nodered  |         "latitudeI": 0,
nodered  |         "longitudeI": 0,
nodered  |         "altitude": 0,
nodered  |         "time": 1734684283,
nodered  |         "locationSource": 0,
nodered  |         "altitudeSource": 0,
nodered  |         "timestamp": 0,
nodered  |         "timestampMillisAdjust": 0,
nodered  |         "altitudeHae": 0,
nodered  |         "altitudeGeoidalSeparation": 0,
nodered  |         "PDOP": 0,
nodered  |         "HDOP": 0,
nodered  |         "VDOP": 0,
nodered  |         "gpsAccuracy": 0,
nodered  |         "groundSpeed": 0,
nodered  |         "groundTrack": 0,
nodered  |         "fixQuality": 0,
nodered  |         "fixType": 0,
nodered  |         "satsInView": 0,
nodered  |         "sensorId": 0,
nodered  |         "nextUpdate": 0,
nodered  |         "seqNumber": 0
nodered  |       },
nodered  |       "wantResponse": false,
nodered  |       "dest": 0,
nodered  |       "source": 0,
nodered  |       "requestId": 0,
nodered  |       "replyId": 0,
nodered  |       "emoji": 0
nodered  |     },
nodered  |     "id": 672638178,
nodered  |     "rxTime": 1734684306,
nodered  |     "rxSnr": -17.25,
nodered  |     "hopLimit": 2,
nodered  |     "wantAck": false,
nodered  |     "priority": 0,
nodered  |     "rxRssi": -128,
nodered  |     "delayed": 0
nodered  |   },
nodered  |   "channelId": "LongFast",
nodered  |   "gatewayId": "!7b076002"
nodered  | }

[ThingsTinkerer]

What is writing to file exactly? Is it node red std out? Or is that individual package writing its own logs internally to file, regardless of node red?

[Steve-Mcl]

To be fair, that node does so little (and not very well either) you could just copy it to a function node.

do you have a sample buffer I could use to test this theory?

Add a debug BEFORE this node & capture the output using the Copy Value button to ensure data is correct format.

---

How to grab data from a debug node

There’s a great page in the docs (Working with messages : Node-RED) that will explain how to use the debug panel to find the right path/value for any data item.

Pay particular attention to the part about the buttons that appear under your mouse pointer when you over hover a debug message property in the sidebar.

[Steve-Mcl]

@Steve-Mcl

To be fair, that node does so little

Unfortunately, it is written in typescript and has some dependencies that are only in JSR (deno registry) so its not straight forwards.

I see you added a comment to the repository asking for console.debugs to be reduced/removed already - you could wait to see it the author responds?

NOTE: The reason they cant be silenced by node-red logging is because the author used console instead of red.log (see JavaScript file : Node-RED)

Lastly, the node emits a new msg object every time (e.g. node.send({payload:result}) instead of being a good boy and returning the original message (e.g. msg.payload=result; node.send(msg)) - effectively throwing away important props you may need down stream!

If the author does not respond you might consider forking the node and fixing it up yourself. Essentially, remove (or change) all 9 of the console.debug lines to this.debug (where this is the node) in this file: node-red-contrib-meshtastic/src/decode.ts at master · meshtastic/node-red-contrib-meshtastic · GitHub

[Steve-Mcl]

Alternatively, you could just edit the file ~/.node-red/node_modules/@meshtastic/node-red-contrib-meshtastic/dist/decode.js and replace all 9 instances of console.debug with (()=>{}) - this will silence them.

you would need to restart node-red afterwards for the file to load.

warning, this could break your node-red installation (make a copy of the file before messing with it)

[WJ4IoT]

Thanks you all for this quick investigation, Steve you are great, I am a BTE guy and your guiding is more than straight enough so I can to handle it. From scratch I would have been lost.

Just started with Node-Red so not worrying about loosing too much and some parts I already have in my fingers. Yes, I did have other balloons in the air already to tackle this problem but thought that this question was basic and for some simple to answer. Tomorrow when my Pauwel Kwak Amber is out of my system again I try it out or maybe in 5 minutes.

Buldog Trial & Error

[TotallyInformation]

Hmm, having finally after some years worked out how to correctly intercept console (without adding a function to the stack and therefore throwing off the trace), I wonder if it would be useful to have a node-red runtime plugin to intercept and redirect/block console outputs?

[ThingsTinkerer]

What's up with community packages doing stuff like this? And what about sending out a new message, removing all incoming props. Another unrelated example, it feels like conventions of stuff like topic and payload is sometimes doing harm. A cron scheduler node requiring the use of topic, but it had no purpose in the node itself. I added an empty space... Don't add limitations and restrictions just because no other usage can be envisioned!?!

[Steve-Mcl]

@ThingsTinkerer

A cron scheduler node requiring the use of topic, but it had no purpose in the node itself

If you are referring to cron-plus, the topic is for differentiating schedules (since there can be more than one) when the message is emitted, permitting the user to route it in a switch or (for example) send directly to MQTT.

[WJ4IoT]

Hi Steve good morning, your last solution " (()=>{})" does do the job for me; avoiding decode results polluting the logging file. So I am completely served and I will mark it as the solution which suited my case. So thanks again.

My request does not go beyond this. A request for re-writing this decode node/function is certainly not included. But to answer your question below is the buffer from MQTT (before DECODE):

[10,84,13,136,169,229,226,21,255,255,255,255,34,42,8,3,18,36,13,0,0,12,31,21,0,0,140,2,24,13,37,45,121,168,103,40,2,88,180,1,120,0,128,1,216,140,252,5,152,1,7,184,1,13,72,0,53,22,198,168,55,61,46,121,168,103,69,0,0,48,64,72,5,96,150,255,255,255,255,255,255,255,255,1,120,5,18,8,76,111,110,103,70,97,115,116,26,9,33,52,51,51,99,57,102,52,99]

I give you some back round, a bit in a NOOB way; Meshtastic is a LoraWan device which main function is to send data over RF with a potential coverage (*) of 10 km or more. This data can be pickup by other Meshtastic device and send through creating a Mesh. You can activate MQTT on them, at that point it will use the GSM of Wifi/BT to send data to a MQTT server, this can be in JSON format (not often activated) but it is always in "protobuf style".

There is only one MQTT topic, this topic contain an encrypted PortNum. This PortNum divides the topic in different type of encrypted data like text, position, metrics, etc. The example itself has PortNum = 1 with a text message.

There are other solutions available to extract this data. In example there is a newly added NodeRed plugin and there are two integrations in Home Assistant (or the simple MQTT-JSON version).

potential coverage can also be reduced to 777 meter when your antenna is not on high ground, indoor, concrete jungle so location can be important.

[jbudd]

@WJ4IoT

Meshtastic is a LoraWan device which main function is to send data over RF with a potential coverage (*) of 10 km or more. This data can be pickup by other Meshtastic device and send through creating a Mesh. You can activate MQTT on them, at that point it will use the GSM of Wifi/BT to send data to a MQTT server, this can be in JSON format (not often activated) but it is always in "protobuf style".

This sounds like an interesting application of Node-red. Maybe you could consider a more detailed account of how you are using it in the "Share your projects" section?

[WJ4IoT]

It is but I am likely not the right person for it, I need already CPR for simple thinks like having the correct syntax of a if-statement in a function.

Also there are alternatives out there, including a Python one I forgot to mention, but I want to work with Nodered because of the flow style it offers.

CPR = Cut, Paste & Replace

[jbudd]

@WJ4IoT

I need already CPR for simple thinks like having the correct syntax of a if-statement in a function.

You and me too!

[ThingsTinkerer]

Yeah cron-plus is the one. I fully get there are plenty of reasons to include topic as an option! What bothers me is that it is mandatory. Can't those that always include topic see that others may not want/need it? Regardless, just a very minor rant from me. Sorry for off topic.

[Steve-Mcl]

@ThingsTinkerer

What bothers me is that it is mandatory

It is hardly a hardship - I always try to add nice touches like sane defaults or incrementing defaults:

I get what you are saying however when I designed cron-plus, it was 100% with the intention that the schedules would be emitted with the topic as the identifying property for later routing & therefore necessary. Partly too, to prevent folk saying "how do i know which schedule was triggered" (early versions did NOT have the "fan out" option). Later, as it evolved more & dynamic ops were added, that meant a unique name for the schedule was necessary - so in theory, now, topic could be left blank. Instead, what I did was, if you leave it blank it will auto generate a default for you! In short, if you dont need it, dont use it (let it be what ever it defaults to).

Now for a (somewhat contrived) use case of topic to demonstrate why - I am sure you can infer what it would be used for in downstream nodes :

name	when	topic	payload	notes
break-start	10:00, 13:00	restrrom/1/light	true
break-end	10:15, 13:45	restrrom/1/light	false
break-info1	10:00, 13:00	restroom/1/ohd/text	"Break time"
break-info2	10:10, 13:40	restroom/1/ohd/text	"Time to get a wriggle on"
break-info3	10:15, 13:45	restroom/1/ohd/text	"Mush Mush"
free-vend-start	13:00	restrrom/1/coffee-machine/1/price	0
free-vend-end	13:10	restrrom/1/coffee-machine/1/price	125

In short, it is there for a reason. It is not really enforced (since it gets defaulted)

[TotallyInformation]

@WJ4IoT

I need already CPR for simple thinks like having the correct syntax of a if-statement in a function.

Well, that's what ChatGPT is for!

[Colin]

@ThingsTinkerer

Can't those that always include topic see that others may not want/need it?

Do you mean that a missing topic should be a valid value for a schedule, or that the node should somehow know to ignore the topic passed in?

[ThingsTinkerer]

@Steve-Mcl

Partly too, to prevent folk saying "how do i know which schedule was triggered" (early versions did NOT have the "fan out" option). Later, as it evolved more & dynamic ops were added, that meant a unique name for the schedule was necessary - so in theory, now, topic could be left blank. Instead, what I did was, if you leave it blank it w

To avoid derailing further, I made a separate discussion here: Simplify flow using cronplus