Debug node store somewhere and use space on sd card?

General
1

hello,
I have the issue, that I run out of space on my sd card wit hraspberry. I have 16GB card, it happens after I added smartmeter to read the IR from my smartmeter at home. I put a debug node behind it and didnt deleted. Everything worked so far, but today I wanted to make some adjustments and it says there is no space left. I thought maybe a new start of raspberry will delete the cashe or something, but now my raspberry wont boot and I am afraid to lose all my flows.

Is there a way to boot raspberry with 0kb space left?
Is there a way to recover my flows?
Do debug or to much reading some nodes use space on sd?

2

Debug nodes do not as a rule store any information anywhere on disk.

If you enable the system console option the it will also output what ever was sent to the node to the Node-RED log. Depending on how you have instaledl and configured Node-RED it is possible that this has driven the disk usage. (It appears that the default journald file size is set to 10% of total disk space, so unlikely)

As for working out what is using the space, your best bet is to mount the card on another linux machine (Unless you install extra drivers I don't beleive you can not read the card directly from Windows, thought you might be able to use WSL).

Once you have mounted the card you should be able to recover the flows/settings/creds from /home/pi/.node-red

3

Thank you very much so far. I managed to install ubuntu on virtualbox and to read the files stored on the sd. So when I clear some space, raspi should manage to boot again, right? Where can I find node-red log? Can I just delete this log file?

4

On your ubuntu machine install ncdu
sudo apt install ncdu
Than go into the top level directory of the card and run
sudo ncdu .
That is a dot on the end. Wait for it to complete (which will probably take a few minutes on an SD card) then it will show you where the large directories are.

5

Often it's huge files in /var/log that makes a Raspberry rum out of space.

You can fairly safely delete /var/log/*.gz (If the card is mounted on another device you will have to include the mount point in that path.)

6

Find out exactly what the files are before you delete them. All log files generated by installed apps should rotate so a sequence of files is used and old ones are deleted. If that is not working for something then you need to know what it is in order to fix the problem. Otherwise it will just happen again.

7

I used this comman ncdu for var/log folder, because for whole card it takes a while. I found out first, that my log files are 10gb huge. The largest file is daemon.log with 4,3gb and the second is syslog.1 with 3,6gb.
Why are they so large, how could I prevent them getting this big?

8

In the syslog folder show us what this gives
ls -l

Then run
more daemon.log
which will show you a page of data at a time. See what is being written there to fill it up.

9 
insgesamt 10367912
-rw-r--r-- 1 root     root          0 Apr  1  2021 alternatives.log
-rw-r--r-- 1 root     root        409 Mär  6  2021 alternatives.log.1
drwxr-xr-x 2 root     root       4096 Okt 18 00:00 apt
-rw-r----- 1 root     adm      515429 Mär 14 20:36 auth.log
-rw-r----- 1 root     adm       36361 Mär 12 23:17 auth.log.1
-rw-r----- 1 root     adm        1929 Mär  5 23:17 auth.log.2.gz
-rw-r----- 1 root     adm        1919 Feb 26 23:17 auth.log.3.gz
-rw-r----- 1 root     adm        2057 Feb 19 23:17 auth.log.4.gz
-rw------- 1 root     root     173633 Mär 14 20:17 boot.log
-rw-r--r-- 1 root     root          0 Jan 11  2021 bootstrap.log
-rw-rw---- 1 root     utmp          0 Mär  1 00:00 btmp
-rw-rw---- 1 root     utmp          0 Feb  1 00:00 btmp.1
drwxr-xr-x 2 root     root       4096 Mär 14 00:00 cups
-rw-r----- 1 root     adm  4596390311 Mär 14 20:36 daemon.log
-rw-r----- 1 root     adm  1395026890 Mär 13 00:00 daemon.log.1
-rw-r----- 1 root     adm       18377 Mär  6 00:00 daemon.log.2.gz
-rw-r----- 1 root     adm       20069 Feb 27 00:00 daemon.log.3.gz
-rw-r----- 1 root     adm       30769 Feb 20 00:00 daemon.log.4.gz
-rw-r----- 1 root     adm       13586 Mär 14 20:17 debug
-rw-r----- 1 root     adm       16068 Mär 12 15:19 debug.1
-rw-r----- 1 root     adm         824 Feb 19 10:31 debug.2.gz
-rw-r----- 1 root     adm         632 Feb 12 11:56 debug.3.gz
-rw-r----- 1 root     adm         635 Nov 19 21:57 debug.4.gz
-rw-r--r-- 1 root     root          0 Okt 18 00:00 dpkg.log
-rw-r--r-- 1 root     root       1026 Okt 17 20:59 dpkg.log.1
-rw-r--r-- 1 root     root        906 Sep 26 09:53 dpkg.log.2.gz
-rw-r--r-- 1 root     root       6039 Mär 15  2021 dpkg.log.3.gz
-rw-r--r-- 1 root     root       2808 Sep 26 09:53 faillog
-rw-r--r-- 1 root     root       4010 Sep 26 09:53 fontconfig.log
drwxr-xr-x 3 root     root       4096 Jan 11  2021 hp
-rw-r----- 1 root     adm      281836 Mär 14 20:36 kern.log
-rw-r----- 1 root     adm       40608 Mär 12 16:01 kern.log.1
-rw-r----- 1 root     adm       16244 Feb 19 10:32 kern.log.2.gz
-rw-r----- 1 root     adm        8282 Feb 12 11:57 kern.log.3.gz
-rw-r----- 1 root     adm        8399 Nov 19 21:57 kern.log.4.gz
-rw-rw-r-- 1 root     utmp     292292 Mär 14 20:17 lastlog
drwx--x--x 2 root     root       4096 Mär 14 20:36 lightdm
-rw-r----- 1 root     adm      688128 Mär 14 20:36 messages
-rw-r----- 1 root     adm       43467 Mär 13 00:00 messages.1
-rw-r----- 1 root     adm         362 Mär  6 00:00 messages.2.gz
-rw-r----- 1 root     adm         288 Feb 27 00:00 messages.3.gz
-rw-r----- 1 root     adm       16448 Feb 20 00:00 messages.4.gz
drwxr-xr-x 2 avahi    root       4096 Mär  1 00:00 mosquitto
-rw-r--r-- 1 root     root          0 Mär 14  2021 nodered-install.log
-rw-r--r-- 1 root     root      34074 Mär  6  2021 nodered-install.log.1
drwx------ 2 root     root       4096 Jan 11  2021 private
-rw-r----- 1 root     adm   765624320 Mär 14 20:36 syslog
-rw-r----- 1 root     adm  3831547237 Mär 14 00:00 syslog.1
-rw-r----- 1 root     adm    24894531 Mär 13 00:00 syslog.2.gz
-rw-r----- 1 root     adm        3157 Mär 12 00:00 syslog.3.gz
-rw-r----- 1 root     adm        3115 Mär 11 00:00 syslog.4.gz
-rw-r----- 1 root     adm        3954 Mär 10 00:00 syslog.5.gz
-rw-r----- 1 root     adm        3156 Mär  9 00:00 syslog.6.gz
-rw-r----- 1 root     adm        3086 Mär  8 00:00 syslog.7.gz
-rw-r----- 1 root     adm      415925 Mär 14 20:36 user.log
-rw-r----- 1 root     adm        4480 Mär 12 16:01 user.log.1
-rw-r----- 1 root     adm         222 Mär  4 11:48 user.log.2.gz
-rw-r----- 1 root     adm         163 Feb 25 16:32 user.log.3.gz
-rw-r----- 1 root     adm         665 Feb 19 10:32 user.log.4.gz
-rw-rw-r-- 1 root     utmp     520192 Mär 14 20:36 wtmp
-rw-r--r-- 1 root     root          0 Mär 14 20:36 Xorg.0.log
-rw-r--r-- 1 root     root      13507 Mär 14 20:36 Xorg.0.log.old
-rw-r----- 1 kernoops adm       61440 Mär 14 20:17 xrdp.log
-rw-r----- 1 root     adm       14623 Mär 14 20:17 xrdp-sesman.log

this is deamon log:

Mar 13 00:00:02 raspberrypi Node-RED[351]: NEW DATA
Mar 13 00:00:02 raspberrypi Node-RED[351]: CURRENT PROCESS STEP 4 IN CHECKMESSAGE
Mar 13 00:00:02 raspberrypi Node-RED[351]: MATCH-RESULT DATA: "\r\n1-0:0.0.0*255(0273260481404)\r\n1-0:1.8.0*255(00012911.0592
550*kWh)\r\n1-0:21.7.0*255(000093.80*W)\r\n1-0:41.7.0*255(000014.45*W)\r\n1-0:61.7.0*255(000012.70*W)\r\n1-0:1.7.0*255(000120.
95*W)\r\n1-0:96.5.5*255(80)\r\n0" -> null
Mar 13 00:00:02 raspberrypi Node-RED[351]: NEW DATA
Mar 13 00:00:02 raspberrypi Node-RED[351]: CURRENT PROCESS STEP 4 IN CHECKMESSAGE
Mar 13 00:00:02 raspberrypi Node-RED[351]: MATCH-RESULT DATA: "\r\n1-0:0.0.0*255(0273260481404)\r\n1-0:1.8.0*255(00012911.0592
550*kWh)\r\n1-0:21.7.0*255(000093.80*W)\r\n1-0:41.7.0*255(000014.45*W)\r\n1-0:61.7.0*255(000012.70*W)\r\n1-0:1.7.0*255(000120.
95*W)\r\n1-0:96.5.5*255(80)\r\n0-" -> null

I was scrolling for about 30 seconds and still on second 2.

10

Do you recognise those messages? Are they coming from a debug node which is set to log to the console? If so then it looks as if you may be stuck in a loop in the flow continually hitting that output.

You can see from the sequence of files for daemon.log that it is rotating the files ok, it is just that you are outputing a vast amount of information to it. You can delete those big files then the pi should boot again, then work out why node-red is, apparently, stuck in a loop. If you have trouble getting into node red because it is all clogged up writing that data then stop node-red and start it again using
node-red --safe
which will start the editor without starting the flows. You can then edit the flows and they will start when you deploy.

11

this data I got in daemon.log looks like the date I get from my IR head, that I placed on my smartmeter. It looks like it communicate all the time and store everything in the log files.

I use smartmeter node with 1 sec. poll rate and yes it was shown in the debug node, also I put it on my dashboard.
I didnt noticed, that node red is slow or something. Everything worked fine. Even today when I had zero space, I was able to edit the flows, but I could not deploy, because I had no space.

12

If you look in the debug node in the editor is it outputting just once each second?

13

yes, exactly

14

and is it showing exactly the messages that you see in syslog? The format of the data does not look correct for a debug node. Try disconnecting the wire to the debug node and see if it stops outputting to the file. Is suspect it is not the debug node that is writing that.
Once you have the Pi running again then you can watch the file by running
tail -f /var/log/daemon.log

15

How exactly is the meter being read ? Is it an external app being called that returns a value ? I would guess whatever is reading the data is what is writing to that log file.

16

it is this one node-red-contrib-smartmeter
I dont know how it does it. The IR-Reader is connected to an USB on raspberry.

17

I have my pi running again, thank you for your help. I deleted deamon.log and I am diconected from the reader-head now. I will connect tomorrow and check again.

but this is was the returned data I was using from the node:

msg.payload={
    phase1 : msg.payload["1-0:21.7.0*255"].values[0].value,
    phase2 : msg.payload["1-0:41.7.0*255"].values[0].value,
    phase3 : msg.payload["1-0:61.7.0*255"].values[0].value,
    gesamt : msg.payload["1-0:1.7.0*255"].values[0].value
}

it looks pretty much the same, as stored in the daemon.log

18

Have you got anything in your flow that outputs a message like that? Use the menu search feature to search for
CHECKMESSAGE
and see if it finds anything. That will tell you whether it is coming from something explicit in your flows.

19

Aha! What would we do without Google?

20

No, I dont have any output with CHECKMESSAGE.

Sry Colin, I dont know what I got to see with the link you sent. I'm not norwegian and dont have this smartmeter. I'm in Germany and using Easymeter. Also I dont use any 3rd party software or programms or scripts. Only this node, I've mentioned before.