Running NodeRed on Pi 3B+.
Raspbian version = 10 ("Buster")
The flows on NodeRed are working. However we had a case where after about 1 week, the unit stopped working. We re-installed another new SD card with new image and all working again. So it is related to SD card.
Question:
In Node Red is the NR, writing logs/data to the SD card in the background?
Would like to know if this could be possible cause.
How do we check to see what data NR is Writing to the SD card?
Any help would be appreciated.
We already have requested users to not power on / off the PI as we have read that this may also corrupt the SD card.
Hello
Have a read here https://www.zdnet.com/article/raspberry-pi-extending-the-life-of-the-sd-card/
Nodered also writes to the log folder.
Mounting some folders to tmpfs so there are less sd card writes is a very good idea on a pi anyway and pi should also always be shut down properly as just pulling the power will indeed destroy sd cards very quickly.
But as you are running a Pi 3b+ you could also think about booting from usb as this is supported out of the box for that model. This way you can use a cheap ssd which will give you a much longer lifespan as most ssds have things like write leveling and error correction build in.
Johannes
Worth saying that the log is generally fairly quiet when things are running 'normally' - there aren't many nodes that log unless there is a warning or error that needs logging.
From previous discussions on this topic, the general advice is to make sure you use a good quality SD card and power supply.
Hi Nick and Team.
First of all, the more I use Node Red, the more we are impressed- its an elegant , platform. You made the complex much easier!
I did look at the /var/log/syslog . It looks like some new info is recorded every 1 hour which seems very reasonable. Is this the correct place to look?
We are going to switch to the Samsung EVO+ 32 GB ( after reading other posts recommendations)
In order to get a better idea of what the problem was you need to analyse in what way the unit was not working.
With a good quality card normal logging and so on will not wear the card out for many years.
However if you, for example, run a browser on the Pi then features such as cacheing in the browser can write many megabytes of data and over a period that might become a problem.
The tmpfs in RAM seems to have solve an issue with a couple of PiZweo-W that were eating SD cards. Its a BLE bluetooth dongle "presence" detector and wrote a small file of raw bluetooth data every 10-15 seconds.
The "shutdown properly" is not truly possible with IOT devices -- had an 8 minute whole neighborhood power outage last night, so unless you have them all on UPS and will always be around to shut them down "properly" its gonna happen.
I'm a bit late to this, but I would point out that assuming you are using a decent make of card and that it isn't tiny, SD-Card problems are mostly a thing of the past. I haven't changed a card in several years now despite 2 Pi's running full-time with occasional crashes (not related to Node-RED) or power outages.
So even unexpected halts don't seem to cause issues any more even with my Pi2.
Just make sure that your power supply is beefy enough for the job (especially if you have any other hardware attached), and that it isn't overheating.
And most of all, get a decent make of card (I always use Samsung EVO or EVO-Pro cards) - of a LARGE size, I always use 32GB cards.
That last part is particularly important as it ensures that there is loads of space available for wear levelling.
I run several services on both Pi's, I take no steps to reduce writes, I don't think I've brought a new card in at least 3 years, maybe more.
Last time I checked, writes were happening about once a second. I use Telegraf to capture system stats to InfluxDB which are then charted using Grafana.
To reinforce what @TotallyInformation said, I have just discovered that on a Pi 3 that has been running for about three years, I think, I had the mosquitto logging set so as to log all transactions. I don't remember setting that, it must have been quite some time ago. The result was that it was logging 125MB/day which was then being zipped into 90MB, so writing 215MB/day. This is on an 8GB SD card. It has been running like that for many months, possibly a couple of years, with no problems.
In contrast my syslog is about 80kB/day, there really isn't any need to worry about that magnitude of data.
Just off to check the mosquitto logging on my other systems
While you are at it, check for ever-growing databases too
InfluxDB is great for this - it automatically shards and keeps extending the db and it is easy to forget about it. I've caused myself some issues on the Pi because although I automatically trim my detail recording (e.g. sensor recordings every minute) to 7d, I let my aggregate db (hourly avg, max, min) grow indefinitely and eventually the indexes get so large that a Pi will start to struggle. A good way to get InfluxDB to peak to 100% CPU periodically.
On my new setup, my hourly db is trimmed to a few years of data.
Incidentally, InfluxDB is much smarter than MongoDB - the 32bit version in particular likes to simply crash without warning and corrupt the db. It also couldn't manage to create a DB anywhere near the size it should have been able to without crashing.
I pulled this out of my old node-red google groups post.
It is the notes file I keep for working with influx and storing data in buckets.
#I keep a set of notes around to jog my memory for working with it. I've greatly expanded on it with how my brain handles the ideas. you may think vastly differently then me.
#To access influxdb
influx
#To Create a DataBase
#CREATE DATABASE "database_name_here"
CREATE DATABASE TempSensors_F
#To Show a DataBase I've Created
SHOW DATABASES TempSensors_F
#To delete a database i've created
#Delete database with
#DROP DATABASE "database_name_here"
DROP DATABASE TempSensors_F
#Retention Policy - How Long to store information in the database
#Think of this as a bucket and its volume is represented as time. its a bucket of time.
#In this bucket you can store time. But you have to define what kind of time it stores. every second? every min? every hour? (how_far_data_points_are_separated)
#How many days are the data points separated for (how_many_days)
#How long do you want to keep the stored time in the bucket (ammount_of_time_to_hold_onto_information)
#CREATE RETENTION POLICY "how_far_data_points_are_separated" _for_"how_many_days" " ON "database_name_here" DURATION "ammount_of_time_to_hold_onto_information" REPLICATION 1 "is_this_the_default_database?"
CREATE RETENTION POLICY "1m_for_45d" ON TempSensors_F DURATION 45d REPLICATION 1 DEFAULT
#to show a retention policy ive created
#SHOW RETENTION POLICIES ON "database_name_here"
SHOW RETENTION POLICIES ON TempSensors_F
#To delete a retention policy ive created
#DROP RETENTION POLICY "how_far_data_points_are_separated" _for_"how_many_days" " ON "database_name_here" DURATION "time_to_hold_onto_information" REPLICATION 1 "is_this_the_default_database?"
DROP RETENTION POLICY "1m_for_45d" ON TempSensors_F DURATION 45d REPLICATION 1 DEFAULT
#Granularity - the scale or level of detail present in a set of data. your going to need more buckets! You want high amount of data for the last month. Do you need every bit of data from 10years ago. lets space out them data points.
#the whole thing would look like this
CREATE RETENTION POLICY "1m_for_45d" ON TempSensors_F DURATION 45d REPLICATION 1 DEFAULT
CREATE RETENTION POLICY "2m_for_90d" ON TempSensors_F DURATION 90d REPLICATION 1
CREATE RETENTION POLICY "5m_for_180d" ON TempSensors_F DURATION 180d REPLICATION 1
CREATE RETENTION POLICY "10m_for_366d" ON TempSensors_F DURATION 366d REPLICATION 1
CREATE RETENTION POLICY "30m_for_731d" ON TempSensors_F DURATION 731d REPLICATION 1
CREATE RETENTION POLICY "60m_for_1826d" ON TempSensors_F DURATION 1826d REPLICATION 1
#Sorting to the next bucket - from the DEFAULT bucket
#Think of this. The first bucket has values every 1 min. We want to sort it into the next bucket at
#CREATE CONTINUOUS QUERY ""query_how_far_data_points_are_separated"_for_"query_for_how_many_days"" ON "database_name_here" BEGIN SELECT mean(*) INTO "database_name_here"."place_how_far_data_points_are_separated"_for_"place_for_how_many_days".:MEASUREMENT FROM /.*/ GROUP BY time("where_to place_simalar_buckets_of_time"),* END
CREATE CONTINUOUS QUERY "cq_2m_for_90d" ON "TempSensors_F" BEGIN SELECT mean(*) INTO "TempSensors_F"."2m_for_90d".:MEASUREMENT FROM /.*/ GROUP BY time(2m),* END
#Sorting to the next bucket - from another previously sorted bucket
#CREATE CONTINUOUS QUERY ""query_how_far_data_points_are_separated"_for_"query_for_how_many_days"" ON "database_name_here" BEGIN SELECT mean(*) INTO "database_name_here".""place_how_far_data_points_are_separated"_for_""place_for_how_many_days""".:MEASUREMENT FROM "database_name_here"."previous_bucket_of_how_far_data_points_are_separated"_for_"previous_bucket_of_for_how_many_days""./.*/ GROUP BY time("where_to place_simalar_buckets_of_time"),* END
CREATE CONTINUOUS QUERY "cq_5m_for_180d" ON "TempSensors_F" BEGIN SELECT mean(*) INTO "TempSensors_F"."5m_for_180d".:MEASUREMENT FROM TempSensors_F."2m_for_90d"./.*/ GROUP BY time(5m),* END
#the whole thing of sorting buckets would look like this
CREATE CONTINUOUS QUERY "cq_2m_for_90d" ON "TempSensors_F" BEGIN SELECT mean(*) INTO "TempSensors_F"."2m_for_90d".:MEASUREMENT FROM /.*/ GROUP BY time(2m),* END
CREATE CONTINUOUS QUERY "cq_5m_for_180d" ON "TempSensors_F" BEGIN SELECT mean(*) INTO "TempSensors_F"."5m_for_180d".:MEASUREMENT FROM TempSensors_F."2m_for_90d"./.*/ GROUP BY time(5m),* END
CREATE CONTINUOUS QUERY "cq_10m_for_366d" ON "TempSensors_F" BEGIN SELECT mean(*) INTO "TempSensors_F"."10m_for_366d".:MEASUREMENT FROM TempSensors_F."5m_for_180d"./.*/ GROUP BY time(10m),* END
CREATE CONTINUOUS QUERY "cq_30m_for_731d" ON "TempSensors_F" BEGIN SELECT mean(*) INTO "TempSensors_F"."30m_for_731d".:MEASUREMENT FROM TempSensors_F."10m_for_366d"./.*/ GROUP BY time(30m),* END
CREATE CONTINUOUS QUERY "cq_60m_for_1826d" ON "TempSensors_F" BEGIN SELECT mean(*) INTO "TempSensors_F"."60m_for_1826d".:MEASUREMENT FROM TempSensors_F."30m_for_731d"./.*/ GROUP BY time(60m),* END
Haha, nice one. Who has time and energy to remember all that text though?! I use a Grafana extension to help me remember some commands. Then some might find their way into Node-RED (of course). Some notes find themselves in my "Research" OneNote Notebook and the ones that might be useful more publicly will end up on my Blog.
The thing about retention and continuous queries is that you might only ever do them once. Don't know about you but that is nowhere near enough to be able to remember them.
Like Einstein, I don't believe in cluttering my brain with things I don't actually need to remember so I write them down and forget them. (Pretty much my only similarity with Einstein to be sure ).
).