Hi friends,
Just wanted to know if it is possible to setup SQLite as context-storage instead of built-in context storage (memory or localfilesystem). ?
As its already recommended, setting localfilesystem flush interval lower than 30 seconds will lead to SD card wear and reduce in life expectancy. Hence, reducing flush-interval is not an option.
I have also read some days ago(I think!) that flow fuse team has implemented SQLite based context storage (to avoid loss of data) meaning it must be possible somehow.
My goal is to buffer incoming data and process it to SQL server later. No data should be lost in case node-red crashes.
Though i have implemented SQLite based buffering to avoid data-loss but it is done at front-end(node-red editor). I was looking for backened solution.
Regards,
Shubham
It is possible. However, I don't believe anyone has written a suitable plugin.
That is not necessarily true. It depends very much on how often you are updating things and how you are structuring your context variables.
Localising variables will create more separate files which helps minimise writes. And even a large, global variable, if only written to rarely will not cause any real wear.
The biggest factor though is the card itself. If you have a good card that has wear-levelling (Such as the various Samsung EVO series) and has plenty of free space, card wear will not be an issue for many years.
When I was running on Pi's, I had EVO cards running for years and I never did anything special to reduce wear. I had many logs and other fast-changing files writing to those cards.
Of course, any SD-Card will eventually fail - that happens even if you don't use them in some cases. So you should always have external backups. But then SSD's and HDD's also fail.
ALL databases buffer in memory and write in batch periodically, that's how they are so efficient. A full DB engine such as MySQL may also maintain a changelog file that allows some additional recovery but no db running on an SD-Card is going to be free of potential corruption. In fact, they may well be worse since you may be able to recover useful data from a text file (e.g. a .json file) that would not be recoverable from a more complex db file.
So you would need to run your db server on a different platform.
If you have data that is absolutely critical not to loose any, write it direct to a DB. Or write it indirectly using MQTT or some other MQ. Either way it needs to be over to a separate server using something more reliable than an SD-Card for storage. And properly backed up of course.
