Question about context and RAM

General
1

Hello,

I have a question about where data is saved when I save it to the context(memoryOnly)?

I am using a rasberry pi 3 and get a huge amount of data via MQTT and don't want to write it at sd card at once, but store it at RAM and after ram usage gets to 80% I want to write all data to the .txt file.

So here is 2 questions:

1.Is data stored in RAM when I am using a context.set()
(With memory only option)?

2.Is there any way to track a usage of RAM(in %)?

Thanks in advance for answers!

2

Yes, but instead of storing it to RAM, you could save it to the filesystem (disk). If you did so, it initially saves the data to RAM, and then flushes to disk every 30 seconds.
The 30 seconds is set by default, although it can be changed, 1 minute, 5 minutes etc.

Have you searched the flow catalogue?
node-red-contrib-linux-memory
node-red-contrib-os
node-red-contrib-device-stats
just to name a couple.

1 Like
3

Have a read of Local Filesystem Context Store : Node-RED which probably explains it better than me!

4

Here is an example of increasing the flush time to 5 minutes.

    contextStorage: {
      default: "memoryOnly",
    memoryOnly: { module: 'memory' },
          file: { module: 'localfilesystem',
          config: {
                  flushInterval: '300'
                  },
          },
    },

...so in this example, it only writes to disk every 5 minutes (and if there is something to write).
It will write to disk immediately if node-RED is shut down, or a system reboot is made, so no data should be lost.

1 Like
5

Just remember that I don't believe using the filestore version free's up any memory. I believe it stays in memory as well. Indeed, I suspect that file-based variables are read on startup (though I've not tested it). Certainly since they are available on the context sidebar, they must be in memory when you are viewing that.

Also, in reguard to trying to understand Linux memory utilisation, it generally isn't enough to simply look at how "full" the RAM is. That's because Linux dynamically uses spare RAM to hold file caches.

Similarly, it is very hard to work out how much memory Node.js is using since it does dynamic memory recovery using the "Garbage Collection" method.

Frankly, you are better off not worrying about it until it becomes an issue. Instead just be sensible about how much data you hold in memory.

Also, this thing about the SD-Cards on a Pi keeps coming round on a depressingly regular basis. A good quality card with lots of spare spare will last for years as several of us have proven.

Basic rules are - don't over optimise and don't optimise too soon.

1 Like
6

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.