Large virtual memory usage with 64-bit OS Bullseye

General
1

We are testing 64-bit OS bullseye and noticed large virtual memory usage by Node-RED (more than 10GB), compared to 200MB virtual memory usage of 32-bit OS. See attached pictures of 32-bit OS vs 64-bit OS node-RED virtual memory usage.

Any thoughts on this?

32-bit virtual memory
32-bit virtual memory1036Ă—529 155 KB

64-bit virtual memory
64-bit virtual memory945Ă—565 145 KB

2

Don't worry about it, virtual memory is not real memory. You can see it is still only using 13% of your 1GB of RAM.

If you are really interested then do some research into what the virtual figure actually means.

3

Not unusual - this from a 64bit Intel laptop which is my home server:

image
image798Ă—275 6.86 KB

And yet you can see that the server is very happy:

image
image1127Ă—165 7.14 KB

4

@Colin @TotallyInformation Thanks much. This confirms my thoughts.

After testing for a month, 64-bit OS provides better performance than 32-bit. Continuous writing to InfluxDB with large amount of data per second can cause RPI4 4GB version CPU and memory overload in 1 day and half. Switching to 64-bit OS and 64-bit InfluxDB, continuous writing to RPI4 2GB version can go up to 3 days.

So it is worth upgrading to 64-bit OS for RPI 4 with 2GB memory and above.

Will run some testing on RPI3 1GB version.

5

64bit is more than just memory handling.

6

Unless you are running too fast for the system to keep up then it should not fail after 3 days or 300.

1 Like
7

4 sensors at 3.2KHz sampling rate in parallel. Write data to influxDB in batch (grouping data points together and write in batch. Otherwise the database will have problem much earlier.)

8

Wow! What data are you sampling?

1 Like
9

Wireless vibration sensors with triaxial acceleration and temperature measurements :grinning_face_with_smiling_eyes:

2 Likes
10

Frankly, I'm amazed that you don't lose data!

3 Likes
11

Are you writing the data to the SD card or have you got a hard disk plugged in?

12

Testing uses old 32GB micro SD cards UHS type I.

This is an extreme test case to check system limit on our ends.

Normal usage won't write that much data and that often. Typical DAQ time is less than 5 minutes per hour.

13

Probably you are writing too quickly and the card can't keep up. You can check that using the iotop command.

If it is important that the data are not lost then don't use the card for storing the data, use a hard disc or SSD.

14

The card speed is well within the limit. Please see attached iotop output screen shot.

IOtop
IOtop1142Ă—358 78.5 KB

In real applications, we use faster micro SD card than the one in testing.

Hard disc or SSD is not ideal for industrial environments, where the micro SD cards are installed inside the wireless gateways that could have tough conditions. Yes we use hard drives or SSDs for servers inside the control room.

15

When using SD-Cards:

  • Use a good, branded make such as the Samsung EVO and EVO Pro. The main thing to look for is a card that has built-in wear levelling. Most of the cheap cards don't.
  • Specify a card that is significantly oversized for the application. That way the wear levelling will work to your best advantage.
  • Schedule card replacements. Ensure that all of the cards get replaced on a schedule. For home use I've had cards last well over 6 years but under heavy use this is likely to be less.

Obviously, you will want to have spare cards in stock and have them preprepared ready to slot in. You may want to consider a simple config script to ensure that the card can be made production ready with all software updates and device specific settings with minimal effort.

16

We have been using Sandisk Micro SDs since 2015 and have excellent results. Very reliable in environments that temperature goes more than 70 degree Celsius in the summer (under direct sunshine in metal structure), and minus 30 degree Celsius in the winter.

You are right that extra card space gives extra life, since there is algorithm that writes to the card intelligently to reduce the wear and tear.

1 Like
17

What is causing the system to fail after three days then?

18

There are different reasons for 32-bit and 64-bit OS. At the testing, we used InfluxDB 1.8.10 for both 32-bit and 64-bit OS.

32-bit InfluxDB is installed at 32-bit Bullseye OS RPI 4 4GB, 64GB 70MB/s fast card. 64-bit InfluxDB is installed on 64-bit Bullseye RPI 4 2GB with 32GB 10MB/s slow card. So the hardware set up of the 64-bit system is inferior to the 32-bit system.

For 32-bit OS and database, the problem could happen after 1 and half day (Please note that the DAQ duration is much longer than our recommended system usage limit. Typical system usage is a few minutes per hour. )

For 64-bit OS and database, the failure may be due to the reason that the system run out of space.
At the end of the 3rd day, the database size is more than 25GB. The card space is only 32GB. When we add the virtual memory size of 10GB and database size of 25GB, the total is more than 32GB. After 3 days, database read and write still work well. But the CPU will go more than 100% usage very often at idle state.

19

I've seen that happen with InfluxDB on a Pi before. Letting the db's get too large will do that I think.

20

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.