Context not updating according to flushInterval & Missing data

General
1

Hello!

The setup I have is 4 Node-RED instances, each one in a Docker container.

I want to share the global context between them, and therefore, in each instance's settings.js file, I have set the following:

contextStorage: {
    default: {
        module: 'localfilesystem',
        dir: '/data/',
        cache: true,
        flushInterval: 1
    }
}

and I turned /data/global into a Docker volume shared by all instances (for those not familiar with Docker, this just means that every instance's /docker/global folder is the same - if once instance's folder is updated, they are all updated accordingly).

Upon using an exec node to execute cat /data/global/global.json, I see that the file is indeed readable by all instances.

First problem - File does not update according to flushInterval

To make sure the file gets updated, I am executing global.set('_time', Date.now()) every second.

The first problem is that I noticed that the file /data/global/global.json is updating every 30 seconds and not 1 as mentioned in flushInterval.

Using an exec node to execute cat /data/global/global.json, I see that the file is updated accordingly by all instances every 30 seconds as well.

Second problem - The context does not update on other instances

I have one "master" Node-RED container which is the one that sets most of the global context.

As mentioned previously, executing global.set('_time', Date.now()) successfully updates global.json, and every Node-RED instance can read that file.

The problem is, when opening the "Context Data" tab on the other instances, it is empty (except for the values set by the itself).

If I execute global.set('_time2', Date.now()) every second in that instance, the content of global.json starts to switch between the values of the contexts of the instances.

And this makes sense, but this wouldn't happen in the first place if the other instances were updating their contexts according to the global.json file.

When I restart all the containers, they all get the same context. But of course, this is useless since after that, they become independent again.

Third problem - Sharing functions in context

In the context, I have a entry with functions inside. These are not written in the file (which makes sense), but I want to share them between instances.

I have found a solution for this problem: turning the functions into strings and then using eval() in each instance to turn them back into functions. But it would be great to find an alternative.

2

What happens if you change cache: true to cache: false

3

Thanks for the reply,

I don't want to do this as I will have to change the hundreds of global.get() calls and all their following code to use callbacks, and even nested callbacks. I have already started working on a MongoDB context, but fell into this asynchronous problem and decided to not use it.

I also don't know how to return and go to the next node from a callback, as returning from a callback does not return to the next node.

4

From what I interpret the documentation saying (Local Filesystem Context Store : Node-RED)
cache Whether to cache context in memory. Default: true
If you set cache to true, (in settings.js) it stores it in memory. I’m wondering if this is causing what you are seeing.

5

Try this;

contextStorage: {
    default: {
        module: 'localfilesystem',
           config: {
                    dir: '/data/',
                    flushInterval: '1'
                   }
          }
}

*Note the speech marks around the flushInterval time period, that are missing in your post.

6

@zenofmud Yes you're right, though that will make the calls asynchronous as you can see here.

@Paul-Reed Thank you, I tried it and it's still the same (I restarted Node-RED of course). The documentation shows it as an integer anyway since there are no quotes around it like the default of the base option.

7

To test, I've changed my settings.js flushInterval to 1 second;

    contextStorage: {
      default: "RAM",
                RAM: { module: 'memory' },
                Disk: { module: 'localfilesystem',
                        config: {
                                flushInterval: '1'
                                },
                },
        },

...and injecting a timestamp every second into a change node, saving it to global context (test).
Then 'tail' the context file, $ tail -F /home/pi/.node-red/context/global/global.json and I get;

log

which shows the context data being written to disk every second.
So, if this does not work for you, it at least eliminates issues with the settings.js entries.

3 Likes
8

Thank you, though it does not work for some reason.

Every other settings that I update does have an effect, so it is not me who does not know how to update settings.

Anyway, it's okay, I found a workaround - sending the needed variables through HTTP. It's not my favorite solution but it's fine.

9

Are you editing the correct settings.js?

Look at your node-red start up output - it will state where the settings.js is that it is using.

Also, did you change the syntax like @Paul-Reed showed you (nested config object)? ...

contextStorage: {
    default: {
        module: 'localfilesystem',
        config: {
            dir: '/data/',
            flushInterval: '1'
        }
    }
}

or ...

contextStorage: {
    default: "Disk",
        RAM: { module: 'memory' },
        Disk: { 
            module: 'localfilesystem',
            config: { flushInterval: '1' },
        }
    }
}
1 Like
10

Yes, since changing other settings takes effect, including adding entries in contextStorage

Ah, I did not pay attention to that! I am not using the filesystem anymore but will keep that in mind in case I will in the future, thank you everyone

11

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.