Context file not written when using multiple localfilesystem entries

General
42

It seems that you are using PM2 to run node-red. I don't use PM2, it isn't the recommended way with Ubuntu, but if you want to use it then it is up to you to make sure you know how to use it. Somewhere in the setup I would expect that you set a user that the node-red process runs under.

Do you mean that you write to that folder from inside node-red? If so then it probably isn't a permission problem.

43

Correct. Plus I don't have a problem with just default single file store and we see other data being written. Could there be an issue with config "directory" parameters - they are being ignored and some kind of files are being created with data written in default context folder (in a mixed up way).

44

Perhaps you should mention that it's not recommended in the guide section - that's how I ended up with this set up since I was using ec2/ubuntu configuration.

https://nodered.org/docs/getting-started/aws#running-on-aws-ec2-with-ubuntu

Regardless, do you believe there is a bug related to using multiple file stores?

45

That is going to a different folder, in the .node-red heirarchy. You have specified that the others go to a different folder (node-red without the dot.) They are both under the same user home folder though.

What does these commands show
ls -al /home/ubuntu/node-red
ls -al /home/ubuntu/.node-red

Try creating the directories that you have told it to save the context in. I can't test your flow till the morning.

46

I did create those directories. And just to be certain (there was no some overlooked permission issue in that particular sub-directory (/home/ubuntu/node-red) I changed the directory to (/home/ubuntu) and that's where I have other nodes (like exec node) writing/creating files without issues.

I did the same inject/change node test and got the same outcome. It is really strange, after restart all three context stores get populated with a value that was stored in one of them (same one as before)

image
image1072Ɨ203 8.64 KB

47

I can confirm that the alternate folder settings for storage do not work.

settings.js

    contextStorage: {
        default: {
            module:'memory',
        },
        file: {
            module: 'localfilesystem',
            // config: {
            //     flushInterval: '10', // default is 30s
            // },
        },
        altFile: { 
            module: 'localfilesystem', 
            dir: '/home/julian/nrAltFileStore',
        }
    },

Flow

[{"id":"1d32ca5b3fdd573d","type":"inject","z":"8f633713fe222323","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":220,"y":240,"wires":[["07688e5a74b5e1eb"]]},{"id":"07688e5a74b5e1eb","type":"change","z":"8f633713fe222323","name":"","rules":[{"t":"set","p":"test1","pt":"flow","to":"This is the default store - a flow var","tot":"str"},{"t":"set","p":"#:(file)::test2","pt":"flow","to":"This is the file store - a flow var","tot":"str"},{"t":"set","p":"#:(altFile)::test3","pt":"flow","to":"This is the altFile store - a flow var","tot":"str"},{"t":"set","p":"test4","pt":"global","to":"This is the default store - a global var","tot":"str"},{"t":"set","p":"#:(file)::test5","pt":"global","to":"This is the file store - a global var","tot":"str"},{"t":"set","p":"#:(altFile)::test6","pt":"global","to":"This is the altFile store - a global var","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":400,"y":240,"wires":[["6e468a657e5df5d3"]]},{"id":"6e468a657e5df5d3","type":"debug","z":"8f633713fe222323","name":"debug 1","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":640,"y":240,"wires":[]}]

image
image713Ɨ1067 57.4 KB

Nothing appears in /home/julian/nrAltFileStore.

Latest node-red running under Debian (via WSL).

48

On a later re-run I did get an error - but related to the flow variable to the file context store:

Welcome to Node-RED
===================
6 Dec 22:18:29 - [info] Node-RED version: v3.0.2
6 Dec 22:18:29 - [info] Node.js  version: v16.18.1
6 Dec 22:18:29 - [info] Linux 5.15.74.2-microsoft-standard-WSL2 x64 LE
6 Dec 22:18:29 - [info] Loading palette nodes
6 Dec 22:18:29 - [info] Settings file  : /home/julian/nrlive/data/settings.js
6 Dec 22:18:29 - [info] HTTP Static    : /home/julian/nrlive/public > /
6 Dec 22:18:29 - [info] Context store  : 'default' [module=memory]
6 Dec 22:18:29 - [info] Context store  : 'file' [module=localfilesystem]
6 Dec 22:18:29 - [info] Context store  : 'altFile' [module=localfilesystem]
6 Dec 22:18:29 - [info] User directory : /home/julian/nrlive/data
6 Dec 22:18:29 - [warn] Projects disabled : editorTheme.projects.enabled=false
6 Dec 22:18:29 - [info] Flows file     : /home/julian/nrlive/data/flows.json
6 Dec 22:18:29 - [info] Server now running at http://127.0.0.1:1881/red/
6 Dec 22:18:29 - [info] Starting flows
6 Dec 22:18:29 - [info] Started flows
6 Dec 22:18:49 - [info] Stopping flows
6 Dec 22:18:49 - [info] Stopped flows
6 Dec 22:18:49 - [info] Updated flows
6 Dec 22:18:49 - [info] Starting flows
6 Dec 22:18:49 - [info] Started flows
6 Dec 22:19:22 - [error] Error writing context: Error: ENOENT: no such file or directory, rename '/home/julian/nrlive/data/context/8f633713fe222323/flow.json.1670365162017.tmp' -> '/home/julian/nrlive/data/context/8f633713fe222323/flow.json'

And here is what I found in the file:

{
    "test2": "This is the file store - a flow var"
}    "test3": "This is the altFile store - a flow var - 2"
}

Oops!

I had changed the node-red settings to this (and restarted):

    contextStorage: {
        default: {
            module:'memory',
        },
        file: {
            module: 'localfilesystem',
            // config: {
            //     flushInterval: '10', // default is 30s
            // },
        },
        altFile: { 
            module: 'localfilesystem', 
            dir: '/home/julian/nrAltFileStore',
            base: 'context',
            cache: false,
            flushInterval: 1,
        }
    },
49

Do you see (after restart) the same data being written in both ?

Does the "base" need to be specified?

50

Not quite, you can see what it wrote - to the wrong place.

I added the other properties to see if that made a difference.

There is clearly a bug when using a local file store with a different directory. We need to let the devs have a look when they can so that they can fix.

51

Would one of you guys be kind enough to raise an issue on the node-red repo please & I'll check it out.

Thanks.

52

Done.

Sorry, details a bit light as it is late but all details are in this thread.

1 Like
53

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.

55

Finally got around to investigating this properly.

Everything is working as designed. The problem in this thread was I gave bad advice in my comment right at the start over how to properly configure multiple file stores.

I said to use the following configuration:

contextStorage: {
   default: "memoryOnly",
   memoryOnly: { module: 'memory' },
   file: { module: 'localfilesystem', dir: '/home/user/.node-red/context1' },
   file2: { module: 'localfilesystem', dir: '/home/user/.node-red/context2' },
   file3: { module: 'localfilesystem', dir: '/home/user/.node-red/context3' }
},

Had I read the docs I wrote I would have spotted that the configuration options must be under a config object:

contextStorage: {
   default: "memoryOnly",
   memoryOnly: { module: 'memory' },
   file: {
      module: 'localfilesystem',
      config: {
         dir: '/home/user/.node-red/context1'
      }
   },
   file2: {
      module: 'localfilesystem',
      config: {
         dir: '/home/user/.node-red/context2'
      }
   }
},
2 Likes
56

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.