I expect I'm being thick or just tired but I am having a problem with persistent context stores. No, I'm not being thick, I can see the problem happening. It is because there is a delay of around 30 seconds before persistent variables are written to the fs. While that is fine normally, Node-RED, I …

There is clearly no way to do this totally reliably across all platforms from within a Node.js service itself. I've come to the conclusion that if you want more reliability from a persistent data store, you need to hand the data off to a dedicated managed data engine. Either a database engine or so…

[image] TotallyInformation: Just to be clear, this is not a Node-RED issue, it is a Node.js one. I would even say it's neither. I'd blame Windows. :grin: In my case, the only reason I went for the file context store was it's simplicity by not requiring an extra service and that the data is n…

[image] TotallyInformation: one day to create a context store backed by PouchDB or similar Now that indeed would be useful and interesting.

[image] kuema: I would even say it's neither. I'd blame Windows Well, the issue I linked to makes it clear that isn't the only issue. The main debate being whether Node.js should create its own solutions or whether that should be handed off to the JavaScript standards committee. The Windows i…

[image] dceejay: Now that indeed would be useful and interesting. Oh dear, Dave. Now you've gone and done it! I sense another distracting project bubbling up!

Yes, would be good for several reasons, not least as it would give the API a workout. @kuema the bones of the redis design is here https://github.com/node-red/node-red-context-redis/pull/1

[image] TotallyInformation: Well, the issue I linked to makes it clear that isn't the only issue. Yes, I fully agree with that. But the lack of proper signalling support in Windows is an issue not only with NodeJS, but extends to other environments (Java, Python, ...) as well. [image] Tot…

Don't forget there is always the option (which is suitable, or even superior for many use cases) of using MQTT with retained topics rather than persistent context. The same problem will arise if MQTT is shut down unexpectedly as it also saves its data at a configurable interval to the file system, …

Hmm, will the existing MQTT config nodes let me connect to them from my own node? They should do shouldn't they? The main issue for me would be that MQTT wouldn't work well for a cache without significant extra logic. I wouldn't want to just re-push a single serialised topic for example as that wou…

@TotallyInformation I'm sorry, I must have missed a step - why are you creating your own cache nodes to duplicate the functionality of Context?

A catch with persistent file stores

General

zenofmud 4 February 2020 11:17 54

FYI - I discovered this and reported it back in November (see Node-RED crashing at startup due to bad flow context (reproducible)) and opened an Issue about it (see https://github.com/node-red/node-red/issues/2369 )

Topic		Replies	Views
Node-red won't start General	63	3663	23 November 2020
Context File corrupt/truncated - node-red auto-restart failure General	56	1160	30 July 2023
Durable and reliable contextStorage - DB/mysql/mariadb based General	55	772	10 February 2025
A guide to understanding 'Persistent Context' General	87	45802	4 January 2022
Problems understanding and implementing Persistent Storage General	42	1301	31 August 2023

A catch with persistent file stores

Related topics