RBE state storage

Hello,

The RBE node is great to automate MQTT based on a state-change event. However quite often it is needed to check the last published state of a sensor, this information is now hidden inside the RBE node (can be achieved with an extra change node).

My proposal is to add the following functionality:

  • a setting in RBE to select state storage between: nodeContext, flowContext, globalContext
  • in case of flow and global, user should be able to define a context key

This functionality will provide the following scenarios:

  • at any time the last published state could be read from the flow or global context,
  • last state is preserved between deploys (IMO this sucks right now),
  • if context key is shared between RBE nodes, they all act as kind of one node, which can sometimes be useful too

This change seems to be quite easy to be implemented. I will of course develop it, however, I need to know if it makes sense in your opinion, so the pull request will be accepted.

Hi,

interesting thoughts. The basic idea sounds ok but I suspect there may be a few edge cases to think about. Currently the node works per topic - so multiple different messages can be handled by the one node - and if not they are all packed under a dummy topic. So the user would need to know that to interrogate the right one.

Also if now in wider context then any other node can also manipulate the state - you mention other RBE nodes which indeed may be an interesting use case - but I'm not sure letting any node mess with the state is a good idea.

But yes certainly up for a discussion to see what makes sense for a PR.

PS - there is an outstanding PR waiting to go in so will need to take that into account also.

I have already seen the "Optional topic" feature, as otherwise, this will be a part of this feature request :slight_smile:

I agree that this is a little bit more expert setting and for a casual user, it may be confusing. As you noticed, the state can be influenced by other nodes (afaik there is no read-only flag for contexts), so it also can break the locality of the node - however, I see a great benefit of having access to the state, and of course, by default, it will keep the current, local behavior.

I'm not sure what this great benefit is... - apart from being a convenience to save state (which the change node is designed to do for you). The premise of an event driven system is that nothing has changed from the last time it changed so any subsequent nodes should really be holding their own state - polling for status is an anti-pattern.

1 Like

In most automations, at some point (in some flow) I need to know the current state of the sensor.

Simplified examples if you detect movement (on PIR) you want to "lock" the light if door is already closed. At 11 pm I want to set the night temperature on my thermostat, but not if the window is opened (and the thermostat is off).

Right now I'm using HA input, so I can get states from it (without additional pooling of sensors),. As I want to move into pure-mqtt, when the state is reported I need to:

  1. Trigger an action if the state is changed (RBE node)
  2. Save the state for possible future use (to not pool it from the sensor, which I agree is anti-pattern).

I agree that if you only need to save the state, change-node is the way to go. If you only need to do an action on change, rbe-node is the way to go. But at least for me, it is usually both, I need to do some action and also to have access to the saved state.

Or course I can always put rbe-node + change-node to achieve that, but then the same data is somehow saved in two locations. I can also use switch-node + change-node to simulate rbe+save state functionality. Nevertheless getting access to the state in rbe-node looks very natural for me.

PS: if the locality is the issue, rbe can simply publish the state to flowContext while the whole logic is still based on a local variable (closure is used for that in rbe-node). But this imho is basically limiting this functionality.

PPS: switch+change in a subflow will basically emulate it, but I thought that having this as core functionality will be useful for more ppl.

That is what Retained Topics are for in MQTT. That will automatically keep the previous value for you and when the flow restarts the initial values of all the retained topics are provided to the flows.

If you want the RBE not to pass on the first value it is given then use the Ignore Initial Value option.

Yes - I hear what you are saying. The same argument could be made for many nodes... (that of adding in extra functionality as options). But that comes at the cost of complexity and understanding and readability of the flow. I could argue that it is easier to have them as separate as that the two uses you are making of them are two different uses. Most typically the access to a saved state is for late arriving readers like displays - (though the dashboard nodes already store state as well :slight_smile: - or can be recovered from (say) a retained MQTT topic (stored there again)...

A basic principle is that nodes should well defined in its purpose (Creating Nodes : Node-RED) ie should do what they do and do it well - so in this instance we chose that the change node is the way to get things in and out of context and the rbe node should do rbe... but...

I'm not 100% against the idea - just trying to work out how really necessary/time saving/useful it is.

PS -re more people - well you are the first person in 5 years to ask I think - but let's see who else joins in :slight_smile:

Sure - it is a good discussion, and tbh that's why I asked before doing any work, to not spend time on something not useful for more ppl. For myself, I can use a subflow or program a function node.

I wonder how people deal with that scenario, to access the current state in a flow.

PS: I don't think that retained topic is addressing this usecase.

There are two different things here.

  1. allowing the RBE's internal state to be stored in context so it can be restored after a restart (assuming the user has enabled a persistent context store).
  2. allowing that state to be stored in flow/global context so it can be modified/accessed outside of the node.

I think there is value in doing #1 as it solves a real problem that cannot be easily solved today. Yes you could construct a flow that re-injects the last known value to the RBE node, but you'd also need a way to stop that re-inject from being passed on by the RBE node. Whilst its academically possible to do, it is not exactly a simple task.

As for #2 - that's where it becomes less clear as to whether we should do it. There is a simple workaround for that today as has been said. Adding support for this will also make the RBE code more complicated as it now has to re-read the current context value every time it wants to make a comparison in case it has changed. (It doesn't have to do that in the case of #1 - it only needs to read back the value when the node is restarted).

Storing internal state in node context is what node context is there for. It just isn't something we've added to the nodes yet. I'm sure there will be other nodes that will benefit from it - so before rushing to add it to the RBE node, it may be worth considering some of the other likely candidates to see how this might work for them. We could then consider adding it in a clear and consistent way, rather than in an ad-hoc manner.

One of the RBE modes is to ignore first value so this can be used for exactly that scenario.

Ah yes, forgot that was there already. But that only solves one part of it. You still need to create the extra flows to store the value and reinject it at start up. Certainly less complicate with the ignore inital value option in the RBE node - and that has probably solved the problem for users up to now.

But it's a reasonable discussion - which we've flirted with a few times before - about making more wide use of node-context in the core nodes to do exactly what it was designed for.

Isn't that handled by my suggestion of using a retained mqtt topic (at least for the case here), as I suggested earlier, where I also pointed out that it could be used along with the ignore first value option.

Yes that would do it. But it really isn't the point of the discussion I'm trying to get to.

Even using MQTT, you still need extra nodes and flows to do the work (pus, in that case, it also requires an MQTT broker to be running). I'm really not have that discussion with you yet again.

1 Like

The first message in the thread describes the use case as exactly that. MQTT into RBE node, so whilst it is valid to discuss the more general situation the use case here would appear to be solved by my suggestion. Unless I am misinterpreting the problem completely.

That's assuming that there is a MQTT flow.
What about feeds arriving via http, or in my user case from a serial port.

indeed that is what @colin assumed from the very original question ...
and yes it has expanded from there to other potential cases.

rbe is great because it is one very simple node to achieve the task of passing messages only when the state changes. If I have to do mqtt retain, or store state somewhere and reinject on startup / deploy or any other more complicated thing, it will be much easier is to use flowcontext based switch + change-node to store state to the flow context.

Adding support for this will also make the RBE code more complicated as it now has to re-read the current context value every time it wants to make a comparison in case it has changed. (It doesn't have to do that in the case of #1 - it only needs to read back the value when the node is restarted).

rbe code is 86 loc, it is a very small and simple module (which is good). Adding support for that is imo not more than ~10 additional lines (of js), it will remain small and simple.

I don't get the point with making extra comparisons. It always has to read the previous state when it makes the decision to pass the message. Now it is taken from the clousere (node.previous). If implemented it will call get/set on xxxxContext. Unless context storage is badly implemented and using it is resource intensive, I really don't see the point.