Detect when context's variable changes its value

Core Development Feature Requests
1

I'm working on a new node that will output a message when a variable receives new value. The format of the message will be similar to:

{
  variable: "the name of the variable",
  oldValue: theOldValue,
  newValue: theNewValue  
}

For this to happen the node need to be notified when someone modifies variable's value trough flow.set(), global.set() or the Change node . To me the obvious way of doing it is to hook the context.set() method (I don't like polling for changes), but I really have no idea how to do it.

Is this possible at all? Or is there already a node that does the same job?

2

The fact that you need to do this suggests that you are using a context variable where you would be better to send a message. Rather than setting the context variable wherever you set it, send a message to the node that needs to know, with the new value.

Alternatively write the value to MQTT and then subscribe to that topic wherever you need to know about it. Then the node that needs to know will automatically get told. Also, if the variable is required to be persistent over a restart then make the MQTT topic persistent and then the nodes that need to know the value will get informed on startup.

3

The idea is that the node will generate new message when it detects a variable change. I'm aware of MQTT ways of achieving that. But I don't need MQTT in this case. It is NodeRED runtime related question.

4

One of the main thoughts behind Node-RED is that you should be able to see what is going on, so messages flow along wires etc. And these events trigger other events. Having invisible connections is not really part of the current thinking.

5

I see this new node as "IN" node (i.e. without inputs). You subscribe for a variable (a.k.a. topic) and receive notifications from every Set operation. That's similar to MQTT but is more like a pub/sub model for the Context (global or flow). Yes there is a hidden connection between the node and the Context. But this is also true for MQTT node and the MQTT server - the connection between them is also hidden.

The practical side of having such node: there are cases when you need to be notified when a value had been changed and you're interested to see how much is that change. Then build logic like "if the change is bigger than, then do X". Or if the last state was X and the new state is Y, then do Z. I think in most of the cases people use the Function node, write code, store the last state etc. This is error prone.
Why do I want to have this at Context level? Well to achieve simple change detection in most of the cases people use the Function node, write some code, store the last state, compare values etc. So they already have the value stored in the context. So why bother with MQTT and complex custom flows at all? I think such node will be useful to many NodeRED users and will simplify the logic and flows.

My question was is it possible to receive such notifications. If not, then I want to make a proposal for adding capability for subscribing for "context().Set" operations. I think it would not be hard to be implemented. The moment is right since you're implementing variable persistence i.e. you're working / modifying the Context class.

I see this as a new Context API:

context.subscribe(varNameOfInterest, callback);

where callback's signature is: function(oldValue, newValue) {}

1 Like
6

No, it is not possible to do with the current API and we're not going to rush anything in at this point with 0.19 ready to ship in the next couple weeks.

I am not in favour of building a pub/sub system (which is what you're proposing) around the Context. As Dave said, the wires in a flow represent the logical flow of events (with specific exceptions for the Status and Catch nodes). The MQTT nodes represent a well understand interaction with an external system.

One of the use cases for the persistent context feature for 0.19 is to allow multiple runtime instances to share a common context. If implementations had to support observability of changes to any value that would greatly complicate any implementation of the context store api.

So no, this isn't something we're going to hold up the 0.19 release for. We can discuss further, but it needs to be done so in full consideration of the changes being introduced and whether it fits with the whole flow-based programming model.

7

Not in a rush. I'll be happy to discuss the idea. My proposal is based on my overall experience with NodeRED with real life tasks.

8

What happens if for instance you inject every second on a function that pulls out the value of the variable and this is directed to a rbe node (what makes is that message only will pass if msg.payload has changed, then from then to wherever you wanted to be notified?

Will do exactly what you expect.

Regards

1 Like
9

Yeah. But polling. Yuk. :wink:
Then again there was once Object.observe but that got deprecated/removed pretty quick so there must have been fairly good reasons.

You could possibly add a link node next to/ near where it gets written and use the other end of that to expose it where you want

10

I have a been working on a flow for a while now and appear to have a very similar mindset to @PKGeorgiev. I am currently using lots of both flow and function context variables. However, thanks to the comments I received in a separate thread, I am starting to realize the power of using topics and mqtt and subflows to achieve the same result.

The main drawback that I am seeing though is a lack of the ability to debug a flow that uses subflows and mqtt. The context variables have been immensely helpful because I can open the context data tab and view exactly what is going on. Subflows do not even allow you to enable/disable debug nodes and while you can have context variables in the subflow you cannot view them in the context data tab (unless I am mistaken). Is there any validity to my thoughts here or do I still just need to better learn the Node-Red mindset so I do not have to rely on debugging and storing data the way I am now?

Thank you, and I'm having a blast working with Node-RED so I truly appreciate your efforts!

11

I can kind of see what you are meaning and sure I can see a way of doing it.

But honestly, I am behind the other people who say there is an easier way.

But to indulge your idea, this is how I would do it:
(Sorry this if being typed on the fly and I am making it up as I go along. So there may be bugs in what I say.)

Get an inject node and set it up to pulse outputs at .... what ever frequency you want.
That output goes into your node.
The code will be something like: (and here comes the seat of your pants part)

var a = flow.get('name');           //  The global thing you are watching for change.
var b = context.get('name');     //  A local store so it knows if the global thing has changed.
if (a !=b)
{
    //  The flow variable has changed.
    //  Do something here and save the new value to context 'name'
    //  But you will need (most likely) an output to action something happening.
}

But MQTT would simplify and allow so much more use of what you are wanting to do that how you are doing it.

I can get that you are wanting to do this to learn. Been there a lot myself.
By all means try.

I hope you can get it working.

But I think that you are making something which can be done easily a lot harder than it needs to be.

Don't be afraid to try MQTT once you have this nut cracked open.

Good luck.

12

Thank you for the reply. I have integrated a great deal of MQTT now into my flow and it's been a very powerful tool. Like you said I am beginning to see "there is an easier way". Some of my difficulty was just being so new to working with javascript objects and JSON, but I am now understanding how to work efficiently with this data.

1 Like
13

Hello,
I'm coming too late ...
I also have the same problem and I am new to NodeRed. I still have a simpler solution compared to MQTT to detect the change of a variable or a context.

When I change a context or variable, I use the Storage / file and Advanced / Watch pair instead.
The Watch node intercepts the writing in the file and can thus link it to another node that will test if the variable has changed.

For my part it simplifies the connections of a duplicated subflow 6 times in the same Flow. Otherwise the Flow would be too 'hairy' and would be less readable.

Image here : Example :

Best regards

14

Do you mean you are using the built in localfilesystem storage as defined in the link below? If so what have you got the cache and flushInterval set to?
https://nodered.org/docs/api/context/store/localfilesystem

15

No, I'm just using the two nodes described above as this : https://mega.nz/#!dod3CASC!mTzttaBphaYzckpbqbDAwHJqBz1tsaHPOTqH16PnGAs

16

I don't want to start downloading stuff thanks, just export that bit of flow and post it here please.

17

In french : voilĂ ... voilĂ ...

[{"id":"13a868fb.5b337f","type":"debug","z":"c7653a56.17239","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","x":630,"y":700,"wires":[]},{"id":"f84f27ca.964398","type":"file","z":"c7653a56.17239","name":"","filename":"test.txt","appendNewline":true,"createDir":false,"overwriteFile":"true","encoding":"none","x":320,"y":700,"wires":[[]]},{"id":"dfc16141.9afcb","type":"watch","z":"c7653a56.17239","name":"","files":"test.txt","recursive":"","x":470,"y":700,"wires":[["13a868fb.5b337f","e8bb3ce4.de7668"]]},{"id":"fb00b32c.2ccd6","type":"change","z":"c7653a56.17239","name":"","rules":[{"t":"set","p":"payload","pt":"msg","to":"sdfdsfqsfdqsdf","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":130,"y":700,"wires":[["f84f27ca.964398"]]},{"id":"e8bb3ce4.de7668","type":"function","z":"c7653a56.17239","name":"The context as changed ?","func":"var fff = env.get(\"FOO\")\nmsg.payload = fff\nreturn msg;","outputs":2,"noerr":0,"x":710,"y":760,"wires":[["578f2343.ed750c"],["d7909979.02d8a"]]},{"id":"578f2343.ed750c","type":"debug","z":"c7653a56.17239","name":"YES","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","x":950,"y":740,"wires":[]},{"id":"d7909979.02d8a","type":"debug","z":"c7653a56.17239","name":"NO","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","x":950,"y":780,"wires":[]}]
18

Sorry, that doesn't make sense to me. It appears to write to a file, then a watch node sees the file has changed and a function node picks up an environment variable and writes to o/p 1 but nothing is written to o/p 2. I don't understand what it is trying to do.

19

So is this idea... not implemented or being considered? As a new feature? I would rather not do polling at a solution as well, which is what I am doing now. I have a situation where several flows need to know the state of a CEC controlled device. So I poll the CEC controlled device, update a global context variable. To then have ever other flow, do pulling of the context variable? Just eats a lot of cycles. Would rather have a event trigger when the original global context variable changes state, so I have only one polling flow, not many. The downstream responders to change of state are different devices, in some cases, so it is not a simple case from a communication perspective, as in all flows exist in the same instance of NR, so I have to use MQTT as well, but the NR instance doing the polling is getting hammered.

20

@Steve-Mcl posted a function with context listeners here