How to join two payloads, when one is intermittent?

Hi

I am trying to build a flow that can handle the output of two separate payloads from an alarm system, where one (payload A) tells me the name of the room/sensor and the other (payload B) tells me if that sensor is open or closed.

image

The trouble I have is that the name of the room/sensor (Payload A) e.g Office sensor is only sent the first time it’s tripped, and then all subsequent messages e.g open or closed, will be Payload B messages, until such a point that another room sensor trips (A) and the same open/close payloads (B) will occur again.

How best do you handle such a situation, so that I can log each event as something like this.

Office sensor open
Office sensor closed
Office sensor open
Office sensor closed

Note: it’s possible that two sensors could be opened at the same time, but in each case it seems the room sensor name would be announced first.

I hope that makes sense ?

When the sensor is triggered, doesn't the data include an ID that identifies the actual sensor? If not, that seems rather limiting & not terrible usable.

If the A payload simply maps the ID to a real location, that would be easy to handle as you would simply add this to a flow or global variable to track ID's against locations. Then, when an alarm is triggered, you can simply re-map the location back to the sensor ID. You already have the location map in-memory.

Sorry for not being clearer,

The alarm panel is connected to a Vera Home Automation controller, which captures a numeric value via VenderStatusData, which I convert via a global lookup to its room/sensor name. E.g 6 = Office

Via the other VenderStatusCode route a numeric value for open and closed is received which I also convert via a global look up e.g 901 = opened 902 = closed

I’ve done all the conversions, I just can’t work out how to handle the flows of data as the room/sensor data is not sent every time - it’s just seems to be sent the first time a room/sensor is tripped (if it was not the last room/sensor value sent/recorded)

So somehow my flow needs to remember the last room/sensor name (from VenderStatusData) and keep resending it with the separate open/close data (coming from VenderStatusCode) until it’s updated with a new room/sensor name - and then that room/sensor needs to be stored and resent until the next change ?

Is there a node out there that can receive a payload, store it and allow itself to be resent repeatedly when requested/injected until and new value is recieved, and when that new value is, the process starts again with the new value ?

A sequence would be..

Payload A = 6 (office)
Payload B = 901 (open)
Payload B = 902 (closed)
Payload B = 901 (open)
Payload B = 902 (closed)
Payload B = 901 (open)
Payload B = 902 (closed)
Payload A = 10 (kitchen)
Payload B = 901 (open)
Payload B = 902 (closed)
Payload B = 901 (open)
Payload A = 6 (office)
Payload B = 901 (open)
Payload A = 10 (kitchen)
Payload B = 902 (closed)
Payload A = 6 (office)
Payload B = 902 (closed)

Payload A is sent to state the sensor being tripped, but it's stored and not sent again until it's replaced to identify another sensors that's tripped. Payload B always shows the status of the sensor (Open or Closed) in focus

There is a pattern for that sort of thing in the cookbook - Send placeholder messages when a stream stops sending : Node-RED

However, I don't think you need it.

If you use a Join node configured in manual mode to create a key/value Object, it will use the value of msg.topic as the key and msg.payload as the value in the Object.

Under the After a number of message parts option you want to set 2 and make sure the and every subsequent message is ticked.

So lets say your two topics are foo and bar. The result of the Join node will be a payload of:

{
   foo: <most recent value from topic foo>,
   bar: <most recent value from topic bar>
}

I think that will give you the information you need in a single message.

2 Likes

Hi Nick

I can’t seem to get this to work, as the payloads come from two sources, I have two different msg.topics, and if I try to join them, with the key/value object option, I receive the error.

"Message missing key property 'msg.topic' - cannot add to object"

I’ve tried changing the topic on one feed in, and remove it from the other, but I still get the same error. Here is a snapshot of how I’ve converted the two data feeds payloads before they go into the join node.

The error says "Message missing key property 'msg.topic' and the middle message in that screenshot clearly has a blank topic property.

For this to work, they must both have msg.topic set to something that distinguishes them.

Many Thanks, that helps and it seems to have moved things forward - the first two are from the two separate flows feeding in, and the 3rd is after the join node has done its job - see below

I now need to see how it works when multiple sensors are tripped simultaneously..

How does the join decide which of the incoming topics to use as the key ?

That question doesn't really make sense. It doesnt decide because it handles each message as it arrives. The message has a tooic, the join node will add that message to it's output object, using that message's topic as the key and it's payload as the value.

Agreed, but as we’re bring the contents of two separate flows together, doesn’t each message coming into the Join node have it’s own topic and payload ?

If you look at the third image above (the result of the join) - the msg.topic that was chosen is ‘sensor status’, but why not the other msg.topic from the other flow, which was ‘sensor’

I’m obviously missing something - so I appreciate your patience :grin:

The topic that is left at the top level is the last to arrive.

Leaving the topic set to that from the last message can sometimes be very useful as it means that in the payload you have the latest values for all topics passed to the Join node, but you can also interrogate the topic to see which was the last message to arrive. This allows you to do things like recalculating something based on the latest values for topic A and topic B each time a topic A message is received, but doing nothing when a topic B message arrives.

1 Like

Couldn't you use a retained Variable to capture the sensor location as text and then concatenate it to the status data? Using a Flow scope Variable retains the last date until overwritten.

Hi @tonylane26

I’m not quite sure what you mean, would you have an actual mock up/example that you could share ?

The problem is already solved using the Join node isn't it?

You’re right, I do have something in place with the Join mode, but it’s not perfect, I assume due to the order they can occasionally be processed - hence I was curious how @tonylane26 suggestion would work.

(With NR there’s often more than one way to skin a cat: i’ve no idea where that saying came from :rofl:)

How have you configured the Join node and what is not perfect about it?

It’s still as Nick suggested; and works well - but as the Join order is dictated by when they arrive, on a very few occasions that has reversed/changed

(Keep in mind the source alarm panel data is taken from my Vera Home Automation (MIOS) controller, which processes it first, so that’s a factor in this too, which I can’t control)

Have you got a switch after the join to ignore the result when the input message is the location message?
If sometimes the state message is arriving before the location message then I suspect that logically there is nothing you can do. How can you tell whether the state message that arrives before the location is a state message for the previous location or for the one that hasn't arrived yet.
You could put a small delay in the state message path, but then you run the risk that a state message for the current location might get delayed till after the next location message.
The problem you are trying to solve is, I believe, unsolvable, as the incoming data is not reliable. Is there really no way that whatever is giving you the data can provide them in one message? Can you not read the two vendor status values in one go somehow rather than having them appear along separate paths.

Can you expand on that? The properties of an object have no natural order so the order messages arrive should irrelevant. Or is this related to the msg.topic of the result being set to that of the most recently received message?