Redundant node-red instances plugged on AWS IOT CORE

General
1

I am using node red after some IOT connectivity layer: IBM IOT cloud now, but migrating to AWS core IOT.
The IOT messages come through the IOT layer on MQTT, and are sent to one or several rednode Instances.
The issue I have is to make sure a MQTT message is handled only once by one node-red instance and not all of the running instances...

It works fine on IBM cloud thanks to shared subscriptions (feature implemented in IBM only).

On AWS, I can't find a documented way to do this.
I was thinking using SQS queues because it is an action available at the output of the core IOT layer:
IOT COre (mqtt) --> SQS queue --> Node-red instances with a SQS node to retrieve SQS messqges.

Does that make sense or am I reinventing the wheel and there are easier ways?

2

In pretty sure you can use mqtt shared subscriptions anywhere with v5 mqtt broker. Just give it a go.

Node-red 1
Subscribe to $share/group1/test

Node-red 2
Subscribe to $share/group1/test

Node-red 3
Publish to test

You should only get a message to one or the other.

3

Thanks Steeve. This is what I did on IBM core and it works perfectly.
But on AWS, issue is: AWS IOT core does not support this feature. Their mqtt is based on 3.1.1
https://forums.aws.amazon.com/thread.jspa?messageID=757689

4

That thread was only raised in 2016 - shouldnt be too long now :rofl:

I seen your comment - i fear mine would be more strongly worded :roll_eyes:

Can you not use another broker?

5

Well I could probably but that complexifies my solution, and increases cost (I quickly checked HiveMQ for instance). I wrote a DM to the guy who is apparently still a community manager at aws and answered at the time, but no answer so far...

I am always amazed that such basic redundancy usecase such as mine seem so undocumented. Was the same in IBM btw (but they did support shared subscription...)

6

I have implemented what I initially thought of:

  • AWS IOT has an action defined to send all messages to a SQS queue.
  • I have created a "worker" instance of Elastic Beanstalk with node-red on it. It takes inputs from the above queue.
  • With auto scaling I can increase number of instances, they all pick their messages in the SQS queue, and only once. In node red it translates in a httpIN node listening on "/", as this is where the queue will deliver the messaqes.

Now I am missing how to send messages back to my devices.
In IBM IOT there were the IBMIOT OUT node where you could easily send data down to the device.
I realise that I never looked at what this node was doing. Publishing to a device specific topic?

I need to find an equivalent node to send data on AWS (not just status as the shadow service seem to be doing) to a specific device.

Can my device subscribe to a topic with its own ID, and then I will publish on this one?
Is this what IBM IOT is doing begind the scene?

7

Providing my own answer to whom it might help.

  • On node-red the entry point is a HTTP IN node listening for POST on localhost:80
  • You also need the HTTP OUT node at some point to ack with a 200OK, otherwise the message is not removed from the queue.
  • To send info byck to my device, I use a MQTT OUT node and publish shadow updates to the device specifically. The device will receive the "/accepted" answer.
8

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.