First integration experience between Django and Node-RED

[tahasamy]

I am preparing an integration tool to connect Django with Node-RED, which will allow for significant flexibility in granting permissions and access rights for users to control devices, check their statuses, and track actions taken by individuals. For example, in a factory setting with an entire team, not every person will have the same permissions.

I have also modified the Node-RED instance to accept only registered users from Django. This means that even if someone manages to access the IP address of Node-RED, it will only function with full reliance on Django for access rights. Node-RED can only be opened within an iframe on a Django page because it depends on sending the user's session ID to Django to determine whether that user has the right to access it.

I am currently adding functionality for cases where Node-RED and Django are in different locations using the internet instead of a local network. I am using encryption techniques to ensure that message information cannot be revealed under any circumstances, employing AES with Base64. However, when encrypting using Django with Python, I cannot decrypt it using Node-RED's JavaScript; the output is nothing, and the same happens in reverse, despite using the same settings on both sides. Encryption is just an option that I will leave out, as it may consume resources when it is not essential, such as when both are on the same local network or server.

I am happy to hear your suggestions.
there are two tab one for django and another from node red

github link

This example demonstrates my ability to pass data within Node-RED messages. In this example, I was able to add the user's username.

json
{"payload":false,"group_id":"Toggle","user":"taha","_msgid":"dd78072b9895ed10","_event":"node:8c17425cf6b74635","is_disconnected":false}

Note: This is done through the backend, not the frontend.
Note: The group_id works as the group name for the channel and as the key for caching the device status.
Note: Devices can be added or removed, and the number of points (here, points refer to the last status of the device) can be increased or decreased without stopping Django or modifying it. In some cases, I store the last 20 statuses, similar to a chart. There is a table through which devices can be added or removed, as well as changing the number of points.

[TotallyInformation]

Not sure how you are passing messages. But if Django and Node-RED are not on the same server, you really should use TLS wire encryption to protect the point-to-point traffic. This is really important for any commercial setting.

[tahasamy]

---

TLS must be used 100%. It will provide very high security, but the difficulty I am facing now is that I need to implement a WebSocket node that supports TLS.

I will explain the work concept simply. The project consists of two parts:

Part One:

When a person opens a page displaying the status of a device, they create a WebSocket that connects the browser to the Django server. The URL pattern looks like this:
re_path(r'ws/mywebsocket/(?P<group_id>\w+)/$', consumers.test.as_asgi()),
where they enter a group_id that equals device_id. In this case, the code (in django server) responsible for creating the WebSocket between the Django server and the browser will create another WebSocket connecting Django server to Node-RED. The difference is that all devices are connected between Django and Node-RED on the same URL.

But how will I distinguish between each message if it pertains to the first, second, or third device? This will be done through Django. Any message sent must be accompanied by its corresponding group_id. Here comes the question: someone might create a fake group_id. In this case, it will be rejected because there is a table that contains all group_id values and all their information, such as how many states you want to save for each device. Thus, I can add devices and control them as well. There are also two tables for permissions, which I will leave for the end, to give permissions to people who have the right to change or read the status of the device.

Part Two:

There is a problem in Part One: there must be a browser running that displays the dashboard for the WebSocket responsible for connecting Django to Node-RED to work. In this case, I need another process to run alongside Django, which will act as a WebSocket that receives the device statuses from Node-RED and sends them in the form of a JSON request with a key stored in the request header to confirm the validity of the request. The external process is designed to handle many requests sent to Django. This process is designed so that when it receives a request from Django, it creates a cache for the last states of each device according to the settings placed for it. Previously, I used to create the cache on the WebSocket side, which delayed the WebSocket, especially in cases where data was flowing at very high speeds.

Note: When the state changes, for example, when the lights turn on, the request goes to Node-RED. If Node-RED does not route the request back to users, it means something is wrong. This is a form of authentication to ensure that any state change of a device comes from Node-RED and not from any person.

---

[TotallyInformation]

@tahasamy

implement a WebSocket node that supports TLS

On the Node-RED side of things, the websocket nodes automatically uses wss if Node-RED itself is configured for TLS.

@tahasamy

rejected because there is a table that contains all group_id values and all their information, such as how many states you want to save for each device

I assume that you randomise the group id's so that they are not predictable?

@tahasamy

In this case, I need another process to run alongside Django, which will act as a WebSocket that receives the device statuses from Node-RED and

It feels like you should be using MQTT here rather than websockets. It is easy enough to configure a broker to use TLS and a quick search reveals that you can use MQTT with Django.

MQTT will make it much easier to manage the connections and will take care of a lot of the details for you.

While it is another daemon that must be run, using Mosquitto is VERY lightweight and should easily run on the same server as Node-RED.

[tahasamy]

@TotallyInformation

I assume that you randomise the group id's so that they are not predictable?

Making the group ID randomized acts as a first line of defense, but it is not a fundamental solution against intruders.
The primary line of defense is Django. In this scenario, we have two individuals trying to access the same page: one is authorized, and the other is not. Ideally, a non-authorized user should not be able to access a specific page unless granted permission. This example assumes the unauthorized user somehow reaches the page or WebSocket. In such a case, the connection should be terminated immediately, informing them that access is denied.

Additionally, using TLS with WebSocket adds an extra layer of security. As previously mentioned, we have two tables: one for users and another for groups, to define the permissions for each. Permissions are categorized into 'read' and 'read with change'.

[TotallyInformation]

@tahasamy

Making the group ID randomized acts as a first line of defense, but it is not a fundamental solution against intruders.

I am aware of that but you wouldn't be the first programmer to get caught out with that little bug.

[tahasamy]

Here’s an example illustrating the difference in permissions between users:

This is the message received by the unauthorized person.
i pass another two parameter one for permission and one for observation

{"payload":true,"group_id":"Toggle","user":"super","Permissions":"R","_msgid":"8b998903bc3fe134","_event":"node:8c17425cf6b74635","observation":"You do not have permission to access"}

[tahasamy]

@TotallyInformation

On the Node-RED side of things, the websocket nodes automatically uses wss if Node-RED itself is configured for TLS.

I mean creating a WebSocket server using Node-RED. I can't find anything in the documentation about adding TLS or not.

[tahasamy]

@TotallyInformation

It feels like you should be using MQTT here rather than websockets. It is easy enough to configure a broker to use TLS and a quick search reveals that you can use MQTT with Django.

MQTT will make it much easier to manage the connections and will take care of a lot of the details for you.

While it is another daemon that must be run, using Mosquitto is VERY lightweight and should easily run on the same server as Node-RED.

There are two features of the WebSocket server:

1. You don't need to rely on external software like Mosquitto. I made the external process creation happen from Django, so it starts with Django by creating a custom command for it. This makes it integrated within Django.
   python manage.py run_websocket_client
2. WebSocket is capable of handling higher traffic compared to MQTT.
   Just to clarify, all messages related to device statuses will pass through the external process.

I’m really happy to have this discussion with you.

If you have any advice, I’m excited to hear it. I’ll start converting it into an app extension with documentation so that anyone can easily add it within Django.

[TotallyInformation]

@tahasamy

I mean creating a WebSocket server using Node-RED. I can't find anything in the documentation about adding TLS or not.

The documentation for those nodes is a little light. But rest-assured, if you configure Node-RED to use TLS, the websocket nodes automatically use it.

[TotallyInformation]

@tahasamy

You don't need to rely on external software like Mosquitto

Understood but on the other hand, you get a message Q system specifically designed for lightweight interfacing between disparate systems. And in practice, Mosquitto is so well built it is pretty rock solid and has the advantage that it doesn't change that often so, compared to other server software, it requires little to no maintenance.

Without it, you will likely need to manually reproduce much of what it does anyway in order to ensure that you don't loose messages.

@tahasamy

WebSocket is capable of handling higher traffic compared to MQTT.

Well, we haven't spoken about capacity. I've run Mosquitto on a Raspberry Pi 3 along side Node-RED, InfluxDB, Grafana, Telegraf, and several other things with no issues at all in handling thousands of messages per minute.

I'm currently running all that plus docker (running Java-based Ubiquiti WiFi controller and a lightweight Linux remote desktop) and some other stuff on an old lenovo thinkpad laptop running headless Debian. An i5 with 8GB RAM and a 512GB spinny rust hard drive. Set to constant low-power mode so that the fans never come on. Here is some idea of the current throughput:

So it might be interesting for you to do some rough calculations as to what throughput you are expecting.

Here is a sample of the 1minute received message load - you can see it hits 3 million messages.

Sheesh, I really need to clear out the cruft from my home automation system! Just bear in mind that I have a LOAD of Telegraf system metrics all of which not only go to InfluxDB but also all go through the broker. That's where a lot of the messages come from.