Python Script into .JSON file for node red?

Hello there!

I wrote a Scipt in python to read data from a sensor which is connected via usb to my raspberry pi 4B.

Is it possible to convert this skript into a .json file and import it into node red to get the readed data into the debug node or into the dashboard ?

Thanks.

What is the sensor?
Have you looked to see if there is a node that uses that sensor?
You could use the exec node to run the script but how does it report the data?

the sensor is the sds011, I once tried to use the available node but it didn't worked. You told me an another topic I made that the author for this node doesn't update it for almost 3 years and thats why the node doenst work, it seems to be abonded. So i wrote a script to read it and now i want to import it to node red, if possible.

The script i wrote shows the data in an array, but I can change that into the form it must have.

If I may suggest, alternative solutions with various degree of work to do

  1. Add MQTT to the script and deliver the array as a message via MQTT to NR. Very easy if you have enough experience in Python. Controol the execution of the script from NR; use an exec node to start it, add a heartbeat function as well to the script, monitor it and if needed restart it. Also not too complicated to achieve

  2. Re-write the logic in javascript and put into a function node but I do not know if there are libraries available to also read the sensor readings directly from javascript or if you still have to have some python script in between (if so, I would stick to alternative 1) ) EDIT: I see now, it is just serial so it should be ppossible to use the serial node in NR

I forgot to say that the whole system must work without any internet connection.
I am not really a professional, more an absolute beginner. does mqtt requiere an internet connection?

Not at all, you can run it locally

Have you tried the serial node and checked if you can receive the raw data directly from the sensor?
Bit rate :9600
Data bit :8
Parity bit:NO
Stop bit :1

I tried the "serial in" node and the "serial request" node, both are connected but in the debug window appears nothing, even when i switch to "full report".

Screenshot 2021-05-12 134605

Because I#m a beginner, can you tell me in detail how i add he MQTT & exec Node and how i set them up?

Thank you very much. :slight_smile:

Sure, share your Python script here, I will modify it and also make a simple flow that monitors it via heartbeat function. If you have any personal credentials in the Python script, please remove them first

Thank you !

import array
import sys
import usb.core
import usb.util

VID =0x1a86
PID = 0x7523
DATA_SIZE = 4

device = usb.core.find(idVendor = VID, idProduct = PID)
if device is None:
sys.exit("Could not find sds011 Sensor.")

if device.is_kernel_driver_active(0):
try:
device.detach_kernel_driver(0)
except usb.core.USBError as e:
sys.exit("Could not detatch kernel driver: %s" % str(e))

try:
device.reset()
device.set_configuration()
except usb.core.USBError as e:
sys.exit("Could not set configuration: %s" % str(e))

endpoint = device[0][(0,0)][0]
print(endpoint.wMaxPacketSize)

data = array.array('B',(0,)*4)
while data[0] != 3:
try:
data = device.read(endpoint.bEndpointAddress, endpoint.wMaxPacketSize)
print(data)

except usb.core.USBError as e:
    if e.args == ('Operation timed out',):
        print("timeout)")
        continue

print("work!")

I have to say that i write it with the help of a friend, so i don't really know how it works :smiley:

No problem, I assume you run it in a Raspberry Pi or at least debian/ubuntu system?

You can start to install the mosquitto MQTT broker. See here. It also explains a bit about MQTT

I installed mosquitto but when i try to test it, like on the website shown, I get the following.

Screenshot 2021-05-12 142612

ok Now i got it ! MQTT & Mosquitto is installed succesfully. it also passed the test.

Perfect! Give me some few minutes more, soon you have something to test :wink:

So below is an example to try, hope it does, I could not test the actual usb communication :wink:

First you have to install the paho mqtt client for Python3. Like this

sudo pip3 install paho-mqtt

Next put the script in your directory (/home/pi) and change extension from .txt to .py

Import the flow to your Node-RED, deploy

Start the script via the "Start script" button. I made it so the exec nodes runs the script visible on your monitor screen or vnc view. If this does not work we will get an error message

Abort the script via the "Abort script" button

Heartbeat checks will start automatically and is sent once per minute

Hopefully it works, I moved the usb communication to a separate thread

You should now see data coming into NR in the debug view

[{"id":"76342fda.61ba7","type":"exec","z":"44db0721.1c8138","command":"export DISPLAY=:0 && lxterminal -e python3 /home/pi/sds011.py","addpay":false,"append":"","useSpawn":"false","timer":"","oldrc":false,"name":"","x":570,"y":180,"wires":[[],[],[]]},{"id":"cb8849f3.036878","type":"inject","z":"44db0721.1c8138","name":"Start script","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"true","payloadType":"bool","x":160,"y":180,"wires":[["76342fda.61ba7"]]},{"id":"260e8f38.3f615","type":"mqtt out","z":"44db0721.1c8138","name":"","topic":"sds011/abort","qos":"","retain":"","respTopic":"","contentType":"","userProps":"","correl":"","expiry":"","broker":"397bfe63.7d8242","x":400,"y":510,"wires":[]},{"id":"3fb86da5.3e02e2","type":"inject","z":"44db0721.1c8138","name":"Abort script","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"abort","payloadType":"str","x":170,"y":510,"wires":[["260e8f38.3f615"]]},{"id":"634dd6c6.21d108","type":"inject","z":"44db0721.1c8138","name":"Heartbeat check","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"60","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"Are you alive?","payloadType":"str","x":190,"y":450,"wires":[["2dbd1fb6.009ef"]]},{"id":"2dbd1fb6.009ef","type":"mqtt out","z":"44db0721.1c8138","name":"","topic":"sds011/heartbeat_check","qos":"","retain":"","respTopic":"","contentType":"","userProps":"","correl":"","expiry":"","broker":"397bfe63.7d8242","x":440,"y":450,"wires":[]},{"id":"7f2b6c64.37d954","type":"mqtt in","z":"44db0721.1c8138","name":"","topic":"sds011/heartbeat_response","qos":"2","datatype":"auto","broker":"397bfe63.7d8242","nl":false,"rap":false,"x":220,"y":390,"wires":[["ad4c3c00.0776b8","38edfbd5.dc9ca4"]]},{"id":"ad4c3c00.0776b8","type":"debug","z":"44db0721.1c8138","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":810,"y":390,"wires":[]},{"id":"38edfbd5.dc9ca4","type":"trigger","z":"44db0721.1c8138","name":"","op1":"","op2":"Script is restarting","op1type":"nul","op2type":"str","duration":"5","extend":true,"overrideDelay":false,"units":"min","reset":"","bytopic":"all","topic":"topic","outputs":1,"x":600,"y":260,"wires":[["76342fda.61ba7","ad4c3c00.0776b8","cf3eecb5.6f8d9"]]},{"id":"cf3eecb5.6f8d9","type":"switch","z":"44db0721.1c8138","name":"","property":"payload","propertyType":"msg","rules":[{"t":"nempty"}],"checkall":"true","repair":false,"outputs":1,"x":600,"y":330,"wires":[["38edfbd5.dc9ca4"]]},{"id":"37b069c1.c685d6","type":"debug","z":"44db0721.1c8138","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":400,"y":600,"wires":[]},{"id":"84ee7b81.3236d8","type":"mqtt in","z":"44db0721.1c8138","name":"","topic":"sds011/data","qos":"2","datatype":"auto","broker":"397bfe63.7d8242","nl":false,"rap":true,"rh":0,"x":170,"y":600,"wires":[["37b069c1.c685d6"]]},{"id":"397bfe63.7d8242","type":"mqtt-broker","name":"","broker":"127.0.0.1","port":"1883","clientid":"","usetls":false,"protocolVersion":"4","keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthPayload":"","birthMsg":{},"closeTopic":"","closeQos":"0","closePayload":"","closeMsg":{},"willTopic":"","willQos":"0","willPayload":"","willMsg":{},"sessionExpiry":""}]

sds011.txt (2.3 KB)

1 Like

Just updated the script, it did not abort fully correctly

First of all, I don't know how to thank you for all this, this seems to be a lot of work you've done for me :smiley:

I installed paho mqtt like you said and reboot everything.
The Script is already in .py, when i start it in the console the following appears.

Than I imported the flow, but when i press start nothing appears in the debug window. Everything shows "connected" and when i press start, under the exec node appears for a second "pid: 1264" with a little blue dot infront of it.
When i press start through the vnc connection, the console is opening for a sec., I think thats because the script starts.
so i think everything you've done is correct but something is wrong with my script :smiley:
Or did I anything wrong ?

You can also consider launching the python script through node-red-node-daemon. In that case you must assure that your python script is outputting the sensor data to standard output (and not to a specific file).

No problem, my pleasure!

I see an error message related to python 2.7

When you did try your original script, how did you start it? With python or python3?

You could maybe try to run it also directly from a command prompt like python3 /home/pi/sds011.py or python /home/pi/sds011.py so we can see what is happening

But to make it work with python 2.7 you have to install paho mqtt client again but this time sudo pip install paho-mqtt

EDIT: Maybe we are hitting the usb port too hard, we can eventually put in a delay between the readings. How frequent do we have to read it?

EDIT AGAIN: I see now you started the script incorrectly, you should try it with sudo python3 /home/pi/sds011.py
If that works you have to edit the command in the exec node to add sudo in front