Import data from sensor into node-red

[snax]

Hey people!

I’m doing a uni assignment on Iot, and we are doing a soil moisture system with a water pump. One stipulation is that we need to use Node-Red.

List of components

*MCP3008 8-channel ADC
Raspberry Pi 4

*Dfrobot Gravity: Analog Waterproof Capacitive Soil Moisture Sensor

*Piico Dev oLED

*Piico Dev adapter for raspberry pi

*Dfrobot: Amphibious Horizontal Submersible Pump

*Using Mac and controlling raspberry pi through the terminal.

Okay. Im a full rookie here… Apart from connecting a dht22 humidity this is my first project…. Here’s where I’m at:

Installed MCP3XXX library and done this…. sudo pip3 install adafruit-circuitpython-mcp3xxx As per everything on this thread Python & CircuitPython | MCP3008 - 8-Channel 10-Bit ADC With SPI Interface | Adafruit Learning System

Set up a mosquito mqtt broker and
node red is connected (check screen shots)`

Heres the code that has been stored
in ` “/home/pi/./code.py”

# SPDX-FileCopyrightText: 2021 ladyada for Adafruit Industries
# SPDX-License-Identifier: MIT

import busio
import digitalio
import board
import adafruit_mcp3xxx.mcp3008 as MCP
from adafruit_mcp3xxx.analog_in import AnalogIn

# create the spi bus
spi = busio.SPI(clock=board.SCK, MISO=board.MISO, MOSI=board.MOSI)

# create the cs (chip select)
cs = digitalio.DigitalInOut(board.D5)

# create the mcp object
mcp = MCP.MCP3008(spi, cs)

# create an analog input channel on pin 0
chan = AnalogIn(mcp, MCP.P0)

print("Raw ADC Value: ", chan.value)
print("ADC Voltage: " + str(chan.voltage) + "V")

Now I run this
code: ` python3 ./code.py and I get data coming in from the sensor. And it does change when I put the sensor in water… so that seems to be working (Look at screenshot).

So what I need is to import this into node-red I’m assuming we need to use an “mqtt in” node, but I just can’t figure it out… I’ve watched probably 40 different YouTube videos, and it looks so easy but nothing seems to work.

When I use a “mqtt in”node I get nothing coming through
the debugger… It shows it is connect but no error msg or data. (check
screenshots) `

Is there a
different way to import the data coming in from the raspberry pi, or is there
something I’m missing?`

I think I will need to calibrate the sensor and somehow set the parameters to interpret the data as come sort of high, medium, low soil moisture level. Not sure if i can do this in Node-Red or do I have to do this in Terminal.

And I haven’t even
attempted to connect the water pump and oLED ,to display if the soil is wet or
not, yet. So many more questions will
probably be on the way haha`

Can some legend out
there same me!!`

[jbudd]

Hello @snax and welcome to the forum.

@snax

I’m assuming we need to use an “mqtt in” node

Why are you assuming that? MQTT may indeed be a good way to exchange data, it may not be the best or the only way. Probably how the different components are connected is relevant to the choice of protocol.

@snax

Is there a different way to import the data coming in from the raspberry pi, or is there
something I’m missing?`

Have you tested that you can successfully publish and subscribe to MQTT from your Node-red setup? Very simple to do.

Have you tested that you can publish to MQTT from a Python script on your Node-red machine?
How does this work? There are plenty of online tutorials for using MQTT from Python.

Have you tested that you can publish to MQTT from a Python script on your Pico?

[Colin]

I don't see where in the python script you are publishing the data to MQTT for node-red to pick up.

[snax]

Hey jbudd,

Thanks for helping out!!

I was assuming because I have no idea how to do it properly. And i've so many different tutorials and the mqtt hasnt worked.

@jbudd

Have you tested that you can successfully publish and subscribe to MQTT from your Node-red setup? Very simple to do.

yep i've done that works fine.

@jbudd

Have you tested that you can publish to MQTT from a Python script on your Node-red machine?
How does this work? There are plenty of online tutorials for using MQTT from Python.

Have you tested that you can publish to MQTT from a Python script on your Pico?

I have not tried that. I don't know python, is there an easy tutorial that you recommend?

[snax]

Hey Colin,

Ohh I didn't know I had to, I'm a full rookie!! Do you know how I could do that?

Thanks for replying so quick!!

[bakman2]

You will need to import the mqtt library for circuitpython and connect to the broker
Example script from adafruit.

[Colin]

To understand how MQTT works then see this tutorial MQTT Essentials - All Core Concepts explained

[snax]

Thanks for the link, its very informative! I had a good look over the documents, but i'm not sure how I will implement the correct protocols when I don't know how to write the code..

[zenofmud]

Try a google search with ‘python and MQTT tutorial’ and see what you find

[manglemender]

@snax Firstly good luck with your project.

I did something similar for my greenhouse last year. I used an ESP32 in the greenhouse to interface 4 moisture sensors and publish the data to my MQTT server over a wireless network. NodeRed reads the raw values from MQTT and a function node then applies the calibration to give moisture readings of 0-100%.

For calibration: each sensor has its own "wet" and "dry" value stored in a csv file and read in to flow variables on startup. For each sensor note down the raw value when it is fully dry and fully wet and use these values to calculate the 0-100% values.

My advice is to break the task down to small chuncks:
Read the sensors
publish to MQTT
Read from MQTT
Scale the output
Display the output

Start with just one sensor and get that working then think about adding more sensors.
After that you can add more refinements such as a calibration interface to edit and store calibration values.

[snax]

thanks @manglemender!! much help but I just couldn't figure it out.. i've converted to wrting a json file and trying to read that from node-red through a http response.. seems to be just as difficult haha.. i think i might just lay down the battle tools and admit defeat. Thanks everyone for helping!!! really appreciate it

[bakman2]

You will need to connect to the mqtt broker as part of your python code. You will not progress without it.

[jbudd]

@snax

i think i might just lay down the battle tools and admit defeat.

Oh dear, there goes your 1st class honours!

@snax

i've converted to wrting a json file and trying to read that from node-red through a http response.. seems to be just as difficult haha..

While http may be more familiar, MQTT is a much better choice for an IOT application.
Once you get used to it, it's much simpler and more elegant than http (in my opinion).
I think someone suggested you look at a tutorial on Adafruit's website, did you try that? Usually Adafruit's tutorials are well written and reliable.

[manglemender]

@jbudd

MQTT is a much better choice for an IOT application

100% agreed.
Steve's internet guide is always a good learning resource: Using The Mosquitto_pub and Mosquitto_sub MQTT Client Tools- Examples

[snax]

here's an updated code that has a mqtt broker and it looks like it should all be working... but i still cant access the data in node-red.

I tried changing the port number to 1886 because, according to chatgpt, there might have been an issue with conflicting port usage. But now the mqtt wont connect in node-red and I get this error in my terminal window but not it the debug in node-red..

26 Sep 23:14:12 - [info] [mqtt-broker:d1b995653bfca26d] Connection failed to broker: mqtt://192.168.0.100:1886

import os
import time
import paho.mqtt.client as mqtt
import busio
import digitalio
import board
import adafruit_mcp3xxx.mcp3008 as MCP
from adafruit_mcp3xxx.analog_in import AnalogIn

# create the spi bus
spi = busio.SPI(clock=board.SCK, MISO=board.MISO, MOSI=board.MOSI)

# create the cs (chip select)
cs = digitalio.DigitalInOut(board.D22)

# create the mcp object
mcp = MCP.MCP3008(spi, cs)

# create an analog input channel on pin 0
chanel = AnalogIn(mcp, MCP.P0)

# Set up MQTT client
mqtt_client = mqtt.Client()
mqtt_client.connect("192.168.0.100", 1886)  # Use the correct port (usually 1883)

# Start the MQTT client's loop
mqtt_client.loop_start()

while True:
    # Read the soil moisture value
    soil_moisture = chanel.value  # You should replace this with your actual value
    # Print the value of soil_moisture
    print('Soil Moisture:', soil_moisture)  
    print('Raw ADC Value: ', soil_moisture)
    print('ADC Voltage: ' + str(chanel.voltage) + 'V')

    # Publish the soil moisture value to an MQTT topic
    mqtt_client.publish("soil_moisture", soil_moisture)

    time.sleep(0.5)

[jbudd]

What MQTT broker are you using - Mosquitto?
Have you set it up to allow connections from remote devices?

This might be relevant

[bakman2]

I tried changing the port number to 1886 because, according to chatgpt,

But chatgpt talks nonsense as usual. In the screenshot in your initial post, your broker lives on port 1883. Change it back to 1883 in both node-red and in the python code.
Check what @jbudd is pointing out, in that particular article the related item is this:

allow_anonymous true

By default this is turned off (false). Set it to true and restart the mqtt broker. Restart device.

[Colin]

As well as
allow_anonymous true
you need
listener 1883

[dynamicdave]

Did you ever try the very basic check of your local MQTT broker (as suggested by @jbudd above) ?
You need to make sure that is working.

[{"id":"0e90d9248cf5a9ce","type":"tab","label":"Basic MQTT check","disabled":false,"info":"","env":[]},{"id":"82cbe9a3221a369b","type":"inject","z":"0e90d9248cf5a9ce","name":"","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"Hello world","payloadType":"str","x":260,"y":200,"wires":[["fb0ee3e59c1f0826"]]},{"id":"f90c59804fd08621","type":"debug","z":"0e90d9248cf5a9ce","name":"debug 3363","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":470,"y":260,"wires":[]},{"id":"2370f7c230b79c2f","type":"mqtt in","z":"0e90d9248cf5a9ce","name":"","topic":"test","qos":"2","datatype":"auto-detect","broker":"846a8c23e24a78db","nl":false,"rap":true,"rh":0,"inputs":0,"x":230,"y":260,"wires":[["f90c59804fd08621"]]},{"id":"fb0ee3e59c1f0826","type":"mqtt out","z":"0e90d9248cf5a9ce","name":"Local MQTT broker","topic":"test","qos":"","retain":"","respTopic":"","contentType":"","userProps":"","correl":"","expiry":"","broker":"846a8c23e24a78db","x":470,"y":200,"wires":[]},{"id":"cbb6c83fa987e42b","type":"comment","z":"0e90d9248cf5a9ce","name":"A very basic test to check your local MQTT broker","info":"","x":370,"y":140,"wires":[]},{"id":"846a8c23e24a78db","type":"mqtt-broker","name":"","broker":"127.0.0.1","port":"1883","clientid":"","autoConnect":true,"usetls":false,"protocolVersion":"4","keepalive":"60","cleansession":true,"autoUnsubscribe":true,"birthTopic":"","birthQos":"0","birthRetain":"false","birthPayload":"","birthMsg":{},"closeTopic":"","closeQos":"0","closeRetain":"false","closePayload":"","closeMsg":{},"willTopic":"","willQos":"0","willRetain":"false","willPayload":"","willMsg":{},"userProps":"","sessionExpiry":""}]

[Colin]

Assuming that works then change the server address in the mqtt config in node-red to use the IP of the machine running the broker, instead of localhost, that will prove the broker can be accessed from other devices.