My Node Red just died

General
1

My Node red has been working fine for quite some time now. But after installing an MQTT broker in NR and trying to do something with it Node Red crashed and I can't get it to start again.

https://flows.nodered.org/node/node-red-contrib-mqtt-broker

When I manually start Node Red in the Terminal I get the messages below. How can I bring my NR back to life?

pi@RENDERFARM-MONITOR:~ $ node-red-start

Start Node-RED
 
Once Node-RED has started, point a browser at http://192.168.1.179:1880
On Pi Node-RED works better with the Firefox or Chrome browser
 
Use   node-red-stop                          to stop Node-RED
Use   node-red-start                         to start Node-RED again
Use   node-red-log                           to view the recent log output
Use   sudo systemctl enable nodered.service  to autostart Node-RED at every boot
Use   sudo systemctl disable nodered.service to disable autostart on boot
 
To find more nodes and example flows - go to http://flows.nodered.org
 
Starting as a systemd service.
19 Oct 17:49:43 - [info]
Welcome to Node-RED
===================
19 Oct 17:49:43 - [info] Node-RED version: v2.0.6
19 Oct 17:49:43 - [info] Node.js  version: v14.17.6
19 Oct 17:49:43 - [info] Linux 5.10.60-v7+ arm LE
19 Oct 17:49:44 - [info] Loading palette nodes
19 Oct 17:49:48 - [info] Dashboard version 3.0.4 started at /ui
19 Oct 17:49:48 - [info] Settings file  : /home/pi/.node-red/settings.js
19 Oct 17:49:48 - [info] Context store  : 'default' [module=memory]
19 Oct 17:49:48 - [info] User directory : /home/pi/.node-red
19 Oct 17:49:48 - [warn] Projects disabled : editorTheme.projects.enabled=false
19 Oct 17:49:48 - [info] Flows file     : /home/pi/.node-red/flows.json
19 Oct 17:49:49 - [info] Server now running at http://127.0.0.1:1880/
19 Oct 17:49:49 - [warn]
---------------------------------------------------------------------
Your flow credentials file is encrypted using a system-generated key.
If the system-generated key is lost for any reason, your credentials
file will not be recoverable, you will have to delete it and re-enter
your credentials.
You should set your own key using the 'credentialSecret' option in
your settings file. Node-RED will then re-encrypt your credentials
file using your chosen key the next time you deploy a change.
---------------------------------------------------------------------
19 Oct 17:49:49 - [info] Starting flows
19 Oct 17:49:49 - [info] [aedes broker:8b97a1b39788fbd5] Start persistence to MongeDB
19 Oct 17:49:49 - [info] [aedes broker:8b97a1b39788fbd5] Publish output wired. Enable broker publish event messages.
19 Oct 17:49:49 - [info] Started flows
19 Oct 17:49:49 - [error] [aedes broker:8b97a1b39788fbd5] Error: Port 1883 is already in use
19 Oct 17:49:49 - [red] Uncaught Exception:
19 Oct 17:49:49 - MongoParseError: Invalid connection string
    at parseConnectionString (/home/pi/.node-red/node_modules/mongodb/lib/core/uri_parser.js:585:21)
    at connect (/home/pi/.node-red/node_modules/mongodb/lib/operations/connect.js:283:3)
    at /home/pi/.node-red/node_modules/mongodb/lib/mongo_client.js:284:5
    at maybePromise (/home/pi/.node-red/node_modules/mongodb/lib/utils.js:692:3)
    at MongoClient.connect (/home/pi/.node-red/node_modules/mongodb/lib/mongo_client.js:280:10)
    at Function.MongoClient.connect (/home/pi/.node-red/node_modules/mongodb/lib/mongo_client.js:426:22)
    at MongoPersistence._connect (/home/pi/.node-red/node_modules/aedes-persistence-mongodb/persistence.js:62:23)
    at MongoPersistence._setup (/home/pi/.node-red/node_modules/aedes-persistence-mongodb/persistence.js:72:8)
    at processImmediate (internal/timers.js:464:21)
nodered.service: Main process exited, code=exited, status=1/FAILURE
nodered.service: Failed with result 'exit-code'.
nodered.service: Service RestartSec=20s expired, scheduling restart.
nodered.service: Scheduled restart job, restart counter is at 1.
Stopped Node-RED graphical event wiring tool.
Started Node-RED graphical event wiring tool.
19 Oct 17:50:12 - [info]
Welcome to Node-RED
===================
19 Oct 17:50:12 - [info] Node-RED version: v2.0.6
19 Oct 17:50:12 - [info] Node.js  version: v14.17.6
19 Oct 17:50:12 - [info] Linux 5.10.60-v7+ arm LE
19 Oct 17:50:13 - [info] Loading palette nodes
19 Oct 17:50:18 - [info] Dashboard version 3.0.4 started at /ui
19 Oct 17:50:18 - [info] Settings file  : /home/pi/.node-red/settings.js
19 Oct 17:50:18 - [info] Context store  : 'default' [module=memory]
19 Oct 17:50:18 - [info] User directory : /home/pi/.node-red
19 Oct 17:50:18 - [warn] Projects disabled : editorTheme.projects.enabled=false
19 Oct 17:50:18 - [info] Flows file     : /home/pi/.node-red/flows.json
19 Oct 17:50:18 - [info] Server now running at http://127.0.0.1:1880/
19 Oct 17:50:18 - [warn]
---------------------------------------------------------------------
Your flow credentials file is encrypted using a system-generated key.
If the system-generated key is lost for any reason, your credentials
file will not be recoverable, you will have to delete it and re-enter
your credentials.
You should set your own key using the 'credentialSecret' option in
your settings file. Node-RED will then re-encrypt your credentials
file using your chosen key the next time you deploy a change.
---------------------------------------------------------------------
19 Oct 17:50:18 - [info] Starting flows
19 Oct 17:50:19 - [info] [aedes broker:8b97a1b39788fbd5] Start persistence to MongeDB
19 Oct 17:50:19 - [info] [aedes broker:8b97a1b39788fbd5] Publish output wired. Enable broker publish event messages.
19 Oct 17:50:19 - [info] Started flows
19 Oct 17:50:19 - [error] [aedes broker:8b97a1b39788fbd5] Error: Port 1883 is already in use
19 Oct 17:50:19 - [red] Uncaught Exception:
19 Oct 17:50:19 - MongoParseError: Invalid connection string
    at parseConnectionString (/home/pi/.node-red/node_modules/mongodb/lib/core/uri_parser.js:585:21)
    at connect (/home/pi/.node-red/node_modules/mongodb/lib/operations/connect.js:283:3)
    at /home/pi/.node-red/node_modules/mongodb/lib/mongo_client.js:284:5
    at maybePromise (/home/pi/.node-red/node_modules/mongodb/lib/utils.js:692:3)
    at MongoClient.connect (/home/pi/.node-red/node_modules/mongodb/lib/mongo_client.js:280:10)
    at Function.MongoClient.connect (/home/pi/.node-red/node_modules/mongodb/lib/mongo_client.js:426:22)
    at MongoPersistence._connect (/home/pi/.node-red/node_modules/aedes-persistence-mongodb/persistence.js:62:23)
    at MongoPersistence._setup (/home/pi/.node-red/node_modules/aedes-persistence-mongodb/persistence.js:72:8)
    at processImmediate (internal/timers.js:464:21)
nodered.service: Main process exited, code=exited, status=1/FAILURE
nodered.service: Failed with result 'exit-code'.
nodered.service: Service RestartSec=20s expired, scheduling restart.
nodered.service: Scheduled restart job, restart counter is at 2.
Stopped Node-RED graphical event wiring tool.
Started Node-RED graphical event wiring tool.
19 Oct 17:50:41 - [info]
Welcome to Node-RED
===================
19 Oct 17:50:41 - [info] Node-RED version: v2.0.6
19 Oct 17:50:41 - [info] Node.js  version: v14.17.6
19 Oct 17:50:41 - [info] Linux 5.10.60-v7+ arm LE
19 Oct 17:50:43 - [info] Loading palette nodes
19 Oct 17:50:47 - [info] Dashboard version 3.0.4 started at /ui
19 Oct 17:50:47 - [info] Settings file  : /home/pi/.node-red/settings.js
19 Oct 17:50:47 - [info] Context store  : 'default' [module=memory]
19 Oct 17:50:47 - [info] User directory : /home/pi/.node-red
19 Oct 17:50:47 - [warn] Projects disabled : editorTheme.projects.enabled=false
19 Oct 17:50:47 - [info] Flows file     : /home/pi/.node-red/flows.json
19 Oct 17:50:47 - [info] Server now running at http://127.0.0.1:1880/
19 Oct 17:50:47 - [warn]
---------------------------------------------------------------------
Your flow credentials file is encrypted using a system-generated key.
If the system-generated key is lost for any reason, your credentials
file will not be recoverable, you will have to delete it and re-enter
your credentials.
You should set your own key using the 'credentialSecret' option in
your settings file. Node-RED will then re-encrypt your credentials
file using your chosen key the next time you deploy a change.
---------------------------------------------------------------------
19 Oct 17:50:47 - [info] Starting flows
19 Oct 17:50:48 - [info] [aedes broker:8b97a1b39788fbd5] Start persistence to MongeDB
19 Oct 17:50:48 - [info] [aedes broker:8b97a1b39788fbd5] Publish output wired. Enable broker publish event messages.
19 Oct 17:50:48 - [info] Started flows
19 Oct 17:50:48 - [error] [aedes broker:8b97a1b39788fbd5] Error: Port 1883 is already in use
19 Oct 17:50:48 - [red] Uncaught Exception:
19 Oct 17:50:48 - MongoParseError: Invalid connection string
    at parseConnectionString (/home/pi/.node-red/node_modules/mongodb/lib/core/uri_parser.js:585:21)
    at connect (/home/pi/.node-red/node_modules/mongodb/lib/operations/connect.js:283:3)
    at /home/pi/.node-red/node_modules/mongodb/lib/mongo_client.js:284:5
    at maybePromise (/home/pi/.node-red/node_modules/mongodb/lib/utils.js:692:3)
    at MongoClient.connect (/home/pi/.node-red/node_modules/mongodb/lib/mongo_client.js:280:10)
    at Function.MongoClient.connect (/home/pi/.node-red/node_modules/mongodb/lib/mongo_client.js:426:22)
    at MongoPersistence._connect (/home/pi/.node-red/node_modules/aedes-persistence-mongodb/persistence.js:62:23)
    at MongoPersistence._setup (/home/pi/.node-red/node_modules/aedes-persistence-mongodb/persistence.js:72:8)
    at processImmediate (internal/timers.js:464:21)
nodered.service: Main process exited, code=exited, status=1/FAILURE
nodered.service: Failed with result 'exit-code'.
nodered.service: Service RestartSec=20s expired, scheduling restart.
nodered.service: Scheduled restart job, restart counter is at 3.
Stopped Node-RED graphical event wiring tool.
Started Node-RED graphical event wiring tool.
19 Oct 17:51:11 - [info]
Welcome to Node-RED
===================
19 Oct 17:51:11 - [info] Node-RED version: v2.0.6
19 Oct 17:51:11 - [info] Node.js  version: v14.17.6
19 Oct 17:51:11 - [info] Linux 5.10.60-v7+ arm LE
^A19 Oct 17:51:12 - [info] Loading palette nodes
2

First, stop the node-red service with node-red-stop.

Then you can start it manually in safe mode - this starts Node-RED without starting the flows running:

node-red --safe

You should be able to access the editor and fix the mqtt broker configuration. When you next click deploy the flows will be started with the updated configuration.

If all is good, stop node-red using ctrl-c in the terminal and restart the service with node-red-start.

3

Not sure why you have installed that module as it was retired 18 months ago and no longer maintained. Especially as you already have node-red-contrib-aedes installed, which is also a MQTT broker.

You will note that in your log;
19 Oct 17:49:49 - [error] [aedes broker:8b97a1b39788fbd5] Error: Port 1883 is already in use which suggests that aedes cannot access port 1883 because node-red-contrib-mqtt-broker is probably using the same port.

I would remove node-red-contrib-mqtt-broker, and either use aedes as your broker, or remove aedes before installing another broker.

1 Like
4

Thank you both, I used the Safe Start mode and Removed the Old Broker. And now I'm back on track.
Reason why I was experimenting with the old broker is because of the next:
I'm running a Python script that sends data to NR using the "paho.mqtt.client", this works fine. However I'm trying to also read data that is generated by NR in Python. I couldn't get that to work with the MQTT node that I already had installed on NR, so I tried the "node-red-contrib-aedes" that brought me to this post.

So if any of you can give me a tip on how to get data from NR to Python using the "paho.mqtt.client" I would be very grateful.
Because sending Data from Python to NR works just fine with paho.

5

Generally it is recommended to use the Mosquitto broker rather than the aedes node. Then it runs as a completely separate process. Mosquitto is very easy to install and configure and has features that are missing from aedes.

Whichever broker you use then I suggest using MQTT Explorer to connect to the broker and check that the communication between python and the broker is working ok. Once that is ok then the standard MQTT nodes should connect to the broker with no problems.

6

I have the Mosquitto Broker installed on a RPI. The Python script is running on a seperate PC. I'm using the "MQTT in" node to read the data coming from my Python script. That works fine. I tried to use the "MQTT out" to send my NR data to Python. But that doesn't seem to work.

As a test I used the "inject" node and had it send data every sec to the "MQTT out" node. That data I could read with Python. But as soon as I feed my msg.payload coming from a function. I couldn't read the msg.payload with Python. It seems that the msg.payload has passed and that Python couldn't fetch it for some reason.

NR_mqtt
NR_mqtt988Ă—401 42.9 KB

7

This is the Python code I'm using. I'm trying to get the "NR/AVG_GPU_TEMP" data ( all the way at the bottom of the code) from NR into Python.

BTW. I'm not a programmer I can help myself so I'm probably doing something wrong. I just don't know if it is wrong at the side of Python or at the side of NR.

import paho.mqtt.client as mqtt
import time

def on_log(client, userdata, level, buf):
    print("log: "+buf)


def on_connect(client, userdata, flags, rc):
    if rc==0:
        print("MQTT Connected OK")
    else:
        print("Bad MQTT Connection, ReturnCode: ",rc)

def on_disconnect(client, userdata, flags, rc=0):
    print("Disconnected, ReturnCode: "+str(rc))


def on_message(client, userdata,msg):
    topic=msg.topic
    global m_decode
    m_decode=str(msg.payload.decode("utf-8","ignore"))
    print("Message Received: ",m_decode)


def get_mqtt_message(topic):
    broker="renderfarm-monitor.local"
    client =mqtt.Client("CHECK_RENDER")

    client.on_connect=on_connect
    client.on_disconnect=on_disconnect
    #client.on_log=on_log
    client.on_message=on_message
    msg=on_message

    print("RX: Connecting to broker ",broker)
    try:
        client.connect(broker)
        client.loop_start()
        client.subscribe(topic)
        time.sleep(0.5)
        client.loop_stop()
        client.disconnect()
    except:
        print("RX: "+time.ctime(time.time())+" // Failed to connect to MQTT server! ")

def send_mqtt_message(topic, send_msg):
    broker="renderfarm-monitor.local"
    client =mqtt.Client("CHECK_RENDER")

    client.on_connect=on_connect
    client.on_disconnect=on_disconnect
    #client.on_log=on_log
    client.on_message=on_message
    msg=on_message

    print("TX: Connecting to broker ",broker)
    try:
        client.connect(broker)
        client.loop_start()
        #client.publish("c3d_pc/renderfarm/message","Hallo wij zijn begonnen...")
        client.publish(topic,send_msg)
        time.sleep(0.5)
        client.loop_stop()
        client.disconnect()
    except:
        print("TX: "+time.ctime(time.time())+" // Failed to connect to MQTT server! ")


 ######  ########    ###    ########  ########
##    ##    ##      ## ##   ##     ##    ##
##          ##     ##   ##  ##     ##    ##
 ######     ##    ##     ## ########     ##
      ##    ##    ######### ##   ##      ##
##    ##    ##    ##     ## ##    ##     ##
 ######     ##    ##     ## ##     ##    ##


get_mqtt_message("esp/renderfarm/temperature2")
gpu_temp= str(float(str(float(m_decode)*10).split('.')[0])/10)

send_msg="Last Update: "+time.ctime(time.time())+" _____ AV GPU TEMP: "+gpu_temp+"°C"
topic="check_render/timestamp"
send_mqtt_message(topic,send_msg)

# Test Table
send_msg='''[
{"SHOT":"HOUDINI_SHOT_01_HOR_ALL-LAYERS_GRAG","RANGE":"(1..10)","PROG":100,"FRAMES":"< COMPLETE >"},
{"SHOT":"HOUDINI_SHOT_02_HOR_ALL-LAYERS_GRAG-B","RANGE":"(1..10)","PROG":100,"FRAMES":"< COMPLETE >"},
{"SHOT":"HOUDINI_SHOT_01_VER_ALL-LAYERS_GRAG","RANGE":"(1..10)","PROG":100,"FRAMES":"< COMPLETE >"},
{"SHOT":"HOUDINI_SHOT_02_VER_ALL-LAYERS_GRAG-B","RANGE":"(1..10)","PROG":100,"FRAMES":"< COMPLETE >"},
{"SHOT":"HOUDINI_SHOT_01_SQR_ALL-LAYERS_GRAG","RANGE":"(1..10)","PROG":100,"FRAMES":"< COMPLETE >"},
{"SHOT":"HOUDINI_SHOT_02_SQR_ALL-LAYERS_GRAG-B","RANGE":"(1..10)","PROG":100,"FRAMES":"< COMPLETE >"}
]'''
topic="check_render/table"
send_mqtt_message(topic,send_msg)

print("*"*50)
get_mqtt_message("NR/AVG_GPU_TEMP")
print(">>>>>>>>>>>>>>>>> "+m_decode)
print("*"*50)
8

I don’t see a mqtt-out node in that image of your flow….
What are you using as the topic in the msg sent to the broker?
Where do you define, in the python code, the topic you are subscribing to?

9

If it works with an inject node but not with a function node, both feeding the same MQTT node then the message from the function must be different to that from the inject. Feed them both into a debug node and compare them carefully. If you have not specified a topic in the MQTT node then check the topics in the two messages are both the same, as well as the payload.

10

Oh Sorry the MQTT out was on another page... here it is.

NR_mqtt_2
NR_mqtt_21888Ă—473 76.6 KB

11

the msg.payload is just a number for both the inject node as from the function node. The only difference is that the function node generates a msg.payload every 10 sec while the inject node generates a msg.payload every sec.
If I set the Repeat interval of the inject node to 10 sec. I also can't read the data. So it has something to do with messages getting lost before Python can read them.

12

In your image you show you are connection to the locahost:1883 broker
in your python code are connecting to the renderfarm-monitor.local broker

Are they the same broker?

and I'll ask once again, in the python code, where are you defining the topic used to publish a message

and what is with the big START in the code?

13

The locahost:1883 and the renderfarm-monitor.local are one and the same.

The big START is just a visual indicator I use in Sublime Text. When the code becomes very long I can see very clearly in the overview to the right of the Sublime UI ( the zoomed out version of the script ) where my main program starts. I always use this big text technique to place markers in my script. Just a visual helper to clearly distinguish the important parts of my script and make them still readable in the zoomed out version of the script to the right of the Sublime UI.

This is how I use the Topic for receiving data from NR, the topic here is NR/AVG_GPU_TEMP

def get_mqtt_message(topic):
    client.subscribe(topic)

get_mqtt_message("NR/AVG_GPU_TEMP")

And this is how I use the Topic for sending data to NR, the topic here is check_render/table and the message is BlaBlaBLa

def send_mqtt_message(topic, send_msg):
    client.publish(topic,send_msg)

send_msg="BlaBlaBLa"
topic="check_render/table"
send_mqtt_message(topic,send_msg)
14

That is very clever! I don't know why but I missed that it was all in # so would be a big comment, very clever indeed. I'm going to add that to my toolbar.

Could you try an experiment and change the mqtt nodes to use renderfarm-monitor.local instead of locahost:1883 and see what happens.

15

In this situation I would install MQTT Explorer on a machine that can access the broker to check whether the problem is in node-red or in Python.

Edit: I just noticed that I already suggested this. So how did that experiment work out @CarloJongen

16

Changing the locahost:1883 to renderfarm-monitor.local doesn't make any difference. Since they are the same I was sort of expecting that.

Just an overview of what I'm trying to accomplish:

  • I have an ESP8266 reading 3 temp sensors ( DS18B20 ) and one Digital Input
  • The ESP8266 also drives one Digital Output.
  • The ESP8266 sends this data every 10sec to NR.
  • In NR I can read those 3 temperatures and the status of the Digital input.
  • In NR I can also trigger the Digital Output of the ESP8266
  • In NR I calculate the average temp from those 3 temperatures in a Junction Node and I can display this on the NR UI.

But now I would like to take the value of the average temp and send that value over to the Python script. And that is the part that doesn't work.

What I don't understand is how does the MQTT Out Node in NR know when the Python script asks for the msg.payload? What triggers the input of the MQTT Out Node to send the payload over to the MQTT broker. I still believe I'm doing something wrong here.

17

It doesn't.

It does that when you send the Out node a message

The broker then passes it on any clients subscribed to that topic. Node-red doesn't know anything about that.

This excellent tutorial will tell you pretty much all you need to know about MQTT. Well worth 30 minutes looking through. MQTT Essentials - All Core Concepts Explained

1 Like
18

That is what I thought... Thanks for the link I will look into this.

But meanwhile, if I publish a message to a Topic. How long will the MQTT Broker keep that message? Until somebody request for it? Or is there also a timeout for the MQTT Broker for it to drop that message?
Because:

  • Sending a message every sec to the MQTT Out node works.
  • Sending a message every 10 sec to the MQTT Out node doesn't work.
    NR_mqtt_3
19

You could probably have answered that yourself by now by reading the tutorial. If the topic is not a Retained topic then, when a messages is published, the broker immediately circulates it to all subscribed clients. It then discards the message. If the topic is a Retained topic then it will keep it indefinitely and send it whenever a client subscribes to it. If using Mosquitto then the default is that retained topics are saved in the file system so they will remain known over a restart so clients will get the previous value immediately they subscribe. Now please work through the tutorial.

1 Like
20

As suggested earlier, please use something like MQTT Explorer to see if that is correct. I think it more likely that the problem is at the Python end.