TL;DR: It worked last year, updating to newest versions of NR and MQTT broke it. Many UI elements were particularly affected. Older version still works but no MQTT.
OLD Versions / (NEW Versions are the latest provided via NR and latest updates)
Pi: Buster Lite / Buster Lite 5.10
NR: v1.0.6 (from browser
Node Js: 12.16.3-1nodesource1
MQTT: 1.5.7-1
Software updates can do funny things, they usually work out well. This one didn't (for me).
Background: Last year, and with the great help of reading comments here on the boards, I built the usual RasbPi/ESP8266 weather/pool temps setup and used NR to pull it all together in a dashboard. It worked great, I added as the summer went on and I was very happy with how it ended up.
Dusting it off this season, I noticed the MQTT portion wasn't working. So I went down the rabbit hole of updates, both with the PiOS and MQTT and realized things were not happening for me. I then figured just to update everything to latest (including NR using the install script), and I was left with no dashboard, nothing working and nothing to even troubleshoot. So many modules were now "dashed" and non-functioning. I was dead. Luckily I saved the old version and re-flashed the Pi as it was last summer. I still have MQTT problems, but I have my external weather sources and weather alerts working. Nothing coming from the ESP8266 (air/water/humid) is making it into the dashboard due to the MQTT issue.
I'm posting this here in the hopes that A) I can get help getting the MQTT issue solved and B) that I can get help in upgrading to the latest versions of things while keeping my project functioning.
Here's the NR info I get from console to get started:
26 May 14:07:52 - [info] Node-RED version: v1.0.6
26 May 14:07:52 - [info] Node.js version: v12.16.3
26 May 14:07:52 - [info] Linux 5.4.42-v7+ arm LE
26 May 14:07:53 - [info] Loading palette nodes
26 May 14:07:56 - [info] Dashboard version 2.21.0 started at /ui
26 May 14:07:56 - [info] Settings file : /home/pi/.node-red/settings.js
26 May 14:07:56 - [info] Context store : 'default' [module=memory]
26 May 14:07:56 - [info] User directory : /home/pi/.node-red
26 May 14:07:56 - [warn] Projects disabled : editorTheme.projects.enabled=false
26 May 14:07:56 - [info] Flows file : /home/pi/.node-red/flows_poolpi.json
26 May 14:07:56 - [error] Unable to listen on http://127.0.0.1:1880/
26 May 14:07:56 - [error] Error: port in use
Port in use. Usually means you already have a version running. You need to stop that first before restarting. It also reports node-red version 1.0.6. Which is not the latest so not sure which you are trying to run.
Thanks for the reply. that output is from the Pi that is running everything and has a fresh install of the old .img I saved from last summer. It runs NR and MQTT server/broker. I have NR restart automatically on bootup. I dont think there are 2 instances running. I know it is the older version of NR, but it works except MQTT.
well something is using port 1880 already so that one in the log isn't running. So you need to find whatever is grabbing port 1880 and stop it... usually it is Node-RED.
Not sure how you are autostarting it - but running node-red-stop may stop it... then restart it.
i did that and got the same output that I posted here. port 1880 jammed up
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:1883 0.0.0.0:* LISTEN 430/mosquitto
tcp 0 0 0.0.0.0:ssh 0.0.0.0:* LISTEN 457/sshd
tcp 0 0 0.0.0.0:1880 0.0.0.0:* LISTEN 314/node-red
tcp6 0 0 [::]:1883 [::]:* LISTEN 430/mosquitto
tcp6 0 0 [::]:ssh [::]:* LISTEN 457/sshd
udp 0 0 0.0.0.0:bootpc 0.0.0.0:* 403/dhcpcd
udp 0 0 0.0.0.0:mdns 0.0.0.0:* 338/avahi-daemon: r
udp 0 0 0.0.0.0:37235 0.0.0.0:* 338/avahi-daemon: r
udp6 0 0 [::]:mdns [::]:* 338/avahi-daemon: r
udp6 0 0 [::]:35051 [::]:* 338/avahi-daemon: r
well that tells you node-red IS already running on 1880... so yes - you can't start it again... Depending how long ago the script was run you may have a node-red-log command which may just show the existing log rather than start a new one.
Every time I stop NR, it restarts. I noticed the log had the date of September on it. So, I don't know what the next steps are. this is the install that was working fine last summer and obviously into fall.
I've run node-red-stop, node-red-log and sudo netstat -lptu to obtain this information
EDIT: I think things must take a real long time to spin up, because thi sis my log file now:
Started Node-RED graphical event wiring tool.
15 Sep 19:17:10 - [info]
Welcome to Node-RED
15 Sep 19:17:10 - [info] Node-RED version: v1.0.6
15 Sep 19:17:10 - [info] Node.js version: v12.16.3
15 Sep 19:17:10 - [info] Linux 5.4.42-v7+ arm LE
15 Sep 19:17:11 - [info] Loading palette nodes
15 Sep 19:17:16 - [info] Dashboard version 2.21.0 started at /ui
15 Sep 19:17:17 - [info] Settings file : /home/pi/.node-red/settings.js
15 Sep 19:17:17 - [info] Context store : 'default' [module=memory]
15 Sep 19:17:17 - [info] User directory : /home/pi/.node-red
15 Sep 19:17:17 - [warn] Projects disabled : editorTheme.projects.enabled=false
15 Sep 19:17:17 - [info] Flows file : /home/pi/.node-red/flows_poolpi.json
15 Sep 19:17:17 - [info] Server now running at http://127.0.0.1:1880/
15 Sep 19:17:17 - [info] Starting flows
15 Sep 19:17:18 - [info] Started flows
15 Sep 19:17:18 - [info] [mqtt-broker:e9732694.8b0b28] Connection failed to broker: mqtt://192.168.1.113:1883
15 Sep 19:17:18 - [error] [http request:UVscrape] Error: getaddrinfo EAI_AGAIN www.theweathernetwork.com
15 Sep 19:17:18 - [error] [openweathermap:Current weather] Error: getaddrinfo EAI_AGAIN api.openweathermap.org
15 Sep 19:17:33 - [info] [mqtt-broker:e9732694.8b0b28] Connected to broker: mqtt://192.168.1.113:1883
Does the Pi actually have access to the internet ? (as the getaddrinfo errors suggest it can't reach those external apis's. Sp maybe its clock is just hopelessly wrong ?
It looks like it has connected to the broker ok though.
It is on my network. I have reserved an IP for it, and I am getting this info via SSH from another computer
That doesn't answer Daves questions though.
One thing I'll note - I've had a few issues with updates to Mosquitto over the last couple of years where they have made ill-judged changes to configuration parameters and defaults (and sometimes changed them back). There have been several occasions where, after an apt update, I belatedly realised that MQTT wasn't working as expected.
I've not had any real issues with Node-RED updates though for quite a long time.
When considering updates, also watch out for node.js major updates because those can require a rebuild of Node-RED and its packages.
The pi is accessing the internet to get weather info, UV readings, NWS stuff. Yes. It is connected to the interwebs.
but the results in the log would indicate that it does not...
either that or your dns server is not working so it can't find the correct addresses.
dont know how else to show you. In my investigation, I think I'm zeroing in on the PubSub thing in the arduino code... will report back
Still back to my initial question: Is it possible to upgrade NR and MQTT to most recent versions without breaking everything? My first go-around proved that it was a complete nightmare and I don't feel like rebuilding everything from scratch. Some things like user-UI stuff are totally broken on the new updates.
There is no such thing as software ALWAYS working and so you can never guarantee that an upgrade won't break something.
You have to make a choice. Lag behind and maybe get caught out by security or other non-compatibility issues and also potentially by difficult, multi-step upgrades. Or stay reasonably up-to-date and suffer occasional heart palpitations while you do some quick footwork.
Experience (a lot of it), shows that keeping things reasonably update is generally much less problematic than leaving things for ages and praying that nothing breaks. The smaller the update, the less likely you are to have problems.
Node-RED is fairly fast moving, it is built on Node.js which is also fast moving. Mosquitto not so fast moving but has had some recent hicups (prior to those, I'd have said that Mosquitto was one of the most stable apps, it is still stable as an app but the config changes were a pain).
Don't stay at the leading edge of change but with tools like this, don't let them lag too far either. I would aim to be updating every month or two. I wouldn't leave it more than 3 months personally and if I did, I'd want to be ready for some upgrade related issues.
Your experience is unusual for Node-RED. I have several instances across several platforms and as I've said, I've not had major problems with upgrades in quite a while.
2 Likes
I read here that other people were having issues with user-ui stuff as well since the project was abandoned and the github erased years ago.
Either way, I got everything running now circa 2020 (including wLED control), and I am running that old stuff until I can slowly port the functionality over to a new NR/MQTT platform.
How was it fixed? Sadly, I have no idea. The only thing I did was update an openweather node and re-enter my API key, which I doubt would have caused all this mess since it had nothing to do with mqtt.
I've created a github for this so I hope to help others in the future as I migrate.
So you still have all your old stuff running ?
I assume you still have our Arduino Source for the ESP8266 - was it programmed in the Arduino IDE or Platform I/O.
List what you have the ESP8266 doing as it may be smarter to move to a precompiled binary so you do not need to maintain this component and the build environment on an onging basis. Tasmota, ESPEasy, Espurna etc all cover most of the use cases for the ESP8266 nowadays
If you want to stay with your own build on the ESP - then check out versions of libraries etc - and try to slowly bring them up to date.
Deploy your new environment in a docker image - start off on your PC etc as you build it up and when ready deploy it to your PI on a new SD card - that way you can go for very little downtime.
Craig
@bodylinksystems lets focus on the issues in your original question regarding the failed upgrade.
- How did you install Node-RED originally?
- How did you upgrade it that led to the missing nodes?
Can you remember which modules were left in the 'dashed' state?
Did you happen to check the Node-RED log to look for any error messages?
In general, upgrades should just work. To figure out why you've hit some issues we need to see some more specific details of those issues.
Thanks for the response.
I did not keep track of which things were not functioning, but it was quite a few. I wasn't even able to get the UI to show anything at all after a fresh install and import of the old .json
NR was installed via the provided script, and it was updated using the apt-get update and -upgrade commands on the pi, IIRC.
imagine doing a fresh install of the latest PiOS on a new microSD, installing the latest NR and MQTT then importing your old .json, then nothing worked. that was the state I was in.
Sorry I can't provide more info, I tried to put as much version detail as I could in a previous reply.
thanks @craigcurtin I'm going to look into these solutions as I am not familiar with them.
If you used our install script, then an apt-get update etc shouldn't have touched anything to do with Node-RED.
The recommended way to upgrade NR is to run the script again.
