Npm, versioning and palette manager confusions

It should run npm install of all the listed modules to bring them up to the latest versions. So, for example, the sqlite node should be up to date so that when you run the script you shouldn't have the build problems that you encountered (the version you currently have installed is not compatible with the newer nodejs). At least that is the current thinking.

First question

npm i $(npm outdated --parseable | grep -v git | cut -d ":" -f 4 | xargs)

Do I do that in the PI folder or the .node-red folder?

Pete

Hi Pete, in the .node-red folder please

I heartily agree, but (once again) you've found the rub. There isn't much point in an intuitive, low-code programming environment if keeping it working requires deep knowledge of operating system, package management, and version control stuff. That's why Dave's installation script is such a big contribution and worth the effort to maintain. I could live without it now but not when I started using NR. Sure there are edge cases, but there are also a lot of questions on the forum from folks who have tried DIY or followed outdated or incomplete recipes from YouTube.

[EDIT]: None of this was meant as a knock on @scargill's script, blog, or YouTube channel!!! I have followed them for years and suspect that was where I first learned of NR.

1 Like

Something missing there I think, Dave. I ran

npm i $(npm outdated --parseable | grep -v git | cut -d ":" -f 4 | xargs)

in the .node-red folder
(which didn't throw any errors)
I then ran the NR install/upgrade script in the /home/pi folder .. It removed the "old" node-red but left the existing node v10.24.1 and npm 6.14.12

the operation took only a couple of minutes.

This can take 20-30 minutes on the slower Pi versions - please wait.
  Stop Node-RED                       ✔
  Remove old version of Node-RED      ✔
  Remove old version of Node.js       -
  Leave existing Node.js              -   Node v10.24.1   Npm 6.14.12
  Clean npm cache                     -
  Install Node-RED core               ✔   1.3.2
  Move global nodes to local          -
  Install extra Pi nodes              ✔
  Npm rebuild existing nodes          -
  Add shortcut commands               ✔
  Update systemd script               ✔
Any errors will be logged to   /var/log/nodered-install.log
All done.
  You can now start Node-RED with the command  node-red-start
  or using the icon under   Menu / Programming / Node-RED
  Then point your browser to localhost:1880 or http://{your_pi_ip-address}:1880
Started  Thu 15 Apr 09:50:58 CEST 2021  -  Finished  Thu 15 Apr 09:52:42 CEST 2021
pi@rpi418buster:~:09:52[0]>

So, what happened to updating nodejs etc?
Here's the output - and that Tasmota error is new - never had that before...

node-red-start

Start Node-RED

Once Node-RED has started, point a browser at http://192.168.1.19:1880
On Pi Node-RED works better with the Firefox or Chrome browser

Use   node-red-stop                          to stop Node-RED
Use   node-red-start                         to start Node-RED again
Use   node-red-log                           to view the recent log output
Use   sudo systemctl enable nodered.service  to autostart Node-RED at every boot
Use   sudo systemctl disable nodered.service to disable autostart on boot

To find more nodes and example flows - go to http://flows.nodered.org

Starting as a systemd service.
15 Apr 09:57:01 - [info]
Welcome to Node-RED
===================
15 Apr 09:57:01 - [info] Node-RED version: v1.3.2
15 Apr 09:57:01 - [info] Node.js  version: v10.24.1
15 Apr 09:57:01 - [info] Linux 5.10.17-v7l+ arm LE
15 Apr 09:57:02 - [info] Loading palette nodes
15 Apr 09:57:09 - [info] Worldmap version 2.13.2
15 Apr 09:57:09 - [info] Dashboard version 2.28.2 started at /ui
15 Apr 09:57:11 - [info] Settings file  : /home/pi/.node-red/settings.js
15 Apr 09:57:11 - [info] HTTP Static    : /home/pi/.node-red/public
15 Apr 09:57:11 - [info] Context store  : 'default' [module=localfilesystem]
15 Apr 09:57:11 - [info] User directory : /home/pi/.node-red
15 Apr 09:57:11 - [warn] Projects disabled : set editorTheme.projects.enabled=true to enable
15 Apr 09:57:11 - [info] Flows file     : /home/pi/.node-red/flows.json
15 Apr 09:57:11 - [info] Server now running at http://127.0.0.1:1880/
15 Apr 09:57:11 - [info] Starting flows
15 Apr 09:57:12 - [warn] [Tasmota Light:lt21 light] Broker configuration is wrong or missing, please review the node settings
15 Apr 09:57:12 - [error] [Tasmota Light:d6eb53f7.49dc7] TypeError: Cannot read property 'subscribe' of null
15 Apr 09:57:12 - [warn] [Tasmota Sensor:lt21 sense] Broker configuration is wrong or missing, please review the node settings
15 Apr 09:57:12 - [error] [Tasmota Sensor:85004139.b711a] TypeError: Cannot read property 'subscribe' of null
15 Apr 09:57:12 - [warn] [Tasmota Switch:lt21 switch] Broker configuration is wrong or missing, please review the node settings
15 Apr 09:57:12 - [error] [Tasmota Switch:1e77de9c.1b5321] TypeError: Cannot read property 'subscribe' of null
15 Apr 09:57:13 - [info] Started flows
15 Apr 09:57:13 - [info] [sqlitedb:6e9a0125.59bc4] opened /home/pi/dbs/iot.db ok
15 Apr 09:57:13 - [info] [sqlitedb:fdbe27fe.fb7848] opened /home/pi/dbs/iot.db ok
15 Apr 09:57:13 - [info] [sqlitedb:33875a87.57ade6] opened /home/pi/dbs/iot.db ok
15 Apr 09:57:13 - [info] [mqtt-broker:4c682b3a.2ab5c4] Connected to broker: mqtt://127.0.0.1:1883
15 Apr 09:57:20 - [info] [e-mail:peterscargill@gmail.com] Message sent: 250 2.0.0 OK  1618473440 h81sm1739176wmf.41 - gsmtp

Definitely still old node - node -v gives v10.24.1

The script won't upgrade nodejs v10 as v10 is ok. How did you get to v12 last time?

The tasmota error suggests there is a problem with the upgraded tasmota node. I don't think that is anything to do with the other problems you have been having.

The tasmota problem is probably this one, though the symptom is not quite the same. There is a recovery strategy there if you read through the thread. Personally I prefer to use MQTT directly rather than going through a contrib node for controlling tasmota devices. Problem after update to 0.9.8 · Issue #26 · DaveMDS/node-red-contrib-tasmota · GitHub

@dceejay, Pete's issue with the Tasmota node does bring up a further question about forcing nodes to be upgraded. Ideally the user should upgrade everything and then check that the system still runs ok before upgrading node-red and nodejs. Otherwise one may end up with issues where it isn't clear whether the contrib node upgrade is the problem or the nodejs/node-red upgrade.
Perhaps another option would be to check for outdated nodes, tell the user what they are, tell him/her he should upgrade them first and only carry with the nodejs/node-red update if the user says to do it anyway.

Hi Colin

The Tasmota node - ignore that - red herring - it seems the (Blitzwold BW-LT21) bulb is shot - won't show up on my nwtwork.. how did I get to NPM 12 last time - no idea.. I though the idea of dave's latest suggestion was to get the setup to move to v14.... I'm wondering if he made an assumption - I just used his latest instruction above and the NR upgrade script... so I'm no further forward :slight_smile: - do I need to do anything else now to get v14?

I think that was the nub of my personal approach and recommendation. If you don't know how to use apt and npm, how do you know in what order to do things?

You make a good point of course. However, I might counter by saying that if this is the case, Dave's script needs more work and needs to become more of a central feature. And then it raises issues about other platforms and we get into problems with 2-tier approaches and certain OS's being left behind.

And yes, this is by no means a dig at Dave's script which is excellent but we are highlighting the issues. The more a script attempts to take on, the more complex and potentially less robust it becomes. At the least, you start finding all the edge-cases - as indeed we are.

At what point do you say that actually showing people how to use your OS's package manager and npm is the best approach? Since you certainly need to know something about managing at least your OS just to use your computer. You also need to know something about managing npm packages to realistically work with Node-RED even if that is just using the nice palette manager.

If you are a Windows, MacOS or Fedora user, you already can't use the script so you are already on your own (actually not quite fair since the docs do tell you how to install things).

1 Like

If you uninstall nodejs using apt then the script will install nodejs 12. It would probably be worth doing that first to make sure the problems with sqlite node rebuild have gone.

Then if you want to get 14 (do you have a reason for wanting to do that?) then edit the file /etc/apt/sources.list.d/nodesource.list and change all the 10s to 14s. Then run
sudo apt update && sudo apt full-upgrade
which should install nodejs 14. Then go into your .node-red folder and run
npm rebuild
then run the script to rebuild node-red.

The existing script leaves 10 or 12 if they are there (unless it is the apt installed old version of 10 :-)... If there is no node.js at all it will install 12.

So if the node updates have worked you should be on node10 plus latest node-red plus latest versions of all extra nodes... (which indeed as Colin points out may break some - so yeah need to think about that - as I said this is mainly me thinking about Node-RED v2 - so maybe I need a different script so it obviously different etc)

If you want to jump to node 14 then if you are on Pi3 or 4 then (for now) the way is to edit your /etc/apt/sources.list.d/nodesource.list file and change node_10.x to node_14.x - then use the use the usual sudo apt update && sudo apt full-upgrade to get node14 ... and then finally in the .node-red folder - run npm rebuild
If you are on arm6 (Pi0,1,2) then it's a bit more involved as we need to hack the script.

Isn't it necessary to re-install node-red after a nodejs upgrade, or am I wrong there? Or possibly run
npm rebuild -g
though I haven't tried that.

You shouldn't need to reinstall, no. Just do a rebuild.

You mean with -g I presume, as well as running it without -g.

1 Like

Node-RED should be ok... though you could always just re-run the script - it will just take longer... as it'll re-re-install Node-RED yet again etc.

Re - forcing upgrades... would it be better to have two scripts... one as-is and one for NR v2 that has disclaimers in big letters saying it will force upgrade to node14 and all nodes etc (and install NRv2) - OR to add a --force option to existing script that basically blows away whatever node.js is already there and installs latest etc... with an extra step of "Are you really sure" in it ?

1 Like

We need a proper design proposal and discussion for how the script will handle the upgrade. Personally I think its clear there are a number of options that could exist on the script to provide users a bit more discretion and control over what it does - whilst having a good default behaviour for users who just want it to work.

I would prefer we don't thrash out the details at the bottom of a thread in the forum and had a discussion on github that we can track changes against.

1 Like

OK - have created an issue for discussion Changes required for updating to Node14 / Node-RED v2 · Issue #13 · node-red/linux-installers · GitHub

1 Like

So... right now, do we have any suggestions other than the ones above that don't work (for me anyway). I have a perfectly up to date NR setup on Pi4 other than it is running node -v = v10.24.1 - and I have currently have no idea how to upgrade from here. The two unsolvable issues seem to be i2c and sqlite...

Here's what my rmote powerup says.
Linux rpi419buster 5.10.17-v7l+ #1403 SMP Mon Feb 22 11:33:35 GMT 2021 armv7l
and node-red is 1.3.2

Regards

Pete.