Npm, versioning and palette manager confusions

The tasmota problem is probably this one, though the symptom is not quite the same. There is a recovery strategy there if you read through the thread. Personally I prefer to use MQTT directly rather than going through a contrib node for controlling tasmota devices. Problem after update to 0.9.8 · Issue #26 · DaveMDS/node-red-contrib-tasmota · GitHub

@dceejay, Pete's issue with the Tasmota node does bring up a further question about forcing nodes to be upgraded. Ideally the user should upgrade everything and then check that the system still runs ok before upgrading node-red and nodejs. Otherwise one may end up with issues where it isn't clear whether the contrib node upgrade is the problem or the nodejs/node-red upgrade.
Perhaps another option would be to check for outdated nodes, tell the user what they are, tell him/her he should upgrade them first and only carry with the nodejs/node-red update if the user says to do it anyway.

Hi Colin

The Tasmota node - ignore that - red herring - it seems the (Blitzwold BW-LT21) bulb is shot - won't show up on my nwtwork.. how did I get to NPM 12 last time - no idea.. I though the idea of dave's latest suggestion was to get the setup to move to v14.... I'm wondering if he made an assumption - I just used his latest instruction above and the NR upgrade script... so I'm no further forward :slight_smile: - do I need to do anything else now to get v14?

I think that was the nub of my personal approach and recommendation. If you don't know how to use apt and npm, how do you know in what order to do things?

You make a good point of course. However, I might counter by saying that if this is the case, Dave's script needs more work and needs to become more of a central feature. And then it raises issues about other platforms and we get into problems with 2-tier approaches and certain OS's being left behind.

And yes, this is by no means a dig at Dave's script which is excellent but we are highlighting the issues. The more a script attempts to take on, the more complex and potentially less robust it becomes. At the least, you start finding all the edge-cases - as indeed we are.

At what point do you say that actually showing people how to use your OS's package manager and npm is the best approach? Since you certainly need to know something about managing at least your OS just to use your computer. You also need to know something about managing npm packages to realistically work with Node-RED even if that is just using the nice palette manager.

If you are a Windows, MacOS or Fedora user, you already can't use the script so you are already on your own (actually not quite fair since the docs do tell you how to install things).

1 Like

If you uninstall nodejs using apt then the script will install nodejs 12. It would probably be worth doing that first to make sure the problems with sqlite node rebuild have gone.

Then if you want to get 14 (do you have a reason for wanting to do that?) then edit the file /etc/apt/sources.list.d/nodesource.list and change all the 10s to 14s. Then run
sudo apt update && sudo apt full-upgrade
which should install nodejs 14. Then go into your .node-red folder and run
npm rebuild
then run the script to rebuild node-red.

The existing script leaves 10 or 12 if they are there (unless it is the apt installed old version of 10 :-)... If there is no node.js at all it will install 12.

So if the node updates have worked you should be on node10 plus latest node-red plus latest versions of all extra nodes... (which indeed as Colin points out may break some - so yeah need to think about that - as I said this is mainly me thinking about Node-RED v2 - so maybe I need a different script so it obviously different etc)

If you want to jump to node 14 then if you are on Pi3 or 4 then (for now) the way is to edit your /etc/apt/sources.list.d/nodesource.list file and change node_10.x to node_14.x - then use the use the usual sudo apt update && sudo apt full-upgrade to get node14 ... and then finally in the .node-red folder - run npm rebuild
If you are on arm6 (Pi0,1,2) then it's a bit more involved as we need to hack the script.

Isn't it necessary to re-install node-red after a nodejs upgrade, or am I wrong there? Or possibly run
npm rebuild -g
though I haven't tried that.

You shouldn't need to reinstall, no. Just do a rebuild.

You mean with -g I presume, as well as running it without -g.

1 Like

Node-RED should be ok... though you could always just re-run the script - it will just take longer... as it'll re-re-install Node-RED yet again etc.

Re - forcing upgrades... would it be better to have two scripts... one as-is and one for NR v2 that has disclaimers in big letters saying it will force upgrade to node14 and all nodes etc (and install NRv2) - OR to add a --force option to existing script that basically blows away whatever node.js is already there and installs latest etc... with an extra step of "Are you really sure" in it ?

1 Like

We need a proper design proposal and discussion for how the script will handle the upgrade. Personally I think its clear there are a number of options that could exist on the script to provide users a bit more discretion and control over what it does - whilst having a good default behaviour for users who just want it to work.

I would prefer we don't thrash out the details at the bottom of a thread in the forum and had a discussion on github that we can track changes against.

1 Like

OK - have created an issue for discussion Changes required for updating to Node14 / Node-RED v2 · Issue #13 · node-red/linux-installers · GitHub

1 Like

So... right now, do we have any suggestions other than the ones above that don't work (for me anyway). I have a perfectly up to date NR setup on Pi4 other than it is running node -v = v10.24.1 - and I have currently have no idea how to upgrade from here. The two unsolvable issues seem to be i2c and sqlite...

Here's what my rmote powerup says.
Linux rpi419buster 5.10.17-v7l+ #1403 SMP Mon Feb 22 11:33:35 GMT 2021 armv7l
and node-red is 1.3.2

Regards

Pete.

Well - as long as the updating nodes went ok - (does npm outdated report anything ?) - then you should be ok to push node.js up to 14 (as per Npm, versioning and palette manager confusions - #52 by dceejay)

Or you should be able to get to 12 by running
sudo apt remove nodejs
then running the install script. That should work ok now that all your nodes are up to date. Or when you say that none of the suggestions have worked that this still doesn't work. In which case we need the log to work out why.

Hi Dave

I've just now ran npm outdated....

Package Current Wanted Latest Location
dateformat 3.0.3 3.0.3 4.5.1 global
node-red-node-sqlite 0.3.7 0.3.7 0.6.0 global
pi@rpi419buster:~:14:26[1]>

No idea why it is showing anything outdated after all the updating I've done.. and updating the two nodes - should there be a -g tag on the update - seems to be suggesting these are global?

OK, this is starting to make some sense. Somehow you have installed the sqlite node globally, so it isn't getting updated. I suggest you remove it
npm remove -g node-red-node-sqlite
then make sure it is installed locally
npm i --unsafe-perm node-red-node-sqlite
note, no -g, and restart node-red. I don't know why you have dateformat installed. Is it something you use in your flows? It probably isn't relevant to the issues here.

2 Likes

It gets even more complicated.. I did the install as you requested – from the pi folder….

Result was: up to date in 0.092s

So, I did the same again without the -g

Result was a proper uninstall.

I then checked the .node-red folder – still had a sqlite install in there…. So… I uninstalled THAT (without the -g).

Could you please confirm therefore – what folder should I be sitting in, in order to do the new install?

While waiting I tried going into NR (no sqlite) and used the pallette manager to install sqlite - that failed...(after stopping and restarting NR). So I uninstalled the sqlite it installed into /home/pi/.node-red/node_modules.... and installed it the way you suggested above - from the /home/pi folder...a long time later (but no ERRORS)... I ended up with sqlite in the home/pi/node_modules folder and ran node-red...

STILL no sqlite - yet before I started all of this I had SQLITE working (remember I'm on nodejs 10).

:blush:
Finally, the non-global --unsafe-perm version in .node-red folder - restarted node-red - works...

but now. npm outdated returns:

Package     Current  Wanted  Latest  Location
dateformat    3.0.3   3.0.3   4.5.1  global
sqlite3       4.0.9   4.2.0   5.0.2  global

I thought the point of this was to install the latest sqlite?

Welcome to Node-RED
===================

16 Apr 17:31:43 - [info] Node-RED version: v1.3.2
16 Apr 17:31:43 - [info] Node.js  version: v10.24.1
16 Apr 17:31:43 - [info] Linux 5.10.17-v7l+ arm LE
16 Apr 17:31:44 - [info] Loading palette nodes
16 Apr 17:31:49 - [info] Worldmap version 2.13.2
16 Apr 17:31:49 - [info] Dashboard version 2.28.2 started at /ui
16 Apr 17:31:50 - [info] Settings file  : /home/pi/.node-red/settings.js
16 Apr 17:31:50 - [info] HTTP Static    : /home/pi/.node-red/public
16 Apr 17:31:50 - [info] Context store  : 'default' [module=localfilesystem]
16 Apr 17:31:50 - [info] User directory : /home/pi/.node-red
16 Apr 17:31:50 - [warn] Projects disabled : set editorTheme.projects.enabled=true to enable
16 Apr 17:31:50 - [info] Flows file     : /home/pi/.node-red/flows.json
16 Apr 17:31:50 - [info] Server now running at http://127.0.0.1:1880/
16 Apr 17:31:50 - [info] Starting flows
16 Apr 17:31:52 - [info] Started flows
16 Apr 17:31:52 - [info] [sqlitedb:6e9a0125.59bc4] opened /home/pi/dbs/iot.db ok
16 Apr 17:31:52 - [info] [sqlitedb:fdbe27fe.fb7848] opened /home/pi/dbs/iot.db ok
16 Apr 17:31:52 - [info] [sqlitedb:33875a87.57ade6] opened /home/pi/dbs/iot.db ok
16 Apr 17:31:52 - [info] [tasmota-mqtt-broker:34268cd3.bfa344] Connected to broker: mqtt://127.0.0.1:1883
16 Apr 17:31:52 - [info] [mqtt-broker:4c682b3a.2ab5c4] Connected to broker: mqtt://127.0.0.1:1883
16 Apr 17:31:59 - [info] [e-mail:peterscargill@gmail.com] Message sent: 250 2.0.0 OK  1618587119 m15sm7803175wrx.32 - gsmtp


Pete

Hi Pete.

All nodes should be installed into the userDir (~/.node-red). Only Node-RED itself should be installed globally.

The only real reason to install any npm package globally is for when a package implements a command line command that you want to be able to run anywhere. Everything else should be locally installed.

If you get stuck with something, you can always just delete it manually. For a globally installed package, you can find out where it lives with:

npm -g list npm-module-name

On my Debian system, that is /usr/lib/node_modules/. Just delete the folder with the matching name.