New Utility: alternate-node-red-installer

Hi all, as I'm needing to install some clean installations of Node-RED for testing, I thought that I would finally get round to writing a proper tool for installing multiple independent instances of Node-RED.

I've just finished that off and published to GitHub and npm so you should be able to use it straight away if you have a need for such a thing. It gives you a command-line installer that takes a single parameter controlling the root folder for installation. It should work on all platforms.

This follows the installation approach that I've used for a couple of years now as I don't like to do global installs of server packages. A global install of Node-RED also restricts you to only having a single version installed which is not ideal for testing.

The advantages of this approach I believe are:

  • You can have as many different versions of Node-RED installed as you like making it easy to compare behaviours.
  • Everything is in one place which makes backup and restore (including a known, working version of Node-RED itself) really simple.
  • Everything is in a place that you have chosen. No wasted time hunting for folders in weird places.
  • Users don't need admin rights to install an instance of Node-RED.

Hope others find this useful. Let me know in this thread if you do or indeed if you don't!

As always, feedback and suggestions (preferably polite ones :slight_smile:) are welcome.

7 Likes

Is it possible to use this to run an earlier copy of Node-RED (e.g 0.20.8)?

Well the script itself will install the latest version. However, it is dead simple to then install whatever version you like.

So just let the script run, cd into the resulting master folder and then use npm install --unsafe-perm --production node-red@xxxx where xxxx is the version you want.

I regularly install a clean version for testing or a previous (or next) version for comparison.

1 Like

Perfect thank you :slight_smile:

Slight issue
Last week I installed it on my Win10 machine so I got 0.20.8

I wanted to update but it didn't work - I still seem to have 0.20.8
(PS same procedure worked fine on a Pi on Tues following launch of 1.0.0)

That's odd, I did that on mine and it worked ok - though I was "upgrading" from the @next version to the live v1

Instead, try a full install. From the master folder:

npm install --production --unsafe-perm node-red

That should work fine.

It did ta :slight_smile:
image

1 Like

[FeatureRequest] :slight_smile:

Everytime I use the alternate installer, I need to change the port used as I'm installing it on a machine that is already running standard Node-RED on 1880

Could we have a switch (-p ?) on the installer so that it saves me the trouble :slight_smile: of editing the settings.js file?

PS I'd accept a clever awk/sed/grep one liner (assuming in base folder not ./data) that does it in the meantime :slight_smile:

We could :slight_smile: - whether we will depends on me finding some time :face_with_raised_eyebrow:

Of course, there is already a method for setting the port on startup simply by setting the PORT environment variable before starting Node-RED.

So in theory, it would be possible to add this to the start npm script rather than trying to hack the settings.js which I'd prefer to leave alone since it does change from time-to-time.

1 Like

Feature request? Would it be possible to add an argument to the installer with the version to have it installed that way immediately, for example when testing bugs on potentially a specific version of NR? Or maybe add this to the documentation on github :slight_smile:

2 Likes

Ah, that sounds like an excellent idea :grinning:

Of course, as Nick and Dave like to say: PR's are always welcome :face_with_raised_eyebrow:

I'll add an issue when I get some time.

2 Likes

Julian,
I need some help. Going through instructions I have this...

Installed Node-RED with your system on C:\NodeRed\data

first I tried C:\NodeRed>npm run update-master and then, C:\NodeRed\data>npm run update-master. I tried to upgrade with the instructions and i'm getting this.

C:\NodeRed\data>npm run update-master

> node-red-userdir@1.0.1 update-master C:\NodeRed\data
> cd .. && npm run update-master


> node-red-master@1.0.0 update-master C:\NodeRed
> npm install --production --unsafe-perm

npm WARN node-red-master@1.0.0 No repository field.

audited 352 packages in 2.042s

6 packages are looking for funding
  run `npm fund` for details

found 4 moderate severity vulnerabilities
  run `npm audit fix` to fix them, or `npm audit` for details

...
C:\NodeRed>node-red

C:\NodeRed>"C:/Program Files/nodejs/node.exe" c:/nodered/node_modules/node-red/red.js --userDir c:/nodered/data
15 Apr 11:14:29 - [info]

Welcome to Node-RED
===================

15 Apr 11:14:29 - [info] Node-RED version: v1.2.9
15 Apr 11:14:29 - [info] Node.js  version: v14.16.1
15 Apr 11:14:29 - [info] Windows_NT 10.0.16299 x64 LE
15 Apr 11:14:29 - [info] Loading palette nodes
15 Apr 11:14:33 - [info] Dashboard version 2.28.2 started at /ui
15 Apr 11:14:33 - [info] Settings file  : c:\nodered\data\settings.js
15 Apr 11:14:33 - [info] Context store  : 'default' [module=memory]
15 Apr 11:14:33 - [info] User directory : c:\nodered\data

i.e. It did not update to Node-RED 3.1.2. :unamused:

UPDATE!

C:\NodeRed>npm install --production --unsafe-perm node-red
...
Welcome to Node-RED
===================

15 Apr 11:29:33 - [info] Node-RED version: v1.3.2
15 Apr 11:29:33 - [info] Node.js  version: v14.16.1
15 Apr 11:29:33 - [info] Windows_NT 10.0.16299 x64 LE
15 Apr 11:29:38 - [info] Loading palette nodes
15 Apr 11:29:42 - [info] Dashboard version 2.28.2 started at /ui
15 Apr 11:29:42 - [info] Settings file  : c:\nodered\data\settings.js
15 Apr 11:29:42 - [info] Context store  : 'default' [module=memory]
15 Apr 11:29:42 - [info] User directory : c:\nodered\data

In the nodered folder you just run npm update and

Julian,
Will npm update uninstall any nodes not in the original install then? Or will it leave the ones you've installed after installing Node-RED? The "install" actually left everything there.

So does npm update "sense" how its installed?

Install:
C:\NodeRed>npm install --production --unsafe-perm node-red

Thus,

Update:   npm update [-g] [<pkg>...]

C:\NodeRed>npm update --production --unsafe-perm node-red

?

Thanks

npm uses the appropriate package.json file to determine what should and shouldn't be installed.

In the alternate installer, the master folder's package.json just has node-red in it (you can add other things if you want/need to of course, indeed good place to install nodes if you don't want users to be able to mess with them).

In the userDir folder, if you have a package installed but for some odd reason not listed npm will warn you. Can't remember whether it automatically removes it I'm afraid. The npm docs will have the answer.

In respect to installing node-red, I tend to always run the full install as you've listed it but I doubt that it is actually needed to do that and I have occasionally just done an update with no apparent harm.

@TotallyInformation,
I have a question for you as I am installing Node-RED on an internet facing server for the express purpose of having it to copy the files after it is installed to a series of offline PC's.

I did not use your installer.... If I botched it all up, I can uninstall and do it again.
I am asking to understand the folders which are installing the node-red instance and then the support of the nodes.

I used the syntax for Powershell as this.

PS C:\NodeRed> npm install --production --unsafe -perm node-red

it has installed in my C:\NodeRed folder "node_modules" and "package-lock.json"
Inside the 'node_modules' folder there are 239 folders of "node modules"

Now, I do not know so this is why I ask and I'd like to comprehend the folder structure necessary
I believe this folder is the basis of Node-RED nodes. And another folder typically called ".node-red" is installed in the GLOBAL context which hosts the nodes installed and when Node-RED runs for the first time.... but I'm guessing.

Can you please help me out to understand what is supposed to be there and how I can avoid putting files, modules, nodes in places where they shouldn't be? As per your install package it places the nodes in the "Data" folder... When I installed --production --unsafe -perm Did it put anything anywhere else?

Kind Thanks for your help
@tree-frog

This is the standard features of a node.js based application. Any installed node.js app will have the same folder and files - it is simply that installing node-red globally hides these from you.

The content of the node_modules folder is maintained by npm. You can copy it if you like but if you always do an install, you don't need to. Because of the number of small files involved, copying that folder can take a long time.

The content of the package-lock.json file is similar to the package.json file that is also installed. However, it contains the exact versions of the installed dependencies whereas package.json contains an indication of what version should be installed. So the lock file will produce an exact copy of the installation (it wouldn't pick up any updated module versions for example). Details are in the npm documentation.

The typically created ~/.node-red folder is replaced by the ./data sub-folder in my alternate installer. This is properly called the "userDir" folder and appears as such in the node-red settings. I create it as a sub-folder so that everything is in 1 master folder that is more easily copied or backed-up. That folder holds your node-red settings.js, flows file, credentials, any persisted context variables, the uibuilder folders if you are using that and quite often a bunch of other files and folders. You will also see another node_modules folder in here and this is where the code for your installed nodes goes, typically not the one that is in the parent folder (though you can actually install nodes into that one too which can be useful if you want some nodes that a Node-RED editor can't mess with but can still use).

Yup, that is right and that is where node-red itself puts the modules for installed nodes if you use the palette manager. That is mostly where you want your node packages and any uibuilder installed packages to be.

Nothing of any importance no. npm itself will have updated a few other things but you don't need to be concerned about that.

The whole point of the alternate installer is that everything is together - in significant contrast to the "normal" installation which spreads out files and folders to dark places users will not normally inhabit! :mage:

When you use the alternate installer (or manually reproduce the same type of install), the master folder that you supply to the installer is the single root folder for everything.

The content of the backup scripts in the alternate installer templates/master/scripts folder will also show you where things are that you need to keep/reproduce:

#! /usr/bin/env bash

# Redirect stdout to syslog - 
#   show with: `sudo journalctl -t nrmain-backup`
#   or `sudo cat /var/log/syslog | grep nrmain-backup`
exec 1> >(logger -t nrmain-backup -p local0.info)
# redirect stderr to syslog
exec 2> >(logger -t nrmain-backup -p local0.err)

# --- SET THESE TO THE CORRECT LOCATIONS --- #
NR_SERVICE_NAME=nrmain
NR_SOURCE_PATH=/home/home/nrmain
NR_DEST_PATH=/home/home/nrmain-backup
# ------------------------------------------ #

STARTDATE=$(date +'%Y-%m-%d %T')

echo " "
echo "Starting daily backup of $NR_SOURCE_PATH/ to $NR_DEST_PATH/ ..."
echo "Rotating snapshots ..."

# Delete oldest daily backup
if [ -d $NR_DEST_PATH/daily.7 ] ; then
    echo "  Deleting oldest daily backup $NR_DEST_PATH/daily.7"

    # The slow but understandable way:
    #rm -rf $NR_DEST_PATH/daily.7
    # The faster way (needs an empty folder)
    rsync -rd --delete $NR_DEST_PATH/empty/ $NR_DEST_PATH/daily.7/
fi

# Shift all other daily backups ahead one day
for OLD in 6 5 4 3 2 1	; do
	if [ -d $NR_DEST_PATH/daily.$OLD ] ; then
		NEW=$(($OLD+1))

		echo "  Moving $NR_DEST_PATH/daily.$OLD to $NR_DEST_PATH/daily.$NEW"
		
		# Backup last date
		# ISSUE: touch does not support options on synology (busybox) system
		touch $NR_DEST_PATH/.dtimestamp -r $NR_DEST_PATH/daily.$OLD
		mv $NR_DEST_PATH/daily.$OLD $NR_DEST_PATH/daily.$NEW
		# Restore timestamp
		touch $NR_DEST_PATH/daily.$NEW -r $NR_DEST_PATH/.dtimestamp

	fi
done

# Copy hardlinked snapshot of level 0 to level 1 (before updating 0 via rsync)
if [ -d $NR_DEST_PATH/daily.0 ] ; then

	echo "  Copying hardlinks from $NR_DEST_PATH/daily.0 to $NR_DEST_PATH/daily.1"

	cp -al $NR_DEST_PATH/daily.0 $NR_DEST_PATH/daily.1
fi

echo "Finished rotating snapshots ..."

if ! [ -d $NR_DEST_PATH/daily.0 ] ; then
	mkdir -p $NR_DEST_PATH/daily.0
fi

# Set today's date on the current backup folder
touch $NR_DEST_PATH/daily.0

ENDDATE=$(date --iso-8601=s)

# Back up
echo "Performing rsync backup ..."
rsync --archive --hard-links --delete --delete-excluded \
      --exclude 'node_modules' --exclude 'data/node_modules' --exclude 'data/externalModules/node_modules' \
      $NR_SOURCE_PATH/ $NR_DEST_PATH/daily.0

# Validate return code
# 0 = no error,
# 24 is fine, happens when files are being touched during sync (logs etc)
# all other codes are fatal -- see man (1) rsync
# You can output the result to MQTT or to a Node-RED http-in endpoint
if ! [ $? = 24 -o $? = 0 ] ; then
	echo "Fatal: Node-RED daily backup finished with errors!"
    #curl --insecure -I 'https://localhost:1880/nrnotify?type=backup&schedule=daily&result=fail'
    mosquitto_pub -r -t services/$NR_SERVICE_NAME/backup/daily/fail -m $ENDDATE
else
    echo "Finished Node-RED daily backup, no errors."
    #curl --insecure -I 'https://localhost:1880/nrnotify?type=backup&schedule=daily&result=success'
    mosquitto_pub -r -t services/$NR_SERVICE_NAME/backup/daily/success -m $ENDDATE
fi

# Sync disks to make sure data is written to disk
sync

#EOF

@TotallyInformation,
Thanks for the speedy reply. I have C:\NodeRed and the files which were installed by my first attempt.

I have an install on another PC which I have Node-RED 1.3.6 installed via your installer...
Structure is here..

C:\NodeRed
                  \bin
                  \data
                 \node_modules
                 \system
                 package.json
                 package-lock.json
                ... 

in \data is

          \node_modules

Can I just copy the folders in the \data which host the installed nodes and run it from the new system ?

Was there a question in there?

I hit the tab key and it dropped me from the editor.

And I mean copy the \data folder + contents over to my new system and not have issues?