Can't bring back the manage palette - a new wrinkle?

Howdy, I am in the process of bringing up several node-red instances on an ubuntu 22.04 server and have had some hard fought successes but I have run into a problem that I cannot seem to find any clues on where to go next.

I will start with some basic information:

  1. I am running as a standard user but have added sudo power along the way to my user "beetred"
  2. I am starting all instances of node-red as this user
  3. my version of npm is 9.2.0, node is v18.13.0 (Thanks to TotallyInformation below)
  4. my startup script for one of the instances (node-orange) is as follows:
[Unit]
Description=Node-ORANGE
After=network.target
[Service]
Type=simple
ExecStart=/home/beetred/node-orange/node_modules/node-red/bin/node-orange --settings'/home/beetred/node-orange/.data/settings.js'
Restart=on-failure
KillSignal=SIGINT
# non-root user to run as
WorkingDirectory=/home/beetred/node-orange/.data
User=beetred
Group=beetred
[Install]
WantedBy=multi-user.target

and the results of the boot are good:

â—Ź node-orange.service - Node-ORANGE
     Loaded: loaded (/etc/systemd/system/node-orange.service; disabled; vendor preset: enabled)
     Active: active (running) since Sun 2023-01-29 12:23:40 MST; 56min ago
   Main PID: 5245 (node-red)
      Tasks: 11 (limit: 28471)
     Memory: 54.3M
        CPU: 2.091s
     CGroup: /system.slice/node-orange.service
             └─5245 node-red "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" >

Jan 29 12:23:40 plex node-orange[5245]: You should set your own key using the 'credentialSecret' option in
Jan 29 12:23:40 plex node-orange[5245]: your settings file. Node-RED will then re-encrypt your credentials
Jan 29 12:23:40 plex node-orange[5245]: file using your chosen key the next time you deploy a change.
Jan 29 12:23:40 plex node-orange[5245]: ---------------------------------------------------------------------
Jan 29 12:23:40 plex node-orange[5245]: 29 Jan 12:23:40 - [warn] Encrypted credentials not found
Jan 29 12:23:40 plex node-orange[5245]: 29 Jan 12:23:40 - [info] Server now running at http://127.0.0.1:1893/
Jan 29 12:23:40 plex node-orange[5245]: 29 Jan 12:23:40 - [info] Starting flows
Jan 29 12:23:40 plex node-orange[5245]: 29 Jan 12:23:40 - [debug] red/nodes/flows.start : starting flow : global
Jan 29 12:23:40 plex node-orange[5245]: 29 Jan 12:23:40 - [debug] red/nodes/flows.start : starting flow : 69e6e59182af629b
Jan 29 12:23:40 plex node-orange[5245]: 29 Jan 12:23:40 - [info] Started flows

But when I go into the web page there is no manage palette option.

if I do a which npm at the command line as a beetred user I get:

$ which npm
/home/beetred/.nvm/versions/node/v18.13.0/bin/npm

which seems good to me

my settings file is as follows (with all comments removed for brevity)

module.exports = {
    flowFile: '/home/beetred/node-orange/.data/flows.json',
    flowFilePretty: true,
    userDir: '/home/beetred/node-orange/.data',
    uiPort: process.env.PORT || 1893,
    diagnostics: {
        enabled: true,
        ui: true,
    },
    runtimeState: {
        enabled: false,
        ui: false,
    },
    logging: {
        console: {
            level: "debug",
            metrics: false,
            audit: false
        }
    },
    exportGlobalContextKeys: false,
    externalModules: {
    },
    editorTheme: {
        page: {title: "Node-ORANGE"},
        header:{title: "Node-ORANGE"},
        palette: {
        },
        projects: {
            enabled: false,
            workflow: {
                mode: "manual"
            }
        },
        codeEditor: {
            lib: "monaco",
            options: {
            }
        }
    },
    functionExternalModules: true,
    functionGlobalContext: {
    },
    debugMaxLength: 1000,
    mqttReconnectTime: 15000,
    serialReconnectTime: 15000,
}

and my exec file for node-red is the same as the node-red-pi file

OPTIONS=""
ARGS=""

for arg in "$@"
do
   case $arg in
      --userDir|--settings|--help) ARGS="$ARGS $arg";;
            --*) OPTIONS="$OPTIONS $arg";;
              *) ARGS="$ARGS $arg";;
   esac
done

# Find the real location of this script
CURRENT_PATH=$(pwd)
SCRIPT_PATH=$(readlink -f "$0")
while [ -h "${SCRIPT_PATH}" ]; do
    cd "$(dirname "${SCRIPT_PATH}")" || exit 1
    P=$(basename "${SCRIPT_PATH}")
    SCRIPT_PATH=$(readlink "${P}")
done
cd "$(dirname "${SCRIPT_PATH}")" > /dev/null || exit 1
SCRIPT_PATH=$(pwd)
cd "$CURRENT_PATH" || exit 1

# Run Node-RED
exec /home/beetred/.nvm/versions/node/v18.13.0/bin/node ${OPTIONS} ${SCRIPT_PATH}/../red.js ${ARGS}

and my PATH for the beetred user is:

/home/beetred/.nvm/versions/node/v18.13.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/usr/games:/usr/local/games:/snap/bin

Any ideas on where I should look or what I should try next would be very much appreciated. Right now I am at a point I can't think of anything else to try.

This really is quite a bad idea I'm afraid. Node.js versions with odd numbers are effectively development versions and generally poorly supported - including by Node-RED and its dependencies. It is best to stick to LTS versions of Node.js which are all even numbered. Typically Node.js has 3 LTS versions running in parallel, any of them should work with Node-RED.

I assume that isn't all of it since a fair bit seems to be missing - including the closing }.

First step, downgrade Node.js to v18 LTS.

Thanks for the tip on moving to the LTS versions, I am new to Node-Red and Javascript and didn't know I was on the bleeding edge.

The settings file is in fact my complete file with the exception of all comments were removed so I wouldn't take up space with comments (the last curly brace is probably just a cut and paste error when I copied over the uncommented file). I assumed the other settings which are not in the file were OK as defaults since when I installed they were not set in the file. So do I need to go here and work to populate all possible configuration settings? That seems a little extreme to me and also I didn't see settings that would keep me from having a working palette. If you know of the settings I need to pay attention to for this issue please could you narrow it down some for me?

Thanks again for taking the time to reply.
Best,
Steve

OK, now running v18.13.0. Still no manage palette available after restarting node-red and reloading web page but I feel like it was still a step forward.

I am still at the same point I was at before, wondering where to go next.

Steve

I think the next step is to rename your settings.js then restart node-red so that it creates a new one (or copy the master from the Node-RED source. I also have a working and annotated settings.js example in the alternate installer repo.

Well, that didn't work as advertised. Instead of giving me the new file it just gave me an error "Error loading settings file:" and died "node-orange.service: Deactivated successfully." I looked for your working and annotated settings.js example but couldn't find it. Could you provide a link?

I decided to rename my settings file back to settings.js and run the start command interactively and guess what, it worked fine and the resulting web page did in fact show the manage palette line in the hamburger icon drop down. I wonder what you make of that. Here's the console output.

root@plex:/home/beetred/node-orange/node_modules/node-red/bin# /home/beetred/.nvm/versions/node/v18.13.0/bin/node ../red.js --settings /home/beetred/node-orange/.data/settings.js
29 Jan 21:07:50 - [info]

Welcome to Node-RED
===================

29 Jan 21:07:50 - [info] Node-RED version: v3.0.2
29 Jan 21:07:50 - [info] Node.js  version: v18.13.0
29 Jan 21:07:50 - [info] Linux 5.15.0-58-generic x64 LE
29 Jan 21:07:50 - [info] Loading palette nodes
29 Jan 21:07:50 - [debug] Module: node-red-contrib-influxdb 0.6.1 /home/beetred/node-orange/.data/node_modules/node-red-contrib-influxdb
29 Jan 21:07:50 - [debug] Module: node-red-contrib-stackhero-influxdb-v2 1.0.4 /home/beetred/node-orange/.data/node_modules/node-red-contrib-stackhero-influxdb-v2
29 Jan 21:07:50 - [debug] Module: node-red-node-mysql 1.0.3 /home/beetred/node-orange/.data/node_modules/node-red-node-mysql
29 Jan 21:07:51 - [info] Settings file  : /home/beetred/node-orange/.data/settings.js
29 Jan 21:07:51 - [info] Context store  : 'default' [module=memory]
29 Jan 21:07:51 - [info] User directory : /home/beetred/node-orange/.data
29 Jan 21:07:51 - [warn] Projects disabled : editorTheme.projects.enabled=false
29 Jan 21:07:51 - [info] Flows file     : /home/beetred/node-orange/.data/flows.json
29 Jan 21:07:51 - [debug] loaded flow revision: 6ef76a7838326782121ff0e92f77b24b
29 Jan 21:07:51 - [debug] red/runtime/nodes/credentials.load : no user key present
29 Jan 21:07:51 - [debug] red/runtime/nodes/credentials.load : using default key
29 Jan 21:07:51 - [debug] red/runtime/nodes/credentials.load : keyType=system
29 Jan 21:07:51 - [warn]

---------------------------------------------------------------------
Your flow credentials file is encrypted using a system-generated key.

If the system-generated key is lost for any reason, your credentials
file will not be recoverable, you will have to delete it and re-enter
your credentials.

You should set your own key using the 'credentialSecret' option in
your settings file. Node-RED will then re-encrypt your credentials
file using your chosen key the next time you deploy a change.
---------------------------------------------------------------------

29 Jan 21:07:51 - [warn] Encrypted credentials not found
29 Jan 21:07:51 - [info] Server now running at http://127.0.0.1:1893/
29 Jan 21:07:51 - [info] Starting flows
29 Jan 21:07:51 - [debug] red/nodes/flows.start : starting flow : global
29 Jan 21:07:51 - [debug] red/nodes/flows.start : starting flow : 69e6e59182af629b
29 Jan 21:07:51 - [info] Started flows

If I cancel and start the node-red using the systemd command then I get no manage palette option. The difference seems to me to be the user I am running under where I am root at the command line and a limited user when running from systemd. So to test this I then tried to run as the limited user at the command line. I bet you can guess what happens, but I couldn't, because I thought there would be no manage palette, but I was wrong, running as the beetred user from the command line I also had the manage palette capability displayed. What follows in the beetred user output.

beetred@plex:~/node-orange/node_modules/node-red/bin$ /home/beetred/.nvm/versions/node/v18.13.0/bin/node ../red.js --settings /home/beetred/node-orange/.data/settings.js
29 Jan 21:24:54 - [info]

Welcome to Node-RED
===================

29 Jan 21:24:54 - [info] Node-RED version: v3.0.2
29 Jan 21:24:54 - [info] Node.js  version: v18.13.0
29 Jan 21:24:54 - [info] Linux 5.15.0-58-generic x64 LE
29 Jan 21:24:55 - [info] Loading palette nodes
29 Jan 21:24:55 - [debug] Module: node-red-contrib-influxdb 0.6.1 /home/beetred/node-orange/.data/node_modules/node-red-contrib-influxdb
29 Jan 21:24:55 - [debug] Module: node-red-contrib-stackhero-influxdb-v2 1.0.4 /home/beetred/node-orange/.data/node_modules/node-red-contrib-stackhero-influxdb-v2
29 Jan 21:24:55 - [debug] Module: node-red-node-mysql 1.0.3 /home/beetred/node-orange/.data/node_modules/node-red-node-mysql
29 Jan 21:24:55 - [info] Settings file  : /home/beetred/node-orange/.data/settings.js
29 Jan 21:24:55 - [info] Context store  : 'default' [module=memory]
29 Jan 21:24:55 - [info] User directory : /home/beetred/node-orange/.data
29 Jan 21:24:55 - [warn] Projects disabled : editorTheme.projects.enabled=false
29 Jan 21:24:55 - [info] Flows file     : /home/beetred/node-orange/.data/flows.json
29 Jan 21:24:55 - [debug] loaded flow revision: 6ef76a7838326782121ff0e92f77b24b
29 Jan 21:24:55 - [debug] red/runtime/nodes/credentials.load : no user key present
29 Jan 21:24:55 - [debug] red/runtime/nodes/credentials.load : using default key
29 Jan 21:24:55 - [debug] red/runtime/nodes/credentials.load : keyType=system
29 Jan 21:24:55 - [warn]

---------------------------------------------------------------------
Your flow credentials file is encrypted using a system-generated key.

If the system-generated key is lost for any reason, your credentials
file will not be recoverable, you will have to delete it and re-enter
your credentials.

You should set your own key using the 'credentialSecret' option in
your settings file. Node-RED will then re-encrypt your credentials
file using your chosen key the next time you deploy a change.
---------------------------------------------------------------------

29 Jan 21:24:55 - [warn] Encrypted credentials not found
29 Jan 21:24:55 - [info] Server now running at http://127.0.0.1:1893/
29 Jan 21:24:55 - [info] Starting flows
29 Jan 21:24:55 - [debug] red/nodes/flows.start : starting flow : global
29 Jan 21:24:55 - [debug] red/nodes/flows.start : starting flow : 69e6e59182af629b
29 Jan 21:24:55 - [info] Started flows

So at this point I am left with the following:

  • The theory that it is related to the user I am running under (probably permissions) is rubbish.

  • The theory that it is related to the settings.js file contents is rubbish.

  • I am left again without a clue as to why the manage palette is disappearing but perhaps it will come to me in a dream, and it is bedtime here in the southwest US so I am off to ponder the mysteries of my node-red environment and the lack of breadcrumbs it leaves making resolution of problems a matter of chance.

Steve

What do you get from this in the debug panel? (running via systemd)

Screenshot 2023-01-30 at 08.51.29

If this output is different from what you believe it should be, maybe add PATH yourself in your systemd file?

[Service]
...
Environment="PATH=/home/beetred/.nvm/versions/node/v18.13.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/usr/games:/usr/local/games:/snap/bin"
...

Just my 2c

1 Like

Good idea except that I don't think the example you share is correct. And even if it is, it will fail on an npm update. This is the path on my working Debian Linux system:

PATH: "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

WAIT! Just spotted: .nvm/versions/node/v18.13.0.

That is almost certainly an issue. You have NVM installed and this very often causes unexpected problems. You don't want to run NVM on a system that you need to rely on. Only use it on a dedicated development device.

1 Like

I appreciate your 2c very much and you seem to be onto something, and no, I do not get what I expected.

Here's what I got:

/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin

Which obviously is missing the path needed to npm. Your tip actually shows exactly what the problem is and shares what I believe to be the correct solution.

I used a simple flow with the change node set up as in your example with an inject and a debug node around it.

image

I tried it and it worked. I want to thank all who weighed in on my issue and especially thank Marcus who got there first with the solution.

Best,
Steve

Julian, can you offer a link or citation that will allow me to read up on the issues with nvm so I can educate myself on these issues affecting reliability when using nvm? Also can you please provide me with same or at least directions on how to get to your annotated example settings.js file? Thanks in advance for helping me to learn more about these issues with the node-red environment.

Best,
Steve

I think the fundamental problem with using nvm is that which version of nodejs you get depends on where, and in what environment, it is run from. Therefore it is very easy to find that node-red runs with a different version of nodejs to that in a command shell. Apart from causing confusion, this causes havoc if nodes are installed using the command line, as one ends up with nodes in node_modules built using different versions of nodejs dependant on whether they were installed via node-red or via the command line.

NVM for JavaScript a bit like what virtualenv is for Python. It creates a new environment for each version of node.js you install. So every version install is separate and it is very hard to know where everything lives and whether anything will talk to anything else. So if you install a new version of node.js - possibly thinking that you are doing a node.js upgrade - everything breaks. So it is entirely unsuited to a "production" environment.

For the settings file:

Thank you Julian and Colin for the words of what I assume are experience and to you Julian for the annotated settings file (and the location of where the source is at).

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.