Installing dependency modules when cloning project to server

bs-eng · 16 March 2024 17:24

Hi there!
Currently I am cloning my project to two other linux PCs, which are set up identical to the the source system.
When starting node-red first time on the clone I setup the git and switch to the branch needed.
Then I restart node-red.

then I get the message that there are many nodes missing and it offers me to manage the missing dependencies. However the list is empty.

On the forum I found that the project package.json is not used by node-red at startup, so it needs to be copied to the local user first. If I copy the package.json from the project to the local user (only the dependencies section) and restart node-red I get the same. Meaning:

project dependencies windows is empty
palette manager is empty

Nowehere I can see the list of missing modules in order to install them.

What is the intended procedure to install modules upon fresh clone of a project?

Thanks a lot!

FYI: my dependencies list looks like this:

  "dependencies": {
        "@abandonware/noble": "^1.9.2-14",
        "node-beacon-scanner-duplicates-check": "^0.2.0",
        "node-red-contrib-beacon-scanner": "~1.0.1",
        "node-red-contrib-tts-ultimate": "~1.0.40",
        "node-red-contrib-ui-media": "~1.2.0",
        "node-red-dashboard": "~2.30.0",
        "node-red-node-mysql": "~1.0.1",
        "node-red-node-tail": "~0.3.2",
        "node-red-node-ui-table": "~0.3.12"
    }

Colin · 16 March 2024 18:17

Copy the whole package.json across, go into the .node-red folder and run npm install in that folder, then start node red.

bs-eng · 16 March 2024 19:14

Thanks for the pointer, Colin!
Unfortunately this started to install things and then procuded a lot of errors.

I started node-red anyway to see if any module got installed and noticed that suddenly I got a proper list of missing dependencies. So I thought let's install them as proposed by the node-red GUI.
I started with ui-table and it was not possible to get it installed. The error log is as follows:

-----------------------------------------------------------
2024-03-16T19:08:04.963Z Installieren : node-red-node-ui-table ~0.3.12

2024-03-16T19:08:04.164Z npm install --no-audit --no-update-notifier --no-fund --save --save-prefix=~ --production --engine-strict node-red-node-ui-table@~0.3.12
2024-03-16T19:08:05.339Z [err] npm
2024-03-16T19:08:05.340Z [err]  WARN config production Use `--omit=dev` instead.
2024-03-16T19:08:23.070Z [err] npm
2024-03-16T19:08:23.071Z [err]  WARN deprecated debug@4.1.1: Debug versions >=3.2.0 <3.2.7 || >=4 <4.3.1 have a low-severity ReDos regression when used in a Node.js environment. It is recommended you upgrade to 3.2.7 or 4.3.1. (https://github.com/visionmedia/debug/issues/797)
2024-03-16T19:08:23.073Z [err] npm 
2024-03-16T19:08:23.074Z [err] WARN deprecated debug@4.1.1: Debug versions >=3.2.0 <3.2.7 || >=4 <4.3.1 have a low-severity ReDos regression when used in a Node.js environment. It is recommended you upgrade to 3.2.7 or 4.3.1. (https://github.com/visionmedia/debug/issues/797)
2024-03-16T19:08:23.076Z [err] npm 
2024-03-16T19:08:23.076Z [err] WARN 
2024-03-16T19:08:23.077Z [err] deprecated debug@4.1.1: Debug versions >=3.2.0 <3.2.7 || >=4 <4.3.1 have a low-severity ReDos regression when used in a Node.js environment. It is recommended you upgrade to 3.2.7 or 4.3.1. (https://github.com/visionmedia/debug/issues/797)
2024-03-16T19:08:23.111Z [err] npm
2024-03-16T19:08:23.112Z [err]  
2024-03-16T19:08:23.113Z [err] WARN
2024-03-16T19:08:23.113Z [err]  
2024-03-16T19:08:23.113Z [err] deprecated
2024-03-16T19:08:23.114Z [err]  querystring@0.2.0: The querystring API is considered Legacy. new code should use the URLSearchParams API instead.
2024-03-16T19:08:23.158Z [err] npm WARN
2024-03-16T19:08:23.159Z [err]  deprecated formidable@1.2.2: Please upgrade to latest, formidable@v2 or formidable@v3! Check these notes: https://bit.ly/2ZEqIau
2024-03-16T19:08:23.439Z [err] npm 
2024-03-16T19:08:23.439Z [err] WARN deprecated uuid@3.3.2: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
2024-03-16T19:08:58.744Z [err] npm
2024-03-16T19:08:58.745Z [err]  
2024-03-16T19:08:58.745Z [err] ERR!
2024-03-16T19:08:58.745Z [err]  
2024-03-16T19:08:58.746Z [err] code
2024-03-16T19:08:58.746Z [err]  1
2024-03-16T19:08:58.747Z [err] npm
2024-03-16T19:08:58.747Z [err]  
2024-03-16T19:08:58.748Z [err] ERR!
2024-03-16T19:08:58.748Z [err]  
2024-03-16T19:08:58.749Z [err] path
2024-03-16T19:08:58.749Z [err]  /home/nodered/.node-red/node_modules/microsoft-cognitiveservices-speech-sdk
2024-03-16T19:08:58.754Z [err] npm
2024-03-16T19:08:58.755Z [err]  
2024-03-16T19:08:58.755Z [err] ERR!
2024-03-16T19:08:58.755Z [err]  command failed
2024-03-16T19:08:58.756Z [err] npm
2024-03-16T19:08:58.756Z [err]  
2024-03-16T19:08:58.756Z [err] ERR!
2024-03-16T19:08:58.757Z [err]  
2024-03-16T19:08:58.757Z [err] command
2024-03-16T19:08:58.757Z [err]  sh -c npm install --package-lock-only --ignore-scripts --no-audit && npx npm-force-resolutions
2024-03-16T19:08:58.758Z [err] npm
2024-03-16T19:08:58.758Z [err]  
2024-03-16T19:08:58.758Z [err] ERR!
2024-03-16T19:08:58.758Z [err]  npm ERR! code EBADENGINE
2024-03-16T19:08:58.759Z [err] npm
2024-03-16T19:08:58.759Z [err]  
2024-03-16T19:08:58.759Z [err] ERR!
2024-03-16T19:08:58.759Z [err]  npm ERR! engine Unsupported engine
2024-03-16T19:08:58.760Z [err] npm
2024-03-16T19:08:58.760Z [err]  
2024-03-16T19:08:58.760Z [err] ERR!
2024-03-16T19:08:58.760Z [err]  npm ERR! engine Not compatible with your version of node/npm: eslint-plugin-jsdoc@39.9.1
2024-03-16T19:08:58.761Z [err] npm
2024-03-16T19:08:58.761Z [err]  
2024-03-16T19:08:58.761Z [err] ERR!
2024-03-16T19:08:58.761Z [err]  npm ERR! notsup Not compatible with your version of node/npm: eslint-plugin-jsdoc@39.9.1
2024-03-16T19:08:58.762Z [err] npm
2024-03-16T19:08:58.762Z [err]  
2024-03-16T19:08:58.762Z [err] ERR!
2024-03-16T19:08:58.762Z [err]  npm ERR! notsup Required: {"node":"^14 || ^16 || ^17 || ^18 || ^19"}
2024-03-16T19:08:58.762Z [err] npm
2024-03-16T19:08:58.763Z [err]  
2024-03-16T19:08:58.763Z [err] ERR!
2024-03-16T19:08:58.763Z [err]  npm ERR! notsup Actual:   {"npm":"10.2.4","node":"v20.11.1"}
2024-03-16T19:08:58.763Z [err] npm
2024-03-16T19:08:58.767Z [err]  ERR! 
2024-03-16T19:08:58.767Z [err] npm ERR! npm ERR! A complete log of this run can be found in: /home/nodered/.npm/_logs/2024-03-16T19_08_28_495Z-debug-0.log
2024-03-16T19:08:58.772Z [err] 
2024-03-16T19:08:58.773Z [err] npm
2024-03-16T19:08:58.773Z [err]  ERR! A complete log of this run can be found in: /home/nodered/.npm/_logs/2024-03-16T19_08_05_185Z-debug-0.log
2024-03-16T19:08:58.850Z rc=1

From the last lines do I guess correct, that I may have a too high version of node.js (v20)?

Colin · 16 March 2024 19:50

Yes, you should submit an error against the nodes that say that. Then install 18 and run

npm rebuild
And
npm install

bs-eng · 16 March 2024 20:09

Ok, thanks!

Did as you proposed and tried installing again the ui-table. Now I am getting errors related to a module that is also on the list (abandonwarenoble), but different from what I am trying to install now (ui-table).
I removed the module from the user node-red directory and tried again. Failure.
The log I get is:

-----------------------------------------------------------
2024-03-16T20:00:46.118Z Installieren : node-red-node-ui-table 0.x

2024-03-16T20:00:45.335Z npm install --no-audit --no-update-notifier --no-fund --save --save-prefix=~ --production --engine-strict node-red-node-ui-table@0.x
2024-03-16T20:00:46.765Z [err] npm
2024-03-16T20:00:46.766Z [err]  WARN
2024-03-16T20:00:46.767Z [err]  config production Use `--omit=dev` instead.
2024-03-16T20:01:33.118Z [err] npm
2024-03-16T20:01:33.119Z [err]  WARN deprecated querystring@0.2.0: The querystring API is considered Legacy. new code should use the URLSearchParams API instead.
2024-03-16T20:01:33.178Z [err] npm 
2024-03-16T20:01:33.179Z [err] WARN deprecated formidable@1.2.2: Please upgrade to latest, formidable@v2 or formidable@v3! Check these notes: https://bit.ly/2ZEqIau
2024-03-16T20:01:33.560Z [err] npm WARN
2024-03-16T20:01:33.560Z [err]  deprecated debug@4.1.1: Debug versions >=3.2.0 <3.2.7 || >=4 <4.3.1 have a low-severity ReDos regression when used in a Node.js environment. It is recommended you upgrade to 3.2.7 or 4.3.1. (https://github.com/visionmedia/debug/issues/797)
2024-03-16T20:01:33.748Z [err] npm 
2024-03-16T20:01:33.749Z [err] WARN deprecated uuid@3.3.2: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
2024-03-16T20:01:33.792Z [err] npm WARN
2024-03-16T20:01:33.792Z [err]  deprecated debug@4.1.1: Debug versions >=3.2.0 <3.2.7 || >=4 <4.3.1 have a low-severity ReDos regression when used in a Node.js environment. It is recommended you upgrade to 3.2.7 or 4.3.1. (https://github.com/visionmedia/debug/issues/797)
2024-03-16T20:01:34.137Z [err] npm WARN
2024-03-16T20:01:34.137Z [err]  deprecated debug@4.1.1: Debug versions >=3.2.0 <3.2.7 || >=4 <4.3.1 have a low-severity ReDos regression when used in a Node.js environment. It is recommended you upgrade to 3.2.7 or 4.3.1. (https://github.com/visionmedia/debug/issues/797)
2024-03-16T20:03:25.671Z [err] npm
2024-03-16T20:03:25.671Z [err]  ERR! code 1
2024-03-16T20:03:25.672Z [err] npm 
2024-03-16T20:03:25.672Z [err] ERR! path /home/nodered/.node-red/node_modules/@abandonware/noble
2024-03-16T20:03:25.678Z [err] npm ERR!
2024-03-16T20:03:25.679Z [err]  command failed
2024-03-16T20:03:25.679Z [err] npm ERR! command sh -c node-gyp-build
2024-03-16T20:03:25.679Z [err] npm ERR!
2024-03-16T20:03:25.679Z [err]  gyp info it worked if it ends with ok
2024-03-16T20:03:25.679Z [err] npm ERR! gyp info using node-gyp@10.0.1
2024-03-16T20:03:25.679Z [err] npm
2024-03-16T20:03:25.680Z [err]  ERR! gyp info using node@18.19.1 | linux | x64
2024-03-16T20:03:25.680Z [err] npm 
2024-03-16T20:03:25.680Z [err] ERR! gyp info find Python using Python version 3.11.2 found at "/usr/bin/python3"
2024-03-16T20:03:25.680Z [err] npm ERR!
2024-03-16T20:03:25.680Z [err]  gyp info spawn /usr/bin/python3
2024-03-16T20:03:25.680Z [err] npm 
2024-03-16T20:03:25.681Z [err] ERR! gyp info spawn args [
2024-03-16T20:03:25.681Z [err] npm 
2024-03-16T20:03:25.681Z [err] ERR! gyp info spawn args '/home/nodered/.node-red/node_modules/node-gyp/gyp/gyp_main.py',
2024-03-16T20:03:25.681Z [err] npm ERR!
2024-03-16T20:03:25.681Z [err]  gyp info spawn args 'binding.gyp',
2024-03-16T20:03:25.681Z [err] npm ERR!
2024-03-16T20:03:25.681Z [err]  gyp info spawn args '-f',
2024-03-16T20:03:25.681Z [err] npm ERR!
2024-03-16T20:03:25.681Z [err]  gyp info spawn args 'make',
2024-03-16T20:03:25.681Z [err] npm ERR!
2024-03-16T20:03:25.681Z [err]  gyp info spawn args '-I',
2024-03-16T20:03:25.681Z [err] npm ERR!
2024-03-16T20:03:25.682Z [err]  gyp info spawn args '/home/nodered/.node-red/node_modules/@abandonware/noble/build/config.gypi',
2024-03-16T20:03:25.682Z [err] npm ERR! gyp info spawn args '-I',
2024-03-16T20:03:25.682Z [err] npm ERR! gyp info spawn args '/home/nodered/.node-red/node_modules/node-gyp/addon.gypi',
2024-03-16T20:03:25.682Z [err] npm
2024-03-16T20:03:25.682Z [err]  ERR! gyp info spawn args '-I',
2024-03-16T20:03:25.682Z [err] npm
2024-03-16T20:03:25.682Z [err]  ERR! gyp info spawn args '/home/nodered/.cache/node-gyp/18.19.1/include/node/common.gypi',
2024-03-16T20:03:25.682Z [err] npm
2024-03-16T20:03:25.682Z [err]  ERR! gyp info spawn args '-Dlibrary=shared_library',
2024-03-16T20:03:25.682Z [err] npm 
2024-03-16T20:03:25.682Z [err] ERR! gyp info spawn args '-Dvisibility=default',
2024-03-16T20:03:25.682Z [err] npm 
2024-03-16T20:03:25.682Z [err] ERR! gyp info spawn args '-Dnode_root_dir=/home/nodered/.cache/node-gyp/18.19.1',
2024-03-16T20:03:25.682Z [err] npm 
2024-03-16T20:03:25.683Z [err] ERR! gyp info spawn args '-Dnode_gyp_dir=/home/nodered/.node-red/node_modules/node-gyp',
2024-03-16T20:03:25.683Z [err] npm ERR!
2024-03-16T20:03:25.683Z [err]  gyp info spawn args '-Dnode_lib_file=/home/nodered/.cache/node-gyp/18.19.1/<(target_arch)/node.lib',
2024-03-16T20:03:25.683Z [err] npm ERR!
2024-03-16T20:03:25.683Z [err]  gyp info spawn args '-Dmodule_root_dir=/home/nodered/.node-red/node_modules/@abandonware/noble',
2024-03-16T20:03:25.683Z [err] npm ERR!
2024-03-16T20:03:25.683Z [err]  gyp info spawn args '-Dnode_engine=v8',
2024-03-16T20:03:25.683Z [err] npm ERR!
2024-03-16T20:03:25.683Z [err]  gyp info spawn args '--depth=.',
2024-03-16T20:03:25.683Z [err] npm ERR! gyp info spawn args '--no-parallel',
2024-03-16T20:03:25.683Z [err] npm ERR! gyp info spawn args '--generator-output',
2024-03-16T20:03:25.684Z [err] npm ERR! gyp info spawn args 'build',
2024-03-16T20:03:25.684Z [err] npm ERR! gyp info spawn args '-Goutput_dir=.'
2024-03-16T20:03:25.684Z [err] npm ERR! gyp info spawn args ]
2024-03-16T20:03:25.684Z [err] npm ERR! gyp ERR! build error 
2024-03-16T20:03:25.684Z [err] npm ERR! gyp ERR! stack Error: not found: make
2024-03-16T20:03:25.684Z [err] npm ERR! gyp ERR! stack at getNotFoundError (/home/nodered/.node-red/node_modules/which/lib/index.js:16:17)
2024-03-16T20:03:25.684Z [err] npm
2024-03-16T20:03:25.685Z [err]  ERR! gyp ERR! stack at which (/home/nodered/.node-red/node_modules/which/lib/index.js:77:9)
2024-03-16T20:03:25.685Z [err] npm
2024-03-16T20:03:25.685Z [err]  ERR! gyp ERR! stack at async doWhich (/home/nodered/.node-red/node_modules/node-gyp/lib/build.js:112:22)
2024-03-16T20:03:25.685Z [err] npm 
2024-03-16T20:03:25.685Z [err] ERR! gyp ERR! stack at async loadConfigGypi (/home/nodered/.node-red/node_modules/node-gyp/lib/build.js:77:7)
2024-03-16T20:03:25.685Z [err] npm 
2024-03-16T20:03:25.685Z [err] ERR! gyp ERR! stack at async build (/home/nodered/.node-red/node_modules/node-gyp/lib/build.js:35:3)
2024-03-16T20:03:25.685Z [err] npm 
2024-03-16T20:03:25.685Z [err] ERR! gyp ERR! stack at async run (/home/nodered/.node-red/node_modules/node-gyp/bin/node-gyp.js:81:18)
2024-03-16T20:03:25.685Z [err] npm ERR!
2024-03-16T20:03:25.685Z [err]  gyp ERR! System Linux 6.1.0-18-amd64
2024-03-16T20:03:25.685Z [err] npm ERR!
2024-03-16T20:03:25.685Z [err]  gyp ERR! command "/usr/bin/node" "/home/nodered/.node-red/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
2024-03-16T20:03:25.685Z [err] npm ERR!
2024-03-16T20:03:25.686Z [err]  gyp ERR! cwd /home/nodered/.node-red/node_modules/@abandonware/noble
2024-03-16T20:03:25.686Z [err] npm ERR! gyp ERR! node -v v18.19.1
2024-03-16T20:03:25.686Z [err] npm ERR! gyp ERR! node-gyp -v v10.0.1
2024-03-16T20:03:25.686Z [err] npm
2024-03-16T20:03:25.686Z [err]  ERR! gyp ERR! not ok
2024-03-16T20:03:25.693Z [err] 
2024-03-16T20:03:25.693Z [err] npm ERR! A complete log of this run can be found in: /home/nodered/.npm/_logs/2024-03-16T20_00_46_577Z-debug-0.log
2024-03-16T20:03:25.792Z rc=1

I am a bit out of ideas, short of reinstaling the whole system again and installing every module by hand. That process worked on the first clone I made a week ago. However to me it seems not the right way to go for clean cloning of the server.

Other ideas?

Colin · 16 March 2024 20:39

What do you mean? What exactly did you do and why?

bs-eng · 16 March 2024 22:03

Well, due to the messages relating to that module I removed it from the node-red user directory. It seemed to cause trouble and was not fully installed.
npm rebuild said it was successful after that.

try to install ui-table => got errors related to another module (abandonwarenoble)
delete abandonwarenoble from node-red user dir
npm rebuild
install ui-table again => still errors relating to abandonwarenoble

Steve-Mcl · 16 March 2024 23:36

Are you 100% certain this is your actual node-red user directory? Usually it is /home/.node-red/

This is a good clue that your system doesn't have all the necessary tools. I.e if on Ubuntu, you should do sudo apt-get install build-essential first (or the equivalent on non Debian system)

Colin · 17 March 2024 09:03

Did you install node red using the recommended Debian/Ubuntu/pi script? I thought it checked that build-essential is installed.

bs-eng · 17 March 2024 19:50

/home/nodered/.node-red/

yes, this is the user directory of the nodered user. Why would it be ´/home/.node-red´? Under which user is node-red running in such a case?

Did you install node red using the recommended Debian/Ubuntu/pi script? I thought it checked that build-essential is installed.

Yes, I used the recommended Debian script - and yes, I also thought it would install build essentials. On the original (running) system it did and it was never a problem to install nodes from source. Also on dev machines that was always fine. So I really wonder what changed this time.

I will completely set up the system new. As I am not certain what has been different. Maybe I did miss a step in my procedure.

Thanks for the helpful pointers! I will keep you posted onthe result of the new install.
Cheers

Colin · 17 March 2024 20:17

Just install build-essential

dceejay · 18 March 2024 09:24

Note - the script does not check or install build-essential ... as per the docs

(as it is not 100% needed in all circumstances) - so it is recommended but up to the user to install if necessary.

bs-eng · 18 March 2024 10:13

OK, then we have the reason for the failures or manual install.

So what is left, is the question: What is the intended procedure to transfer projects to another machine and take care of all dependencies?

Steve-Mcl · 18 March 2024 10:25

Same as before, however install the required tools first (the command in the yellow tip section)

bs-eng · 20 March 2024 16:01

As a quick follow up. The command npm build worked well right after installing build essentials.

I will try to recreate the procedure on a virgin server one of these days to see if package dependencies get shown properly in node-red.
Cheers

system · 3 April 2024 16:02

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Creating a node with dependency General	5	1381	11 January 2019
Q about Node-red in project mode, dependencies in package.json, project dependencies General docker	2	76	25 November 2024
Installing a module fails to install dependencies? Developing Nodes	12	1091	9 June 2022
Is it possible to install missing node types automatically? General	12	1193	8 August 2021
Node-RED node installed as dependency requires restart Developing Nodes	6	543	2 April 2022

Installing dependency modules when cloning project to server

Related topics