Using npm in script file, installs seem to fail?

General
1

I have a new OS build of a Raspberry Pi, I install node red via the official installation script. No problems so far. I copy my flows.json file into the correct location ~/.node-red/ for example. All good so far. I then run the following script. I stop and start node red via systemctl restart nodered. Here is where something goes south, node red comes up fine, the flows load of course, but... maybe 1/2 to 1/3 of the packages loaded via npm install cli as per the script below have failed to load, no errors, just are not visible in node red. If I look at the palette via manage palette, sure enough, the packages are not loading. Scratching my head over this one, thoughts? Am I doing something wrong?

#!/bin/bash
#
#

declare -a MODULES=(
                'node-red-dashboard'
                'node-red-contrib-weather'
                'node-red-dashboard'
                'node-red-contrib-advanced-ping'
                'node-red-node-email'
                'node-red-contrib-email-out'
                'node-red-node-ui-table'
                'node-red-contrib-os'
                'node-red-contrib-counter'
                'node-red-contrib-bme280'
                'node-red-contrib-simple-gate'
                'node-red-contrib-loop-processing'
                'node-red-node-pi-gpiod'
                'node-red-contrib-linux-network-stats'
                'node-red-contrib-simple-queue'
                'node-red-contrib-traffic'
                'node-red-contrib-queue-gate'
                'node-red-node-mysql'
                'node-red-contrib-i2c'
                'node-red-node-openweathermap'
                'node-red-contrib-advanced-ping'
                'node-red-contrib-fs-ops'
                'node-red-contrib-cron-plus'
)

declare -i KABOOM=255
declare -i ONE=1
declare -i ZERO=0
#declare -i FIVE=5

declare -i theResult=${ZERO}
declare thePath=`pwd`

#

clear

#

cd /home/pi/.node-red

#

for theIndex in "${MODULES[@]}"
do
{
        #

        echo "Install ${theIndex}..."

        npm install ${theIndex}
        theResult=$?

        echo "...Install ${theIndex} (Result: ${theResult})."
        sleep ${ONE}
}
done

#

cd ${thePath}

#

exit ${theResult}

Update... I added a call per install, to query npm list output to try to track the if even npm is not installing and not reporting errors. But this REALLY slows the script down because of the way npm list works.

2

Perhaps a fatal error is causing script to exit?

What happens if you try npm install package1 package2 package3 etc?

3

Script always completes. It never aborts... the packages just are not installed per node red. I get the pop up window that states there are X packages references by flows missing. Which should not happen when I explicitly stop and start node red. Hence I am scratching my head, on this one. I was hoping someone would point out something I am doing wrong. Otherwise, the install success rate of npm is horrible, which does not make sense, right?

Update... trying a different variant of the script that calls npm install once, rather than multiple times, so see if that works better.

4

If you just copy a suitable package.json, all you need to do is a single npm install.

1 Like
5

In my case there may not always be a suitable package.json. But, as I have been doing more testing, doing a single call to npm with a rather long parameter list, may be working better. Still testing.

Long term, what I want to do is parse a flows.json file and if possible find the modules that should be loaded, and load them accordingly. When we import, it would be nice if the missing modules were automatically loaded, maybe with a prompt asking for approval. That said, I understand why the current design does not do that, automatic or even prompted load of modules that you have not first reviewed, is a risk scenario that should not be ignored.

6

Calling npm install once, seems to work a lot better, than calling it per module. It should not make a difference but apparently it does, based on the several tests I did today.

7

I wonder - was your earlier version was spawning multiple npm processes in parallel (not waiting for npm to finish before looping around to the next npm call)?

If so, then perhaps that was causing some file locking or overwriting each others updates in package.json (i.e. npm1 reads package.json & begins install, npm2 reads package.json & begins install, npm1 updates package.json, npm2 overwrites package.json with its in memory copy adding package2 but ultimately has no awareness of package1)

This is speculation as I don't know at what point npm reads & rewrites package.json but I'd put money on it being a concurrency issue.

It would also explain why running npm once with all packages in the command line works better (as npm will likely do things in series not parallel)

1 Like
8

There is already a web page that does that here on this site - I've lost the reference to it though.

npm is a very strange beast as I discovered when trying to integrate with uibuilder. So I can't say I'm surprised you are finding oddities. It should work though.

9

The page you want is for the flow inspector - New thing: Flow Inspector

and yes - calling npm once with a big list is the preferred way as it lets the dependency optimiser work a lot better.
(PS you have dashboard and advanced-ping in the list twice)
(PPS not sure why you should need node-red-contrib-email-out - we would prefer to fix the node-red-node one if it is broken than require another overlapping one)

2 Likes
10

@dceejay
The duplicates... where an attempt to see if I could improve the success rate while trying to figure out why the limited results. But good catch. :slight_smile:

I have some modules/packages I will retire... I was trying different ones as I was learning, I have a couple of flows I need to revise to get rid of the modules I did not... enjoy as much as others, if you will.

I am still looking for a straight-forward SMS option, but I have email working, and of course dashboard windows popups for immediate alerting to operator when needed.

11

I played a bit with nexmo just to see if it worked, and yes was working fine, think it was free to start experimenting also but a paid plan is needed for permanent usage I guess

In daily operation of my "home production system" I just use Telegram for short messages including snap shots to my iPhone

If you don't want to use such services, maybe an ESP alternative coud be of interest?

Just need to figure out adding MQTT to it and voila!

12

Yes, it just so happens I have reasonable experience with ESP32, ESP8266 based devices. On one (very) remote Pi device. I have ESP8266 connected to the serial console of the Pi, so it the Pi gets into trouble I can see what happened. I can even reset or power on/off the Pi, if the model Pi supports and has the right jumper pins.

13

I am looking at using google voice as transport layer for SMS. This limits where I can send the text messages, but for my need it works. I just have google voice forward to cell phone.

14

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.