Should Palette Manager show when a node downloads something from another server?

Core Development Feature Requests
1

I noticed something in the Palette Manager and I wonder if it should be shown to users.

When you install a module, Node-RED runs npm install for you.

Sometimes a node has a dependency that is downloaded from another website, not from the npm registry.

For example, node-red-contrib-modbus currently has this dependency in its package.json:

"@openp4nr/modbus-serial": "https://dl.cloudsmith.io/public/iniationware-gmbh/plus4nodered/npm/openp4nr/modbus-serial/8.4.0/openp4nr-modbus-serial-8.4.0.tgz"

So during installation, this file is downloaded from dl.cloudsmith.io.

I am not saying this is bad or unsafe. node-red-contrib-modbus is only an easy example.

My point is about transparency.

Many users just click Install and do not know that the installation may also download files from another website.

Maybe Palette Manager could show a small message like:

“This module also downloads a dependency from xyz web”

Another option could be an extra confirmation before installation:

“This module will also download a dependency from xyz web. Continue?”

This would let the user see where files are coming from before the installation starts.

It could also help when an installation fails because a company network blocks that website, or the machine has limited internet access.

I am not suggesting that external dependencies should be blocked or treated as unsafe.

I just think it could be useful for the user to know what is being downloaded and from where.

What do you think? Would this information be useful in the Palette Manager?

2

The important thing to note is that no external install is "safe".

Doesn't really matter that much whether the install is from npmjs, GitHub or elsewhere. You need to vet every install.

I would, of course, agree that there is some additional risk involved in installing something from a "random" site and really npm should probably auto-reject such installs unless explicitly told otherwise. But we are where we are.

Bottom line is that, I agree that it would be nice to see a warning about such installs (if that is possible). However, there is still a danger that people might assume the lack of a warning means that an installed module/node is "Safe". So I'm a bit torn on this myself.

3

Where does this actually stop though? The modbus node shows one dependency pointing at a tarball URL, but that's just the top level. npm resolution is transitive ( any package anywhere in the tree can declare a dependency as a tarball URL, git URL, or GH shorthand instead of a registry semver entry). A complete version of this warning wouldn't just read the node's own package.json, it would need to walk the entire resolved dependency graph.

Now that in itself is not too difficult (sandbox an npm install with --package-lock-only & parse the generated lock file) but there is more - it could have install scripts that download code. Then potentially there is alt registries in npmrc to consider too. And would it be cheap enough to run on each result in the palette results etc

I'm not saying it shouldn't or wouldn't be useful, but a complete/confident solution is not straightforward either.

4

Right, it might be nested inside the module, and then another module inside that module...

5

This is why I say that it needs something at the npm level, not Node-RED. I don't believe npm has the tools to deal with this.

The only "proper" answer to this problem is to only allow specific packages to be installed - preferably from a controlled npm library running "locally". Coupled with aggressive vetting of allows packages.

For less critical environments (e.g. your own home), you should always check what is being installed before installation. You should also set npm to refuse installing updates less than a few days old (as this is a common supply-chain attack issue). Also, you should occasionally audit your installed packages.

Most of all, always install Node-RED under a restricted user and preferably locally, not globally. Then use your OS and filing system to restrict access to only necessary services and folders/files.

As I often say, Node-RED is an incredibly powerful tool. Ignore that at your peril.

6

If you check the history for that node, you'll see that it's dependency got deleted from npm, so this is the only way to keep it running (since the @openp4nr/modbus-serial was not on GH either to fork iirc.

Also if you really really want to scare yourself, consider that lots of npm packages are just wrappers round native libraries (e.g. the serial port node is a good example). As Steve hinted many of these use post-install scripts to pull pre-compiled binary blobs from random hosting sites (e.g. s3 buckets) rather than try and build the library at install time as it can be very slow to build the library. And unless you real really are very skilled there is no easy way to compare those binary blobs to the src code that may (or may not) be shipped bundled into the node.

Now the next version of npm is going to block these scripts by default, so I hope you all have a full and complete set of all the required build tools and development versions of the pre-req libraries needed to build the binary components of all the pacakges you use?

While well intentioned I'm 90% sure npm are going to have to roll back some of these changes or most people will end up disabling to new "safe guards" because so much if just going to break.

7

I doubt they will since the other equivalent tools to npm have banned post-install scripts for some while now. There are overrides available thankfully should you need them but people should really think carefully whether they want to take the risk given the number of supply-chain attacks we are seeing. Until developer communities all update things to be stricter, AI will continue to create ongoing supply-chain disasters to be honest.

To cover for supply-chain hijacks, at the very least EVERYONE should set their minimum allowed update date to 3 days or more:

Try this for a system wide setting for your user:

npm config set min-release-age 7 --location=user

But also make sure that if you are creating custom nodes or other packages using npm, create a .npmrc file in the root of your project and add this to it:

min-release-age=7

You will see different recommendations on the minimum age to use but from what I've seen, 3d may be a little short. 7d should cover pretty much all issues with reasonably well used packages.

8

Surely, since the tarball exists, the correct thing to do would be to create a new version and publish that to npm and use it (assuming the original license allowed it) - or even incorporate the code directly into the node package if it isn't used elsewhere.

Personally, I would not use a node in a production environment that had a tarball dependency like that. It would trigger all manner of security warnings that wouldn't easily be solvable I don't think.

9

I'd love to see their market share values, I expect low single digits for them all combined (I've seen a website that claims npm is only 53% but I really find that hard to believe given npm ships with nodejs and you need to go out of your way to install something different. This is no doubt doubly so for Node-RED users who it appears mainly get it bundled into a container from Home Assistant or use the install script to pull both NodeJS and Node-RED)

I suspect the only place to get the real market share numbers would be npmjs's request logs.

I've set dependabot to only offer updates over a few days old (but I have bigger problems with that as it tends to mostly just patch the pacakge-lock.json file which doesn't get shipped with packaged modules so every person who installs it runs the risk of pulling the very latest every time)

Surely, since the tarball exists, the correct thing to do would be to create a new version and publish that to npm and use it (assuming the original license allowed it) - or even incorporate the code directly into the node package if it isn't used elsewhere.

People do what is most expedient at the time to fix what is broken, and also in this current climate to avoid becoming responsible for maintaining yet another public package. In fact I find this approach is probably actually better, it's basically acting like a closed source bundle which means you are unlikely to get a hoard of AI scipt kiddies beating your door down with BS security reports because they haven't even read the output of the tools, let alone understood it.

You are free to choose to not use this package, but I understand it is one of the better Modbus nodes, so you either accept it and have well working nodes or use a worse node.

I am really starting to see the appeal of some of the closed source paid for packages in this industrial space. But hey, these are just my current Sunday afternoon thoughts on the downward trajectory of this space.

10

This is what Gemini is saying:

Package Manager Estimated Market Share / Usage Estimated Download Volume Primary Strengths & Characteristics
npm 65% – 75% ~100M+ weekly Bundled with Node.js; default choice for production and enterprise.
pnpm 12% – 18% ~12M weekly Highly popular in monorepos; uses content-addressable storage to cut disk space.
Yarn (v1 Classic & v4 Berry) 8% – 12% ~6M weekly Pioneer of parallel installs and yarn.lock; heavily used in established projects.
Bun 2% – 5% ~2M weekly Integrated runtime and ultra-fast package manager; rapidly growing for new projects.

So the others are certainly a minority but I've seen them used in some pretty large projects.

Yes, but not relevant to the likelihood of npm backing off the changes.

Which is why users need to set the global setting too.

While I understand the position, I don't feel it applies in this specific instance give the organisation that is publishing the nodes, they are selling commercial solutions as far as I can see so this feels a bit like laziness rather than a measured decision. Of course, that is just my opinion based on an external view. It is a perception - but perceptions can be the difference between a sale and a miss.

Thankfully I don't currently need any modbus stuff. However, this seems to be a significant weakness for Node-RED in general (and perhaps FlowFuse specifically) given the number of people who are raising issues and clearly using modbus. I'm only sharing my perceptions.

Paying for convenience is a tried and trusted approach. :smiley:

Sadly, in the current environments, lack of visibility might be worse?

A sad thing to be happening if this is correct. And somewhat against the grain given that AI is helping many projects (my own included) to move away from proprietary solutions or even 3rd-party open source to using primary source. I've been getting rid of a bunch of external dependencies over the course of this year.

Anyway, as you say, just spitballing on a quiet Sunday afternoon.