Why multiple node_modules directories?

General
1

On my Raspberry Pi install, I used to find the core nodes in '/usr/lib/node_modules/node-red/nodes/core'

After running the UPGRADE command found in the Running on Raspberry Pi page, it looks like the core nodes have been moved to '/usr/lib/node_modules/node-red/node_modules/@node-red/nodes/core'

I poked around some more and found at least three paths for node-modules on my Raspberry Pi, and it looks like a lot of modules themselves are duplicated.
For example, there is a module folder for 'are-we-there-yet' in all of the following directories:
/home/pi/.node-red/node_modules
/usr/lib/node_modules
/usr/lib/node_modules/npm/node_modules

Is there a legitimate reason for the multiple locations, or are some of them redundant or leftover from the upgrade? I kind of get why there is a /home/pi... directory for MY nodes, but it seems silly to maintain the installed nodes in multiple locations.

Thanks in advance for any feedback.

2

I believe this is part of the rework being done on Node-RED in preparation for the move to v1. The core is being restructured. Doubtless this will resolve itself in time.

3

Yes and no.

The fact the core nodes have moved directory is because of the repackaging we did in 0.20.

The fact there are lots of node_modules directories is just how npm works.

Yes. /usr/lib/node_modules is the global npm module location - anything installed with -g as a global module.

You also then get locally installed modules - such as those you install just for node-red under /home/pi/.node-red/node_modules.

There are whole essays written about how npm lays out modules when running an install - I'm not going to go over it all, but here's the basics.

First thing to know is that when some code requires a module, it looks in the current directory for a node_modules directory, and looks in that directory for the module being loaded. If it doesn't find it, it moves to the parent directory and again looks for node_modules - and so on, all the way to the root of the file system.

Every module defines its own list of modules it depends on. One of the tasks npm has to do is to figure out how to install all of those dependencies.

In the early days of npm, it would install all dependencies below the modules own directory - so if you had module foo that depended on bar, and also module baz that also depended on bar, you'd end up with two copies of bar:

/usr/lib/node_modules/foo/node_modules/bar/
/usr/lib/node_modules/baz/node_modules/bar/

More recent versions of npm tries to reduce this duplication by installing dependencies as high as possible in the directory path. So you would get:

/usr/lib/node_modules/foo
/usr/lib/node_modules/baz
/usr/lib/node_modules/bar

However, if foo and baz depend on different versions of bar, npm hasto put individual copies of bar under the node_modules directory of the individual modules.

So yes you can end up with lots of copies of the same module installed; but npm does a lot of work to minimise that duplication.

2 Likes