Installing a module fails to install dependencies?

tve · 18 March 2022 22:12

I'm having difficulties with NPM dependency installation. It's not happening the way I would hope it would... Specifically:

node-red-fd-corewidgets depends on node-red-flexdash
if I install node-red-fd-corewidgets using the palette manager then npm installs both modules:

    bash-5.0$ ls /data/node_modules/@flexdash
    node-red-fd-corewidgets  node-red-flexdash

but the palette manager doesn't show node-red-flexdash installed:

image788×420 14.5 KB
the nodes of node-red-flexdash don't show up in the palette
when I instantiate a node from node-red-fd-corewidgets it needs a config node from the other repo, and the input field that should show a drop down with the config nodes or "Add a new config node..." doesn't work
if I go back to the palette manager and manually install node-red-flexdash then its nodes get recognized, become available in the palette, and everything works

Am I doing something wrong or is there a bug or is this "as designed"?

Node-RED log:

Welcome to Node-RED
===================

18 Mar 22:00:38 - [info] Node-RED version: v2.2.2
18 Mar 22:00:38 - [info] Node.js  version: v14.18.2
18 Mar 22:00:38 - [info] Linux 5.13.0-27-generic x64 LE
18 Mar 22:00:38 - [info] Loading palette nodes
18 Mar 22:00:39 - [info] Settings file  : /data/settings.js
18 Mar 22:00:39 - [info] Context store  : 'default' [module=memory]
18 Mar 22:00:39 - [info] User directory : /data
18 Mar 22:00:39 - [warn] Projects disabled : editorTheme.projects.enabled=false
18 Mar 22:00:39 - [info] Flows file     : /data/flows.json
18 Mar 22:00:39 - [warn]
18 Mar 22:00:39 - [info] Server now running at http://127.0.0.1:1880/
18 Mar 22:00:39 - [info] Starting flows
18 Mar 22:00:39 - [info] Started flows
18 Mar 22:01:08 - [info] Installing module: @flexdash/node-red-fd-corewidgets, version: 0.1.4
18 Mar 22:01:19 - [info] Installed module: @flexdash/node-red-fd-corewidgets
18 Mar 22:01:19 - [info] Added node types:
18 Mar 22:01:19 - [info]  - @flexdash/node-red-fd-corewidgets:date-time
18 Mar 22:01:19 - [info]  - @flexdash/node-red-fd-corewidgets:gauge
...
18 Mar 22:01:19 - [info]  - @flexdash/node-red-fd-corewidgets:value-sequence
18 Mar 22:01:19 - [info]  - @flexdash/node-red-fd-corewidgets:wind-plot

knolleary · 18 March 2022 22:31

When you install a module, the runtime only knows you want to use that one module. So once the install completes it loads that module.

The runtime doesn't rescan the entire npm module tree to see what other modules got installed that might also be Node-RED modules.

A restart of Node-RED would trigger a full rescan and find the extra modules.

There is a long standing item in the backlog to support this type of scenario - but there hasnt been much demand so it hasn't got off the backlog.

tve · 18 March 2022 22:42

I see, thanks for the quick reply! Is there a call I can make for now that would scan the other package for nodes?

hardillb · 18 March 2022 22:55

No, the only option is to restart node-red, or ask the manage pallet option to install the dependencies again

tve · 18 March 2022 23:46

Looks like the backlog item is Allow node modules to have node modules as dependencies · Issue #569 · node-red/node-red · GitHub
Am I right that what's really missing is a call

registry.addModule('node-red-flexdash')

github.com

node-red/node-red/blob/master/packages/node_modules/@node-red/registry/lib/loader.js#L475

      
        
            
            
    return Promise.all(promises).then(function() {
                    if (settings.available()) {
                        return registry.saveNodeList();
                    } else {
                        return;
                    }
                });
            }
            
            
function addModule(module) {
                if (!settings.available()) {
                    throw new Error("Settings unavailable");
                }
                var nodes = [];
                var existingInfo = registry.getModuleInfo(module);
                if (existingInfo) {
                    // TODO: nls
                    var e = new Error("module_already_loaded");
                    e.code = "module_already_loaded";
                    return Promise.reject(e);

?

knolleary · 18 March 2022 23:52

If you're looking for API calls you could make, then no. That is a deeply internal API and shouldn't be called directly.

tve · 19 March 2022 00:10

I was wondering about starting a PR, but it looks like the enthusiasm for getting this fixed is near zero, so be it...

knolleary · 19 March 2022 00:32

It would be great to get it fixed to support this behaviour.

My lack of enthusiasm was for someone to hack something into their node to try and bypass what we have. Sadly that happens quite often - people choose to do their own thing rather than contribute back.

If this is something you're interested in contributing to, that would be awesome. I'd be happy to give some more detailed pointers - I'm just not in a position to do that right now.

tve · 19 March 2022 01:31

Hmm, trying to understand what's needed. So we land in installModule:

github.com

node-red/node-red/blob/master/packages/node_modules/@node-red/registry/lib/installer.js#L245-L247

      
        
            } else if (!isUpgrade) {
                log.info(log._("server.install.installed",{name:module}));
                return require("./index").addModule(module).then(reportAddedModules);

and then it proceeds to addModule, which has the following intriguing loop:

github.com

node-red/node-red/blob/master/packages/node_modules/@node-red/registry/lib/loader.js#L490-L508

      
        
            while(moduleStack.length > 0) {
                var moduleToLoad = moduleStack.shift();
                var files = localfilesystem.getModuleFiles(moduleToLoad);
                if (files[moduleToLoad]) {
                    moduleFiles[moduleToLoad] = files[moduleToLoad];
                    if (moduleFiles[moduleToLoad].dependencies) {
                        log.debug(`Loading dependencies for ${module}`)
                        for (var i=0; i<moduleFiles[moduleToLoad].dependencies.length; i++) {
                            var dep = moduleFiles[moduleToLoad].dependencies[i]
                            if (!registry.getModuleInfo(dep)) {
                                log.debug(` - load ${dep}`)
                                moduleStack.push(dep);
                            } else {
                                log.debug(` - already loaded ${dep}`)
                                registry.addModuleDependency(dep,moduleToLoad)
                            }
                        }
                    }
                }

But that already looks at dependencies!? Specifically moduleFiles[moduleToLoad].dependencies and adds those to the stack!? The dependencies seem to be established in

github.com

node-red/node-red/blob/master/packages/node_modules/@node-red/registry/lib/localfilesystem.js#L425

      
        
                nodes: {},
                plugins: {},
                resources: nodeModuleFiles.resources,
                icons: nodeModuleFiles.icons,
                examples: nodeModuleFiles.examples
            };
            if (moduleFile.package['node-red'].version) {
                nodeList[moduleFile.package.name].redVersion = moduleFile.package['node-red'].version;
            }
            if (moduleFile.package['node-red'].dependencies) {
                nodeList[moduleFile.package.name].dependencies = moduleFile.package['node-red'].dependencies;
            }
            if (moduleFile.usedBy) {
                nodeList[moduleFile.package.name].usedBy = moduleFile.usedBy;
            }
            nodeModuleFiles.nodeFiles.forEach(function(node) {
                nodeList[moduleFile.package.name].nodes[node.name] = node;
                nodeList[moduleFile.package.name].nodes[node.name].local = moduleFile.local || false;
            });
            nodeModuleFiles.pluginFiles.forEach(function(plugin) {
                nodeList[moduleFile.package.name].plugins[plugin.name] = plugin;

Does that mean all I'm missing is some json in my package.json? E.g. in addition to

  "dependencies": {
    "@flexdash/node-red-flexdash": ">=0.3.7"
  },

I should also have

  "node-red": {
    "dependencies": [
      "@flexdash/node-red-flexdash"
    ]
  }

Or am I completely mis-understanding what the code is trying to do?

Steve-Mcl · 19 March 2022 07:26

@tve, it would be worth a shot adding that to your package. Please let us know if that does the trick.

tve · 25 March 2022 04:14

Nope, doesn't work. In fact, even if I install the dependency manually that "dependencies" added to package.json will actively prevent the nodes from the dependency from being added to the palette.

tve · 10 April 2022 22:50

As a follow-up, the current state I'm facing is:

Flexdash needs 3 packages, prob 4:
- node-red-flexdash: core of flexdash, primarily configuration nodes
- node-red-flexdash-plugin: flexdash plugin due to bug described in How do Node-RED plugins work? (can't reliably have a plugin and a node in the same pkg)
- node-red-fd-corewidgets: core set of widget nodes, really separable from the core of flexdash as someone could install a different set of widgets
- node-red-flexdash-sidebar: sidebar for FlexDash, which perhaps should/could be integrated into the node-red-flexdash package, or perhaps not
While the instructions can tell someone to "npm install node-red-fd-corewidgets" and get everything else pulled in as dependencies, due to the bug described here this is not possible through the palette manager: the dependencies get pulled in at the npm level but not their nodes and thus users end up with missing nodes. Instead users have to select all 3 or 4 packages for install manually in the palette manager.
In order to allow the manual install of all the dependencies in the palette manager, I have to publish all these packages in the flow library, which due to the fact that it can't be automated is both painful and error prone (see Script to trigger flow library update after npm publish?)

If I'm missing a solution to all this I'd be grateful to hear!

Edit: Ticket about the failure to install nodes in dependencies: When installing packages using palette manager the nodes of dependencies are not discovered · Issue #3529 · node-red/node-red · GitHub

system · 9 June 2022 22:51

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
[ Flexdash ] [@flexdash/mode-red-fd-corewidgets ] Dashboard flexdash	11	234	2 April 2023
FlexDash Not Installing General flexdash	7	322	1 August 2023
Manage Palette Node install error General	4	927	30 July 2020
Palette manager install fails General	5	3619	5 January 2020
Downloading node palettes General	7	1282	13 March 2020

Installing a module fails to install dependencies?

Related topics