Serving my own catalog via Node-RED

[nlecaude]

Hello,
I'm working on my own Docker image and would like to have a custom node catalog available. Ideally I'd like to host that catalog myself from within Node-RED.
Is there some way I could access RED.httpAdmin from within the settings.js file ? Or perhaps I need to create a configuration node for this that would be invisible but open the endpoint there ?

Thanks !

[knolleary]

The only option via the settings file would be httpStatic to expose a directory via the http server - and then ensure the catalog is the only thing in there.

[nlecaude]

And if not via settings.js, could a configuration node work even if it’s not deployed in the flows ?

[knolleary]

Nothing requires you to call RED.nodes.registerType if you have nothing to register.

So you could create a module that contains something that looks like a node in that it has the .js and .html files - but leave the .html largely blank and the .js do whatever you need.

In 1.3 the new plugins feature arrives and would be a semantically better way of doing it.

[nlecaude]

Nice thanks ! I’ll do that for now and migrate to a plugin when it’s ready !

[dceejay]

I think @hardillb had a version with a catalogue in its own container as part of his multi-tenent how-to series of blog posts

[nlecaude]

I'll have a look !
My use case is a bit specific as the packages are hosted on Azure Devops and I'd rather have it in the main container but I'll check his blog post for inspiration !

[hardillb]

Yep, I've using Verdaccio private NPM repo, setup to make a call to a web endpoint when ever any of the packages get updated. This triggers the catalogue app to rebuild the catalogue.json

[nlecaude]

Ended up creating an 'invisible' node as @knolleary suggested with the following:

const axios = require('axios')

module.exports = function (RED) {
  RED.httpAdmin.get('/catalog', async (req, res, next) => {
    const options = {
      auth: {
        username: 'user@npm',
        password: process.env.NPM_REGISTRY_TOKEN
      }
    }
    const response = await axios.get('https://feeds.dev.azure.com/orgname/_apis/packaging/Feeds/orgname/packages?api-version=6.0-preview.1&includeDescription=true', options)
    const azureNodes = response.data.value.filter(pkg => pkg.name.includes('node-red-contrib'))
    const catalog = {
      name: 'Node-RED Catalog',
      updated_at: Date.now(),
      modules: azureNodes.map(pkg => {
        return {
          id: pkg.normalizedName,
          version: pkg.versions[0].normalizedVersion,
          description: pkg.versions[0].packageDescription,
          updated_at: pkg.versions[0].publishDate
        }
      })
    }
    res.end(JSON.stringify(catalog, '', 2))
  })
}

Works like a charm ! Thanks !

[nlecaude]

I've encountered a small bump in the road: Is the catalog loaded by the editor or by the runtime ?

[shrickus]

FWIW, I found a way to serve my own "curated" list of nodes from within my own node-red instance -- Here is the flow I'm using:

[{"id":"829c111f.31f9c","type":"http in","z":"636b4cd7.6c8974","name":"","url":"/catalog.json","method":"get","upload":false,"swaggerDoc":"","x":180,"y":500,"wires":[["84e9294b.c06cd8"]]},{"id":"84e9294b.c06cd8","type":"http request","z":"636b4cd7.6c8974","name":"https://catalogue.nodered.org/catalogue.json","method":"GET","ret":"obj","url":"https://catalogue.nodered.org/catalogue.json","tls":"","x":510,"y":480,"wires":[["69bb335a.fdd94c"]]},{"id":"69bb335a.fdd94c","type":"change","z":"636b4cd7.6c8974","name":"curated node filter","rules":[{"t":"set","p":"payload.name","pt":"msg","to":"zPaper Nodes Catalog","tot":"str"},{"t":"set","p":"payload.modules","pt":"msg","to":"(\t    $contribs := [\t        \"node-red-contrib-aftership\",\t        \"node-red-contrib-aws\",\t        \"node-red-contrib-github\",\t        \"node-red-contrib-google\",\t        \"node-red-contrib-jira\",\t        \"node-red-contrib-moment\",\t        \"node-red-contrib-npm\",\t        \"node-red-contrib-pdf\",\t        \"node-red-contrib-redis-ssl-support\",\t        \"node-red-contrib-salesforce\",\t        \"node-red-contrib-salesforce-connection-emitter\",\t        \"node-red-contrib-salesforce-platform-event\",\t        \"node-red-contrib-slack\",\t        \"node-red-contrib-xsltransform\",\t        \"node-red-contrib-zip\",\t        \"node-red-contrib-uibuilder\",\t        \"node-red-dashboard\"\t    ];\t    payload.modules[id.$match(/^node-red-node-/) or id in $contribs]\t)","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":830,"y":480,"wires":[["e0bfde23.781e7"]]},{"id":"bd1a45f8.074d88","type":"http response","z":"636b4cd7.6c8974","name":"","statusCode":"","headers":{},"x":1170,"y":500,"wires":[]},{"id":"e0bfde23.781e7","type":"switch","z":"636b4cd7.6c8974","name":"req?","property":"req","propertyType":"msg","rules":[{"t":"null"},{"t":"nnull"}],"checkall":"true","repair":false,"outputs":2,"x":1030,"y":480,"wires":[["c5fce441.cc9598"],["bd1a45f8.074d88"]]},{"id":"c5fce441.cc9598","type":"debug","z":"636b4cd7.6c8974","name":"catalog.json","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","x":1190,"y":460,"wires":[]},{"id":"d508b24d.b2dff","type":"inject","z":"636b4cd7.6c8974","name":"invoke the endpoint","topic":"","payload":"true","payloadType":"bool","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":190,"y":460,"wires":[["84e9294b.c06cd8"]]},{"id":"d83abe6.a20f84","type":"comment","z":"636b4cd7.6c8974","name":"Generate a curated list of nodes for Palette Manager","info":"","x":230,"y":400,"wires":[]},{"id":"379ea62b.75d2fa","type":"comment","z":"636b4cd7.6c8974","name":"Modify this list as necessary","info":"","x":840,"y":440,"wires":[]},{"id":"1e4c116b.28152f","type":"comment","z":"636b4cd7.6c8974","name":"Retrieve the full list of nodes from the flows library","info":"","x":520,"y":440,"wires":[]}]

It starts with the /red/catalog.json http endpoint on the left, fetches the actual node-red catalog, then uses a JSONata expression to filter only the ones I want people to see. Then i added this to the editorTheme section of my settings.js file:

        palette: {
            catalogues: ['http://localhost:1880/red/catalog.json']
        },

Of course, anyone can just change the filter to add whatever they want, so if real security is needed for your use case, then this would not be a good solution. But for our purposes, it serves to focus people on those nodes that we would expect them to use.

[nlecaude]

The one thing I wonder is I think the editor calls the catalog url directly, meaning if we put localhost it will only work if Node-RED runs on the same machine, so in theory it would need to be the publicly reachable URL.

[dceejay]

Does it work if the url is just relative ? IE just /red/catalog.json ?

[nlecaude]

Thanks Dave, that worked like a charm ! Feel dumb for not thinking about that first

[hardillb]

The catalogue is loaded by the editor so needs to be accessible by the browser

[nlecaude]

Yup my main issue was that I was using the full URL in my settings file but since the catalog is served in the container itself it would only work for localhost. Using a relative URL solved the issue.