Auto-upgrade of palette and its dependencies?

Background

Heyo! Posting here to ask about a potential auto-upgrade use-case that seems to be overlooked.

Specifically, multiple node types that talk to external services often break:

  • protocols get deprecated/discontinued
  • external services have API BC breaks
  • newer/better/more reliable communication methods are introduced
  • security issues lead to necessary upgrades in handshaking and filtering logic

All of the above are generally part of the logic of custom palette nodes, and their dependencies (node_modules).

I already use unattended upgrades on my host OS ( Unattended Upgrades - auto update OS ), then I run node-red as a docker container, and I keep it up-to-date with watchtower ( GitHub - containrrr/watchtower: A process for automating Docker container base image updates. ).

Still, that only leads to the "outer shell" (node-red itself) being updated, while the palette lags behind. Every few weeks/months, I have to log into the node-red instance, and manually upgrade the palette, then restart the container, usually because something stopped working. After an upgrade + restart, all good.

This is relatively OK: kudos to the team for providing an easy way to perform one-click upgrades :slight_smile:

Still, this does not cover security issues and general stability upgrades that could happen in background.

Idea

My idea is that we could have an "auto-upgrade" schedule in node-red:

  • run as a cronjob (nightly or similar)
  • run as an internal flow (timer-based, user can enable/disable it) that talks to the node-red internal API
  • have a CLI script that users can trigger manually
  • have a version history of package-lock.json or yarn.lock files (not sure which one node-red uses for the palette, nor where they are located in internals)

Run as cronjob

Depending on a setting, we could allow auto-upgrades to happen "out of the box", after a big disclaimer "it will break, and we can't do anything about it" (it's OSS, after all).

This could literally be a checkbox in the palette option, with minimal config

Run as a flow

node-red is amazing at automating tasks: this could be an "eat your own dogfood" flow that talks to the node-red API and effectively does what the UI interactions in admin currently do, when upgrading flows.

It would then be trivial to add alerting and further nodes to such an internal flow (add a downstream node with your preferred logging system).

Users could also enable/disable the flow itself by themselves.

Perhaps nodes that check when a palette upgrade is available?

Have it as CLI script

Having a CLI script that does an "update all" and then "restart" would be a good option (although unsure how complex it would be to handle it in docker): also here, the user can decide to have a flow that executes a shell script, or they could have their own crontab entries.

Have a history of upgrades

Upgrades come with risk, and to mitigate risk, it would be a good idea to stash away the previous versions of package-lock.json or yarn.lock, to allow for manual rollback.

Since this is an edge case, perhaps leaving a README.md with instructions on how to roll-back a palette would suffice.

1 Like

If you are auto-updating Node-RED, you can auto-update all of the installed nodes as well.

cd ~/.node-red
npm update

Obviously though, you have to restart Node-RED afterwards since the nodes need to re-initialise.

You should also note that if you are auto-updating Node.js past major version numbers, you may need to rebuild some of your npm packages.

Oh wow, that is indeed simple!

Only difference seems to be that the docker image has:

  • /usr/src/node-red as ~, and contains the package.json for the main Node-RED instance
  • /data containing all flow information.

Perhaps a good way to handle this could be to have /data managed as git repo, and updated by dependabot, but really unsure about it.

Running npm commands on /data seems viable, and the tooling is already mature: just needs a viable way to restart the whole container too :thinking: