Script to trigger flow library update after npm publish?

General
1

Has someone written a script to trigger a flow library update after publishing a module to npm?

The npm part is fully automated on git push and I routinely forget to manually go to the flow library, find my module, log in, press the button... (Never mind the hour it often takes to actually update, by which time I also forget to test it...) :roll_eyes:

2

There used to be an automatic routine that scanned npm for the keyword node-red but (for reasons I forget - probably "flakiness") it was switched off.

Nick will know more. But for now, you will just have to remember :slight_smile:

3

I would not associate a smiley with that...

4

It is what it is. It was not intended to be dismissive, more a subtle nod nod wink wink to say "I realise this it is not what you want to hear but for reasons i forget right now, this feature was disabled (some time back) & is not currently on anyone's radar"

1 Like
5

OK. I'm also finding the delay after pressing the button manually extremely frustrating. It may seem that in the big scheme of things it's not that much of a problem if the update happens an hour later, but when I'm publishing something and writing a post to describe how to install & use my stuff it actually is extremely frustrating. I like to walk through the steps before posting them and so that basically means I get to switch to something else and then have to remember an hour later to look and come back. Of course more often than not there's some small issue and it repeats.

6

Are you sure it’s an hour ? That was when it was automatic. For me, now that it is manual, it is normally more like a minute, until the flows page updates.

7

What I'm looking at is when I have the Node-RED flow editor open, with the palette manager import tab open, a search term entered, and I click the refresh button and it shows the new version. Yes, the web page updates within a few seconds, but that doesn't actually make it available to NR.

Also, when I submit multiple update requests, either due to a quickly-fixed mistake or because I need to update multiple packages I believe I have seen it update within a couple of minutes for the first one, but then take an eternity for the second.

It's not like this is easy to measure, sitting there clicking the refresh button in Node-RED every few seconds...

8

If you are simply testing the install from the pallet manager instead of publishing to npm, you could always use npm pack & install using the upload tgz button. This has the benefit of not having to wait for the flows library to pick up the new version and also not having lots of updates going to npm.

Another alternative is to npm install path/to/your/contrib/node. This creates a sim link to your development directory in the node_modules directory meaning any source code changes you make are picked up by a simple restart of node red. A benefit of this method is you can actually step debug your own node source code.

9

I whipped up a script to test. It checks about every 30 seconds the flow library web page and the catalogue used by Node-RED. I updated one of my packages to 0.4.9 and let it run:

Sun 10 Apr 2022 09:26:38 PM PDT
        <h1 class="flow-title" style="margin-bottom: 10px;">@flexdash&#x2F;node-red-flexdash <span c
lass="flow-version">0.4.8</span></h1>
"@flexdash/node-red-flexdash"
"0.4.8"
Sun 10 Apr 2022 09:27:09 PM PDT
        <h1 class="flow-title" style="margin-bottom: 10px;">@flexdash&#x2F;node-red-flexdash <span c
lass="flow-version">0.4.9</span></h1>
"@flexdash/node-red-flexdash"
"0.4.8"
Sun 10 Apr 2022 09:27:40 PM PDT
        <h1 class="flow-title" style="margin-bottom: 10px;">@flexdash&#x2F;node-red-flexdash <span c
lass="flow-version">0.4.9</span></h1>
"@flexdash/node-red-flexdash"
"0.4.8"

...

Sun 10 Apr 2022 09:31:16 PM PDT
        <h1 class="flow-title" style="margin-bottom: 10px;">@flexdash&#x2F;node-red-flexdash <span c
lass="flow-version">0.4.9</span></h1>
"@flexdash/node-red-flexdash"
"0.4.9"

So that took 4-5 minutes, which is reasonably OK. But then I pushed 0.4.10:

Sun 10 Apr 2022 09:32:49 PM PDT
        <h1 class="flow-title" style="margin-bottom: 10px;">@flexdash&#x2F;node-red-flexdash <span c
lass="flow-version">0.4.10</span></h1>
"@flexdash/node-red-flexdash"
"0.4.9"

...

Sun 10 Apr 2022 09:53:55 PM PDT
        <h1 class="flow-title" style="margin-bottom: 10px;">@flexdash&#x2F;node-red-flexdash <span c
lass="flow-version">0.4.10</span></h1>
"@flexdash/node-red-flexdash"
"0.4.9"
Sun 10 Apr 2022 09:54:25 PM PDT
        <h1 class="flow-title" style="margin-bottom: 10px;">@flexdash&#x2F;node-red-flexdash <span c
lass="flow-version">0.4.10</span></h1>
"@flexdash/node-red-flexdash"
"0.4.10"

That now took more than 20 minutes, which means being already somewhere else and having to come back to pick up the task.

I ran it one more time by updating to 0.4.11 and it took 25 minutes.

While in isolation, this may not seem such a big deal, when you're writing a post or documentation and need to test the "final product" to make sure it actually works as described and you hit this ~ 25 minute delay every time you adjust something because you have to push 2 packages, and then there's some minor problem you want to fix and it starts all over again, and you actually don't know it's 25 minutes you have to wait 'cause you haven't yet written a script to time the situation.. it becomes really frustrating real quick. For me writing the flexdash integration has become "death by 1000 paper cuts"...

10

Thanks for the tip. Can you guarantee that the upload tgz button is identical to installing from the flow library? I've gotten to the point where I absolutely do not trust that kind of assumption anymore...

I'm very well aware of the npm install and symlink options, however, I've come to find out the result is absolutely not the same as using the palette manager.

11

The pallet manager, the upload tgz & install path methods all use npm under the hood. There should not be any difference except that install path creates a sim link instead of installing the files. But to answer your question, no without a thorough review of the core, I cannot guarantee the same code paths are executed.

If you are seeing problems or differences, it would be great if you could report these but from my experience if it works using the install path or upload tgz methods then it works from the pallet manager and npm. Every time.

The install path method is my primary method of developing since I can step debug in VS code. Only once I am happy with the operation do I npm publish. That has been successful for me for several years.

12

Yeah, I thought the same until: Installing a module fails to install dependencies?

I just tested the npm pack + upload (took me a minute to find the upload button...) and I can joyfully report that it seems to be bug-compatible with the install-from-library method.

Is there documentation or do you have some short write-up on how to do the step debug in vscode, preferably when using docker to run NR?

NB: I'm not publishing before debugging my code, I'm publishing 'cause I have dependencies between 4 packages and am working with a collaborator so on an almost daily basis I need to make sure that when I write "I pushed a fixed version for you to test" it actually works...

Also, when I'm publishing for someone else, while I can test that dependencies work using the npm pack, that doesn't actually give me an end-to-end check that it's gonna work for others 'cause I may have forgotten/botched some update of a dependent package (yup, I'm already guilty). If I could update the flow library automatically as part of CI then this would not be an issue, but since I can't...

13

Unfortunately not using docker but you should be able to adapt my method to a remote container...

If I get a moment later today or this week, I'll try with a container & let you know how it goes (no promises im afraid)

2 Likes
14

Just to fill in some blanks, the catalogue used by the palette manager in the editor is regenerated every 30 minutes - flow-library/build.yml at e50418d36dbdfdd1e8dcbe3e44103a476a483ae1 · node-red/flow-library · GitHub

It would be fairly simple to change that to get triggered whenever a node is updated in the library.

1 Like
15

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.