Retrieving node IDs for members of flows and subflows/subflow instances

[shimbone]

The metrics log seems to be the best source of what a NR server is doing at any given time. To maximize its benefit one needs to have a manifest of how the node IDs are members of individual flows, subflows, and subflow instances.

It appears that when trace logging is enabled, NR outputs exactly this data to its log file on restart. It looks something like this with a "table" for each flow and subflow instance.

14 Oct 15:00:41 - [trace] [subflow:1a545064.28bce] ------------------|--------------|-----------------
14 Oct 15:00:41 - [trace] [subflow:1a545064.28bce]  id               | type         | alias
14 Oct 15:00:41 - [trace] [subflow:1a545064.28bce] ------------------|--------------|-----------------
14 Oct 15:00:41 - [trace] [subflow:1a545064.28bce]  4944e0123614db3f | comment      | ec4c98c7.032bf8
14 Oct 15:00:41 - [trace] [subflow:1a545064.28bce]  3fcef4a2ece6dee6 | function     | 4958392a.c6a268
14 Oct 15:00:41 - [trace] [subflow:1a545064.28bce]  fe3da7efd9caee7d | switch       | b6293df6.36443
14 Oct 15:00:41 - [trace] [subflow:1a545064.28bce]  f62dd4943432a5ef | change       | 2aa585ac.7f583a
14 Oct 15:00:41 - [trace] [subflow:1a545064.28bce]  73bd56874daa5354 | change       | b1684a5c.181048
14 Oct 15:00:41 - [trace] [subflow:1a545064.28bce]  3222bb2dfad8bb22 | change       | 286c0768.1a9ea8
14 Oct 15:00:41 - [trace] [subflow:1a545064.28bce]  ba1e3de940d6e084 | change       | 2d7ed569.c82ada
14 Oct 15:00:41 - [trace] [subflow:1a545064.28bce]  71fb3033aae1e54f | function     | 659978c1.b1b718
14 Oct 15:00:41 - [trace] [subflow:1a545064.28bce]  8e2f2c56950b9e05 | change       | 5272a9c.7c1f858
14 Oct 15:00:41 - [trace] [subflow:1a545064.28bce]  7fd7e228221aa248 | template     | dc814d78.a1dda
14 Oct 15:00:41 - [trace] [subflow:1a545064.28bce]  a588f138e03d4140 | function     | 6694d9bd.eaba98
14 Oct 15:00:41 - [trace] [subflow:1a545064.28bce]  89925f5447eddf07 | switch       | dd285dbe.26ac8
14 Oct 15:00:41 - [trace] [subflow:1a545064.28bce]  962f07f412169c93 | function     | 3de0650d.7b995a
14 Oct 15:00:41 - [trace] [subflow:1a545064.28bce]  84f596e34782b523 | change       | a9406493.671828
14 Oct 15:00:41 - [trace] [subflow:1a545064.28bce]  986c714e7002d855 | subflow:9b5d0796.da2928 |
14 Oct 15:00:41 - [trace] [subflow:1a545064.28bce]  715fe568acbb3150 | subflow:8fef06e6.af0ca8 |
14 Oct 15:00:41 - [trace] [subflow:1a545064.28bce] ------------------|--------------|-----------------

The problem I keep bumping into is that the virtual node IDs (that is, the node IDs that are created by NR for the member nodes of subflow instances-- and are therefore no assigned IDs by the editor) seem to change when the server is restarted, or even in some (all?) circumstances, when nodes are deployed. This makes it difficult to take a given chunk of the metrics log and understand it because it is so difficult to page back through the log far enough to find where the manifests were last output.

Does anyone know if there is there any other way to get this data? I was thinking of maybe an API endpoint? Or maybe a settings.js directive that would cause the manifest data to be written to a file each time the NR server changes it?

Thanks in advance!

[knolleary]

Hi @shimbone

Whilst it seems obvious now you've pointed it out, this disconnect between the generated ids and the metrics output is something we need to address. I'm not sure there is a sensible way to workaround it today.

There are a few options for us to look at.

Nodes already have an internal _path property that unambiguously identifies the node by including all the parent flow/subflow ids. We could add this to the metrics output.

Another option would be to make the generated ids more deterministic.

I'll make sure we do something on this in the 3.1 release due in the next month.

[shimbone]

@knolleary That sounds great!

My particular use case is that I'll have a server act up with an unusual increase in load and assuming I previously enabled metrics logging, I'll take a chunk of the metrics file by redirecting node-red-log to a file for a period of time while the server is misbehaving. I've also added

sudo timeout 30 node-red-log > /home/pi/.node-red/NRHead.log 2>&1 &

to node-red-restart and node-red-pi so that I capture the first 30 seconds of logging (and therefore the node -> flow and subflow mapping) after running a restart or rebooting the server. This works well as long as I haven't done a node deploy in the mean time. Note that I can't do a restart of NR before taking the metrics sample or I'd lose the state of the flows are causing the problem, which can take days or weeks to surface after restarting.

Making the generated ids more deterministic so they didn't change would work great for me. Adding path to metrics logging would also work really well as long as each log entry contained the path. If the information was was output similar to how trace logging does it today, but moved to the metrics log, while I think that would make more sense, it would still be tricky for me to figure out how to capture the information.

Thanks again for the consideration!