Problem detected when migrating subflows between machines

[Trying_to_learn]

Yes, how I am doing it may be making my own life miserable, but it's the only way I know.

Details - as best I can give:
EXPORTING FROM: NR 4.0.9
IMPORTING TO: NR 3.0.2 and 4.0.9

I would import it, REPLACE the node, and do a bit of fixing up of other stuff.

ALSO: The subflow had been severely altered in that how many outputs there are/were.
Reduced.

Re-wire it and DEPLOY.

Things weren't working.

I'd inject messages and NOTHING happened.
After a few attempts, I would open the subflow and look at it's internals.

Half/all/most of the wires were missing.

Now, I'm not sure if I also re-exported it or just imported it again.

All wires intact.

WEIRD.

Just to put it out there.

[Trying_to_learn]

HUGE DEVELOPMENT

I don't understand what is going on.
I am only reporting what I'm seeing.

I said that I import subflows and sometimes the internal wires are either all or some missing.

I was working on the tab/flow which such a subflow and had only just updated the machine to NR 4.x

Me debug is set to this tab.
I'm on that tab and I am getting WEIRD messages.

W.E.I.R.D

TypeError: sendEvent.destination.node.receive is not a function
    at deliverMessageToDestination (/usr/lib/node_modules/node-red/node_modules/@node-red/runtime/lib/flows/Flow.js:811:40)
    at Immediate._onImmediate (/usr/lib/node_modules/node-red/node_modules/@node-red/runtime/lib/flows/Flow.js:827:21)
    at process.processImmediate (node:internal/timers:476:21)

But wanting to remain focused as possible.... (Look! Squirrel!) I was wanting to get to the bottom of why the subflow wasn't working.

Lowe and behold:
I opened the subflow....

NADA!
NOTHING!

drum roll and build up!

So, this is getting very interesting.

Again:

I don't know/understand what's going on.

I'm sharing it to (maybe) help someone else suffering the same problem/s.

I'll re-import the subflow and post back.

[Trying_to_learn]

Well, it made a liar of me.

Didn't fix the problem.

Still happening and there is nothing to identify which node is causing it.

[Sean-McG]

I don't use subflows, but clearly yours are "messed up" (technical term).

This is almost certainly the cause of the error messages in the logs, as NR is not able to route messages via missing wires and nodes.

How this has occurred I'm afraid I don't know, and will defer to others !

[Trying_to_learn]

Steve, no problems.

Thanks.

No offence taken. I know I'm messed up.

But HOW it (NR, the node, or someone) got access to that path....
There are 4 files there.

They are all .json and don't seem malicious.

Just ROGUE.

Could you maybe point me to where a useful log would be?
I have looked in the places I know but they are more install logs and npm logs.

[Trying_to_learn]

(Only to not go back and edit pervious posts)
(I should also dig just that bit deeper.)

I was wrong again.

Just opened the subflow - a third time - and it is STILL missing wires!

Be right back with more soon.

[Trying_to_learn]

Third/fourth? time lucky.

All the wires are now there on THAT subflow.

Still getting the rogue messages.
(Don't know if they should be a separate thread)

I quickly checked a couple of other subflows on that tab.... All seem ok.

I'll do a bit more comparing now.

[Trying_to_learn]

Ok, I have copied the Offending files to my machine.

They are all .json files.

And dated recent.

I opened a new tab and tried to import the Flow.js.
Wouldn't.

Anyone recognise this code?
(Incomplete. Too long to post)

/**
 * Copyright JS Foundation and other contributors, http://js.foundation
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 * http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 **/

const clone = require("clone");
const redUtil = require("@node-red/util").util;
const events = require("@node-red/util").events;
const flowUtil = require("./util");
const context = require('../nodes/context');
const hooks = require("@node-red/util").hooks;
const credentials = require("../nodes/credentials");

let Subflow;
let Log;
let Group;

let nodeCloseTimeout = 15000;
let asyncMessageDelivery = true;

/**
 * This class represents a flow within the runtime. It is responsible for
 * creating, starting and stopping all nodes within the flow.
 */
class Flow {

    /**
     * Create a Flow object.
     * @param {[type]} parent     The parent flow
     * @param {[type]} globalFlow The global flow definition
     * @param {[type]} flow       This flow's definition
     */
    constructor(parent,globalFlow,flow) {
        this.TYPE = 'flow';
        this.parent = parent;
        this.global = globalFlow;
        if (typeof flow === 'undefined') {
            this.flow = globalFlow;
            this.isGlobalFlow = true;
        } else {
            this.flow = flow;
            this.isGlobalFlow = false;
        }
        this.id = this.flow.id || "global";
        this.groups = {}
        this.groupOrder = []
        this.activeNodes = {};
        this.subflowInstanceNodes = {};
        this.catchNodes = [];
        this.statusNodes = [];
        this.path = this.id;
        // Ensure a context exists for this flow
        this.context = context.getFlowContext(this.id,this.parent.id);
        
        // env is an array of env definitions
        // _env is an object for direct lookup of env name -> value
        this.env = this.flow.env
        this._env = {}
    }

    /**
     * Log a debug-level message from this flow
     * @param  {[type]} msg [description]
     * @return {[type]}     [description]
     */
    debug(msg) {
        Log.log({
            id: this.id||"global",
            level: Log.DEBUG,
            type:this.TYPE,
            msg:msg
        })
    }

    /**
     * Log an error-level message from this flow
     * @param  {[type]} msg [description]
     * @return {[type]}     [description]
     */
    error(msg) {
        Log.log({
            id: this.id||"global",
            level: Log.ERROR,
            type:this.TYPE,
            msg:msg
        })
    }

    /**
     * Log a info-level message from this flow
     * @param  {[type]} msg [description]
     * @return {[type]}     [description]
     */
    info(msg) {
        Log.log({
            id: this.id||"global",
            level: Log.INFO,
            type:this.TYPE,
            msg:msg
        })
    }

    /**
     * Log a trace-level message from this flow
     * @param  {[type]} msg [description]
     * @return {[type]}     [description]
     */
    trace(msg) {
        Log.log({
            id: this.id||"global",
            level: Log.TRACE,
            type:this.TYPE,
            msg:msg
        })
    }

    /**
     * [log description]
     * @param  {[type]} msg [description]
     * @return {[type]}     [description]
     */
    log(msg) {
        if (!msg.path) {
            msg.path = this.path;
        }
        this.parent.log(msg);
    }

    /**
     * Start this flow.
     * The `diff` argument helps define what needs to be started in the case
     * of a modified-nodes/flows type deploy.
     * @param  {[type]} msg [description]
     * @return {[type]}     [description]
     */
    async start(diff) {
        this.trace("start "+this.TYPE+" ["+this.path+"]");
        var node;
        var newNode;
        var id;
        this.catchNodes = [];
        this.statusNodes = [];
        this.completeNodeMap = {};


        if (this.isGlobalFlow) {
            // This is the global flow. It needs to go find the `global-config`
            // node and extract any env properties from it
            const configNodes = Object.keys(this.flow.configs);
            for (let i = 0; i < configNodes.length; i++) {
                const node = this.flow.configs[configNodes[i]]
                if (node.type === 'global-config' && node.env) {
                    const globalCreds = credentials.get(node.id)?.map || {}
                    const nodeEnv = await flowUtil.evaluateEnvProperties(this, node.env, globalCreds)
                    this._env = { ...this._env, ...nodeEnv }
                }
            }
        }

        if (this.env) {
            this._env = { ...this._env, ...await flowUtil.evaluateEnvProperties(this, this.env, credentials.get(this.id)) }
        }

        // Initialise the group objects. These must be done in the right order
        // starting from outer-most to inner-most so that the parent hierarchy
        // is maintained.
        this.groups = {}
        this.groupOrder = []
        const groupIds = Object.keys(this.flow.groups || {})
        while (groupIds.length > 0) {
            const id = groupIds.shift()
            const groupDef = this.flow.groups[id]
            if (!groupDef.g || this.groups[groupDef.g]) {
                // The parent of this group is available - either another group
                // or the top-level flow (this)
                const parent = this.groups[groupDef.g] || this
                this.groups[groupDef.id] = new Group(parent, groupDef)
                this.groupOrder.push(groupDef.id)
            } else {
                // Try again once we've processed the other groups
                groupIds.push(id)
            }
        }

        for (let i = 0; i < this.groupOrder.length; i++) {
            // Start the groups in the right order so they
            // can setup their env vars knowning their parent
            // will have been started
            await this.groups[this.groupOrder[i]].start()
        }

        var configNodes = Object.keys(this.flow.configs);
        var configNodeAttempts = {};
        while (configNodes.length > 0) {
            id = configNodes.shift();
            node = this.flow.configs[id];
            if (!this.activeNodes[id]) {
                if (node.d !== true) {
                    var readyToCreate = true;
                    // This node doesn't exist.
                    // Check it doesn't reference another non-existent config node
                    for (var prop in node) {

[Trying_to_learn]

This is group.js
(Short)

const flowUtil = require("./util");
const credentials = require("../nodes/credentials");
const clone = require("clone");

/**
 * This class represents a group within the runtime.
 */
class Group {

    /**
     * Create a Group object.
     * @param {[type]} parent     The parent flow/group
     * @param {[type]} groupDef   This group's definition
     */
    constructor(parent, groupDef) {
        this.TYPE = 'group'
        this.name = groupDef.name
        this.parent = parent
        this.group = groupDef
        this.id = this.group.id
        this.g = this.group.g
        this.env = this.group.env
        this._env = {}
    }

    async start() {
        if (this.env) {
            this._env = await flowUtil.evaluateEnvProperties(this, this.env, credentials.get(this.id))
        }
    }
    /**
     * Get a group setting value.
     * @param  {[type]} key [description]
     * @return {[type]}     [description]
     */
    getSetting(key) {
        if (key === "NR_GROUP_NAME") {
            return this.name;
        }
        if (key === "NR_GROUP_ID") {
            return this.id;
        }
        if (!key.startsWith("$parent.")) {
            if (this._env.hasOwnProperty(key)) {
                return (this._env[key] && Object.hasOwn(this._env[key], 'value') && this._env[key].__clone__) ? clone(this._env[key].value) : this._env[key]
            }
        } else {
            key = key.substring(8);
        }
        return this.parent.getSetting(key);
    }

    error(msg) {
        this.parent.error(msg);
    }

    getContext(scope) {
        return this.parent.getContext(scope);
    }
}

module.exports = {
    Group
}

ALL the files were in (path)
/usr/lib/node_modules/node-red/node_modules/@node-red/runtime/lib/flows/

Way beyond what I thought was allowable.

[xx_Nexus_xx]

I had a similar issue a while back , when moving code (incl subflows) from my dev machine to my prod. I check the code on a WIN and MAC machine and the lines were missing in both systems

A full shutdown of my PROD system solved the issue.

[Trying_to_learn]

I'm not sure I understand what you did to fix it.

[Trying_to_learn]

I notice a very similar problem happened a long time ago:

Something similar

[Trying_to_learn]

ANOTHER DEVELOPMENT

So thing are really getting bad here.

I explained how I imported the subflow; all the wires aren't getting there.

After several attempts it was all working.

Yeah, nice.

BUT!

The machine started to get these rogue messages.
Out of desperation, I updated it to the lates NR.
4.0.9 NJS 18.x
Buster.

Messed around a couple of times as it hung.

Got it up and running again.

It isn't working 100%.

Opened up the most recent imported subflow. (Machine health V2) as per this thread.

NO WIRES!

HOW???!!!

This is very disheartening.

Ok, more:
(still with me?)

I FINALLY for the subflow all imported.
All wires.
Everything.

Still getting those errors/messages.

Opened it up. Yes, all wires there.

Copied all the nodes out of the subflow, and put them at flow level.
Disabled the subflow, wired the nodes as though they were the subflow.

WORKS!

THOUGHT

MAYBE - somewhere in the depths of all this:
The routine that handles subflows has lost its wheels (Aka broke)
Has it been looked at since..... ?????

(and I did wait to be sure before posting.)

It has (again) made a liar of me.

Still getting the rogue messages even with the subflow disabled.

THOUGH!

I do have other subflows on that tab.
Maybe this precipitated all this mess.

But on machine 3 which has the same subflow etc....
Same tab open. No errors.

[Trying_to_learn]

AND ANOTHER IMPORTANT UPDATE ON WHAT'S HAPPENING

Well, this is a weird one.

The subflow was DISABLED.
All the wires into/out of it were rerouted to the exported nodes that were sitting at the same level/layer as the others.

Note: It was DISABLED.
I left the wires going in/out of it.

(This is important)

The errors/messages kept happening.

I DELETED the wires going INTO it.

DEPLOYED.

Messages stopped!

Go figure.

Please - someone.

[xx_Nexus_xx]

My system is running on docker, so I completely restarted the NR docker ... full system restart for Node-Red

[Trying_to_learn]

I am stupid.

I am not understanding what you mean.

What? You simply go into the CLI/Terminal and node-red-restart?

[GogoVega]

This is the internal code of the runtime

[Trying_to_learn]

Sorry folks, I'm writing War and piece with this.

But...... Not to be outwitted.

All I feel I can do is Throw crap at it and take notes.

So let's start.

This is going to be long. But with pictures!

Base line:

Walk through
This is the part of the flow that does things.
The join points.... used to switch where the existing signal to go.
Up high is the exported insides of the subflow (green area)
Note time and debug window.
subflow input is NOT connected! (Important)
Also note the red/green lines showing WHO is connected.

Some time later.....
No errors.

Note how the wires have now moved and the top part is disconnected and the SUBFLOW input is now connected.

But note:

The subflow is still DISABLED
Now I'll press DEPLOY

Deployed.
Note times. 19:17

Time 19:18
ONE MINUTE
ERROR
subflow DISABLED!

Let's go further!
Wire removed to subflow.
Then DEPLOY

After deployment.
Errors wiped after I deploy to show any new ones.
NO errors.

Time passes.
No errors.
Note times.
Now let's look at it a bit more.
This is a copy of the subflow (duplicate) with some inputs.
DEPLOYED

Some time later.
No errors

More time.
No errors
I added a node to show the time, so you can see how long has gone by with no errors.

Injecting messages. Quite a few.
NO ERRORS

Still no errors

And still no errors.

and still none

Now we'll go back to the original subflow.
Note times, and wires into the subflow.
(yes a bit repetitive, but to show I am not hiding anything)

No errors

Move the wires and look at that!
ERRORS!

PLEASE....
SOMEONE....

What have I found?

[GogoVega]

Can you retry with a full deploy

[Trying_to_learn]

Gee I'm impressed you read it.

Here's what happened ITMT between me posting that and when you replied.

I copied the second subflow and put it below the first one.
Connected it and it works.

I have since wired up the rest and it works.

I'll leave the original one there for now and when I have caught my breath, I may give it a go.

But my head hurts with this happening as it has.