Merge flows with git

Hello,

I've been using Tortoise git for different projects and now that I work with Node-Red, I continued using it. But I found that merging Json file (flows) doesn't work well when some people work in Node-Red.

I wanted to try the built-in git in Node-Red, but it doesn't work here for now (probablement network config)

So, what is the best git tool to work with Node-RED?

If the answer is the built-in git tool, what network port I have to open to make it work? Or any other ideas why it doesn't work on a corporate location? When I try to clone a project, it just wait for 2 minutes, then I have an empty folder.

Thanks a lot

Hello,

I didn't get any help on this. I'm sure I'm not the only one in this situation.

  • Still need to know to best git client on Windowsto work with Node-RED Json file.
  • Also, if you have any hint with when I clonde a repositorie in Node-RED, I get nothing, no error, just a waiting for 2 minutes.

Thanks

The big problem is that the entire node-red runtime flow is defined in a single JSON text file. As you have already found out, merging text changes is not as straight-forward for text as it is for structured lines of code...

The "Projects" feature at least allows your team members to commit versions of the entire flow, which replace the previous version. If you weren't having problems with git, that would be a way to go. Node-RED uses whatever "git" command/system is installed on the host machine, so if you can clone a repo from the command line, it should work inside node-red also. Are there any git-related errors in the logs?

Assuming you do not want to use the built-in Projects, here are some ideas to make managing changes easier. Some tools like Beyond Compare make merging easier by showing the lines side-by-side. If you have not already done so, make sure to enable this section in your settings.js file:

// To enabled pretty-printing of the flow within the flow file, set the following
//  property to true:
flowFilePretty: true,

This only applies to the JSON structure of your flows, not the code within your function nodes, and so will not help with merging changes to that code.

There is also a contributed storage solution that converts your JSON flow to YAML before saving it to disk. Since the YAML format is a superset of JSON, it can also retain your newlines (which are not valid in JSON) -- meaning that function code will be easier to merge/compare.

Thanks for the answer Shrickus!

To be more precise:

  • We do use projects
  • flowFilePretty is enabled
  • cloning the Node-Red repositoy works in command line, but not in Node-Red
  • I don't see any log error about git. Even when activating all traces.

For now, we don't have much problem merging function code. Maybe we've been lucky. But that's good to know about yaml.

We usually have conflicts to every nodes that have been modified by multiple users. We always have conflicts on all nodes added at the end of the file. Git just doesn't understand that it have to put each new nodes on after the other.

We still want to try the git ui in Node-RED

We are trying to find a good merge tool that we could plug in tortoise git that could de a better job merging json by parsing it structure.

That is innevitable. One user adds a node at the end, and another user adds a node at the end. Git has no way of knowing which one has to go in first. In fact it doesn't matter, but git cannot know that.

1 Like

@Colin

That's why we need a tools that understand the json and will detect that there are new nodes and has to add them.

@shrickus
I tried the YAML contrib, but when I add the storageModule in my settings file, I get this error

Error loading settings file: C:\UBX_GIT\Server\node_modules\node-red\settings.js
Error: Cannot find module '../node-red/red/runtime/log'

Did you get this palette to work? I'm wondering if it is still valid as there's no recent update.

Thanks

I looked at the code for the Yaml contrib, as it has not been updated, it is not compatible with Node-Red anymore. It is also not compatible with projects. I did not worked a lot on this to make it work again as I'm not sure it is the way to go.

I think that the idea to have a new file structure would help versionning in git:

But no idea when it will be done.

Another idea would be to divide the flow json in multiple file. (one file per flow, on file per template/function node, one file per subflow). I did not found the good way to do it.

I ran into the same problem and made myself a bash script to help. It doesn't actually help to automate the merge, but it makes it easy to see the 3-way differences between the two branches and the merge-base. At least you can easily see what lines are modified in the function. You can then manually merge them.

Note that the pretty print is half hearted as it doesn't handle "; but it's good enough for me:)

    #!/bin/bash

    # This bash script creates 3 pretty version of the flows-*.json 
    # file: mine branch, their branch, merge-base branch.
    # Usage: in node-red directory:
    #   ../flow_merge_tool.sh <mine> <theirs>
    # It will generate 3 files.  branch names could be SHA1 too


    if [[ $# != 2 ]]; then
        echo Pretty node-red flow merge branches and base maker
        echo Usage: $0 \<branch 1\> \<branch 2\>
    else
        export B1SHA1=$(git show $1 | grep -E '^commit ' | sed 's/commit //' | sed -E 's/(.......).*/\1/')
        export B2SHA1=$(git show $2 | grep -E '^commit ' | sed 's/commit //' | sed -E 's/(.......).*/\1/')

        export BASESHA1=$(git merge-base $B1SHA1 $B2SHA1 | cut - -c -7)

        echo SHA1 $B1SHA1 is $1
        echo SHA1 $B2SHA1 is $2
        echo SHA1 $BASESHA1 is merge-base


        git show $B1SHA1:node-red/flows-my.json | sed 's/\\n/\n/g' | sed 's/\\"/"/g' > flow.$B1SHA1.mine.pretty
        git show $B2SHA1:node-red/flows-my.json | sed 's/\\n/\n/g' | sed 's/\\"/"/g' > flow.$B2SHA1.theirs.pretty
        git show $BASESHA1:node-red/flows-my.json | sed 's/\\n/\n/g' | sed 's/\\"/"/g' > flow.$BASESHA1.base.pretty

        ls -la flow.*.pretty
    fi