Showing visual node diff when git repo is not at Project level

Core Development Feature Requests
1

We have one Git repository which contains a number of folders, one for each service. One of these folders is for Node-RED.

The Projects feature is very useful, especially the node diffs that show the changes to a Flow at a higher level of abstraction than a textual diff. When creating a Project, Node-RED also wants to create a Git repo at the level of the project. This doesn't work so well with our desired git pattern of having one 'monorepo'.

I have tried removing the .git/ folder from inside the project, and letting Node-RED pick up the Git repository several levels higher up. The Node-RED history panel picks up this git history, but no longer shows the visual diffs for changes to a Flow, it falls back to a text diff of the JSON.

I've looked at the diff code a little bit, and I think I can see why.

At line 1726 it checks if the file in the commit is the flow file. When you change the root of the git repository, this line compares commitOptions.project.files.flow ("flows.json") and file.file ("node-red/data/projects/Test/flows.json").

One way to fix this without risking breaking other code would be to introduce a new key in the options which contains the canonicalised path of the project, relative to the git root, (this may be different to the package.json root).

2

I wonder if a guy git submodule could solve this issue?

3

Hi @danielcompton

The projects feature does allow you to have the flow files in subfolders of the git repo - which was an attempt to allow some flexibility in how the repo could be structured. It does however require the package.json to be at the root of the repo - as that contains the configuration of where the flow files are.

But I can see the benefit of being able to have the whole project dir be contained below the root. The configuration for this would have to be stored outside of the project; so when you clone the project, if NR can't find the package.json it would have to ask you to identify where it was. Currently if it can't find package.json at the root it offers to create a default one for you - this would have been to be an alternative option at that point.

So... yes. This would be a good enhancement - just needs some thought around the UX.

4

Yep, that is the other option, but I'd prefer not to go down that route if I can help it.

I've found that everything worked fine when I had the package.json at a far deeper level than the git root, except for the aforementioned git diffing.

I think another possible option to get diffing working without changing Node-RED's architecture and expectations of where package.json lives would be to add another key like commitOptions.project.files.flowRelativeToGitRoot (with a better name) and use that when checking whether to show the custom diff viewer.

For more context, here's the directory layout I'm working with. Git is at the root of this directory.

$ tree -I "node_modules"
.
├── README.md
├── docker-compose.yml
├── nginx
│   └── app.conf
├── node-red
│   └── data
│       ├── flows.json
│       ├── flows_cred.json
│       ├── lib
│       │   └── flows
│       ├── package.json
│       ├── projects
│       │   └── Test
│       │       ├── README.md
│       │       ├── flows.json
│       │       ├── flows_cred.json
│       │       ├── new.json
│       │       └── package.json
│       ├── settings.js
│       └── yarn.lock

9 directories, 17 files

This is the relevant section from my docker-compose file:

  node-red:
    image: nodered/node-red-docker:0.19.5-v8
    command: "npm start -- --userDir /data/node-red/data"
    ports:
      - "1880:1880"
    links:
      - postgres
    volumes:
      - ./:/data

I mount the root of the host directory to /data on the container and then start Node-RED with a custom userDir folder.

5

Hi @danielcompton

I've just pushed a change that lets you have package.json to not be at the root of the git repo.

image
image.png1120×572 15.8 KB

The Project Settings dialog now has a Package entry that lets you select the package file to use in the repo - that determines the root of the project under which the flow/credential files can then be found.

This then means the merge and visual flow diff all work as expected as well.

6

Awesome, thanks! I'll check that out soon.

7

I'm also developing a node-red setup with docker-compose and was wondering if I should integrate the node-red projects feature or not. The benefit would be, as you have pointed out, the flow diff feature.

I could not get the feature that Knolleary introduced in this thread to work for me. After creating the project, deleting the project's .git and integrating the files into the outer git, I could not edit the package or flow locations that he pointed out in project settings: The UI entry fields would become empty and non-editable after clicking on the choose-file icon in project settings. Also, I cannot confirm what you saw, I do not see the git history of the outer project in the node-red git tab -- it remains empty in my case. See https://github.com/ianmacs/home/tree/1edf00ddcbb70f7735708c52065e79569679f192

But I have found a solution:

  1. After deleting the project's .git, move the project's files flow_cred.json, flow.json, .gitignore, package.json, README.md to the outer git repository's root directory:
.
├── docker-compose.yml
├── flow_cred.json
├── flow.json
├── .git
...
├── nodered
│   └── data
│       ├── .config.json
│       ├── .gitignore
│       ├── lib
│       │   └── flows
│       │       └── .gitignore
│       ├── package.json
│       ├── projects
│       │   ├── home
│       │   │   └── .gitignore
│       │   └── .sshkeys
│       │       └── .gitignore
│       └── settings.js
├── package.json
└── README.md

Then, through docker volume mounts, mount the root git directory as the node-red project directory. See docker-compose.yml excerpt:

version: "3"
services:

  nodered:
    image: nodered/node-red
    ports:
      - "1880:1880"
    volumes:
      - ./nodered/data:/data
      - .:/data/projects/home
    restart: always

Basically, you bind-mount the outer directory as a sub-directory of the inner node-red data directory. See current state at https://github.com/ianmacs/home/tree/04bb22cc3bd50886ed4d62a9b39198ac6bd6d4ba

8

Hi @danielcompton did you ever get this working? I have the same issue here, trying to create a dev/prod environment in a docker container with projects enabled for my dev env and disabled for my prod env. So I need to have the .git/ in the root and not in the projects folder of my nr project.
I am trying to use the Package entry in the project settings menu but with nu succes so far.
If you got this working, would you mind explaining me what your last config looked like? I'm mainly confused by the package.json. do you still need the original package.json in the node-red root folder next to a package.json specific for the project? should both be merged? if so, how does it look like and where to place it?