We need to see all the files in the directory that has the compose file (since that is the build context)
Specifically the Dockerfile to start with, but probably more.
But you probably should NOT be mixing the volume mounted on /data with the environment used to build the Docker container (build context is the same as volume source, this feels like a really bad idea)
As it is a custom docker build, you should also drop to the containers console and check that D2 is not installed in the container globally or anywhere where node-red will pick it up.
As I mentioned sharing the same directory for the root container build and the /data means that they will have clashing package.json file. These should NOT be the same file, they should contain different things (one is for installing Node-RED into the container, the other (in /data) is for keeping track of installed nodes.
If you have rebuilt the container after installing the dashbaord it will have installed the dashbaord nodes (and any other nodes previously installed) under /usr/src/nodered when the container was built and you can not remove those nodes as they are baked into the container.
This would show the symptoms you have described.
I suggest you start again and ensure that you use use a separate directory mounted as /data
This is really intriguing. So I could create a separate ./docker folder to put my docker files in. Keep the docker config files there as well as an image package.json. However, the current root level package.json is where the runtime package changes are applied to? And as seems common in NR, we make these changes live in production. So let's say we add or update a package needed for integrating a system (for example modbus node in NR), we do that in palette manager and it is added to /data/package.json (runtime). However, would we then need to synchronize this to also add to image build stage package.json? Or perhaps keep the image context package.json minimal and limited to core dependencies like node red only.
I did a quick test to see the contents of /usr/src/node-red inside the container, and while it has the node_modules folder, it doesn't currently have a package.json. But this is of course post build stage, so perhaps irrelevant.
While I wasn't initially aware of this fumble I made, my thinking was whatever changes is done to runtime /data/package.json needs to be shared, synced and applied to future images/containers.
Thanks all, great help particularly with docker which seems to be the problem. I learned loads and restructured project to accommodate docker build image context. Added a docker folder for that and put a separate docker-package.json in it (together with Dockerfile). This only have node red dependency + engine. The root package.json is now exclusively for runtime dependencies.
It seemed to work to remove dependencies now even after restart. But I can only test on my local windows pc at the moment, will try target environment next week.