Hello,
I know that through export/import I can copy flows between two hosts. However, other elements, such as additional nodes installed via the palette, or configuration nodes will not be copied with this method.
Is there a way to easily mode everything from host A to host B ? To make it "simpler" let's assume that both hosts are running the same and official docker version.
Thanks,
This is a bit sticky. In general that following should work:
- same version on both, of NPM and NR
- make sure any require modules are installed in the npm tree under NR
- change the secret key in settings.js file to be same for both systems. And make settings.js file generic, no user specific, and that the global context references said js modules correctly. See documentation for this.
- copy package.json under .node-red over.
- same module packages on both, you can use package.json file to do a npm install, it will use the
package.json if no explicit package referenced on npm install command - copy the flows.json and flows_cred.json file from A to B.
There is some official documentation that explains this, but last I looked the documentation was still draft, and not finalized. So I developed my own process which I outlined below.
Also the projects feature is an option.
I still find some odd issues, some nodes store credentials in odd ways, so even when I have the cred file correct, the settings.js file correct as noted above. I still get a flow or two that lose credential information. Have to re-enter it.
Have you taken a look at the projects feature?
Kind of... I have not really understood the purpose 
All node information is stored in either the flow file or the credentials file. Nodes should not be storing information anywhere else of their own accord. Without derailing this topic, can you let me know what nodes you find lose their credentials so I can take a look at what they are doing?
Sure, does cronplus and json-DB not store stuff of their own accord? It is the the mysql node that seems to gripe about credentials at times, even though I have settings.js secret in common across NR instances, and copy over the cred file as well as the flow file. I don't get the credentials lost warning on import of flows, so that tells me I have the secret set right and the cred file is accepted, right?
And, I am just doing file copies, not using the admin API... plan to at some point, just have not had time to switch over.
Thanks. Sorry, I don't understand step 5... The additional nodes installed on A using the palette (node-red-contrib...), will have to be added on B through npm? There is no way by "importing/copying" some files that all additional nodes to be installed automagically?
In addition, if you want to make sure you have exactly the same versions of third party nodes installed then copy package-lock.json as well as package.json before you go into the .node-red directory and run npm install (which is what step 5 is indicating I think).
If you don't copy package-lock then it might install later versions of the nodes than those in the original.
All of the extra nodes should be listed in the dependencies section of the package.json file.
Once you've copied that file over, run npm install to get them installed.
@knolleary @Colin @Nodi.Rubrum : Thanks for the help. I will try this asap!!
Oh, dang, I should have listed the package lock file too. That was an oversight on my part. Sorry about that. That is what i get for responding from memory and not checking my notes.
There are several nodes that store other information on the filing system - including uibuilder. That's because they need more complex and/or different types of data. Other examples might come from using Node-RED's static web folder, output to file or SQLlite, etc.
Generally, unless your own flow is exporting something, you would expect all file data to reside in the userDir or a subfolder of that.
In general, I would always assume that I would copy everything from the userDir with the exception of the node_modules folder. That should take everything except things that you've configured for yourself outside that folder.
Then on the other side, you can run npm install inside the new userDir and get a working system again. If the target does not have direct Internet connection, you can include the node_modules folder as well as long as the source and target systems use the same version of Node.js, Node-RED and have the same platform type & OS.
Of course, the default for the flows and credentials files is that Node-RED assumes they are named from the machine ID - which is why I always specify the flows filename on my Node-RED instances.
Is the 'lib' directory still valid? I believe the draft documentation mentioned it as well, but every instance I have, the 'lib' directory is empty, thus far. I copy the entire .node-red directory structure when I archive, this is a bit over kill because your can recreate the node_modules directory from other files, i.e. package*.json files. But hey, disk space is cheap [cough].
I should have mentioned, I run two instances of NR where the directory structure is mirrored. This is a poor man active/passive model. I run other instances of NR on things as well, but for my main monitoring/information collection 'nodes' the mirror copy seems to work just fine so far. The one thing I had to do in the actual flows was do a bit of coding so the right 'host' or 'publisher' and 'subscriber' references are correct for both of the 'mirrored' nodes. Oh, I only run one broker at any given time as well, but that is different story.
Like already mentioned, it's not a good idea to include the node_modules dir as it can include native modules compiled for the specific system. Even if the Node-RED, Node.js version, OS and the host processor architecture is the same, some native module might link to a C/C++ library on the system that might be missing.
A manually run npm installwill compile all the native modules based on the OS, CPU architecrure and available native libraries which might still be missing from the system NR is being copied over.
Soi unless you know the systems have been setup exactly same same, the extra npm install command on the new host might indicate missing external dependencies and fail.
Yes, and just to be clear, I said I archive node_modules, I did not say I use it to create anything, if you review what I wrote above, I did suggest to use the package*.json method to create the npm repository.
Above said, my two monitoring NR instances are on the same hardware, the same OS version, and the same NR and NPM versions. Which would be expected IMHO if you are planning to run a two instance active/passive infrastructure.
I believe I have done the suggested things.
- Copy flows.json, flows_cred.json, package.json
- Put the same credentialSecret in both settings.js
- Run npm install
- Restart the container
On in the logs I can see this:
11 Sep 17:43:41 - [info] Flows file : /data/flows.json
11 Sep 17:43:41 - [info] Server now running at http://127.0.0.1:1880/
11 Sep 17:43:41 - [warn] Error loading credentials: SyntaxError: Unexpected token L in JSON at position 0
11 Sep 17:43:41 - [warn] Error loading flows: Error: Failed to decrypt credentials
11 Sep 17:43:41 - [info] Starting flows
This is displayed on the node red window:
What have I missed ?
I'm not 100% sure of this but I think if you're using projects I believe the flow credentials file is located in the project directory $HOME/.node-red/projects/PROJECT_NAME. Maybe you should then copy this structure instead?
With that said, I recently I somehow lost my credentials file after I had to force close NR when it got too busy from me pushing the limits. On startup I also received a similar unexpected token error and it didn't go away even by replacing the creds file with the automatically backed up version. There credentials decryption key is set in the project settings but I did not find where it's stored in the file system even after setting a new key. Maybe @knolleary or @dceejay could shine a light here?
I am not using projects... So it shouldn't be the issue.
Unexpected token L in JSON at position 0
That doesn't sound like an encryption key error. That error suggests the files contents is just wrong - it should never have an L at the start.
