The challenge is, this will be used on systems with little/no connectivity. Relying on a local /data/ volume mount seems less "portable" as well (including updates and changes in a build system is a relatively easy lift so I am much less concerned about that). Additionally, I have a local path that must be accessible to the flows (currently mapped to something like /data/subdir). I suppose I could map elsewhere (or make a location in a Dockerfile)?
The goal is to have an image with all the needed pieces "baked in" so to speak (such that, upon start, nothing further needs downloaded and is already present).
Maybe this will help explain the alternatives (as I understand them) and the challenges I am seeing with them...
Inputs
docker-compose.yaml
services:
not-baked-in:
container_name: not-baked-in
image: nodered/node-red:3.0.0-16-minimal
volumes:
- ./:/data/
ports:
- "1880:1880" # web ui
baked-in-workdir:
container_name: baked-in-workdir
build:
context: .
dockerfile: Dockerfile.baked-in-workdir
ports:
- "1880:1880" # web ui
baked-in-userdir:
container_name: baked-in-userdir
build:
context: .
dockerfile: Dockerfile.baked-in-userdir
ports:
- "1880:1880" # web ui
Dockerfile.baked-in-workdir
FROM nodered/node-red:3.0.0-16-minimal
RUN npm i \
node-red-debugger@1.1.1 \
pg@8.6.0 \
--unsafe-perm --no-update-notifier --no-fund --omit=dev
Dockerfile.baked-in-userdir
FROM nodered/node-red:3.0.0-16-minimal
RUN cd /data/
COPY package.json flows.json settings.js ./
RUN npm i \
--unsafe-perm --no-update-notifier --no-fund --omit=dev
package.json
{
"name": "my-node-red",
"version": "0.0.1",
"dependencies": {
"node-red-debugger": "1.1.1",
"pg": "8.6.0"
}
}
flows.json
[{"id":"0219bb93a4e9f932","type":"tab","label":"Flow 1","disabled":false,"info":"","env":[]},{"id":"c3aaf3a15f27e455","type":"inject","z":"0219bb93a4e9f932","name":"init","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":90,"y":60,"wires":[["a2c53a9c97709bfc"]]},{"id":"a2c53a9c97709bfc","type":"function","z":"0219bb93a4e9f932","name":"try pg lib","func":"let PgPool = pg.Pool;\nlet pool = new PgPool();\n\npool.query(\"SELECT 'Test' as [TestMsg]\", (err, res) => {\n if (err) {\n msg.result = err;\n }\n \n msg.result = res.rows[0];\n})\n\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[{"var":"pg","module":"pg@8.6.0"}],"x":240,"y":60,"wires":[["6fbe3d1f1df9f9da"]]},{"id":"6fbe3d1f1df9f9da","type":"debug","z":"0219bb93a4e9f932","name":"dbg","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":390,"y":60,"wires":[]}]
settings.js
As per original, except...
Outputs
not-baked-in
> docker compose up not-baked-in
TLDR/Results:
no pg/lib install
no error message
flow debug installed
flow stop feature enabled
baked-in/portable image
inject output in debug panel
Details:
not-baked-in |
not-baked-in | > node-red-docker@3.0.0 start
not-baked-in | > node $NODE_OPTIONS node_modules/node-red/red.js $FLOWS "--userDir" "/data"
not-baked-in |
not-baked-in | 18 Jul 22:00:11 - [info]
not-baked-in |
not-baked-in | Welcome to Node-RED
not-baked-in | ===================
not-baked-in |
not-baked-in | 18 Jul 22:00:11 - [info] Node-RED version: v3.0.0
not-baked-in | 18 Jul 22:00:11 - [info] Node.js version: v16.16.0
not-baked-in | 18 Jul 22:00:11 - [info] Linux 5.4.72-microsoft-standard-WSL2 x64 LE
not-baked-in | 18 Jul 22:00:13 - [info] Loading palette nodes
not-baked-in | 18 Jul 22:00:16 - [info] Settings file : /data/settings.js
not-baked-in | 18 Jul 22:00:16 - [info] Context store : 'default' [module=memory]
not-baked-in | 18 Jul 22:00:16 - [info] User directory : /data
not-baked-in | 18 Jul 22:00:16 - [warn] Projects disabled : editorTheme.projects.enabled=false
not-baked-in | 18 Jul 22:00:16 - [info] Flows file : /data/flows.json
not-baked-in | 18 Jul 22:00:16 - [info] Server now running at http://127.0.0.1:1880/
not-baked-in | 18 Jul 22:00:16 - [warn]
not-baked-in |
not-baked-in | ---------------------------------------------------------------------
not-baked-in | Your flow credentials file is encrypted using a system-generated key.
not-baked-in |
not-baked-in | If the system-generated key is lost for any reason, your credentials
not-baked-in | file will not be recoverable, you will have to delete it and re-enter
not-baked-in | your credentials.
not-baked-in |
not-baked-in | You should set your own key using the 'credentialSecret' option in
not-baked-in | your settings file. Node-RED will then re-encrypt your credentials
not-baked-in | file using your chosen key the next time you deploy a change.
not-baked-in | ---------------------------------------------------------------------
not-baked-in |
not-baked-in | 18 Jul 22:00:16 - [warn] Encrypted credentials not found
not-baked-in | 18 Jul 22:00:16 - [info] Starting flows
not-baked-in | 18 Jul 22:00:16 - [error] [function:a2c53a9c97709bfc] Error: Cannot find module 'pg@8.6.0'
not-baked-in | Require stack:
not-baked-in | - /data/node_modules/pg@8.6.0
not-baked-in | 18 Jul 22:00:16 - [info] Started flows
Cleanup: > docker compose down not-baked-in
baked-in-workdir
> docker compose up baked-in-workdir
TLDR/Results:
no pg/lib install
no error message
flow debug installed
flow stop feature enabled
baked-in/portable image
inject output in debug panel
Details:
baked-in-workdir |
baked-in-workdir | > node-red-docker@3.0.0 start
baked-in-workdir | > node $NODE_OPTIONS node_modules/node-red/red.js $FLOWS "--userDir" "/data"
baked-in-workdir |
baked-in-workdir | 18 Jul 22:44:38 - [info]
baked-in-workdir |
baked-in-workdir | Welcome to Node-RED
baked-in-workdir | ===================
baked-in-workdir |
baked-in-workdir | 18 Jul 22:44:38 - [info] Node-RED version: v3.0.0
baked-in-workdir | 18 Jul 22:44:38 - [info] Node.js version: v16.16.0
baked-in-workdir | 18 Jul 22:44:38 - [info] Linux 5.4.72-microsoft-standard-WSL2 x64 LE
baked-in-workdir | 18 Jul 22:44:40 - [info] Loading palette nodes
baked-in-workdir | 18 Jul 22:44:42 - [info] Settings file : /data/settings.js
baked-in-workdir | 18 Jul 22:44:42 - [info] Context store : 'default' [module=memory]
baked-in-workdir | 18 Jul 22:44:42 - [info] User directory : /data
baked-in-workdir | 18 Jul 22:44:42 - [warn] Projects disabled : editorTheme.projects.enabled=false
baked-in-workdir | 18 Jul 22:44:42 - [info] Flows file : /data/flows.json
baked-in-workdir | 18 Jul 22:44:42 - [warn]
baked-in-workdir |
baked-in-workdir | ---------------------------------------------------------------------
baked-in-workdir | Your flow credentials file is encrypted using a system-generated key.
baked-in-workdir |
baked-in-workdir | If the system-generated key is lost for any reason, your credentials
baked-in-workdir | file will not be recoverable, you will have to delete it and re-enter
baked-in-workdir | your credentials.
baked-in-workdir |
baked-in-workdir | You should set your own key using the 'credentialSecret' option in
baked-in-workdir | your settings file. Node-RED will then re-encrypt your credentials
baked-in-workdir | file using your chosen key the next time you deploy a change.
baked-in-workdir | ---------------------------------------------------------------------
baked-in-workdir |
baked-in-workdir | 18 Jul 22:44:42 - [info] Server now running at http://127.0.0.1:1880/
baked-in-workdir | 18 Jul 22:44:42 - [warn] Encrypted credentials not found
baked-in-workdir | 18 Jul 22:44:42 - [info] Installing module: pg, version: 8.6.0
baked-in-workdir | 18 Jul 22:44:49 - [info] Installed module: pg@8.6.0
baked-in-workdir | 18 Jul 22:44:49 - [info] Starting flows
baked-in-workdir | 18 Jul 22:44:49 - [error] [function:a2c53a9c97709bfc] Error: Cannot find module 'pg@8.6.0'
baked-in-workdir | Require stack:
baked-in-workdir | - /data/node_modules/pg@8.6.0
baked-in-workdir | 18 Jul 22:44:49 - [info] Started flows
Note: despite "installed" message, and lib being in both workDir and userDir, a "Cannot find" error is returned
> docker exec -it baked-in-workdir sh
~ $ pwd
/usr/src/node-red
~ $ npm ls --depth=0
node-red-docker@3.0.0 /usr/src/node-red
+-- node-red-debugger@1.1.1
+-- node-red@3.0.0
`-- pg@8.6.0
~ $ cd /data
/data $ npm ls --depth=0
node-red-project@0.0.1 /data
`-- pg@8.6.0
Cleanup: > docker compose down baked-in-workdir
baked-in-userdir
> docker compose up baked-in-userdir
TLDR/Results:
no pg/lib install
no error message
flow debug installed
flow stop feature enabled
baked-in/portable image
inject output in debug panel
Details:
aked-in-userdir | npm ERR! Missing script: "start"
baked-in-userdir | npm ERR!
baked-in-userdir | npm ERR! Did you mean one of these?
baked-in-userdir | npm ERR! npm star # Mark your favorite packages
baked-in-userdir | npm ERR! npm stars # View packages marked as favorites
baked-in-userdir | npm ERR!
baked-in-userdir | npm ERR! To see a list of scripts, run:
baked-in-userdir | npm ERR! npm run
baked-in-userdir |
baked-in-userdir | npm ERR! A complete log of this run can be found in:
baked-in-userdir | npm ERR! /data/.npm/_logs/2022-07-18T22_55_00_980Z-debug-0.log
baked-in-userdir exited with code 1
Cleanup: > docker compose down baked-in-userdir
I imagine if I added node-red@3.0.0 to the package.json to the baked-in-userdir service, it would work... but then why use FROM nodered/node-red:3.0.0-16-minimal in dockerfile/etc? Changing functionExternalModules to false might prevent installation of packages, but it is unclear why it thinks it needs to install them to begin with?
What am I missing?
PS: functionExternalModules = false for baked-in-workdir gives...
19 Jul 02:01:31 - [error] [function:7313a330f49cc770] Error: Function node not allowed to load external modules
...I evaluated the options once upon a time, and I dont recall now what the reasoning was for going this route (some feature gap at the time, IIRC). I'll look into it again.