Automatically import a flow

I've created a very minimal flow that subscribes to an MQTT broker (mosquitto) with topic "readings" and stores whatever it receives into a file "readings.json".
The content of the file storage (the messages) is not really a valid JSON - every JSON message coming from the MQTT broker is just appended to the file, but I am trying to figure out how to import/export flows between different instances in an automatic way - without human interaction.

For completeness - the message is published with this command:

mosquitto_pub -h ${MY_LOCAL_IP} -p 1883 -t "readings" -i foo001 -m '{"deviceId":"foo001","fooMetric":42.42}' -d`)

If I put the JSON representing the flow in various places e.g.:

  • ~/.node-red/
  • ~/.node-red/lib/
  • ~/.node-red/lib/flows/

Then I restart the Systemd service with: sudo systemctl restart nodered.service

Then I open the web browser on <MY_LOCAL_IP>:1880 and check around in the dashboard, I am not able to find this job I would like to import.

Clicking on menu > Import > Library I get an empty list, but I was expecting to find my custom flow.

  • Is there a way to automatically import a flow from a JSON file? I am looking for something that does not require human interactions.
  • What should I do if I want to automatically deploy this automatically imported flow?

The only useful doc I found is this one regarding the manual process of importing/exporting flows: https://nodered.org/docs/user-guide/editor/workspace/import-export

Also: various people on the internet claim they have automatically imported flows, but the screenshots they show and the procedure they describe look like it's referring an outdated version of Node-RED.

My setup:

  • Node-RED version: v0.19.5
  • Node.js version: v10.15.0

This is the flow for reference:

[
  {
    "id": "cd5dc7af.0934f8",
    "type": "tab",
    "label": "Store readings",
    "disabled": false,
    "info": ""
  },
  {
    "id": "70da9d53.26da74",
    "type": "mqtt in",
    "z": "cd5dc7af.0934f8",
    "name": "mqtt-subscriber",
    "topic": "readings",
    "qos": "2",
    "broker": "b073e910.c235e8",
    "x": 260,
    "y": 100,
    "wires": [
      [
        "2b5cce9.cde0d32"
      ]
    ]
  },
  {
    "id": "2b5cce9.cde0d32",
    "type": "file",
    "z": "cd5dc7af.0934f8",
    "name": "",
    "filename": "readings.json",
    "appendNewline": true,
    "createDir": false,
    "overwriteFile": "false",
    "x": 510,
    "y": 100,
    "wires": [
      []
    ]
  },
  {
    "id": "b073e910.c235e8",
    "type": "mqtt-broker",
    "z": "",
    "name": "mosquitto-mqtt-broker",
    "broker": "localhost",
    "port": "1883",
    "clientid": "",
    "usetls": false,
    "compatmode": true,
    "keepalive": "60",
    "cleansession": true,
    "birthTopic": "",
    "birthQos": "0",
    "birthPayload": "",
    "closeTopic": "",
    "closeQos": "0",
    "closePayload": "",
    "willTopic": "",
    "willQos": "0",
    "willPayload": ""
  }
]

The editor provides an http API for administering your flows.

The /flows endpoint can be used to get/set the complete flown configuration - it's what the editor uses when you hit deploy.

The /flow endpoint can be used to manage individual flows (ie tabs in the editor). It uses a slightly different payload format, but it shouldnt be too hard to figure out.

https://nodered.org/docs/api/admin/methods/

1 Like

Thanks for that, very helpful.

Do I have to HTTP POST all the /flows at once or can I perform multiple calls one per each flow?
I am thinking at having a directory with multiple JSON files - each file with a meaningful name, looping via bash and submitting each JSON file content as a separate flow with a dedicated curl request.

Edit probably need to use PUT/POST /flow multiple times... I need to figure out if a flow ID is always required or if there is some sort of UPSERT/PATCH where someone could replace whatever is already there.

This HTTP POST request to add a new flow (see at the bottom) gives this error:

{"error":"unexpected_error","message":"Error: missing nodes property"}

The flow I am trying to submit has been exported from the web UI, so I am assuming it is valid.

According to this: https://nodered.org/docs/api/admin/methods/post/flow/ it seems I need to wrap the exported content into this as the field nodes, is this the intended way? Should I manually remove all the fields id as they are replaced by Node-RED anyway?

{
  "id": "91ad451.f6e52b8",
  "label": "Sheet 1",
  "nodes": [ ],
  "configs": [ ]
}

Also in the reference doc there's this " Requires permission: flows.write", which I am assuming it's not relevant unless I start introducing authentication and all of that security stuff, right?

This is the call:

curl -X POST \
  http://${MY_LOCAL_IP_ADDR}:1880/flow \
  -H 'content-type: application/json' \
  -d '[
  {
    "id": "cd5dc7af.0934f8",
    "type": "tab",
    "label": "Store readings",
    "disabled": false,
    "info": ""
  },
  {
    "id": "70da9d53.26da74",
    "type": "mqtt in",
    "z": "cd5dc7af.0934f8",
    "name": "mqtt-subscriber",
    "topic": "readings",
    "qos": "2",
    "broker": "b073e910.c235e8",
    "x": 260,
    "y": 100,
    "wires": [
      [
        "2b5cce9.cde0d32"
      ]
    ]
  },
  {
    "id": "2b5cce9.cde0d32",
    "type": "file",
    "z": "cd5dc7af.0934f8",
    "name": "",
    "filename": "readings.json",
    "appendNewline": true,
    "createDir": false,
    "overwriteFile": "false",
    "x": 510,
    "y": 100,
    "wires": [
      []
    ]
  },
  {
    "id": "b073e910.c235e8",
    "type": "mqtt-broker",
    "z": "",
    "name": "mosquitto-mqtt-broker",
    "broker": "localhost",
    "port": "1883",
    "clientid": "",
    "usetls": false,
    "compatmode": true,
    "keepalive": "60",
    "cleansession": true,
    "birthTopic": "",
    "birthQos": "0",
    "birthPayload": "",
    "closeTopic": "",
    "closeQos": "0",
    "closePayload": "",
    "willTopic": "",
    "willQos": "0",
    "willPayload": ""
  }
]
'

Thanks again for the help :slight_smile:

Edit

When wrapping the exported flow into the required structure with nodes, then the API was complaining about the first node in the list of type "type": "tab",. So I tweaked a bit the request making that "special node" as the label in the wrapper struct for nodes.

That worked, then when manually exporting the flow again from the web UI I found that "wrapper struct" to actually be a new "synthetic node" of type tab included as the first in the nodes array.

Is there a way to get all this stuff done automatically? Like some export option or something?

What's the purpose for that "configs": [ ] field in the wrapper?

I noticed that after running this HTTP request, Node-RED will prompt the user to merge. Is it possible to suppress the prompt, making adding a workflow completely automatic?

It is added to the workflow. The message is because the code in the browser (which might be the other side of the world remember, and knows nothing about the fact that you are updating the flow on the server) notices that it has changed in the server and doesn't know what to do. That may not be your use case, but it is a valid use case.

I notice that you have asked this on a new thread also. Please don't do that. I will close this one.