Automate updating nodes to the Flow Library (updated with XonSh)

Developing Nodes
1

Thanks to this post
I have update the script to xonsh script

import re
COOKIES_FILE="cookies.txt"
MODULE_NAME=$ARG1 #ex:@malysus/node-red-contrib-mqtt-auto

touch @(COOKIES_FILE)
rm @(COOKIES_FILE)

# get the _csrf token
cmd= f"".split(" ")
print(" ".join(cmd))
csfr=$(curl -s -c @(COOKIES_FILE) -b @(COOKIES_FILE) -XGET 'https://flows.nodered.org/add/node')

regex = r"_csrf\" type=\"hidden\" value=\"([^\"]+)\""
matches = list(re.finditer(regex, csfr, re.MULTILINE))
try:
    token=matches[0].group(1)
    print(f"_csrf token was found to be {token}")
    module=f"module={MODULE_NAME}&_csrf={token}"
    result=$(curl -s -XPOST "https://flows.nodered.org/add/node" -c @(COOKIES_FILE) -b @(COOKIES_FILE) -H "referer: https://flows.nodered.org/add/node" -H "Content-Type: application/x-www-form-urlencoded" -H "x-requested-with: XMLHttpRequest" -d @(module))
    print(result)
except Exception as e:
    print(e)
2

Here's the same as a Node-RED flow (using exec & curl):

[{"id":"b3f710d3656ec0bd","type":"exec","z":"73f57341a6bdef33","command":"rm -f /tmp/cookies.txt","addpay":"","append":"","useSpawn":"false","timer":"","winHide":false,"oldrc":false,"name":"","x":1417,"y":1204,"wires":[[],[],["7134a0a01a78b0a4"]]},{"id":"3870e85d9e363b6c","type":"inject","z":"73f57341a6bdef33","name":"","props":[{"p":"module_name","v":"@gregoriusrippenstein/node-red-contrib-introspection","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":1254,"y":1091,"wires":[["b3f710d3656ec0bd"]]},{"id":"7134a0a01a78b0a4","type":"switch","z":"73f57341a6bdef33","name":"","property":"payload.code","propertyType":"msg","rules":[{"t":"eq","v":"0","vt":"num"}],"checkall":"true","repair":false,"outputs":1,"x":1556,"y":1277,"wires":[["d7bb19775d60ba5f"]]},{"id":"d7bb19775d60ba5f","type":"change","z":"73f57341a6bdef33","name":"","rules":[{"t":"set","p":"payload","pt":"msg","to":"curl -s -c /tmp/cookies.txt -b /tmp/cookies.txt -XGET 'https://flows.nodered.org/add/node'","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":1670,"y":1332,"wires":[["329cb745ab148644"]]},{"id":"329cb745ab148644","type":"exec","z":"73f57341a6bdef33","command":"","addpay":"payload","append":"","useSpawn":"false","timer":"","winHide":false,"oldrc":false,"name":"","x":1776,"y":1393,"wires":[["8b4462e71d18d914"],[],[]]},{"id":"8b4462e71d18d914","type":"switch","z":"73f57341a6bdef33","name":"","property":"rc.code","propertyType":"msg","rules":[{"t":"eq","v":"0","vt":"num"}],"checkall":"true","repair":false,"outputs":1,"x":1928,"y":1399,"wires":[["c447b660ffb100b7"]]},{"id":"c447b660ffb100b7","type":"split","z":"73f57341a6bdef33","name":"","splt":"\\n","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":2008,"y":1343,"wires":[["24a6bdedc55adb51"]]},{"id":"24a6bdedc55adb51","type":"switch","z":"73f57341a6bdef33","name":"","property":"payload","propertyType":"msg","rules":[{"t":"regex","v":"name=\"_csrf\" type=\"hidden\" value=\"(.+)\">","vt":"str","case":false}],"checkall":"true","repair":false,"outputs":1,"x":2077,"y":1289,"wires":[["e045b2b86c04bede"]]},{"id":"e045b2b86c04bede","type":"change","z":"73f57341a6bdef33","name":"","rules":[{"t":"set","p":"payload","pt":"msg","to":"$substringBefore( $substringAfter($$.payload, \"value=\\\"\"), \"\\\">\")","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":2134,"y":1235,"wires":[["5096c688782c5d5b"]]},{"id":"5096c688782c5d5b","type":"change","z":"73f57341a6bdef33","name":"","rules":[{"t":"set","p":"data_string","pt":"msg","to":"\"module=\" & $$.module_name & \"&_csrf=\" & $$.payload","tot":"jsonata"},{"t":"set","p":"payload","pt":"msg","to":"\"curl -v -XPOST 'https://flows.nodered.org/add/node' -c /tmp/cookies.txt -b /tmp/cookies.txt -H 'referer: https://flows.nodered.org/node/\" & $$.module_name & \"' -H 'Content-Type: application/x-www-form-urlencoded' -H 'x-requested-with: XMLHttpRequest' -d '\" & $$.data_string & \"'\"","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":2345,"y":1325,"wires":[["4260147961b82058"]]},{"id":"4260147961b82058","type":"exec","z":"73f57341a6bdef33","command":"","addpay":"payload","append":"","useSpawn":"false","timer":"","winHide":false,"oldrc":false,"name":"","x":2465,"y":1455,"wires":[["f4dc513b63d2486a"],["3210b2328892fb83"],["2782388af7e4a3e3"]]},{"id":"f4dc513b63d2486a","type":"debug","z":"73f57341a6bdef33","name":"debug 345","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":2639,"y":1404,"wires":[]},{"id":"3210b2328892fb83","type":"debug","z":"73f57341a6bdef33","name":"debug 346","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":2642,"y":1455,"wires":[]},{"id":"2782388af7e4a3e3","type":"debug","z":"73f57341a6bdef33","name":"debug 347","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":2639,"y":1511,"wires":[]}]

The initial inject is given the module name and it should just work.

Also I changed the referer on the second request to be https://flows.nodered.org/node/<module> as that's what the browser was sending.

3

i try to make the equivalent for "https://flows.nodered.org/flow/xxxxxx/refresh" have any idea how it's managed authentication?

I HAVE CREATED A GIST with your code.

I should be easier to make change and share it.

4

No idea. It's non-trivial because of the login via github option. It would all be far simpler if there were to be an REST api that could be utilised to do all this.

5

of course It's not depending on me :wink: