Best way to run a curl command?

General
1

Guys,

I have a CURL command from my Electricity provider to query an API and return (supposedly) as JSON object.

I can run this from the command line in Linux and is does return data which i can store into a file.

Simplest way would be to then just read this data file and work on it - however i was wondering is there any node that will essentially take a curl commandline and encapsulate it or convert it for use within NR itself rather than execing out etc ?

The comnand line is

curl -X POST https://api.amberelectric.com.au/prices/listprices -H 'Content-Type: application/json' -d '{ "postcode": "2069" }'

Craig

2

You can run that command with an exec node and the output of the exec node should be the data you want. No need to go via a file.

3

That is a fairly easy http request. No need to exec a curl.

Add the content type header & push the JSON into a http request node set to your URL & type POST.

1 Like
4

if you do end up running exec node, add a timeout in shell command too so that it doesn't get stuck forever. (don't ask me how i know :slight_smile: ) .

1 Like
5

you can also add a timeout into the node itself, which should kill the command if it gets stuck.

6

One question to ask is what degree of reporting or error handling do you want? There are a few different ways to execute commands, such as the exec node. Sometimes I want unique exception handling, so go beyond the basic exec model. Or may be you want to use spawn as via asynch execution, for example. The way to execute commands from node js cover wide range of features. I am actually working on a node, an experiment, on parallel command execution, using worker threads for example, using spawn.

Here is a short discussion on when you might want to use the different options to execute commands. Not al modes or types of such are visible in node red that I have discovered so far, but people are adding new nodes all the time.
https://nodejs.org/en/knowledge/child-processes/how-to-spawn-a-child-process/

7

I'm not certain why everyone is talking about exec node :confused: am i missing something?

There is not a node to do the work for you - but it isn't that difficult to do by hand.

Here is how you translate a simple CURL to "The node-red way"

image
image780Ă—726 54.7 KB

[{"id":"3fbcc579.1dc12a","type":"inject","z":"60184b60.424444","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":800,"y":140,"wires":[["37d01ae9.869d66"]]},{"id":"dea77fd.b480f8","type":"http request","z":"60184b60.424444","name":"","method":"POST","ret":"obj","paytoqs":"ignore","url":"https://api.amberelectric.com.au/prices/listprices","tls":"","persist":false,"proxy":"","authType":"","x":950,"y":260,"wires":[["261e81ee.00f02e"]]},{"id":"37d01ae9.869d66","type":"change","z":"60184b60.424444","name":"","rules":[{"t":"set","p":"headers","pt":"msg","to":"{\"Content-Type\":\"application/json\"}","tot":"json"},{"t":"set","p":"payload","pt":"msg","to":"{\"postcode\":\"2069\"}","tot":"json"}],"action":"","property":"","from":"","to":"","reg":false,"x":880,"y":200,"wires":[["dea77fd.b480f8"]]},{"id":"261e81ee.00f02e","type":"debug","z":"60184b60.424444","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":990,"y":320,"wires":[]}]

The benefits are...

  • No file to write / read
  • No process spawning
  • Response is ready to use JS object (no parsing after spawning).
  • Far simpler (IMO)
5 Likes
8

Excellent thanks Steve - yep the conversation was interesting but not really what i wanted - i will give your solution a try shortly and report back !

regards

Craig

9

Yep thats perfect thanks Steve - i have just run it now and did exactly what i wanted. Now to build a dashboard with it !

Craig

10

@dceejay that never worked reliability for me.

11

Though http request node would work in most situations, I found that node-red adds an additional header which can't be removed (forgot the exact header name). Sometimes receiving servers are not forgiving on headers and you would not know why request is failing.

12

Did you raise an issue or post it on the forum?

13

Indeed, as Steve mentions, we would be interested in any examples you have of it not working. Both timeout, and headers. Thanks.

14

Isn't the kill (on timeout or manually) timeout not working reliably with the exec node a problem that was discussed recently (and I think fixed, but not certain) due to it running a shell which runs the command, and the kill killed the shell but not the command.

15

My pull request to fix it on systems where bash is available was merged so i think it should now work on those linux systems if nodered is up to date as the problem was that debian based systems symlinked /bin/sh to dash and not bash so that was what the exec was using. It should now default to bash if available where the killing works.

1 Like
16

OK - and what about the headers @hoolahoous ?

17

Since i write advance scripts using axios/request I use http request node only occasionally. i could have sworn that i saw a header added automatically with something like 'nodeid' . I thought it was a feature for http request node to track the response. Now I can't seem to reproduce it.

1 Like
18

on topic of timeout, I have several puppeteer script which I run from node-red in headless mode. to prevent runaway puppeteer instances which would never exit, i tried putting timeout on exec node. however it would work only on spawn mode not in exec mode. And spawn mode was not suitable for my use case, so I just used timeout in command to make sure process gets killed after certain timeout.

1 Like
19

I have had a similar experience, where using spawnSync worked with timeouts better than more generic exec. I was doing it in straight Javascript, however, not using the exec node spawn option. This somewhat suggests the issue maybe with something below node red if you will. Also been working with spawn in async mode, but decided that was overkill for what I needed.

20

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.