Browser request works but exact same node red request fails

davidmnz · 5 April 2025 00:37

I am new to node red (so links are fudged to htttp due to new user limits) and I expected a very simple http request would just work. This is the request: "htttps://api.forecast.solar/estimate/-41.51/173.96/25/170/3.84". It does work as expected in Firefox but the very simple flow fails quickly with the message "no response from server" from the http-request node and the following error message captured at the debug node:

{"_msgid":"db9f3f80dae3053e","payload":"RequestError : htttps://api.forecast.solar/estimate/-41.51/173.96/25/170/3.84","topic":"","statusCode":"ETIMEDOUT"}

The browser and node red are running on the same platform, a Raspberry Pi 4 running Ubuntu 22.04. Version info:

Node-RED version: v4.0.9
Node.js version: v20.19.0
Linux 5.15.0-1070-raspi arm64 LE

I think I must be missing something very simple about node red or nodejs and networking but have so far had no luck searching for a solution.

Here is a screenshot and the JSON for the flow:

[
    {
        "id": "0df4284c28cd9b46",
        "type": "tab",
        "label": "Flow 2",
        "disabled": false,
        "info": "",
        "env": []
    },
    {
        "id": "70abc65628136c33",
        "type": "inject",
        "z": "0df4284c28cd9b46",
        "name": "",
        "props": [
            {
                "p": "payload"
            },
            {
                "p": "topic",
                "vt": "str"
            }
        ],
        "repeat": "",
        "crontab": "",
        "once": false,
        "onceDelay": 0.1,
        "topic": "",
        "payload": "",
        "payloadType": "date",
        "x": 140,
        "y": 160,
        "wires": [
            [
                "ed895fa7f3f3d1bf"
            ]
        ]
    },
    {
        "id": "ed895fa7f3f3d1bf",
        "type": "http request",
        "z": "0df4284c28cd9b46",
        "name": "",
        "method": "GET",
        "ret": "obj",
        "paytoqs": "ignore",
        "url": "https://api.forecast.solar/estimate/-41.51/173.96/25/170/3.84",
        "tls": "",
        "persist": false,
        "proxy": "",
        "insecureHTTPParser": false,
        "authType": "",
        "senderr": false,
        "headers": [],
        "x": 350,
        "y": 160,
        "wires": [
            [
                "fc5f9282a54b38a5"
            ]
        ]
    },
    {
        "id": "fc5f9282a54b38a5",
        "type": "debug",
        "z": "0df4284c28cd9b46",
        "name": "debug 3",
        "active": true,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "true",
        "targetType": "full",
        "statusVal": "",
        "statusType": "auto",
        "x": 600,
        "y": 160,
        "wires": []
    }
]

jbudd · 5 April 2025 01:26

Welcome to the forum @davidmnz .

Your Node-red flow worked for me, returning a statusCode 200 and a bunch of forecast data. A temporary glitch maybe, or does it still not work for you?

Are you using Docker?

The URL you posted didn't work though, until I changed htttps://.... to https://....

davidmnz · 5 April 2025 01:46

Thanks. I did have to change the link to htttps because newbies are only allowed to post two links. But I guess you knew that.

It has never worked for me and still doesn't after trying just now.

I am running node red from the command line in a Bash terminal window, not as a service. Could this be a problem? Otherwise no VM, Docker or any other hosting involved. Same hardware and OS. Works in Firefox every time but never in node red.

jbudd · 5 April 2025 01:57

Does this flow work? It's just a different (unrelated) API

[{"id":"70abc65628136c33","type":"inject","z":"0df4284c28cd9b46","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":140,"y":160,"wires":[["ed895fa7f3f3d1bf"]]},{"id":"ed895fa7f3f3d1bf","type":"http request","z":"0df4284c28cd9b46","name":"opentopodata.org api","method":"GET","ret":"obj","paytoqs":"ignore","url":"https://api.opentopodata.org/v1/srtm30m?locations=-41.51, 173.96","tls":"","persist":false,"proxy":"","insecureHTTPParser":false,"authType":"","senderr":false,"headers":[],"x":340,"y":160,"wires":[["fc5f9282a54b38a5"]]},{"id":"fc5f9282a54b38a5","type":"debug","z":"0df4284c28cd9b46","name":"debug 3","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":540,"y":160,"wires":[]}]

davidmnz · 5 April 2025 02:16

Yes, that flow did work. Returned status code 200 and (I think) a small array. So why does my flow, which seems very similar, not work?

davidmnz · 5 April 2025 05:06

I think something weird is going on with the http-request node. I used wireshark to capture packets from the working browser requests and the failing node red requests. Node red is resetting the TCP connection. Unlike the browser capture which shows many packets to and fro, the node red capture is only 3 packets: request (TCP SYN), reply (server TCP SYN ACK) and response (TCP RST).

Is there something in the node-red or nodejs config that can affect TCP in strange ways?

E1cid · 5 April 2025 06:40

If you are using the free tier you only get 6 calls per day. Try from a different IP address.

davidmnz · 5 April 2025 20:30

I think the limit with the free account is more like 12 per hour. Anyway that is not the root cause because the browser and node red requests come from the same IP address and the browser has never failed while node red has never succeeded.

Also see my previous post where I captured packets. It is node red that is breaking the connection, not the server side.

craigcurtin · 7 April 2025 01:31

Did you pick up the typo that was pointed out earlier you have htttps (i.e. one too many t's)

I have tried your flow (after changing that) and it works

Craig

davidmnz · 7 April 2025 01:52

That typo was deliberate because newbies are only allowed to post 2 hyperlinks.

I have tried many tweaks for my flow and it just does not work whereas the opentopo flow always works and the browser always works. I have even had a quick look at the http request source code but it's too complex for me to go any further via that debugging route. However I am convinced there is a bug or a configuration issue somewhere that causes my flow to fail every time on my machine.

Steve-Mcl · 7 April 2025 06:47

Your flow works perfectly fine for me (as it did for @jbudd above)

Where is the browser running? On the same device as where Node-RED is installed?

Is you internet connection via a proxy or is it firewalled?

Have you tried a cURL from the Node-RED servers terminal?
curl https://api.forecast.solar/estimate/-41.51/173.96/25/170/3.84

davidmnz · 8 April 2025 02:20

As noted in one of my posts above, the browser and node red are running on the same hardware and OS. My internet connection is a home fibre connection with a wifi router connecting to the Raspberry Pi. There is some degree of firewalling in both the router and the Pi but I have never experienced any connections failing due to firewall rules.

What do you mean by a curl from the node red servers terminal? Do you mean running curl in the same terminal session as node red is launched from? So I guess I would have to launch node red as a background process or a service so I can actually get at the command prompt. Might just try this anyway.

BTW note that I said "I am convinced there is a bug or a configuration issue" so I'm not saying it's definitely a bug but could be something I got wrong in my node red config.

davidmnz · 8 April 2025 02:30

I launched node red in background and then did a curl from the same terminal session. That worked. I also did a curl from a different terminal session and that also worked. But I'm not sure what that tells me. Browser works and curl works but node red http-request fails.

davidmnz · 8 April 2025 02:39

And running node red as a service doesn't make any difference to running it from the command line.

kitori · 8 April 2025 05:03

i did a tcpdump of of these two request:
grafik

[{"id":"085abd113d47e6eb","type":"inject","z":"1bc8ef14da938222","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":280,"y":280,"wires":[["655120214e1b2b9d"]]},{"id":"655120214e1b2b9d","type":"http request","z":"1bc8ef14da938222","name":"","method":"GET","ret":"txt","paytoqs":"ignore","url":"https://api.forecast.solar/estimate/-41.51/173.96/25/170/3.84","tls":"dff49c68b8cdf54e","persist":false,"proxy":"","insecureHTTPParser":false,"authType":"","senderr":false,"headers":[],"x":450,"y":280,"wires":[["64425574a6b9f0d2"]]},{"id":"64425574a6b9f0d2","type":"debug","z":"1bc8ef14da938222","name":"debug 13","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":640,"y":280,"wires":[]},{"id":"dff49c68b8cdf54e","type":"tls-config","name":"","cert":"","key":"","ca":"","certname":"","keyname":"","caname":"","servername":"","verifyservercert":true,"alpnprotocol":""}]

and

 curl https://api.forecast.solar/estimate/-41.51/173.96/25/170/3.84

<---- Left: curl -- Right: Node-RED------>

I wonder why I’m getting the RST packages from the server
Can someone explain this? I have no idea, this is not my main thing.

Steve-Mcl · 8 April 2025 05:33

I'll analyse your replies a little later. In the meantime, could you:

Put node.warn({env: process.env})in a function node (you'll also need to add process to the imports on the setup tab), inject a msg to trigger it, hit the "copy value"button on the debug message & paste into a code block (sanitise any values you need to)
Post the log from the console of node-red from start to flows running
Provide a list of installed contrib nodes (easiest method is ctrl-shift-p -> enter system info)

davidmnz · 8 April 2025 06:06

Thanks. I have done 1 and 2 but I can't figure out how to do 3. In what context does ctrl-shift-p work?

{"LESSOPEN":"| /usr/bin/lesspipe %s","LANGUAGE":"en_NZ:en","USER":"david","XDG_SESSION_TYPE":"wayland","SHLVL":"1","HOME":"/home/david","OLDPWD":"/usr/lib/node_modules/node-red/bin","DESKTOP_SESSION":"ubuntu","GNOME_SHELL_SESSION_MODE":"ubuntu","GTK_MODULES":"gail:atk-bridge","SYSTEMD_EXEC_PID":"2074","DBUS_SESSION_BUS_ADDRESS":"unix:path=/run/user/1000/bus","COLORTERM":"truecolor","IM_CONFIG_PHASE":"1","WAYLAND_DISPLAY":"wayland-0","LOGNAME":"david","_":"/usr/bin/node-red-pi","XDG_SESSION_CLASS":"user","USERNAME":"david","TERM":"xterm-256color","GNOME_DESKTOP_SESSION_ID":"this-is-deprecated","PATH":"/home/david/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin","SESSION_MANAGER":"local/david-desktop:@/tmp/.ICE-unix/1792,unix/david-desktop:/tmp/.ICE-unix/1792","XDG_MENU_PREFIX":"gnome-","GNOME_TERMINAL_SCREEN":"/org/gnome/Terminal/screen/cf734d07_99a6_4dc0_87c0_e5dd81639ae6","GNOME_SETUP_DISPLAY":":1","XDG_RUNTIME_DIR":"/run/user/1000","DISPLAY":":0","LANG":"en_NZ.UTF-8","XDG_CURRENT_DESKTOP":"ubuntu:GNOME","XMODIFIERS":"@im=ibus","XDG_SESSION_DESKTOP":"ubuntu","XAUTHORITY":"/run/user/1000/.mutter-Xwaylandauth.RLXS42","LS_COLORS":"rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;3...","GNOME_TERMINAL_SERVICE":":1.109","SSH_AGENT_LAUNCHER":"gnome-keyring","SSH_AUTH_SOCK":"/run/user/1000/keyring/ssh","SHELL":"/bin/bash","QT_ACCESSIBILITY":"1","GDMSESSION":"ubuntu","LESSCLOSE":"/usr/bin/lesspipe %s %s","QT_IM_MODULE":"ibus","PWD":"/home/david","XDG_CONFIG_DIRS":"/etc/xdg/xdg-ubuntu:/etc/xdg","XDG_DATA_DIRS":"/usr/share/ubuntu:/usr/local/share/:/usr/share/:/var/lib/snapd/desktop","VTE_VERSION":"6800","NODE_RED_HOME":"/usr/lib/node_modules/node-red","PYTHONUNBUFFERED":"1"}

david@david-desktop:~$ node-red-pi --max-old-space-size=256
8 Apr 17:35:57 - [info] 

Welcome to Node-RED
===================

8 Apr 17:35:57 - [info] Node-RED version: v4.0.9
8 Apr 17:35:57 - [info] Node.js  version: v20.19.0
8 Apr 17:35:57 - [info] Linux 5.15.0-1070-raspi arm64 LE
8 Apr 17:35:57 - [info] Loading palette nodes
8 Apr 17:35:59 - [warn] rpi-gpio : Raspberry Pi specific node set inactive
8 Apr 17:36:00 - [info] Settings file  : /home/david/.node-red/settings.js
8 Apr 17:36:00 - [info] Context store  : 'default' [module=memory]
8 Apr 17:36:00 - [info] User directory : /home/david/.node-red
8 Apr 17:36:00 - [warn] Projects disabled : editorTheme.projects.enabled=false
8 Apr 17:36:00 - [info] Flows file     : /home/david/.node-red/flows.json
8 Apr 17:36:00 - [info] Server now running at http://127.0.0.1:1880/
8 Apr 17:36:00 - [info] Starting flows
8 Apr 17:36:00 - [info] Started flows

Steve-Mcl · 8 April 2025 06:14

In the main workspace of node-red editor. Same effect as right-click whitespace -> "action list".

davidmnz · 8 April 2025 06:29

Ah! Actually Firefox catches ctrl-shift-p and opens a private browser window. Maybe other browsers differ. Anyway:

{
    "report": "diagnostics",
    "scope": "basic",
    "time": {
        "utc": "Tue, 08 Apr 2025 06:26:58 GMT",
        "local": "8/04/2025, 6:26:58 pm"
    },
    "intl": {
        "locale": "en-NZ",
        "timeZone": "Pacific/Auckland"
    },
    "nodejs": {
        "version": "v20.19.0",
        "arch": "arm64",
        "platform": "linux",
        "memoryUsage": {
            "rss": 108584960,
            "heapTotal": 47951872,
            "heapUsed": 40537392,
            "external": 2737491,
            "arrayBuffers": 74014
        }
    },
    "os": {
        "wsl": false,
        "totalmem": 8134819840,
        "freemem": 4227461120,
        "arch": "arm64",
        "loadavg": [
            1.75,
            0.83,
            0.93
        ],
        "platform": "linux",
        "release": "5.15.0-1070-raspi",
        "type": "Linux",
        "uptime": 16802.19,
        "version": "#73-Ubuntu SMP PREEMPT Thu Dec 19 17:56:10 UTC 2024"
    },
    "runtime": {
        "version": "4.0.9",
        "isStarted": true,
        "flows": {
            "state": "start",
            "started": true
        },
        "modules": {
            "node-red": "4.0.9",
            "node-red-contrib-buffer-parser": "3.2.2",
            "node-red-contrib-play-audio": "2.5.0",
            "node-red-node-pi-gpio": "2.0.6",
            "node-red-node-ping": "0.3.3",
            "node-red-node-random": "0.4.1",
            "node-red-node-serialport": "2.0.3",
            "node-red-node-smooth": "0.1.2",
            "node-red-contrib-chartjs": "0.5.2",
            "node-red-contrib-https": "2.0.0"
        },
        "settings": {
            "available": true,
            "apiMaxLength": "UNSET",
            "disableEditor": false,
            "contextStorage": {},
            "debugMaxLength": 1000,
            "editorTheme": {
                "palette": {},
                "projects": {
                    "enabled": false,
                    "workflow": {
                        "mode": "manual"
                    }
                },
                "codeEditor": {
                    "lib": "monaco",
                    "options": {}
                },
                "markdownEditor": {
                    "mermaid": {
                        "enabled": true
                    }
                },
                "multiplayer": {
                    "enabled": false
                }
            },
            "flowFile": "flows.json",
            "mqttReconnectTime": 15000,
            "serialReconnectTime": 15000,
            "socketReconnectTime": "UNSET",
            "socketTimeout": "UNSET",
            "tcpMsgQueueSize": "UNSET",
            "inboundWebSocketTimeout": "UNSET",
            "runtimeState": {
                "enabled": false,
                "ui": false
            },
            "adminAuth": "SET",
            "httpAdminRoot": "/",
            "httpAdminCors": "UNSET",
            "httpNodeAuth": "UNSET",
            "httpNodeRoot": "/",
            "httpNodeCors": "UNSET",
            "httpStatic": "UNSET",
            "httpStaticRoot": "UNSET",
            "httpStaticCors": "UNSET",
            "uiHost": "SET",
            "uiPort": "SET",
            "userDir": "SET",
            "nodesDir": "UNSET"
        }
    }
}

Steve-Mcl · 8 April 2025 06:34

Can you uninstall this, restart node-red, try again?

Topic		Replies	Views
Probems with http request-Node: ERR_GOT_REQUEST_ERROR General	16	76	18 February 2025
Simple HTTP Request not working General	9	1841	12 October 2021
Http-Request - API fatal error Forum Help	3	1194	24 April 2019
Http request kills node-red General	8	1940	27 March 2019
Http request node stopped working after update to node red 2.1 - but only for https://tesla.com (solved) General http-request	3	1049	15 January 2022

Browser request works but exact same node red request fails

Related topics