### Author:  [@hazbot](https://discourse.nodered.org/u/hazbot)
#### Post date: [23 June 2024 19:57 UTC](https://discourse.nodered.org/t/strange-issue-with-http-request-after-upgrade-to-4-0/88892/1 "2024-06-23T19:57:15Z")
Hi all
Have been running NodeRed for quite a few years and up until recently was running 3.1.9 using the official amd64 Docker image. Last Friday I updated the container to 4.0, with no other changes. Since then I have been having a weird issue and wondering if anyone else has seen similar.
One of my flows sends notification messages (maybe 50 or 100+ messages per day) by using the http-request node to send commands to the Discord bot API. After the upgrade to 4.0, these messages stopped abruptly.
When I view the output from the http-request node, it shows HTTP error 403, and an error text returned from the remote server "internal network error, code 40333"
A bit of googling suggests this error is coming from the Cloudflare frontend deployed by Discord and usually occurs when a HTTP request sent to the API has been malformed in some way.
The timing of the issue coincides exactly with my upgrade to 4.0 so I wonder if something in NodeRed, or in node version 20, might have changed the handling of http requests.
If I send the same HTTP request (with same url, same payload and same headers) using cURL on a command-line, the request is successful.
I also deployed a new container of NodeRed 3.1.9 on a different machine and copied just that part of my flow which sends the message to Discord. This flow still works fine on 3.1.9
My current workaround in my original flow under 4.0 is to use an exec node which calls cURL with the same arguments, and this is successful.
Bit stumped here and wondering if anyone else has encountered anything similar.
Fergus
---
### Author:  [@GogoVega](https://discourse.nodered.org/u/GogoVega)
#### Post date: [23 June 2024 21:19 UTC](https://discourse.nodered.org/t/strange-issue-with-http-request-after-upgrade-to-4-0/88892/2 "2024-06-23T21:19:52Z")
Also another issue with this node:
>
>
> \### Current Behavior
>
> The ver3.1.3's http-request flow back "statusCode: 500"… after upgrading to 4.0,\[hassos node-red addons\]
> Copy the flow to node-red 3.13 can back the right msg；
> I checked the request information and found that the “content-type” was not in the returned msg, but I configured it in the function.
>
> After reading the 4.0 release, I found the following updates. Not sure if these updates are the cause of this issue.
> \*\*Other updates
> The following configuration options have been added to the default settings file:
> httpAdminCookieOptions can be used to customise the options set on cookies as part of the authentication system.
> httpStaticCors can be used to set custom cross-origin resource sharing rules for the content served up via the httpStatic option
> node-red --version now reports the node-red, node.js and os information withou\*\*
>
> \### Expected Behavior
>
> Wrong Msg\[ver:4.0.0\]
> !\[image\](https://github.com/node-red/node-red/assets/51348277/7cbcba6e-aace-4e0c-ac73-b962cbc351a6)
> Right Msg \[ver:3.1.3\]
> !\[image\](https://github.com/node-red/node-red/assets/51348277/daedb661-b74f-454a-93a6-1666c879430e)
>
>
>
> \### Steps To Reproduce
>
> \_No response\_
>
> \### Example flow
>
> \_No response\_
>
> \### Environment
>
> \- Node-RED version:4.0.0
> \- Node.js version:20.0/HASSOS DOCKER
> \- npm version:10.3.0
> \- Platform/OS:Windows10/hassos
> \- Browser:Chroms126.0.6478.63
---
### Author:  [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)
#### Post date: [24 June 2024 08:28 UTC](https://discourse.nodered.org/t/strange-issue-with-http-request-after-upgrade-to-4-0/88892/3 "2024-06-24T08:28:59Z")
> [@hazbot](#):
>
> code 40333
> [@hazbot](#):
>
> Cloudflare
These 2 comments lead me to this: [Misleading API response for Cloudflare blocked requests · Issue #6473 · discord/discord-api-docs · GitHub](https://github.com/discord/discord-api-docs/issues/6473#issuecomment-1955405994)
I now wonder if the addition of user-agent (albeit recommended by the RFC) is the problem.
Could you try adding an empty (or alterative) `user-agent` header to your requests?
Essentially, set
```auto
msg.headers['user-agent'] = ''
```
or
```auto
msg.headers['user-agent'] = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.5112.79 Safari/537.36'
```
---
### Author:  [@hazbot](https://discourse.nodered.org/u/hazbot)
#### Post date: [24 June 2024 09:04 UTC](https://discourse.nodered.org/t/strange-issue-with-http-request-after-upgrade-to-4-0/88892/4 "2024-06-24T09:04:14Z")
Yes I did also stumble on that link while googling my issue 🙂 I do suspect that this is somehow related to a User-Agent header.
In my flow I normally send a valid User-Agent, but I tried just now to send a null User-Agent as a test, but the Cloudflare 403 error still occurs.
I suspect the http-request node itself might be overwriting the User-Agent with some internal value of its own, but I don't have a way to confirm this.
Another issue I noticed: while trying to inspect the traffic generated by the http-request node, I set up an intercepting proxy ("Charles" -- [https://www.charlesproxy.com/](https://www.charlesproxy.com/)) . I configured the http-request node to send requests via this proxy. However in my NodeRed 4.0 instance, the proxy setting appears to be ignored and the connection goes out direct, so I can't inspect the headers.
In my NodeRed 3.1.9 instance, the proxy setting in http-request does work and I can see the connection going via Charles as expected
---
### Author:  [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)
#### Post date: [24 June 2024 09:07 UTC](https://discourse.nodered.org/t/strange-issue-with-http-request-after-upgrade-to-4-0/88892/5 "2024-06-24T09:07:55Z")
> [@hazbot](#):
>
> I suspect the http-request node itself might be overwriting the User-Agent with some internal value of its own, but I don't have a way to confirm this.
Correct, i just discovered the http node cannot sent an empty string (will be raising an issue shortly) HOWEVER, i have confirmed that if you set a user agent in `msg.headers` or in the UI it will be sent.
> [@hazbot](#):
>
> Another issue I noticed: while trying to inspect the traffic generated by the http-request node, I set up an intercepting proxy ("Charles" -- [https://www.charlesproxy.com/](https://www.charlesproxy.com/)) . I configured the http-request node to send requests via this proxy. However in my NodeRed 4.0 instance, the proxy setting appears to be ignored and the connection goes out direct, so I can't inspect the headers.
Could you share (private DM if required) further details? Perhaps share an example flow and proxy details so I can test and debug this? There were a lot of changes around proxy handling in NR 4. There may be a bug here. also, please include any `*_proxy` ENV VARs you set (including uppercase/lowercase variants)
---
### Author:  [@hazbot](https://discourse.nodered.org/u/hazbot)
#### Post date: [24 June 2024 09:27 UTC](https://discourse.nodered.org/t/strange-issue-with-http-request-after-upgrade-to-4-0/88892/6 "2024-06-24T09:27:16Z")
Thanks for looking into this.
For the proxy setup, I installed "charles" locally on my Ubuntu machine using the steps from here: [APT repository • Charles Web Debugging Proxy](https://www.charlesproxy.com/documentation/installation/apt-repository/)
Then launch the "charles" application and it starts listening on port 8888. By default there is no authentication.
Inside NodeRed I configured the http-request node with the IP of my Ubuntu machine and port 8888.
To enable SSL decryption in the proxy there is an additional step ... to grab the SSL certificate which Charles generates, and configure that in the http-request node TLS config:
By the way , when I noticed that the proxy settings I had defined in the http-request node did not appear to take effect on NR 4.0 ... I restarted my NodeRed container with the charles proxy defined in the variable "http\_proxy" instead. But this also didn't have any effect that I could see, and the connection still went out direct.
---
### Author:  [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)
#### Post date: [24 June 2024 09:44 UTC](https://discourse.nodered.org/t/strange-issue-with-http-request-after-upgrade-to-4-0/88892/7 "2024-06-24T09:44:39Z")
> [@hazbot](#):
>
> "http\_proxy"
Are you making requests to `http` or `https`
According to my understanding, `http_proxy` is used for `http` requests (and `https_proxy` is used for `https` requests.
---
### Author:  [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)
#### Post date: [24 June 2024 09:54 UTC](https://discourse.nodered.org/t/strange-issue-with-http-request-after-upgrade-to-4-0/88892/8 "2024-06-24T09:54:21Z")
> [@hazbot](#):
>
> In my flow I normally send a valid User-Agent, but I tried just now to send a null User-Agent as a test, but the Cloudflare 403 error still occurs.
Are you able to share details of this endpoint so that I can debug the issue? It would make this much quicker to have a known problematic endpoint to poke at. Share details via DM if you can. If you cannot, then no worries.
PS, i have discovered we are not able to set a header value empty (issue raised [here](https://github.com/node-red/node-red/issues/4789)) so instead, try setting an alternative user-agent is the only option currently.
---
### Author:  [@hazbot](https://discourse.nodered.org/u/hazbot)
#### Post date: [24 June 2024 11:10 UTC](https://discourse.nodered.org/t/strange-issue-with-http-request-after-upgrade-to-4-0/88892/9 "2024-06-24T11:10:34Z")
Hi, that's strange about the User-Agent header, because I have been setting a value for this (tested both by setting msg.headers, and explicitly in the http-request node itself). But I am still getting the Cloudflare error.
I will send a DM with details of a curl command and flow which you can try. The curl command works, but the flow (which I believe should send the exact same http request) does not.
Thanks!
---
### Author:  [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)
#### Post date: [24 June 2024 12:02 UTC](https://discourse.nodered.org/t/strange-issue-with-http-request-after-upgrade-to-4-0/88892/10 "2024-06-24T12:02:58Z")
Thanks to @hazbot sharing some flows and info privately, I have confirmed the issue.
It is fixed in `master` and will be part of a maintenance release soon.
However, in the mean time, for anyone with a similar issue, a simple workaround is to set the `user-agent` manually (note the LOWER\_CASE `user-agent`)
Via UI
Or, via the `msg`
---
### Author:  [@hazbot](https://discourse.nodered.org/u/hazbot)
#### Post date: [24 June 2024 12:55 UTC](https://discourse.nodered.org/t/strange-issue-with-http-request-after-upgrade-to-4-0/88892/11 "2024-06-24T12:55:47Z")
thank you for digging into this @Steve-Mcl
I have changed my flow to send the header as "user-agent" instead of "User-Agent" and can confirm that the Discord API calls are working now.
---
### Author:  [@system](https://discourse.nodered.org/u/system)
#### Post date: [8 July 2024 12:56 UTC](https://discourse.nodered.org/t/strange-issue-with-http-request-after-upgrade-to-4-0/88892/12 "2024-07-08T12:56:03Z")
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.