I have a webhook listener set up (HTTP In Node), and an online service sending JSON webhook requests to this URL. After each restart of Node-RED nothing comes out of this node.
As soon as I disable parsing in the node -> deploy -> enable parsing -> deploy, then I start getting the messages.
When I request the URL manually I get a response (something like "nothing here" which makes sense for a GET request when the node is setup for a POST request).
It seems like the HTTP In node does not properly start up or something.
Is this a known issue? (I think it was the same in 4.1.7 at least; but I am not sure).
This sounds like a known issue where the incoming request claims to contain JSON, but the actual body is invalid and the error gets swallowed. This is why disabling parsing allows the messages through.
What does body of the request look like if you pass the message to a Debug node (configured to show full message)?
Thanks. But the odd thing is the input is not changing. I just disable and re-enable parsing, and then the messages start coming through.
I have no reason to suspect anything wrong with the requests. There are heartbeat requests regularly. For the first few hours after starting Node-Red, I don't see these. After disabling and re-enabling parsing (with deploys in between) I start seeing these messages.
I'd check the very first request after restart with parsing turned off and log msg.req.headers, msg.req.rawBody (or the raw payload) before anything else touches it. This smells like the sender is claiming application/json but the first body is empty or slightly malformed, and toggling parse resets the parser state.
If you need a workaround now, leave the HTTP In node as plain text / buffer and run it through a JSON node after a quick guard like if (msg.payload && msg.payload.length). That usually makes the bad first request obvious instead of swallowing it.
Yes, messages with raw body data come through when disabling parsing and deploying. It carries the raw bytes. I have not tried parsing this, but I assume it is just the JSON data.
Thanks. But I don't see why the requests should be malformed for the exact period of time between start, and me toggling the parsing setting. So for example heartbeats being sent every 10 minutes (it is from eNode) that are malformed for these exact hours only.
The sender is eNode, so I think the requests are likely to be correct. Also it has been working for years without problems, and all requests received after I toggle the parsing setting are correct.
To me it looks more like a "server side" problem in Node-Red. And it started recently, possibly coinciding with upgrading to 4.1.8.
Node-Red is running in a Docker container on a Synology NAS. There is a reverse proxy in front for TLS-offloading, but all of this has also been working correctly for years.
I'm not sure if there is anything I could do to better troubleshoot this issue?
One thing, though: When setting the node to not parse, it emits a message with byte values (shown as hex values in the debug console) and not a string. Maybe that is a hint, somehow?
Capturing it on the wire before it is processed by Node-RED at all will help determine exactly which path it takes in the parsing middleware.
Also if you can supply a sample of the capture from the wire then we may be able to try and reproduce the problem to debug it.
But if it's showing as bytes that does imply that it may have incorrect content-type headers.
Also do not assume the bytes are correct, run them through the JSON node to prove it. Also clicking on the bytes in the debug side bar should convert it to a string if possible so you can see if it is correct
To rule out the possibility that it is something in your flow, feed the http node only into a debug node, and nothing else. Then restart and make sure that you see correct messages after the disable/enable parsing.
I set the HTTP In node to only send its message to a debug node and the http response node (with parsing still enabled), and restarted the container. I waited past an expected heartbeat message (every 10 minutes).
When the first heartbeat message was expected (but not possible to verify it was actually sent) the debug node received nothing.
I then disabled parsing, and the next message contained (in raw):
After that I re-enabled parsing, and the next heartbeat arrived just when expected (plus a few additional actual data payloads), and correctly parsed.
I did not capture raw packets as these are TLS encrypted anyway. Not sure if it is workable to packet capture in the Docker image, without too much trouble.
So it seems quite clear to me that this is somehow caused by the Http In node. I find it extremely unlikely that eNode sends bad requests every time I restart my Node-Red container, only to start sending correct requests as soon as I change the parsing option of this node.
By the way: I hit on another quirk: after removing the connection to the debug node, setting the correct connections, and redeploying, I still got the console log entries for each message. No matter how many times I added a debug connection and removed it again, it just kept appearing the debug console for each message. So no way to stop debugging again. Strange.
A few of these are my own development. But this has been running for a long time with no issues like this before. I did not update an nydes recently, as far as I recall. But I did update from 4.1.7 to 4.1.8 recently.
Yes, well it occurs now, after restarting and trying the same procedure again. I would like to limit the number of restarts as this is a production system, so I have not restarted after removing the connection. And also I would like to prioritize the [Http In] issue. But I could perhaps try restarting tomorrow.
I don't have enough imagination to see how these two issues could be related
