Debugging http in node issues

General
1

Sometime in the past week, an automation I'd set up stopped working. I have a rule in a "Sonoff s31 15A" outlet running tasmota to make a webhook call under certain circumstances to a pair of http-in (GET) nodes on node red.

I attached a debug node to those http-in nodes and noted that nothing was coming out of them when I triggered the call on tasmota. Tasmota's concole shows an error saying "Connect failed" whenever it tries to trigger its WebSend protocol.

So I tried eliminating tasmota to rule out an issue with the plug - and just went to the URL (http://raspberrypi.local:1880/desklightoff and http://raspberrypi.local:1880/desklighton) in a web browser. The response in the browser window was just {} and the automation didn't happen and nothing came out in the debug pane.

I then restarted node-red and tried again. It worked once and then wouldn't work every subsequent time.

Then I tried the domain raspberrypi.localdomain instead of raspberrypi.local. I actually forget what happened (I've tried so many things), but when I went back to using .local, it started working in the browser every time. But it still wouldn't work when coming from tasmota.

I suppose the next thing I'll do is reboot my router, but does anyone know what could be going on here? The only thing that may have happened in the past week was I cut all my office automations from one flow and pasted them into another - including those http-in nodes. I can't imagine that could have affected anything, but other than that, I can't think of anything that changed.

2

Router reboot didn't fix it. Oh, and I've also tried restarting the tasmota device multiple times. Rebooting the pi now...

3

Pi reboot didn't seem to resolve it either. And now I'm back to the web browser call not working either. Neither .local and .localdomain works. Both just respond {}... And just now, it worked again. It seems intermittent? I'm stumped. Maybe my pi is dying...

4

What do you see if you run, on the machine running the browser (I presume that is on your PC)
ping raspberrypi.local
and on the pi run
ip addr
and show us what that shows too.

Also add a Catch node on the HTTP In node and see if that triggers.

Those are obviously when it isn't working.

5 
>ping raspberrypi.local
ping: Lacking privilege for raw socket.

I did localdomain too:

>ping raspberrypi.localdomain
ping: Lacking privilege for raw socket.

And...

>ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether b8:27:eb:70:2e:5e brd ff:ff:ff:ff:ff:ff
3: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether b8:27:eb:25:7b:0b brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.56/24 brd 192.168.1.255 scope global wlan0
       valid_lft forever preferred_lft forever
    inet6 fe80::ba27:ebff:fe25:7b0b/64 scope link 
       valid_lft forever preferred_lft forever
4: tun0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN group default qlen 100
    link/none 
    inet 10.8.0.1/24 brd 10.8.0.255 scope global tun0
       valid_lft forever preferred_lft forever
    inet6 fe80::e9f4:1667:7703:d523/64 scope link flags 800 
       valid_lft forever preferred_lft forever

That's what I see when it isn't working.

6

If you get a response then node-red (and the endpoint) is working - perhaps something your flow does downstream from the http in node is misbehaving?

I would put debug node on every node between the http-in and the http-response node & see if you can spot some oddity.

you should see the debug nodes output to the side bar of the node-red editor, every time you access the URL in your browser.

7

I have put a number of debug nodes in the flow, including directly from the http-in node. Nothing gets into the debug console.

8

Could there be a separate node red instance running?

9

I don't think so. Unless you are using a different port number for the endpoint to what you have in the browser where the node-red editor is showing.

Either that, or you have a DNS issue and where your browser is pointing is not where you think.

Could you try IP addresses instead of host names for your testing (accessing editor, testing endpoint etc) & see what happens?

10

As Steve suggests, try using 192.168.1.56 instead of the domain name.

The purpose of using ping was to check what IP the PC is using to get to the Pi. I see you are using a mac. If you google for
ping: Lacking privilege for raw socket
then I expect you will find solutions to fix that. It is worth fixing so you can check it is using the correct IP address .

I presume that means you are running a VPN server on the Pi. I suggest you stop that, run ip addr again to make sure it is stopped and try again. See if that makes any difference at all. Don't forget that the server will probably start again if you reboot the pi.

11

I will try the IP address next chance. I was teaching a dance class last night, so I didn't have time for debugging any more yesterday.

I don't run a VPN server on the pi. I don't know why there's a 10.x.x.x scheme mentioned. I've had a UDM for a few years now and it uses 192.168.x.x addresses, though even with it and a remote UDM wired backhaul in the attic, their signals aren't strong enough for my security cameras, so I still have my AirPort Extreme on the network (in bridge mode or something - I don't remember). When it was my main router, it used the 10.x.x.x addresses and I had to switch everything over to new fixed DHCP addresses.

I don't know how any of that could be relevant to an issue that started after years of working in the current configuration, but I'm no network expert.

I have had issues with Bonjour/mDNS in the past, but nothing that manifested like what I'm seeing here (I.e. a response in the browser but nothing in the debug console). But I'm optimistic that you guys may be right about it being a DNS issue. I will try using the IP.

12

So I just tried ping again just now before doing anything and this time I didn't get the error. (Perhaps ping failed yesterday because I had a VPN connection to work going on my mac? Does that make sense?)

>ping raspberrypi.local
PING raspberrypi.local (192.168.1.56) 56(84) bytes of data.
64 bytes from raspberrypi.localdomain (192.168.1.56): icmp_seq=1 ttl=64 time=0.212 ms
64 bytes from raspberrypi.localdomain (192.168.1.56): icmp_seq=2 ttl=64 time=0.193 ms
64 bytes from raspberrypi.localdomain (192.168.1.56): icmp_seq=3 ttl=64 time=0.134 ms
13

It's still not working using .local (or .localdomain). I still see {} in the browser and nothing in the console.

BUT, when I use the IP address, I still see {} in the browser, but now I see this in the console coming out of the http-in node:

response

I don't see that when I use the mDNS address (raspberrypi.local), so it looks like it is a DNS issue.

I was having lots of bonjour/mDNS issues in the distant past, but I thought they'd gone away and everything has been working fine for years now... sigh

But I don't understand why:

  • ping doesn't have that error today (unless it was my VPN connection to work?)
  • I see a response in the browser and nothing in the debug console (when using .local)
  • My Sonoff S31 15A shows "Connect Failed" when my browser seems to think it's getting a response
14

Oh geez. So I switched the tasmota config over to using the IP address instead of the Bonjour/mDNS address and I still get a "Connect failed" error when it tries to do a WebSend command to make the webhook call! Ugh. What the heck?

I have a second Sonoff S31 15A set up the same way. Let me check if it's doing the same thing to rule out an issue with the smart plug itself.

15

Certainly you should stop that while doing tests in case it is an issue.

Well something has created that interface. I am not sure there is any way of identifying what though. If you don't know what is causing something as major as creating an ip interface then you are in a difficult situation.

Looking through syslog from boot might identify it. In fact, searching syslog for 10.8.0 might be fruitful.

Have you got a Catch node on the http in node, as suggested earlier?
Also on the http in node output make sure the debug node is showing the complete message to see if anything helpful is there.

16

Oh, I forgot the catch node suggestion, thanks. I'll do that.

I can say that I did set up the ability for the pi to connect to a VPN provider a long time ago, but I never use it. But what little I know about network config, I wouldn't think that setting up the ability to connect to a VPN would "create an interface", but then again, I only know enough about network config to get myself in trouble, so I don't mess with it unless I have a reason to do so. And I can say that I haven't done anything in that category in years...

I was poking around in my UDM settings and I noted that auto-update is turned on. It runs every Monday at 12AM. That coincides with when I started having this issue. I'm in a queue right now for Ubiquiti support live chat to see if they can help.

The other sonoff, BTW is doing the same thing as this one. "Connect failed".

Off to do the catch...

17

The tun0 interface isn't from connecting to a VPN it is something running on your system that creates an interface, such as a VPN.

18

Now I can't make it not work in the browser (whether I use the IP or mDNS), but tasmota gives me the error every time, whether I'm using the IP or mDNS. This is exasperating.

(Though one thing I noticed is that if I use the browser's back button, it seems to get a response, but nothing shows up in the console - my guess is that it's giving me a cached version of the response, because if I reload, it works.)

OK, let me search the log like you suggested. And I'll have to think more about the interface. Maybe I kept some notes that would explain it somewhere. I'll search my files for tun0. Perhaps it has to do with my efforts to fix mDNS issues from years ago...?

19

boot.log has:

         My IP address is 10.8.0.1

!?

20

I have not heard of boot.log. Where is it?
/var/log/syslog is the normal place for system log.