Debugging http in node issues

General
41

I'm a novice when it comes to network debugging, so I was just reading about tcpdump. I found this example to isolating http GET requests:

tcpdump -vvAls0 | grep 'GET'

Is that continuous output like a tail -f and would that work if I run it on the UDM as well as the pi?

42

You need to look for traffic on port 1880. Close down any browser windows connected to node-red so that they don't add to the traffic. Or maybe look for traffic from the Tasmota IP
I don't use tcpdump very often but my notes tell me that this site has lots of useful examples. https://danielmiessler.com/study/tcpdump/. I see right near the beginning there is a command that you could use to show traffic from the Tasmota IP

1 Like
43

Yeah, that's where I got the command I pasted above. Thanks! I will try and see.

44

OK! I installed tcpdump and ran the test. It appears that the webhook IS hitting the pi and that it's not a unifios issue at all:

This is the command issued on tasmota:

WebSend [http://raspberrypi.local:1880] /desklighton

raspberrypi[2022-10-30 11:07:17]:~>sudo tcpdump host 192.168.1.43
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on wlan0, link-type EN10MB (Ethernet), capture size 262144 bytes
11:07:29.265139 ARP, Request who-has unifi.localdomain tell desklight-0587.localdomain, length 28
11:07:29.266459 IP desklight-0587.localdomain.63942 > 255.255.255.255.0: Flags [S], seq 32780624, win 5840, options [mss 1460], length 0
11:07:32.037427 IP desklight-0587.localdomain.63942 > 255.255.255.255.0: Flags [S], seq 32780624, win 5840, options [mss 1460], length 0
11:07:34.346889 IP desklight-0587.localdomain.63942 > 255.255.255.255.0: Flags [R.], seq 32780625, ack 0, win 53270, length 0

And the second call to the second GET webhook (WebSend [http://raspberrypi.local:1880] /desklightoff)...

11:07:56.916772 IP desklight-0587.localdomain.54401 > 255.255.255.255.0: Flags [S], seq 33178615, win 5840, options [mss 1460], length 0
11:07:59.680865 IP desklight-0587.localdomain.54401 > 255.255.255.255.0: Flags [S], seq 33178615, win 5840, options [mss 1460], length 0
11:08:02.138295 IP desklight-0587.localdomain.54401 > 255.255.255.255.0: Flags [R.], seq 33178616, ack 0, win 53270, length 0

And when I control-c:

7 packets captured
7 packets received by filter
0 packets dropped by kernel

So now where do we look?

45

I also tried:

raspberrypi[2022-10-30 11:17:33]:~>sudo tcpdump dst 192.168.1.43
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on wlan0, link-type EN10MB (Ethernet), capture size 262144 bytes
^C
0 packets captured
0 packets received by filter
0 packets dropped by kernel

Nothing is sent back.

46

Is that desklight the Tasmota device? If so then isn't that sending to 255.255.255.255 rather than to the Pi?

What do you see if you put the address of the pi in for the host? That should show you traffic going to and from it. Add the port to reduce the amount shown.

47

I don't know what the 255 address stuff is. I assumed it was some sort of mask thing. Yes. The tasmota is running on the Sonoff, which is named desk light. I would have used it's static dhcp address which is 192.168.1.43 when I ran the test, but I couldn't remember it at the time. My other tests had shown it makes no different if I use the mDNS address or static DHCP address, so I assumed equivalency.

I'll run it on the dhcp address. And I'll also check the 255 thing, even though I have no idea what that is, and report back.

48

If you look at the tcpdump guide you will see that it should show the ip address the message is addressed to.

49

Wait. I think I'm confused. I'm running the tcpdump command on the pi. I can't run it on the tasmota, to my knowledge. It's sending its webhooks to http://raspberrypi.local:1880/desklighton. So it's getting to the pi, because the tcpdump running on the pi shows activity when I run the tasmota websend. I think the pi DHCP address is 192.168.1.56. Let me try that on the tasmota console.

50

I believe tcpdump will show you any traffic that passes by the pi, even if it is not for the pi. To show traffic that is specifically for the pi then specify dst with the pi ip address in the tcpdump command.

[Edit] You said you had other webhooks that were working. You can also look at those with tcpdump to see what you should see.

51

Thanks for that! I'm on a learning curve here. You're right! It looks like the tasmota websend is going to 255.255.255.255. And my tests show that that's happening for both sonoff devices. When I monitor dst 255... it shows the same output as above.

Now I feel like I'm getting somewhere! You're a great help! Now the question is, what is causing these communications to go to 255... I tried your suggestion and used a working webhook (from node red running on a mac to the pi:

>sudo tcpdump src 192.168.1.97
13:07:40.400678 IP 192.168.1.97.62169 > raspberrypi.localdomain.1880: Flags [S], seq 4051085406, win 65535, options [mss 1460,nop,wscale 6,nop,nop,TS val 659720069 ecr 0,sackOK,eol], length 0
13:07:40.402239 IP 192.168.1.97.62169 > raspberrypi.localdomain.1880: Flags [.], ack 4186889120, win 2058, options [nop,nop,TS val 659720074 ecr 712417234], length 0
13:07:40.403769 IP 192.168.1.97.62169 > raspberrypi.localdomain.1880: Flags [P.], seq 0:172, ack 1, win 2058, options [nop,nop,TS val 659720074 ecr 712417234], length 172
13:07:40.440047 IP 192.168.1.97.62169 > raspberrypi.localdomain.1880: Flags [.], ack 319, win 2053, options [nop,nop,TS val 659720111 ecr 712417271], length 0
13:07:40.441084 IP 192.168.1.97.62169 > raspberrypi.localdomain.1880: Flags [F.], seq 172, ack 319, win 2053, options [nop,nop,TS val 659720112 ecr 712417271], length 0
13:07:40.659841 IP 192.168.1.97.62169 > raspberrypi.localdomain.1880: Flags [.], ack 320, win 2053, options [nop,nop,TS val 659720332 ecr 712417492], length 0
13:07:41.410892 IP 192.168.1.97.mdns > 224.0.0.251.mdns: 0*- [0q] 2/0/6 TXT "model=Macmini8,1" "osxvers=21", PTR golrath._googlecast._tcp.local. (357)
13:07:42.130129 IP 192.168.1.97.mdns > 224.0.0.251.mdns: 0*- [0q] 2/0/0 TXT "model=Macmini8,1" "osxvers=21", PTR golrath._companion-link._tcp.local. (121)
13:07:45.486477 ARP, Reply 192.168.1.97 is-at 14:9d:99:7b:5e:9f (oui Unknown), length 46
^C
9 packets captured
9 packets received by filter
0 packets dropped by kernel

192.168.1.97 is the mac mini. It's doing a POST (not a GET like the tasmota - all my other webhooks are POSTS) to http://raspberrypi.local:1880/notify. And it worked. Node red running on the pi acted on the input.

I'm back to suspecting unifios. Could it be making an mDNS mistake and mismapping desklight-0587.localdomain to 255.255.255.255?

52

Don't you mean it is mismapping the pi domain? But that would only be when the desklight asks for it. Can you put the pi ip address in the desklight to check if it then works?

53

Yeah, sorry, I misspoke. And yes, I tried the actual pi's static dhcp address. Same non-result. (I've tried it many times.) I still get 255.

What I mean by the mismapping is that the routing reported by tcpdump shows that in the working state, it's sending the traffic to raspberrypi.localdomain.1880, which I infer to mean that it is defaulting to its own generated .localdomain DNS addressing, but in the broken state, it's taking 192.168.1.56.1880 and sending to 255.255.255.255.0.

54

What device you suggesting is doing that?

Google suggests you can also use tcpdump in your router, plus other tools to work out what is going on.

55

Well, my bet is on the UDM and its unifios update. Technically, it could be the tasmota, but I have not updated it since initially setting it up and I don't think it even has an autoupdate capability.

I can't think of anything else that could be determining the 255 outcome.

56

You know, I have a discord chat going with the tasmota developer in another window. They say that that 255 address is the "broadcast" address. I don't know what that means exactly, but my feeling is that it could suggest an mDNS issue.

57

Are you running with tasmota configured with the IP address? That takes DNS out of the problem.

58

I don't see a place in the tasmota config where you set its address (mDNS or IP). I'm pretty sure I never gave it an address. I just looked up what address it was assigned on the UDM and set it to static in the UDM's device settings for that device (on the UDM).

But it's the pi's address that's getting mismapped (but only when the GET is coming from the sonoffs/tasmota). For all I know, it could be all GETs that are messing up. So I suppose that could be a test I could do. But, it's getting late. I need to get some other stuff done before 4, so I'm going to have to set this aside for now. I did create another Ubiquiti ticket with what we've learned though.

59

I meant the pi ip address as configured in Tasmota.

60

Right. That's the first thing I tried last week. I changed the rule from raspberrypi.local to 192.168.1.56, but to test, I'm running the websend command manually and I have been trying all 3 variants: 192.168.1.56 , raspberrypi.local, and raspberrypi.localdomain.