"Wifi scan" (python scrpit) different outputs from different machines

I have two RPIs scanning for available WAPs.

One is showing me this:

{"payload":"[\"TelstraCA5E31\", \"Fon WiFi\", \"Telstra Air\", \"BigPond121C\", \"Telstra6C2C0D\", \"Telstra6C2C0D\", \"PiNet\", \"Tango2\", \"Telstra Air\", \"Fon WiFi\", \"TelstraAC4579\", \"Redmi\", \"TPG-VZBS\", \"Slowest Internet in the World\", \"TelstraDAE48B\", \"Chilli\", \"Telstra Air\", \"Fon WiFi\", \"TPG-S66J\", \"\", \"Interweb Thingy\"]","_msgid":"647867e3.c260e8"}

The other is showing me this:

["Telstra6C2C0D", "Telstra Air", "Fon WiFi", "TPG-VZBS", "TelstraDAE48B", "Telstra Air", "Fon WiFi", "Tango2", "Chilli", "TelstraCA5E31", "Telstra Air", "Fon WiFi", "BigPond121C"]

The problem is that #1 is showing ("zooming in on it")

"Telstra6C2C0D", "Telstra6C2C0D"

But #2 is only showing:

"["Telstra6C2C0D"

Now, ok, I just looked at the bigger list from #1 and I can't see the double of said WAP, but on the debug screen it is there.

(Pictures if requested)

Ok, yes, it will be a dumb question. But they are my speciality.

I've checked the python script on both machines.
Both are the same. So that is confusing. I could get it if they were different, but them being the same....

Please.
Someone?

My guess is you have one Debug node set to show the whole message and one Debug node set to just show msg.payload

1 Like

(If only I had half your skills.)

Thanks, that is true, but alas that explains why I am seeing it in the debug node.

But further down the flow, I am getting a double appearance of that WAP.
(That is why I am chasing the problem. The fact I am seeing two with the debug node is only because I activated them.)

That is a bigger picture of what I am seeing.
I do the scan, split it and switch on the name.

Two messages.

Had you explained that the problem is that you are seeing the WAP repeated it might have saved some time. How are you fetching the array?
What do you see if you run a scan using the command line on the device?
[Edit] Hold on, I am not sure that you are saying that is the problem. Perhaps you should start again and explain the issue, posting a short flow exhibiting it.

Apologies.
(ok, edit now seen)
The two machines scan WiFI WAPs. Each produces its own list on it's "screen" (NR)
I am/was getting weird things happening and so I started looking at the messages sent to the lists.
I saw that one machine was double listing one of the WAPs, but the other isn't.

Digging around I looked at the output of the wifiscan part of the node. (Both machines.)
I saw the double listing there too on one machine, of course.

Going a bit further here is a CLI breakdown of what the commands get.
That way (alas) it seems NR can be excused for being the problem.

But......
The two scripts are the same.
Run on one machine it is one, on the other the WAP isn't double listed.

Just wanting to try and work out why and how to stop it.

(I do remember I asked about something on WAP a long time ago. That was resolved and I don't think it really ties in with this. But I am known to be wrong.)

Any ideas?

Machine #1: (in this case "good")

pi@TimePi:~ $ python3 wifiscan.py 
["Telstra6C2C0D", "Telstra Air", "Fon WiFi", "TelstraDAE48B", "Telstra Air", "Fon WiFi", "Tango2", "TelstraCA5E31", "Telstra Air", "Fon WiFi", "", "BigPond121C"]
pi@TimePi:~ $ 

Machine #2:

pi@TelePi:~ $ python3 wifiscan.py 
["TelstraCA5E31", "Fon WiFi", "Telstra Air", "BigPond121C", "Telstra6C2C0D", "Telstra6C2C0D", "PiNet", "Tango2", "Telstra Air", "Fon WiFi", "TPG-VZBS", "TelstraAC4579", "TelstraDAE48B", "Telstra Air", "Fon WiFi", "", "Slowest Internet in the World", "WiFi-D463", "Chilli"]
pi@TelePi:~ $ 

See the duplicate list for Telstra6C2C0D

Scripts:
Machine #1:

pi@TimePi:~ $ cat wifiscan.py 
#wifiscan.py
import subprocess
import json
import re
 
child = subprocess.Popen('sudo iwlist wlan0 scan | grep ESSID',shell=True,stdout=subprocess.PIPE)
output = child.communicate()[0]
output = output.decode("utf8")
output = output.replace('                   ESSID:','')
output = output.replace(' "','')
output = output.replace('"','')
output = output.splitlines()
 
output = json.dumps(output)
 
print (output)
pi@TimePi:~ $ 

Machine #2:

pi@TelePi:~ $ cat wifiscan.py 
#wifiscan.py
import subprocess
import json
import re
 
child = subprocess.Popen('sudo iwlist wlan0 scan | grep ESSID',shell=True,stdout=subprocess.PIPE)
output = child.communicate()[0]
output = output.decode("utf8")
output = output.replace('                   ESSID:','')
output = output.replace(' "','')
output = output.replace('"','')
output = output.splitlines()
 
output = json.dumps(output)
 
print (output)
pi@TelePi:~ $ 

Try
sudo iwlist scan

So what exactly is the question?

  1. why are you getting duplicating entries on one Pi and not the other?
  2. how to remove the duplicate in your Python code?
  3. how to remove the duplicate in your Node-RED flow?

For #1 - I don't think we can tell you why iwlist outputs one of the ssids twice. I would look at the full output of iwlist and compare what it says about the two entries for Telstra6C2C0D. Are they identical models of Pi? The Pi3 can work with 5Ghz networks as well as 2.4Ghz networks, so if one was a Pi3 then that may explain two entries (lots of ifs and maybes there... can only guess).

For #2, a quick google finds https://www.w3schools.com/python/python_howto_remove_duplicates.asp

For #3, well, I'd personally go with #2.

1 Like

Well, yes, 1, 2 and 3.

But that is helpful.

Thanks.

I shall look at the link supplied.

One is a RP2B and one is a RPZ(W)

The zero doesn't support 5gig - does it? (ok, rhetorical. I'll look.)
As that is the one which is showing/giving the double entry.

(After looking)

This is the new script:

#wifiscan.py
import subprocess
import json
import re
 
child = subprocess.Popen('sudo iwlist wlan0 scan | grep ESSID',shell=True,stdout=subprocess.PIPE)
output = child.communicate()[0]
output = output.decode("utf8")
output = output.replace('                   ESSID:','')
output = output.replace(' "','')
output = output.replace('"','')
output = output.splitlines()

#  new stuff here
output = list(dict.fromkeys(output))

output = json.dumps(output)
 
print (output)

Yeah, I like to add comments. That seemed to fix it.
As far as I know it never used to happen.

But anyway: Thanks.

Just to show all things declared:

This is a log file of what I was getting:
Note: No double entries.
For a while today (I think only today) there were.

2019-6-17 07:23:50 TimePi  - Online
2019-6-17 07:23:50 Main__  - Online
>> -- Mark -- 2019-6-17 17:16:33------------
>> -- Mark -- 2019-6-17 17:51:26------------
>> -- Mark -- 2019-6-19 19:55:05------------
>> -- Mark -- 2019-6-27 07:16:56------------
>> -- Mark -- 2019-6-29 23:53:11------------
----< 2019/7/1 >----
2019-7-1 07:35:45 Main__  - Online
2019-7-1 07:35:46 TimePi  - Online
2019-7-1 07:36:25 Main__  - Online
2019-7-1 07:36:25 TimePi  - Online
2019-7-1 07:38:25 Main__  - Online
2019-7-1 07:38:26 TimePi  - Online
----< 2019/7/2 >----
----< 2019/7/3 >----
----< 2019/7/9 >----

As I said, I would personally run the sudo iwlist wlan0 scan command manually and then compare the duplicate entries it returns for any clues.

Or I would just deduplicate it in the python code and get on with the next thing.

I edited the python script to remove duplicates.

The "weird" thing is that this has only recently started happening.

That log shows it used to not have duplicates. Today I noticed it was.
(So something must have caused it.)

I'm not going to lose too much sleep on it, but it threw me doing other things.

I did do the command you said (and Colin) and there is another interesting thing:
There are TWO entries with the same name, etc.

One is a lot bigger than the other. But they have the same address (mac?) and details.
Just one or two lines are different.

It is what it is.
But I found the solution to the problem as I see it.

(Mr. Paranoia speaking)
Maybe it is someone spoofing my WAP or something? (Eeeek!) Kidding.

I have the output and could post it, but I think it isn't really that important.
But if you wouldn't mind having a quick look....

Post it, I am interested. Just the duplicate entries of course, not the whole thing.

The two entries:

          Cell 05 - Address: C4:EA:1D:6C:2C:0D      <--------------
                    Channel:6
                    Frequency:2.437 GHz (Channel 6)
                    Quality=70/70  Signal level=-38 dBm  
                    Encryption key:on
                    ESSID:"Telstra6C2C0D"              <-------------------------
                    Bit Rates:1 Mb/s; 2 Mb/s; 5.5 Mb/s; 11 Mb/s; 18 Mb/s
                              24 Mb/s; 36 Mb/s; 54 Mb/s
                    Bit Rates:6 Mb/s; 9 Mb/s; 12 Mb/s; 48 Mb/s
                    Mode:Master
                    Extra:tsf=0000000000000000
                    Extra: Last beacon: 68014440ms ago
                    IE: Unknown: 000D54656C73747261364332433044
                    IE: Unknown: 010882848B9624B0486C
                    IE: Unknown: 030106
                    IE: Unknown: 050400010000
                    IE: Unknown: 2A0100
                    IE: Unknown: 32048C129860
                    IE: IEEE 802.11i/WPA2 Version 1
                        Group Cipher : CCMP
                        Pairwise Ciphers (1) : CCMP
                        Authentication Suites (1) : PSK
                    IE: Unknown: 46053000000000
                    IE: Unknown: 2D1A1C081BFFFF000000000000000000000000000000000000000000
                    IE: Unknown: 3D1606080400000000000000000000000000000000000000
                    IE: Unknown: 7F080400080000400040
                    IE: Unknown: DD1D0050F204104A0001101044000102103C0001031049000600372A000120
                    IE: Unknown: DD090010180201000C0000
                    IE: Unknown: DD180050F2020101040003A4000027A4000042435E0062322F00

          Cell 06 - Address: C4:EA:1D:6C:2C:0D        <-----------------
                    Channel:1
                    Frequency:2.412 GHz (Channel 1)
                    Quality=65/70  Signal level=-45 dBm  
                    Encryption key:on
                    ESSID:"Telstra6C2C0D"               <---------------------------
                    Bit Rates:1 Mb/s; 2 Mb/s; 5.5 Mb/s; 11 Mb/s; 18 Mb/s
                              24 Mb/s; 36 Mb/s; 54 Mb/s
                    Bit Rates:6 Mb/s; 9 Mb/s; 12 Mb/s; 48 Mb/s
                    Mode:Master
                    Extra:tsf=0000000000000000
                    Extra: Last beacon: 90ms ago
                    IE: Unknown: 000D54656C73747261364332433044
                    IE: Unknown: 010882848B9624B0486C
                    IE: Unknown: 030101
                    IE: Unknown: 2A0100
                    IE: Unknown: 32048C129860
                    IE: IEEE 802.11i/WPA2 Version 1
                        Group Cipher : CCMP
                        Pairwise Ciphers (1) : CCMP
                        Authentication Suites (1) : PSK
                    IE: Unknown: 46053000000000
                    IE: Unknown: 2D1A1C081BFFFF000000000000000000000000000000000000000000
                    IE: Unknown: 3D1601081500000000000000000000000000000000000000
                    IE: Unknown: 7F080400080000400040
                    IE: Unknown: DD980050F204104A0001101044000102103B0001031047001043A3BE9E7F60502C97B6A99085E141EE1021000B546563686E69636F6C6F721023000E546563686E69636F6C6F7220544710240006373939766163104200093135343153415757391054000800060050F204000110110014546563686E69636F6C6F72205447373939766163100800020784103C0001031049000600372A000120
                    IE: Unknown: DD090010180201000C0000
                    IE: Unknown: DD180050F2020101040003A4000027A4000042435E0062322F00

Am I going nuts seeing this?

Nope they are on different channels - if i had to guess i would say either the unit has multiple radios (doubtful) or it is jumping around trying to find a channel it is happy with - what model of Telstra Router do you have ? Are you using the PIs as WIFI extenders or just straight WAPs with an ethernet backhaul into the main network ?

Yeah, ok "different channels".....

But I don't get why my router/WAP is broadcasting on two channels. It is set up for Oz. (Dunno actually if it even has an option for country. It is an ISP supplied router.)

So it is "broadcasting" on two channels - dunno why - why ......

Ok. It is saying "I am a WAP. Here I am."

But, it never used to.

As shown in the short log I posted back in # 9.
(Main__) The __ are to format the columns so the names match.
Only 1 listing. So, somewhere/time between "then" and "now" that modem decided to start broadcasting on two channels.

W E I R D.

(Well, to me it is.)

Anxiously awaiting sparky to come over and install a new power point so I can commission my new router and get rid of that one.

I believe that is actually the case with some of them, allowing for connection to both T.Air and Fon. Perhaps that is what he is seeing.

I think (actually I'm pretty sure) those would have different ID's as they would be different devices within the router.

I think it much more likely that this is a timing/caching issue with the software. It is quite common for smart WiFi firmware to hop channels so I'm thinking that the software has simply seen the two channels within a scanning period?

Probably something to raise with the authors of iwlist. The most definitive source I could find for that tool seems to belong to HP:

Maybe raise an issue against that repo.

Ok, Craig:

I think it is the "Gateway MAX".

Black, flat modem.
Looking at this link:Second modem in picture - the flat one.

The router is one WAP. One IP range. (Main in the list.)
TimePi is another WAP on a different IP range. NOT an extender.
They are connected to each other via Cat-5 as you said.
TimePi also scans for WAPS - but can't see itself.
TelePi is a RPZ(w) and is connected to the Main WAP.

Ok, so the Main one is broadcasting on two channels. (Indulging theories.)
That's ok as stated, but why is it only TelePi was seeing it and not both TimePi and TelePi?

They are about 2 (3) meters apart, with little between them.
So if one can "see" a WAP, the other should too. You'd think.

It isn't too much of a worry, just that it caused a bit of a glitch when I was checking something the other day.
As stated: TimePi didn't used to see two entries of the one WAP before.

(Spoke to sparky again and maybe next week he will come and install the 240 socket so I can get the Linksys router up rather than that one.)

Anyway, thanks to all.

It was just an interesting journey with what happened and why.

I just hope it doesn't happen again when the new router is brought online.

Neighbors ? I mean, I see 66 ESSID's and a number have the same names :')

Yeah, but although I didn't change the name, I believe (foolishly?) that all routers are given unique names.

But gee, there are some cheeky names I am seeing in my lists.