Does not everyone that gets into NR, at some point write a flow to ping monitor something? Just saying.
Speaking of simulation of a mouse click, I did that once a long time ago. There are a few javascript examples of how to do this, basically you just have to pull the page, parse to get the button control identifier and then you post and event to it. Dang, I don't know where i saved that script, will look for it.
Have you seen this?
My modem happens to use JS, so when I could click the reboot button, I did similar to this example.
There's also browser automation libraries for Node.js (used for automated testing, web crawling). Some seem to have contrib node wrappers also:
- https://flows.nodered.org/node/node-red-contrib-puppeteer-new
- https://flows.nodered.org/node/node-red-contrib-nbrowser
- https://flows.nodered.org/node/node-red-contrib-selenium-webdriver
The first one is quite recent and Puppeteer is one of the most modern libraries for the purpose so it might even work.
The newest AFAIK is Playwright made by the same team as Puppeteer but I doubt there's a NR wrapper for it.
Nice. Good idea too.
You just gave me an idea how to drive my dashboard my invisible hand.
If you're wishing to run these on a Raspberry it might not be straightforward as they might depend on native code (often i686/x86_64) for the browser part. I just remembered I did a quick test using function node and Playwright on a Raspberry Pi to answer a question on the forum: Automate a screen capture?. This might help if the contrib node's don't work.
Edit: Oops, I remembered wrong and I didn't get it to work on a Raspberry as I mention on the topic.
Edit 2: Nightmare.js used by the nbrowser node seems to run on a Raspberry (based on the latest comments) but the version required by contrib-nbrowser is 2.x while the newest is 3.x. I doubt it will work either.
Edit 3: Some say they've got it to work on a Raspberry with some hurdles.
Thanks yes need to verify which resolve the issue:
- Reboot router
- Power off/on router
- Reboot modem
- Power off/on modem (confirmed)
Today power off/on modem completely resolved the issue. Before ping to google between 90-250ms. After ping 13-17ms. Router is on dd wrt and I could event have reboot every night if needed or use some smartphone based automations with tasker.
I have node red running on windows 10 computer.
Thanks for the comment. I would need to resolve entering password on login screen and then clicking on the second screen. I inspected the page and can see javascript called out but I am not smart enough to get the function/event. I will check with a friend.
In this case you might very well have success with the puppeteer node I linked earlier. Based on a quick glance it seemed to have quite a comprehensive set of nodes to work with but it will for sure require some probing with the browser development tools (F12 on Chrome) which you might or might not be familiar with.
My ISP loves me and hates me. I have a python script (which this week I retired for NR flow) that captures the entire state of the modem, logs, statistics, etc., it all gets dropped into a maria DB instance. For about 10 years, I have had to prove various times, that my ISP is doing some odd things, that my local AMP was failing, that the wiring (external to my house) trunk line was failing, etc. The average tech for ISP would come out, do a spot check, declare that I was smoking the coax cable insulation, and claim the issue was resolved and leave. Well of course it was not fixed. Hence I collect the data, and proved the trending evidence... which can be done if you have the data, and get a tech that will listen.
About 2 years ago, a very good tech happened to answer an issue call I made, he told me flat out, I was 'known' to the local ISP tech team as 'trouble' maker, and because I am an IT guy and an avid online gamer, I was always griping about something. I challenged him, and pulled my data. I asked him to look at the down and up signal strength, channel jumps, noise ratios, line resets, etc.
After a few minutes, he said he would be back. Wow, what a difference, he replaced the AMP, he line balanced the signal loading so the cable modem had appropriate leveling, and the modem was no longer screaming back at the ISP because of the significant uncorrectable codeword errors, and of course noise ratio was insane.
While he watched, the number of line resets continued however, as everything else improved with each change he did. Then he walked down the street and opened the local hub junction, and in seconds was in his van, grabbing a spool of coax, and additional tools. In short, he told me the local hub had several damaged terminations, including mine, and because he knew I was a gamer, changed to order of connections in the hub, so I was inline first on the given bus line, in short I get bandwidth before any of my other 7 neighbors... Don't tell them I know this. Even the odd pixelation on my various TVs improved as he validated the new AMP configuration.
I know this is a long story or comment, but it was what he said as he left, that really nailed this entire visit in my memory... "And you'd show this collection of data to other techs that came before?" I replied "I tried." He picked up his tools, told me to call him directly if there were any issues, and as he walked away, said, I am sure just loud enough for me to here, one word, "Idiots."
Thanks
For me it is still puzzling that there are no well / widely known ways to check internet quality. For a service that has more than 95% penetration. Speed test is of course of no use
What i have been using to test so far is MOS from pingplotter and also the puma modem test from dslreports. (I have a non puma modem)
Please let me know if you know of another way
I did notice poor MOS when my video call quality was crap
#1 thing to check is the up and down stream signal values and the DB noise ratio. That is the place to start, that tells you what your modem is doing. If the noise ratio is outside of about -10 to +10, and you have a lot of channel number changes over time, or device resets, then you uncorrectable codeword errors are high, meaning counting up frequently... you should be able to prove to your ISP that they have work to do. If you do a lot of vidoe streaming... it buffers and thus hides a lot of ugly issues. But if you are a gamer, and the upload bandwidth and stability is bad, it makes game play horrible. This is easier to spot as well but because buffering is not usually done with gaming.
The big deal, is to set a baseline, and watch how things change from the base line over time. This data should be applicable to any discussion you have with your ISP. Also, when you talk to the ISP, ask for a level 2 support tech, so you get past the call center script readers... 1st level support, and get to someone that actually knows how things work, not just canned information echo back to you.
Thanks, Right now my internet was a bit iffy (some lite distortion during voip call). I am not restarting since my daughter is in the middle of a lecture.
I just checked (wired LAN) ping to google and it is all over the place between 16ms and 160ms. Other runs have similar spread. After a fresh modem reboot ping to google was 13-18ms very solid
Right now below is what I see on my modem. Would you be able to advise what are the red flags? I have done page scrubbing with node red and maybe I can set alarm from returned values below. I am not sure how to interpret your NR because the SNR below is in dB
Start Up Procedure
Procedure | Status | Comment |
---|---|---|
Acquire Downstream Channel | 621000000 Hz | Locked |
Connectivity State | OK | Operational |
Boot State | OK | Operational |
Configuration File | OK | bac10c000106c47154183cb2 |
Security | Enabled | BPI+ |
Downstream Bonded Channels
Channel | Status | Modulation | Channel ID | Frequency | Power | SNR |
---|---|---|---|---|---|---|
1 | Locked | QAM256 | 12 | 621000000 Hz | -3.8 dBmV | 40.2 dB |
2 | Locked | QAM256 | 5 | 579000000 Hz | -3.3 dBmV | 40.6 dB |
3 | Locked | QAM256 | 6 | 585000000 Hz | -3.1 dBmV | 40.7 dB |
4 | Locked | QAM256 | 7 | 591000000 Hz | -3.2 dBmV | 40.5 dB |
5 | Locked | QAM256 | 8 | 597000000 Hz | -3.5 dBmV | 40.3 dB |
6 | Locked | QAM256 | 9 | 603000000 Hz | -3.6 dBmV | 40.3 dB |
7 | Locked | QAM256 | 10 | 609000000 Hz | -3.3 dBmV | 40.4 dB |
8 | Locked | QAM256 | 11 | 615000000 Hz | -3.4 dBmV | 40.2 dB |
9 | Locked | QAM256 | 1 | 279000000 Hz | -3.8 dBmV | 39.8 dB |
10 | Locked | QAM256 | 13 | 633000000 Hz | -4.2 dBmV | 39.9 dB |
11 | Locked | QAM256 | 14 | 639000000 Hz | -4.3 dBmV | 38.0 dB |
12 | Locked | QAM256 | 15 | 645000000 Hz | -4.0 dBmV | 40.0 dB |
13 | Locked | QAM256 | 16 | 651000000 Hz | -3.9 dBmV | 39.9 dB |
14 | Locked | QAM256 | 17 | 657000000 Hz | -3.7 dBmV | 39.9 dB |
15 | Locked | QAM256 | 18 | 663000000 Hz | -4.0 dBmV | 39.7 dB |
16 | Locked | QAM256 | 19 | 669000000 Hz | -4.2 dBmV | 39.7 dB |
17 | Locked | QAM256 | 20 | 675000000 Hz | -4.2 dBmV | 39.0 dB |
18 | Locked | QAM256 | 21 | 681000000 Hz | -4.0 dBmV | 39.3 dB |
19 | Locked | QAM256 | 22 | 687000000 Hz | -3.7 dBmV | 39.4 dB |
20 | Locked | QAM256 | 23 | 693000000 Hz | -3.8 dBmV | 39.5 dB |
21 | Locked | QAM256 | 24 | 699000000 Hz | -3.9 dBmV | 39.4 dB |
22 | Locked | QAM256 | 25 | 705000000 Hz | -3.8 dBmV | 39.4 dB |
23 | Locked | QAM256 | 26 | 711000000 Hz | -3.4 dBmV | 39.6 dB |
24 | Locked | QAM256 | 27 | 717000000 Hz | -3.2 dBmV | 39.9 dB |
Upstream Bonded Channels
Channel | Status | Channel Type | Channel ID | Symbol Rate | Frequency | Power |
---|---|---|---|---|---|---|
1 | Locked | ATDMA | 4 | 5120 ksym/sec | 38700000 Hz | 46.3 dBmV |
2 | Locked | ATDMA | 2 | 5120 ksym/sec | 25900000 Hz | 46.5 dBmV |
3 | Locked | ATDMA | 3 | 5120 ksym/sec | 32300000 Hz | 47.0 dBmV |
4 | Locked | TDMA and ATDMA | 1 | 2560 ksym/sec | 21100000 Hz | 45.8 dBmV |
5 | Not Locked | Unknown | 0 | 0 ksym/sec | 0 Hz | 0.0 dBmV |
6 | Not Locked | Unknown | 0 | 0 ksym/sec | 0 Hz | 0.0 dBmV |
7 | Not Locked | Unknown | 0 | 0 ksym/sec | 0 Hz | 0.0 dBmV |
8 | Not Locked | Unknown | 0 | 0 ksym/sec | 0 Hz | 0.0 dBmV |
If you are getting odd noise when you talk on a call, that is an upstream issue, if people say you are breaking up, choppy for example. If you hear breaking up of voices, or choppy, that is a downstream issue. Make sure you don't here any breaking of audio (up or down) when NO ONE else is using your bandwidth, that will define your baseline is ok. If you get bad quality when no one else is using your bandwidth, that means either your ISP is limiting something or you are sending and receiving a lot of bad data or modem maybe retrying to get good data.
You should have power between 10/-10, which is ok for most modems. For your make and modem of modem there should be a specification sheet on what is normal ranges, and your ISP should publish the same as well.
As for your ping returns say to www.google.com being both faster and slower, that could be from a number of scenarios. Need more data to know if that is an issue or not. Each route of a ping can vary greatly based on what others are doing at the time you do the ping test. Cable is a SHARED bus, so if others are streaming a lot, you would expect your pings to be slow, etc.
Do you have any view that shows the correctable and uncorrectable codeword errors? This is also a good metric for tracking issues.
Here is a snapshot of my cable modem...
Downstream Bonding Channel Value
Channel ID 15 16 17 18 19 20 21 22
Frequency 663000000 Hz 669000000 Hz 675000000 Hz 681000000 Hz 687000000 Hz 693000000 Hz 699000000 Hz 705000000 Hz
Signal to Noise Ratio 36 dB 36 dB 36 dB 36 dB 37 dB 36 dB 36 dB 36 dB
Downstream Modulation QAM256 QAM256 QAM256 QAM256 QAM256 QAM256 QAM256 QAM256
Power Level
The Downstream Power Level reading is a snapshot taken at the time this page was requested. Please Reload/Refresh this Page for a new reading
-9 dBmV -9 dBmV -9 dBmV -9 dBmV -9 dBmV -9 dBmV -10 dBmV -10 dBmV
Upstream Bonding Channel Value
Channel ID 17 20 19 18
Frequency 17800000 Hz 37000000 Hz 30600000 Hz 24200000 Hz
Ranging Service ID 9281 9281 9281 9281
Symbol Rate 5.120 Msym/sec 5.120 Msym/sec 5.120 Msym/sec 5.120 Msym/sec
Power Level 44 dBmV 47 dBmV 46 dBmV 45 dBmV
Upstream Modulation [3] QPSK
[1] 16QAM
[2] 64QAM
[3] QPSK
[1] 16QAM
[2] 64QAM
[3] QPSK
[1] 16QAM
[2] 64QAM
[3] QPSK
[1] 16QAM
[2] 64QAM
Ranging Status Success Success Success Success
Signal Stats (Codewords) Bonding Channel Value
Channel ID 15 16 17 18 19 20 21 22
Total Unerrored Codewords 3782278605 3782294806 3782259374 3782317826 3782261177 3782258912 3782254714 3782258905
Total Correctable Codewords 8780 9058 10931 4134 687 3032 9960 4669
Total Uncorrectable Codewords 1661 2428 1458 1527 2111 1658 620 2687
Here is the datasheet for my modem...
https://arris.secure.force.com/consumers/articles/General_FAQs/SB6141-Cable-Signal-Level/?l=en_US&fs=RelatedArticle#:~:text=Acceptable%20SNR%20Levels%20(dB)%3A,be%2033%20dB%20or%20greater.
Notice my downstream SNR is below 39, the higher the better is the general rule, but it varies based on the QAM settings, so I have good quality signals coming my AMP, and I have a signal filter on the AMP, so that helps keep the SNR up even though by cable modem is a long distance from my AMP and the ISP hub. But too high is also an issue, which most tutorials do not explain as an issue. But lets ignore that for now.
My upstream SNR is a bit off specification, but again, I have a long distance between point to point as noted above. As long as I am just gaming and surfing, I am not sending a lot of data back to the ISP compared to a business or such, so I am not flooding the up channels. Thus a bit more noise, or lower ratio is ok. If either my downstream or upstream signals degrade, I will see the correctable and uncorrectable errors start adding up faster than in the past. As you can (not) see my error counts are low and stay low... no spiking.
If you can see the codeword counts and the modem logs, to watch the number of line resets, then you can develop over time a good baseline and have NR report anything that varies against the baseline. This is what I do as well. The second I see an increase of resets (and the ISP is not pushing firmware updates or such) or the number of uncorrectable codewords spikes, I have reason to talk to my ISP. Of course if the SNR goes above 40 for downstream and above 45 for upstream in my case as well I talk to my ISP.
Oh, I should have said this before, if anything is out of range for power levels and you have a dedicated AMP, your ISP should be able to adjust accordingly. This should improve both the stability and performance of your internet experience. Even if you have a shared AMP, say in apartment building, there is still some adjustments that the ISP can do to improve things if your SNRs are out of acceptable ranges, it just depend on how many lines you have and how many devices you have on each line.
Thanks, I only know how to access the web interface of the modem, I am not sure if there are other way. What I can see is what I shared. I can also see system log but seem short and some detail is cut out from the web interface. I will try to find out if there a (hidden) way to extend the info on the web interface
One thing I wanted to point out:
- My house is relatively new in a relatively new residential suburb area (built 2014). I would hope this eliminates certain potential issues related to old infrastructure
- My ISP is a small one that buys wholesales bandwidth from one of the large ISPs
Does this make a difference? Would the discussion with my ISP change (i.e. they may have limited capacity to address an issue?)
System Log
Time | Priority | Description |
---|---|---|
Tue Sep 15 08:55:07 2020 | Warning (5) | DHCP WARNING - Non-critical field invalid in response ;CM-MAC=BLURRED!;CMTS-MAC=BLURRED2!;CM-QOS=1.1;CM-VER=... |
Tue Sep 15 08:55:12 2020 | Notice (6) | Honoring MDD; IP provisioning mode = IPv4 |
Tue Sep 15 08:55:01 2020 | Critical (3) | No Ranging Response received - T3 time-out;CM-MAC=BLURRED!;CMTS-MAC=BLURRED2!;CM-QOS=1.0;CM-VER=3.0; |
Tue Sep 15 08:54:50 2020 | Critical (3) | Received Response to Broadcast Maintenance Request, But no Unicast Maintenance opportunities received - T4 time out;CM-MAC=c4... |
Tue Sep 15 08:54:17 2020 | Critical (3) | Started Unicast Maintenance Ranging - No Response received - T3 time-out;CM-MAC=BLURRED!;CMTS-MAC=BLURRED2!;... |
Time Not Established | Warning (5) | DHCP WARNING - Non-critical field invalid in response ;CM-MAC=BLURRED!;CMTS-MAC=BLURRED2!;CM-QOS=1.0;CM-VER=... |
Time Not Established | Notice (6) | Honoring MDD; IP provisioning mode = IPv4 |
Sat Sep 12 21:58:05 2020 | Critical (3) | Started Unicast Maintenance Ranging - No Response received - T3 time-out;CM-MAC=BLURRED!;CMTS-MAC=BLURRED2!;... |
Time Not Established | Critical (3) | No Ranging Response received - T3 time-out;CM-MAC=BLURRED!;CMTS-MAC=BLURRED2!;CM-QOS=1.0;CM-VER=3.0; |
Your log, is pretty much a typical log, ISPs often send control codes/messages, and your modem often reacquire its ip unless you have a static ip, which typically is only done if you host a server or have a business account. You can always request a static ip for extra $$$ with most ISPs.
The log information you showed here, does not show a reset or reboot, which is good, you don't want resets or reboots that are not scheduled, or expected.
Infrastructure issues can happen at anytime, typically water damage is common on the truck lines between your local hub and your house being typically underground. I have had to have my trunk line replaced very few years, even through I live out in the desert, the problem out here is heat, wires have memory, meaning they don't like to be flexed, or move once set in place, every time a trunk line is moved, pushed, etc. say when new lines are pulled or added to the same conduit, it can impact the quality of the physical cable you are using.
Now if you can find in your model setup, where the codeword statistics are, we would have the full picture of what you have to work with. A lot of uncorrectable and/or correctable codewords, adding up over a short period of time is an indicator of several possible issues.
Your ISP is just a VAR (value added reseller), so all technical support comes from the parent ISP right?
My ISP (Carrytel) is not a pure reseller. Below describe their technical model even though is for a different ISP.
In my case, my house came wired by Rogers (a large player) and I got their service for a year. With carrytel, I believe they "reuse" (pay?) to use the last mile getting to my home. Carrytel then connect to their backbone infrastructure and to the internet.
TekSavvy Solutions Inc. is a Canadian residential, business, and wholesale telecommunications company based in Chatham, Ontario. In most of the country, it is a wholesale-network-access-based service provider and voice reseller, connecting its service to existing last mile networks from telecom carriers Bell Canada (including Bell Aliant) and Telus Communications, and cable carriers Rogers Communications, Cogeco Cable, Shaw and Vidéotron. However, in parts of rural southwestern Ontario, the service is provided over TekSavvy's own fixed wireless network (branded as "Sky Fi").
Sadly, based on a quick research sounds like my modem doesnt log codewords...
https://www.amazon.com/review/R1G0O8LGRWNOS8/ref=cm_cr_srp_d_rdp_perm
That is really a shame. You might want to consider replacing it if possible to get such data. It really is key to finding and resolving some of the common issues.