Help with how to get "speed test" results saved

(Ok, maybe dashboard but I want the mechanics more than the dressing)

I'm guessing most countries have a speed test site that tests their down/up load speeds and given it to you in/on a nice screen.

I'm wanting to see if I can do that from within NR and get the results saved.
Or, at worst: build my own flow that could do this.

Possible?

P.S.
It isn't so much the DOWNLOAD speed, but the UPLOAD speed.
For the past few times I've tested it, it has been <1mb/s.

ADSL speed. :frowning:

ITMT, this is a flow I knocked up.

(Maybe someone can cast their eyes over it.)

External node needed:
speedtest node

[{"id":"ef4a762fbd189044","type":"inject","z":"28c631fa.7e866e","g":"7f16dc04ae3040ee","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":3580,"y":1920,"wires":[["cdec8dfd8222d653"]]},{"id":"cdec8dfd8222d653","type":"speedtest","z":"28c631fa.7e866e","g":"7f16dc04ae3040ee","name":"","maxTime":5000,"serverId":"","acceptLicense":true,"acceptGdpr":true,"x":3730,"y":1920,"wires":[["cca85169024b2e11","9283b97f60564c7e"]]},{"id":"cca85169024b2e11","type":"function","z":"28c631fa.7e866e","g":"7f16dc04ae3040ee","name":"Now()","func":"msg.time = time();\nfunction time() { return { time: new Date().toLocaleTimeString() } };\n//msg.time = msg.goo.time\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":3855,"y":1920,"wires":[["e0dca7fc6115e6ba"]],"l":false},{"id":"e0dca7fc6115e6ba","type":"moment","z":"28c631fa.7e866e","g":"7f16dc04ae3040ee","name":"","topic":"","input":"time.time","inputType":"msg","inTz":"Australia/Sydney","adjAmount":0,"adjType":"days","adjDir":"add","format":"YYYY-MM-DD HH:mm:ss","locale":"en_AU","output":"time","outputType":"msg","outTz":"Australia/Sydney","x":3925,"y":1920,"wires":[["4698096ddefe25ac","6c633d88076d5b40"]],"l":false},{"id":"4698096ddefe25ac","type":"template","z":"28c631fa.7e866e","g":"7f16dc04ae3040ee","name":"","field":"payload","fieldType":"msg","format":"handlebars","syntax":"mustache","template":"Host {{payload.server.host}} Time {{time}} speeds: Down {{payload.download.bandwidth}} Up {{payload.upload.bandwidth}}","output":"str","x":4030,"y":1880,"wires":[["b2a22de9f60c0993"]]},{"id":"b2a22de9f60c0993","type":"junction","z":"28c631fa.7e866e","g":"7f16dc04ae3040ee","x":4120,"y":1880,"wires":[["e581d46f103cd330","bf23f55d279e5a40"]]},{"id":"e581d46f103cd330","type":"debug","z":"28c631fa.7e866e","g":"7f16dc04ae3040ee","name":"Speedtest","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":4210,"y":1840,"wires":[]}]

I installed that node and tried your flow but for some reason I could not get the template to output JSON.

Instead I used a function node:

msg.payload = {
    download: msg.payload.download.bandwidth,
    upload: msg.payload.upload.bandwidth,
    host: msg.payload.server.host
    }
return msg;

which gave me

(I omitted the timestamp in my tests)

An alternative to consider

An alternative approach that I have used on a Pi is to install speedtest-cli using apt. I think it accesses the same speedtest systems as your node. Then this flow

image

[{"id":"a20640c3c271f237","type":"json","z":"5441cb819ed3cbad","name":"","property":"payload","action":"","pretty":false,"x":450,"y":140,"wires":[["467a78ce8cf38955"]]},{"id":"dab97dde5c22a5d7","type":"exec","z":"5441cb819ed3cbad","command":"speedtest-cli --json","addpay":"","append":"","useSpawn":"false","timer":"","winHide":false,"oldrc":false,"name":"","x":290,"y":140,"wires":[["a20640c3c271f237"],[],[]]},{"id":"6d1a55f1ac3092c8","type":"inject","z":"5441cb819ed3cbad","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":120,"y":140,"wires":[["dab97dde5c22a5d7"]]},{"id":"467a78ce8cf38955","type":"debug","z":"5441cb819ed3cbad","name":"debug 2","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":580,"y":140,"wires":[]}]

Gives me this, It has the same fields you are interested in - download & upload speed, server and time (So no need to your moment node)

However

Both your flow and mine (on a Pi) report very different results from the speedtest website (on a PC). It makes me question the value of all the results.
1 Like

Tried using your flow on a RPi-4B and on my PC - get results that are close to each other.



Note: The server used by 'speedtest-cli' can vary each time you run the command.

Mine seems odd because the download shown by the flows is consistently much lower than the upload, and about 10% of the value shown by the Web page.

It's a Pi zero 2 with iffy wifi. Mobile broadband too.

Yes. You can specify it with a speedtest-cli command line flag though.

1 Like

My RPi-4B and PC are using 6e cables from the 1Gbs Ethernet sockets back to my main router.

1 Like

Well that can certainly effect the results then :wink:

Are you running the browser on the pi? Otherwise you are possibly limited by the comms between the pi and the router. Wifi versus wired for example?

Nonono, let's not hijack Andrew's thread.

I dont use either flow regularly and not seeking to diagnose them.

The browser is on the pc.
Both pi and browser are wireless, though the pi is on 2.4GHz and the pc 5GHz

The question is, what do you want to measure?

  1. The network speed of a local device in your local LAN
  2. Network speed of your router/modem/gateway to your ISP

Most people are interested in network speed against your ISP.

If you run a speed-test from a local device, be aware of the following limitations:

1 The Ethernet port capacity of your device

Raspberry Pi 2B(+) and 3B will give about 94.8 Mbps, the Pi 3+ will give 224 Mbps and the Pi 4B will give 933 Mbps.

Although a Raspberry Pi 3B+ is advertised with a network speed of 1Gbps, it can never be more than 300Mbps as the network port is shared with the USB.

  1. The network cabling in your local network. It should be at least Cat 5e, Cat 6 or better.
  2. Limitation of the port speed of your switches between your device and the Router/Modem.
  3. Quality of the RJ-45 connections. Did you make your own connectors or has the cable been factory made.
  4. Be aware that the repository of Rasbian (Debian) Buster contains a version of speedtest-cli that contains a bug, which results in lower test results.

I recommend to run the original speedtest-cli on your Router/Modem?Gateway (if possible)
You can find the OOKLA speedtest at: Speedtest CLI: Internet speed test for the command line
This is also the backend, used at the NR Contrib node (speedtest)

I use the command line tool on my router and push the values to NR by MQTT on a Raspberry Pi.
From there you can push the data to a Dashboard, another Home Automation System or Google Smart home, etc.

The way I integrated it in Domoticz, including a discussion, you can find at: https://forum.domoticz.com/viewtopic.php?t=31898

I created that solution about 5 years ago and so far I have no issues.

The NR flow, probably needs some maintenance (as I used var).

Does this, what you need?

Regards

1 Like

Another thing to note - the raw network speed is only part of the data you need.

Network Jitter is also important and I don't believe Speedtest tells you that? Latency is also important of course. And if you have ANY packet loss, you have an issue.

https://freeola.com/line-test/

Many advantages of being on a BT Full Fibre connection is both low latency and low jitter. :smiley:

All measured from my desktop which has a 1Gb connection to my 1Gb switch with 100Mb to the router which runs direct to the fibre outlet. All with 1Gb capable wiring. I pay for a 100Mb fibre and consistently get 50% more than that.

You should also check your DNS responses

I've not actually tested my home server (laptop) but wouldn't expect anywhere near the same as my desktop since it is only a laptop and has older CAT 5a wiring. But then I don't need my server to have masses of bandwidth. I do also have a NAS with a dual 100Mb network.


I should also have said that I use Telegraf to record network and other performance metrics and so can get charting and historic views on Grafana.

Hello @TotallyInformation,

I agree with you, that not only Download speed and Upload speed is important but also jitter, latency and of course packet loss.

The OOKLA command line tool gives you that all.

As I have a 1 Gbps fiber optics connection (symmetrical), I currently have a download speed 924.49 Mbps and an upload speed of 933.89 Mbps.

Regards

Ah, cool. I didn't quite get time to test all of that today. Then I decided I had too many other things going on anyway and I no longer have any real need to measure such things - at least for now. :smiley:

Very nice.

Ok, first off: Greatest apologies to all.

The reason:

I am on a (there abouts) 50/25 plan.
50 (meg) upload and 25 (meg) download.

Alas I have sort of drifted to the dark side and have a Windows PC and play games on it.
Alas (number 2) seems these days the games want to talk to Mummy and Daddy while playing.
Which shouldn't be too bad - but as anyone here from Oz will know: that can be a big ask.

Recently I'm happily playing and it all falls apart because the game can't talk to the required places.
Adding insult to injury a couple of times I wanted to upload a simple (well ok, not quite) email AND a PICTURE!
5mb size.

It takes 10 minutes.

WHY?

Upload speed:
500kb/s
Note the K in that. Not M

And it stays like that for a while.

It happens at different times of the day and is annoying.

I've complained to my Provider and from past experience, I know to not say something without backup data. (A whole other story)

I know that there is no fancy display in this scenario. But all I want is data.
Yes, it is interesting that what I see if I go to speedtest.com and what this node gives me are different.

Much? Not exactly sure. But I think it will/should be enough to show the trend.

Quick example:
Pictures:

Alas this one doesn't give jitter.
They were also taken at slightly different times.

and 24 and 18 are quite different.

Problems as of now:
I can't get the one Telstra uses.
It doesn't give jitter.

The two tests are done of DIFFERENT machines. :frowning:
Yes, I know that is bad.
But they are BOTH ethernet connected. Including/introducing WiFi is adding an unwanted variable.

One (the NR one) is on a RasPi 3b+ and the fancy one is on a NUC.

I put it on the RasPi because that machine is on 24/7 and I put the GUI so I can use my tablet (or on the games PC) can access the GUI and force a log of what's happening. (Information gathering)

If - and I really hope it does - it shows a huge disparity with normal I can use this to show them the problem and not have them just sweep it under the mat.

@dynamicdave
So I would like to use the same server every time and not change it as that is introducing a variable/change which could be used to invalidate my data.
As said, I don't know which one it being used via speedtest.com, but I hope that isn't a deal breaker when I get the problem.

Sod's law will step in how (of course) and it won't happen to me now for a long time. :wink:

But better I am ready than not.

And in case anyone is wondering:
The left most button is the go button and the Y is a toggle button to log the results.
(OCD sucks) :wink:

On the side:
It is interesting how this got so many replies so quickly.
Is it an underlying curiosity for others too?

Thanks to all who replied though.
As much as I kind of did get it working, I felt it better to ask first again to avert Sod's law happening to me. :slight_smile:

Oh, P.S.
The network is all 1Gig and so I am hoping that the speeds are basically/effectively from the Modem upstream.

Interesting update - and probably way off topic.
(But I may as well share) :slight_smile:
These two machines are on different VLANS.

Just now I see a HUGE difference in upload speeds between them.
One is good.... 25 MB the other is 0.4MB.

I do NOT have NR on the other VLAN. :frowning:

How could I test speeds from it to my last point to better determine where the problem is?
P.S. It is windows 10 and I am not that well in tune with its workings.

I may dig up a RASPI and plug it on that network and do some testing.

But am looking for options.

IMPORTANT UPDATE

(This is from left field)

The Windows machine has 2 ethernet ports.
That wasn't a deal breaker for me when I bought it, but it did come in handy.
I also have ANOTHER windows machine next to this/that one.
It too needs internet (well, not really, but for the sake of simplifying things.....
So I BRIDGED the first ethernet port so the second one can see the network through the first machine's port.

Said machine has GRUNT.

After a bit of reading - lucky me - I see that:

If you bridge an ethernet port, it's speed will be **SEVERELY IMPACTED**

So that may be biting my butt.

I remember it took a LONG TIME to get it like that and I am not keen on breaking that - just now - to prove/test the new theory.

I remember windows (XP days) had restore point options, but that was a long time ago and how to do it in W10 is - at this point - beyond me.

So I shall declare this new information now only to be honest on what is happening this end.

Second update:

Ok, BRIDGING the adaptor can cause problems.

But if I take THIS machine and put it onto that network's socket I am still getting vastly different speeds.

Crown Jewels - main (This) network.
Windows - the suspected slower network.

That's a factor of TEN difference.

So it indicates to me there is more going on here.

I should think Telstra would refuse to consider any evidence unless the computer generating it is directly plugged in to their router.

For me that would mean a dedicated Pi 3 running speedtest-cli hourly alternating between a couple of specified servers and reporting back to a database on my main server Pi.

Well, indulging the question:

A while back now I had a problem where my internet was rather frequently changing it's IP address.

Granted I am not on a FIXED IP scheme. But none the less, it was just changing.
Sometimes like 5 times a day, or more.

This caused problems now and then and initially I didn't suspect anything but when it got beyond a joke I dug down.

I complained and they rather dismissively replied, and so I sent them THREE YEARS of logging my IP address every 20 seconds - only changes shown.

After a week I got a new modem (Router) and it continued. So they sent out someone to check my line. Signal good. But they (the person) swapped my NTU. PROBLEM FIXED.

I'm proud of that because their (Telstra's) attitude was I didn't know what I was talking about.
But when I showed them, they actually took notice.

Ok, it did take 3 weeks: start to end. Which isn't too bad in the scheme of things.

1 Like

Right, so

  1. What router are you using?
  2. Why not get a switch to handle the 2nd PC? Put both PC's into the switch and the switch into the router. That is a much better configuration.
  3. You need to test consistently over time. Do a speedtest to a fixed endpoint every hour - BUT NOT ON THE HOUR (I got caught out by that before). Do a ping test every 10 minutes to a known and trusted public endpoint. Testing must be from a decent device though. Unless you have a Pi 4, I wouldn't use a Pi, simply leave one of your PC's (connected via switch to the router) running for a week.

I have Telegraf running on my home server (i5, 8GB RAM, Thinkpad laptop, connected direct to my switch). It can capture all sorts of metrics and record them direct to InfluxDB. There is even a Grafana Speedtest dashboard that uses a Speedtest Telegraf extension.

Using Telegraf, I also regularly ping some common endpoints such as YouTube and the BBC because, sometimes in the past, I thought my broadband was bad but it was actually something else.

Here is an interesting example from my own stats:

As you can see, there was some kind of a blip between 4 and 8 this morning. However, it is possible that might have been due to my NAS running some backups.

And the 15d averages:

Again, you can see the regular dips which I believe are due to some big, regular backup batch jobs.

Wow, that looks nice.

To answer your question/s.

My network is a bit complicated. :confused:

I'll reply to you to not share too much stuff.