Internet performance monitoring (new approach)

General
1

After the discussions about network speed, etc (see: Help with how to get "speed test" results saved ), I recently discovered something like https://orb.net/.

The first initial release was April 30th 2025 and since then a lot of publications has appeared on the internet.

I wonder, what your opinion is about this new approach?

Home Assistant (I do not use HA) has already an integration (beta)
Will it replace the traditional way?

Regards

2

Would need to look more closely at how it measures things. One measure often overlooked but important is jitter. Another possible issue to look for with Orb though would be how much bandwidth does it use. And finally, they potentially collect a LOT of detailed info from you, you better be sure that you trust them. :slight_smile:

Going to give it a go on my phone for a bit to see if it is any use there.

3

Most speed test sites are QOS’d by your ISP so if you use one and want it to be accurate use one that isn’t mainstream

4

@leftymuller,

Most speed test sites are QOS’d by your ISP so if you use one and want it to be accurate use one that isn’t mainstream

Agree, but this was not the issue. My question was, what do you think about this approach of former developers of OOKLA speed-test.

As a test I installed a Orb sensor on a Raspberry Pi 3B. This Pi runs Raspberry Pi OS Bookworm. Installation was fast and without any issues. After registering an account, you can log in to https://cloud.orb.net/status

I also installed the app from the play store om my Android phone.
It turns out that the app also contains a sensor as, the phone showed up in the Dashboud after installation.

Below a screenshot for the orb cloud website.

Screenshot_orb_cloud
Screenshot_orb_cloud1913Ă—354 37.1 KB

In my opinion, it does not offer much, except an overview of the installed sensors.

The app on the phone gives you more information.
See the screenshots below:

Screenshot_Orb-app2
Screenshot_Orb-app21080Ă—2400 79.5 KB

Screenshot_Orb-app1
Screenshot_Orb-app11080Ă—2400 197 KB

This shows us Responsiveness and Reliability. But what does that mean.
Currently I do not have a feeling about it.

It shows also the Download and Upload speed of the device.
Not bad, for a Pi 3B, which has an 100Mb Ethernet port (wired)
I shows “Lag” as well, 2 ms to the router and 6 ms to Internet.
Obviously 2 + 6 = 8 and that results in a score of 92 (100 - 8)>

According to the Home Assistant website, the following is available in their integration.
See: GitHub - orbforge/addon-orb: Orb Sensor Home Assistant Addon

  • Bandwidth Score

  • Reliability Score

  • Responsiveness Score

  • Bandwidth Upload

  • Bandwidth Download

  • Lag

  • Packet Loss

However they seem to be able to communicate with the Orb server, but I cannot find any API or MQTT server.
So I will not be able to integrate something in Node-Red or its Dashboard.

Regards

5

Where is it measuring to? Do they have endpoints all over the world? Where? And what performance.

This doesn't really feel like any kind of scientific measure at present. Too many unknowns.

1 Like
6

If there isn’t API you could just do an old fashioned web scrap… very uncivilized and unreliable but it should do what you want

If you want true speed test values IPERF is the only way to go.

7

Hello @leftymuller,

No, it is not that I’m looking for a way to measure the Internet speed.
I do already so and I know Iperf. I’m also aware of the different limitations of the various devices. So that is not the issue.

But I was curious, if someone has already discovered this new approach or has already used it.
Or what is your opinion of this method of measuring?

Regards

8

I think that the bottom line is that this doesn't look like anything actually new. And, we have no visibility of how they are actually measuring things and where the measurement endpoint is.

9

Hi @TotallyInformation

You said:

I think that the bottom line is that this doesn't look like anything actually new.

I didn’t give up and did some more research.

I installed a sensor on a Raspberry Pi 3B and it turned out that an application has been installed for the user “orb”. In its home directory you will find a number of hidden files and a hidden config directory.

An orb executable exist and with the following options, you will receive data:

Orb: Measure what matters (``orb.net``)

Usage:
orb [command]

Available Commands:
sensor Run the Orb sensor
listen Connect to a running Orb service
link Link this Orb to an account
version Current version of Orb
summary Show the latest summary for this Orb
help This screen

Flags:
-h, --help help for example
-r --remote connect to remote host

I also found the Home Assistant add-on source code and they do exactly what I thought.
They push orb summary to MQTT. They do it even every 5 seconds. I.m.o too much.
I have the orb sensor on the same Pi and an instance of Node RED.
So a simple exec node with the above command does the job.
To be a little more flexible, I created a simple bash script, that pushes the result to MQTT.

image
image314Ă—826 42.4 KB

Next step is to find, what the values really represent and what is their meaning.
In HA users are very enthusiastic, but I have to understand where I look at.
To be continued. :slightly_smiling_face:

Regards

10

All well-and-good, but this still doesn't touch on the fundamentals. We don't know what the orb service is connecting to, nor do we know where that endpoint (or points - we just don't know) physically sit. Nor do we know exactly what and how they are measuring things. The bandwidth for example, what does that "score" actually mean?

But for the paranoid version of me, the biggest reason to avoid this is that we don't know what we are connecting to, where it is, how it is controlled/protected, or what information about us is being retained.

Maybe their endpoint is in North Korea? China? Russia? The USA? None of those countries can be considered "safe" right now for most of us.

2 Likes
11

Hello Julian (@TotallyInformation )

You wrote:

You forgot Iran :grinning_face:

No, serious

In But for the paranoid version of me, the biggest reason to avoid this is that we don't know what we are connecting to, where it is, how it is controlled/protected, or what information about us is being retained.

I looked somewhat further and their way is quite well documented.

In the other post Help with how to get "speed test" results saved - #11 by TotallyInformation you wrote:
Another thing to note - the raw network speed is only part of the data you need.

Network Jitter is also important and I don't believe Speedtest tells you that? Latency is also important of course. And if you have ANY packet loss, you have an issue.

I found that they describe that in their documentation, as well.
See: Orb Detail View

I think, it works more or less the same as all the other applications, such as speedtest.net from Ookla. Not surprising, as the current CEO of Orb was the previous CEO of Ookla.
Even node-red-contrib-speedtest is based on speedtest.net.

But perhaps some articles are interesting to read.

or

https://www.howtogeek.com/ookla-founder-launches-orb-a-smarter-internet-quality-tracker/

Regards

12

I noticed that as well. You don't need an addon to publish to mqtt. It should only need access to your mqtt server. Installing the addon gives it direct access to HA for no reason.