internet_speed giving much lower download results (<200 Mbps vs 900 Mbps)

[cliffjao]

Relevant telegraf.conf

[[inputs.internet_speed]]
  interval = "8h"

Logs from Telegraf

NA

System info

Telegraf 1.23.0 (git: HEAD 806dc28)

Docker

No response

Steps to reproduce

Run internet_speed plugin according to config. Compare result to speedtest.net.

Expected behavior

Download speeds and upload speeds are similar to manually running speedtest.net

Actual behavior

Download speeds much slower (<200 Mbps vs 900 Mbps)
Upload speeds are comparable to speedtest.net (30 Mbps)

Additional info

No response

[powersj]

Hi,

Telegraf uses the showwin/speedtest-go library that interacts with the speedtest.net service. There are a wide variety of reasons that you could be getting different results:

• servers: using different servers or one that is somehow bandwidth-limited at the time of the test
• system bandwidth: your own system could be doing something to affect the test at the time or if one a wireless connection could have a reduced connection at the time of the test
• network contention: your internet speed could be used by some other device or limited at the time of the test

I have a similar network setup and have run this plugin in a variety of times and do see some fluctuation with the results depending on my own network usage, and the occasional drops due to a network or server blip. My suggestion is to play around with the speedtest library binary I linked above and see if you are getting similar results as well as running telegraf in debug mode to see what server you are using for your tests.

Without any additional details, it is not clear how this is a bug, particularly in Telegraf.

[cliffjao]

Yea, it looks like the speedtest-go library isn't very accurate for me. I will file a bug with that project. Feel free to close this one.

[cliffjao]

Actually, one thing that might help on Telegraf side is to implement the server ID switch in the config. It appears that some servers are more accurate for me than others.

[cliffjao]

And note that I get the same results with enable_file_download = true

[srebhan]

@cliffjao PRs are welcome! :-)

[powersj]

Hi,

Actually, one thing that might help on Telegraf side is to implement the server ID switch in the config. It appears that some servers are more accurate for me than others.

I do think that this is something that could be helpful. Do note that the code currently does try to return and use the closet server available to you, which should in theory, be the fastest. However, I can understand if this is not always the case due to load or some other issue in between you and that server.

It looks like the library does have a findServer function that could be used. Telegraf could add a server option that takes the ID of the server.

[srebhan]

As a remark, the package currently only queries 10 servers from the list which might prevent you from finding the server even though it exists...

[powersj]

@cliffjao,

Can you try one of the test artifacts on #12617 and use the server_id_exclude option to ignore certain servers?

Thanks!

[melroy89]

Sorry, I still have issues with the speedtest-go binary.

I now run the latest speedtest-go binary on my server and compare the results with the official Ookla speedtest binary. In both cases I'm using the same server ID (Amstelveen)!!

First the speedtest binary by Ookla (these results are correct):

speedtest -s 61186

   Speedtest by Ookla

      Server: KPN B.V. - Amstelveen (id: 61186)
         ISP: KPN
Idle Latency:     4.33 ms   (jitter: 0.42ms, low: 3.86ms, high: 4.87ms)
    Download:  2328.09 Mbps (data used: 1.3 GB)                                                   
                  7.50 ms   (jitter: 1.06ms, low: 4.34ms, high: 13.19ms)
      Upload:  2322.84 Mbps (data used: 2.9 GB)                                                   
                  6.69 ms   (jitter: 0.91ms, low: 3.78ms, high: 19.10ms)
 Packet Loss:     0.0%
  Result URL: https://www.speedtest.net/result/c/0a4adc67-2e39-407a-98fc-c6f60bc766f3

Now I executed directly after the test above test the following speedtest-go command, using the same server from Amstelveen... And I get lower upload speed, this is INcorrect:

 ./speedtest-go -s 61186

    speedtest-go v1.7.8 (git-a052d7f) @showwin

✓ ISP: 77.61.56.117 (KPN) [52.3664, 4.8496] 
✓ Found 1 Specified Public Server(s)

✓ Test Server: [61186] 0.00km Amstelveen by KPN B.V.
✓ Latency: 4.998629ms Jitter: 317.485µs Min: 4.462946ms Max: 5.79003ms
✓ Packet Loss Analyzer: Running in background (<= 30 Secs)
✓ Download: 2346.21 Mbps (Used: 2968.90MB) (Latency: 15ms Jitter: 5ms Min: 6ms Max: 20ms)
✓ Upload: 875.56 Mbps (Used: 1168.78MB) (Latency: 9ms Jitter: 0ms Min: 8ms Max: 11ms)
✓ Packet Loss: N/A

[powersj]

I now run the latest speedtest-go binary on my server and compare the results with the official Ookla speedtest binary. In both cases I'm using the same server ID (Amstelveen)!!

As a next step, I would follow up with the speedtest-go project then and see if there is an option you need to set to replicate the same result.

There are a lot of factors and variables that could be at play here, including how reliable both results are, how the tests are actually run, and even as far as the implementation at those specific speeds.

[melroy89]

I now run the latest speedtest-go binary on my server and compare the results with the official Ookla speedtest binary. In both cases I'm using the same server ID (Amstelveen)!!

As a next step, I would follow up with the speedtest-go project then and see if there is an option you need to set to replicate the same result.

There are a lot of factors and variables that could be at play here, including how reliable both results are, how the tests are actually run, and even as far as the implementation at those specific speeds.

OK. done: showwin/speedtest-go#226

Let's see what the outcome will be..

[melroy89]

@powersj Small update: We found the issue. It seems that the HTTP test is bottleneck somehow. However speedtest.net in the browser and the speedtest binary of Ookla are all using TCP websockets for their tests, that is why we see a upload difference. After running the new prototype binary of r3inbowari which is also using TCP for the speedtest, I get the correct upload speed.

Now, r3inbowari is busy implementing TCP in the speedtest-go binary next to the existing HTTP test. Hopefully once this will be released, your Telegraf plugin might need to be extended to allow this configuration to select either TCP or HTTP for the speedtest.

See: showwin/speedtest-go#226 (comment)

I will keep you posted once this version is out!

[powersj]

it seems that the HTTP test is bottleneck somehow.

[powersj]

Ugh sorry, hit the button before writing this - re-opening to track this.

Thanks for follow-up! Happy to update the dependency of sppedtest-go, and we could certainly add an option for how the tests are run. We would want it to continue to default to HTTP but allow a TCP option.

Thanks again!

[melroy89]

@powersj Small update: We found the issue. It seems that the HTTP test is bottleneck somehow. However speedtest.net in the browser and the speedtest binary of Ookla are all using TCP websockets for their tests, that is why we see a upload difference. After running the new prototype binary of r3inbowari which is also using TCP for the speedtest, I get the correct upload speed.

Now, r3inbowari is busy implementing TCP in the speedtest-go binary next to the existing HTTP test. Hopefully once this will be released, your Telegraf plugin might need to be extended to allow this configuration to select either TCP or HTTP for the speedtest.

See: showwin/speedtest-go#226 (comment)

I will keep you posted once this version is out!

At the end it wasn't a HTTP bottlebeck after all. So the TCP implementation is coming later.

The root-cause was a bug in the code that didn't close the upload requests (broken keep-alive). The latest version now (see PR) solves this issue, closing the connection and allow port re-use, resulting in the correct upload speeds.

[r3inbowari]

@melroy89 thank you for your patient feedback and active tracking.

[melroy89]

@melroy89 thank you for your patient feedback and active tracking.

Thank you for debugging the problem and fixing the upload speed issue!