Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: unpin nss #2110

Closed
wants to merge 1 commit into from
Closed

ci: unpin nss #2110

wants to merge 1 commit into from

Conversation

jschanck
Copy link
Contributor

Copy link

Failed Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

neqo-latest as server

All results

Succeeded Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

neqo-latest as server

Unsupported Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

neqo-latest as server

Copy link

codecov bot commented Sep 13, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.35%. Comparing base (d4978de) to head (769f5a9).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2110   +/-   ##
=======================================
  Coverage   95.35%   95.35%           
=======================================
  Files         112      112           
  Lines       36325    36325           
=======================================
  Hits        34637    34637           
  Misses       1688     1688           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link

Benchmark results

Performance differences relative to d4978de.

coalesce_acked_from_zero 1+1 entries: Change within noise threshold.
       time:   [99.693 ns 100.01 ns 100.36 ns]
       change: [+0.0798% +0.5175% +1.0014%] (p = 0.03 < 0.05)

Found 10 outliers among 100 measurements (10.00%)
4 (4.00%) high mild
6 (6.00%) high severe

coalesce_acked_from_zero 3+1 entries: No change in performance detected.
       time:   [117.95 ns 118.29 ns 118.65 ns]
       change: [-0.1276% +0.4604% +0.9468%] (p = 0.09 > 0.05)

Found 17 outliers among 100 measurements (17.00%)
2 (2.00%) low severe
2 (2.00%) low mild
4 (4.00%) high mild
9 (9.00%) high severe

coalesce_acked_from_zero 10+1 entries: No change in performance detected.
       time:   [117.53 ns 117.87 ns 118.32 ns]
       change: [-0.2316% +0.4650% +1.0812%] (p = 0.17 > 0.05)

Found 18 outliers among 100 measurements (18.00%)
4 (4.00%) low severe
3 (3.00%) low mild
3 (3.00%) high mild
8 (8.00%) high severe

coalesce_acked_from_zero 1000+1 entries: Change within noise threshold.
       time:   [98.579 ns 98.732 ns 98.902 ns]
       change: [+0.6521% +1.5380% +2.4842%] (p = 0.00 < 0.05)

Found 8 outliers among 100 measurements (8.00%)
2 (2.00%) high mild
6 (6.00%) high severe

RxStreamOrderer::inbound_frame(): Change within noise threshold.
       time:   [111.45 ms 111.51 ms 111.57 ms]
       change: [-0.5776% -0.3764% -0.2408%] (p = 0.00 < 0.05)

Found 11 outliers among 100 measurements (11.00%)
9 (9.00%) low mild
2 (2.00%) high mild

transfer/pacing-false/varying-seeds: Change within noise threshold.
       time:   [26.283 ms 27.205 ms 28.126 ms]
       change: [-9.8545% -5.0715% -0.0275%] (p = 0.05 < 0.05)
transfer/pacing-true/varying-seeds: No change in performance detected.
       time:   [34.768 ms 36.362 ms 37.953 ms]
       change: [-9.6389% -4.1373% +1.7435%] (p = 0.17 > 0.05)

Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) low mild

transfer/pacing-false/same-seed: Change within noise threshold.
       time:   [31.243 ms 32.045 ms 32.826 ms]
       change: [-7.6161% -4.2222% -1.1332%] (p = 0.01 < 0.05)

Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) low mild

transfer/pacing-true/same-seed: No change in performance detected.
       time:   [37.916 ms 40.484 ms 43.047 ms]
       change: [-7.7034% +2.5078% +13.956%] (p = 0.65 > 0.05)
1-conn/1-100mb-resp (aka. Download)/client: No change in performance detected.
       time:   [112.36 ms 112.86 ms 113.34 ms]
       thrpt:  [882.32 MiB/s 886.03 MiB/s 889.97 MiB/s]
change:
       time:   [-0.7714% -0.2327% +0.2893%] (p = 0.42 > 0.05)
       thrpt:  [-0.2885% +0.2332% +0.7774%]

Found 5 outliers among 100 measurements (5.00%)
5 (5.00%) low mild

1-conn/10_000-parallel-1b-resp (aka. RPS)/client: No change in performance detected.
       time:   [309.94 ms 313.56 ms 317.14 ms]
       thrpt:  [31.532 Kelem/s 31.892 Kelem/s 32.264 Kelem/s]
change:
       time:   [-1.7523% -0.2988% +1.2888%] (p = 0.72 > 0.05)
       thrpt:  [-1.2724% +0.2997% +1.7836%]
1-conn/1-1b-resp (aka. HPS)/client: Change within noise threshold.
       time:   [34.111 ms 34.325 ms 34.559 ms]
       thrpt:  [28.936  elem/s 29.134  elem/s 29.316  elem/s]
change:
       time:   [+0.1877% +0.9730% +1.8019%] (p = 0.02 < 0.05)
       thrpt:  [-1.7700% -0.9636% -0.1874%]

Found 8 outliers among 100 measurements (8.00%)
1 (1.00%) low mild
7 (7.00%) high severe

Client/server transfer results

Transfer of 33554432 bytes over loopback.

Client Server CC Pacing Mean [ms] Min [ms] Max [ms] Relative
msquic msquic 217.9 ± 127.7 102.5 543.3 1.00
neqo msquic reno on 315.2 ± 98.0 220.7 464.0 1.00
neqo msquic reno 232.6 ± 56.6 202.5 408.9 1.00
neqo msquic cubic on 256.7 ± 70.3 213.0 425.7 1.00
neqo msquic cubic 253.3 ± 68.4 215.8 430.1 1.00
msquic neqo reno on 176.4 ± 98.7 85.9 357.2 1.00
msquic neqo reno 156.8 ± 85.2 83.1 356.9 1.00
msquic neqo cubic on 204.7 ± 130.0 93.9 647.1 1.00
msquic neqo cubic 175.0 ± 102.2 84.8 366.4 1.00
neqo neqo reno on 203.3 ± 91.4 136.1 432.5 1.00
neqo neqo reno 186.1 ± 77.3 124.6 392.1 1.00
neqo neqo cubic on 247.9 ± 128.3 125.1 532.4 1.00
neqo neqo cubic 223.6 ± 134.8 127.7 586.4 1.00

⬇️ Download logs

Copy link

Firefox builds for this PR

The following builds are available for testing. Crossed-out builds did not succeed.

@larseggert
Copy link
Collaborator

Thanks! I prefer to go with #2108, since this PR reverts some changes I'd like to keep (like using the GitHub action to check out the source.)

@larseggert larseggert closed this Sep 16, 2024
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants