-
Notifications
You must be signed in to change notification settings - Fork 286
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update system-tests to 06cbfad5f9facbc6e258d9c8208c466cd46efeea and use Python 3.12 #7297
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
smola
added
comp: testing
Testing
tag: no release notes
Changes to exclude from release notes
labels
Jul 9, 2024
PerfectSlayer
approved these changes
Jul 9, 2024
PerfectSlayer
approved these changes
Jul 9, 2024
BenchmarksStartupParameters
See matching parameters
SummaryFound 1 performance improvements and 0 performance regressions! Performance is the same for 51 metrics, 11 unstable metrics.
Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.38.0-SNAPSHOT~ab7632ac0d, baseline=1.38.0-SNAPSHOT~0e5e2749b0
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.063 s) : 0, 1063364
Total [baseline] (8.512 s) : 0, 8512199
Agent [candidate] (1.077 s) : 0, 1076680
Total [candidate] (8.595 s) : 0, 8594515
section iast
Agent [baseline] (1.178 s) : 0, 1178186
Total [baseline] (8.967 s) : 0, 8967013
Agent [candidate] (1.173 s) : 0, 1172847
Total [candidate] (8.975 s) : 0, 8974934
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.179 s) : 0, 1178981
Total [baseline] (8.961 s) : 0, 8961458
Agent [candidate] (1.175 s) : 0, 1175207
Total [candidate] (8.963 s) : 0, 8962563
section iast_TELEMETRY_OFF
Agent [baseline] (1.169 s) : 0, 1168516
Total [baseline] (8.961 s) : 0, 8960883
Agent [candidate] (1.168 s) : 0, 1168009
Total [candidate] (8.968 s) : 0, 8968309
gantt
title insecure-bank - break down per module: candidate=1.38.0-SNAPSHOT~ab7632ac0d, baseline=1.38.0-SNAPSHOT~0e5e2749b0
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (665.522 ms) : 0, 665522
BytebuddyAgent [candidate] (673.967 ms) : 0, 673967
GlobalTracer [baseline] (305.234 ms) : 0, 305234
GlobalTracer [candidate] (308.402 ms) : 0, 308402
AppSec [baseline] (49.827 ms) : 0, 49827
AppSec [candidate] (50.919 ms) : 0, 50919
Remote Config [baseline] (678.574 µs) : 0, 679
Remote Config [candidate] (690.741 µs) : 0, 691
Telemetry [baseline] (7.618 ms) : 0, 7618
Telemetry [candidate] (7.731 ms) : 0, 7731
section iast
BytebuddyAgent [baseline] (786.476 ms) : 0, 786476
BytebuddyAgent [candidate] (780.485 ms) : 0, 780485
GlobalTracer [baseline] (296.177 ms) : 0, 296177
GlobalTracer [candidate] (296.164 ms) : 0, 296164
AppSec [baseline] (48.932 ms) : 0, 48932
AppSec [candidate] (50.501 ms) : 0, 50501
Remote Config [baseline] (569.653 µs) : 0, 570
Remote Config [candidate] (566.622 µs) : 0, 567
Telemetry [baseline] (6.991 ms) : 0, 6991
Telemetry [candidate] (6.941 ms) : 0, 6941
IAST [baseline] (25.459 ms) : 0, 25459
IAST [candidate] (24.708 ms) : 0, 24708
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (786.191 ms) : 0, 786191
BytebuddyAgent [candidate] (782.056 ms) : 0, 782056
GlobalTracer [baseline] (296.859 ms) : 0, 296859
GlobalTracer [candidate] (295.899 ms) : 0, 295899
AppSec [baseline] (49.208 ms) : 0, 49208
AppSec [candidate] (48.162 ms) : 0, 48162
Remote Config [baseline] (567.183 µs) : 0, 567
Remote Config [candidate] (565.189 µs) : 0, 565
Telemetry [baseline] (7.016 ms) : 0, 7016
Telemetry [candidate] (7.776 ms) : 0, 7776
IAST [baseline] (25.487 ms) : 0, 25487
IAST [candidate] (27.173 ms) : 0, 27173
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (778.289 ms) : 0, 778289
BytebuddyAgent [candidate] (777.352 ms) : 0, 777352
GlobalTracer [baseline] (295.246 ms) : 0, 295246
GlobalTracer [candidate] (294.919 ms) : 0, 294919
AppSec [baseline] (47.281 ms) : 0, 47281
AppSec [candidate] (47.382 ms) : 0, 47382
Remote Config [baseline] (574.736 µs) : 0, 575
Remote Config [candidate] (594.673 µs) : 0, 595
Telemetry [baseline] (6.824 ms) : 0, 6824
Telemetry [candidate] (6.781 ms) : 0, 6781
IAST [baseline] (26.758 ms) : 0, 26758
IAST [candidate] (27.418 ms) : 0, 27418
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.38.0-SNAPSHOT~ab7632ac0d, baseline=1.38.0-SNAPSHOT~0e5e2749b0
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.073 s) : 0, 1073108
Total [baseline] (10.401 s) : 0, 10400635
Agent [candidate] (1.064 s) : 0, 1064104
Total [candidate] (10.347 s) : 0, 10347239
section appsec
Agent [baseline] (1.185 s) : 0, 1184587
Total [baseline] (10.491 s) : 0, 10490964
Agent [candidate] (1.189 s) : 0, 1189021
Total [candidate] (10.538 s) : 0, 10537806
section iast
Agent [baseline] (1.173 s) : 0, 1173408
Total [baseline] (10.676 s) : 0, 10676182
Agent [candidate] (1.169 s) : 0, 1168735
Total [candidate] (10.792 s) : 0, 10792160
section profiling
Agent [baseline] (1.266 s) : 0, 1266053
Total [baseline] (10.597 s) : 0, 10597431
Agent [candidate] (1.267 s) : 0, 1266578
Total [candidate] (10.58 s) : 0, 10580183
gantt
title petclinic - break down per module: candidate=1.38.0-SNAPSHOT~ab7632ac0d, baseline=1.38.0-SNAPSHOT~0e5e2749b0
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (671.377 ms) : 0, 671377
BytebuddyAgent [candidate] (665.768 ms) : 0, 665768
GlobalTracer [baseline] (308.258 ms) : 0, 308258
GlobalTracer [candidate] (305.379 ms) : 0, 305379
AppSec [baseline] (50.172 ms) : 0, 50172
AppSec [candidate] (50.086 ms) : 0, 50086
Remote Config [baseline] (681.477 µs) : 0, 681
Remote Config [candidate] (667.761 µs) : 0, 668
Telemetry [baseline] (7.677 ms) : 0, 7677
Telemetry [candidate] (7.552 ms) : 0, 7552
section appsec
BytebuddyAgent [baseline] (676.92 ms) : 0, 676920
BytebuddyAgent [candidate] (678.504 ms) : 0, 678504
GlobalTracer [baseline] (299.575 ms) : 0, 299575
GlobalTracer [candidate] (300.61 ms) : 0, 300610
AppSec [baseline] (153.665 ms) : 0, 153665
AppSec [candidate] (154.091 ms) : 0, 154091
Remote Config [baseline] (623.646 µs) : 0, 624
Remote Config [candidate] (623.701 µs) : 0, 624
Telemetry [baseline] (8.908 ms) : 0, 8908
Telemetry [candidate] (8.894 ms) : 0, 8894
IAST [baseline] (20.328 ms) : 0, 20328
IAST [candidate] (22.982 ms) : 0, 22982
section iast
BytebuddyAgent [baseline] (780.852 ms) : 0, 780852
BytebuddyAgent [candidate] (779.662 ms) : 0, 779662
GlobalTracer [baseline] (295.556 ms) : 0, 295556
GlobalTracer [candidate] (295.743 ms) : 0, 295743
AppSec [baseline] (51.077 ms) : 0, 51077
AppSec [candidate] (47.251 ms) : 0, 47251
Remote Config [baseline] (577.672 µs) : 0, 578
Remote Config [candidate] (576.372 µs) : 0, 576
Telemetry [baseline] (7.76 ms) : 0, 7760
Telemetry [candidate] (7.049 ms) : 0, 7049
IAST [baseline] (24.031 ms) : 0, 24031
IAST [candidate] (24.911 ms) : 0, 24911
section profiling
BytebuddyAgent [baseline] (662.936 ms) : 0, 662936
BytebuddyAgent [candidate] (663.861 ms) : 0, 663861
GlobalTracer [baseline] (390.086 ms) : 0, 390086
GlobalTracer [candidate] (389.129 ms) : 0, 389129
AppSec [baseline] (51.676 ms) : 0, 51676
AppSec [candidate] (51.756 ms) : 0, 51756
Remote Config [baseline] (664.828 µs) : 0, 665
Remote Config [candidate] (656.969 µs) : 0, 657
Telemetry [baseline] (7.336 ms) : 0, 7336
Telemetry [candidate] (7.395 ms) : 0, 7395
ProfilingAgent [baseline] (96.088 ms) : 0, 96088
ProfilingAgent [candidate] (96.509 ms) : 0, 96509
Profiling [baseline] (96.113 ms) : 0, 96113
Profiling [candidate] (96.534 ms) : 0, 96534
LoadParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 17 unstable metrics. Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.38.0-SNAPSHOT~ab7632ac0d, baseline=1.38.0-SNAPSHOT~0e5e2749b0
dateFormat X
axisFormat %s
section baseline
no_agent (369.438 µs) : 349, 390
. : milestone, 369,
iast (478.47 µs) : 458, 499
. : milestone, 478,
iast_FULL (549.799 µs) : 529, 571
. : milestone, 550,
iast_GLOBAL (510.095 µs) : 488, 532
. : milestone, 510,
iast_HARDCODED_SECRET_DISABLED (481.659 µs) : 461, 503
. : milestone, 482,
iast_INACTIVE (455.475 µs) : 434, 477
. : milestone, 455,
iast_TELEMETRY_OFF (473.531 µs) : 452, 495
. : milestone, 474,
tracing (450.732 µs) : 430, 471
. : milestone, 451,
section candidate
no_agent (370.564 µs) : 351, 390
. : milestone, 371,
iast (481.613 µs) : 460, 503
. : milestone, 482,
iast_FULL (559.854 µs) : 539, 581
. : milestone, 560,
iast_GLOBAL (510.229 µs) : 488, 532
. : milestone, 510,
iast_HARDCODED_SECRET_DISABLED (487.854 µs) : 467, 509
. : milestone, 488,
iast_INACTIVE (461.576 µs) : 440, 483
. : milestone, 462,
iast_TELEMETRY_OFF (473.245 µs) : 452, 494
. : milestone, 473,
tracing (447.484 µs) : 426, 469
. : milestone, 447,
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.38.0-SNAPSHOT~ab7632ac0d, baseline=1.38.0-SNAPSHOT~0e5e2749b0
dateFormat X
axisFormat %s
section baseline
no_agent (1.341 ms) : 1322, 1361
. : milestone, 1341,
appsec (1.715 ms) : 1691, 1738
. : milestone, 1715,
appsec_no_iast (1.704 ms) : 1679, 1729
. : milestone, 1704,
iast (1.482 ms) : 1459, 1504
. : milestone, 1482,
profiling (1.503 ms) : 1477, 1530
. : milestone, 1503,
tracing (1.469 ms) : 1444, 1493
. : milestone, 1469,
section candidate
no_agent (1.339 ms) : 1320, 1358
. : milestone, 1339,
appsec (1.708 ms) : 1684, 1732
. : milestone, 1708,
appsec_no_iast (1.729 ms) : 1705, 1753
. : milestone, 1729,
iast (1.478 ms) : 1455, 1501
. : milestone, 1478,
profiling (1.543 ms) : 1517, 1569
. : milestone, 1543,
tracing (1.474 ms) : 1451, 1497
. : milestone, 1474,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics. Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.38.0-SNAPSHOT~ab7632ac0d, baseline=1.38.0-SNAPSHOT~0e5e2749b0
dateFormat X
axisFormat %s
section baseline
no_agent (1.45 ms) : 1439, 1462
. : milestone, 1450,
appsec (2.213 ms) : 2178, 2248
. : milestone, 2213,
iast (1.952 ms) : 1911, 1994
. : milestone, 1952,
iast_GLOBAL (2.002 ms) : 1959, 2045
. : milestone, 2002,
profiling (2.333 ms) : 2145, 2521
. : milestone, 2333,
tracing (1.829 ms) : 1796, 1862
. : milestone, 1829,
section candidate
no_agent (1.455 ms) : 1444, 1466
. : milestone, 1455,
appsec (2.199 ms) : 2165, 2234
. : milestone, 2199,
iast (1.966 ms) : 1925, 2008
. : milestone, 1966,
iast_GLOBAL (2.005 ms) : 1962, 2047
. : milestone, 2005,
profiling (1.854 ms) : 1818, 1889
. : milestone, 1854,
tracing (1.829 ms) : 1796, 1862
. : milestone, 1829,
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.38.0-SNAPSHOT~ab7632ac0d, baseline=1.38.0-SNAPSHOT~0e5e2749b0
dateFormat X
axisFormat %s
section baseline
no_agent (15.01 s) : 15010000, 15010000
. : milestone, 15010000,
appsec (15.005 s) : 15005000, 15005000
. : milestone, 15005000,
iast (18.67 s) : 18670000, 18670000
. : milestone, 18670000,
iast_GLOBAL (18.01 s) : 18010000, 18010000
. : milestone, 18010000,
profiling (15.352 s) : 15352000, 15352000
. : milestone, 15352000,
tracing (15.095 s) : 15095000, 15095000
. : milestone, 15095000,
section candidate
no_agent (15.067 s) : 15067000, 15067000
. : milestone, 15067000,
appsec (15.173 s) : 15173000, 15173000
. : milestone, 15173000,
iast (18.662 s) : 18662000, 18662000
. : milestone, 18662000,
iast_GLOBAL (17.835 s) : 17835000, 17835000
. : milestone, 17835000,
profiling (15.944 s) : 15944000, 15944000
. : milestone, 15944000,
tracing (14.871 s) : 14871000, 14871000
. : milestone, 14871000,
|
mcculls
approved these changes
Jul 9, 2024
smola
force-pushed
the
smola/update-system-tests-20240709
branch
2 times, most recently
from
July 11, 2024 12:14
450de52
to
ed43051
Compare
smola
changed the title
Update system-tests to 2640b90ad5641717b6012a840646e85bad3a24fc and use Python 3.12
Update system-tests to 5330ad86a59bd0f055d88287f5e64af994e22908 and use Python 3.12
Jul 11, 2024
smola
force-pushed
the
smola/update-system-tests-20240709
branch
from
July 12, 2024 14:46
6bb9f74
to
ab7632a
Compare
smola
changed the title
Update system-tests to 5330ad86a59bd0f055d88287f5e64af994e22908 and use Python 3.12
Update system-tests to 06cbfad5f9facbc6e258d9c8208c466cd46efeea and use Python 3.12
Jul 12, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What Does This Do
Motivation
Additional Notes
Based on DataDog/dd-trace-php#2748