Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Do not gather coverage for skippable tests #7139

Merged
merged 1 commit into from
Jun 7, 2024
Merged

Do not gather coverage for skippable tests #7139

merged 1 commit into from
Jun 7, 2024

Conversation

nikita-tkachenko-datadog
Copy link
Contributor

@nikita-tkachenko-datadog nikita-tkachenko-datadog commented Jun 7, 2024 •
edited
Loading

What Does This Do

Updates ITR per-test code coverage logic: if a test case is known to be skippable (i.e. is included in the list of skippable tests retrieved from the backend), then code coverage will not be gathered for that test case.

Motivation

Reducing coverage performance overhead.
There is no point in gathering coverage for skippable tests: if a test is skippable it means no covered files were changed. So if coverage is gathered, it will be the same as before.

Jira ticket: SDTEST-441

@nikita-tkachenko-datadog nikita-tkachenko-datadog added tag: performance Performance related changes comp: ci visibility Continuous Integration Visibility labels Jun 7, 2024
@pr-commenter
Copy link

pr-commenter bot commented Jun 7, 2024

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nikita-tkachenko/itr-no-coverage-for-skippable-tests
git_commit_date 1717749941 1717755953
git_commit_sha edb2c3f b58dc72
release_version 1.36.0-SNAPSHOT~edb2c3fa17 1.36.0-SNAPSHOT~b58dc72569
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1717758810 1717758810
ci_job_id 535787117 535787117
ci_pipeline_id 36142247 36142247
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 53 metrics, 18 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.36.0-SNAPSHOT~b58dc72569, baseline=1.36.0-SNAPSHOT~edb2c3fa17

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.076 s) : 0, 1075847
Total [baseline] (10.404 s) : 0, 10404191
Agent [candidate] (1.064 s) : 0, 1063811
Total [candidate] (10.323 s) : 0, 10322905
section appsec
Agent [baseline] (1.185 s) : 0, 1185338
Total [baseline] (10.551 s) : 0, 10551353
Agent [candidate] (1.183 s) : 0, 1183080
Total [candidate] (10.563 s) : 0, 10563312
section iast
Agent [baseline] (1.169 s) : 0, 1168891
Total [baseline] (10.731 s) : 0, 10731301
Agent [candidate] (1.178 s) : 0, 1177580
Total [candidate] (10.676 s) : 0, 10676327
section profiling
Agent [baseline] (1.261 s) : 0, 1260523
Total [baseline] (10.744 s) : 0, 10743650
Agent [candidate] (1.267 s) : 0, 1267360
Total [candidate] (10.706 s) : 0, 10705828
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.076 s -
Agent appsec 1.185 s 109.491 ms (10.2%)
Agent iast 1.169 s 93.044 ms (8.6%)
Agent profiling 1.261 s 184.676 ms (17.2%)
Total tracing 10.404 s -
Total appsec 10.551 s 147.162 ms (1.4%)
Total iast 10.731 s 327.11 ms (3.1%)
Total profiling 10.744 s 339.459 ms (3.3%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.064 s -
Agent appsec 1.183 s 119.269 ms (11.2%)
Agent iast 1.178 s 113.768 ms (10.7%)
Agent profiling 1.267 s 203.548 ms (19.1%)
Total tracing 10.323 s -
Total appsec 10.563 s 240.406 ms (2.3%)
Total iast 10.676 s 353.422 ms (3.4%)
Total profiling 10.706 s 382.923 ms (3.7%) 
gantt
    title petclinic - break down per module: candidate=1.36.0-SNAPSHOT~b58dc72569, baseline=1.36.0-SNAPSHOT~edb2c3fa17

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (674.092 ms) : 0, 674092
BytebuddyAgent [candidate] (665.882 ms) : 0, 665882
GlobalTracer [baseline] (307.47 ms) : 0, 307470
GlobalTracer [candidate] (304.401 ms) : 0, 304401
AppSec [baseline] (50.623 ms) : 0, 50623
AppSec [candidate] (50.3 ms) : 0, 50300
Logs Intake [baseline] (391.993 µs) : 0, 392
Logs Intake [candidate] (390.615 µs) : 0, 391
Remote Config [baseline] (685.751 µs) : 0, 686
Remote Config [candidate] (698.04 µs) : 0, 698
Telemetry [baseline] (7.667 ms) : 0, 7667
Telemetry [candidate] (7.635 ms) : 0, 7635
section appsec
BytebuddyAgent [baseline] (678.596 ms) : 0, 678596
BytebuddyAgent [candidate] (678.051 ms) : 0, 678051
GlobalTracer [baseline] (298.965 ms) : 0, 298965
GlobalTracer [candidate] (298.202 ms) : 0, 298202
AppSec [baseline] (153.934 ms) : 0, 153934
AppSec [candidate] (153.398 ms) : 0, 153398
Logs Intake [baseline] (336.735 µs) : 0, 337
Logs Intake [candidate] (334.612 µs) : 0, 335
Remote Config [baseline] (653.405 µs) : 0, 653
Remote Config [candidate] (634.632 µs) : 0, 635
Telemetry [baseline] (7.278 ms) : 0, 7278
Telemetry [candidate] (7.562 ms) : 0, 7562
IAST [baseline] (22.027 ms) : 0, 22027
IAST [candidate] (20.744 ms) : 0, 20744
section iast
BytebuddyAgent [baseline] (779.096 ms) : 0, 779096
BytebuddyAgent [candidate] (784.88 ms) : 0, 784880
GlobalTracer [baseline] (293.556 ms) : 0, 293556
GlobalTracer [candidate] (295.438 ms) : 0, 295438
AppSec [baseline] (47.321 ms) : 0, 47321
AppSec [candidate] (47.575 ms) : 0, 47575
Logs Intake [baseline] (302.153 µs) : 0, 302
Logs Intake [candidate] (304.994 µs) : 0, 305
Remote Config [baseline] (566.296 µs) : 0, 566
Remote Config [candidate] (566.494 µs) : 0, 566
Telemetry [baseline] (8.389 ms) : 0, 8389
Telemetry [candidate] (7.632 ms) : 0, 7632
IAST [baseline] (26.422 ms) : 0, 26422
IAST [candidate] (27.826 ms) : 0, 27826
section profiling
BytebuddyAgent [baseline] (662.46 ms) : 0, 662460
BytebuddyAgent [candidate] (666.593 ms) : 0, 666593
GlobalTracer [baseline] (386.072 ms) : 0, 386072
GlobalTracer [candidate] (388.483 ms) : 0, 388483
AppSec [baseline] (50.823 ms) : 0, 50823
AppSec [candidate] (51.135 ms) : 0, 51135
Logs Intake [baseline] (329.188 µs) : 0, 329
Logs Intake [candidate] (331.202 µs) : 0, 331
Remote Config [baseline] (732.16 µs) : 0, 732
Remote Config [candidate] (754.39 µs) : 0, 754
Telemetry [baseline] (7.291 ms) : 0, 7291
Telemetry [candidate] (7.37 ms) : 0, 7370
ProfilingAgent [baseline] (96.225 ms) : 0, 96225
ProfilingAgent [candidate] (95.77 ms) : 0, 95770
Profiling [baseline] (96.25 ms) : 0, 96250
Profiling [candidate] (95.794 ms) : 0, 95794
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.36.0-SNAPSHOT~b58dc72569, baseline=1.36.0-SNAPSHOT~edb2c3fa17

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.06 s) : 0, 1060194
Total [baseline] (8.56 s) : 0, 8560242
Agent [candidate] (1.06 s) : 0, 1060039
Total [candidate] (8.542 s) : 0, 8541845
section iast
Agent [baseline] (1.168 s) : 0, 1167964
Total [baseline] (8.993 s) : 0, 8992878
Agent [candidate] (1.175 s) : 0, 1175482
Total [candidate] (9.03 s) : 0, 9029652
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.176 s) : 0, 1175866
Total [baseline] (9.012 s) : 0, 9012096
Agent [candidate] (1.171 s) : 0, 1170651
Total [candidate] (9.0 s) : 0, 8999965
section iast_TELEMETRY_OFF
Agent [baseline] (1.161 s) : 0, 1161370
Total [baseline] (9.03 s) : 0, 9029637
Agent [candidate] (1.173 s) : 0, 1172511
Total [candidate] (9.005 s) : 0, 9004831
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.06 s -
Agent iast 1.168 s 107.77 ms (10.2%)
Agent iast_HARDCODED_SECRET_DISABLED 1.176 s 115.672 ms (10.9%)
Agent iast_TELEMETRY_OFF 1.161 s 101.176 ms (9.5%)
Total tracing 8.56 s -
Total iast 8.993 s 432.636 ms (5.1%)
Total iast_HARDCODED_SECRET_DISABLED 9.012 s 451.853 ms (5.3%)
Total iast_TELEMETRY_OFF 9.03 s 469.395 ms (5.5%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.06 s -
Agent iast 1.175 s 115.443 ms (10.9%)
Agent iast_HARDCODED_SECRET_DISABLED 1.171 s 110.612 ms (10.4%)
Agent iast_TELEMETRY_OFF 1.173 s 112.472 ms (10.6%)
Total tracing 8.542 s -
Total iast 9.03 s 487.807 ms (5.7%)
Total iast_HARDCODED_SECRET_DISABLED 9.0 s 458.119 ms (5.4%)
Total iast_TELEMETRY_OFF 9.005 s 462.985 ms (5.4%) 
gantt
    title insecure-bank - break down per module: candidate=1.36.0-SNAPSHOT~b58dc72569, baseline=1.36.0-SNAPSHOT~edb2c3fa17

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (663.569 ms) : 0, 663569
BytebuddyAgent [candidate] (663.85 ms) : 0, 663850
GlobalTracer [baseline] (303.814 ms) : 0, 303814
GlobalTracer [candidate] (303.378 ms) : 0, 303378
AppSec [baseline] (50.023 ms) : 0, 50023
AppSec [candidate] (49.896 ms) : 0, 49896
Logs Intake [baseline] (404.583 µs) : 0, 405
Logs Intake [candidate] (405.54 µs) : 0, 406
Remote Config [baseline] (662.54 µs) : 0, 663
Remote Config [candidate] (666.142 µs) : 0, 666
Telemetry [baseline] (7.489 ms) : 0, 7489
Telemetry [candidate] (7.501 ms) : 0, 7501
section iast
BytebuddyAgent [baseline] (778.357 ms) : 0, 778357
BytebuddyAgent [candidate] (784.347 ms) : 0, 784347
GlobalTracer [baseline] (292.873 ms) : 0, 292873
GlobalTracer [candidate] (295.006 ms) : 0, 295006
AppSec [baseline] (46.897 ms) : 0, 46897
AppSec [candidate] (47.195 ms) : 0, 47195
IAST [baseline] (26.45 ms) : 0, 26450
IAST [candidate] (27.002 ms) : 0, 27002
Logs Intake [baseline] (303.91 µs) : 0, 304
Logs Intake [candidate] (319.05 µs) : 0, 319
Remote Config [baseline] (575.986 µs) : 0, 576
Remote Config [candidate] (605.506 µs) : 0, 606
Telemetry [baseline] (9.229 ms) : 0, 9229
Telemetry [candidate] (7.633 ms) : 0, 7633
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (783.759 ms) : 0, 783759
BytebuddyAgent [candidate] (780.762 ms) : 0, 780762
GlobalTracer [baseline] (295.206 ms) : 0, 295206
GlobalTracer [candidate] (294.417 ms) : 0, 294417
AppSec [baseline] (47.057 ms) : 0, 47057
AppSec [candidate] (46.54 ms) : 0, 46540
IAST [baseline] (28.571 ms) : 0, 28571
IAST [candidate] (27.089 ms) : 0, 27089
Logs Intake [baseline] (306.829 µs) : 0, 307
Logs Intake [candidate] (301.89 µs) : 0, 302
Remote Config [baseline] (582.422 µs) : 0, 582
Remote Config [candidate] (573.863 µs) : 0, 574
Telemetry [baseline] (6.966 ms) : 0, 6966
Telemetry [candidate] (7.603 ms) : 0, 7603
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (774.656 ms) : 0, 774656
BytebuddyAgent [candidate] (781.807 ms) : 0, 781807
GlobalTracer [baseline] (293.155 ms) : 0, 293155
GlobalTracer [candidate] (295.755 ms) : 0, 295755
AppSec [baseline] (49.126 ms) : 0, 49126
AppSec [candidate] (50.774 ms) : 0, 50774
IAST [baseline] (22.718 ms) : 0, 22718
IAST [candidate] (22.457 ms) : 0, 22457
Logs Intake [baseline] (299.141 µs) : 0, 299
Logs Intake [candidate] (297.473 µs) : 0, 297
Remote Config [baseline] (562.487 µs) : 0, 562
Remote Config [candidate] (553.55 µs) : 0, 554
Telemetry [baseline] (7.534 ms) : 0, 7534
Telemetry [candidate] (7.439 ms) : 0, 7439
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-06-07T10:44:05 2024-06-07T10:50:55
git_branch master nikita-tkachenko/itr-no-coverage-for-skippable-tests
git_commit_date 1717749941 1717755953
git_commit_sha edb2c3f b58dc72
release_version 1.36.0-SNAPSHOT~edb2c3fa17 1.36.0-SNAPSHOT~b58dc72569
start_time 2024-06-07T10:43:52 2024-06-07T10:50:42
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1717757801 1717757801
ci_job_id 535787118 535787118
ci_pipeline_id 36142247 36142247
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 16 unstable metrics.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.36.0-SNAPSHOT~b58dc72569, baseline=1.36.0-SNAPSHOT~edb2c3fa17
    dateFormat X
    axisFormat %s
section baseline
no_agent (371.181 µs) : 352, 391
.   : milestone, 371,
iast (480.125 µs) : 459, 501
.   : milestone, 480,
iast_FULL (550.13 µs) : 529, 571
.   : milestone, 550,
iast_GLOBAL (503.7 µs) : 482, 526
.   : milestone, 504,
iast_HARDCODED_SECRET_DISABLED (479.744 µs) : 458, 501
.   : milestone, 480,
iast_INACTIVE (449.11 µs) : 428, 470
.   : milestone, 449,
iast_TELEMETRY_OFF (474.676 µs) : 453, 496
.   : milestone, 475,
tracing (442.181 µs) : 422, 463
.   : milestone, 442,
section candidate
no_agent (374.921 µs) : 354, 396
.   : milestone, 375,
iast (480.168 µs) : 459, 502
.   : milestone, 480,
iast_FULL (551.792 µs) : 530, 573
.   : milestone, 552,
iast_GLOBAL (506.088 µs) : 484, 528
.   : milestone, 506,
iast_HARDCODED_SECRET_DISABLED (481.838 µs) : 460, 503
.   : milestone, 482,
iast_INACTIVE (443.553 µs) : 423, 464
.   : milestone, 444,
iast_TELEMETRY_OFF (472.546 µs) : 451, 494
.   : milestone, 473,
tracing (445.106 µs) : 424, 466
.   : milestone, 445,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 371.181 µs [351.846 µs, 390.515 µs] -
iast 480.125 µs [458.854 µs, 501.396 µs] 108.944 µs (29.4%)
iast_FULL 550.13 µs [528.901 µs, 571.359 µs] 178.95 µs (48.2%)
iast_GLOBAL 503.7 µs [481.895 µs, 525.505 µs] 132.519 µs (35.7%)
iast_HARDCODED_SECRET_DISABLED 479.744 µs [458.479 µs, 501.009 µs] 108.563 µs (29.2%)
iast_INACTIVE 449.11 µs [428.428 µs, 469.793 µs] 77.929 µs (21.0%)
iast_TELEMETRY_OFF 474.676 µs [452.883 µs, 496.47 µs] 103.496 µs (27.9%)
tracing 442.181 µs [421.522 µs, 462.84 µs] 71.001 µs (19.1%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 374.921 µs [353.592 µs, 396.249 µs] -
iast 480.168 µs [458.791 µs, 501.546 µs] 105.248 µs (28.1%)
iast_FULL 551.792 µs [530.272 µs, 573.312 µs] 176.871 µs (47.2%)
iast_GLOBAL 506.088 µs [484.094 µs, 528.082 µs] 131.167 µs (35.0%)
iast_HARDCODED_SECRET_DISABLED 481.838 µs [460.34 µs, 503.336 µs] 106.917 µs (28.5%)
iast_INACTIVE 443.553 µs [422.761 µs, 464.346 µs] 68.633 µs (18.3%)
iast_TELEMETRY_OFF 472.546 µs [450.645 µs, 494.447 µs] 97.625 µs (26.0%)
tracing 445.106 µs [423.798 µs, 466.414 µs] 70.185 µs (18.7%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.36.0-SNAPSHOT~b58dc72569, baseline=1.36.0-SNAPSHOT~edb2c3fa17
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.358 ms) : 1339, 1377
.   : milestone, 1358,
appsec (1.728 ms) : 1704, 1751
.   : milestone, 1728,
appsec_no_iast (1.727 ms) : 1702, 1751
.   : milestone, 1727,
iast (1.481 ms) : 1457, 1505
.   : milestone, 1481,
profiling (1.51 ms) : 1485, 1534
.   : milestone, 1510,
tracing (1.462 ms) : 1438, 1486
.   : milestone, 1462,
section candidate
no_agent (1.346 ms) : 1327, 1366
.   : milestone, 1346,
appsec (1.736 ms) : 1712, 1760
.   : milestone, 1736,
appsec_no_iast (1.713 ms) : 1688, 1738
.   : milestone, 1713,
iast (1.478 ms) : 1455, 1501
.   : milestone, 1478,
profiling (1.545 ms) : 1520, 1571
.   : milestone, 1545,
tracing (1.461 ms) : 1438, 1485
.   : milestone, 1461,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.358 ms [1.339 ms, 1.377 ms] -
appsec 1.728 ms [1.704 ms, 1.751 ms] 369.619 µs (27.2%)
appsec_no_iast 1.727 ms [1.702 ms, 1.751 ms] 368.533 µs (27.1%)
iast 1.481 ms [1.457 ms, 1.505 ms] 122.874 µs (9.0%)
profiling 1.51 ms [1.485 ms, 1.534 ms] 151.286 µs (11.1%)
tracing 1.462 ms [1.438 ms, 1.486 ms] 103.605 µs (7.6%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.346 ms [1.327 ms, 1.366 ms] -
appsec 1.736 ms [1.712 ms, 1.76 ms] 389.785 µs (29.0%)
appsec_no_iast 1.713 ms [1.688 ms, 1.738 ms] 366.941 µs (27.3%)
iast 1.478 ms [1.455 ms, 1.501 ms] 131.95 µs (9.8%)
profiling 1.545 ms [1.52 ms, 1.571 ms] 198.794 µs (14.8%)
tracing 1.461 ms [1.438 ms, 1.485 ms] 115.038 µs (8.5%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nikita-tkachenko/itr-no-coverage-for-skippable-tests
git_commit_date 1717749941 1717755953
git_commit_sha edb2c3f b58dc72
release_version 1.36.0-SNAPSHOT~edb2c3fa17 1.36.0-SNAPSHOT~b58dc72569
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1717758299 1717758299
ci_job_id 535787119 535787119
ci_pipeline_id 36142247 36142247
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.36.0-SNAPSHOT~b58dc72569, baseline=1.36.0-SNAPSHOT~edb2c3fa17
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.469 ms) : 1458, 1481
.   : milestone, 1469,
appsec (2.202 ms) : 2169, 2236
.   : milestone, 2202,
iast (1.963 ms) : 1923, 2004
.   : milestone, 1963,
iast_GLOBAL (2.009 ms) : 1967, 2050
.   : milestone, 2009,
profiling (1.855 ms) : 1822, 1888
.   : milestone, 1855,
tracing (1.828 ms) : 1796, 1859
.   : milestone, 1828,
section candidate
no_agent (1.465 ms) : 1454, 1476
.   : milestone, 1465,
appsec (2.207 ms) : 2173, 2241
.   : milestone, 2207,
iast (1.962 ms) : 1921, 2002
.   : milestone, 1962,
iast_GLOBAL (2.006 ms) : 1965, 2046
.   : milestone, 2006,
profiling (1.851 ms) : 1818, 1884
.   : milestone, 1851,
tracing (1.828 ms) : 1796, 1860
.   : milestone, 1828,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.469 ms [1.458 ms, 1.481 ms] -
appsec 2.202 ms [2.169 ms, 2.236 ms] 732.812 µs (49.9%)
iast 1.963 ms [1.923 ms, 2.004 ms] 493.977 µs (33.6%)
iast_GLOBAL 2.009 ms [1.967 ms, 2.05 ms] 539.144 µs (36.7%)
profiling 1.855 ms [1.822 ms, 1.888 ms] 385.805 µs (26.3%)
tracing 1.828 ms [1.796 ms, 1.859 ms] 358.2 µs (24.4%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.465 ms [1.454 ms, 1.476 ms] -
appsec 2.207 ms [2.173 ms, 2.241 ms] 742.447 µs (50.7%)
iast 1.962 ms [1.921 ms, 2.002 ms] 496.705 µs (33.9%)
iast_GLOBAL 2.006 ms [1.965 ms, 2.046 ms] 540.686 µs (36.9%)
profiling 1.851 ms [1.818 ms, 1.884 ms] 386.357 µs (26.4%)
tracing 1.828 ms [1.796 ms, 1.86 ms] 362.903 µs (24.8%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.36.0-SNAPSHOT~b58dc72569, baseline=1.36.0-SNAPSHOT~edb2c3fa17
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.59 s) : 15590000, 15590000
.   : milestone, 15590000,
appsec (15.205 s) : 15205000, 15205000
.   : milestone, 15205000,
iast (18.767 s) : 18767000, 18767000
.   : milestone, 18767000,
iast_GLOBAL (17.748 s) : 17748000, 17748000
.   : milestone, 17748000,
profiling (15.466 s) : 15466000, 15466000
.   : milestone, 15466000,
tracing (15.025 s) : 15025000, 15025000
.   : milestone, 15025000,
section candidate
no_agent (15.287 s) : 15287000, 15287000
.   : milestone, 15287000,
appsec (14.791 s) : 14791000, 14791000
.   : milestone, 14791000,
iast (18.901 s) : 18901000, 18901000
.   : milestone, 18901000,
iast_GLOBAL (17.713 s) : 17713000, 17713000
.   : milestone, 17713000,
profiling (15.891 s) : 15891000, 15891000
.   : milestone, 15891000,
tracing (14.862 s) : 14862000, 14862000
.   : milestone, 14862000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.59 s [15.59 s, 15.59 s] -
appsec 15.205 s [15.205 s, 15.205 s] -385.0 ms (-2.5%)
iast 18.767 s [18.767 s, 18.767 s] 3.177 s (20.4%)
iast_GLOBAL 17.748 s [17.748 s, 17.748 s] 2.158 s (13.8%)
profiling 15.466 s [15.466 s, 15.466 s] -124.0 ms (-0.8%)
tracing 15.025 s [15.025 s, 15.025 s] -565.0 ms (-3.6%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.287 s [15.287 s, 15.287 s] -
appsec 14.791 s [14.791 s, 14.791 s] -496.0 ms (-3.2%)
iast 18.901 s [18.901 s, 18.901 s] 3.614 s (23.6%)
iast_GLOBAL 17.713 s [17.713 s, 17.713 s] 2.426 s (15.9%)
profiling 15.891 s [15.891 s, 15.891 s] 604.0 ms (4.0%)
tracing 14.862 s [14.862 s, 14.862 s] -425.0 ms (-2.8%)

@nikita-tkachenko-datadog nikita-tkachenko-datadog marked this pull request as ready for review June 7, 2024 12:34
@nikita-tkachenko-datadog nikita-tkachenko-datadog requested review from a team as code owners June 7, 2024 12:34
@nikita-tkachenko-datadog nikita-tkachenko-datadog merged commit 7afb3eb into master Jun 7, 2024
83 checks passed
@nikita-tkachenko-datadog nikita-tkachenko-datadog deleted the nikita-tkachenko/itr-no-coverage-for-skippable-tests branch June 7, 2024 12:49
@github-actions github-actions bot added this to the 1.35.0 milestone Jun 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@anmarchenko anmarchenko anmarchenko approved these changes

@PerfectSlayer PerfectSlayer Awaiting requested review from PerfectSlayer PerfectSlayer is a code owner automatically assigned from DataDog/apm-java

@nayeem-kamal nayeem-kamal Awaiting requested review from nayeem-kamal nayeem-kamal is a code owner automatically assigned from DataDog/apm-java

Assignees
No one assigned
Labels
comp: ci visibility Continuous Integration Visibility tag: performance Performance related changes
Projects
None yet
Milestone
1.35.0
Development

Successfully merging this pull request may close these issues.
2 participants
@nikita-tkachenko-datadog @anmarchenko