Generalize TestRetryPolicy to TestExecutionPolicy #8302

nikita-tkachenko-datadog · 2025-01-29T15:31:53Z

What Does This Do

Renames TestRetryPolicy to TestExecutionPolicy.

Motivation

Test execution policies will be used for tests quarantining, which does not involve retries but alters the normal test execution flow (by suppressing exceptions).

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the public documentation in case of new configuration flag or behavior

Jira ticket: SDTEST-1483

pr-commenter · 2025-01-29T16:13:59Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nikita-tkachenko/tests-quarantine
git_commit_date	1738161747	1738164569
git_commit_sha	`a41d151`	`0176671`
release_version	1.46.0-SNAPSHOT~a41d1512de	1.46.0-SNAPSHOT~0176671e93

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1738167193	1738167193
ci_job_id	785691222	785691222
ci_pipeline_id	54331900	54331900
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-axymc9my-project-304-concurrent-1-r3o1cneg 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-axymc9my-project-304-concurrent-1-r3o1cneg 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 51 metrics, 12 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.46.0-SNAPSHOT~0176671e93, baseline=1.46.0-SNAPSHOT~a41d1512de

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.043 s) : 0, 1042909
Total [baseline] (8.618 s) : 0, 8618429
Agent [candidate] (1.038 s) : 0, 1038266
Total [candidate] (8.608 s) : 0, 8608032
section iast
Agent [baseline] (1.176 s) : 0, 1176403
Total [baseline] (9.289 s) : 0, 9288799
Agent [candidate] (1.177 s) : 0, 1176596
Total [candidate] (9.284 s) : 0, 9283975
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.176 s) : 0, 1175986
Total [baseline] (9.234 s) : 0, 9233611
Agent [candidate] (1.173 s) : 0, 1172877
Total [candidate] (9.178 s) : 0, 9177842
section iast_TELEMETRY_OFF
Agent [baseline] (1.163 s) : 0, 1163130
Total [baseline] (9.231 s) : 0, 9231263
Agent [candidate] (1.173 s) : 0, 1173466
Total [candidate] (9.237 s) : 0, 9237052

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.043 s	-
Agent	iast	1.176 s	133.494 ms (12.8%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.176 s	133.078 ms (12.8%)
Agent	iast_TELEMETRY_OFF	1.163 s	120.222 ms (11.5%)
Total	tracing	8.618 s	-
Total	iast	9.289 s	670.37 ms (7.8%)
Total	iast_HARDCODED_SECRET_DISABLED	9.234 s	615.181 ms (7.1%)
Total	iast_TELEMETRY_OFF	9.231 s	612.834 ms (7.1%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.038 s	-
Agent	iast	1.177 s	138.331 ms (13.3%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.173 s	134.612 ms (13.0%)
Agent	iast_TELEMETRY_OFF	1.173 s	135.201 ms (13.0%)
Total	tracing	8.608 s	-
Total	iast	9.284 s	675.943 ms (7.9%)
Total	iast_HARDCODED_SECRET_DISABLED	9.178 s	569.809 ms (6.6%)
Total	iast_TELEMETRY_OFF	9.237 s	629.02 ms (7.3%)

gantt
    title insecure-bank - break down per module: candidate=1.46.0-SNAPSHOT~0176671e93, baseline=1.46.0-SNAPSHOT~a41d1512de

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (718.573 ms) : 0, 718573
BytebuddyAgent [candidate] (714.25 ms) : 0, 714250
GlobalTracer [baseline] (240.57 ms) : 0, 240570
GlobalTracer [candidate] (239.334 ms) : 0, 239334
AppSec [baseline] (56.239 ms) : 0, 56239
AppSec [candidate] (55.001 ms) : 0, 55001
Remote Config [baseline] (718.219 µs) : 0, 718
Remote Config [candidate] (707.811 µs) : 0, 708
Telemetry [baseline] (11.508 ms) : 0, 11508
Telemetry [candidate] (13.649 ms) : 0, 13649
section iast
BytebuddyAgent [baseline] (839.591 ms) : 0, 839591
BytebuddyAgent [candidate] (839.543 ms) : 0, 839543
GlobalTracer [baseline] (232.339 ms) : 0, 232339
GlobalTracer [candidate] (232.181 ms) : 0, 232181
IAST [baseline] (24.853 ms) : 0, 24853
IAST [candidate] (27.274 ms) : 0, 27274
AppSec [baseline] (54.822 ms) : 0, 54822
AppSec [candidate] (52.801 ms) : 0, 52801
Remote Config [baseline] (628.691 µs) : 0, 629
Remote Config [candidate] (627.333 µs) : 0, 627
Telemetry [baseline] (8.744 ms) : 0, 8744
Telemetry [candidate] (8.754 ms) : 0, 8754
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (838.965 ms) : 0, 838965
BytebuddyAgent [candidate] (837.917 ms) : 0, 837917
GlobalTracer [baseline] (231.569 ms) : 0, 231569
GlobalTracer [candidate] (230.046 ms) : 0, 230046
IAST [baseline] (25.997 ms) : 0, 25997
IAST [candidate] (23.625 ms) : 0, 23625
AppSec [baseline] (54.642 ms) : 0, 54642
AppSec [candidate] (56.482 ms) : 0, 56482
Remote Config [baseline] (632.607 µs) : 0, 633
Remote Config [candidate] (624.939 µs) : 0, 625
Telemetry [baseline] (8.794 ms) : 0, 8794
Telemetry [candidate] (8.839 ms) : 0, 8839
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (829.593 ms) : 0, 829593
BytebuddyAgent [candidate] (837.566 ms) : 0, 837566
GlobalTracer [baseline] (230.347 ms) : 0, 230347
GlobalTracer [candidate] (232.048 ms) : 0, 232048
IAST [baseline] (24.181 ms) : 0, 24181
IAST [candidate] (25.31 ms) : 0, 25310
AppSec [baseline] (54.641 ms) : 0, 54641
AppSec [candidate] (53.935 ms) : 0, 53935
Remote Config [baseline] (610.121 µs) : 0, 610
Remote Config [candidate] (612.726 µs) : 0, 613
Telemetry [baseline] (8.508 ms) : 0, 8508
Telemetry [candidate] (8.573 ms) : 0, 8573

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.46.0-SNAPSHOT~0176671e93, baseline=1.46.0-SNAPSHOT~a41d1512de

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.039 s) : 0, 1038972
Total [baseline] (10.571 s) : 0, 10571407
Agent [candidate] (1.044 s) : 0, 1044403
Total [candidate] (10.612 s) : 0, 10611835
section appsec
Agent [baseline] (1.185 s) : 0, 1185307
Total [baseline] (10.728 s) : 0, 10728261
Agent [candidate] (1.182 s) : 0, 1182119
Total [candidate] (10.779 s) : 0, 10778725
section iast
Agent [baseline] (1.178 s) : 0, 1178214
Total [baseline] (10.939 s) : 0, 10939453
Agent [candidate] (1.168 s) : 0, 1167603
Total [candidate] (10.904 s) : 0, 10903907
section profiling
Agent [baseline] (1.258 s) : 0, 1258031
Total [baseline] (10.863 s) : 0, 10863037
Agent [candidate] (1.256 s) : 0, 1256192
Total [candidate] (10.842 s) : 0, 10842421

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.039 s	-
Agent	appsec	1.185 s	146.335 ms (14.1%)
Agent	iast	1.178 s	139.243 ms (13.4%)
Agent	profiling	1.258 s	219.059 ms (21.1%)
Total	tracing	10.571 s	-
Total	appsec	10.728 s	156.853 ms (1.5%)
Total	iast	10.939 s	368.046 ms (3.5%)
Total	profiling	10.863 s	291.63 ms (2.8%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.044 s	-
Agent	appsec	1.182 s	137.716 ms (13.2%)
Agent	iast	1.168 s	123.201 ms (11.8%)
Agent	profiling	1.256 s	211.79 ms (20.3%)
Total	tracing	10.612 s	-
Total	appsec	10.779 s	166.89 ms (1.6%)
Total	iast	10.904 s	292.072 ms (2.8%)
Total	profiling	10.842 s	230.585 ms (2.2%)

gantt
    title petclinic - break down per module: candidate=1.46.0-SNAPSHOT~0176671e93, baseline=1.46.0-SNAPSHOT~a41d1512de

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (714.035 ms) : 0, 714035
BytebuddyAgent [candidate] (719.395 ms) : 0, 719395
GlobalTracer [baseline] (240.628 ms) : 0, 240628
GlobalTracer [candidate] (241.306 ms) : 0, 241306
AppSec [baseline] (55.463 ms) : 0, 55463
AppSec [candidate] (56.036 ms) : 0, 56036
Remote Config [baseline] (715.025 µs) : 0, 715
Remote Config [candidate] (724.694 µs) : 0, 725
Telemetry [baseline] (12.939 ms) : 0, 12939
Telemetry [candidate] (11.645 ms) : 0, 11645
section appsec
BytebuddyAgent [baseline] (735.966 ms) : 0, 735966
BytebuddyAgent [candidate] (734.34 ms) : 0, 734340
GlobalTracer [baseline] (238.21 ms) : 0, 238210
GlobalTracer [candidate] (237.442 ms) : 0, 237442
AppSec [baseline] (175.673 ms) : 0, 175673
AppSec [candidate] (175.32 ms) : 0, 175320
Remote Config [baseline] (658.862 µs) : 0, 659
Remote Config [candidate] (655.838 µs) : 0, 656
Telemetry [baseline] (8.722 ms) : 0, 8722
Telemetry [candidate] (8.32 ms) : 0, 8320
IAST [baseline] (21.549 ms) : 0, 21549
IAST [candidate] (21.538 ms) : 0, 21538
section iast
BytebuddyAgent [baseline] (840.447 ms) : 0, 840447
BytebuddyAgent [candidate] (832.912 ms) : 0, 832912
GlobalTracer [baseline] (232.693 ms) : 0, 232693
GlobalTracer [candidate] (230.536 ms) : 0, 230536
AppSec [baseline] (54.239 ms) : 0, 54239
AppSec [candidate] (55.371 ms) : 0, 55371
Remote Config [baseline] (625.445 µs) : 0, 625
Remote Config [candidate] (608.544 µs) : 0, 609
Telemetry [baseline] (8.838 ms) : 0, 8838
Telemetry [candidate] (8.729 ms) : 0, 8729
IAST [baseline] (25.127 ms) : 0, 25127
IAST [candidate] (24.151 ms) : 0, 24151
section profiling
BytebuddyAgent [baseline] (704.795 ms) : 0, 704795
BytebuddyAgent [candidate] (704.798 ms) : 0, 704798
GlobalTracer [baseline] (350.829 ms) : 0, 350829
GlobalTracer [candidate] (349.959 ms) : 0, 349959
AppSec [baseline] (55.106 ms) : 0, 55106
AppSec [candidate] (54.119 ms) : 0, 54119
Remote Config [baseline] (708.976 µs) : 0, 709
Remote Config [candidate] (714.28 µs) : 0, 714
Telemetry [baseline] (8.801 ms) : 0, 8801
Telemetry [candidate] (8.896 ms) : 0, 8896
ProfilingAgent [baseline] (95.592 ms) : 0, 95592
ProfilingAgent [candidate] (95.378 ms) : 0, 95378
Profiling [baseline] (95.615 ms) : 0, 95615
Profiling [candidate] (95.403 ms) : 0, 95403

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2025-01-29T15:41:06	2025-01-29T15:48:09
git_branch	master	nikita-tkachenko/tests-quarantine
git_commit_date	1738161747	1738164569
git_commit_sha	`a41d151`	`0176671`
release_version	1.46.0-SNAPSHOT~a41d1512de	1.46.0-SNAPSHOT~0176671e93
start_time	2025-01-29T15:40:51	2025-01-29T15:47:55

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1738166046	1738166046
ci_job_id	785691223	785691223
ci_pipeline_id	54331900	54331900
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-tpd8mfxw-project-304-concurrent-0-nd5abg35 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-tpd8mfxw-project-304-concurrent-0-nd5abg35 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 18 unstable metrics.

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.46.0-SNAPSHOT~0176671e93, baseline=1.46.0-SNAPSHOT~a41d1512de
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.341 ms) : 1322, 1360
.   : milestone, 1341,
appsec (1.748 ms) : 1724, 1772
.   : milestone, 1748,
appsec_no_iast (1.767 ms) : 1744, 1789
.   : milestone, 1767,
iast (1.52 ms) : 1497, 1544
.   : milestone, 1520,
profiling (1.539 ms) : 1515, 1563
.   : milestone, 1539,
tracing (1.48 ms) : 1454, 1505
.   : milestone, 1480,
section candidate
no_agent (1.346 ms) : 1326, 1366
.   : milestone, 1346,
appsec (1.756 ms) : 1732, 1780
.   : milestone, 1756,
appsec_no_iast (1.762 ms) : 1738, 1786
.   : milestone, 1762,
iast (1.501 ms) : 1476, 1525
.   : milestone, 1501,
profiling (1.505 ms) : 1482, 1529
.   : milestone, 1505,
tracing (1.493 ms) : 1469, 1517
.   : milestone, 1493,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.341 ms [1.322 ms, 1.36 ms]	-
appsec	1.748 ms [1.724 ms, 1.772 ms]	407.053 µs (30.4%)
appsec_no_iast	1.767 ms [1.744 ms, 1.789 ms]	425.637 µs (31.7%)
iast	1.52 ms [1.497 ms, 1.544 ms]	179.561 µs (13.4%)
profiling	1.539 ms [1.515 ms, 1.563 ms]	197.925 µs (14.8%)
tracing	1.48 ms [1.454 ms, 1.505 ms]	138.722 µs (10.3%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.346 ms [1.326 ms, 1.366 ms]	-
appsec	1.756 ms [1.732 ms, 1.78 ms]	409.497 µs (30.4%)
appsec_no_iast	1.762 ms [1.738 ms, 1.786 ms]	415.807 µs (30.9%)
iast	1.501 ms [1.476 ms, 1.525 ms]	154.333 µs (11.5%)
profiling	1.505 ms [1.482 ms, 1.529 ms]	158.939 µs (11.8%)
tracing	1.493 ms [1.469 ms, 1.517 ms]	146.837 µs (10.9%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.46.0-SNAPSHOT~0176671e93, baseline=1.46.0-SNAPSHOT~a41d1512de
    dateFormat X
    axisFormat %s
section baseline
no_agent (382.209 µs) : 355, 409
.   : milestone, 382,
iast (514.103 µs) : 491, 538
.   : milestone, 514,
iast_FULL (745.373 µs) : 723, 767
.   : milestone, 745,
iast_GLOBAL (548.119 µs) : 526, 570
.   : milestone, 548,
iast_HARDCODED_SECRET_DISABLED (508.774 µs) : 485, 532
.   : milestone, 509,
iast_INACTIVE (458.11 µs) : 437, 480
.   : milestone, 458,
iast_TELEMETRY_OFF (493.376 µs) : 470, 516
.   : milestone, 493,
tracing (446.463 µs) : 426, 467
.   : milestone, 446,
section candidate
no_agent (378.076 µs) : 359, 398
.   : milestone, 378,
iast (510.326 µs) : 487, 534
.   : milestone, 510,
iast_FULL (740.246 µs) : 718, 763
.   : milestone, 740,
iast_GLOBAL (557.803 µs) : 534, 581
.   : milestone, 558,
iast_HARDCODED_SECRET_DISABLED (504.562 µs) : 482, 528
.   : milestone, 505,
iast_INACTIVE (459.881 µs) : 438, 482
.   : milestone, 460,
iast_TELEMETRY_OFF (496.287 µs) : 473, 519
.   : milestone, 496,
tracing (449.391 µs) : 429, 470
.   : milestone, 449,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	382.209 µs [355.432 µs, 408.985 µs]	-
iast	514.103 µs [490.562 µs, 537.644 µs]	131.894 µs (34.5%)
iast_FULL	745.373 µs [723.309 µs, 767.437 µs]	363.164 µs (95.0%)
iast_GLOBAL	548.119 µs [525.775 µs, 570.464 µs]	165.911 µs (43.4%)
iast_HARDCODED_SECRET_DISABLED	508.774 µs [485.244 µs, 532.305 µs]	126.566 µs (33.1%)
iast_INACTIVE	458.11 µs [436.658 µs, 479.562 µs]	75.901 µs (19.9%)
iast_TELEMETRY_OFF	493.376 µs [470.352 µs, 516.4 µs]	111.167 µs (29.1%)
tracing	446.463 µs [425.718 µs, 467.208 µs]	64.254 µs (16.8%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	378.076 µs [358.531 µs, 397.622 µs]	-
iast	510.326 µs [486.895 µs, 533.757 µs]	132.25 µs (35.0%)
iast_FULL	740.246 µs [717.509 µs, 762.983 µs]	362.17 µs (95.8%)
iast_GLOBAL	557.803 µs [534.126 µs, 581.48 µs]	179.727 µs (47.5%)
iast_HARDCODED_SECRET_DISABLED	504.562 µs [481.601 µs, 527.522 µs]	126.485 µs (33.5%)
iast_INACTIVE	459.881 µs [438.202 µs, 481.56 µs]	81.805 µs (21.6%)
iast_TELEMETRY_OFF	496.287 µs [473.293 µs, 519.28 µs]	118.211 µs (31.3%)
tracing	449.391 µs [428.524 µs, 470.258 µs]	71.315 µs (18.9%)

Dacapo

| Package | Type | Package file | Manager | Update | Change | |---|---|---|---|---|---| | [com.google.cloud:google-cloud-datastore](https://github.com/googleapis/java-datastore) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `2.25.4` -> `2.26.0` | | [com.datadoghq:dd-trace-api](https://github.com/datadog/dd-trace-java) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `1.45.2` -> `1.46.0` | | [com.datadoghq:dd-trace-ot](https://github.com/datadog/dd-trace-java) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `1.45.2` -> `1.46.0` | | [software.amazon.awssdk:sdk-core](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.30.9` -> `2.30.10` | | [software.amazon.awssdk:dynamodb-enhanced](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.30.9` -> `2.30.10` | | [software.amazon.awssdk:dynamodb](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.30.9` -> `2.30.10` | | [software.amazon.awssdk:aws-core](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.30.9` -> `2.30.10` | | [software.amazon.awssdk:bom](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.30.9` -> `2.30.10` | | [software.amazon.awssdk:auth](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.30.9` -> `2.30.10` | --- ### Release Notes <details> <summary>googleapis/java-datastore (com.google.cloud:google-cloud-datastore)</summary> ### [`v2.26.0`](https://github.com/googleapis/java-datastore/blob/HEAD/CHANGELOG.md#2260-2025-01-29) ##### Features - Add firestoreInDatastoreMode for datastore emulator ([#1698](googleapis/java-datastore#1698)) ([50f106d](googleapis/java-datastore@50f106d)) ##### Dependencies - Update dependency com.google.cloud:sdk-platform-java-config to v3.42.0 ([#1725](googleapis/java-datastore#1725)) ([1cbaf22](googleapis/java-datastore@1cbaf22)) </details> <details> <summary>datadog/dd-trace-java (com.datadoghq:dd-trace-api)</summary> ### [`v1.46.0`](https://github.com/DataDog/dd-trace-java/releases/tag/v1.46.0): 1.46.0 ##### Breaking Changes > \[!WARNING] > jnr-unixsocket is now an external dependency of dd-trace-ot and must be included when deploying dd-trace-ot. > \[!NOTE] > The API `TracerScope.setAsync(boolean)`, used to manually control asynchronous span propagation, does no more apply to the scope instance but to the active span scope. ##### Components ##### Application Security Management (IAST) - 🐛 Fix String.replace instrumentation for IAST ([#8281](DataDog/dd-trace-java#8281) - [@Mariovido](https://github.com/Mariovido)) - ✨ Apply the standard nomenclature to the stacktrace configs ([#8244](DataDog/dd-trace-java#8244) - [@jandro996](https://github.com/jandro996)) - 🐛 Exclude false positive weak randomness ([#8232](DataDog/dd-trace-java#8232) - [@jandro996](https://github.com/jandro996)) - ✨ Propagation of translateEscapes of String class ([#8186](DataDog/dd-trace-java#8186) - [@sezen-datadog](https://github.com/sezen-datadog)) - ✨ Add security control metrics ([#8175](DataDog/dd-trace-java#8175) - [@jandro996](https://github.com/jandro996)) - ✨ Increase IAST propagation to StringBuffer setLength ([#8128](DataDog/dd-trace-java#8128) - [@Mariovido](https://github.com/Mariovido)) - ✨ Add IAST taint tracking for DB values ([#8072](DataDog/dd-trace-java#8072) - [@Mariovido](https://github.com/Mariovido)) ##### Application Security Management (WAF) - 🐛 Prevents a NPE when there is no subscriber for user events ([#8258](DataDog/dd-trace-java#8258) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) - ✨ Apply the standard nomenclature to the stacktrace configs ([#8244](DataDog/dd-trace-java#8244) - [@jandro996](https://github.com/jandro996)) - 🐛 Ensure cached subscriptions are cleared on reconfiguration via RC ([#8229](DataDog/dd-trace-java#8229) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) - ✨ Add support for session tracking in Vertx ([#8167](DataDog/dd-trace-java#8167) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) - ✨ Create span tag: \_dd.appsec.rasp.timeout ([#8269](DataDog/dd-trace-java#8269) - [@Mariovido](https://github.com/Mariovido)) ##### Build & Tooling - 🐛 Ensure shaded helpers have unique names when injected into class-loaders ([#8192](DataDog/dd-trace-java#8192) - [@mcculls](https://github.com/mcculls)) ##### Configuration at Runtime - 🐛 Remove filtering of `DD_SERVICE` and `DD_ENV` from the tracer ([#8176](DataDog/dd-trace-java#8176) - [@mhlidd](https://github.com/mhlidd)) ##### Continuous Integration Visibility - 🧹 Generalize TestRetryPolicy to TestExecutionPolicy ([#8302](DataDog/dd-trace-java#8302) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - 🧹 Parallelize CI Visibility settings requests ([#8299](DataDog/dd-trace-java#8299) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - 🧹 Generalize test retry logic ([#8289](DataDog/dd-trace-java#8289) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - 🧹 Generalize tests skipping logic ([#8288](DataDog/dd-trace-java#8288) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - 🧹 Remove skip and shouldBeSkipped methods from TestEventsHandler in favor of isSkippable ([#8286](DataDog/dd-trace-java#8286) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨⚡ Optimize Git repository information computation ([#8270](DataDog/dd-trace-java#8270) - [@dougqh](https://github.com/dougqh)) - ✨ Always request known tests from the backend ([#8268](DataDog/dd-trace-java#8268) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨ Fix NPE when trying to get retry analyzer in Test NG ([#8253](DataDog/dd-trace-java#8253) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - 🐛 Set test framework and test framework version tags atomically ([#8252](DataDog/dd-trace-java#8252) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨ Add debug logging to Android Gradle module layout logic ([#8251](DataDog/dd-trace-java#8251) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - 🐛 Fix source and destination folders computation for Android Gradle projects ([#8190](DataDog/dd-trace-java#8190) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨ Add basic Scala Weaver sbt support ([#8189](DataDog/dd-trace-java#8189) - [@daniel-mohedano](https://github.com/daniel-mohedano)) - ✨ Implement impacted tests detection ([#8188](DataDog/dd-trace-java#8188) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) ##### Data Streams Monitoring - ✨ Change hash computation for protobuf to better represent impacting changes + save proto number in schema ([#8201](DataDog/dd-trace-java#8201) - [@vandonr](https://github.com/vandonr)) ##### Database Monitoring - Add peer service tag in dbm sql commenter ([#7913](DataDog/dd-trace-java#7913) - [@jordan-wong](https://github.com/jordan-wong)) ##### Dynamic Instrumentation - ✨ Add support for SymDB to scan directories ([#8306](DataDog/dd-trace-java#8306) - [@jpbempel](https://github.com/jpbempel)) - ✨ Add SymDB report for any jar scanning failures ([#8300](DataDog/dd-trace-java#8300) - [@jpbempel](https://github.com/jpbempel)) - ✨ Use two budgets depending on type ([#8283](DataDog/dd-trace-java#8283) - [@evanchooly](https://github.com/evanchooly)) - ✨ Institute a 10 snapshot per probe per trace budget ([#8277](DataDog/dd-trace-java#8277) - [@evanchooly](https://github.com/evanchooly)) - 🐛 Avoid double snapshots for Exception Replay ([#8273](DataDog/dd-trace-java#8273) - [@jpbempel](https://github.com/jpbempel)) - ✨ Simplify code origins. Separate out snapshot generation. ([#8263](DataDog/dd-trace-java#8263) - [@evanchooly](https://github.com/evanchooly)) - ✨ Add Exception probe custom instrumentation ([#8230](DataDog/dd-trace-java#8230) - [@jpbempel](https://github.com/jpbempel)) - ✨ Enhance log probes to honor debug session tags ([#8215](DataDog/dd-trace-java#8215) - [@evanchooly](https://github.com/evanchooly)) - 🐛 Don't redact env tokens from debugger probe snapshots ([#8211](DataDog/dd-trace-java#8211) - [@watson](https://github.com/watson)) - ✨⚡ Move Trace/SpanId capture at commit time ([#8184](DataDog/dd-trace-java#8184) - [@jpbempel](https://github.com/jpbempel)) - 🐛 Capture values at entry for method probe ([#8169](DataDog/dd-trace-java#8169) - [@jpbempel](https://github.com/jpbempel)) ##### JMX fetch - 🐛 Mute JMXFetch Shutdown in progress error ([#8068](DataDog/dd-trace-java#8068) - [@ygree](https://github.com/ygree)) ##### OpenTracing - ⚠️🧹 Make jnr-unixsocket an explicit dependency of dd-trace-ot ([#8307](DataDog/dd-trace-java#8307) - [@mcculls](https://github.com/mcculls)) ##### Profiling - 🐛 Avoid unsupported API call for creating folders on windows ([#8304](DataDog/dd-trace-java#8304) - [@jbachorik](https://github.com/jbachorik)) - ✨ Tag profiles for serverless ([#8279](DataDog/dd-trace-java#8279) - [@jbachorik](https://github.com/jbachorik)) - ✨ add queue type and length to queue events ([#8242](DataDog/dd-trace-java#8242) - [@richardstartin](https://github.com/richardstartin)) - 🐛 TempLocationManager Fixes and Improvements ([#8191](DataDog/dd-trace-java#8191) - [@jbachorik](https://github.com/jbachorik)) - ✨ Bump ddprof to 1.18.0 ([#8173](DataDog/dd-trace-java#8173) - [@jbachorik](https://github.com/jbachorik)) - ✨ Report profiler initialization and configuration errors to telemetry ([#8171](DataDog/dd-trace-java#8171) - [@jbachorik](https://github.com/jbachorik)) ##### Telemetry - ✨ Add pending traces report in tracer flares ([#8053](DataDog/dd-trace-java#8053) - [@mhlidd](https://github.com/mhlidd)) ##### Testing - ✨ Test http server requests in parallel ([#8222](DataDog/dd-trace-java#8222) - [@amarziali](https://github.com/amarziali)) ##### Trace context propagation - ✨ Add non default propagator registration ([#8310](DataDog/dd-trace-java#8310) - [@PerfectSlayer](https://github.com/PerfectSlayer)) ##### Tracer core - ✨ Probe for existence of IBMSASL or ACCP security providers ([#8276](DataDog/dd-trace-java#8276) - [@mcculls](https://github.com/mcculls)) - ✨⚡ Overhead improvement to agent feedback based sampling ([#8265](DataDog/dd-trace-java#8265) - [@dougqh](https://github.com/dougqh)) - 🧹 Move async propagation API from scope to tracer ([#8231](DataDog/dd-trace-java#8231) - [@PerfectSlayer](https://github.com/PerfectSlayer)) - ✨ Introduce context propagation API ([#8161](DataDog/dd-trace-java#8161) - [@PerfectSlayer](https://github.com/PerfectSlayer)) - ✨🧪 Use env-entry to add tags per webapp deployment ([#8138](DataDog/dd-trace-java#8138) - [@amarziali](https://github.com/amarziali)) - ✨ Introduce context helpers API ([#8134](DataDog/dd-trace-java#8134) - [@PerfectSlayer](https://github.com/PerfectSlayer)) - ✨ Support IPv6 values for `DD_AGENT_HOST` and `DD_TRACE_AGENT_URL` ([#7984](DataDog/dd-trace-java#7984) - [@mhlidd](https://github.com/mhlidd)) ##### Instrumentations ##### Apache HttpComponents - 🐛 Properly finish spans and support latest apache httpclient5 ([#8272](DataDog/dd-trace-java#8272) - [@amarziali](https://github.com/amarziali)) ##### AWS Lambda instrumentation - 🐛 Properly capture lambda payloads for all handler types. ([#8264](DataDog/dd-trace-java#8264) - [@purple4reina](https://github.com/purple4reina)) ##### AWS S3 instrumentation - 💡 Create S3 instrumentation + add span pointers ([#8075](DataDog/dd-trace-java#8075) - [@nhulston](https://github.com/nhulston)) ##### AWS SDK instrumentation - 🐛 Revert "Add avoid double instrumenting lambda non-streaming handlers." ([#8247](DataDog/dd-trace-java#8247) - [@nhulston](https://github.com/nhulston)) ##### Cassandra - ✨ Allow extracting keyspace from statement result ([#8239](DataDog/dd-trace-java#8239) - [@amarziali](https://github.com/amarziali)) ##### Core Java language instrumentation - ✨ Propagation of translateEscapes of String class ([#8186](DataDog/dd-trace-java#8186) - [@sezen-datadog](https://github.com/sezen-datadog)) ##### Eclipse Vert.x instrumentation - 🐛 Fix vertx worker propagation and error handling ([#8237](DataDog/dd-trace-java#8237) - [@amarziali](https://github.com/amarziali)) - ✨ Support vertx 5 ([#8220](DataDog/dd-trace-java#8220) - [@amarziali](https://github.com/amarziali)) - ✨ Add support for session tracking in Vertx ([#8167](DataDog/dd-trace-java#8167) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) ##### Kafka instrumentation - 🐛 Prevent possible NPE calculating Kafka record header size ([#8292](DataDog/dd-trace-java#8292) - [@ygree](https://github.com/ygree)) ##### Mule instrumentation - 🐛 Fix crash using Mule with JPMS ([#8187](DataDog/dd-trace-java#8187) - [@amarziali](https://github.com/amarziali)) ##### Protocol Buffer instrumentation - ✨ Change hash computation for protobuf to better represent impacting changes + save proto number in schema ([#8201](DataDog/dd-trace-java#8201) - [@vandonr](https://github.com/vandonr)) ##### Spring instrumentation - 🐛 Preserve getQualifier from spring scheduling runnables ([#8293](DataDog/dd-trace-java#8293) - [@amarziali](https://github.com/amarziali)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - "after 6pm every weekday,before 2am every weekday" in timezone Australia/Melbourne, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://github.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). GitOrigin-RevId: bb09d47e4eed77a003f630273b4d0a84003eb899

Generalize TestRetryPolicy to TestExecutionPolicy

0176671

nikita-tkachenko-datadog added type: refactoring comp: ci visibility Continuous Integration Visibility labels Jan 29, 2025

nikita-tkachenko-datadog marked this pull request as ready for review January 30, 2025 10:55

nikita-tkachenko-datadog requested review from a team as code owners January 30, 2025 10:55

nikita-tkachenko-datadog requested review from anmarchenko, mcculls and daniel-mohedano January 30, 2025 10:55

daniel-mohedano approved these changes Jan 30, 2025

View reviewed changes

nikita-tkachenko-datadog merged commit 57a74ad into master Jan 30, 2025
198 of 200 checks passed

nikita-tkachenko-datadog deleted the nikita-tkachenko/tests-quarantine branch January 30, 2025 11:19

github-actions bot added this to the 1.46.0 milestone Jan 30, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Generalize TestRetryPolicy to TestExecutionPolicy #8302

Generalize TestRetryPolicy to TestExecutionPolicy #8302

nikita-tkachenko-datadog commented Jan 29, 2025 •

edited

Loading

pr-commenter bot commented Jan 29, 2025

Generalize TestRetryPolicy to TestExecutionPolicy #8302

Generalize TestRetryPolicy to TestExecutionPolicy #8302

Conversation

nikita-tkachenko-datadog commented Jan 29, 2025 • edited Loading

What Does This Do

Motivation

Contributor Checklist

pr-commenter bot commented Jan 29, 2025

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

nikita-tkachenko-datadog commented Jan 29, 2025 •

edited

Loading