Generalize tests skipping logic #8288

nikita-tkachenko-datadog · 2025-01-27T16:10:56Z

What Does This Do

Does some preparatory work for future flaky test management features.
When flaky test management is implemented, the tracer will need to skip quarantined test cases.
This skipping will be done the same way ITR (aka TIA) skips tests at the moment.
So this PR generalises tests skipping logic, separating it from ITR-specific things.

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the public documentation in case of new configuration flag or behavior

Jira ticket: SDTEST-1483

pr-commenter · 2025-01-27T16:54:51Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nikita-tkachenko/generalize-test-skipping
git_commit_date	1737992274	1737994187
git_commit_sha	`af5f441`	`dc06b5e`
release_version	1.46.0-SNAPSHOT~af5f4412a1	1.46.0-SNAPSHOT~dc06b5ec8e

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1737996724	1737996724
ci_job_id	781850036	781850036
ci_pipeline_id	54081720	54081720
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-8wlcx2s5-project-304-concurrent-1-hwgmq64r 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-8wlcx2s5-project-304-concurrent-1-hwgmq64r 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 60 metrics, 3 unstable metrics.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.46.0-SNAPSHOT~dc06b5ec8e, baseline=1.46.0-SNAPSHOT~af5f4412a1

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.046 s) : 0, 1046341
Total [baseline] (10.481 s) : 0, 10481166
Agent [candidate] (1.04 s) : 0, 1040283
Total [candidate] (10.537 s) : 0, 10537089
section appsec
Agent [baseline] (1.182 s) : 0, 1182196
Total [baseline] (10.781 s) : 0, 10781025
Agent [candidate] (1.178 s) : 0, 1178490
Total [candidate] (10.723 s) : 0, 10723488
section iast
Agent [baseline] (1.171 s) : 0, 1170518
Total [baseline] (10.925 s) : 0, 10925487
Agent [candidate] (1.176 s) : 0, 1176042
Total [candidate] (11.012 s) : 0, 11012236
section profiling
Agent [baseline] (1.258 s) : 0, 1257740
Total [baseline] (10.905 s) : 0, 10905044
Agent [candidate] (1.255 s) : 0, 1254561
Total [candidate] (10.885 s) : 0, 10885309

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.046 s	-
Agent	appsec	1.182 s	135.855 ms (13.0%)
Agent	iast	1.171 s	124.177 ms (11.9%)
Agent	profiling	1.258 s	211.4 ms (20.2%)
Total	tracing	10.481 s	-
Total	appsec	10.781 s	299.859 ms (2.9%)
Total	iast	10.925 s	444.321 ms (4.2%)
Total	profiling	10.905 s	423.878 ms (4.0%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.04 s	-
Agent	appsec	1.178 s	138.206 ms (13.3%)
Agent	iast	1.176 s	135.759 ms (13.1%)
Agent	profiling	1.255 s	214.278 ms (20.6%)
Total	tracing	10.537 s	-
Total	appsec	10.723 s	186.399 ms (1.8%)
Total	iast	11.012 s	475.147 ms (4.5%)
Total	profiling	10.885 s	348.22 ms (3.3%)

gantt
    title petclinic - break down per module: candidate=1.46.0-SNAPSHOT~dc06b5ec8e, baseline=1.46.0-SNAPSHOT~af5f4412a1

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (719.94 ms) : 0, 719940
BytebuddyAgent [candidate] (715.967 ms) : 0, 715967
GlobalTracer [baseline] (241.125 ms) : 0, 241125
GlobalTracer [candidate] (239.463 ms) : 0, 239463
AppSec [baseline] (56.206 ms) : 0, 56206
AppSec [candidate] (55.573 ms) : 0, 55573
Remote Config [baseline] (734.956 µs) : 0, 735
Remote Config [candidate] (710.362 µs) : 0, 710
Telemetry [baseline] (13.041 ms) : 0, 13041
Telemetry [candidate] (13.543 ms) : 0, 13543
section appsec
BytebuddyAgent [baseline] (734.186 ms) : 0, 734186
BytebuddyAgent [candidate] (731.971 ms) : 0, 731971
GlobalTracer [baseline] (237.71 ms) : 0, 237710
GlobalTracer [candidate] (236.51 ms) : 0, 236510
AppSec [baseline] (176.341 ms) : 0, 176341
AppSec [candidate] (176.482 ms) : 0, 176482
Remote Config [baseline] (677.663 µs) : 0, 678
Remote Config [candidate] (661.639 µs) : 0, 662
Telemetry [baseline] (8.259 ms) : 0, 8259
Telemetry [candidate] (8.203 ms) : 0, 8203
IAST [baseline] (19.467 ms) : 0, 19467
IAST [candidate] (19.423 ms) : 0, 19423
section iast
BytebuddyAgent [baseline] (835.033 ms) : 0, 835033
BytebuddyAgent [candidate] (840.945 ms) : 0, 840945
GlobalTracer [baseline] (230.983 ms) : 0, 230983
GlobalTracer [candidate] (230.83 ms) : 0, 230830
AppSec [baseline] (58.401 ms) : 0, 58401
AppSec [candidate] (58.264 ms) : 0, 58264
Remote Config [baseline] (651.534 µs) : 0, 652
Remote Config [candidate] (660.676 µs) : 0, 661
Telemetry [baseline] (8.687 ms) : 0, 8687
Telemetry [candidate] (8.747 ms) : 0, 8747
IAST [baseline] (21.446 ms) : 0, 21446
IAST [candidate] (21.469 ms) : 0, 21469
section profiling
ProfilingAgent [baseline] (95.798 ms) : 0, 95798
ProfilingAgent [candidate] (95.566 ms) : 0, 95566
BytebuddyAgent [baseline] (705.102 ms) : 0, 705102
BytebuddyAgent [candidate] (703.834 ms) : 0, 703834
GlobalTracer [baseline] (349.726 ms) : 0, 349726
GlobalTracer [candidate] (349.243 ms) : 0, 349243
AppSec [baseline] (55.194 ms) : 0, 55194
AppSec [candidate] (54.264 ms) : 0, 54264
Remote Config [baseline] (692.926 µs) : 0, 693
Remote Config [candidate] (717.61 µs) : 0, 718
Telemetry [baseline] (8.918 ms) : 0, 8918
Telemetry [candidate] (8.894 ms) : 0, 8894
Profiling [baseline] (95.822 ms) : 0, 95822
Profiling [candidate] (95.59 ms) : 0, 95590

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.46.0-SNAPSHOT~dc06b5ec8e, baseline=1.46.0-SNAPSHOT~af5f4412a1

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.047 s) : 0, 1046800
Total [baseline] (8.643 s) : 0, 8643343
Agent [candidate] (1.04 s) : 0, 1040278
Total [candidate] (8.646 s) : 0, 8645650
section iast
Agent [baseline] (1.167 s) : 0, 1167425
Total [baseline] (9.189 s) : 0, 9188603
Agent [candidate] (1.165 s) : 0, 1165387
Total [candidate] (9.226 s) : 0, 9225750
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.172 s) : 0, 1171619
Total [baseline] (9.231 s) : 0, 9230537
Agent [candidate] (1.166 s) : 0, 1165528
Total [candidate] (9.154 s) : 0, 9154139
section iast_TELEMETRY_OFF
Agent [baseline] (1.172 s) : 0, 1171677
Total [baseline] (9.189 s) : 0, 9189297
Agent [candidate] (1.171 s) : 0, 1170500
Total [candidate] (9.221 s) : 0, 9221368

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.047 s	-
Agent	iast	1.167 s	120.625 ms (11.5%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.172 s	124.819 ms (11.9%)
Agent	iast_TELEMETRY_OFF	1.172 s	124.877 ms (11.9%)
Total	tracing	8.643 s	-
Total	iast	9.189 s	545.26 ms (6.3%)
Total	iast_HARDCODED_SECRET_DISABLED	9.231 s	587.194 ms (6.8%)
Total	iast_TELEMETRY_OFF	9.189 s	545.954 ms (6.3%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.04 s	-
Agent	iast	1.165 s	125.108 ms (12.0%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.166 s	125.25 ms (12.0%)
Agent	iast_TELEMETRY_OFF	1.171 s	130.222 ms (12.5%)
Total	tracing	8.646 s	-
Total	iast	9.226 s	580.1 ms (6.7%)
Total	iast_HARDCODED_SECRET_DISABLED	9.154 s	508.489 ms (5.9%)
Total	iast_TELEMETRY_OFF	9.221 s	575.718 ms (6.7%)

gantt
    title insecure-bank - break down per module: candidate=1.46.0-SNAPSHOT~dc06b5ec8e, baseline=1.46.0-SNAPSHOT~af5f4412a1

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (723.093 ms) : 0, 723093
BytebuddyAgent [candidate] (715.716 ms) : 0, 715716
GlobalTracer [baseline] (240.676 ms) : 0, 240676
GlobalTracer [candidate] (240.781 ms) : 0, 240781
AppSec [baseline] (56.257 ms) : 0, 56257
AppSec [candidate] (55.696 ms) : 0, 55696
Remote Config [baseline] (715.013 µs) : 0, 715
Remote Config [candidate] (718.933 µs) : 0, 719
Telemetry [baseline] (10.685 ms) : 0, 10685
Telemetry [candidate] (12.264 ms) : 0, 12264
section iast
BytebuddyAgent [baseline] (833.071 ms) : 0, 833071
BytebuddyAgent [candidate] (832.596 ms) : 0, 832596
GlobalTracer [baseline] (230.406 ms) : 0, 230406
GlobalTracer [candidate] (229.42 ms) : 0, 229420
AppSec [baseline] (58.061 ms) : 0, 58061
AppSec [candidate] (57.074 ms) : 0, 57074
IAST [baseline] (21.276 ms) : 0, 21276
IAST [candidate] (22.065 ms) : 0, 22065
Remote Config [baseline] (645.892 µs) : 0, 646
Remote Config [candidate] (644.833 µs) : 0, 645
Telemetry [baseline] (8.669 ms) : 0, 8669
Telemetry [candidate] (8.541 ms) : 0, 8541
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (834.542 ms) : 0, 834542
BytebuddyAgent [candidate] (832.791 ms) : 0, 832791
GlobalTracer [baseline] (231.762 ms) : 0, 231762
GlobalTracer [candidate] (229.369 ms) : 0, 229369
AppSec [baseline] (58.797 ms) : 0, 58797
AppSec [candidate] (57.609 ms) : 0, 57609
IAST [baseline] (21.628 ms) : 0, 21628
IAST [candidate] (21.319 ms) : 0, 21319
Remote Config [baseline] (676.297 µs) : 0, 676
Remote Config [candidate] (654.996 µs) : 0, 655
Telemetry [baseline] (8.875 ms) : 0, 8875
Telemetry [candidate] (8.7 ms) : 0, 8700
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (836.375 ms) : 0, 836375
BytebuddyAgent [candidate] (835.799 ms) : 0, 835799
GlobalTracer [baseline] (231.229 ms) : 0, 231229
GlobalTracer [candidate] (231.015 ms) : 0, 231015
AppSec [baseline] (58.427 ms) : 0, 58427
AppSec [candidate] (58.31 ms) : 0, 58310
IAST [baseline] (20.961 ms) : 0, 20961
IAST [candidate] (20.91 ms) : 0, 20910
Remote Config [baseline] (652.651 µs) : 0, 653
Remote Config [candidate] (646.631 µs) : 0, 647
Telemetry [baseline] (8.657 ms) : 0, 8657
Telemetry [candidate] (8.695 ms) : 0, 8695

Load

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nikita-tkachenko/generalize-test-skipping
git_commit_date	1737992274	1737994187
git_commit_sha	`af5f441`	`dc06b5e`
release_version	1.46.0-SNAPSHOT~af5f4412a1	1.46.0-SNAPSHOT~dc06b5ec8e

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1737996278	1737996278
ci_job_id	781850039	781850039
ci_pipeline_id	54081720	54081720
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-8wlcx2s5-project-304-concurrent-2-3d7hb9qk 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-8wlcx2s5-project-304-concurrent-2-3d7hb9qk 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.46.0-SNAPSHOT~dc06b5ec8e, baseline=1.46.0-SNAPSHOT~af5f4412a1
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.463 ms) : 1452, 1474
.   : milestone, 1463,
appsec (2.348 ms) : 2305, 2391
.   : milestone, 2348,
iast (2.102 ms) : 2047, 2156
.   : milestone, 2102,
iast_GLOBAL (2.142 ms) : 2087, 2197
.   : milestone, 2142,
profiling (1.984 ms) : 1939, 2028
.   : milestone, 1984,
tracing (1.938 ms) : 1896, 1980
.   : milestone, 1938,
section candidate
no_agent (1.466 ms) : 1455, 1478
.   : milestone, 1466,
appsec (2.356 ms) : 2313, 2399
.   : milestone, 2356,
iast (2.1 ms) : 2045, 2155
.   : milestone, 2100,
iast_GLOBAL (2.147 ms) : 2092, 2202
.   : milestone, 2147,
profiling (1.954 ms) : 1910, 1997
.   : milestone, 1954,
tracing (1.933 ms) : 1891, 1974
.   : milestone, 1933,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.463 ms [1.452 ms, 1.474 ms]	-
appsec	2.348 ms [2.305 ms, 2.391 ms]	885.443 µs (60.5%)
iast	2.102 ms [2.047 ms, 2.156 ms]	638.773 µs (43.7%)
iast_GLOBAL	2.142 ms [2.087 ms, 2.197 ms]	679.365 µs (46.4%)
profiling	1.984 ms [1.939 ms, 2.028 ms]	520.827 µs (35.6%)
tracing	1.938 ms [1.896 ms, 1.98 ms]	475.52 µs (32.5%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.466 ms [1.455 ms, 1.478 ms]	-
appsec	2.356 ms [2.313 ms, 2.399 ms]	889.872 µs (60.7%)
iast	2.1 ms [2.045 ms, 2.155 ms]	633.518 µs (43.2%)
iast_GLOBAL	2.147 ms [2.092 ms, 2.202 ms]	680.835 µs (46.4%)
profiling	1.954 ms [1.91 ms, 1.997 ms]	487.43 µs (33.2%)
tracing	1.933 ms [1.891 ms, 1.974 ms]	466.331 µs (31.8%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.46.0-SNAPSHOT~dc06b5ec8e, baseline=1.46.0-SNAPSHOT~af5f4412a1
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.439 s) : 15439000, 15439000
.   : milestone, 15439000,
appsec (14.956 s) : 14956000, 14956000
.   : milestone, 14956000,
iast (18.594 s) : 18594000, 18594000
.   : milestone, 18594000,
iast_GLOBAL (17.983 s) : 17983000, 17983000
.   : milestone, 17983000,
profiling (15.068 s) : 15068000, 15068000
.   : milestone, 15068000,
tracing (14.984 s) : 14984000, 14984000
.   : milestone, 14984000,
section candidate
no_agent (15.625 s) : 15625000, 15625000
.   : milestone, 15625000,
appsec (15.077 s) : 15077000, 15077000
.   : milestone, 15077000,
iast (18.545 s) : 18545000, 18545000
.   : milestone, 18545000,
iast_GLOBAL (17.538 s) : 17538000, 17538000
.   : milestone, 17538000,
profiling (15.02 s) : 15020000, 15020000
.   : milestone, 15020000,
tracing (14.971 s) : 14971000, 14971000
.   : milestone, 14971000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.439 s [15.439 s, 15.439 s]	-
appsec	14.956 s [14.956 s, 14.956 s]	-483.0 ms (-3.1%)
iast	18.594 s [18.594 s, 18.594 s]	3.155 s (20.4%)
iast_GLOBAL	17.983 s [17.983 s, 17.983 s]	2.544 s (16.5%)
profiling	15.068 s [15.068 s, 15.068 s]	-371.0 ms (-2.4%)
tracing	14.984 s [14.984 s, 14.984 s]	-455.0 ms (-2.9%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.625 s [15.625 s, 15.625 s]	-
appsec	15.077 s [15.077 s, 15.077 s]	-548.0 ms (-3.5%)
iast	18.545 s [18.545 s, 18.545 s]	2.92 s (18.7%)
iast_GLOBAL	17.538 s [17.538 s, 17.538 s]	1.913 s (12.2%)
profiling	15.02 s [15.02 s, 15.02 s]	-605.0 ms (-3.9%)
tracing	14.971 s [14.971 s, 14.971 s]	-654.0 ms (-4.2%)

| Package | Type | Package file | Manager | Update | Change | |---|---|---|---|---|---| | [com.google.cloud:google-cloud-datastore](https://github.com/googleapis/java-datastore) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `2.25.4` -> `2.26.0` | | [com.datadoghq:dd-trace-api](https://github.com/datadog/dd-trace-java) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `1.45.2` -> `1.46.0` | | [com.datadoghq:dd-trace-ot](https://github.com/datadog/dd-trace-java) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `1.45.2` -> `1.46.0` | | [software.amazon.awssdk:sdk-core](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.30.9` -> `2.30.10` | | [software.amazon.awssdk:dynamodb-enhanced](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.30.9` -> `2.30.10` | | [software.amazon.awssdk:dynamodb](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.30.9` -> `2.30.10` | | [software.amazon.awssdk:aws-core](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.30.9` -> `2.30.10` | | [software.amazon.awssdk:bom](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.30.9` -> `2.30.10` | | [software.amazon.awssdk:auth](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.30.9` -> `2.30.10` | --- ### Release Notes <details> <summary>googleapis/java-datastore (com.google.cloud:google-cloud-datastore)</summary> ### [`v2.26.0`](https://github.com/googleapis/java-datastore/blob/HEAD/CHANGELOG.md#2260-2025-01-29) ##### Features - Add firestoreInDatastoreMode for datastore emulator ([#1698](googleapis/java-datastore#1698)) ([50f106d](googleapis/java-datastore@50f106d)) ##### Dependencies - Update dependency com.google.cloud:sdk-platform-java-config to v3.42.0 ([#1725](googleapis/java-datastore#1725)) ([1cbaf22](googleapis/java-datastore@1cbaf22)) </details> <details> <summary>datadog/dd-trace-java (com.datadoghq:dd-trace-api)</summary> ### [`v1.46.0`](https://github.com/DataDog/dd-trace-java/releases/tag/v1.46.0): 1.46.0 ##### Breaking Changes > \[!WARNING] > jnr-unixsocket is now an external dependency of dd-trace-ot and must be included when deploying dd-trace-ot. > \[!NOTE] > The API `TracerScope.setAsync(boolean)`, used to manually control asynchronous span propagation, does no more apply to the scope instance but to the active span scope. ##### Components ##### Application Security Management (IAST) - 🐛 Fix String.replace instrumentation for IAST ([#8281](DataDog/dd-trace-java#8281) - [@Mariovido](https://github.com/Mariovido)) - ✨ Apply the standard nomenclature to the stacktrace configs ([#8244](DataDog/dd-trace-java#8244) - [@jandro996](https://github.com/jandro996)) - 🐛 Exclude false positive weak randomness ([#8232](DataDog/dd-trace-java#8232) - [@jandro996](https://github.com/jandro996)) - ✨ Propagation of translateEscapes of String class ([#8186](DataDog/dd-trace-java#8186) - [@sezen-datadog](https://github.com/sezen-datadog)) - ✨ Add security control metrics ([#8175](DataDog/dd-trace-java#8175) - [@jandro996](https://github.com/jandro996)) - ✨ Increase IAST propagation to StringBuffer setLength ([#8128](DataDog/dd-trace-java#8128) - [@Mariovido](https://github.com/Mariovido)) - ✨ Add IAST taint tracking for DB values ([#8072](DataDog/dd-trace-java#8072) - [@Mariovido](https://github.com/Mariovido)) ##### Application Security Management (WAF) - 🐛 Prevents a NPE when there is no subscriber for user events ([#8258](DataDog/dd-trace-java#8258) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) - ✨ Apply the standard nomenclature to the stacktrace configs ([#8244](DataDog/dd-trace-java#8244) - [@jandro996](https://github.com/jandro996)) - 🐛 Ensure cached subscriptions are cleared on reconfiguration via RC ([#8229](DataDog/dd-trace-java#8229) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) - ✨ Add support for session tracking in Vertx ([#8167](DataDog/dd-trace-java#8167) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) - ✨ Create span tag: \_dd.appsec.rasp.timeout ([#8269](DataDog/dd-trace-java#8269) - [@Mariovido](https://github.com/Mariovido)) ##### Build & Tooling - 🐛 Ensure shaded helpers have unique names when injected into class-loaders ([#8192](DataDog/dd-trace-java#8192) - [@mcculls](https://github.com/mcculls)) ##### Configuration at Runtime - 🐛 Remove filtering of `DD_SERVICE` and `DD_ENV` from the tracer ([#8176](DataDog/dd-trace-java#8176) - [@mhlidd](https://github.com/mhlidd)) ##### Continuous Integration Visibility - 🧹 Generalize TestRetryPolicy to TestExecutionPolicy ([#8302](DataDog/dd-trace-java#8302) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - 🧹 Parallelize CI Visibility settings requests ([#8299](DataDog/dd-trace-java#8299) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - 🧹 Generalize test retry logic ([#8289](DataDog/dd-trace-java#8289) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - 🧹 Generalize tests skipping logic ([#8288](DataDog/dd-trace-java#8288) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - 🧹 Remove skip and shouldBeSkipped methods from TestEventsHandler in favor of isSkippable ([#8286](DataDog/dd-trace-java#8286) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨⚡ Optimize Git repository information computation ([#8270](DataDog/dd-trace-java#8270) - [@dougqh](https://github.com/dougqh)) - ✨ Always request known tests from the backend ([#8268](DataDog/dd-trace-java#8268) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨ Fix NPE when trying to get retry analyzer in Test NG ([#8253](DataDog/dd-trace-java#8253) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - 🐛 Set test framework and test framework version tags atomically ([#8252](DataDog/dd-trace-java#8252) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨ Add debug logging to Android Gradle module layout logic ([#8251](DataDog/dd-trace-java#8251) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - 🐛 Fix source and destination folders computation for Android Gradle projects ([#8190](DataDog/dd-trace-java#8190) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨ Add basic Scala Weaver sbt support ([#8189](DataDog/dd-trace-java#8189) - [@daniel-mohedano](https://github.com/daniel-mohedano)) - ✨ Implement impacted tests detection ([#8188](DataDog/dd-trace-java#8188) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) ##### Data Streams Monitoring - ✨ Change hash computation for protobuf to better represent impacting changes + save proto number in schema ([#8201](DataDog/dd-trace-java#8201) - [@vandonr](https://github.com/vandonr)) ##### Database Monitoring - Add peer service tag in dbm sql commenter ([#7913](DataDog/dd-trace-java#7913) - [@jordan-wong](https://github.com/jordan-wong)) ##### Dynamic Instrumentation - ✨ Add support for SymDB to scan directories ([#8306](DataDog/dd-trace-java#8306) - [@jpbempel](https://github.com/jpbempel)) - ✨ Add SymDB report for any jar scanning failures ([#8300](DataDog/dd-trace-java#8300) - [@jpbempel](https://github.com/jpbempel)) - ✨ Use two budgets depending on type ([#8283](DataDog/dd-trace-java#8283) - [@evanchooly](https://github.com/evanchooly)) - ✨ Institute a 10 snapshot per probe per trace budget ([#8277](DataDog/dd-trace-java#8277) - [@evanchooly](https://github.com/evanchooly)) - 🐛 Avoid double snapshots for Exception Replay ([#8273](DataDog/dd-trace-java#8273) - [@jpbempel](https://github.com/jpbempel)) - ✨ Simplify code origins. Separate out snapshot generation. ([#8263](DataDog/dd-trace-java#8263) - [@evanchooly](https://github.com/evanchooly)) - ✨ Add Exception probe custom instrumentation ([#8230](DataDog/dd-trace-java#8230) - [@jpbempel](https://github.com/jpbempel)) - ✨ Enhance log probes to honor debug session tags ([#8215](DataDog/dd-trace-java#8215) - [@evanchooly](https://github.com/evanchooly)) - 🐛 Don't redact env tokens from debugger probe snapshots ([#8211](DataDog/dd-trace-java#8211) - [@watson](https://github.com/watson)) - ✨⚡ Move Trace/SpanId capture at commit time ([#8184](DataDog/dd-trace-java#8184) - [@jpbempel](https://github.com/jpbempel)) - 🐛 Capture values at entry for method probe ([#8169](DataDog/dd-trace-java#8169) - [@jpbempel](https://github.com/jpbempel)) ##### JMX fetch - 🐛 Mute JMXFetch Shutdown in progress error ([#8068](DataDog/dd-trace-java#8068) - [@ygree](https://github.com/ygree)) ##### OpenTracing - ⚠️🧹 Make jnr-unixsocket an explicit dependency of dd-trace-ot ([#8307](DataDog/dd-trace-java#8307) - [@mcculls](https://github.com/mcculls)) ##### Profiling - 🐛 Avoid unsupported API call for creating folders on windows ([#8304](DataDog/dd-trace-java#8304) - [@jbachorik](https://github.com/jbachorik)) - ✨ Tag profiles for serverless ([#8279](DataDog/dd-trace-java#8279) - [@jbachorik](https://github.com/jbachorik)) - ✨ add queue type and length to queue events ([#8242](DataDog/dd-trace-java#8242) - [@richardstartin](https://github.com/richardstartin)) - 🐛 TempLocationManager Fixes and Improvements ([#8191](DataDog/dd-trace-java#8191) - [@jbachorik](https://github.com/jbachorik)) - ✨ Bump ddprof to 1.18.0 ([#8173](DataDog/dd-trace-java#8173) - [@jbachorik](https://github.com/jbachorik)) - ✨ Report profiler initialization and configuration errors to telemetry ([#8171](DataDog/dd-trace-java#8171) - [@jbachorik](https://github.com/jbachorik)) ##### Telemetry - ✨ Add pending traces report in tracer flares ([#8053](DataDog/dd-trace-java#8053) - [@mhlidd](https://github.com/mhlidd)) ##### Testing - ✨ Test http server requests in parallel ([#8222](DataDog/dd-trace-java#8222) - [@amarziali](https://github.com/amarziali)) ##### Trace context propagation - ✨ Add non default propagator registration ([#8310](DataDog/dd-trace-java#8310) - [@PerfectSlayer](https://github.com/PerfectSlayer)) ##### Tracer core - ✨ Probe for existence of IBMSASL or ACCP security providers ([#8276](DataDog/dd-trace-java#8276) - [@mcculls](https://github.com/mcculls)) - ✨⚡ Overhead improvement to agent feedback based sampling ([#8265](DataDog/dd-trace-java#8265) - [@dougqh](https://github.com/dougqh)) - 🧹 Move async propagation API from scope to tracer ([#8231](DataDog/dd-trace-java#8231) - [@PerfectSlayer](https://github.com/PerfectSlayer)) - ✨ Introduce context propagation API ([#8161](DataDog/dd-trace-java#8161) - [@PerfectSlayer](https://github.com/PerfectSlayer)) - ✨🧪 Use env-entry to add tags per webapp deployment ([#8138](DataDog/dd-trace-java#8138) - [@amarziali](https://github.com/amarziali)) - ✨ Introduce context helpers API ([#8134](DataDog/dd-trace-java#8134) - [@PerfectSlayer](https://github.com/PerfectSlayer)) - ✨ Support IPv6 values for `DD_AGENT_HOST` and `DD_TRACE_AGENT_URL` ([#7984](DataDog/dd-trace-java#7984) - [@mhlidd](https://github.com/mhlidd)) ##### Instrumentations ##### Apache HttpComponents - 🐛 Properly finish spans and support latest apache httpclient5 ([#8272](DataDog/dd-trace-java#8272) - [@amarziali](https://github.com/amarziali)) ##### AWS Lambda instrumentation - 🐛 Properly capture lambda payloads for all handler types. ([#8264](DataDog/dd-trace-java#8264) - [@purple4reina](https://github.com/purple4reina)) ##### AWS S3 instrumentation - 💡 Create S3 instrumentation + add span pointers ([#8075](DataDog/dd-trace-java#8075) - [@nhulston](https://github.com/nhulston)) ##### AWS SDK instrumentation - 🐛 Revert "Add avoid double instrumenting lambda non-streaming handlers." ([#8247](DataDog/dd-trace-java#8247) - [@nhulston](https://github.com/nhulston)) ##### Cassandra - ✨ Allow extracting keyspace from statement result ([#8239](DataDog/dd-trace-java#8239) - [@amarziali](https://github.com/amarziali)) ##### Core Java language instrumentation - ✨ Propagation of translateEscapes of String class ([#8186](DataDog/dd-trace-java#8186) - [@sezen-datadog](https://github.com/sezen-datadog)) ##### Eclipse Vert.x instrumentation - 🐛 Fix vertx worker propagation and error handling ([#8237](DataDog/dd-trace-java#8237) - [@amarziali](https://github.com/amarziali)) - ✨ Support vertx 5 ([#8220](DataDog/dd-trace-java#8220) - [@amarziali](https://github.com/amarziali)) - ✨ Add support for session tracking in Vertx ([#8167](DataDog/dd-trace-java#8167) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) ##### Kafka instrumentation - 🐛 Prevent possible NPE calculating Kafka record header size ([#8292](DataDog/dd-trace-java#8292) - [@ygree](https://github.com/ygree)) ##### Mule instrumentation - 🐛 Fix crash using Mule with JPMS ([#8187](DataDog/dd-trace-java#8187) - [@amarziali](https://github.com/amarziali)) ##### Protocol Buffer instrumentation - ✨ Change hash computation for protobuf to better represent impacting changes + save proto number in schema ([#8201](DataDog/dd-trace-java#8201) - [@vandonr](https://github.com/vandonr)) ##### Spring instrumentation - 🐛 Preserve getQualifier from spring scheduling runnables ([#8293](DataDog/dd-trace-java#8293) - [@amarziali](https://github.com/amarziali)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - "after 6pm every weekday,before 2am every weekday" in timezone Australia/Melbourne, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://github.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). GitOrigin-RevId: bb09d47e4eed77a003f630273b4d0a84003eb899

Generalize tests skipping logic

dc06b5e

nikita-tkachenko-datadog added type: refactoring comp: ci visibility Continuous Integration Visibility labels Jan 27, 2025

daniel-mohedano approved these changes Jan 28, 2025

View reviewed changes

nikita-tkachenko-datadog marked this pull request as ready for review January 28, 2025 16:43

nikita-tkachenko-datadog requested a review from a team as a code owner January 28, 2025 16:43

nikita-tkachenko-datadog requested a review from tonyredondo January 28, 2025 16:43

nikita-tkachenko-datadog merged commit 9efa100 into master Jan 28, 2025
200 of 202 checks passed

nikita-tkachenko-datadog deleted the nikita-tkachenko/generalize-test-skipping branch January 28, 2025 16:47

github-actions bot added this to the 1.46.0 milestone Jan 28, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Generalize tests skipping logic #8288

Generalize tests skipping logic #8288

nikita-tkachenko-datadog commented Jan 27, 2025 •

edited

Loading

pr-commenter bot commented Jan 27, 2025

Generalize tests skipping logic #8288

Generalize tests skipping logic #8288

Conversation

nikita-tkachenko-datadog commented Jan 27, 2025 • edited Loading

What Does This Do

Contributor Checklist

pr-commenter bot commented Jan 27, 2025

Benchmarks

Startup

Parameters

Summary

Load

Dacapo

Parameters

Summary

nikita-tkachenko-datadog commented Jan 27, 2025 •

edited

Loading