-
Notifications
You must be signed in to change notification settings - Fork 291
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Collect WAF headers on user sdk events #7014
Collect WAF headers on user sdk events #7014
Conversation
04a5d6e
to
46284b6
Compare
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 49 metrics, 14 unstable metrics. Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.35.0-SNAPSHOT~67ae62b445, baseline=1.35.0-SNAPSHOT~440212338d
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.084 s) : 0, 1083572
Total [baseline] (8.541 s) : 0, 8541393
Agent [candidate] (1.077 s) : 0, 1077005
Total [candidate] (8.554 s) : 0, 8554388
section iast
Agent [baseline] (1.202 s) : 0, 1202372
Total [baseline] (9.014 s) : 0, 9013942
Agent [candidate] (1.202 s) : 0, 1202432
Total [candidate] (9.015 s) : 0, 9014872
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.201 s) : 0, 1201045
Total [baseline] (9.019 s) : 0, 9019356
Agent [candidate] (1.2 s) : 0, 1200209
Total [candidate] (9.01 s) : 0, 9010385
section iast_TELEMETRY_OFF
Agent [baseline] (1.211 s) : 0, 1210907
Total [baseline] (9.078 s) : 0, 9077760
Agent [candidate] (1.21 s) : 0, 1209846
Total [candidate] (9.011 s) : 0, 9011486
gantt
title insecure-bank - break down per module: candidate=1.35.0-SNAPSHOT~67ae62b445, baseline=1.35.0-SNAPSHOT~440212338d
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (678.847 ms) : 0, 678847
BytebuddyAgent [candidate] (673.577 ms) : 0, 673577
GlobalTracer [baseline] (312.206 ms) : 0, 312206
GlobalTracer [candidate] (310.997 ms) : 0, 310997
AppSec [baseline] (49.627 ms) : 0, 49627
AppSec [candidate] (49.724 ms) : 0, 49724
Remote Config [baseline] (653.996 µs) : 0, 654
Remote Config [candidate] (663.074 µs) : 0, 663
Telemetry [baseline] (7.561 ms) : 0, 7561
Telemetry [candidate] (7.633 ms) : 0, 7633
section iast
BytebuddyAgent [baseline] (795.826 ms) : 0, 795826
BytebuddyAgent [candidate] (795.886 ms) : 0, 795886
GlobalTracer [baseline] (290.65 ms) : 0, 290650
GlobalTracer [candidate] (290.695 ms) : 0, 290695
AppSec [baseline] (50.043 ms) : 0, 50043
AppSec [candidate] (50.729 ms) : 0, 50729
IAST [baseline] (23.657 ms) : 0, 23657
IAST [candidate] (23.555 ms) : 0, 23555
Remote Config [baseline] (609.208 µs) : 0, 609
Remote Config [candidate] (586.375 µs) : 0, 586
Telemetry [baseline] (7.375 ms) : 0, 7375
Telemetry [candidate] (6.584 ms) : 0, 6584
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (795.149 ms) : 0, 795149
BytebuddyAgent [candidate] (793.877 ms) : 0, 793877
GlobalTracer [baseline] (290.203 ms) : 0, 290203
GlobalTracer [candidate] (290.099 ms) : 0, 290099
AppSec [baseline] (50.825 ms) : 0, 50825
AppSec [candidate] (50.712 ms) : 0, 50712
IAST [baseline] (23.38 ms) : 0, 23380
IAST [candidate] (24.079 ms) : 0, 24079
Remote Config [baseline] (584.552 µs) : 0, 585
Remote Config [candidate] (585.333 µs) : 0, 585
Telemetry [baseline] (6.551 ms) : 0, 6551
Telemetry [candidate] (6.59 ms) : 0, 6590
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (802.092 ms) : 0, 802092
BytebuddyAgent [candidate] (799.561 ms) : 0, 799561
GlobalTracer [baseline] (292.87 ms) : 0, 292870
GlobalTracer [candidate] (293.278 ms) : 0, 293278
AppSec [baseline] (50.352 ms) : 0, 50352
AppSec [candidate] (49.866 ms) : 0, 49866
IAST [baseline] (22.164 ms) : 0, 22164
IAST [candidate] (25.205 ms) : 0, 25205
Remote Config [baseline] (669.922 µs) : 0, 670
Remote Config [candidate] (669.522 µs) : 0, 670
Telemetry [baseline] (8.14 ms) : 0, 8140
Telemetry [candidate] (6.503 ms) : 0, 6503
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.35.0-SNAPSHOT~67ae62b445, baseline=1.35.0-SNAPSHOT~440212338d
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.083 s) : 0, 1083318
Total [baseline] (10.421 s) : 0, 10420740
Agent [candidate] (1.085 s) : 0, 1084517
Total [candidate] (10.457 s) : 0, 10457376
section appsec
Agent [baseline] (1.202 s) : 0, 1202464
Total [baseline] (10.531 s) : 0, 10531448
Agent [candidate] (1.202 s) : 0, 1202314
Total [candidate] (10.52 s) : 0, 10520215
section iast
Agent [baseline] (1.205 s) : 0, 1204823
Total [baseline] (10.861 s) : 0, 10861181
Agent [candidate] (1.211 s) : 0, 1211413
Total [candidate] (10.732 s) : 0, 10731545
section profiling
Agent [baseline] (1.271 s) : 0, 1271358
Total [baseline] (10.542 s) : 0, 10542229
Agent [candidate] (1.27 s) : 0, 1270287
Total [candidate] (10.61 s) : 0, 10609557
gantt
title petclinic - break down per module: candidate=1.35.0-SNAPSHOT~67ae62b445, baseline=1.35.0-SNAPSHOT~440212338d
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (677.819 ms) : 0, 677819
BytebuddyAgent [candidate] (678.542 ms) : 0, 678542
GlobalTracer [baseline] (312.933 ms) : 0, 312933
GlobalTracer [candidate] (312.795 ms) : 0, 312795
AppSec [baseline] (49.736 ms) : 0, 49736
AppSec [candidate] (50.185 ms) : 0, 50185
Remote Config [baseline] (659.128 µs) : 0, 659
Remote Config [candidate] (665.906 µs) : 0, 666
Telemetry [baseline] (7.574 ms) : 0, 7574
Telemetry [candidate] (7.694 ms) : 0, 7694
section appsec
BytebuddyAgent [baseline] (701.335 ms) : 0, 701335
BytebuddyAgent [candidate] (700.911 ms) : 0, 700911
GlobalTracer [baseline] (294.385 ms) : 0, 294385
GlobalTracer [candidate] (295.148 ms) : 0, 295148
AppSec [baseline] (152.924 ms) : 0, 152924
AppSec [candidate] (153.033 ms) : 0, 153033
Remote Config [baseline] (623.919 µs) : 0, 624
Remote Config [candidate] (622.965 µs) : 0, 623
Telemetry [baseline] (9.085 ms) : 0, 9085
Telemetry [candidate] (8.493 ms) : 0, 8493
IAST [baseline] (18.908 ms) : 0, 18908
IAST [candidate] (18.874 ms) : 0, 18874
section iast
BytebuddyAgent [baseline] (797.953 ms) : 0, 797953
BytebuddyAgent [candidate] (801.629 ms) : 0, 801629
GlobalTracer [baseline] (291.351 ms) : 0, 291351
GlobalTracer [candidate] (292.838 ms) : 0, 292838
AppSec [baseline] (51.357 ms) : 0, 51357
AppSec [candidate] (52.022 ms) : 0, 52022
Remote Config [baseline] (591.395 µs) : 0, 591
Remote Config [candidate] (612.183 µs) : 0, 612
Telemetry [baseline] (6.577 ms) : 0, 6577
Telemetry [candidate] (6.687 ms) : 0, 6687
IAST [baseline] (22.57 ms) : 0, 22570
IAST [candidate] (23.014 ms) : 0, 23014
section profiling
BytebuddyAgent [baseline] (678.644 ms) : 0, 678644
BytebuddyAgent [candidate] (677.489 ms) : 0, 677489
GlobalTracer [baseline] (381.479 ms) : 0, 381479
GlobalTracer [candidate] (381.939 ms) : 0, 381939
AppSec [baseline] (50.43 ms) : 0, 50430
AppSec [candidate] (50.267 ms) : 0, 50267
Remote Config [baseline] (714.237 µs) : 0, 714
Remote Config [candidate] (697.097 µs) : 0, 697
Telemetry [baseline] (7.464 ms) : 0, 7464
Telemetry [candidate] (7.459 ms) : 0, 7459
ProfilingAgent [baseline] (96.039 ms) : 0, 96039
ProfilingAgent [candidate] (95.778 ms) : 0, 95778
Profiling [baseline] (96.063 ms) : 0, 96063
Profiling [candidate] (95.802 ms) : 0, 95802
LoadParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 17 unstable metrics. Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.35.0-SNAPSHOT~67ae62b445, baseline=1.35.0-SNAPSHOT~440212338d
dateFormat X
axisFormat %s
section baseline
no_agent (1.344 ms) : 1325, 1363
. : milestone, 1344,
appsec (1.722 ms) : 1698, 1746
. : milestone, 1722,
appsec_no_iast (1.737 ms) : 1713, 1761
. : milestone, 1737,
iast (1.462 ms) : 1440, 1484
. : milestone, 1462,
profiling (1.514 ms) : 1489, 1540
. : milestone, 1514,
tracing (1.464 ms) : 1440, 1487
. : milestone, 1464,
section candidate
no_agent (1.341 ms) : 1322, 1359
. : milestone, 1341,
appsec (1.729 ms) : 1705, 1753
. : milestone, 1729,
appsec_no_iast (1.725 ms) : 1700, 1749
. : milestone, 1725,
iast (1.474 ms) : 1451, 1498
. : milestone, 1474,
profiling (1.512 ms) : 1486, 1537
. : milestone, 1512,
tracing (1.464 ms) : 1440, 1488
. : milestone, 1464,
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.35.0-SNAPSHOT~67ae62b445, baseline=1.35.0-SNAPSHOT~440212338d
dateFormat X
axisFormat %s
section baseline
no_agent (372.178 µs) : 352, 392
. : milestone, 372,
iast (487.249 µs) : 466, 509
. : milestone, 487,
iast_FULL (550.103 µs) : 529, 571
. : milestone, 550,
iast_GLOBAL (505.838 µs) : 484, 528
. : milestone, 506,
iast_HARDCODED_SECRET_DISABLED (475.702 µs) : 455, 497
. : milestone, 476,
iast_INACTIVE (450.847 µs) : 430, 472
. : milestone, 451,
iast_TELEMETRY_OFF (476.364 µs) : 455, 498
. : milestone, 476,
tracing (440.969 µs) : 420, 461
. : milestone, 441,
section candidate
no_agent (371.095 µs) : 351, 391
. : milestone, 371,
iast (480.124 µs) : 457, 504
. : milestone, 480,
iast_FULL (557.917 µs) : 537, 579
. : milestone, 558,
iast_GLOBAL (504.622 µs) : 484, 525
. : milestone, 505,
iast_HARDCODED_SECRET_DISABLED (477.243 µs) : 456, 498
. : milestone, 477,
iast_INACTIVE (449.454 µs) : 428, 471
. : milestone, 449,
iast_TELEMETRY_OFF (476.503 µs) : 455, 498
. : milestone, 477,
tracing (440.083 µs) : 420, 461
. : milestone, 440,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics. Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.35.0-SNAPSHOT~67ae62b445, baseline=1.35.0-SNAPSHOT~440212338d
dateFormat X
axisFormat %s
section baseline
no_agent (1.456 ms) : 1444, 1467
. : milestone, 1456,
appsec (2.202 ms) : 2168, 2236
. : milestone, 2202,
iast (1.944 ms) : 1904, 1985
. : milestone, 1944,
iast_GLOBAL (1.967 ms) : 1927, 2007
. : milestone, 1967,
profiling (1.831 ms) : 1799, 1864
. : milestone, 1831,
tracing (1.824 ms) : 1792, 1856
. : milestone, 1824,
section candidate
no_agent (1.455 ms) : 1443, 1466
. : milestone, 1455,
appsec (2.199 ms) : 2165, 2234
. : milestone, 2199,
iast (1.954 ms) : 1913, 1996
. : milestone, 1954,
iast_GLOBAL (1.995 ms) : 1955, 2035
. : milestone, 1995,
profiling (1.834 ms) : 1801, 1866
. : milestone, 1834,
tracing (1.817 ms) : 1785, 1849
. : milestone, 1817,
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.35.0-SNAPSHOT~67ae62b445, baseline=1.35.0-SNAPSHOT~440212338d
dateFormat X
axisFormat %s
section baseline
no_agent (14.754 s) : 14754000, 14754000
. : milestone, 14754000,
appsec (15.148 s) : 15148000, 15148000
. : milestone, 15148000,
iast (18.672 s) : 18672000, 18672000
. : milestone, 18672000,
iast_GLOBAL (17.902 s) : 17902000, 17902000
. : milestone, 17902000,
profiling (15.358 s) : 15358000, 15358000
. : milestone, 15358000,
tracing (15.135 s) : 15135000, 15135000
. : milestone, 15135000,
section candidate
no_agent (14.99 s) : 14990000, 14990000
. : milestone, 14990000,
appsec (14.937 s) : 14937000, 14937000
. : milestone, 14937000,
iast (18.993 s) : 18993000, 18993000
. : milestone, 18993000,
iast_GLOBAL (17.772 s) : 17772000, 17772000
. : milestone, 17772000,
profiling (15.254 s) : 15254000, 15254000
. : milestone, 15254000,
tracing (15.015 s) : 15015000, 15015000
. : milestone, 15015000,
|
ab74adc
to
4f60289
Compare
28cc48e
to
462c276
Compare
dd-java-agent/appsec/src/main/java/com/datadog/appsec/gateway/GatewayBridge.java
Show resolved
Hide resolved
462c276
to
da68ce8
Compare
private static boolean isTruthy(final Object value) { | ||
return value != null && "true".equalsIgnoreCase(value.toString()); | ||
} | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since isTruthy()
is used only in hasUserTrackingEvent()
, should we merge them? Or you think will be more cases to use isTruthy
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Merged, we can always extract it again if needed
* @param key key of the tag | ||
* @param sanitize indicates is key need to be sanitized | ||
*/ | ||
Object getTagCurrent(String key, boolean sanitize); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have doubts about getters in TraceSegment. If I recall correctly, it was designed only to write to isolate writing to local root span
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are already other getters in the TraceSegment
like Object getDataCurrent(String key)
and they seem to be working fine. I did test this PR with the related tests for this functionality and it seems to be working fine.
Are there any other mechanism to access the local root span and extract the tags?
56328af
to
1345bcc
Compare
1345bcc
to
34c9fa4
Compare
@@ -62,6 +62,11 @@ public class GatewayBridge { | |||
private static final Pattern QUERY_PARAM_SPLITTER = Pattern.compile("&"); | |||
private static final Map<String, List<String>> EMPTY_QUERY_PARAMS = Collections.emptyMap(); | |||
|
|||
/** User tracking tags that will force the collection of request headers */ | |||
private static final List<String> USER_TRACKING_TAGS = |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
minor: since this is only used internally in a foreach you could make it a simple String[]
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
changed! thanks for the input
34c9fa4
to
67ae62b
Compare
What Does This Do
Collects all WAF headers whenever a user tracking event is found in the span.
Motivation
WAF related headers are very useful while investigating ATO (account take-over) campaigns, this PR ensures that the backend have the required information to deal with them.
Additional Notes
Jira ticket: APPSEC-53088