Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v4.47.0 #4735

Merged
merged 60 commits into from
Oct 1, 2024
Merged

v4.47.0 #4735

merged 60 commits into from
Oct 1, 2024

Conversation

juan-fernandez
Copy link
Collaborator

@juan-fernandez juan-fernandez commented Sep 30, 2024

Fixes

Features

Improvements

Excluded from Release notes

bengl and others added 30 commits September 30, 2024 10:54
Co-authored-by: Jordi Bertran de Balanda <jordi.bertran@datadoghq.com>
…4645)

* Add DSM support for KafkaJS batch consume operation
Remote Config updates coming from the Agent needs to be acknowledged. This PR
changes the existing behavior by optionally allowing the acknowledgement to be
async. This is achieved by:

- Changing the way you register products from using the `EventEmitter` API to a
  custom API that only allows a single handler per product (event emitters can
  have more than one listener, which is problematic as we can't support more
  than one of the listeners to acknowledge the config). The new API is as
  follows:
  - `rc.setProductHandler(product, handler)` (prevously `rc.on(product,
    handler)`)
  - `rc.removeProductHandler(product)` (prevously `rc.off(product, handler)`)

- The new product handler is called similar to the old event listener, with the
  following exception: A new optional 4th argument is supplied called `ack`.
  This is a callback which can be called (sync or async) either without any
  arguments (to set the state to `ACKNOWLEDGED`) or with a single error argument
  (to set the state to `ERROR`).
  - If the handler function signature takes less than 4 arguments or doesn't use
    the rest operator (`...args`), and...
    - ...the handler doesn't return a `Promise`: The state is set to
      `ACKNOWLEDGED` right away without the handler having to do anything
      (existing behavior)
    - ...the handler returns a `Promise`, we wait until the promise is resolved
      or rejected and set the state accordingly (new behavior)
  - If the handler function signature takes 4 or more arguments or use the rest
    operator (`...args`): The state is left as `UNACKNOWLEDGED` until the `ack`
    callback is called (new behavior)
  - In any case, the state is still set to `ERROR` if the handler throws
    (existing behavior)

The `RemoteConfigManager` is still an `EventEmitter` however because the
`kPreUpdate` symbol is still being emitted.
* Check WAF context disposal before run

* Tests for disposal check and warn log

* Move the check to the beginning of run()

* Merge tests
---------

Co-authored-by: simon-id <simon.id@datadoghq.com>
Indicating that a backoff has expired isn't enough information to
diagnose what's causing the backoff in the first place, so rather than
creating a new error object, this change just passes the last error
along.
Prepare the Node.js tracer to be used with the Dynamic Instrumentation product
(DI).

Probes are added as breakpoints using the V8 Inspector Protocol. The inspector
runs in a worker thread and will pause the main thread temporarily while
gathering the required information related to the probe (as of this commit no
information is gathered, but this is a required step once we start to gather
the local state snapshot).

DI features included in this commit:
- Support for line based log probes
- Support for loading probes via Remote Configuration
- Support for sending probe status to the debugger-backend
- Support for sending log data to debugger-backend

Please note, that with this commit it's still not possible to use DI with the
Node.js tracer as support for using the Node.js tracer with DI has not yet been
enabled in the Datadog UI.
juan-fernandez and others added 9 commits September 30, 2024 11:02
Two files have been modified, which was added to the repo 5 years ago
and never touched again:

The `.vscode/launch.json` file was outdated and didn't actually work, as
it expected `tdd` to exist in the path.

Read more about `launch.json` attributes here:
https://code.visualstudio.com/Docs/editor/debugging#_launchjson-attributes

The `.vscode/settings.json` file was outdated and used a config property
no longer supported. The proper equivalent setting would have been:

    "debug.javascript.autoAttachFilter": "always"

But I don't think we want that as it will auto-attach a debugger to
every script you run in the console. I've opted for removing this
altogether.

Read more about auto-attaching here:
https://code.visualstudio.com/docs/nodejs/nodejs-debugging#_auto-attach
* use DD_VERSION as version for tracer computed stats if available

* test version for tracer computed stats
Separating Rhea into its own CI test job.
If the payload being sent to the agent contained any two-or-more-byte
characters, the calculation would be incorrect because `str.length`
returns the number of characters and not the number of bytes in the
string.

If the payload was JSON, this would make it malformed and the receiver
would not be able to parse it (as it would be missing one or more bytes
from the end of the payload).
Copy link

github-actions bot commented Sep 30, 2024

Overall package size

Self size: 7.21 MB
Deduped: 64.46 MB
No deduping: 64.74 MB

Dependency sizes | name | version | self size | total size | |------|---------|-----------|------------| | @datadog/native-appsec | 8.1.1 | 18.67 MB | 18.68 MB | | @datadog/native-iast-taint-tracking | 3.1.0 | 12.27 MB | 12.28 MB | | @datadog/pprof | 5.3.0 | 9.85 MB | 10.22 MB | | protobufjs | 7.2.5 | 2.77 MB | 7.01 MB | | @datadog/native-iast-rewriter | 2.4.1 | 2.14 MB | 2.23 MB | | @opentelemetry/core | 1.14.0 | 872.87 kB | 1.47 MB | | @datadog/native-metrics | 2.0.0 | 898.77 kB | 1.3 MB | | @opentelemetry/api | 1.8.0 | 1.21 MB | 1.21 MB | | jsonpath-plus | 9.0.0 | 580.4 kB | 1.03 MB | | import-in-the-middle | 1.11.2 | 112.74 kB | 826.22 kB | | msgpack-lite | 0.1.26 | 201.16 kB | 281.59 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | pprof-format | 2.1.0 | 111.69 kB | 111.69 kB | | @datadog/sketches-js | 2.1.0 | 109.9 kB | 109.9 kB | | semver | 7.6.3 | 95.82 kB | 95.82 kB | | lodash.sortby | 4.7.0 | 75.76 kB | 75.76 kB | | lru-cache | 7.14.0 | 74.95 kB | 74.95 kB | | ignore | 5.3.1 | 51.46 kB | 51.46 kB | | int64-buffer | 0.1.10 | 49.18 kB | 49.18 kB | | shell-quote | 1.8.1 | 44.96 kB | 44.96 kB | | istanbul-lib-coverage | 3.2.0 | 29.34 kB | 29.34 kB | | rfdc | 1.3.1 | 25.21 kB | 25.21 kB | | tlhunter-sorted-set | 0.1.0 | 24.94 kB | 24.94 kB | | limiter | 1.1.5 | 23.17 kB | 23.17 kB | | dc-polyfill | 0.1.4 | 23.1 kB | 23.1 kB | | retry | 0.13.1 | 18.85 kB | 18.85 kB | | jest-docblock | 29.7.0 | 8.99 kB | 12.76 kB | | crypto-randomuuid | 1.0.0 | 11.18 kB | 11.18 kB | | koalas | 1.0.2 | 6.47 kB | 6.47 kB | | path-to-regexp | 0.1.10 | 6.38 kB | 6.38 kB | | module-details-from-path | 1.0.3 | 4.47 kB | 4.47 kB |

🤖 This report was automatically generated by heaviest-objects-in-the-universe

@juan-fernandez juan-fernandez marked this pull request as ready for review September 30, 2024 10:10
@juan-fernandez juan-fernandez requested a review from a team as a code owner September 30, 2024 10:10
Co-authored-by: Ugaitz Urien <ugaitz.urien@datadoghq.com>
@pr-commenter
Copy link

pr-commenter bot commented Sep 30, 2024

Benchmarks

Benchmark execution time: 2024-10-01 08:14:08

Comparing candidate commit 2c3ce03 in PR branch v4.47.0-proposal with baseline commit c1fe1db in branch v4.x.

Found 0 performance improvements and 7 performance regressions! Performance is the same for 769 metrics, 22 unstable metrics.

scenario:log-skip-log-16

  • 🟥 cpu_user_time [+24.103ms; +33.266ms] or [+7.028%; +9.699%]
  • 🟥 execution_time [+24.216ms; +24.925ms] or [+6.323%; +6.508%]

scenario:log-with-debug-16

  • 🟥 execution_time [+24.499ms; +25.261ms] or [+6.401%; +6.600%]

scenario:log-with-error-16

  • 🟥 cpu_user_time [+19.991ms; +28.938ms] or [+5.821%; +8.426%]
  • 🟥 execution_time [+24.299ms; +25.099ms] or [+6.345%; +6.553%]

scenario:log-without-log-16

  • 🟥 execution_time [+24.755ms; +26.099ms] or [+6.595%; +6.954%]

scenario:scope-manager-async_hooks-18

  • 🟥 cpu_user_time [+3.989ms; +10.134ms] or [+6.115%; +15.534%]

Copy link

codecov bot commented Sep 30, 2024

Codecov Report

Attention: Patch coverage is 46.82081% with 276 lines in your changes missing coverage. Please review.

Please upload report for BASE (v4.x@c1fe1db). Learn more about missing BASE report.

Files with missing lines Patch % Lines
packages/dd-trace/src/appsec/rasp/index.js 14.00% 43 Missing ⚠️
...ackages/dd-trace/src/profiling/profilers/events.js 2.38% 41 Missing ⚠️
...kages/dd-trace/src/appsec/remote_config/manager.js 16.66% 35 Missing ⚠️
packages/dd-trace/src/appsec/rasp/sql_injection.js 19.04% 34 Missing ⚠️
packages/dd-trace/src/debugger/index.js 19.51% 33 Missing ⚠️
packages/dd-trace/src/appsec/rasp/utils.js 29.62% 19 Missing ⚠️
packages/dd-trace/src/appsec/rasp/ssrf.js 35.29% 11 Missing ⚠️
...ackages/dd-trace/src/appsec/remote_config/index.js 8.33% 11 Missing ⚠️
packages/dd-trace/src/datastreams/processor.js 21.42% 11 Missing ⚠️
...ges/dd-trace/src/encode/agentless-ci-visibility.js 50.00% 9 Missing ⚠️
... and 10 more
Additional details and impacted files
@@           Coverage Diff           @@
##             v4.x    #4735   +/-   ##
=======================================
  Coverage        ?   65.36%           
=======================================
  Files           ?      280           
  Lines           ?    12137           
  Branches        ?       33           
=======================================
  Hits            ?     7933           
  Misses          ?     4204           
  Partials        ?        0           
Flag Coverage Δ
65.36% <46.82%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

uurien
uurien previously approved these changes Sep 30, 2024
wconti27
wconti27 previously approved these changes Sep 30, 2024
wconti27 and others added 2 commits October 1, 2024 09:59
…vailable (#4741)

* fix dsm checkpointing when no parent context is available when consuming
@juan-fernandez juan-fernandez merged commit b42c17c into v4.x Oct 1, 2024
198 checks passed
@juan-fernandez juan-fernandez deleted the v4.47.0-proposal branch October 1, 2024 08:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.