[serverless] Add DynamoDB Span Pointers

[nhulston]

Large PR, but 1000+ lines are tests. Only ~250 lines are actual code from this PR. All these tests are necessary to test all the different possible cases. Please review commit by commit to make the review process easier.

What does this PR do?

Adds span pointers in DynamoDB for putItem, updateItem, deleteItem, transactWriteItems, and batchWriteItem requests.

For more info on the details of how span pointers work for each of these operations, see https://github.com/DataDog/dd-span-pointer-rules/blob/main/AWS/DynamoDB/Item/README.md.

Span pointers are similar to Span Links, but for cases when it is impossible to pass the Trace ID and Span ID between the spans that need to be linked.

When the calculated hashes for the upstream and downstream lambdas match, the Datadog backend will automatically link the two traces together.

When clicking on the linked span, a new tab opens linking to the downstream Lambda function that was triggered by this Dynamo object update.

Motivation

This feature already exists in Python, and I'm working on adding it to all other runtimes (Node, .NET, Java, Golang).

Additional Notes

• This handles the 'upstream' case, for Lambdas that make Dynmo requests. I will have another PR in datadog-lambda-js to handle the 'downstream' case, for Lambdas that are triggered by Dynamo updates.
• My changes in packages/dd-trace/src/util.js are intentionally there because it is a public API, since datadog-lambda-js will reuse the exported fields.

Testing Guidelines

Easy: Checkout this span, enable the feature flag, and you will see that it's pointing to the downstream Lambda.

Thorough testing: Run this Lambda function with the event payload

{
  "shouldUseV2": false,
  "twoKeyTable": false,
  "shouldPutItem": false,
  "shouldUpdateItem": false,
  "shouldDeleteItem": false,
  "shouldBatchWriteItems": false,
  "shouldTransactWriteItems": false
}

and change one of the bools to true. Also, try both with AWS SDK v2 and v3, and try with twoKeyTable vs oneKeyTable. Enable the span pointers feature flag, and check Datadog to ensure that the spans are properly linked.

I also added tests:

# Unit tests
yarn test packages/dd-trace/test/util.spec.js

# Integration tests
docker run -d -p 4566:4566 localstack/localstack
PLUGINS=aws-sdk SERVICES=localstack DD_DATA_STREAMS_ENABLED=true yarn test:plugins:ci 

[github-actions]

Overall package size

Self size: 8.14 MB
Deduped: 94.65 MB
No deduping: 94.98 MB

Dependency sizes

| name | version | self size | total size | |------|---------|-----------|------------| | @datadog/libdatadog | 0.2.2 | 29.27 MB | 29.27 MB | | @datadog/native-appsec | 8.3.0 | 19.37 MB | 19.38 MB | | @datadog/native-iast-taint-tracking | 3.2.0 | 13.9 MB | 13.91 MB | | @datadog/pprof | 5.4.1 | 9.76 MB | 10.13 MB | | protobufjs | 7.2.5 | 2.77 MB | 5.16 MB | | @datadog/native-iast-rewriter | 2.5.0 | 2.51 MB | 2.65 MB | | @opentelemetry/core | 1.14.0 | 872.87 kB | 1.47 MB | | @datadog/native-metrics | 3.0.1 | 1.06 MB | 1.46 MB | | @opentelemetry/api | 1.8.0 | 1.21 MB | 1.21 MB | | import-in-the-middle | 1.11.2 | 112.74 kB | 826.22 kB | | msgpack-lite | 0.1.26 | 201.16 kB | 281.59 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | lru-cache | 7.18.3 | 133.92 kB | 133.92 kB | | pprof-format | 2.1.0 | 111.69 kB | 111.69 kB | | @datadog/sketches-js | 2.1.0 | 109.9 kB | 109.9 kB | | semver | 7.6.3 | 95.82 kB | 95.82 kB | | lodash.sortby | 4.7.0 | 75.76 kB | 75.76 kB | | ignore | 5.3.1 | 51.46 kB | 51.46 kB | | int64-buffer | 0.1.10 | 49.18 kB | 49.18 kB | | shell-quote | 1.8.1 | 44.96 kB | 44.96 kB | | istanbul-lib-coverage | 3.2.0 | 29.34 kB | 29.34 kB | | rfdc | 1.3.1 | 25.21 kB | 25.21 kB | | @isaacs/ttlcache | 1.4.1 | 25.2 kB | 25.2 kB | | tlhunter-sorted-set | 0.1.0 | 24.94 kB | 24.94 kB | | limiter | 1.1.5 | 23.17 kB | 23.17 kB | | dc-polyfill | 0.1.4 | 23.1 kB | 23.1 kB | | retry | 0.13.1 | 18.85 kB | 18.85 kB | | jest-docblock | 29.7.0 | 8.99 kB | 12.76 kB | | crypto-randomuuid | 1.0.0 | 11.18 kB | 11.18 kB | | koalas | 1.0.2 | 6.47 kB | 6.47 kB | | path-to-regexp | 0.1.10 | 6.38 kB | 6.38 kB | | module-details-from-path | 1.0.3 | 4.47 kB | 4.47 kB |

_{🤖 This report was automatically generated by heaviest-objects-in-the-universe}

[pr-commenter]

Benchmarks

Benchmark execution time: 2024-11-22 22:51:48

Comparing candidate commit a65dd20 in PR branch nicholas.hulston/dynamodb-span-pointers with baseline commit a41951c in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 264 metrics, 2 unstable metrics.

[nhulston]

I had to export encodeValue for unit tests, but it's not used in any files; it's just a helper function for extractPrimaryKeys. Is this fine or is there something better practice I can do?

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 88.54%. Comparing base (61c5a32) to head (06558ff).
Report is 24 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff             @@
##           master    #4912       +/-   ##
===========================================
+ Coverage   69.93%   88.54%   +18.60%     
===========================================
  Files         120      119        -1     
  Lines        4025     4216      +191     
===========================================
+ Hits         2815     3733      +918     
+ Misses       1210      483      -727     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

---

🚨 Try these New Features:

• Flaky Tests Detection - Detect and resolve failed and flaky tests
• JS Bundle Analysis - Avoid shipping oversized bundles