Skip to content

Commit

Permalink
[serverless] Create EventBridge Instrumentation and Inject Trace Cont…
Browse files Browse the repository at this point in the history 
…ext (#6096)

## Summary of changes
This creates a new instrumentation for EventBridge and intercepts
`PutEvents` and `PutEventsAsync` to inject trace context. This allows
the agent to combine spans from a distributed (serverless) architecture
into a single trace.

This PR only injects trace context. I'm working on [PR
1](DataDog/datadog-agent#29414) and [PR
2](DataDog/datadog-agent#29551) to update the
Lambda extension to use this trace context to create EventBridge spans.

I am also working on a similar PR in
[dd-trace-java](DataDog/dd-trace-java#7613) and
dd-trace-go.

## Reason for change

SNS and SQS are already supported, and the tracer currently injects
trace context into message attributes fields for them. However,
EventBridge wasn't supported, and this PR aims to fix this problem.

## Implementation details

I followed the
[documentation](https://github.com/DataDog/dd-trace-dotnet/blob/master/docs/development/AutomaticInstrumentation.md)
to create an instrumentation. Much of the logic was mirrored from the
[existing
implementation](https://github.com/DataDog/dd-trace-dotnet/tree/master/tracer/src/Datadog.Trace/ClrProfiler/AutoInstrumentation/AWS/SNS)
of SNS, since EventBridge and SNS are extremely similar.

Overall, AWS's EventBridge API is lacking some features, so we have to
do some hacky solutions.

- SNS and SQS call their custom input field messageAttributes, and
EventBridge calls it detail
- Unlike SNS and SQS, the detail field is given as a raw string.
Therefore, we have to manually modify the detail string using
StringBuilder.
- The agent has no reliable way of getting the start time of the
EventBridge span, so the tracer has to put the current time into
`detail[_datadog]` under the header `x-datadog-start-time`
- The EventBridge API has no way of getting the EventBridge bus name, so
the tracer has to put the bus name (which is used to create the span
resource name) into `detail[_datadog]` under the header
`x-datadog-resource-name`


## Test coverage

I added system tests for SNS/SQS:
DataDog/system-tests#3204

I added [unit
tests](d05eb4c)
and [integration
tests](5ccd8b7).

Unit tests can be ran with:
```
cd tracer
dotnet test ./test/Datadog.Trace.ClrProfiler.Managed.Tests
```

Integration tests can be ran with these commands:
```
cd tracer

# Start docker localstock
docker run --rm -it -p 4566:4566 -p 4571:4571 -e SERVICES=events localstack/localstack

# Run integation tests
./build.sh BuildAndRunOSxIntegrationTests -buildConfiguration Debug -framework net6.0 -Filter AwsEventBridgeTests -SampleName Samples.AWS.EventBridge
```

I also did manual testing:
<img width="505" alt="Screenshot 2024-09-30 at 11 00 47 AM"
src="https://github.com/user-attachments/assets/bdf5d516-8b46-4138-ac25-c45d1822dc56">

## Other details

There are lots of diffs and files changed. I recommend reviewers to
review the PR commit by commit. All the autogenerated files were added
in a single commit, which should make the review process less
overwhelming.

<!-- ⚠️ Note: where possible, please obtain 2 approvals prior to
merging. Unless CODEOWNERS specifies otherwise, for external teams it is
typically best to have one review from a team member, and one review
from apm-dotnet. Trivial changes do not require 2 reviews. -->

---------

Co-authored-by: Steven Bouwkamp <steven.bouwkamp@datadoghq.com>
  • Loading branch information
@nhulston @bouwkast
nhulston and bouwkast authored Oct 15, 2024
1 parent aa5cedc commit 8dc8808
Show file tree
Hide file tree
Showing 71 changed files with 3,413 additions and 351 deletions.
6 changes: 6 additions & 0 deletions Datadog.Trace.sln
View file
Original file line number Diff line number Diff line change
Expand Up @@ -589,6 +589,8 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Generated", "Generated", "{
tracer\build\PackageVersionsLatestSpecific.g.props = tracer\build\PackageVersionsLatestSpecific.g.props
EndProjectSection
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Samples.AWS.EventBridge", "tracer\test\test-applications\integrations\Samples.AWS.EventBridge\Samples.AWS.EventBridge.csproj", "{D6155F26-8245-4B66-8944-79C3DF9F9DA3}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "AssemblyLoadContextResolve", "tracer\test\test-applications\regression\AssemblyLoadContextResolve\AssemblyLoadContextResolve.csproj", "{8B1AF6A7-DD41-4347-B637-90C23D69B50E}"
EndProject
Global
Expand Down Expand Up @@ -1411,6 +1413,10 @@ Global
{2CA0D70C-DFC1-458A-871B-328AB6E87E3A}.Debug|Any CPU.Build.0 = Debug|Any CPU
{2CA0D70C-DFC1-458A-871B-328AB6E87E3A}.Release|Any CPU.ActiveCfg = Release|Any CPU
{2CA0D70C-DFC1-458A-871B-328AB6E87E3A}.Release|Any CPU.Build.0 = Release|Any CPU
{D6155F26-8245-4B66-8944-79C3DF9F9DA3}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{D6155F26-8245-4B66-8944-79C3DF9F9DA3}.Debug|Any CPU.Build.0 = Debug|Any CPU
{D6155F26-8245-4B66-8944-79C3DF9F9DA3}.Release|Any CPU.ActiveCfg = Release|Any CPU
{D6155F26-8245-4B66-8944-79C3DF9F9DA3}.Release|Any CPU.Build.0 = Release|Any CPU
{8B1AF6A7-DD41-4347-B637-90C23D69B50E}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{8B1AF6A7-DD41-4347-B637-90C23D69B50E}.Debug|Any CPU.Build.0 = Debug|Any CPU
{8B1AF6A7-DD41-4347-B637-90C23D69B50E}.Release|Any CPU.ActiveCfg = Release|Any CPU
Expand Down
6 changes: 3 additions & 3 deletions docker-compose.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ services:
localstack_arm64:
image: localstack/localstack
environment:
- SERVICES=sns,sqs,kinesis,dynamodb
- SERVICES=sns,sqs,kinesis,dynamodb,events
- DEBUG=1
- DATA_DIR=/tmp/localstack/data
- DEFAULT_REGION=us-east-1
Expand Down Expand Up @@ -93,7 +93,7 @@ services:
localstack:
image: localstack/localstack
environment:
- SERVICES=sns,sqs,kinesis,dynamodb
- SERVICES=sns,sqs,kinesis,dynamodb,events
- DEBUG=1
- DATA_DIR=/tmp/localstack/data
- DEFAULT_REGION=us-east-1
Expand Down Expand Up @@ -1112,7 +1112,7 @@ services:
localstack_osx_arm64:
image: localstack/localstack
environment:
- SERVICES=sns,sqs,kinesis,dynamodb
- SERVICES=sns,sqs,kinesis,dynamodb,events
- DEBUG=1
- DATA_DIR=/tmp/localstack/data
- DEFAULT_REGION=us-east-1
Expand Down
18 changes: 18 additions & 0 deletions tracer/build/PackageVersionsGeneratorDefinitions.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,24 @@
}
]
},
{
"IntegrationName": "AwsEventBridge",
"SampleProjectName": "Samples.AWS.EventBridge",
"NugetPackageSearchName": "AWSSDK.EventBridge",
"MinVersion": "3.3.100",
"MaxVersionExclusive": "4.0.0",
"SpecificVersions": [
"3.3.*",
"3.5.*",
"3.7.*"
],
"VersionConditions": [
{
"MaxVersionExclusive": "3.3.0",
"IncludeOnlyTargetFrameworks": ["net462"]
}
]
},
{
"IntegrationName": "MongoDB",
"SampleProjectName": "Samples.MongoDB",
Expand Down
42 changes: 42 additions & 0 deletions tracer/build/PackageVersionsLatestMajors.g.props
View file
Original file line number Diff line number Diff line change
Expand Up @@ -222,6 +222,48 @@ NOTE: This code was generated by the GeneratePackageVersions tool. To safely
<RequiresDockerDependency>All</RequiresDockerDependency>
<SampleName>Samples.AWS.SimpleNotificationService</SampleName>
</PackageVersionSample>
<PackageVersionSample Include="test\test-applications\integrations\Samples.AWS.EventBridge\Samples.AWS.EventBridge.csproj">
<Properties>ApiVersion=3.7.401.25;RestoreRecursive=false;BuildProjectReferences=false</Properties>
<TargetFramework>netcoreapp2.1</TargetFramework>
<RequiresDockerDependency>All</RequiresDockerDependency>
<SampleName>Samples.AWS.EventBridge</SampleName>
</PackageVersionSample>
<PackageVersionSample Include="test\test-applications\integrations\Samples.AWS.EventBridge\Samples.AWS.EventBridge.csproj">
<Properties>ApiVersion=3.7.401.25;RestoreRecursive=false;BuildProjectReferences=false</Properties>
<TargetFramework>netcoreapp3.0</TargetFramework>
<RequiresDockerDependency>All</RequiresDockerDependency>
<SampleName>Samples.AWS.EventBridge</SampleName>
</PackageVersionSample>
<PackageVersionSample Include="test\test-applications\integrations\Samples.AWS.EventBridge\Samples.AWS.EventBridge.csproj">
<Properties>ApiVersion=3.7.401.25;RestoreRecursive=false;BuildProjectReferences=false</Properties>
<TargetFramework>netcoreapp3.1</TargetFramework>
<RequiresDockerDependency>All</RequiresDockerDependency>
<SampleName>Samples.AWS.EventBridge</SampleName>
</PackageVersionSample>
<PackageVersionSample Include="test\test-applications\integrations\Samples.AWS.EventBridge\Samples.AWS.EventBridge.csproj">
<Properties>ApiVersion=3.7.401.25;RestoreRecursive=false;BuildProjectReferences=false</Properties>
<TargetFramework>net5.0</TargetFramework>
<RequiresDockerDependency>All</RequiresDockerDependency>
<SampleName>Samples.AWS.EventBridge</SampleName>
</PackageVersionSample>
<PackageVersionSample Include="test\test-applications\integrations\Samples.AWS.EventBridge\Samples.AWS.EventBridge.csproj">
<Properties>ApiVersion=3.7.401.25;RestoreRecursive=false;BuildProjectReferences=false</Properties>
<TargetFramework>net6.0</TargetFramework>
<RequiresDockerDependency>All</RequiresDockerDependency>
<SampleName>Samples.AWS.EventBridge</SampleName>
</PackageVersionSample>
<PackageVersionSample Include="test\test-applications\integrations\Samples.AWS.EventBridge\Samples.AWS.EventBridge.csproj">
<Properties>ApiVersion=3.7.401.25;RestoreRecursive=false;BuildProjectReferences=false</Properties>
<TargetFramework>net7.0</TargetFramework>
<RequiresDockerDependency>All</RequiresDockerDependency>
<SampleName>Samples.AWS.EventBridge</SampleName>
</PackageVersionSample>
<PackageVersionSample Include="test\test-applications\integrations\Samples.AWS.EventBridge\Samples.AWS.EventBridge.csproj">
<Properties>ApiVersion=3.7.401.25;RestoreRecursive=false;BuildProjectReferences=false</Properties>
<TargetFramework>net8.0</TargetFramework>
<RequiresDockerDependency>All</RequiresDockerDependency>
<SampleName>Samples.AWS.EventBridge</SampleName>
</PackageVersionSample>
<PackageVersionSample Include="test\test-applications\integrations\Samples.MongoDB\Samples.MongoDB.csproj">
<Properties>ApiVersion=2.29.0;RestoreRecursive=false;BuildProjectReferences=false</Properties>
<TargetFramework>net462</TargetFramework>
Expand Down
126 changes: 126 additions & 0 deletions tracer/build/PackageVersionsLatestMinors.g.props
View file
Original file line number Diff line number Diff line change
Expand Up @@ -756,6 +756,132 @@ NOTE: This code was generated by the GeneratePackageVersions tool. To safely
<RequiresDockerDependency>All</RequiresDockerDependency>
<SampleName>Samples.AWS.SimpleNotificationService</SampleName>
</PackageVersionSample>
<PackageVersionSample Include="test\test-applications\integrations\Samples.AWS.EventBridge\Samples.AWS.EventBridge.csproj">
<Properties>ApiVersion=3.3.102.16;RestoreRecursive=false;BuildProjectReferences=false</Properties>
<TargetFramework>netcoreapp2.1</TargetFramework>
<RequiresDockerDependency>All</RequiresDockerDependency>
<SampleName>Samples.AWS.EventBridge</SampleName>
</PackageVersionSample>
<PackageVersionSample Include="test\test-applications\integrations\Samples.AWS.EventBridge\Samples.AWS.EventBridge.csproj">
<Properties>ApiVersion=3.5.6.5;RestoreRecursive=false;BuildProjectReferences=false</Properties>
<TargetFramework>netcoreapp2.1</TargetFramework>
<RequiresDockerDependency>All</RequiresDockerDependency>
<SampleName>Samples.AWS.EventBridge</SampleName>
</PackageVersionSample>
<PackageVersionSample Include="test\test-applications\integrations\Samples.AWS.EventBridge\Samples.AWS.EventBridge.csproj">
<Properties>ApiVersion=3.7.401.25;RestoreRecursive=false;BuildProjectReferences=false</Properties>
<TargetFramework>netcoreapp2.1</TargetFramework>
<RequiresDockerDependency>All</RequiresDockerDependency>
<SampleName>Samples.AWS.EventBridge</SampleName>
</PackageVersionSample>
<PackageVersionSample Include="test\test-applications\integrations\Samples.AWS.EventBridge\Samples.AWS.EventBridge.csproj">
<Properties>ApiVersion=3.3.102.16;RestoreRecursive=false;BuildProjectReferences=false</Properties>
<TargetFramework>netcoreapp3.0</TargetFramework>
<RequiresDockerDependency>All</RequiresDockerDependency>
<SampleName>Samples.AWS.EventBridge</SampleName>
</PackageVersionSample>
<PackageVersionSample Include="test\test-applications\integrations\Samples.AWS.EventBridge\Samples.AWS.EventBridge.csproj">
<Properties>ApiVersion=3.5.6.5;RestoreRecursive=false;BuildProjectReferences=false</Properties>
<TargetFramework>netcoreapp3.0</TargetFramework>
<RequiresDockerDependency>All</RequiresDockerDependency>
<SampleName>Samples.AWS.EventBridge</SampleName>
</PackageVersionSample>
<PackageVersionSample Include="test\test-applications\integrations\Samples.AWS.EventBridge\Samples.AWS.EventBridge.csproj">
<Properties>ApiVersion=3.7.401.25;RestoreRecursive=false;BuildProjectReferences=false</Properties>
<TargetFramework>netcoreapp3.0</TargetFramework>
<RequiresDockerDependency>All</RequiresDockerDependency>
<SampleName>Samples.AWS.EventBridge</SampleName>
</PackageVersionSample>
<PackageVersionSample Include="test\test-applications\integrations\Samples.AWS.EventBridge\Samples.AWS.EventBridge.csproj">
<Properties>ApiVersion=3.3.102.16;RestoreRecursive=false;BuildProjectReferences=false</Properties>
<TargetFramework>netcoreapp3.1</TargetFramework>
<RequiresDockerDependency>All</RequiresDockerDependency>
<SampleName>Samples.AWS.EventBridge</SampleName>
</PackageVersionSample>
<PackageVersionSample Include="test\test-applications\integrations\Samples.AWS.EventBridge\Samples.AWS.EventBridge.csproj">
<Properties>ApiVersion=3.5.6.5;RestoreRecursive=false;BuildProjectReferences=false</Properties>
<TargetFramework>netcoreapp3.1</TargetFramework>
<RequiresDockerDependency>All</RequiresDockerDependency>
<SampleName>Samples.AWS.EventBridge</SampleName>
</PackageVersionSample>
<PackageVersionSample Include="test\test-applications\integrations\Samples.AWS.EventBridge\Samples.AWS.EventBridge.csproj">
<Properties>ApiVersion=3.7.401.25;RestoreRecursive=false;BuildProjectReferences=false</Properties>
<TargetFramework>netcoreapp3.1</TargetFramework>
<RequiresDockerDependency>All</RequiresDockerDependency>
<SampleName>Samples.AWS.EventBridge</SampleName>
</PackageVersionSample>
<PackageVersionSample Include="test\test-applications\integrations\Samples.AWS.EventBridge\Samples.AWS.EventBridge.csproj">
<Properties>ApiVersion=3.3.102.16;RestoreRecursive=false;BuildProjectReferences=false</Properties>
<TargetFramework>net5.0</TargetFramework>
<RequiresDockerDependency>All</RequiresDockerDependency>
<SampleName>Samples.AWS.EventBridge</SampleName>
</PackageVersionSample>
<PackageVersionSample Include="test\test-applications\integrations\Samples.AWS.EventBridge\Samples.AWS.EventBridge.csproj">
<Properties>ApiVersion=3.5.6.5;RestoreRecursive=false;BuildProjectReferences=false</Properties>
<TargetFramework>net5.0</TargetFramework>
<RequiresDockerDependency>All</RequiresDockerDependency>
<SampleName>Samples.AWS.EventBridge</SampleName>
</PackageVersionSample>
<PackageVersionSample Include="test\test-applications\integrations\Samples.AWS.EventBridge\Samples.AWS.EventBridge.csproj">
<Properties>ApiVersion=3.7.401.25;RestoreRecursive=false;BuildProjectReferences=false</Properties>
<TargetFramework>net5.0</TargetFramework>
<RequiresDockerDependency>All</RequiresDockerDependency>
<SampleName>Samples.AWS.EventBridge</SampleName>
</PackageVersionSample>
<PackageVersionSample Include="test\test-applications\integrations\Samples.AWS.EventBridge\Samples.AWS.EventBridge.csproj">
<Properties>ApiVersion=3.3.102.16;RestoreRecursive=false;BuildProjectReferences=false</Properties>
<TargetFramework>net6.0</TargetFramework>
<RequiresDockerDependency>All</RequiresDockerDependency>
<SampleName>Samples.AWS.EventBridge</SampleName>
</PackageVersionSample>
<PackageVersionSample Include="test\test-applications\integrations\Samples.AWS.EventBridge\Samples.AWS.EventBridge.csproj">
<Properties>ApiVersion=3.5.6.5;RestoreRecursive=false;BuildProjectReferences=false</Properties>
<TargetFramework>net6.0</TargetFramework>
<RequiresDockerDependency>All</RequiresDockerDependency>
<SampleName>Samples.AWS.EventBridge</SampleName>
</PackageVersionSample>
<PackageVersionSample Include="test\test-applications\integrations\Samples.AWS.EventBridge\Samples.AWS.EventBridge.csproj">
<Properties>ApiVersion=3.7.401.25;RestoreRecursive=false;BuildProjectReferences=false</Properties>
<TargetFramework>net6.0</TargetFramework>
<RequiresDockerDependency>All</RequiresDockerDependency>
<SampleName>Samples.AWS.EventBridge</SampleName>
</PackageVersionSample>
<PackageVersionSample Include="test\test-applications\integrations\Samples.AWS.EventBridge\Samples.AWS.EventBridge.csproj">
<Properties>ApiVersion=3.3.102.16;RestoreRecursive=false;BuildProjectReferences=false</Properties>
<TargetFramework>net7.0</TargetFramework>
<RequiresDockerDependency>All</RequiresDockerDependency>
<SampleName>Samples.AWS.EventBridge</SampleName>
</PackageVersionSample>
<PackageVersionSample Include="test\test-applications\integrations\Samples.AWS.EventBridge\Samples.AWS.EventBridge.csproj">
<Properties>ApiVersion=3.5.6.5;RestoreRecursive=false;BuildProjectReferences=false</Properties>
<TargetFramework>net7.0</TargetFramework>
<RequiresDockerDependency>All</RequiresDockerDependency>
<SampleName>Samples.AWS.EventBridge</SampleName>
</PackageVersionSample>
<PackageVersionSample Include="test\test-applications\integrations\Samples.AWS.EventBridge\Samples.AWS.EventBridge.csproj">
<Properties>ApiVersion=3.7.401.25;RestoreRecursive=false;BuildProjectReferences=false</Properties>
<TargetFramework>net7.0</TargetFramework>
<RequiresDockerDependency>All</RequiresDockerDependency>
<SampleName>Samples.AWS.EventBridge</SampleName>
</PackageVersionSample>
<PackageVersionSample Include="test\test-applications\integrations\Samples.AWS.EventBridge\Samples.AWS.EventBridge.csproj">
<Properties>ApiVersion=3.3.102.16;RestoreRecursive=false;BuildProjectReferences=false</Properties>
<TargetFramework>net8.0</TargetFramework>
<RequiresDockerDependency>All</RequiresDockerDependency>
<SampleName>Samples.AWS.EventBridge</SampleName>
</PackageVersionSample>
<PackageVersionSample Include="test\test-applications\integrations\Samples.AWS.EventBridge\Samples.AWS.EventBridge.csproj">
<Properties>ApiVersion=3.5.6.5;RestoreRecursive=false;BuildProjectReferences=false</Properties>
<TargetFramework>net8.0</TargetFramework>
<RequiresDockerDependency>All</RequiresDockerDependency>
<SampleName>Samples.AWS.EventBridge</SampleName>
</PackageVersionSample>
<PackageVersionSample Include="test\test-applications\integrations\Samples.AWS.EventBridge\Samples.AWS.EventBridge.csproj">
<Properties>ApiVersion=3.7.401.25;RestoreRecursive=false;BuildProjectReferences=false</Properties>
<TargetFramework>net8.0</TargetFramework>
<RequiresDockerDependency>All</RequiresDockerDependency>
<SampleName>Samples.AWS.EventBridge</SampleName>
</PackageVersionSample>
<PackageVersionSample Include="test\test-applications\integrations\Samples.MongoDB\Samples.MongoDB.csproj">
<Properties>ApiVersion=2.0.2;RestoreRecursive=false;BuildProjectReferences=false</Properties>
<TargetFramework>net462</TargetFramework>
Expand Down
Loading

0 comments on commit 8dc8808

Please sign in to comment.