An exporter for OpenTelemetry that allows the user to send the collected traces to AWS X-Ray.
This exporter allows the user to send collected traces to AWS X-Ray, a service that helps with the monitoring and troubleshooting of microservices applications. It collects data about the requests that your application serves and provides tools you can use to view, filter, and gain insights into that data to identify issues and opportunities for optimization.
If you'd like to begin sending traces to XRay and use OpenTelemetry, you need to use the Lambda layer to send traces to XRay.
https://github.com/open-telemetry/opentelemetry-lambda
This Lambda layer needs to be built and there is no simple way to run this locally. Of course, you could also run the OpenTelemetry collector locally
https://github.com/open-telemetry/opentelemetry-collector-contrib
As you can see, for simple setups, this becomes complex. When things don't work as expected, you need to begin debugging the collector using the sparse logs it spews.
To get around this, this project reimplements the core logic found in the "contrib" version of the OpenTelemetry Collector.
Barring a few odd quirks, this project has been meticulously engineered to main compatibility with XRay and ensures feature parity with the AWS Exporter in the OpenTelemetry Exporter.
Install using NPM by using the following command:
npm install --save @mridang/exporter-xrayTip
If you're deploying your project to AWS Lambda, you can reduce your bundle
size by saving the XRay client as a peer dependency. Since this package
relies on the @aws-sdk/client-xrat transitive dependency, it would automatically
be installed, but, on AWS Lambda, the entirety of the AWS SDK v3 is preinstalled
and you do not need to include this transtive dependency. This transitive
dependency can be excluded by simply installing it as a peer dependency.
npm install --save-peer @aws-sdk/client-xrayFirst, set up the necessary dependencies and instantiate the AWS X-Ray exporter:
const { AWSXRayPropagator } = require('@opentelemetry/propagator-aws-xray');
const { AWSXRayIdGenerator } = require('@opentelemetry/id-generator-aws-xray');
const { XraySpanExporter } = require('@mridang/exporter-xray');
// Configure the OpenTelemetry SDK to use AWS X-Ray for context propagation and ID generation
const propagator = new AWSXRayPropagator();
const idGenerator = new AWSXRayIdGenerator();
const exporter = new XraySpanExporter();
// Register the exporter and start tracing
tracer.addSpanProcessor(new BatchSpanProcessor(exporter));When writing a custom exporter for OpenTelemetry in Node.js to export spans to
AWS X-Ray, it is crucial to use the AWSXRayPropagator and AWSXRayIdGenerator
from the @opentelemetry/propagator-aws-xray and @opentelemetry/id-generator-aws-xray
packages, respectively. Here’s why these components are necessary and what
could happen if they aren’t used:
-
Trace Context Propagation:
- AWSXRayPropagator: This propagator ensures that trace context is correctly propagated across different services. It translates the OpenTelemetry trace context to the format expected by AWS X-Ray. Without this, your traces would lack the necessary context to be linked correctly across different services, making it difficult to track a request’s path through a distributed system source source.
- AWSXRayIdGenerator: AWS X-Ray requires a specific format for trace IDs,
which includes a timestamp and a unique identifier. The
AWSXRayIdGeneratorgenerates IDs in this format, ensuring compatibility with X-Ray’s requirements. Using a different ID generator may result in trace IDs that X-Ray cannot recognize, leading to errors or ignored traces source source.
-
Integration and Compatibility:
- Both components are designed to work seamlessly with AWS X-Ray, ensuring that the traces collected by OpenTelemetry are correctly understood and processed by X-Ray. They handle specific details like the structure of trace IDs and the propagation of context headers, which are essential for maintaining the integrity of the trace data source.
-
Loss of Trace Continuity: If you do not use the
AWSXRayPropagator, your application might fail to propagate trace context correctly. This can result in incomplete traces, where the links between different spans are broken. This makes it challenging to get a complete view of the application's performance and trace requests across services source source. -
Trace Rejection: Without the
AWSXRayIdGenerator, the trace IDs generated might not meet X-Ray’s format requirements. X-Ray might reject these traces, causing a loss of valuable tracing information. This can lead to gaps in your monitoring and make it harder to diagnose issues within your application source.
By ensuring that these components are used, you enable proper trace propagation and ID generation, which are critical for accurate and effective monitoring with AWS X-Ray. This results in a more robust observability setup, allowing for comprehensive tracking and debugging of distributed systems.
You can inject the xRayClient and other dependencies into the
XraySpanExporter to customize its behavior:
const {
DefaultIdParser,
DefaultCauseParser,
DefaultHttpParser,
} = require('your-parser-modules');
// Instantiate the parsers
const idParser = new DefaultIdParser();
const causeParser = new DefaultCauseParser();
const httpParser = new DefaultHttpParser();
// Instantiate the custom exporter with injected dependencies
const exporter = new XraySpanExporter(
xRayClient,
idParser,
causeParser,
httpParser,
);The exporter supports two approaches for emitting trace segments:
The SDK-Based Emitter uses the AWS X-Ray SDK to send trace segments and is recommended for environments outside AWS Lambda. This is slower as it uses a TCP-based transport but is easier to set up.
The UDP Daemon Emitter sends trace segments to the X-Ray daemon
using UDP and is recommended for AWS Lambda environments where the
daemon is typically running. By default, this emitter emits segments
over UDP to the daemon running on localhost at port 2000, but you can
switch this by using the AWS_XRAY_DAEMON_ADDRESS environment variable.
https://docs.aws.amazon.com/xray/latest/devguide/xray-daemon.html
The exporter defaults to using the UDPDaemonSegmentEmitter when
running on AWS Lambda, and the SDKBasedSegmentEmitter otherwise.
This is determined automatically based on the presence of the
AWS_LAMBDA_FUNCTION_NAME environment variable. No additional setup is
needed as the daemon is already running on Lambda environments.
Please visit the AWS X-Ray console to view your traces: https://console.aws.amazon.com/xray/home
-
When sending SQL segments there is no support for sending these fields as they aren't added:
database_version– The version number of the database engine.driver_version– The version number of the database driverpreparation- A flag indicating whether query or statement
This is because these fields aren't part of the OpenTelemetry Semantic Conventions and therefore is never set as a part of the automatic instrumentation e.g. in the MySQL instrumentation.
-
There doesn't seem to be any support for AWS AppRunner. This means that when the origin is reported in Xray, there is no way to denote that the instrumentation was running on AppRunner. Even OpenTelemetry internally has no mention of AppRunner but there are some references to it which means that may be incubating
-
There is no documentation around span links and therefore, while this feature has been implemented, it may or may not work. There was code in the original collector to manage span links, but I don't see any mention of this any of the AWS SDKs. An comment has been added on the original pull-request that introduced this feature.
-
Not all outbound HTTP requests show up in a similar fashion. This differs between different HTTP clients e.g.
axios,undiciandfetchand is due to the fact the libraries that auto-instrument these transports, don't add the same attributes. Tweaking this is outside the scope of this project and it is recommended that you fork and patch thehttpand thefetchinstrumentations to resolve this.
- For more information on OpenTelemetry, visit: https://opentelemetry.io/
- For more about OpenTelemetry JavaScript: https://github.com/open-telemetry/opentelemetry-js
- For AWS X-Ray, visit: https://aws.amazon.com/xray/
If you have suggestions for how this app could be improved, or want to report a bug, open an issue - we'd love all and any contributions.
Apache License 2.0 © 2024 Mridang Agarwalla