feat: add tracing sub-package based on Kubo

docs/tracing.md

@@ -0,0 +1,143 @@
+# Tracing
+
+Tracing across the stack follows, as much as possible, the [Open Telemetry]
+specifications. Configuration environment variables are specified in the
+[OpenTelemetry Environment Variable Specification].
+
+We use the [opentelemtry-go] package, which currently does not have default support
+for the `OTEL_TRACES_EXPORTER` environment variables. Therefore, we provide some
+helper functions under [`boxo/tracing`](../tracing/) to support these.
+
+In this document, we document the quirks of our custom support for the `OTEL_TRACES_EXPORTER`,
+as well as examples on how to use tracing, create traceable headers, and how
+to use the Jaeger UI. The [Gateway examples](../examples/gateway/) fully support Tracing.
+
+- [Environment Variables](#environment-variables)
+  - [`OTEL_TRACES_EXPORTER`](#otel_traces_exporter)
+  - [`OTLP Exporter`](#otlp-exporter)
+  - [`Jaeger Exporter`](#jaeger-exporter)
+  - [`Zipkin Exporter`](#zipkin-exporter)
+  - [`File Exporter`](#file-exporter)
+  - [`OTEL_PROPAGATORS`](#otel_propagators)
+- [Using Jaeger UI](#using-jaeger-ui)
+- [Generate `traceparent` Header](#generate-traceparent-header)
+
+## Environment Variables
+
+For advanced configurations, such as ratio-based sampling, please see also the
+[OpenTelemetry Environment Variable Specification].
+
+### `OTEL_TRACES_EXPORTER`
+
+Specifies the exporters to use as a comma-separated string. Each exporter has a
+set of additional environment variables used to configure it. The following values
+are supported:
+
+- `otlp`
+- `jaeger`
+- `zipkin`
+- `stdout`
+- `file` -- appends traces to a JSON file on the filesystem
+
+Default: `""` (no exporters)
+
+### `OTLP Exporter`
+
+Unless specified in this section, the OTLP exporter uses the environment variables
+documented in [OpenTelemetry Protocol Exporter].
+
+#### `OTEL_EXPORTER_OTLP_PROTOCOL`
+Specifies the OTLP protocol to use, which is one of:
+
+- `grpc`
+- `http/protobuf`
+
+Default: `"grpc"`
+
+### `Jaeger Exporter`
+
+See [Jaeger Exporter](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/sdk-environment-variables.md#jaeger-exporter).
+
+### `Zipkin Exporter`
+
+See [Zipkin Exporter](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/sdk-environment-variables.md#zipkin-exporter).
+
+### `File Exporter`
+
+#### `OTEL_EXPORTER_FILE_PATH`
+
+Specifies the filesystem path for the JSON file.
+
+Default: `"$PWD/traces.json"`
+
+### `OTEL_PROPAGATORS`
+
+See [General SDK Configuration](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/sdk-environment-variables.md#general-sdk-configuration).
+
+## Using Jaeger UI
+
+One can use the `jaegertracing/all-in-one` Docker image to run a full Jaeger stack
+and configure the gateway examples, or Kubo, to publish traces to it. Here, in an
+ephemeral container:
+
+```console
+$ docker run --rm -it --name jaeger \
+    -e COLLECTOR_ZIPKIN_HOST_PORT=:9411 \
+    -p 5775:5775/udp \
+    -p 6831:6831/udp \
+    -p 6832:6832/udp \
+    -p 5778:5778 \
+    -p 16686:16686 \
+    -p 14268:14268 \
+    -p 14269:14269 \
+    -p 14250:14250 \
+    -p 9411:9411 \
+    jaegertracing/all-in-one
+```
+
+Then, in other terminal, start the CAR Gateway example (or Kubo), with Jaeger
+tracing enabled:
+
+```
+$ OTEL_TRACES_EXPORTER=jaeger ./car-gateway -c data.car -p 8040
+```
+
+Finally, the [Jaeger UI] is available at http://localhost:16686.
+
+## Generate `traceparent` Header
+
+If you want to trace a specific request and want to have its tracing ID, you can
+generate a `Traceparent` header. According to the [Trace Context] specification,
+the header is formed as follows:
+
+> ```
+> version-format   = trace-id "-" parent-id "-" trace-flags
+> trace-id         = 32HEXDIGLC  ; 16 bytes array identifier. All zeroes forbidden
+> parent-id        = 16HEXDIGLC  ; 8 bytes array identifier. All zeroes forbidden
+> trace-flags      = 2HEXDIGLC   ; 8 bit flags. Currently, only one bit is used. See below for details
+> ```
+
+To generate a valid `Traceparent` header value, the following script can be used:
+
+```bash
+version="00" # fixed in spec at 00
+trace_id="$(cat /dev/urandom | tr -dc 'a-f0-9' | fold -w 32 | head -n 1)"
+parent_id="00$(cat /dev/urandom | tr -dc 'a-f0-9' | fold -w 14 | head -n 1)"
+tace_flag="01"   # sampled
+traceparent="$version-$trace_id-$parent_id-$tace_flag"
+echo $traceparent
+```
+
+**NOTE**: the `tr` command behaves differently on macOS. You may want to install
+the GNU `tr` (`gtr`) and use it instead.
+
+Then, the value can be passed onto the request with `curl -H "Traceparent: $traceparent" URL`.
+If using Jaeger, you can now search by the trace with ID `$trace_id` and see
+the complete trace of this request.
+
+[Open Telemetry]: https://opentelemetry.io/
+[opentelemetry-go]: https://github.com/open-telemetry/opentelemetry-go
+[Trace Context]: https://www.w3.org/TR/trace-context
+[OpenTelemetry Environment Variable Specification]: https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/sdk-environment-variables.md
+[OpenTelemetry Protocol Exporter]: https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/protocol/exporter.md
+[Jaeger UI]: https://github.com/jaegertracing/jaeger-ui

examples/gateway/car/main.go

@@ -18,17 +18,19 @@ import (
 )
 
 func main() {
+	ctx, cancel := context.WithCancel(context.Background())
+	defer cancel()
+
 	carFilePtr := flag.String("c", "", "path to CAR file to back this gateway from")
 	port := flag.Int("p", 8040, "port to run this gateway from")
 	flag.Parse()
 
 	// Setups up tracing. This is optional and only required if the implementer
 	// wants to be able to enable tracing.
-	tp, err := common.SetupTracing("CAR Gateway Example")
+	tp, err := common.SetupTracing(ctx, "CAR Gateway Example")
 	if err != nil {
 		log.Fatal(err)
 	}
-	ctx := context.Background()
 	defer (func() { _ = tp.Shutdown(ctx) })()
 
 	// Sets up a block service based on the CAR file.

examples/gateway/common/tracing.go

@@ -1,41 +1,27 @@
 package common
 
 import (
-	"net/http"
+	"context"
 
-	"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
+	"github.com/ipfs/boxo/tracing"
 	"go.opentelemetry.io/contrib/propagators/autoprop"
 	"go.opentelemetry.io/otel"
 	"go.opentelemetry.io/otel/sdk/resource"
 	"go.opentelemetry.io/otel/sdk/trace"
 	semconv "go.opentelemetry.io/otel/semconv/v1.4.0"
 )
 
-// SetupTracing setups Open Telemetry tracing with some defaults that will enable
-// tracing in this system. Please note that in a real-case scenario this would
-// look slightly different and include more configurations.
-func SetupTracing(serviceName string) (*trace.TracerProvider, error) {
-	// Creates the resources to create a new tracing provider.
-	r, err := resource.Merge(
-		resource.Default(),
-		resource.NewSchemaless(
-			semconv.ServiceNameKey.String(serviceName),
-		),
-	)
+// SetupTracing sets up the tracing based on the OTEL_* environment variables,
+// and the provided service name. It returns a trace.TracerProvider.
+func SetupTracing(ctx context.Context, serviceName string) (*trace.TracerProvider, error) {
+	tp, err := NewTracerProvider(ctx, serviceName)
 	if err != nil {
 		return nil, err
 	}
 
-	// Creates a new tracing provider. In a real-case scenario, this trace provider
-	// will likely be configured with exporters in order to be able to access
-	// the tracing information. https://opentelemetry.io/docs/instrumentation/go/exporters/
-	tp := trace.NewTracerProvider(
-		trace.WithResource(r),
-	)
-
 	// Sets the default trace provider for this process. If this is not done, tracing
 	// will not be enabled. Please note that this will apply to the entire process
-	// as it is set as the default tracer.
+	// as it is set as the default tracer, as per OTel recommendations.
 	otel.SetTracerProvider(tp)
 
 	// Configures the default propagators used by the Open Telemetry library. By
@@ -48,12 +34,28 @@ func SetupTracing(serviceName string) (*trace.TracerProvider, error) {
 	return tp, nil
 }
 
-// NewClient creates a new HTTP Client wraped with the OTel transport. This will
-// ensure correct propagation of tracing headers across multiple HTTP requests when
-// Open Telemetry is configured. Please note that NewClient will use the default
-// global trace provider and propagators. Therefore, SetupTracing must be called first.
-func NewClient() *http.Client {
-	return &http.Client{
-		Transport: otelhttp.NewTransport(http.DefaultTransport),
+// NewTracerProvider creates and configures a TracerProvider.
+func NewTracerProvider(ctx context.Context, serviceName string) (*trace.TracerProvider, error) {
+	exporters, err := tracing.NewSpanExporters(ctx)
+	if err != nil {
+		return nil, err
+	}
+
+	options := []trace.TracerProviderOption{}
+
+	for _, exporter := range exporters {
+		options = append(options, trace.WithBatcher(exporter))
+	}
+
+	r, err := resource.Merge(
+		resource.Default(),
+		resource.NewSchemaless(
+			semconv.ServiceNameKey.String(serviceName),
+		),
+	)
+	if err != nil {
+		return nil, err
 	}
+	options = append(options, trace.WithResource(r))
+	return trace.NewTracerProvider(options...), nil
 }

examples/gateway/proxy/blockstore.go

@@ -6,10 +6,10 @@ import (
 	"io"
 	"net/http"
 
-	"github.com/ipfs/boxo/examples/gateway/common"
 	"github.com/ipfs/boxo/exchange"
 	blocks "github.com/ipfs/go-block-format"
 	"github.com/ipfs/go-cid"
+	"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
 )
 
 type proxyExchange struct {
@@ -19,7 +19,9 @@ type proxyExchange struct {
 
 func newProxyExchange(gatewayURL string, client *http.Client) exchange.Interface {
 	if client == nil {
-		client = common.NewClient()
+		client = &http.Client{
+			Transport: otelhttp.NewTransport(http.DefaultTransport),
+		}
 	}
 
 	return &proxyExchange{

examples/gateway/proxy/main.go

@@ -16,17 +16,19 @@ import (
 )
 
 func main() {
+	ctx, cancel := context.WithCancel(context.Background())
+	defer cancel()
+
 	gatewayUrlPtr := flag.String("g", "", "gateway to proxy to")
 	port := flag.Int("p", 8040, "port to run this gateway from")
 	flag.Parse()
 
 	// Setups up tracing. This is optional and only required if the implementer
 	// wants to be able to enable tracing.
-	tp, err := common.SetupTracing("Proxy Gateway Example")
+	tp, err := common.SetupTracing(ctx, "CAR Gateway Example")
 	if err != nil {
 		log.Fatal(err)
 	}
-	ctx := context.Background()
 	defer (func() { _ = tp.Shutdown(ctx) })()
 
 	// Sets up a blockstore to hold the blocks we request from the gateway

examples/gateway/proxy/main_test.go

@@ -94,11 +94,12 @@ func TestTraceContext(t *testing.T) {
 		traceFlags    = "00"
 	)
 
-	// Creating a trace provider and registering it will make OTel enable tracing.
-	tp, err := common.SetupTracing("Proxy Test")
+	ctx, cancel := context.WithCancel(context.Background())
+	defer cancel()
+
+	tp, err := common.SetupTracing(ctx, "Proxy Test")
 	assert.Nil(t, err)
-	ctx := context.Background()
-	t.Cleanup(func() { _ = tp.Shutdown(ctx) })
+	defer (func() { _ = tp.Shutdown(ctx) })()
 
 	t.Run("Re-use Traceparent Trace ID Of Initial Request", func(t *testing.T) {
 		rs := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {

examples/gateway/proxy/routing.go

@@ -8,12 +8,12 @@ import (
 	"strings"
 
 	"github.com/gogo/protobuf/proto"
-	"github.com/ipfs/boxo/examples/gateway/common"
 	"github.com/ipfs/boxo/ipns"
 	ipns_pb "github.com/ipfs/boxo/ipns/pb"
 	ic "github.com/libp2p/go-libp2p/core/crypto"
 	"github.com/libp2p/go-libp2p/core/peer"
 	"github.com/libp2p/go-libp2p/core/routing"
+	"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
 )
 
 type proxyRouting struct {
@@ -23,7 +23,9 @@ type proxyRouting struct {
 
 func newProxyRouting(gatewayURL string, client *http.Client) routing.ValueStore {
 	if client == nil {
-		client = common.NewClient()
+		client = &http.Client{
+			Transport: otelhttp.NewTransport(http.DefaultTransport),
+		}
 	}
 
 	return &proxyRouting{

examples/go.mod

@@ -25,6 +25,7 @@ require (
 	github.com/alecthomas/units v0.0.0-20210927113745-59d0afb8317a // indirect
 	github.com/benbjohnson/clock v1.3.0 // indirect
 	github.com/beorn7/perks v1.0.1 // indirect
+	github.com/cenkalti/backoff/v4 v4.2.0 // indirect
 	github.com/cespare/xxhash v1.1.0 // indirect
 	github.com/cespare/xxhash/v2 v2.2.0 // indirect
 	github.com/containerd/cgroups v1.0.4 // indirect
@@ -50,6 +51,7 @@ require (
 	github.com/google/gopacket v1.1.19 // indirect
 	github.com/google/pprof v0.0.0-20221203041831-ce31453925ec // indirect
 	github.com/google/uuid v1.3.0 // indirect
+	github.com/grpc-ecosystem/grpc-gateway/v2 v2.7.0 // indirect
 	github.com/hashicorp/errwrap v1.1.0 // indirect
 	github.com/hashicorp/go-multierror v1.1.1 // indirect
 	github.com/hashicorp/golang-lru v0.5.4 // indirect
@@ -108,6 +110,7 @@ require (
 	github.com/onsi/ginkgo/v2 v2.5.1 // indirect
 	github.com/opencontainers/runtime-spec v1.0.2 // indirect
 	github.com/opentracing/opentracing-go v1.2.0 // indirect
+	github.com/openzipkin/zipkin-go v0.4.1 // indirect
 	github.com/pbnjay/memory v0.0.0-20210728143218-7b4eea64cf58 // indirect
 	github.com/petar/GoLLRB v0.0.0-20210522233825-ae3b015fd3e9 // indirect
 	github.com/pkg/errors v0.9.1 // indirect
@@ -134,8 +137,16 @@ require (
 	go.opentelemetry.io/contrib/propagators/b3 v1.15.0 // indirect
 	go.opentelemetry.io/contrib/propagators/jaeger v1.15.0 // indirect
 	go.opentelemetry.io/contrib/propagators/ot v1.15.0 // indirect
+	go.opentelemetry.io/otel/exporters/jaeger v1.14.0 // indirect
+	go.opentelemetry.io/otel/exporters/otlp/internal/retry v1.14.0 // indirect
+	go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.14.0 // indirect
+	go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.14.0 // indirect
+	go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.14.0 // indirect
+	go.opentelemetry.io/otel/exporters/stdout/stdouttrace v1.14.0 // indirect
+	go.opentelemetry.io/otel/exporters/zipkin v1.14.0 // indirect
 	go.opentelemetry.io/otel/metric v0.37.0 // indirect
 	go.opentelemetry.io/otel/trace v1.14.0 // indirect
+	go.opentelemetry.io/proto/otlp v0.19.0 // indirect
 	go.uber.org/atomic v1.10.0 // indirect
 	go.uber.org/dig v1.15.0 // indirect
 	go.uber.org/fx v1.18.2 // indirect
@@ -150,6 +161,8 @@ require (
 	golang.org/x/text v0.7.0 // indirect
 	golang.org/x/tools v0.3.0 // indirect
 	golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 // indirect
+	google.golang.org/genproto v0.0.0-20230110181048-76db0878b65f // indirect
+	google.golang.org/grpc v1.53.0 // indirect
 	google.golang.org/protobuf v1.28.1 // indirect
 	gopkg.in/yaml.v3 v3.0.1 // indirect
 	lukechampine.com/blake3 v1.1.7 // indirect