diff --git a/debug.md b/debug.md index f2a608ce24d7..0c9826544c92 100644 --- a/debug.md +++ b/debug.md @@ -1,21 +1,37 @@ -# Debugging tips and tricks +# Logging, Debugging and Telemetry + +**Warning:The OpenCensus project is obsolete and was archived on July 31st, +2023.** This means that any security vulnerabilities that are found will not be +patched. We recommend that you begin migrating to OpenCensus tracing to +OpenTelemetry, the successor project. See [OpenCensus](#opencensus) below for +details. + +Logging, debugging and telemetry all capture data that can be used for +troubleshooting. Logging records specific events and transactions. Debugging +exposes values for immediate analysis. Telemetry is suitable for production use +and can serve both logging and monitoring purposes. Telemetry tracing follows +requests through a system to provide a view of component interactions. Telemetry +metrics collects data for significant performance indicators, offering insights +into a system's health. + +## Logging and debugging While working with the Go Client libraries you may run into some situations where you need a deeper level of understanding about what is going on in order to solve your problem. Here are some tips and tricks that you can use in these -cases. *Note* that many of the tips in this document will have a performance +cases. *Note* that many of the tips in this section will have a performance impact and are therefore not recommended for sustained production use. Use these tips locally or in production for a *limited time* to help get a better understanding of what is going on. -## HTTP based clients +### HTTP based clients All of our auto-generated clients have a constructor to create a client that uses HTTP/JSON instead of gRPC. Additionally a couple of our hand-written clients like Storage and Bigquery are also HTTP based. Here are some tips for debugging these clients. -### Try setting Go's HTTP debug variable +#### Try setting Go's HTTP debug variable Try setting the following environment variable for verbose Go HTTP logging: GODEBUG=http2debug=1. To read more about this feature please see the godoc for @@ -24,7 +40,7 @@ GODEBUG=http2debug=1. To read more about this feature please see the godoc for *WARNING*: Enabling this debug variable will log headers and payloads which may contain private information. -### Add in your own logging with an HTTP middleware +#### Add in your own logging with an HTTP middleware You may want to add in your own logging around HTTP requests. One way to do this is to register a custom HTTP client with a logging transport built in. Here is @@ -92,16 +108,16 @@ func main() { } ``` -## gRPC based clients +### gRPC based clients -### Try setting grpc-go's debug variables +#### Try setting grpc-go's debug variables Try setting the following environment variables for grpc-go: `GRPC_GO_LOG_VERBOSITY_LEVEL=99` `GRPC_GO_LOG_SEVERITY_LEVEL=info`. These are good for diagnosing connection level failures. For more information please see [grpc-go's debug documentation](https://pkg.go.dev/google.golang.org/grpc/examples/features/debugging#section-readme). -### Add in your own logging with a gRPC interceptors +#### Add in your own logging with a gRPC interceptors You may want to add in your own logging around gRPC requests. One way to do this is to register a custom interceptor that adds logging. Here is @@ -160,3 +176,220 @@ func main() { // Use the client } ``` + +## Telemetry + +**Warning:The OpenCensus project is obsolete and was archived on July 31st, +2023.** This means that any security vulnerabilities that are found will not be +patched. We recommend that you begin migrating to OpenCensus tracing to +OpenTelemetry, the successor project. See [OpenCensus](#opencensus) below for +details. + +The Google Cloud client libraries for Go still use the OpenCensus project by +default. However, opt-in support for +[OpenTelemetry](https://opentelemetry.io/docs/what-is-opentelemetry/) is now +available. The transition from OpenCensus to OpenTelemetry is covered in the +following sections. + +### Tracing (experimental) + +Apart from spans created by underlying libraries such as gRPC, Google Cloud Go +generated clients do not create spans. Only the spans created by following +hand-written clients are in scope for the discussion in this section: + +* [cloud.google.com/go/bigquery](https://pkg.go.dev/cloud.google.com/go/bigquery) +* [cloud.google.com/go/bigtable](https://pkg.go.dev/cloud.google.com/go/bigtable) +* [cloud.google.com/go/datastore](https://pkg.go.dev/cloud.google.com/go/datastore) +* [cloud.google.com/go/firestore](https://pkg.go.dev/cloud.google.com/go/firestore) +* [cloud.google.com/go/spanner](https://pkg.go.dev/cloud.google.com/go/spanner) +* [cloud.google.com/go/storage](https://pkg.go.dev/cloud.google.com/go/storage) + +Currently, the spans created by these clients are for OpenCensus. However, +OpenCensus users are urged to transition to OpenTelemetry as soon as possible, +as explained in the next section. OpenTelemetry users can opt-in to experimental +OpenTelemetry support via an environment variable, as described below. + +#### OpenCensus + +**Warning:The OpenCensus project is obsolete and was archived on July 31st, +2023.** This means that any security vulnerabilities that are found will not be +patched. We recommend that you begin migrating to OpenCensus tracing to +OpenTelemetry, the successor project. + +Using the [OpenTelemetry-Go - OpenCensus Bridge](https://pkg.go.dev/go.opentelemetry.io/otel/bridge/opencensus), you can immediately begin exporting your traces with OpenTelemetry, even while +dependencies of your application remain instrumented with OpenCensus. If you do +not use the bridge, you will need to migrate your entire application and all of +its instrumented dependencies at once. For simple applications, this may be +possible, but we expect the bridge to be helpful if multiple libraries with +instrumentation are used. + +On May 29, 2024, six months after the +[release](https://github.com/googleapis/google-cloud-go/releases/tag/v0.111.0) +of experimental, opt-in support for OpenTelemetry tracing, the default tracing +support in the clients above will change from OpenCensus to OpenTelemetry, and +the experimental OpenCensus support will be marked as deprecated. To continue +using the OpenCensus support after this change, set the environment variable +`GOOGLE_API_GO_EXPERIMENTAL_TELEMETRY_PLATFORM_TRACING` to the case-insensitive +value `opencensus` before loading the client library. + +```sh +export GOOGLE_API_GO_EXPERIMENTAL_TELEMETRY_PLATFORM_TRACING=opencensus +``` + +On December 2nd, 2024, one year after the release of OpenTelemetry support, the +experimental and deprecated support for OpenCensus tracing will be removed. + +Please note that all Google Cloud Go clients currently provide experimental +support for the propagation of both OpenCensus and OpenTelemetry trace context +to their receiving endpoints. The experimental support for OpenCensus trace +context propagation will be removed at the same time as the experimental +OpenCensus tracing support. + +Please refer to the following resources: + +* [Sunsetting OpenCensus](https://opentelemetry.io/blog/2023/sunsetting-opencensus/) +* [OpenTelemetry-Go - OpenCensus Bridge](https://pkg.go.dev/go.opentelemetry.io/otel/bridge/opencensus) + +#### OpenTelemetry + +To opt-in to experimental OpenTelemetry tracing currently available in the +clients listed above, set the environment variable +`GOOGLE_API_GO_EXPERIMENTAL_TELEMETRY_PLATFORM_TRACING` to the case-insensitive +value `opentelemetry` before loading the client library. + +```sh +export GOOGLE_API_GO_EXPERIMENTAL_TELEMETRY_PLATFORM_TRACING=opentelemetry +``` + +On May 29, 2024, the default tracing support will change from OpenCensus to +OpenTelemetry, and this environment variable will no longer be needed. + +Please refer to the following resources: + +* [What is OpenTelemetry?](https://opentelemetry.io/docs/what-is-opentelemetry/) +* [Cloud Trace - Go and OpenTelemetry](https://cloud.google.com/trace/docs/setup/go-ot) +* On GCE, [use Ops Agent and OpenTelemetry](https://cloud.google.com/trace/docs/otlp) + +##### Configuring the OpenTelemetry-Go - OpenCensus Bridge + +To configure the OpenCensus bridge with OpenTelemetry and Cloud Trace: + +```go +import ( + "context" + "log" + "os" + texporter "github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/trace" + octrace "go.opencensus.io/trace" + "go.opentelemetry.io/contrib/detectors/gcp" + "go.opentelemetry.io/otel" + "go.opentelemetry.io/otel/bridge/opencensus" + "go.opentelemetry.io/otel/sdk/resource" + sdktrace "go.opentelemetry.io/otel/sdk/trace" + semconv "go.opentelemetry.io/otel/semconv/v1.7.0" +) + +func main() { + // Create exporter. + ctx := context.Background() + projectID := os.Getenv("GOOGLE_CLOUD_PROJECT") + exporter, err := texporter.New(texporter.WithProjectID(projectID)) + if err != nil { + log.Fatalf("texporter.New: %v", err) + } + // Identify your application using resource detection + res, err := resource.New(ctx, + // Use the GCP resource detector to detect information about the GCP platform + resource.WithDetectors(gcp.NewDetector()), + // Keep the default detectors + resource.WithTelemetrySDK(), + // Add your own custom attributes to identify your application + resource.WithAttributes( + semconv.ServiceNameKey.String("my-application"), + ), + ) + if err != nil { + log.Fatalf("resource.New: %v", err) + } + // Create trace provider with the exporter. + // + // By default it uses AlwaysSample() which samples all traces. + // In a production environment or high QPS setup please use + // probabilistic sampling. + // Example: + // tp := sdktrace.NewTracerProvider(sdktrace.WithSampler(sdktrace.TraceIDRatioBased(0.0001)), ...) + tp := sdktrace.NewTracerProvider( + sdktrace.WithBatcher(exporter), + sdktrace.WithResource(res), + ) + defer tp.Shutdown(ctx) // flushes any pending spans, and closes connections. + otel.SetTracerProvider(tp) + tracer := otel.GetTracerProvider().Tracer("example.com/trace") + // Configure the OpenCensus tracer to use the bridge. + octrace.DefaultTracer = opencensus.NewTracer(tracer) + // Use otel tracer to create spans... +} + +``` + + +##### Configuring context propagation + +In order to pass options to OpenTelemetry trace context propagation, follow the +appropriate example for the client's underlying transport. + +###### Passing options in HTTP-based clients + +```go +ctx := context.Background() +trans, err := htransport.NewTransport(ctx, + http.DefaultTransport, + option.WithScopes(storage.ScopeFullControl), +) +if err != nil { + log.Fatal(err) +} +// An example of passing options to the otelhttp.Transport. +otelOpts := otelhttp.WithFilter(func(r *http.Request) bool { + return r.URL.Path != "/ping" +}) +hc := &http.Client{ + Transport: otelhttp.NewTransport(trans, otelOpts), +} +client, err := storage.NewClient(ctx, option.WithHTTPClient(hc)) +``` + +Note that scopes must be set manually in this user-configured solution. + +###### Passing options in gRPC-based clients + +```go +projectID := "..." +ctx := context.Background() + +// An example of passing options to grpc.WithStatsHandler. +otelOpts := otelgrpc.WithMessageEvents(otelgrpc.ReceivedEvents) +dialOpts := grpc.WithStatsHandler(otelgrpc.NewClientHandler(otelOpts)) + +ctx := context.Background() +c, err := datastore.NewClient(ctx, projectID, option.WithGRPCDialOption(dialOpts)) +if err != nil { + log.Fatal(err) +} +defer c.Close() +``` + + +### Metrics (experimental) + +The generated clients do not create metrics. Only the following hand-written +clients create experimental OpenCensus metrics: + +* [cloud.google.com/go/bigquery](https://pkg.go.dev/cloud.google.com/go/bigquery) +* [cloud.google.com/go/pubsub](https://pkg.go.dev/cloud.google.com/go/pubsub) +* [cloud.google.com/go/spanner](https://pkg.go.dev/cloud.google.com/go/spanner) + +#### OpenTelemetry + +The transition of the experimental metrics in the clients above from OpenCensus +to OpenTelemetry is still TBD. \ No newline at end of file diff --git a/doc.go b/doc.go index d15db660e6a2..a90d072f0f05 100644 --- a/doc.go +++ b/doc.go @@ -250,7 +250,7 @@ situations, including: [testing against fake servers]: https://github.com/googleapis/google-cloud-go/blob/main/testing.md#testing-grpc-services-using-fakes [Vertex AI - Locations]: https://cloud.google.com/vertex-ai/docs/general/locations [Google Application Default Credentials]: https://cloud.google.com/docs/authentication/external/set-up-adc -[Debugging Guide]: https://github.com/googleapis/google-cloud-go/blob/main/debug.md +[Logging, Debugging and Telemetry Guide]: https://github.com/googleapis/google-cloud-go/blob/main/debug.md [Testing Guide]: https://github.com/googleapis/google-cloud-go/blob/main/testing.md */ package cloud // import "cloud.google.com/go"