No traces for go applications even though process has found by instrumentation container.

[jyoti-borade]

I have configured otel auto-instrumetation for three of my go services.
I am getting below logs for each of them:

{"level":"info","ts":1706790698.5967553,"logger":"go.opentelemetry.io/auto","caller":"cli/main.go:94","msg":"building OpenTelemetry Go instrumentation ...","globalImpl":false}
{"level":"info","ts":1706790700.5980866,"logger":"Instrumentation.Analyzer","caller":"process/discover.go:69","msg":"found process","pid":19}

But after this I don't see any logs, even though I hit the api calls for my go services, I don't see any traces getting generated.
I am using below image.
image: ghcr.io/open-telemetry/opentelemetry-go-instrumentation/autoinstrumentation-go:v0.10.1-alpha
also I have set below env variables:

• name: OTEL_SERVICE_NAME
  value: target-service-name
  - name: OTEL_EXPORTER_OTLP_ENDPOINT
  value: http://my-collector-service:13575
  - name: OTEL_TRACES_EXPORTER
  value: otlp

Environment

• OS: LINUX
• Go Version: 1.7.5
• Version: v0.10.0-alpha
  more system info:
  / $ uname -r
  5.10.205-195.804.amzn2.x86_64
  / $ uname -m
  x86_64
  / $

[damemi]

is your go version really 1.7.5? If so that might be the problem. The logging output hanging at found process means it's stuck looking for functions to instrument

[jyoti-borade]

is your go version really 1.7.5? If so that might be the problem. The logging output hanging at found process means it's stuck looking for functions to instrument

Yes it is 1.7 and 1.7.5. The go apps which I am trying to instrument are developed long back. so can't we auto-instrument it as the go version is 1.7?

[damemi]

@jyoti-borade that might be it. Could you please share the full output of go version so I can check?

The real bug is that whatever is causing the instrumentation to fail is not reporting an error. What you're seeing is actually the code hanging trying to report the error. I proposed a fix for this here: #638. You could try building with that PR to see what the exact error is you're getting

[jyoti-borade]

@jyoti-borade that might be it. Could you please share the full output of go version so I can check?

The real bug is that whatever is causing the instrumentation to fail is not reporting an error. What you're seeing is actually the code hanging trying to report the error. I proposed a fix for this here: #638. You could try building with that PR to see what the exact error is you're getting

https://github.com/microservices-demo/catalogue/tree/master This is the repository for the Go application I'm trying to auto-instrument—an open-source sample microservice developed in Go. The specified Go version in the Dockerfile is 1.7. I'm attempting to auto-instrument the publicly available image [version](image: weaveworksdemos/catalogue:0.3.5). Now, I'll check if I can retrieve the actual error details with the mentioned PR.

[jyoti-borade]

@damemi With the new build( #638 ) I am getting below error:

{"level":"info","ts":1706858639.801028,"logger":"go.opentelemetry.io/auto","caller":"cli/main.go:94","msg":"building OpenTelemetry Go instrumentation ...","globalImpl":false}
{"level":"info","ts":1706858641.8052044,"logger":"Instrumentation.Analyzer","caller":"process/discover.go:70","msg":"found process","pid":7}
{"level":"error","ts":1706858641.8076577,"logger":"go.opentelemetry.io/auto","caller":"cli/main.go:103","msg":"failed to create instrumentation","error":"not a Go executable","stacktrace":"main.main\n\t/app/cli/main.go:103\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:267"}
{"level":"info","ts":1706858727.7907786,"logger":"go.opentelemetry.io/auto","caller":"cli/main.go:94","msg":"building OpenTelemetry Go instrumentation ...","globalImpl":false}
{"level":"info","ts":1706858729.8552575,"logger":"Instrumentation.Analyzer","caller":"process/discover.go:70","msg":"found process","pid":7}
{"level":"error","ts":1706858729.860884,"logger":"go.opentelemetry.io/auto","caller":"cli/main.go:103","msg":"failed to create instrumentation","error":"not a Go executable","stacktrace":"main.main\n\t/app/cli/main.go:103\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:267"}

I have set below env variable:

• name: OTEL_GO_AUTO_TARGET_EXE
  value: /app

and 'app' is the go executable present in home directory of container.

[damemi]

Looks like that error comes from getGoDetails() when trying to parse the go build info. I think that the go version or build environment is the issue, but it seems like a bug in our code to not handle it.

@edeNFed it looks like a lot of this was copied from the Go buildinfo package, is there any reason not to use that package directly? Wonder if the extra handling they do supports older versions like this but it's a little over my head

[pellared]

This version of Go is not supported

For reference: https://github.com/open-telemetry/opentelemetry-go-instrumentation/tree/main?tab=readme-ov-file#compatibility

[damemi]

@pellared we should clarify the readme that the go support is for instrumenting applications and for building the auto-instrumentation binary. I don't think it's clear right now, and our offsets track way earlier than the previous 2 go versions (xref #639 for dropping this support)

[edeNFed]

@edeNFed it looks like a lot of this was copied from the Go buildinfo package, is there any reason not to use that package directly? Wonder if the extra handling they do supports older versions like this but it's a little over my head

Using buildinfo package makes total sense, I don't remember the reason for not using it in the first place.
Also, just noticed go1.22 has a new go/version package we can probably leverage as well.

[damemi]

Ok, I think we can close this issue as the version of Go in the original post is not supported. I'll open another issue to track refactoring that code to use buildinfo.