Bug: /otelcol-custom: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /otelcol-custom) #43

arjunvijaygoogle · 2023-10-03T04:37:39Z

Once I create the image using cloud build I get the following issue

arjunvijay@cloudshell:~/.../gke/simple (arjun-demo-123)$ kubectl logs otel-collector-97fd46647-rhhg4 -n $OTEL_NAMESPACE
/otelcol-custom: /lib/x86_64-linux-gnu/libc.so.6: version GLIBC_2.32' not found (required by /otelcol-custom) /otelcol-custom: /lib/x86_64-linux-gnu/libc.so.6: version GLIBC_2.34' not found (required by /otelcol-custom)

The text was updated successfully, but these errors were encountered:

arjunvijaygoogle · 2023-10-04T09:53:47Z

After updating the version from 0.75 to 0.86 for OTEL it still gives errors


arjunvijay-macbookpro:simple arjunvijay$ kubectl logs otel-collector-d89ff6984-n9snc -n $OTEL_NAMESPACE
exec /otelcol-custom: no such file or directory

arjunvijaygoogle · 2023-10-04T10:00:34Z

@damemi Can you please help me here

dashpole · 2023-10-04T13:54:32Z

Sorry, damemi is out. I'll take a look.

dashpole · 2023-10-04T13:55:22Z

Which sample did you apply, and which collector did you apply?

arjunvijaygoogle · 2023-10-04T14:44:00Z

Hi @dashpole I have rerun entire thing with everything rolled back to main branch. I still get the same error

arjunvijay-macbookpro:simple arjunvijay$ kubectl logs otel-collector-d89ff6984-p522m -n otel-collector
/otelcol-custom: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /otelcol-custom)
/otelcol-custom: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /otelcol-custom)

dashpole · 2023-10-04T15:19:42Z

What kind of GKE cluster are you running it on?

arjunvijaygoogle · 2023-10-04T15:55:13Z

I am running GKE Autopilot

dashpole · 2023-10-04T19:29:04Z

gcloud auth login
gcloud container clusters create-auto builder-bug --region us-east1
git clone https://github.com/GoogleCloudPlatform/opentelemetry-collector-builder-sample.git
cd opentelemetry-collector-builder-sample
make setup-artifact-registry
cd build/local
make docker-build
make docker-push
cd ../../deploy/gke/simple/
export GCLOUD_PROJECT=<the Google Cloud project ID to which your IAM service account belongs>
export PROJECT_ID=<your Google Cloud project ID>
gcloud iam service-accounts create otel-collector --project=${GCLOUD_PROJECT}
gcloud projects add-iam-policy-binding $PROJECT_ID \
    --member "serviceAccount:otel-collector@${GCLOUD_PROJECT}.iam.gserviceaccount.com" \
    --role "roles/logging.logWriter"
gcloud projects add-iam-policy-binding $PROJECT_ID \
    --member "serviceAccount:otel-collector@${GCLOUD_PROJECT}.iam.gserviceaccount.com" \
    --role "roles/cloudtrace.agent"
gcloud projects add-iam-policy-binding $PROJECT_ID \
    --member "serviceAccount:otel-collector@${GCLOUD_PROJECT}.iam.gserviceaccount.com" \
    --role "roles/monitoring.metricWriter"
gcloud iam service-accounts add-iam-policy-binding "otel-collector@${GCLOUD_PROJECT}.iam.gserviceaccount.com" \
    --role roles/iam.workloadIdentityUser \
    --member "serviceAccount:${GCLOUD_PROJECT}.svc.id.goog[otel-collector/otel-collector]"
export OTEL_NAMESPACE=otel-collector
kubectl create namespace $OTEL_NAMESPACE
kubectl create serviceaccount otel-collector -n otel-collector
kubectl annotate serviceaccount otel-collector \
    --namespace $OTEL_NAMESPACE \
    iam.gke.io/gcp-service-account=otel-collector@${GCLOUD_PROJECT}.iam.gserviceaccount.com
kubectl create configmap otel-config --from-file=./otel-config.yaml -n $OTEL_NAMESPACE
kubectl apply -f manifest.yaml -n $OTEL_NAMESPACE
kubectl logs deployment/otel-collector -n otel-collector

Output of kubectl logs shows success exporting, and does not contain any errors:

2023-10-04T19:23:34.416Z        info    service/telemetry.go:90 Setting up own telemetry...
2023-10-04T19:23:34.489Z        info    service/telemetry.go:116        Serving Prometheus metrics      {"address": ":8888", "level": "Basic"}
2023-10-04T19:23:34.609Z        info    exporter@v0.75.0/exporter.go:286        Development component. May change in the future.        {"kind": "exporter", "data_type": "traces", "name": "logging"}
2023-10-04T19:23:34.611Z        warn    loggingexporter@v0.75.0/factory.go:109  'loglevel' option is deprecated in favor of 'verbosity'. Set 'verbosity' to equivalent value to preserve behavior.      {"kind": "exporter", "data_type": "traces", "name": "logging", "loglevel": "debug", "equivalent verbosity level": "Detailed"}
2023-10-04T19:23:34.612Z        info    memorylimiterprocessor@v0.75.0/memorylimiter.go:149     Using percentage memory limiter {"kind": "processor", "name": "memory_limiter", "pipeline": "traces", "total_memory_mib": 512, "limit_percentage": 65, "spike_limit_percentage": 20}
2023-10-04T19:23:34.613Z        info    memorylimiterprocessor@v0.75.0/memorylimiter.go:113     Memory limiter configured       {"kind": "processor", "name": "memory_limiter", "pipeline": "traces", "limit_mib": 332, "spike_limit_mib": 102, "check_interval": 1}
2023-10-04T19:23:34.614Z        info    exporter@v0.75.0/exporter.go:286        Development component. May change in the future.        {"kind": "exporter", "data_type": "metrics", "name": "logging"}
2023-10-04T19:23:34.616Z        info    exporter@v0.75.0/exporter.go:286        Development component. May change in the future.        {"kind": "exporter", "data_type": "logs", "name": "logging"}
2023-10-04T19:23:34.621Z        info    service/service.go:129  Starting otelcol-custom...      {"Version": "1.0.0", "NumCPU": 2}
2023-10-04T19:23:34.621Z        info    extensions/extensions.go:41     Starting extensions...
2023-10-04T19:23:34.621Z        info    internal/resourcedetection.go:136       began detecting resource information    {"kind": "processor", "name": "resourcedetection/gke", "pipeline": "traces"}
2023-10-04T19:23:34.631Z        info    internal/resourcedetection.go:150       detected resource information   {"kind": "processor", "name": "resourcedetection/gke", "pipeline": "traces", "resource": {"cloud.account.id":"dashpole-gke-dev","cloud.platform":"gcp_kubernetes_engine","cloud.provider":"gcp","cloud.region":"us-east1","host.id":"3356598686790455806","host.name":"gk3-builder-bug-default-pool-4ff3ccc3-5pb9","k8s.cluster.name":"builder-bug"}}
2023-10-04T19:23:34.632Z        info    prometheusreceiver@v0.75.0/metrics_receiver.go:255      Starting discovery manager      {"kind": "receiver", "name": "prometheus/self", "data_type": "metrics"}
2023-10-04T19:23:34.632Z        info    prometheusreceiver@v0.75.0/metrics_receiver.go:243      Scrape job added        {"kind": "receiver", "name": "prometheus/self", "data_type": "metrics", "jobName": "otel-collector"}
2023-10-04T19:23:34.633Z        info    prometheusreceiver@v0.75.0/metrics_receiver.go:289      Starting scrape manager {"kind": "receiver", "name": "prometheus/self", "data_type": "metrics"}
2023-10-04T19:23:34.633Z        warn    internal/warning.go:51  Using the 0.0.0.0 address exposes this server to every network interface, which may facilitate Denial of Service attacks        {"kind": "receiver", "name": "otlp", "data_type": "logs", "documentation": "https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/security-best-practices.md#safeguards-against-denial-of-service-attacks"}
2023-10-04T19:23:34.633Z        info    otlpreceiver@v0.75.0/otlp.go:94 Starting GRPC server    {"kind": "receiver", "name": "otlp", "data_type": "logs", "endpoint": "0.0.0.0:4317"}
2023-10-04T19:23:34.633Z        warn    internal/warning.go:51  Using the 0.0.0.0 address exposes this server to every network interface, which may facilitate Denial of Service attacks        {"kind": "receiver", "name": "otlp", "data_type": "logs", "documentation": "https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/security-best-practices.md#safeguards-against-denial-of-service-attacks"}
2023-10-04T19:23:34.634Z        info    otlpreceiver@v0.75.0/otlp.go:112        Starting HTTP server    {"kind": "receiver", "name": "otlp", "data_type": "logs", "endpoint": "0.0.0.0:4318"}
2023-10-04T19:23:34.634Z        info    service/service.go:146  Everything is ready. Begin running and processing data.
2023-10-04T19:23:41.649Z        info    MetricsExporter {"kind": "exporter", "data_type": "metrics", "name": "logging", "#metrics": 16}
2023-10-04T19:23:41.650Z        info    ResourceMetrics #0
Resource SchemaURL: https://opentelemetry.io/schemas/1.6.1
Resource attributes:
     -> service.name: Str(otel-collector)
     -> service.instance.id: Str(0.0.0.0:8888)
     -> net.host.port: Str(8888)
     -> http.scheme: Str(http)
     -> cloud.provider: Str(gcp)
     -> cloud.account.id: Str(dashpole-gke-dev)
     -> cloud.platform: Str(gcp_kubernetes_engine)
     -> cloud.region: Str(us-east1)
     -> k8s.cluster.name: Str(builder-bug)
     -> host.id: Str(3356598686790455806)
     -> host.name: Str(gk3-builder-bug-default-pool-4ff3ccc3-5pb9)
ScopeMetrics #0
...

I am not able to reproduce your issue. Can you follow the steps above exactly (filling in the project), and let me know if you are still able to reproduce the issue?

arjunvijaygoogle · 2023-10-05T05:21:34Z

let me try this @dashpole

arjunvijaygoogle · 2023-10-05T05:54:05Z

Hi @dashpole , I followed the above steps and get the following error. I ran this from cloud shell, so local environment should not be a problematic. You can checkout my activity from cloud shell here

arjunvijay@cloudshell:~/opentelemetry-collector-builder-sample/deploy/gke/simple (arjun-demo-123)$ kubectl logs deployment/otel-collector -n otel-collector/otelcol-custom: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /otelcol-custom)
/otelcol-custom: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /otelcol-custom)

dashpole · 2023-10-05T13:26:43Z

Let me try from cloud shell. Based on golang/go#58550 this has something to do with the CGO_ENABLED and the go version. I'm also going to try updating the collector version and go version to see if that helps.

dashpole · 2023-10-05T13:37:37Z

One other question while I work on the above: Did you enable any options when you created your GKE Autopilot cluster?

arjunvijaygoogle · 2023-10-05T13:40:54Z

@dashpole No Used your command gcloud container clusters create-auto builder-bug --region us-east1

dashpole · 2023-10-05T14:39:59Z

I was able to reproduce this in cloud shell, so that seems to be part of the issue.

dashpole · 2023-10-05T14:51:10Z

Using cloud build from the cloud shell does not help

dashpole · 2023-10-05T19:47:05Z

Confirmed that #48 fixes this issue

dashpole self-assigned this Oct 4, 2023

dashpole added the bug Something isn't working label Oct 4, 2023

dashpole added the p3 label Oct 4, 2023

dashpole mentioned this issue Oct 4, 2023

Add workload identity setup to the sample documentation #44

Merged

dashpole mentioned this issue Oct 5, 2023

Update to go1.21, collector v0.86.0, and disable CGO #48

Merged

dashpole closed this as completed in #48 Oct 6, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bug: /otelcol-custom: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /otelcol-custom) #43

Bug: /otelcol-custom: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /otelcol-custom) #43

arjunvijaygoogle commented Oct 3, 2023

arjunvijaygoogle commented Oct 4, 2023 •

edited

Loading

arjunvijaygoogle commented Oct 4, 2023

dashpole commented Oct 4, 2023

dashpole commented Oct 4, 2023

arjunvijaygoogle commented Oct 4, 2023

dashpole commented Oct 4, 2023

arjunvijaygoogle commented Oct 4, 2023

dashpole commented Oct 4, 2023

arjunvijaygoogle commented Oct 5, 2023

arjunvijaygoogle commented Oct 5, 2023 •

edited

Loading

dashpole commented Oct 5, 2023

dashpole commented Oct 5, 2023

arjunvijaygoogle commented Oct 5, 2023 •

edited

Loading

dashpole commented Oct 5, 2023

dashpole commented Oct 5, 2023

dashpole commented Oct 5, 2023

Bug: /otelcol-custom: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /otelcol-custom) #43

Bug: /otelcol-custom: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /otelcol-custom) #43

Comments

arjunvijaygoogle commented Oct 3, 2023

arjunvijaygoogle commented Oct 4, 2023 • edited Loading

arjunvijaygoogle commented Oct 4, 2023

dashpole commented Oct 4, 2023

dashpole commented Oct 4, 2023

arjunvijaygoogle commented Oct 4, 2023

dashpole commented Oct 4, 2023

arjunvijaygoogle commented Oct 4, 2023

dashpole commented Oct 4, 2023

arjunvijaygoogle commented Oct 5, 2023

arjunvijaygoogle commented Oct 5, 2023 • edited Loading

dashpole commented Oct 5, 2023

dashpole commented Oct 5, 2023

arjunvijaygoogle commented Oct 5, 2023 • edited Loading

dashpole commented Oct 5, 2023

dashpole commented Oct 5, 2023

dashpole commented Oct 5, 2023

arjunvijaygoogle commented Oct 4, 2023 •

edited

Loading

arjunvijaygoogle commented Oct 5, 2023 •

edited

Loading

arjunvijaygoogle commented Oct 5, 2023 •

edited

Loading