Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[processor/resourcedetection] k8snode detector queries K8s API for k8s.node.uid when the resource label is disabled #31941

Closed
avanish-vaghela opened this issue Mar 25, 2024 · 2 comments · Fixed by #32039
Assignees
Labels
bug Something isn't working processor/resourcedetection Resource detection processor

Comments

@avanish-vaghela
Copy link

Component(s)

processor/resourcedetection

What happened?

Description

When you configure the K8snode detector to

  • get the node name from an existing variable such as MY_NODE_NAME and
  • disable the k8s.node.uid resource label and
  • disable the k8snode auth by setting auth_type to none
    the k8snode still calls the K8s API to get the K8s node UID, resulting in a resource detection error

This is a bug in the K8s node detector where even if you disable the K8s node uid label it will still query the k8s API.

Steps to Reproduce

Expected Result

k8snode detector should be able to set the k8s.node.name value from a given environment variable without having to call the K8s API and it should not call the K8s APIs to get the k8s.node.uid when it has been explicitly disabled.

Actual Result

2024-03-25T18:42:26.207519786Z 2024-03-25T18:42:26.207Z	warn	internal/resourcedetection.go:130	failed to detect resource	{"kind": "processor", "name": "resourcedetection", "pipeline": "metrics/splunk", "error": "failed getting k8s node UID: failed to fetch node with name sv2-rkeworkervm3-1a-lab from K8s API: Unauthorized"}

Adding a conditional logic around this line should address the issue.

Collector version

0.96.0

Environment information

Environment

OS: Linux
Env: K8s

OpenTelemetry Collector configuration

resourcedetection:
      detectors: [env, k8snode]
      timeout: 2s
      override: false
      k8snode:
        node_from_env_var: MY_NODE_NAME
        auth_type: none
        resource_attributes:
          k8s.node.uid:
            enabled: false

Log output

2024-03-25T18:42:26.207519786Z 2024-03-25T18:42:26.207Z	warn	internal/resourcedetection.go:130	failed to detect resource	{"kind": "processor", "name": "resourcedetection", "pipeline": "metrics/splunk", "error": "failed getting k8s node UID: failed to fetch node with name kv2-workervm3-1a-lab from K8s API: Unauthorized"}

Additional context

Code where detection happens without attributes enablement check. https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/processor/resourcedetectionprocessor/internal/k8snode/k8snode.go#L51

@avanish-vaghela avanish-vaghela added bug Something isn't working needs triage New item requiring triage labels Mar 25, 2024
@github-actions github-actions bot added the processor/resourcedetection Resource detection processor label Mar 25, 2024
Copy link
Contributor

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@crobert-1
Copy link
Member

Thanks for filing, and for the thorough investigation @avanish-vaghela! Super helpful!

I've submitted #32039 to fix this.

@crobert-1 crobert-1 removed the needs triage New item requiring triage label Mar 29, 2024
@crobert-1 crobert-1 self-assigned this Mar 29, 2024
dmitryax pushed a commit that referenced this issue Mar 30, 2024
…e attributes when enabled (#32039)

**Description:** 
Detecting k8s node resource attributes requires specific permissions and
configuration. If they're disabled we shouldn't hit errors trying to
detect them. This change makes it so that these resource attributes are
only attempted to be detected when enabled.

**Link to tracking Issue:**
Resolves
#31941

**Testing:** 
Existing tests are passing. New test was failing before changes,
succeeds after.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working processor/resourcedetection Resource detection processor
Projects
None yet
2 participants