Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

'kubectl node top' get 'error: Metrics API not available' when metrics-server is allocated on agent node #10859

Closed
piliplaker opened this issue Sep 8, 2024 · 1 comment

Comments

@piliplaker
Copy link

Environmental Info:
K3s Version:
v1.30.4+k3s1

Node(s) CPU architecture, OS, and Version:
Linux k3s-master 6.1.0-25-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.106-3 (2024-08-26) x86_64 GNU/Linux
Linux k3s-worker1-debian 6.8.4-2-pve #1 SMP PREEMPT_DYNAMIC PMX 6.8.4-2 (2024-04-10T17:36Z) x86_64 GNU/Linux
Linux k3s-worker2-debian 6.8.4-2-pve #1 SMP PREEMPT_DYNAMIC PMX 6.8.4-2 (2024-04-10T17:36Z) x86_64 GNU/Linux
Linux k3s-worker3-debian 6.8.4-2-pve #1 SMP PREEMPT_DYNAMIC PMX 6.8.4-2 (2024-04-10T17:36Z) x86_64 GNU/Linux

Cluster Configuration:
1 server , 3 agents

Describe the bug:
When the metric server is assigned to the server node, 'kubectl top node'' can work normally, but when it is assigned to the agent node, an 'error: Metrics API not available' will be reported.

Steps To Reproduce:
on server node :

token: 123456
node-label:
- "role=normal"
node-name: master-nas
disable:
- servicelb
embedded-registry: true
write-kubeconfig-mode: "0644"
node-taint:
  - k3s-controlplane=true:PreferNoSchedule

use curl -sfL https://get.k3s.io | sh - to install

on agent node:

token: 123456
node-label:
- "role=normal"
node-name: worker-cube
server: https://192.168.1.116:6443

use curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC=agent sh - to install

Initially, the metric server was running on the server node. and the cpu and mem indicators can be displayed normally in k9s
and then I use k9s to delete the pod running on server node trying to release some resource on server node,
then i found that cpu and mem metrics show n/a , I waited for a while till the pod running and it still looked like this
and then i run 'kubectl top node' get 'error: Metrics API not available'
ant then i delete the pod working on agent node , the new pod was allocated back to the server node, the cpu and mem came back

Expected behavior:
No matter where the metric server is allocated, it should be able to operate normally.

Actual behavior:
it is looks like that metrics server only works well on server node,

Additional context / logs:
when the metric server running on agent node
1725767308922
1725767360019

@piliplaker
Copy link
Author

well, I just reinstalled k3s server on master node, and every thing works fine. but i still don't know why this was happened.
by the way, before i did this, the coredns was not working too, that's why i desided to reinstall

@github-project-automation github-project-automation bot moved this from New to Done Issue in K3s Development Sep 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done Issue
Development

No branches or pull requests

1 participant