Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Topology broken in 1.0.1 with a JS error #9702

Closed
commarla opened this issue Dec 21, 2020 · 4 comments · Fixed by #9733
Closed

Topology broken in 1.0.1 with a JS error #9702

commarla opened this issue Dec 21, 2020 · 4 comments · Fixed by #9733

Comments

@commarla
Copy link

Nomad version

Nomad v1.0.1 (c9c68aa55a7275f22d2338f2df53e67ebfcb9238)

Operating system and Environment details

Debian 9.13

Issue

Topology UI is not displaying
image

with this JS error

image

Maybe related to #9658 and #9617

I have three identical servers clusters, 2 are not working and 1 is working but the last one has only 3 clients and 1 alloc. The non working have 973 allocs / 113 clients and 226 allocs /32 clients

Reproduction steps

Just head to the topology

Nomad Server logs (if appropriate)

Nomad monitor logs in trace
2020-12-21T11:15:51.622+0100 [DEBUG] http: request failed: method=GET path=/v1/acl/token/self error="RPC Error:: 400,ACL support disabled" code=400
2020-12-21T11:15:51.622+0100 [DEBUG] http: request complete: method=GET path=/v1/acl/token/self duration=111.539µs
2020-12-21T11:15:51.622+0100 [ERROR] http: request failed: method=GET path=/v1/operator/license error="Nomad Enterprise only endpoint" code=501
2020-12-21T11:15:51.622+0100 [DEBUG] http: request complete: method=GET path=/v1/operator/license duration=210.584µs
2020-12-21T11:15:51.622+0100 [DEBUG] http: request complete: method=GET path=/v1/agent/members duration=145.748µs
2020-12-21T11:15:51.694+0100 [DEBUG] http: request complete: method=GET path=/v1/nodes?resources=true duration=3.29983ms
2020-12-21T11:15:51.733+0100 [DEBUG] http: request complete: method=GET path=/v1/jobs duration=42.236001ms
2020-12-21T11:15:51.745+0100 [DEBUG] http: request complete: method=GET path=/v1/namespaces duration=141.827µs
2020-12-21T11:15:51.767+0100 [DEBUG] http: request complete: method=GET path=/v1/agent/self duration=589.724µs
2020-12-21T11:15:52.058+0100 [DEBUG] http: request complete: method=GET path=/v1/allocations?namespace=%2A&resources=true&task_states=false duration=367.722724ms
2020-12-21T11:15:52.772+0100 [DEBUG] nomad: memberlist: Stream connection from=127.0.0.1:47800
2020-12-21T11:15:53.807+0100 [DEBUG] nomad: memberlist: Stream connection from=10.32.150.103:46174
@DingoEatingFuzz
Copy link
Contributor

Thank you for reporting this, @commarla! Apologies for the delay, but HashiCorp was observing a holiday break.

Parsing through that error message is always a bit of a challenge with minified code, but it appears to be that a node is being returned from the API without a NodeResources property.

The error is thrown here:

dataForNode(node) {
return {
node,
datacenter: node.datacenter,
memory: node.resources.memory,
cpu: node.resources.cpu,
allocations: [],
isSelected: false,
};
}

And node.resources.memory there maps to NodeResources as defined here:

attrs = {
isDraining: 'Drain',
httpAddr: 'HTTPAddr',
resources: 'NodeResources',
reserved: 'ReservedResources',
};

The easy fix here is to guard against null resources, but I want to dig into why resources is null in the first place first.

@commarla
Copy link
Author

commarla commented Jan 4, 2021

Hi @DingoEatingFuzz

With the info you provided I was able to pin down the nodes where NodeResources.Memory is null simply with a api call
curl $NOMAD_ADDR/v1/nodes?resources=true |jq -r '.[]|"\(.ID) \(.NodeResources.Memory)"'

The faulty nodes are still running nomad 0.8.7. I have to upgrade !

Thanks

@DingoEatingFuzz
Copy link
Contributor

Aha! That'll do it. I'll PR the null check too so the UI won't explode in this scenario.

@github-actions
Copy link

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 26, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants