[question] Topology dashboard overcounting allocations and reserved resources.

[caiohcl]

Hi,

I recently updated Nomad to v1.0.1, and something caught my attention. I checked the topology dashboard, and under "Cluster Details" it shows that 151% of memory is currently in use. Another thing I noticed is that the Alloc number is different than what is shown in the graph.

For example, here it shows that client 046 has 7 Allocs, but the graph shows only 2:

As far as I can tell, the Topology is also using "completed" allocations to get the Cluster Details. I wonder if I'm missing some configuration or if this is the correct behavior?

Thank you!

[idrennanvmware]

Curious - do you happen to have any prestart tasks?

[caiohcl]

Hi @idrennanvmware. No, I'm not using any prestart task.

[manveru]

The same is happening to me across multiple clusters since the 1.0 release candidates. We also have a high number of dead allocations taking up memory and space on the nomad server nodes until we manually GC them and restart the nodes.

[DingoEatingFuzz]

Hi @caiohcl, as @manveru is suggesting, this is due to terminal allocations. These allocations don't actually reserve resources in the mind of the scheduler which is how you're able to get to "151%" of reserved memory used.

This is definitely a bug and it's naturally where I didn't write tests 😂 😭 ☠️

nomad/ui/tests/acceptance/topology-test.js

Lines 8 to 10 in 2867e26

	// TODO: Once we settle on the contents of the info panel, the contents
	// should also get acceptance tests.
	module('Acceptance | topology', function(hooks) {

[github-actions]

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.