CSI volumes API doesn't respect wildcard namespace filter

[dcarbone]

Nomad version

Nomad v1.2.2 (78b8c171a211f967a8b297a88a7e844b3543f2b0)

Operating system and Environment details

Linux 5.11.0-1022-raspi aarch64

Issue

After upgrading servers to v1.2.0+, any / all csi volumes do not appear in the web UI. The volumes are present in the CLI output.

Reproduction steps

1. Create n CSI volumes using csc or some other tool
2. Register volumes with Nomad in any namespace, default or otherwise

Expected Result

1. Volume creation verified using nomad volume status
2. Appropriate volumes are viewable when filtering by namespace
3. All volumes present when not filtering by namespace

Actual Result

Volumes created and registered successfully, but are only viewable in UI when filtering by their appropriate namespace. The "All" filter yields no results.

Nomad Server logs (if appropriate)

No relevant logging appears on the servers, no errors, etc.

[apollo13]

@dcarbone in the volume UI can you switch the namespace to default? For me the volumes which are in the default namespace only show there and not in "All (*)":

[dcarbone]

@apollo13: ah, yep, that's it. Apologies, I noticed this under v1.2.1 and I suspect I was running into #11563 at the same time I was trying to load the UI. After upgrading to v1.2.2, I am able to see them when selecting the appropriate namespace. I quite stupidly only looked at "all" and assumed the same behavior.

I'll update the issue accordingly, thanks!

[tgross]

I took a quick look into this and it looks like this is arguably not a bug in the UI but in the RPC layer for not respecting the wildcard that we implemented for the job API. But the namespace switcher component expects it to be everywhere.

I'm going to rename and retag the issue slightly so that it's assigned to the right folks to fix.

Repro, on a dev agent running the hostpath demo:

$ nomad namespace apply test
Successfully applied namespace "test"!

$ curl -s 'http://localhost:4646/v1/volumes?type=csi&namespace=*' | jq .
[]

$ curl -s 'http://localhost:4646/v1/volumes?type=csi&namespace=default' | jq .
[
  {
    "AccessMode": "single-node-reader-only",
    "AttachmentMode": "file-system",
    "ControllersExpected": 1,
    "ControllersHealthy": 1,
    "CreateIndex": 19,
    "CurrentReaders": 1,
    "CurrentWriters": 0,
    "ExternalID": "2ef22ba6-5123-11ec-ba9f-0242ac110002",
    "ID": "test-volume[0]",
    "ModifyIndex": 24,
    "Name": "test-volume[0]",
    "Namespace": "default",
    "NodesExpected": 1,
    "NodesHealthy": 1,
    "PluginID": "hostpath-plugin0",
    "Provider": "csi-hostpath",
    "Schedulable": true,
    "Topologies": []
  },
  {
    "AccessMode": "single-node-reader-only",
    "AttachmentMode": "file-system",
    "ControllersExpected": 1,
    "ControllersHealthy": 1,
    "CreateIndex": 20,
    "CurrentReaders": 1,
    "CurrentWriters": 0,
    "ExternalID": "2effb981-5123-11ec-ba9f-0242ac110002",
    "ID": "test-volume[1]",
    "ModifyIndex": 25,
    "Name": "test-volume[1]",
    "Namespace": "default",
    "NodesExpected": 1,
    "NodesHealthy": 1,
    "PluginID": "hostpath-plugin0",
    "Provider": "csi-hostpath",
    "Schedulable": true,
    "Topologies": []
  }
]

[lgfa29]

Hi all, just a quick update.

The API for listing volumes now supports the * namespace wildcard, so the UI will display volumes accordingly:

[github-actions]

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.