-
Notifications
You must be signed in to change notification settings - Fork 103
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CSI volume created using nomad_external_volume resource seems to be missing mount_flags #260
Comments
Hi @CarbonCollins!
The |
If this is the case how does creating the volume from the CLI/API direct to nomad differ? If I do it using these methods I do not need to provide the mount_flags in the job spec as they are defined already (The job starts correctly and looking at the CSI controller logs I can see the mount_flags present, I also don't provide any mount_flags or mount_options in the spec file) The docs for the Nomad volume stanza also specifies Though i'm not sure how up to date/in sync the docs are as the terraform provider docs say Though to be fair I might be assuming that the terraform provider would be using the Nomad API... |
You can only register a volume with Terraform's You can include the
It does, but the CSI API for Nomad is still in beta and has changed fairly quickly. I think the Terraform docs might be stale here. |
I'm using the Though it is good to see the potential API changes with the |
Oh that's embarrassing... I didn't realize there was a second resource for that (can you tell it's been a while since I've looked at this?) 😊 The rest of my comments should apply though, so I'll verify that. |
To be honest I had to look at the provider source code to figure out what the difference between the As far as I understood it at the time, the I am mostly setting these mount_flags at the time of volume creation as I could not figure out how to define them in the nomad job file from a vault template as they seem to be limited to the task stanza whereas the volume mount_flags are at the group stanza level. Ideally I would be setting these credentials in the volume secrets but these also don't seem to be passed along either 😅 regardless of this the CSI controller I am using just converts these secrets into mount_flags and appends them onto the end anyway so for simplicity I want to at least get the mount_flags working first before complicating it even more :) |
@CarbonCollins I'm realizing after having just wrapped up hashicorp/nomad#12150 that we redact the mount options coming back from the API, precisely because they can be secrets. Having data redacted in the API definitely seems like it would break Terraform's model for how resources are supposed to work. I'm certain there's a canonical way in TF providers to handle that but I'm going to admit that I'm not an expert on that. Let me tag-in my colleague @lgfa29 to see if he has any suggestions. 👋 |
Ah yes, that was an unfortunate order of events that caused the resource names to be extra confusing, but we have to stick with it because renaming resources is a pretty big breaking change. Sorry about that.
Yup, but already got bitten by it, so it shouldn't be a problem anymore: Using the config you provided and intercepting the request I see that {
"Volumes": [
{
"ID": "plex-data",
"Name": "plex-data",
"ExternalID": "",
"Namespace": "",
"Topologies": null,
"AccessMode": "",
"AttachmentMode": "",
"MountOptions": {
"FSType": "cifs",
"MountFlags": null
},
... We're probably not parsing that structure correctly. I will need some time to investigate this further, but I will let you know once I find the root cause. |
Possibly fixed by hashicorp/nomad#12150, which will ship in Nomad 1.3.0 |
Hey I wanted to follow up on this because unfortunately it doesn't look like the Nomad 1.3.0 fixes actually fixed anything here, and it doesn't look like it's related to redaction. I ran the following on my local environment with the current terraform configprovider "nomad" {
address = "http://localhost:4646"
region = "global"
}
resource "nomad_external_volume" "volume0" {
type = "csi"
plugin_id = "org.democratic-csi.nfs"
volume_id = "csi-volume-nfs"
name = "csi-volume-nfs"
capacity_min = "1GiB"
capacity_max = "1GiB"
capability {
access_mode = "single-node-writer"
attachment_mode = "file-system"
}
mount_options {
mount_flags = ["noatime"]
}
} terraform apply
The results in the CLI:
Ok, so maybe it's a problem with the CLI. Let's check the API:
Nope, that's not it. Just to make sure it's not an issue with the Nomad API's redaction of the outputs, I went directly to the state store:
Finally, I've dumped the API request we get from Terraform and it's nil there too. So the data just isn't there. But if I create the same volume via
I'll dig into the provider code a bit to see if I can figure out what's going on there. |
Should be fixed in #266 |
Hi there,
Thank you for opening an issue. Please note that we try to keep the Terraform issue tracker reserved for bug reports and feature requests. For general usage questions, please see: https://www.terraform.io/community.html.
Terraform Version
Nomad Version
1.2.4
Provider Configuration
Which values are you setting in the provider configuration?
Environment Variables
Do you have any Nomad specific environment variable set in the machine running Terraform?
Affected Resource(s)
Please list the resources as a list, for example:
If this issue appears to affect multiple resources, it may be an issue with Terraform's core, so please mention this.
Terraform Configuration Files
Debug Output
Please provider a link to a GitHub Gist containing the complete debug output: https://www.terraform.io/docs/internals/debugging.html. Please do NOT paste the debug output in the issue; just paste a link to the Gist.
Panic Output
none
Expected Behavior
What should have happened?
CSI volume is created in nomad and when an allocation starts it should be using the mount_flags specified in the resource to mount the cifs share.
Actual Behavior
What actually happened?
CSI volume is created in nomad and when an allocation starts it uses a default set of mount_flags from the CSI driver (democratic-csi) when mounting the cifs share.
Steps to Reproduce
Please list the steps required to reproduce the issue, for example:
terraform apply
nomad job run nomad.hcl
(with job file that makes use of csi volume in this case I am usinglinuxserver/plex:latest
)Important Factoids
References
None
The text was updated successfully, but these errors were encountered: