proxmox_virtual_environment_file with content_type=snippets and source_file does not work

[otopetrik]

Describe the bug
Using proxmox_virtual_environment_file resource with non-empty attribute source_file and content_type=snippets fails with:

proxmox_virtual_environment_file.test_file: Creating...
╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to proxmox_virtual_environment_file.test_file2, provider "provider[\"registry.terraform.io/bpg/proxmox\"]" produced an unexpected new value: Root resource was present, but now absent.
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.
╵

To Reproduce
Steps to reproduce the behavior:

1. Create a main.tf file with following contents:

data "proxmox_virtual_environment_nodes" "available_nodes" {}

resource "proxmox_virtual_environment_file" "test_file" {
  content_type = "snippets"
  datastore_id = "local"
  node_name = data.proxmox_virtual_environment_nodes.available_nodes.names[0]

  source_file {
    path = "main.tf"
    file_name = "main.tf-copy-as-snippet.txt"
  }
}

2. Run 'terraform apply'
3. See the aforementioned error
4. Comment out following line:

   terraform-provider-proxmox/proxmoxtf/resource/file.go

   Line 627 in 926382c

   d.SetId("")

5. Run 'terraform apply' using "patched" provider
6. See no error

Observed behavior
Function fileCreate has special path for snippets and uses SSH for upload, but at the end it calls fileRead,
which assumes that just uploaded file will be available using PVE API result, which is not true for content_type=snippets.
Loop matches nothing, and at the end of the function d.SetId("") is called, causing the error.

Commenting out d.SetId("") works around the problem. But it is NOT a fix! For resources (files) deleted by other means (e.g. SSH, or just two copies of colliding terraform environments using same names), terraform would not know about that, and believe that they still exist.

Proper fix likely requires fileRead to check existence of the file using SSH.

Additional context
Really not sure about the logic of fileRead itself, it looks like it needs to be rewritten.
Following behavior is suspect:

1. It seems to ignore everything from PVE API except the v.VolumeID.
   Looks like it would not detect .img or .iso file in the datastore being replaced by one with same name but different size, because returned DatastoreFileListResponseData.FileSize is not even accessed.
   It probably should at least call something like d.Set(mkResourceVirtualEnvironmentFileFileSize, v.FileSize).

2. When file resource is using source_raw (i.e. no source file), it bails early:
   

   terraform-provider-proxmox/proxmoxtf/resource/file.go

   Line 564 in 926382c

   if len(sourceFile) == 0 {

   and therefore it does not detect if the file is deleted outside of terraform.

• Provider version (ideally it should be the latest version): v0.21.0
• Terraform version: v1.4.6

[dmsimms]

I've been banging my head for a week trying to work around this, I thought I had just something configured wrong!

[otopetrik]

Discovered that content type "iso" actually supports for ".img" files, so the following works:

resource "proxmox_virtual_environment_file" "test_image" {
  ...
  node_name = "..."
  datastore_id = "local"
  content_type = "iso"
  source_file {
    file_name = "$cirros-0.6.2-x86_64-disk.img"
    path = "https://download.cirros-cloud.net/0.6.2/cirros-0.6.2-x86_64-disk.img"
    checksum = "07e44a73e54c94d988028515403c1ed762055e01b83a767edf3c2b387f78ce00"
  }
}

resource "proxmox_virtual_environment_vm" "test_template" {
  ...
  disk {
    datastore_id = "..."
    file_id      = proxmox_virtual_environment_file.test_image.id
    interface    = "scsi0"
  }
}

Btw. the d.Set(mkResourceVirtualEnvironmentFileFileSize, v.FileSize) idea in the original comment is likely wrong. There should be two size attributes: file_size (= size of the source) and uploaded_file_size (= size in pve datastore), and cause resource re-creation if sizes do not match.

[dandaolrian]

Also came across this issue as well yesterday. Use case was to use one of these files for the user_data in cloud-init
Would be nice if this got fixed

Workaround was to use source_raw rather than source_file and then use file() to put the data in

BTW: Cool provider :)

[bpg]

Thanks for reporting!

Interesting thought, I can't reproduce it on my test server using the provided example:

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # proxmox_virtual_environment_file.test_file will be created
  + resource "proxmox_virtual_environment_file" "test_file" {
      + content_type           = "snippets"
      + datastore_id           = "local"
      + file_modification_date = (known after apply)
      + file_name              = (known after apply)
      + file_size              = (known after apply)
      + file_tag               = (known after apply)
      + id                     = (known after apply)
      + node_name              = "pve"

      + source_file {
          + changed   = false
          + file_name = "main.tf-copy-as-snippet.txt"
          + insecure  = false
          + path      = "main.tf"
        }
    }

Plan: 1 to add, 0 to change, 0 to destroy.
proxmox_virtual_environment_file.test_file: Creating...
proxmox_virtual_environment_file.test_file: Creation complete after 1s [id=local:snippets/main.tf-copy-as-snippet.txt]

Could be related to the storage type, or whether snippets are enabled for this storage.
The provider makes an API call to get storage details, in my env it returns this for local:
2023-06-24T00:41:31.320-0400 [DEBUG] provider.terraform-provider-proxmox: Received HTTP Response: @module=proxmox Content-Length=164 Server=pve-api-daemon/3.0 tf_provider_addr=registry.terraform.io/bpg/proxmox @caller=/Users/pasha/go/pkg/mod/github.com/hashicorp/terraform-plugin-sdk/v2@v2.26.1/helper/logging/logging_http_transport.go:159 Date="Sat, 24 Jun 2023 04:41:31 GMT" Expires="Sat, 24 Jun 2023 04:41:31 GMT" tf_http_res_version=HTTP/1.1 tf_mux_provider=tf5to6server.v5tov6Server tf_req_id=bbc2d51a-a447-05de-039d-0b1ad942ffb7 tf_resource_type=proxmox_virtual_environment_file Cache-Control=max-age=0 Content-Type=application/json;charset=UTF-8 tf_http_op_type=response tf_http_res_body={"data":{"shared":0,"type":"dir","path":"/var/lib/vz","storage":"local","digest":"2ef93499b03432022d385884f1593f2f2311729c","content":"iso,vztmpl,backup,snippets"}} tf_http_res_status_code=200 tf_rpc=ApplyResourceChange tf_http_trans_id=73879ec2-0047-1127-f53c-324189bade09 Connection=Keep-Alive Pragma=no-cache tf_http_res_status_reason="200 OK" timestamp=2023-06-24T00:41:31.320-0400

My suspicion is snippets cant be read back via API if they are not enabled for the storage:

Could someone please check if the "Snippets" content type is enabled for the storage, and if not, try to enable it and see if this helps?

[bpg]

Indeed, it was due to missing "Snippets" content type on the storage.

I've added a better diagnostic for this use case:

rraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # proxmox_virtual_environment_file.test_file will be created
  + resource "proxmox_virtual_environment_file" "test_file" {
      + content_type           = "snippets"
      + datastore_id           = "local"
      + file_modification_date = (known after apply)
      + file_name              = (known after apply)
      + file_size              = (known after apply)
      + file_tag               = (known after apply)
      + id                     = (known after apply)
      + node_name              = "pve"

      + source_file {
          + changed   = false
          + file_name = "main.tf-copy-as-snippet.txt"
          + insecure  = false
          + path      = "main.tf"
        }
    }

Plan: 1 to add, 0 to change, 0 to destroy.
proxmox_virtual_environment_file.test_file: Creating...
╷
│ Warning: the datastore "local" does not support content type "snippets"
│
│   with proxmox_virtual_environment_file.test_file,
│   on main.tf line 1, in resource "proxmox_virtual_environment_file" "test_file":
│    1: resource "proxmox_virtual_environment_file" "test_file" {
│
╵
╷
│ Error: failed to read file from "local:snippets/main.tf-copy-as-snippet.txt"
│
│   with proxmox_virtual_environment_file.test_file,
│   on main.tf line 1, in resource "proxmox_virtual_environment_file" "test_file":
│    1: resource "proxmox_virtual_environment_file" "test_file" {
│
╵

[DanielHabenicht]

@bpg I have seen you last comment here and wondered why I did not see the message the datastore "local" does not support content type "snippets" Should I open an Issue for it?.

Without enabling "snippets" it showed this message:

proxmox_virtual_environment_file.cloud_config: Creating...
╷
│ Error: failed to create file /var/lib/vz/snippets/user-data.yml: file does not exist
│
│   with proxmox_virtual_environment_file.cloud_config,
│   on main.tf line 52, in resource "proxmox_virtual_environment_file" "cloud_config":
│   52: resource "proxmox_virtual_environment_file" "cloud_config" {
│
╵

Interestingly after disabling it again (after applying a snippet successfully) and reapplying it the message posted by you shows up.

[bpg]

@DanielHabenicht Yes, please open the issue. Missing warning could be related to auth / permissions, we need to investigate this a bit further.