vcd_org_vdc: adding a storage profile requires the vdc to be replaced #648

carmine73 · 2021-03-01T11:11:06Z

Terraform Version

Terraform v0.14.7
+ provider registry.terraform.io/vmware/vcd v3.1.0

Affected Resource(s)

vcd_org_vdc

Terraform Configuration Files

I have input values like the following:

input = {
  [. . .]
  sdc_storage           = [
    {
      type      = "fc"
      limit     = 102400  # MB, 0 means unlimited
      default   = true
    },
  ]
  [. . .]
}

The org vdc has a dynamic block for storage profiles:

## org-vdc
resource "vcd_org_vdc" "vdc1" {
  name              = local.vcd_org_vdc.name
  description       = local.vcd_org_vdc.descr
  org               = vcd_org.org1.name

  allocation_model  = local.vcd_org_vdc.alloc
  cpu_speed         = local.vcd_org_vdc.cpu_speed
  cpu_guaranteed    = local.vcd_org_vdc.cpu_guaranteed
  memory_guaranteed = local.vcd_org_vdc.mem_guaranteed
  network_pool_name = local.vcd_org_vdc.net_pool
  provider_vdc_name = local.vcd_org_vdc.pvdc

  compute_capacity {
    cpu {
      allocated = local.vcd_org_vdc.cpu_alloc
      limit     = local.vcd_org_vdc.cpu_limit
    }

    memory {
      allocated = local.vcd_org_vdc.mem_alloc
      limit     = local.vcd_org_vdc.mem_limit
    }
  }

  # dynamic storage profile (they could be more than one)
  dynamic "storage_profile" {
    for_each = [ for storage in local.vcd_org_vdc.storage : {
      name    = storage.name
      limit   = storage.limit
      default = storage.default
    } ]

    content {
      name    = storage_profile.value.name
      limit   = storage_profile.value.limit
      default = storage_profile.value.default
    }
  }

  network_quota = 10

  enabled                  = local.vcd_org_vdc.enabled
  enable_thin_provisioning = false
  enable_fast_provisioning = true
  delete_force             = true
  delete_recursive         = true
}

Steps to Reproduce

terraform plan
terraform apply
Change the value of the input to have:

input = {
  [. . .]
  sdc_storage           = [
    {
      type      = "fc"
      limit     = 102400  # MB, 0 means unlimited
      default   = true
    },
    {
      type      = "ssd"
      limit     = 51200   # MB, 0 means unlimited
      default   = false
    },
  ]
  [. . .]
}

terraform plan
terraform apply

Debug Output

First, I had a successful organization creation:

$ terraform apply "orgs/org-template_1.plan"
vcd_org.org1: Creation complete after 5s [id=urn:vcloud:org:58f8ad69-89a2-44d1-9341-fc573f9a8313]
vcd_org_user.org1-admin: Creation complete after 4s [id=urn:vcloud:user:d904f296-8632-4ee4-8ccc-c5b20a3ead3f]
vcd_org_vdc.vdc1: Creation complete after 42s [id=urn:vcloud:vdc:1d263d50-6ba1-44dc-a0aa-30ca6e794c56]
vcd_network_direct.bck_nets[0]: Creation complete after 30s [id=urn:vcloud:network:deda1a77-5117-4778-944d-957bd6b2aa45]
vcd_network_direct.bck_nets[1]: Creation complete after 33s [id=urn:vcloud:network:f8a9ff69-cadd-4718-8099-801bbd1acc62]
vcd_network_direct.dir_nets[0]: Creation complete after 48s [id=urn:vcloud:network:225c9bf9-286a-4d63-ab13-a7c7657ada16]
vcd_edgegateway.edge1: Creation complete after 1m19s [id=urn:vcloud:gateway:e9165d68-66af-435f-9ff8-bf74483b0311]
vcd_network_routed.edge1net1: Creation complete after 32s [id=urn:vcloud:network:b42924c2-f3a3-4d0f-be55-1afb4620ebf6]
vcd_nsxv_firewall_rule.edge1rule1: Creation complete after 54s [id=131075]
vcd_nsxv_snat.edge1snat1: Creation complete after 1m6s [id=196609]

Apply complete! Resources: 10 added, 0 changed, 0 destroyed.

After the input modification I had a plan that requires replacement of the vdc:

$ terraform plan -var-file=orgs/org-template_1.tfvars -out=orgs/org-template_1.plan
vcd_org.org1: Refreshing state... [id=urn:vcloud:org:58f8ad69-89a2-44d1-9341-fc573f9a8313]
vcd_org_user.org1-admin: Refreshing state... [id=urn:vcloud:user:d904f296-8632-4ee4-8ccc-c5b20a3ead3f]
vcd_org_vdc.vdc1: Refreshing state... [id=urn:vcloud:vdc:1d263d50-6ba1-44dc-a0aa-30ca6e794c56]
vcd_network_direct.bck_nets[1]: Refreshing state... [id=urn:vcloud:network:f8a9ff69-cadd-4718-8099-801bbd1acc62]
vcd_network_direct.bck_nets[0]: Refreshing state... [id=urn:vcloud:network:deda1a77-5117-4778-944d-957bd6b2aa45]
vcd_network_direct.dir_nets[0]: Refreshing state... [id=urn:vcloud:network:225c9bf9-286a-4d63-ab13-a7c7657ada16]
vcd_edgegateway.edge1: Refreshing state... [id=urn:vcloud:gateway:e9165d68-66af-435f-9ff8-bf74483b0311]
vcd_nsxv_snat.edge1snat1: Refreshing state... [id=196609]
vcd_nsxv_firewall_rule.edge1rule1: Refreshing state... [id=131075]
vcd_network_routed.edge1net1: Refreshing state... [id=urn:vcloud:network:b42924c2-f3a3-4d0f-be55-1afb4620ebf6]

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
-/+ destroy and then create replacement

Terraform will perform the following actions:

  # vcd_org_vdc.vdc1 must be replaced
-/+ resource "vcd_org_vdc" "vdc1" {
      + allow_over_commit           = (known after apply)
      + default_vm_sizing_policy_id = (known after apply)
      ~ elasticity                  = false -> (known after apply)
      - enable_vm_discovery         = false -> null
      ~ id                          = "urn:vcloud:vdc:1d263d50-6ba1-44dc-a0aa-30ca6e794c56" -> (known after apply)
      ~ include_vm_memory_overhead  = true -> (known after apply)
        name                        = "org-template_1_ALL_1"
      - nic_quota                   = 0 -> null
      - vm_quota                    = 0 -> null
      ~ vm_sizing_policy_ids        = [] -> (known after apply)
        # (15 unchanged attributes hidden)

      ~ compute_capacity {
          ~ cpu {
              ~ limit     = 8000 -> 0
              ~ reserved  = 4000 -> (known after apply)
              ~ used      = 0 -> (known after apply)
                # (1 unchanged attribute hidden)
            }

          ~ memory {
              ~ limit     = 32768 -> 0
              ~ reserved  = 16384 -> (known after apply)
              ~ used      = 0 -> (known after apply)
                # (1 unchanged attribute hidden)
            }
        }

      + storage_profile { # forces replacement
          + default            = false
          + enabled            = true
          + limit              = 51200
          + name               = "ALLOCATION-SSD VM Storage Profile"
          + storage_used_in_mb = (known after apply)
        }
      - storage_profile { # forces replacement
          - default            = true -> null
          - enabled            = true -> null
          - limit              = 102400 -> null
          - name               = "ALLOCATION-FC VM Storage Profile" -> null
          - storage_used_in_mb = 0 -> null
        }
      + storage_profile { # forces replacement
          + default            = true
          + enabled            = true
          + limit              = 102400
          + name               = "ALLOCATION-FC VM Storage Profile"
          + storage_used_in_mb = (known after apply)
        }
    }

Plan: 1 to add, 0 to change, 1 to destroy.

Expected Behavior

Storage profile is added in place to the org vdc

Actual Behavior

org vdc has been replaced, the other resources (edge, networks, ...) are disappeared.

$ terraform apply "orgs/org-template_1.plan"
vcd_org_vdc.vdc1: Destroying... [id=urn:vcloud:vdc:1d263d50-6ba1-44dc-a0aa-30ca6e794c56]
vcd_org_vdc.vdc1: Destruction complete after 20s
vcd_org_vdc.vdc1: Creating...
vcd_org_vdc.vdc1: Creation complete after 1m40s [id=urn:vcloud:vdc:405f7bbe-6279-419a-bc0c-a20e7ccda059]

Apply complete! Resources: 1 added, 0 changed, 1 destroyed.

The text was updated successfully, but these errors were encountered:

dataclouder · 2021-03-01T11:30:17Z

The storage_profile change requires a VDC rebuild. This was made on purpose, because we cannot remove existing profiles.
The way Terraform works, we can make a property modifiable (= does not require rebuild) or not. There is no intermediate state where we allow adding new components but not modify existing ones.
We may achieve such goal, but it would require careful coding outside the current infrastructure.

carmine73 · 2021-03-01T11:59:27Z

There is no intermediate state where we allow adding new components but not modify existing ones.

I can modify the limit of an existing storage profile without destroying the vdc, but I cannot add a new one

carmine73 · 2021-03-01T16:16:09Z

@dataclouder I have one more question:
why adding a storage profile the vdc replace is done using a delete with force=true&recursive=true?

DELETE https://vcloud-host/api/vdc/2ad474ad-b3ae-4611-8df8-af2873a42e7e?force=true&recursive=true

dataclouder · 2021-03-01T18:21:34Z

Replacing the VDC implies a deletion, and such deletion is not possible unless the VDC is emptied recursively.

carmine73 · 2021-03-02T08:20:28Z

Let me summarize:

if I have a complex infrastructure (org, vdc, edge, network) and I add a storage profile to the vdc, terraform will destroy the vdc force=true&recursive=true regardless the resources underneath
if I have a complex infrastructure (org, vdc, edge, network) and I try to destroy it but not all resources are managed by terraform, I cannot do it unless I import the whole infrastructure: destroying vdc is done using force=false&recursive=false even if the vdc has this values with:

        "delete_force": true,
        "delete_recursive": true,

Am I correct?

dataclouder · 2021-03-02T08:39:35Z

Sorry, I don't follow.
I think this issue has been defined, and we will consider it when we can.
If you wish to report another issue with VDC, please do so in a separate instance.

carmine73 · 2021-03-02T09:19:01Z

Sorry, you are right (forget the second bullet - I understand why it doesn't work as I was expecting).
Anyway I believe the actual behavior (force=true&recursive=true) is dangerous.

carmine73 · 2021-07-09T14:24:06Z

Any step forward on that?
A provider needs adding a Storage Profile to a "running" vDC without destroying it.

dataclouder · 2021-07-20T12:49:41Z

I am looking into it now.
The main modification will be to allow storage profiles to be modified. Right now, they force rebuild, as reported.

When we allow the storage profile to be modified, we have three cases:

Add a new storage profile: should not be a problem.
Remove a storage profile: this could be a problem if the storage profile was used for any object that belongs to the VDC. My plan is - rather than over-engineering and figure out if removal is possible - to attempt removal, and let the VCD complain if the storage profile cannot be removed.
Modify parameters of existing storage profiles: this is, sometimes, similar to n.2, when the changes affect existing objects. Also in this case, I will let the VCD report whether the change is compatible.

* Fix Issue vmware#648 adding a storage profile requires the vdc to be replaced * Fix Issue vmware#696 Catalog deletion failure returns as success Signed-off-by: Giuseppe Maxia <gmaxia@vmware.com>

* Fix Issue #648 adding a storage profile requires the vdc to be replaced * Fix Issue #696 Catalog deletion failure returns as success * Bump up version to 3.4.0 * Update CHANGELOG items Signed-off-by: Giuseppe Maxia <gmaxia@vmware.com>

dataclouder · 2021-08-06T07:46:13Z

Fixed in PR #698

carmine73 · 2021-08-30T10:25:25Z

it works!
thanks.

  # vcd_org_vdc.vdc1 will be updated in-place
  ~ resource "vcd_org_vdc" "vdc1" {
      + delete_force                = false
      + delete_recursive            = false
        id                          = "urn:vcloud:vdc:f242f773-7634-4678-8290-5d6eddeaff25"
        name                        = "org-testcol_1_ALL_1"
        # (20 unchanged attributes hidden)


      + storage_profile {
          + default            = false
          + enabled            = true
          + limit              = 7680
          + name               = "ALLOCATION-SSD VM Storage Profile"
          + storage_used_in_mb = (known after apply)
        }
        # (2 unchanged blocks hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

And it is also ok that removal or change are attempted to let VCD complain if the storage profile cannot be removed or modified.

Error: [VDC update] error updating storage profiles: [updateStorageProfiles] error removing storage profile ALLOCATION-SSD VM Storage Profile: task did not complete successfully:  [400:BAD_REQUEST] - [ 776dab5f-71db-4cb0-b407-6d15022c9bd5 ] Storage policy "ALLOCATION-SSD VM Storage Profile" cannot be deleted since it is currently in use.

carmine73 · 2021-09-24T12:15:40Z

@dataclouder any release date for 3.4.0 including this fix? thanks

dataclouder · 2021-09-24T12:18:52Z

It is merged to master. It will be released soon.

dataclouder added the wishlist label Mar 1, 2021

dataclouder self-assigned this Jul 20, 2021

dataclouder mentioned this issue Jul 27, 2021

Update vdc storage profiles vmware/go-vcloud-director#393

Merged

dataclouder mentioned this issue Jul 28, 2021

Fix storage profile update #698

Merged

dataclouder closed this as completed Aug 6, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

vcd_org_vdc: adding a storage profile requires the vdc to be replaced #648

vcd_org_vdc: adding a storage profile requires the vdc to be replaced #648

carmine73 commented Mar 1, 2021

dataclouder commented Mar 1, 2021

carmine73 commented Mar 1, 2021

carmine73 commented Mar 1, 2021 •

edited

Loading

dataclouder commented Mar 1, 2021

carmine73 commented Mar 2, 2021

dataclouder commented Mar 2, 2021

carmine73 commented Mar 2, 2021 •

edited

Loading

carmine73 commented Jul 9, 2021 •

edited

Loading

dataclouder commented Jul 20, 2021

dataclouder commented Aug 6, 2021

carmine73 commented Aug 30, 2021 •

edited

Loading

carmine73 commented Sep 24, 2021

dataclouder commented Sep 24, 2021

vcd_org_vdc: adding a storage profile requires the vdc to be replaced #648

vcd_org_vdc: adding a storage profile requires the vdc to be replaced #648

Comments

carmine73 commented Mar 1, 2021

Terraform Version

Affected Resource(s)

Terraform Configuration Files

Steps to Reproduce

Debug Output

Expected Behavior

Actual Behavior

dataclouder commented Mar 1, 2021

carmine73 commented Mar 1, 2021

carmine73 commented Mar 1, 2021 • edited Loading

dataclouder commented Mar 1, 2021

carmine73 commented Mar 2, 2021

dataclouder commented Mar 2, 2021

carmine73 commented Mar 2, 2021 • edited Loading

carmine73 commented Jul 9, 2021 • edited Loading

dataclouder commented Jul 20, 2021

dataclouder commented Aug 6, 2021

carmine73 commented Aug 30, 2021 • edited Loading

carmine73 commented Sep 24, 2021

dataclouder commented Sep 24, 2021

carmine73 commented Mar 1, 2021 •

edited

Loading

carmine73 commented Mar 2, 2021 •

edited

Loading

carmine73 commented Jul 9, 2021 •

edited

Loading

carmine73 commented Aug 30, 2021 •

edited

Loading