Two NICs on a single VM : VM systematically replaced with primary_nic

[ArnaultMICHEL]

Terraform Version

Terraform v1.6.5
on linux_amd64
+ provider registry.terraform.io/hashicorp/tls v4.0.5
+ provider registry.terraform.io/outscale/outscale v0.11.0

Terraform Configuration Files

resource "outscale_vm" "myvm" {
...
  primary_nic {
    nic_id        = outscale_nic.nic01.nic_id
    device_number = "0"
  }

  nics {
    nic_id         = outscale_nic.myvm_nic02.nic_id
    device_number  = "1"
  }
...
}

Please find below a complete tf file to reproduce the issue

Output

the first terraform apply will create the VM.
But the next terraform apply with the same terraform code will replaced the VM.

$ terraform apply
tls_private_key.rsa_pkey: Refreshing state... [id=5cd5afd9779fcdf9a7b8d920b033e90ba15574ae]
outscale_net.vpc_poc: Refreshing state... [id=vpc-8f56698e]
outscale_keypair.keypair: Refreshing state... [id=keypair-myvm-test]
outscale_subnet.public_subnet: Refreshing state... [id=subnet-6cdf3e99]
outscale_security_group.sg_ssh: Refreshing state... [id=sg-a0ec4c11]
outscale_security_group_rule.allow_ssh_to_pocvm: Refreshing state... [id=sg-a0ec4c11]
outscale_nic.myvm_nic02: Refreshing state... [id=eni-4573bdb9]
outscale_nic.myvm_nic01: Refreshing state... [id=eni-14d90bde]
outscale_vm.myvm: Refreshing state... [id=i-65a32c47]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
-/+ destroy and then create replacement

Terraform will perform the following actions:

  # outscale_vm.myvm must be replaced
-/+ resource "outscale_vm" "myvm" {
...

      - nics { # forces replacement
...
          - description                = "myvm admin interface" -> null
          - device_number              = 0 -> null
 ...
        }
      - nics { # forces replacement
...
          - description                = "myvm service interface for VPN" -> null
          - device_number              = 1 -> null
... 
        }
      + nics { # forces replacement
...
          + device_number              = 1
...
        }

      - primary_nic {
...
          - device_number              = 0 -> null
...
        }
      + primary_nic {
...
          + device_number              = 0
...
        }

        # (1 unchanged block hidden)
    }

Plan: 1 to add, 0 to change, 1 to destroy.

Expected Behavior

I suggest two ways to fix it :

1. Do not recreate the VM on each terraform apply if mixing primary_nic with nics
2. Change the doc to use only nics blocks if a VM has multiple NICS ( and remove primary_nic block?)

Adding a code sample for "Create a VM with two NIC" in the documentation would be great +1

Actual Behavior

Will replace (destroy & recreate) the VM on each terraform apply, even if the terraform code didn't change

Steps to Reproduce

1. terraform init
2. terraform apply -> the vm will be created 🆗
3. terraform apply -> ⚠️ the vm will be systematically replaced ⚠️

Workaround

I found that it is working as expected using two nics instead of primary_nic + nics :

resource "outscale_vm" "myvm" {
  ...
  nics {
    nic_id        = outscale_nic.myvm_nic01.nic_id
    device_number = "0"
  }

  nics {
    nic_id         = outscale_nic.myvm_nic02.nic_id
    device_number  = "1"
  }
  ...
}

Documentation

the documentation for outscale provider invite the user/devops to use primary_nic with nics :
nics - (Optional) One or more NICs. If you specify this parameter, you must not specify the subnet_id and subregion_name parameters. To define a NIC as the primary network interface of the VM, use the primary_nic argument.

Complete terraform code to reproduce

load your secrets in env vars

export TF_VAR_outscale_access_key=XXXAKXXX
export TF_VAR_outscale_secret_key=XXXSKXXX

the apply the following main.tf file

terraform {
  required_providers {
    outscale = {
      source  = "outscale/outscale"
      version = "0.11.0"
    }
  }
  required_version = ">= 1.6.5"
}

provider "outscale" {
  access_key_id = var.outscale_access_key
  secret_key_id = var.outscale_secret_key
  region        = var.outscale_region
  endpoints {
    api  = "api.${var.outscale_region}.outscale.com"
  }
}

## Terraform providers inputs variables

variable "outscale_access_key" {
  type = string
}
variable "outscale_secret_key" {
  type = string
}
variable "outscale_region" {
  description = "Outscale region"
  type        = string
  default     = "eu-west-2"
}
variable "outscale_azs" {
  description = "Outscale availability zones"
  type        = list(string)
  default     = ["eu-west-2a", "eu-west-2b"]
}

### Network - VPC and subnet

resource "outscale_net" "vpc_poc" {
  ip_range = "10.0.0.0/16"
  tags {
    key   = "Name"
    value = "vpc_poc"
  }
}

resource "outscale_subnet" "public_subnet" {
    net_id   = outscale_net.vpc_poc.net_id
    ip_range = "10.0.1.0/24"
    subregion_name = var.outscale_azs[0]
  tags {
    key   = "Name"
    value = "poc_public_subnet"
  }
}

### NICs for VM 
resource "outscale_nic" "myvm_nic01" {
  description         = "myvm admin interface"
  subnet_id          = outscale_subnet.public_subnet.subnet_id
  security_group_ids = ["${outscale_security_group.sg_ssh.id}"]
  private_ips {
    is_primary = true
    private_ip = "10.0.1.10"
  }
  tags {
    key   = "Name"
    value = "nic1-myvm"
  }
}

resource "outscale_nic" "myvm_nic02" {
  description        = "myvm service interface for VPN"
  subnet_id          = outscale_subnet.public_subnet.subnet_id
  security_group_ids = ["${outscale_security_group.sg_ssh.id}"]
  private_ips {
    is_primary = true
    private_ip = "10.0.1.20"
  }
  tags {
    key   = "Name"
    value = "nic2-myvm"
  }
}

### SSH keypair

# generate a new RSA Keypair
resource "tls_private_key" "rsa_pkey" {
  algorithm = "RSA"
  rsa_bits  = "4096"
}

resource "outscale_keypair" "keypair" {
  keypair_name = "keypair-myvm-test"
  public_key   = "${tls_private_key.rsa_pkey.public_key_openssh}"
}

### VM 

resource "outscale_vm" "myvm" {

  image_id           = "ami-a3ca408c"
  vm_type            = "tinav5.c1r2p2"
  keypair_name       = outscale_keypair.keypair.keypair_name
  
  state              = "running"

  #placement_tenancy     = "default"
  nested_virtualization = false

  user_data             = "" #Not yet supported

  block_device_mappings {
    device_name = "/dev/sda1"
    bsu {
      volume_size = "20"
      volume_type = "standard"
      #iops        = 3000
      delete_on_vm_deletion = true
    }
  }

  # issue : Will replace (destroy & recreate) VM each time even if no terraform IaC code change
  primary_nic {
    nic_id        = outscale_nic.myvm_nic01.nic_id
    device_number = "0"
  }
  
  # Will work as expected, instead of the primary_nic block
  # nics {
  #   nic_id        = outscale_nic.myvm_nic01.nic_id
  #   device_number = "0"
  # }

  nics {
    nic_id         = outscale_nic.myvm_nic02.nic_id
    device_number  = "1"
  }

  tags {
        key   = "Name"
        value = "TerraformProviderBug"
  }

}

### FW rules
resource "outscale_security_group" "sg_ssh" {
  security_group_name = "secgroup-incoming-ssh"
  description = "SSH access to POC VM"
  net_id      = outscale_net.vpc_poc.net_id
}
resource "outscale_security_group_rule" "allow_ssh_to_pocvm" {
  flow              = "Inbound"
  security_group_id = outscale_security_group.sg_ssh.id
  rules {
    from_port_range = "22"
    to_port_range   = "22"
    ip_protocol     = "tcp"
    ip_ranges       = ["4.3.2.1/32"]
  }
}

[outscale-toa]

Hi @ArnaultMICHEL,
Thanks for reaching us, we are looking at your issue.

Best regards,

[outscale-toa]

Hi @ArnaultMICHEL,
your workaround:

I found that it is working as expected using two `nics` instead of `primary_nic` + `nics` :

resource "outscale_vm" "myvm" {
  ...
  nics {
    nic_id        = outscale_nic.myvm_nic01.nic_id
    device_number = "0"
  }

  nics {
    nic_id         = outscale_nic.myvm_nic02.nic_id
    device_number  = "1"
  }
  ...
}

It will work until adding an other nic through resource "outscale_nic_link" #376

I advise you to use:

...
resource "outscale_vm" "myvm" {
  ...
  primary_nic {
    nic_id        = outscale_nic.myvm_nic01.nic_id
    device_number = "0"
  }
  ...
}
...
resource "outscale_nic_link" "nic_link01" {                                                                                                                                                                                                                                  
    device_number = "1"                                                                                                                                                                                                                                                      
    vm_id         = outscale_vm.myvm.vm_id                                                                                                                                                                                                                                   
    nic_id        = outscale_nic.myvm_nic02.nic_id                                                                                                                                                                                                                           
}                                

We will fix this issues soon

Best regards,

[ArnaultMICHEL]

in my use case, i want to be 100% sure that the two NICs exists during the first boot of the OS (& cloudinit configuration operations).

So, from my understanding, this is how terraform manage dependencies with resources outscale_vm + outscale_nic_link (from your advise) :

1. terraform first try to create the ressource outscale_vm (otherwize outscale_vm.myvm.vm_id is not available)
2. then terraform will attach the NIC to the VM.

So when you use outscale_nic_link, the attachment could be processed after the VM boot, especially if you have a large number of resources managed by terraform. if it happen after cloudinit network configuration step, the interface won't be properly configured by cloudinit.

I simply want to avoid that and don't use outscale_nic_link

FYI, we faced a similar issue with disk/volume ressource, when we populate a large number of VMs and volumes.
sometimes (~20% to 30% of our VMs), the volume is attached after cloudinit disk managment step.
so the volume is not properly configured (FS formatting + add to /etc/fstab).

[outscale-toa]

Fixed in v0.12.0