kubernetes_manifest getting NullVal instead of UnknownVal

[Phylu]

Terraform Version, Provider Version and Kubernetes Version

Terraform version: Terraform v1.4.5
Kubernetes provider version: registry.terraform.io/hashicorp/kubernetes v2.20.0
Kubernetes version: 1.26 (EKS)

Affected Resource(s)

• kubernetes_manifest

Terraform Configuration Files

# Assume a subnet created like this.
resource "aws_subnet" "example" {
  vpc_id            = aws_vpc.example.id
  availability_zone = "eu-central-1a"

  cidr_block = "100.64.0.0/20"

  secondary_subnets = {
    eu-central-1a = aws_subnet.example.id
  }

# The subnet is used as input for a module
module "kubernetes_manifest" {
  secondary_subnets = {
    eu-central-1a = aws_subnet.example.id
  }
}

# variables.tf of the module
variable "secondary_subnets" {
  type = object({
    eu-central-1a = string
  })
}

# Inside the module the following resource is created
resource "kubernetes_manifest" "vpc_cni_eni_configuration" {
  for_each = var.secondary_subnets

  manifest = {
    "apiVersion" = "crd.k8s.amazonaws.com/v1alpha1"
    "kind"       = "ENIConfig"
    "metadata" = {
      "name" = each.key
    }
    "spec" = {
      "securityGroups" = [module.cluster.node_security_group_id]
      "subnet"         = each.value
    }
  }

}

Debug Output

2023-05-17T12:02:10.831+0200 [DEBUG] provider.terraform-provider-kubernetes_v2.20.0_x5: 2023-05-17T12:02:10.831+0200 [DEBUG] [PlanUpdateResource]: OAPI type="tftypes.Object[\"apiVersion\":tftypes.String, \"kind\":tftypes.String, \"metadata\":tftypes.Object[\"name\":tftypes.String], \"spec\":tftypes.Object[\"securityGroups\":tftypes.Tuple[tftypes.String], \"subnet\":tftypes.String]]"
2023-05-17T12:02:10.831+0200 [DEBUG] provider.terraform-provider-kubernetes_v2.20.0_x5: 2023-05-17T12:02:10.831+0200 [DEBUG] [PlanUpdateResource]: OAPI type="tftypes.Object[\"apiVersion\":tftypes.String, \"kind\":tftypes.String, \"metadata\":tftypes.Object[\"name\":tftypes.String], \"spec\":tftypes.Object[\"securityGroups\":tftypes.Tuple[tftypes.String], \"subnet\":tftypes.String]]"
2023-05-17T12:02:10.831+0200 [DEBUG] provider.terraform-provider-kubernetes_v2.20.0_x5: 2023-05-17T12:02:10.831+0200 [DEBUG] [PlanResourceChange]: morphed manifest="tftypes.Object[\"apiVersion\":tftypes.String, \"kind\":tftypes.String, \"metadata\":tftypes.Object[\"name\":tftypes.String], \"spec\":tftypes.Object[\"securityGroups\":tftypes.Tuple[tftypes.String], \"subnet\":tftypes.String]]<\"apiVersion\":tftypes.String<\"crd.k8s.amazonaws.com/v1alpha1\">, \"kind\":tftypes.String<\"ENIConfig\">, \"metadata\":tftypes.Object[\"name\":tftypes.String]<\"name\":tftypes.String<\"eu-central-1a\">>, \"spec\":tftypes.Object[\"securityGroups\":tftypes.Tuple[tftypes.String], \"subnet\":tftypes.String]<\"securityGroups\":tftypes.Tuple[tftypes.String]<tftypes.String<\"sg-0ae34d6ae24e3a7fb\">>, \"subnet\":tftypes.String<null>>>"
2023-05-17T12:02:10.831+0200 [DEBUG] provider.terraform-provider-kubernetes_v2.20.0_x5: 2023-05-17T12:02:10.831+0200 [DEBUG] [PlanResourceChange]: morphed manifest="tftypes.Object[\"apiVersion\":tftypes.String, \"kind\":tftypes.String, \"metadata\":tftypes.Object[\"name\":tftypes.String], \"spec\":tftypes.Object[\"securityGroups\":tftypes.Tuple[tftypes.String], \"subnet\":tftypes.String]]<\"apiVersion\":tftypes.String<\"crd.k8s.amazonaws.com/v1alpha1\">, \"kind\":tftypes.String<\"ENIConfig\">, \"metadata\":tftypes.Object[\"name\":tftypes.String]<\"name\":tftypes.String<\"eu-central-1c\">>, \"spec\":tftypes.Object[\"securityGroups\":tftypes.Tuple[tftypes.String], \"subnet\":tftypes.String]<\"securityGroups\":tftypes.Tuple[tftypes.String]<tftypes.String<\"sg-0ae34d6ae24e3a7fb\">>, \"subnet\":tftypes.String<\"subnet-01f6c3e642108c359\">>>"
2023-05-17T12:02:10.831+0200 [DEBUG] provider.terraform-provider-kubernetes_v2.20.0_x5: 2023-05-17T12:02:10.831+0200 [DEBUG] [PlanResourceChange]: backfilled manifest="tftypes.Object[\"apiVersion\":tftypes.String, \"kind\":tftypes.String, \"metadata\":tftypes.Object[\"name\":tftypes.String], \"spec\":tftypes.Object[\"securityGroups\":tftypes.Tuple[tftypes.String], \"subnet\":tftypes.String]]<\"apiVersion\":tftypes.String<\"crd.k8s.amazonaws.com/v1alpha1\">, \"kind\":tftypes.String<\"ENIConfig\">, \"metadata\":tftypes.Object[\"name\":tftypes.String]<\"name\":tftypes.String<\"eu-central-1c\">>, \"spec\":tftypes.Object[\"securityGroups\":tftypes.Tuple[tftypes.String], \"subnet\":tftypes.String]<\"securityGroups\":tftypes.Tuple[tftypes.String]<tftypes.String<\"sg-0ae34d6ae24e3a7fb\">>, \"subnet\":tftypes.String<\"subnet-01f6c3e642108c359\">>>"
2023-05-17T12:02:10.831+0200 [DEBUG] provider.terraform-provider-kubernetes_v2.20.0_x5: 2023-05-17T12:02:10.831+0200 [DEBUG] [PlanResourceChange]: creating object="tftypes.Object[\"apiVersion\":tftypes.String, \"kind\":tftypes.String, \"metadata\":tftypes.Object[\"name\":tftypes.String], \"spec\":tftypes.Object[\"securityGroups\":tftypes.Tuple[tftypes.String], \"subnet\":tftypes.String]]<\"apiVersion\":tftypes.String<\"crd.k8s.amazonaws.com/v1alpha1\">, \"kind\":tftypes.String<\"ENIConfig\">, \"metadata\":tftypes.Object[\"name\":tftypes.String]<\"name\":tftypes.String<\"eu-central-1c\">>, \"spec\":tftypes.Object[\"securityGroups\":tftypes.Tuple[tftypes.String], \"subnet\":tftypes.String]<\"securityGroups\":tftypes.Tuple[tftypes.String]<tftypes.String<\"sg-0ae34d6ae24e3a7fb\">>, \"subnet\":tftypes.String<\"subnet-01f6c3e642108c359\">>>"
2023-05-17T12:02:10.832+0200 [WARN]  provider.terraform-provider-kubernetes_v2.20.0_x5: Response contains warning diagnostic: @module=sdk.proto diagnostic_detail="We could not find an OpenAPI schema for this custom resource. Updates to this resource will cause a forced replacement." diagnostic_summary="This custom resource does not have an associated OpenAPI schema." tf_resource_type=kubernetes_manifest diagnostic_severity=WARNING tf_proto_version=5.3 tf_provider_addr=registry.terraform.io/hashicorp/kubernetes tf_req_id=b568f6d6-e3fe-4072-7d1e-27a474b1cf29 tf_rpc=PlanResourceChange @caller=github.com/hashicorp/terraform-plugin-go@v0.14.2/tfprotov5/internal/diag/diagnostics.go:57 timestamp=2023-05-17T12:02:10.831+0200
2023-05-17T12:02:10.832+0200 [DEBUG] provider.terraform-provider-kubernetes_v2.20.0_x5: 2023-05-17T12:02:10.831+0200 [DEBUG] [PlanResourceChange]: backfilled manifest="tftypes.Object[\"apiVersion\":tftypes.String, \"kind\":tftypes.String, \"metadata\":tftypes.Object[\"name\":tftypes.String], \"spec\":tftypes.Object[\"securityGroups\":tftypes.Tuple[tftypes.String], \"subnet\":tftypes.String]]<\"apiVersion\":tftypes.String<\"crd.k8s.amazonaws.com/v1alpha1\">, \"kind\":tftypes.String<\"ENIConfig\">, \"metadata\":tftypes.Object[\"name\":tftypes.String]<\"name\":tftypes.String<\"eu-central-1a\">>, \"spec\":tftypes.Object[\"securityGroups\":tftypes.Tuple[tftypes.String], \"subnet\":tftypes.String]<\"securityGroups\":tftypes.Tuple[tftypes.String]<tftypes.String<\"sg-0ae34d6ae24e3a7fb\">>, \"subnet\":tftypes.String<unknown>>>"
2023-05-17T12:02:10.832+0200 [DEBUG] provider.terraform-provider-kubernetes_v2.20.0_x5: 2023-05-17T12:02:10.831+0200 [DEBUG] [PlanResourceChange]: creating object="tftypes.Object[\"apiVersion\":tftypes.String, \"kind\":tftypes.String, \"metadata\":tftypes.Object[\"name\":tftypes.String], \"spec\":tftypes.Object[\"securityGroups\":tftypes.Tuple[tftypes.String], \"subnet\":tftypes.String]]<\"apiVersion\":tftypes.String<\"crd.k8s.amazonaws.com/v1alpha1\">, \"kind\":tftypes.String<\"ENIConfig\">, \"metadata\":tftypes.Object[\"name\":tftypes.String]<\"name\":tftypes.String<\"eu-central-1a\">>, \"spec\":tftypes.Object[\"securityGroups\":tftypes.Tuple[tftypes.String], \"subnet\":tftypes.String]<\"securityGroups\":tftypes.Tuple[tftypes.String]<tftypes.String<\"sg-0ae34d6ae24e3a7fb\">>, \"subnet\":tftypes.String<unknown>>>"
2023-05-17T12:02:10.833+0200 [ERROR] vertex "module.global-backend-tools-eks.kubernetes_manifest.vpc_cni_eni_configuration[\"eu-central-1a\"]" error: Provider produced invalid plan
2023-05-17T12:02:10.833+0200 [WARN]  provider.terraform-provider-kubernetes_v2.20.0_x5: Response contains warning diagnostic: diagnostic_detail="We could not find an OpenAPI schema for this custom resource. Updates to this resource will cause a forced replacement." tf_proto_version=5.3 tf_provider_addr=registry.terraform.io/hashicorp/kubernetes @caller=github.com/hashicorp/terraform-plugin-go@v0.14.2/tfprotov5/internal/diag/diagnostics.go:57 @module=sdk.proto diagnostic_severity=WARNING diagnostic_summary="This custom resource does not have an associated OpenAPI schema." tf_req_id=7d922182-1837-6cd7-f7bd-217729358677 tf_resource_type=kubernetes_manifest tf_rpc=PlanResourceChange timestamp=2023-05-17T12:02:10.832+0200

https://gist.github.com/Phylu/fef920fe27af4603859fb65cda53bb2c
(Unfortunately, I had to strip the debug for containing too much sensitive information. I hope that I have covered all the important output).

Panic Output

N/A

Steps to Reproduce

1. terraform plan

Expected Behavior

The kubernetes manifest is created.

Actual Behavior

The manifest cannot be created and the following error is shown:

│ Error: Provider produced invalid plan
│
│ Provider "registry.terraform.io/hashicorp/kubernetes" planned an invalid value for
│ module.example.kubernetes_manifest.vpc_cni_eni_configuration["eu-central-1a"].manifest: planned value
│ cty.ObjectVal(map[string]cty.Value{"apiVersion":cty.StringVal("crd.k8s.amazonaws.com/v1alpha1"), "kind":cty.StringVal("ENIConfig"),
│ "metadata":cty.ObjectVal(map[string]cty.Value{"name":cty.StringVal("eu-central-1a")}),
│ "spec":cty.ObjectVal(map[string]cty.Value{"securityGroups":cty.TupleVal([]cty.Value{cty.StringVal("sg-abcde")}), "subnet":cty.NullVal(cty.String)})}) does not match
│ config value cty.ObjectVal(map[string]cty.Value{"apiVersion":cty.StringVal("crd.k8s.amazonaws.com/v1alpha1"), "kind":cty.StringVal("ENIConfig"),
│ "metadata":cty.ObjectVal(map[string]cty.Value{"name":cty.StringVal("eu-central-1a")}),
│ "spec":cty.ObjectVal(map[string]cty.Value{"securityGroups":cty.TupleVal([]cty.Value{cty.StringVal("sg-abcde")}), "subnet":cty.UnknownVal(cty.String)})}).
│
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.

You can see that the provider expects the subnet id to be of type cty.UnknownVal(cty.String) however it receives an cty.NullVal(cty.String). As soon as the subnet has been created before this part of the terraform code is planned or when the subnet-id is hardcoded, the creation works as expected.

Important Factoids

References

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[github-actions]

Marking this issue as stale due to inactivity. If this issue receives no comments in the next 30 days it will automatically be closed. If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. This helps our maintainers find and focus on the active issues. Maintainers may also remove the stale label at their discretion. Thank you!

[awalther52]

I am experiencing the same exact issue right now.