Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unable to bootstrap cluster #156

Open
petterroea opened this issue Apr 1, 2024 · 10 comments
Open

Unable to bootstrap cluster #156

petterroea opened this issue Apr 1, 2024 · 10 comments

Comments

@petterroea
Copy link

Hello

When using this terraform provider, I am unable to bootstrap my cluster. However, if I output the generated talosconfig and use talosctl manually, I am able to bootstrap with no problem. I am running bare-metal on proxmox.

Terraform stuff:

resource "talos_machine_secrets" "secrets" {}

resource "talos_machine_configuration_apply" "control_planes" {
  depends_on = [
    proxmox_vm_qemu.talos-control
  ]

  for_each = data.talos_machine_configuration.machine_configurations

  client_configuration        = talos_machine_secrets.secrets.client_configuration
  machine_configuration_input = each.value.machine_configuration
  node                        = var.control_plane_nodes[each.key].expected_ip
}

resource "talos_machine_bootstrap" "bootstrap" {
  depends_on = [
    talos_machine_configuration_apply.control_planes
  ]

  node                 = var.endpoint_ip

  client_configuration = talos_machine_secrets.secrets.client_configuration
}

And some variables:

variable cluster_name {
    type = string
    default = "homecluster"
}
variable dns_domain {
    type = string
    default = "cluster.local"
}

variable endpoint_ip {
    type = string
    default = "10.0.5.60"
}

Logs from attempting to bootstrap:

talos_machine_bootstrap.bootstrap: Creating...
2024-04-01T06:47:52.599Z [TRACE] terraform.contextPlugins: Schema for provider "registry.terraform.io/siderolabs/talos" is in the global cache
2024-04-01T06:47:52.599Z [INFO]  Starting apply for talos_machine_bootstrap.bootstrap
2024-04-01T06:47:52.600Z [TRACE] terraform.contextPlugins: Schema for provider "registry.terraform.io/siderolabs/talos" is in the global cache
2024-04-01T06:47:52.600Z [TRACE] terraform.contextPlugins: Schema for provider "registry.terraform.io/siderolabs/talos" is in the global cache
2024-04-01T06:47:52.600Z [DEBUG] skipping FixUpBlockAttrs
2024-04-01T06:47:52.601Z [TRACE] terraform.contextPlugins: Schema for provider "registry.terraform.io/siderolabs/talos" is in the global cache
2024-04-01T06:47:52.601Z [DEBUG] talos_machine_bootstrap.bootstrap: applying the planned Create change
2024-04-01T06:47:52.601Z [TRACE] GRPCProvider.v6: ApplyResourceChange
2024-04-01T06:47:52.601Z [TRACE] GRPCProvider.v6: GetProviderSchema
2024-04-01T06:47:52.601Z [TRACE] provider.terraform-provider-talos_v0.2.0: Received request: tf_req_id=8fc0c9e8-13ec-3943-a763-8774afca0305 tf_rpc=ApplyResourceChange @caller=github.com/hashicorp/terraform-plugin-go@v0.15.0/tfprotov6/tf6server/server.go:803 @module=sdk.proto tf_proto_version=6.3 tf_provider_addr=registry.terraform.io/siderolabs/talos tf_resource_type=talos_machine_bootstrap timestamp=2024-04-01T06:47:52.601Z
2024-04-01T06:47:52.601Z [TRACE] provider.terraform-provider-talos_v0.2.0: Sending request downstream: tf_rpc=ApplyResourceChange tf_req_id=8fc0c9e8-13ec-3943-a763-8774afca0305 tf_resource_type=talos_machine_bootstrap @module=sdk.proto tf_proto_version=6.3 tf_provider_addr=registry.terraform.io/siderolabs/talos @caller=github.com/hashicorp/terraform-plugin-go@v0.15.0/tfprotov6/internal/tf6serverlogging/downstream_request.go:17 timestamp=2024-04-01T06:47:52.601Z
2024-04-01T06:47:52.601Z [TRACE] provider.terraform-provider-talos_v0.2.0: Checking ResourceTypes lock: tf_req_id=8fc0c9e8-13ec-3943-a763-8774afca0305 @module=sdk.framework tf_resource_type=talos_machine_bootstrap tf_rpc=ApplyResourceChange @caller=github.com/hashicorp/terraform-plugin-framework@v1.2.0/internal/fwserver/server.go:339 tf_provider_addr=registry.terraform.io/siderolabs/talos timestamp=2024-04-01T06:47:52.601Z
2024-04-01T06:47:52.602Z [TRACE] provider.terraform-provider-talos_v0.2.0: Checking ResourceSchemas lock: @module=sdk.framework tf_req_id=8fc0c9e8-13ec-3943-a763-8774afca0305 tf_resource_type=talos_machine_bootstrap tf_rpc=ApplyResourceChange @caller=github.com/hashicorp/terraform-plugin-framework@v1.2.0/internal/fwserver/server.go:413 tf_provider_addr=registry.terraform.io/siderolabs/talos timestamp=2024-04-01T06:47:52.601Z
2024-04-01T06:47:52.602Z [TRACE] provider.terraform-provider-talos_v0.2.0: ApplyResourceChange received no PriorState, running CreateResource: @module=sdk.framework tf_provider_addr=registry.terraform.io/siderolabs/talos tf_resource_type=talos_machine_bootstrap @caller=github.com/hashicorp/terraform-plugin-framework@v1.2.0/internal/fwserver/server_applyresourcechange.go:42 tf_req_id=8fc0c9e8-13ec-3943-a763-8774afca0305 tf_rpc=ApplyResourceChange timestamp=2024-04-01T06:47:52.602Z
2024-04-01T06:47:52.602Z [DEBUG] provider.terraform-provider-talos_v0.2.0: Calling provider defined Resource Create: @caller=github.com/hashicorp/terraform-plugin-framework@v1.2.0/internal/fwserver/server_createresource.go:96 @module=sdk.framework tf_req_id=8fc0c9e8-13ec-3943-a763-8774afca0305 tf_resource_type=talos_machine_bootstrap tf_provider_addr=registry.terraform.io/siderolabs/talos tf_rpc=ApplyResourceChange timestamp=2024-04-01T06:47:52.602Z
2024-04-01T06:47:52.602Z [INFO]  provider.terraform-provider-talos_v0.2.0: create timeout configuration not found, using provided default: tf_rpc=ApplyResourceChange @caller=github.com/hashicorp/terraform-plugin-framework-timeouts@v0.3.1/resource/timeouts/timeouts.go:105 tf_provider_addr=registry.terraform.io/siderolabs/talos tf_req_id=8fc0c9e8-13ec-3943-a763-8774afca0305 tf_resource_type=talos_machine_bootstrap @module=talos timestamp=2024-04-01T06:47:52.602Z
2024-04-01T06:47:52.602Z [DEBUG] provider.terraform-provider-talos_v0.2.0: 2024/04/01 06:47:52 [DEBUG] Waiting for state to become: [success]
2024-04-01T06:47:52.603Z [DEBUG] provider.terraform-provider-talos_v0.2.0: 2024/04/01 06:47:52 
[TRACE] Waiting 500ms before next try
2024-04-01T06:47:53.106Z [DEBUG] provider.terraform-provider-talos_v0.2.0: 2024/04/01 06:47:53 [TRACE] Waiting 1s before next try
2024-04-01T06:47:54.107Z [DEBUG] provider.terraform-provider-talos_v0.2.0: 2024/04/01 06:47:54 [TRACE] Waiting 2s before next try
2024-04-01T06:47:55.226Z [TRACE] dag/walk: vertex "provider[\"registry.terraform.io/siderolabs/talos\"] (close)" is waiting for "talos_machine_bootstrap.bootstrap"
2024-04-01T06:47:56.109Z [DEBUG] provider.terraform-provider-talos_v0.2.0: 2024/04/01 06:47:56 [TRACE] Waiting 4s before next try
2024-04-01T06:47:57.272Z [TRACE] dag/walk: vertex "root" is waiting for "provider[\"registry.terraform.io/siderolabs/talos\"] (close)"
2024-04-01T06:48:00.111Z [DEBUG] provider.terraform-provider-talos_v0.2.0: 2024/04/01 06:48:00 [TRACE] Waiting 8s before next try
2024-04-01T06:48:00.226Z [TRACE] dag/walk: vertex "provider[\"registry.terraform.io/siderolabs/talos\"] (close)" is waiting for "talos_machine_bootstrap.bootstrap"
2024-04-01T06:48:02.273Z [TRACE] dag/walk: vertex "root" is waiting for "provider[\"registry.terraform.io/siderolabs/talos\"] (close)"
talos_machine_bootstrap.bootstrap: Still creating... [10s elapsed]
2024-04-01T06:48:05.228Z [TRACE] dag/walk: vertex "provider[\"registry.terraform.io/siderolabs/talos\"] (close)" is waiting for "talos_machine_bootstrap.bootstrap"
2024-04-01T06:48:07.275Z [TRACE] dag/walk: vertex "root" is waiting for "provider[\"registry.terraform.io/siderolabs/talos\"] (close)"
2024-04-01T06:48:08.119Z [DEBUG] provider.terraform-provider-talos_v0.2.0: 2024/04/01 06:48:08 [TRACE] Waiting 10s before next try
2024-04-01T06:48:10.230Z [TRACE] dag/walk: vertex "provider[\"registry.terraform.io/siderolabs/talos\"] (close)" is waiting for "talos_machine_bootstrap.bootstrap"
2024-04-01T06:48:12.277Z [TRACE] dag/walk: vertex "root" is waiting for "provider[\"registry.terraform.io/siderolabs/talos\"] (close)"
talos_machine_bootstrap.bootstrap: Still creating... [20s elapsed]

What steps can i take from here to help troubleshoot? This has already cost me a day so I need to get on with running my cluster by manually bootstrapping, but I'd love to help fix this issue.
@frezbo
Copy link
Member

frezbo commented Apr 1, 2024

It's probably due to using wrong ip for the node? Could you let TF wait until the bootstrap times out, or you could set a small timeout and gather the error. Without error there's not much info to proceed.

@petterroea
Copy link
Author

Hello, thank you for your prompt response.

I let TF wait for the bootstrap to time out, here is the result:

talos_machine_bootstrap.bootstrap: Still creating... [9m40s elapsed]
2024-04-01T09:00:02.198Z [TRACE] dag/walk: vertex "root" is waiting for "provider[\"registry.terraform.io/siderolabs/talos\"] (close)"
2024-04-01T09:00:02.199Z [TRACE] dag/walk: vertex "provider[\"registry.terraform.io/siderolabs/talos\"] (close)" is waiting for "talos_machine_bootstrap.bootstrap"
2024-04-01T09:00:02.765Z [DEBUG] provider.terraform-provider-talos_v0.2.0: 2024/04/01 09:00:02 [TRACE] Waiting 10s before next try
2024-04-01T09:00:07.200Z [TRACE] dag/walk: vertex "provider[\"registry.terraform.io/siderolabs/talos\"] (close)" is waiting for "talos_machine_bootstrap.bootstrap"
2024-04-01T09:00:07.200Z [TRACE] dag/walk: vertex "root" is waiting for "provider[\"registry.terraform.io/siderolabs/talos\"] (close)"
talos_machine_bootstrap.bootstrap: Still creating... [9m50s elapsed]
2024-04-01T09:00:12.201Z [TRACE] dag/walk: vertex "root" is waiting for "provider[\"registry.terraform.io/siderolabs/talos\"] (close)"
2024-04-01T09:00:12.201Z [TRACE] dag/walk: vertex "provider[\"registry.terraform.io/siderolabs/talos\"] (close)" is waiting for "talos_machine_bootstrap.bootstrap"
2024-04-01T09:00:12.774Z [DEBUG] provider.terraform-provider-talos_v0.2.0: 2024/04/01 09:00:12 [TRACE] Waiting 10s before next try
2024-04-01T09:00:17.206Z [TRACE] dag/walk: vertex "provider[\"registry.terraform.io/siderolabs/talos\"] (close)" is waiting for "talos_machine_bootstrap.bootstrap"
2024-04-01T09:00:17.206Z [TRACE] dag/walk: vertex "root" is waiting for "provider[\"registry.terraform.io/siderolabs/talos\"] (close)"
2024-04-01T09:00:21.294Z [DEBUG] provider.terraform-provider-talos_v0.2.0: 2024/04/01 09:00:21 [WARN] WaitForState timeout after 10m0s
2024-04-01T09:00:21.294Z [DEBUG] provider.terraform-provider-talos_v0.2.0: 2024/04/01 09:00:21 [WARN] WaitForState starting 30s refresh grace period
2024-04-01T09:00:21.294Z [DEBUG] provider.terraform-provider-talos_v0.2.0: 2024/04/01 09:00:21 [ERROR] Context cancelation detected, abandoning grace period
2024-04-01T09:00:21.294Z [DEBUG] provider.terraform-provider-talos_v0.2.0: Called provider defined Resource Create: tf_provider_addr=registry.terraform.io/siderolabs/talos tf_req_id=a8d8ce18-ae04-db56-bf56-8fffe59fc345 tf_resource_type=talos_machine_bootstrap tf_rpc=ApplyResourceChange @module=sdk.framework @caller=github.com/hashicorp/terraform-plugin-framework@v1.2.0/internal/fwserver/server_createresource.go:98 timestamp=2024-04-01T09:00:21.294Z
2024-04-01T09:00:21.294Z [TRACE] provider.terraform-provider-talos_v0.2.0: Received downstream response: tf_proto_version=6.3 diagnostic_error_count=1 tf_req_id=a8d8ce18-ae04-db56-bf56-8fffe59fc345 tf_req_duration_ms=600008 @caller=github.com/hashicorp/terraform-plugin-go@v0.15.0/tfprotov6/internal/tf6serverlogging/downstream_request.go:37 diagnostic_warning_count=0 tf_provider_addr=registry.terraform.io/siderolabs/talos @module=sdk.proto tf_resource_type=talos_machine_bootstrap tf_rpc=ApplyResourceChange timestamp=2024-04-01T09:00:21.294Z
2024-04-01T09:00:21.294Z [ERROR] provider.terraform-provider-talos_v0.2.0: Response contains error diagnostic: diagnostic_severity=ERROR tf_provider_addr=registry.terraform.io/siderolabs/talos @caller=github.com/hashicorp/terraform-plugin-go@v0.15.0/tfprotov6/internal/diag/diagnostics.go:55 @module=sdk.proto diagnostic_summary="Error bootstrapping node" diagnostic_detail="rpc error: code = Canceled desc = grpc: the client connection is closing" tf_proto_version=6.3 tf_req_id=a8d8ce18-ae04-db56-bf56-8fffe59fc345 tf_rpc=ApplyResourceChange tf_resource_type=talos_machine_bootstrap timestamp=2024-04-01T09:00:21.294Z
2024-04-01T09:00:21.295Z [TRACE] provider.terraform-provider-talos_v0.2.0: Served request: @module=sdk.proto tf_resource_type=talos_machine_bootstrap tf_proto_version=6.3 tf_provider_addr=registry.terraform.io/siderolabs/talos tf_req_id=a8d8ce18-ae04-db56-bf56-8fffe59fc345 tf_rpc=ApplyResourceChange @caller=github.com/hashicorp/terraform-plugin-go@v0.15.0/tfprotov6/tf6server/server.go:829 timestamp=2024-04-01T09:00:21.294Z
2024-04-01T09:00:21.296Z [TRACE] maybeTainted: talos_machine_bootstrap.bootstrap encountered an error during creation, so it is now marked as tainted
2024-04-01T09:00:21.296Z [TRACE] terraform.contextPlugins: Schema for provider "registry.terraform.io/siderolabs/talos" is in the global cache
2024-04-01T09:00:21.296Z [TRACE] NodeAbstractResouceInstance.writeResourceInstanceState to workingState for talos_machine_bootstrap.bootstrap
2024-04-01T09:00:21.296Z [TRACE] NodeAbstractResouceInstance.writeResourceInstanceState: removing state object for talos_machine_bootstrap.bootstrap
2024-04-01T09:00:21.296Z [TRACE] evalApplyProvisioners: talos_machine_bootstrap.bootstrap is tainted, so skipping provisioning
2024-04-01T09:00:21.296Z [TRACE] maybeTainted: talos_machine_bootstrap.bootstrap was already tainted, so nothing to do
2024-04-01T09:00:21.296Z [TRACE] terraform.contextPlugins: Schema for provider "registry.terraform.io/siderolabs/talos" is in the global cache
2024-04-01T09:00:21.296Z [TRACE] NodeAbstractResouceInstance.writeResourceInstanceState to workingState for talos_machine_bootstrap.bootstrap
2024-04-01T09:00:21.296Z [TRACE] NodeAbstractResouceInstance.writeResourceInstanceState: removing state object for talos_machine_bootstrap.bootstrap
2024-04-01T09:00:21.297Z [TRACE] statemgr.Filesystem: have already backed up original terraform.tfstate to terraform.tfstate.backup on a previous write
2024-04-01T09:00:21.302Z [TRACE] statemgr.Filesystem: state has changed since last snapshot, so incrementing serial to 363
2024-04-01T09:00:21.302Z [TRACE] statemgr.Filesystem: writing snapshot at terraform.tfstate
2024-04-01T09:00:21.335Z [ERROR] vertex "talos_machine_bootstrap.bootstrap" error: Error bootstrapping node
2024-04-01T09:00:21.335Z [TRACE] vertex "talos_machine_bootstrap.bootstrap": visit complete, with errors
2024-04-01T09:00:21.335Z [TRACE] dag/walk: upstream of "provider[\"registry.terraform.io/siderolabs/talos\"] (close)" errored, so skipping
2024-04-01T09:00:21.335Z [TRACE] dag/walk: upstream of "root" errored, so skipping
2024-04-01T09:00:21.335Z [TRACE] statemgr.Filesystem: have already backed up original terraform.tfstate to terraform.tfstate.backup on a previous write
2024-04-01T09:00:21.341Z [TRACE] statemgr.Filesystem: state has changed since last snapshot, so incrementing serial to 364
2024-04-01T09:00:21.341Z [TRACE] statemgr.Filesystem: writing snapshot at terraform.tfstate

It is worth noting that if i bootstrap manually, the node in question changes state from booting to running immediately, and there is immediately a lot of console output. This never happens when bootstrapping through terraform.

endpoint_ip should be correct. I am able to ping it from the same box I run terraform on, and this is what the talos monitor shows me:

image

@frezbo
Copy link
Member

frezbo commented Apr 1, 2024

Not sure why the logs say this:

 evalApplyProvisioners: talos_machine_bootstrap.bootstrap is tainted, so skipping provisioning

could you disable debug logs and just share the normal output?

Maybe also try setting node attribute to same as endpoint_ip

@petterroea
Copy link
Author

@frezbo Just setting node to 10.0.5.60, right? Will run with normal output.

@frezbo
Copy link
Member

frezbo commented Apr 1, 2024

@frezbo Just setting node to 10.0.5.60, right? Will run with normal output.

yes

@petterroea
Copy link
Author

Running.

@petterroea
Copy link
Author

Thank you for waiting:

talos_machine_secrets.secrets: Creating...
talos_machine_secrets.secrets: Creation complete after 0s [id=machine_secrets]
data.talos_client_configuration.cc: Reading...
data.talos_client_configuration.cc: Read complete after 0s [id=homecluster]
data.talos_machine_configuration.machine_configurations["1"]: Reading...
data.talos_machine_configuration.machine_configurations["0"]: Reading...
data.talos_machine_configuration.machine_configurations["2"]: Reading...
data.talos_machine_configuration.machine_configurations["1"]: Read complete after 0s [id=homecluster]
data.talos_machine_configuration.machine_configurations["0"]: Read complete after 0s [id=homecluster]
data.talos_machine_configuration.machine_configurations["2"]: Read complete after 0s [id=homecluster]
proxmox_vm_qemu.talos-control["2"]: Creating...
proxmox_vm_qemu.talos-control["1"]: Creating...
proxmox_vm_qemu.talos-control["0"]: Creating...
proxmox_vm_qemu.talos-control["2"]: Still creating... [10s elapsed]
proxmox_vm_qemu.talos-control["1"]: Still creating... [10s elapsed]
proxmox_vm_qemu.talos-control["0"]: Still creating... [10s elapsed]
proxmox_vm_qemu.talos-control["1"]: Creation complete after 11s [id=sanae/qemu/104]
proxmox_vm_qemu.talos-control["2"]: Creation complete after 11s [id=yumi/qemu/102]
proxmox_vm_qemu.talos-control["0"]: Creation complete after 11s [id=yumi/qemu/103]
talos_machine_configuration_apply.control_planes["1"]: Creating...
talos_machine_configuration_apply.control_planes["0"]: Creating...
talos_machine_configuration_apply.control_planes["2"]: Creating...
talos_machine_configuration_apply.control_planes["2"]: Still creating... [10s elapsed]
talos_machine_configuration_apply.control_planes["1"]: Still creating... [10s elapsed]
talos_machine_configuration_apply.control_planes["0"]: Still creating... [10s elapsed]
talos_machine_configuration_apply.control_planes["0"]: Still creating... [20s elapsed]
talos_machine_configuration_apply.control_planes["1"]: Still creating... [20s elapsed]
talos_machine_configuration_apply.control_planes["2"]: Still creating... [20s elapsed]
talos_machine_configuration_apply.control_planes["1"]: Still creating... [30s elapsed]
talos_machine_configuration_apply.control_planes["2"]: Still creating... [30s elapsed]
talos_machine_configuration_apply.control_planes["0"]: Still creating... [30s elapsed]
talos_machine_configuration_apply.control_planes["1"]: Creation complete after 32s [id=machine_configuration_apply]
talos_machine_configuration_apply.control_planes["0"]: Creation complete after 32s [id=machine_configuration_apply]
talos_machine_configuration_apply.control_planes["2"]: Creation complete after 32s [id=machine_configuration_apply]
talos_machine_bootstrap.bootstrap: Creating...
talos_machine_bootstrap.bootstrap: Still creating... [10s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [20s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [31s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [41s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [51s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [1m1s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [1m11s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [1m21s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [1m31s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [1m41s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [1m51s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [2m1s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [2m11s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [2m21s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [2m31s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [2m41s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [2m51s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [3m1s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [3m11s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [3m21s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [3m31s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [3m41s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [3m51s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [4m1s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [4m11s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [4m21s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [4m31s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [4m41s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [4m51s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [5m1s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [5m11s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [5m21s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [5m31s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [5m41s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [5m51s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [6m1s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [6m11s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [6m21s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [6m31s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [6m41s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [6m51s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [7m1s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [7m11s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [7m21s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [7m31s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [7m41s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [7m51s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [8m1s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [8m11s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [8m21s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [8m31s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [8m41s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [8m51s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [9m1s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [9m11s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [9m21s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [9m31s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [9m41s elapsed]
talos_machine_bootstrap.bootstrap: Still creating... [9m51s elapsed]
╷
│ Warning: Qemu Guest Agent support is disabled from proxmox config.
│ 
│   with proxmox_vm_qemu.talos-control["1"],
│   on control-plane.tf line 2, in resource "proxmox_vm_qemu" "talos-control":
│    2: resource "proxmox_vm_qemu" "talos-control" {
│ 
│ Qemu Guest Agent support is required to make communications with the VM
│ 
│ (and 2 more similar warnings elsewhere)
╵
╷
│ Error: Error bootstrapping node
│ 
│   with talos_machine_bootstrap.bootstrap,
│   on talos_machine_config.tf line 45, in resource "talos_machine_bootstrap" "bootstrap":
│   45: resource "talos_machine_bootstrap" "bootstrap" {
│ 
│ rpc error: code = Canceled desc = grpc: the client connection is closing
╵

@petterroea
Copy link
Author

Here is the relevant plan diff:

  # talos_machine_configuration_apply.control_planes["0"] will be created
  + resource "talos_machine_configuration_apply" "control_planes" {
      + apply_mode                  = "auto"
      + client_configuration        = (known after apply)
      + endpoint                    = "10.0.5.60"
      + id                          = (known after apply)
      + machine_configuration       = (sensitive value)
      + machine_configuration_input = (sensitive value)
      + node                        = "10.0.5.60"
    }

  # talos_machine_configuration_apply.control_planes["1"] will be created
  + resource "talos_machine_configuration_apply" "control_planes" {
      + apply_mode                  = "auto"
      + client_configuration        = (known after apply)
      + endpoint                    = "10.0.5.61"
      + id                          = (known after apply)
      + machine_configuration       = (sensitive value)
      + machine_configuration_input = (sensitive value)
      + node                        = "10.0.5.61"
    }

  # talos_machine_configuration_apply.control_planes["2"] will be created
  + resource "talos_machine_configuration_apply" "control_planes" {
      + apply_mode                  = "auto"
      + client_configuration        = (known after apply)
      + endpoint                    = "10.0.5.62"
      + id                          = (known after apply)
      + machine_configuration       = (sensitive value)
      + machine_configuration_input = (sensitive value)
      + node                        = "10.0.5.62"
    }

  # talos_machine_secrets.secrets will be created
  + resource "talos_machine_secrets" "secrets" {
      + client_configuration = (known after apply)
      + id                   = (known after apply)
      + machine_secrets      = (known after apply)
      + talos_version        = "v1.4"
    }

@vdupain
Copy link

vdupain commented May 6, 2024

Have you verified that time is synchronzed?
I encountered this issue when deploying my cluster in WSL. WSL data/time was out of sync with Windows or the actual time.
See: https://www.talos.dev/v1.7/talos-guides/configuration/time-sync/

@petterroea
Copy link
Author

@vdupain time should be synchronized, but i will verify!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@petterroea @vdupain @frezbo