Add machine type availability checks to slurm-gcp-v6-nodeset

[annuay-google]

Issue

Blueprint yaml files allow users to specify machine and zone combinations that do not exist. Infra is provisioned by Terraform during ./ghpc deploy , but autoscaling may fail later if capacity is not found by bulk insert APIs in zones specified

Approach

Created a Terraform precondition to verify that the machine type is available in at least one zone. In case they're not, there is no feasible way for bulk insert to allocate machines, and terraform will exit during plan/apply

Testing

Ran ./ghpc create, ./ghpc deploy on this configuration. The specified machine type does not exist in both zones us-central1-b and us-central1-c. Verified that Terraform exits and error message is as expected

config:

vars:
  deployment_name: hpc-slurm
  region: us-central1
  zone: us-central1-b

deployment_groups:
- group: primary
  modules:
  - id: h3_nodeset
    source: community/modules/compute/schedmd-slurm-gcp-v6-nodeset
    use: [network]
    settings:
      node_count_dynamic_max: 20
      # Note that H3 is available in only specific zones. https://cloud.google.com/compute/docs/regions-zones
      machine_type: h3-standard-88
      # H3 does not support pd-ssd and pd-standard
      # https://cloud.google.com/compute/docs/compute-optimized-machines#h3_disks
      disk_type: pd-balanced
      bandwidth_tier: gvnic_enabled
      allow_automatic_updates: false
      zones:
      - us-central1-c

output:

Testing if deployment group hpc-slurm/primary requires adding or changing cloud infrastructure
Error: exit status 1

Error: Resource precondition failed

  on modules/embedded/community/modules/compute/schedmd-slurm-gcp-v6-nodeset/main.tf line 193, in resource "terraform_data" "machine_type_zone_validation":
 193:       condition     = length(local.zones_with_machine_type) > 0
    ├────────────────
    │ local.zones_with_machine_type is empty tuple

machine type h3-standard-88 is not available in any of the zones
["us-central1-b","us-central1-c"]. To list zones in which it is available, run:

gcloud compute machine-types list --filter="name=h3-standard-88"

Hint: terraform plan for deployment group hpc-slurm/primary failed

Added 1 valid zone (us-central1-a) and verified that infra is correctly provisioned

Submission Checklist

Please take the following actions before submitting this pull request.

• Fork your PR branch from the Toolkit "develop" branch (not main)
• Test all changes with pre-commit in a local branch #
• Confirm that "make tests" passes all tests
• Add or modify unit tests to cover code changes
• Ensure that unit test coverage remains above 80%
• Update all applicable documentation
• Follow Cluster Toolkit Contribution guidelines #

The boolean logic in the check block does not enforce the correct behavior.

[tpdownes]

To make the terraform_data resource and precondition useful, it should be inserted into the terraform resource graph.

[tpdownes]

Please also change versions.tf to require terraform 1.4

[annuay-google]

Please also change versions.tf to require terraform 1.4

Done

[tpdownes]

Please add one final helpful hint to the user. Then squash your commits and this should be ready!

[annuay-google]

Please add one final helpful hint to the user. Then squash your commits and this should be ready!

Done. Verified the error as well

machine type h3-standard-88 is not available in any of the zones
["us-central1-b"]". To list zones in which it is available, run:

gcloud compute machine-types list --filter="name=h3-standard-88"

[tpdownes]

This is a big improvement in user experience. Thank you for the submission!