helper/schema: Online/Network/API validation

[radeksimko]

Depends on hashicorp/terraform#15895
Or it's probably better if it hashicorp/terraform#15895 is implemented first.

---

It is certainly undesirable to perform any "slow" validation which requires network access by default in terraform validate, but there's still value in having such validation.

It can be opt-in for validate command and it can also (more importantly) run as part of plan.

The implementation can be very much similar to ValidateFunc, except that the interface needs access to provider's meta.

Example use cases in AWS provider:

"vpc_security_group_ids": {
	Type:     schema.TypeString,
	Optional: true,
	NetworkValidateFunc: func(k string, v, meta interface{}) (ws []string, es []error) {
		if hasEc2Classic(meta.(*AWSClient).supportedplatforms) {
			es = append(es, fmt.Errorf("Use security_groups (with SG names) in EC2 Classic-enabled region"))
			return
		}
		return
	},
},
"security_groups": {
	Type:     schema.TypeString,
	Optional: true,
	NetworkValidateFunc: func(k string, v, meta interface{}) (ws []string, es []error) {
		if !hasEc2Classic(meta.(*AWSClient).supportedplatforms) {
			es = append(es, fmt.Errorf("Use security_group_ids (with SG IDs) in VPC-enabled region"))
			return
		}
		return
	},
},

I'm not sure if NetworkValidateFunc is the best name, but this is rather a simple "reminder" rather than full-blown proposal with all answers.

Related: hashicorp/terraform-provider-aws#3897

[apparentlymart]

This sort of validation was one of the intended use-cases of the CustomizeDiff functionality, which runs as part of terraform plan and is therefore allowed to do network-sensitive validations.

The [ValidateValue] helper function allows this to be expressed in a more declarative way by function composition:

  CustomizeDiff: customdiff.All(
      customdiff.ValidateValue("security_groups", func (value, meta interface{}) error {
          if !hasEc2Classic(meta.(*AWSClient).supportedplatforms) {
              return fmt.Errorf("Use security_group_ids (with SG IDs) in VPC-enabled region"))
          }
      }),
      // ...
  ),

Of course, there are some ergonomic problems here. Most notably, the validation rule is expressed outside of the schema.Schema instance and this may not be easy to find in a resource with a very large schema.

Having a new schema-specific function for this seems fine to me, but from the perspective of Terraform Core I'd prefer to have the provider SDK do it as part of the existing Diff method (the same as CustomizeDiff) rather than introducing a new idea of "network validation". Once you've got the provider configured enough that it could implement network validation there isn't really any harm in running a full plan, even if you intend to ignore the result.

I think I'd approach this by using the very general CustomizeDiff for this sort of problem right now and then that experience will help us understand which parts of its functionality would be valuable to include as schema.Schema features too. For example, I think we can achieve the sort of validation you described here for hashicorp/terraform-provider-aws#3897 with the features we already have, and then convert to a different approach later.