Skip to content

Commit

Permalink
all: Add dynamic type, attribute, and function support (#931)
Browse files Browse the repository at this point in the history
* initial implementation of dynamic type and value

* fix doc string

* add dynamic defaults

* add plan modifier for dynamic

* add dynamic attribute, validator, and plan modifiers

* add provider and data source attributes and tests

* update comment

* add resource schema attribute test

* add provider metaschema

* add hook into attribute validation

* add hook into plan modification for resources

* add hook into dynamic default logic

* add hooks for dynamic semantic equality

* add function param and return definitions

* add function tests

* modify ToTerraformValue to always return DPT

* switch back

* make reflection update

* update dep

* use safe equal

* return attr.Type

* add todos and docs based on findings

* random doc cleanup

* doc fix

* fix for data path

* switch to tuples and add reflection support

* update docs

* add names to variadic param names

* update doc note

* update docs

* update docs

* add doc note

* add changelog

* delete metaschema and fix broken tests

* updated some docs

* switch changelog to breaking change

* update to tuple index

* update comment wording

* update doc

* add new maintainer note

* remove top part of comment

* add logic to look at value type for dynamic schemas

* add data value tests

* add dynamic get at path tests

* add dynamic get tests

* add path exist tests

* add get provider schema tests

* add comment to weird obj fix

* make changes to support reflection (get)

* function: Switch the representation of variadic arguments to `types.Tuple` (#923)

* switch to tuples and add reflection support

* update docs

* add names to variadic param names

* update doc note

* update docs

* update docs

* add doc note

* add changelog

* switch changelog to breaking change

* update to tuple index

* update comment wording

* update doc

* add new maintainer note

* remove top part of comment

* fixed the odd spacing in the comment :)

* add dynamic interface

* reflection test

* add list type update

* add list value updates

* small refactoring

* quick check

* revert list type and value

* add detailed comments about the lack of dynamic element type support

* add some object tests for good measure

* add tuple tests

* add validation logic for list attribute only

* add block and attribute validation for list

* function: Replace usage of diagnostics with function errors during execution of provider-defined functions (#925)

* Replacing function.RunResponse diagnostics with error

* Adding custom FunctionError

* Adding custom FunctionErrors

* Removing unneeded equateErrors gocmp option

* Switching to using convenience functions for adding errors to FunctionErrors

* Add copyright headers

* Refactor to use Error() method on function errors when converting to tfprotov5/6.FunctionError

* Adding documentation and testing for fwerrors types

* Formatting errors during conversion to tfprotov<5|6>.FunctionError

* Removing argument error and argument warning diagnostics

* Renaming field name for FunctionErrors from Error to Errors

* Modifying documentation to reflect that executing the Run() method of a provider-defined function returns FunctionErrors

* Remove references to AddArgumentError and AddArgumentWarning from diagnostics documentation

* Removing fwerror package and moving FunctionError to function package

* Refactoring to replace FunctionErrors slice with single FunctionError

* Bumping terraform-plugin-go to v0.22.0

* Removing unneeded DiagnosticWithFunctionArgument interface and implementation

* Altering function signature of ConcatFuncErrors

* Removing HasError method

* Updating docs

* Updates following code review

* Adding changelog entries

* Fix naming

* Update website/docs/plugin/framework/functions/errors.mdx

Co-authored-by: Austin Valle <austinvalle@gmail.com>

* Formatting

* Updates following code review

---------

Co-authored-by: Austin Valle <austinvalle@gmail.com>

* Update changelog

* diag: remove incorrect code (#935)

* update plugin-go dependency, fix errors and switch equal method

* function: Add validation for parameter name conflicts and update defaulting logic (#936)

* add validation and refactor defaulting logic

* add changelogs

* test fix

* Update website/docs/plugin/framework/functions/documentation.mdx

Co-authored-by: Brian Flad <bflad417@gmail.com>

* refactor the logic, tests and docs for defaulting

* Update website/docs/plugin/framework/functions/documentation.mdx

Co-authored-by: Benjamin Bennett <ben.bennett@hashicorp.com>

---------

Co-authored-by: Brian Flad <bflad417@gmail.com>
Co-authored-by: Benjamin Bennett <ben.bennett@hashicorp.com>

* resource/schema: Ensure invalid attribute default value errors are raised (#933)

Reference: #590
Reference: #930

Previously the logic handling attribute `Default` values would silently ignore any type errors, which would lead to confusing planning data behaviors. This updates the logic to raise those error properly and adds covering unit testing.

These error messages are using the underlying `tftypes` type system errors which is currently a pragmatic compromise throughout various parts of the framework logic that bridges between both type systems to save additional type assertion logic and potential bugs relating to those conversions. In the future if the internal `tftypes` handling and exported fields are replaced with the framework type system types, this logic would instead return error messaging based on the framework type system errors.

This also will enhance the schema validation logic to check any `Default` response value and compare its type to the schema, which will raise framework type system errors during the `GetProviderSchema` RPC, or during schema unit testing if provider developers have implemented that additional testing.

* Update provider functions testing docs to help users avoid nil pointer error (#940)

* Fix capitalisation

* Update example test to mitigate nil pointer error

See #928

* Update website/docs/plugin/framework/functions/testing.mdx

Co-authored-by: Austin Valle <austinvalle@gmail.com>

* Update website/docs/plugin/framework/functions/testing.mdx

Co-authored-by: Austin Valle <austinvalle@gmail.com>

* Update website/docs/plugin/framework/functions/testing.mdx

Co-authored-by: Austin Valle <austinvalle@gmail.com>

* Update website/docs/plugin/framework/functions/testing.mdx

Co-authored-by: Austin Valle <austinvalle@gmail.com>

* Update website/docs/plugin/framework/functions/testing.mdx

Co-authored-by: Austin Valle <austinvalle@gmail.com>

---------

Co-authored-by: Austin Valle <austinvalle@gmail.com>
Co-authored-by: Brian Flad <bflad417@gmail.com>

* Reinstate go toolchain and add changelog for Go version bump to 1.21 (#937)

* Reinstate go toolchain and add changelog for Go version bump to 1.21

* Adding changelog

* Updating changelog

* Squashed commit of the following:

commit e7415b7
Author: Benjamin Bennett <ben.bennett@hashicorp.com>
Date:   Fri Mar 1 16:16:39 2024 +0000

    Reinstate go toolchain and add changelog for Go version bump to 1.21 (#937)

    * Reinstate go toolchain and add changelog for Go version bump to 1.21

    * Adding changelog

    * Updating changelog

commit 1597a95
Author: Sarah French <15078782+SarahFrench@users.noreply.github.com>
Date:   Fri Mar 1 16:12:22 2024 +0000

    Update provider functions testing docs to help users avoid nil pointer error (#940)

    * Fix capitalisation

    * Update example test to mitigate nil pointer error

    See #928

    * Update website/docs/plugin/framework/functions/testing.mdx

    Co-authored-by: Austin Valle <austinvalle@gmail.com>

    * Update website/docs/plugin/framework/functions/testing.mdx

    Co-authored-by: Austin Valle <austinvalle@gmail.com>

    * Update website/docs/plugin/framework/functions/testing.mdx

    Co-authored-by: Austin Valle <austinvalle@gmail.com>

    * Update website/docs/plugin/framework/functions/testing.mdx

    Co-authored-by: Austin Valle <austinvalle@gmail.com>

    * Update website/docs/plugin/framework/functions/testing.mdx

    Co-authored-by: Austin Valle <austinvalle@gmail.com>

    ---------

    Co-authored-by: Austin Valle <austinvalle@gmail.com>
    Co-authored-by: Brian Flad <bflad417@gmail.com>

commit bd22b58
Author: Brian Flad <bflad417@gmail.com>
Date:   Fri Mar 1 07:20:55 2024 -0500

    resource/schema: Ensure invalid attribute default value errors are raised (#933)

    Reference: #590
    Reference: #930

    Previously the logic handling attribute `Default` values would silently ignore any type errors, which would lead to confusing planning data behaviors. This updates the logic to raise those error properly and adds covering unit testing.

    These error messages are using the underlying `tftypes` type system errors which is currently a pragmatic compromise throughout various parts of the framework logic that bridges between both type systems to save additional type assertion logic and potential bugs relating to those conversions. In the future if the internal `tftypes` handling and exported fields are replaced with the framework type system types, this logic would instead return error messaging based on the framework type system errors.

    This also will enhance the schema validation logic to check any `Default` response value and compare its type to the schema, which will raise framework type system errors during the `GetProviderSchema` RPC, or during schema unit testing if provider developers have implemented that additional testing.

commit f03ca33
Author: Austin Valle <austinvalle@gmail.com>
Date:   Thu Feb 29 14:23:29 2024 -0500

    function: Add validation for parameter name conflicts and update defaulting logic (#936)

    * add validation and refactor defaulting logic

    * add changelogs

    * test fix

    * Update website/docs/plugin/framework/functions/documentation.mdx

    Co-authored-by: Brian Flad <bflad417@gmail.com>

    * refactor the logic, tests and docs for defaulting

    * Update website/docs/plugin/framework/functions/documentation.mdx

    Co-authored-by: Benjamin Bennett <ben.bennett@hashicorp.com>

    ---------

    Co-authored-by: Brian Flad <bflad417@gmail.com>
    Co-authored-by: Benjamin Bennett <ben.bennett@hashicorp.com>

* fix dynamic param

* incorrect merge conflict fix :)

* update comments from feedback

* refactor validation logic

* license headers

* implement remaining resource and datasource validation

* implement provider schema validation

* update error msg and add object validator

* add validation to object attributes

* update existing attributes to use new bool return

* add validation to function parameters and definitions

* refactor to only have one exported function

* add tuple tests for completeness

* create new fwtype package

* add parameter name to validate implementation definition

* various PR fixes

* add more docs

* fix docs from default param change

* update comment

* add more to reflection case + update comment

* comment wording

* remove todos and add package docs

* add changelogs

* Fix the use-case where dynamic value type is known, but the value itself is still null/unknown

* check for dynamic underlying value for null/unknown detection

* removed the unneccessary interface and removed half-implemented reflection support

* doc formatting

* update static default naming

* add happy path tests for datasource + provider validate implementations

* update definition validation messaging with variadic

* add doc explicitly saying that dynamic types aren't supported

* add docs mentioning required dynamic type to functions

* prevent nil panics in `types.Dynamic` implementation w/ tests

* proposal for `NewDynamicValue`

* add recommendations in doc string

* update block msg

* update all doc strings and error messages to make recommendations

* move the function validate interfaces to internal package

* add maintainer note about parameter name

* prevent attribute paths from stepping into dynamic types + new tests

* add helper methods for checking underlying value unknown/null

* add tests and comments about edge case with empty values

* move maintainer note

* Add dynamic type support documentation and considerations

* first pass at dynamic documentation

* Apply suggestions from code review

Co-authored-by: Brian Flad <bflad417@gmail.com>

* update literal representation in attribute page

* drop callouts and link to collection type page

* Update website/docs/plugin/framework/handling-data/attributes/dynamic.mdx

Co-authored-by: Brian Flad <bflad417@gmail.com>

* Apply suggestions from code review

Co-authored-by: Brian Flad <bflad417@gmail.com>

* Update website/docs/plugin/framework/functions/parameters/dynamic.mdx

Co-authored-by: Brian Flad <bflad417@gmail.com>

* Update website/docs/plugin/framework/handling-data/dynamic-data.mdx

Co-authored-by: Brian Flad <bflad417@gmail.com>

* include parameter name in example

* adjust the path documentation to reflect new changes

* rework the first paragraph of the data page

* add callout for null underlying element values

* add underlying value null/unknown information

* Update website/docs/plugin/framework/handling-data/types/dynamic.mdx

Co-authored-by: Brian Flad <bflad417@gmail.com>

* Update website/docs/plugin/framework/handling-data/dynamic-data.mdx

Co-authored-by: Brian Flad <bflad417@gmail.com>

* add float64 and number notes

---------

Co-authored-by: Brian Flad <bflad417@gmail.com>

---------

Co-authored-by: Benjamin Bennett <ben.bennett@hashicorp.com>
Co-authored-by: hc-github-team-tf-provider-devex <github-team-tf-provider-devex@hashicorp.com>
Co-authored-by: John Behm <jxsl13@googlemail.com>
Co-authored-by: Brian Flad <bflad417@gmail.com>
Co-authored-by: Sarah French <15078782+SarahFrench@users.noreply.github.com>
  • Loading branch information
6 people committed Mar 20, 2024
1 parent 66e4fa1 commit 1ded09e
Show file tree
Hide file tree
Showing 207 changed files with 15,841 additions and 146 deletions.
6 changes: 6 additions & 0 deletions .changes/unreleased/FEATURES-20240311-175905.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
kind: FEATURES
body: 'types/basetypes: Added `DynamicType` and `DynamicValue` implementations for
dynamic value handling'
time: 2024-03-11T17:59:05.67474-04:00
custom:
Issue: "147"
7 changes: 7 additions & 0 deletions .changes/unreleased/FEATURES-20240311-180136.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
kind: FEATURES
body: 'types/basetypes: Added interfaces `basetypes.DynamicTypable`, `basetypes.DynamicValuable`,
and `basetypes.DynamicValuableWithSemanticEquals` for dynamic custom type and value
implementations'
time: 2024-03-11T18:01:36.888566-04:00
custom:
Issue: "147"
6 changes: 6 additions & 0 deletions .changes/unreleased/FEATURES-20240311-180351.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
kind: FEATURES
body: 'resource/schema: Added `DynamicAttribute` implementation for dynamic value
handling'
time: 2024-03-11T18:03:51.559347-04:00
custom:
Issue: "147"
6 changes: 6 additions & 0 deletions .changes/unreleased/FEATURES-20240311-180418.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
kind: FEATURES
body: 'datasource/schema: Added `DynamicAttribute` implementation for dynamic value
handling'
time: 2024-03-11T18:04:18.042171-04:00
custom:
Issue: "147"
6 changes: 6 additions & 0 deletions .changes/unreleased/FEATURES-20240311-180430.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
kind: FEATURES
body: 'provider/schema: Added `DynamicAttribute` implementation for dynamic value
handling'
time: 2024-03-11T18:04:30.200616-04:00
custom:
Issue: "147"
5 changes: 5 additions & 0 deletions .changes/unreleased/FEATURES-20240311-180515.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
kind: FEATURES
body: 'function: Added `DynamicParameter` and `DynamicReturn` for dynamic value handling`'
time: 2024-03-11T18:05:15.196275-04:00
custom:
Issue: "147"
6 changes: 6 additions & 0 deletions .changes/unreleased/FEATURES-20240311-180859.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
kind: FEATURES
body: 'resource/schema/dynamicdefault: New package with `StaticValue` implementation
for dynamic schema-based default values'
time: 2024-03-11T18:08:59.479664-04:00
custom:
Issue: "147"
6 changes: 6 additions & 0 deletions .changes/unreleased/FEATURES-20240311-181044.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
kind: FEATURES
body: 'resource/schema/dynamicplanmodifier: New package with built-in implementations
for dynamic value plan modification.'
time: 2024-03-11T18:10:44.015502-04:00
custom:
Issue: "147"
6 changes: 6 additions & 0 deletions .changes/unreleased/FEATURES-20240311-181151.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
kind: FEATURES
body: 'resource/schema/defaults: New `Dynamic` interface for dynamic schema-based
default implementations'
time: 2024-03-11T18:11:51.403326-04:00
custom:
Issue: "147"
6 changes: 6 additions & 0 deletions .changes/unreleased/FEATURES-20240311-181242.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
kind: FEATURES
body: 'resource/schema/planmodifier: New `Dynamic` interface for dynamic value plan
modification implementations'
time: 2024-03-11T18:12:42.945376-04:00
custom:
Issue: "147"
5 changes: 5 additions & 0 deletions .changes/unreleased/FEATURES-20240311-181424.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
kind: FEATURES
body: 'schema/validator: New `Dynamic` interface for dynamic value schema validation'
time: 2024-03-11T18:14:24.809064-04:00
custom:
Issue: "147"
6 changes: 6 additions & 0 deletions attr/value.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,12 @@ const (
// NullValueString should be returned by Value.String() implementations
// when Value.IsNull() returns true.
NullValueString = "<null>"

// UnsetValueString should be returned by Value.String() implementations
// when Value does not contain sufficient information to display to users.
//
// This is primarily used for invalid Dynamic Value implementations.
UnsetValueString = "<unset>"
)

// Value defines an interface for describing data associated with an attribute.
Expand Down
188 changes: 188 additions & 0 deletions datasource/schema/dynamic_attribute.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,188 @@
// Copyright (c) HashiCorp, Inc.
// SPDX-License-Identifier: MPL-2.0

package schema

import (
"github.com/hashicorp/terraform-plugin-framework/attr"
"github.com/hashicorp/terraform-plugin-framework/internal/fwschema"
"github.com/hashicorp/terraform-plugin-framework/internal/fwschema/fwxschema"
"github.com/hashicorp/terraform-plugin-framework/schema/validator"
"github.com/hashicorp/terraform-plugin-framework/types"
"github.com/hashicorp/terraform-plugin-framework/types/basetypes"
"github.com/hashicorp/terraform-plugin-go/tftypes"
)

// Ensure the implementation satisifies the desired interfaces.
var (
_ Attribute = DynamicAttribute{}
_ fwxschema.AttributeWithDynamicValidators = DynamicAttribute{}
)

// DynamicAttribute represents a schema attribute that is a dynamic, rather
// than a single static type. Static types are always preferable over dynamic
// types in Terraform as practitioners will receive less helpful configuration
// assistance from validation error diagnostics and editor integrations. When
// retrieving the value for this attribute, use types.Dynamic as the value type
// unless the CustomType field is set.
//
// The concrete value type for a dynamic is determined at runtime in this order:
// 1. By Terraform, if defined in the configuration (if Required or Optional).
// 2. By the provider (if Computed).
//
// Once the concrete value type has been determined, it must remain consistent between
// plan and apply or Terraform will return an error.
type DynamicAttribute struct {
// CustomType enables the use of a custom attribute type in place of the
// default basetypes.DynamicType. When retrieving data, the basetypes.DynamicValuable
// associated with this custom type must be used in place of types.Dynamic.
CustomType basetypes.DynamicTypable

// Required indicates whether the practitioner must enter a value for
// this attribute or not. Required and Optional cannot both be true,
// and Required and Computed cannot both be true.
Required bool

// Optional indicates whether the practitioner can choose to enter a value
// for this attribute or not. Optional and Required cannot both be true.
Optional bool

// Computed indicates whether the provider may return its own value for
// this Attribute or not. Required and Computed cannot both be true. If
// Required and Optional are both false, Computed must be true, and the
// attribute will be considered "read only" for the practitioner, with
// only the provider able to set its value.
Computed bool

// Sensitive indicates whether the value of this attribute should be
// considered sensitive data. Setting it to true will obscure the value
// in CLI output. Sensitive does not impact how values are stored, and
// practitioners are encouraged to store their state as if the entire
// file is sensitive.
Sensitive bool

// Description is used in various tooling, like the language server, to
// give practitioners more information about what this attribute is,
// what it's for, and how it should be used. It should be written as
// plain text, with no special formatting.
Description string

// MarkdownDescription is used in various tooling, like the
// documentation generator, to give practitioners more information
// about what this attribute is, what it's for, and how it should be
// used. It should be formatted using Markdown.
MarkdownDescription string

// DeprecationMessage defines warning diagnostic details to display when
// practitioner configurations use this Attribute. The warning diagnostic
// summary is automatically set to "Attribute Deprecated" along with
// configuration source file and line information.
//
// Set this field to a practitioner actionable message such as:
//
// - "Configure other_attribute instead. This attribute will be removed
// in the next major version of the provider."
// - "Remove this attribute's configuration as it no longer is used and
// the attribute will be removed in the next major version of the
// provider."
//
// In Terraform 1.2.7 and later, this warning diagnostic is displayed any
// time a practitioner attempts to configure a value for this attribute and
// certain scenarios where this attribute is referenced.
//
// In Terraform 1.2.6 and earlier, this warning diagnostic is only
// displayed when the Attribute is Required or Optional, and if the
// practitioner configuration sets the value to a known or unknown value
// (which may eventually be null). It has no effect when the Attribute is
// Computed-only (read-only; not Required or Optional).
//
// Across any Terraform version, there are no warnings raised for
// practitioner configuration values set directly to null, as there is no
// way for the framework to differentiate between an unset and null
// configuration due to how Terraform sends configuration information
// across the protocol.
//
// Additional information about deprecation enhancements for read-only
// attributes can be found in:
//
// - https://github.com/hashicorp/terraform/issues/7569
//
DeprecationMessage string

// Validators define value validation functionality for the attribute. All
// elements of the slice of AttributeValidator are run, regardless of any
// previous error diagnostics.
//
// Many common use case validators can be found in the
// github.com/hashicorp/terraform-plugin-framework-validators Go module.
//
// If the Type field points to a custom type that implements the
// xattr.TypeWithValidate interface, the validators defined in this field
// are run in addition to the validation defined by the type.
Validators []validator.Dynamic
}

// ApplyTerraform5AttributePathStep always returns an error as it is not
// possible to step further into a DynamicAttribute.
func (a DynamicAttribute) ApplyTerraform5AttributePathStep(step tftypes.AttributePathStep) (interface{}, error) {
return a.GetType().ApplyTerraform5AttributePathStep(step)
}

// Equal returns true if the given Attribute is a DynamicAttribute
// and all fields are equal.
func (a DynamicAttribute) Equal(o fwschema.Attribute) bool {
if _, ok := o.(DynamicAttribute); !ok {
return false
}

return fwschema.AttributesEqual(a, o)
}

// GetDeprecationMessage returns the DeprecationMessage field value.
func (a DynamicAttribute) GetDeprecationMessage() string {
return a.DeprecationMessage
}

// GetDescription returns the Description field value.
func (a DynamicAttribute) GetDescription() string {
return a.Description
}

// GetMarkdownDescription returns the MarkdownDescription field value.
func (a DynamicAttribute) GetMarkdownDescription() string {
return a.MarkdownDescription
}

// GetType returns types.DynamicType or the CustomType field value if defined.
func (a DynamicAttribute) GetType() attr.Type {
if a.CustomType != nil {
return a.CustomType
}

return types.DynamicType
}

// IsComputed returns the Computed field value.
func (a DynamicAttribute) IsComputed() bool {
return a.Computed
}

// IsOptional returns the Optional field value.
func (a DynamicAttribute) IsOptional() bool {
return a.Optional
}

// IsRequired returns the Required field value.
func (a DynamicAttribute) IsRequired() bool {
return a.Required
}

// IsSensitive returns the Sensitive field value.
func (a DynamicAttribute) IsSensitive() bool {
return a.Sensitive
}

// DynamicValidators returns the Validators field value.
func (a DynamicAttribute) DynamicValidators() []validator.Dynamic {
return a.Validators
}
Loading

0 comments on commit 1ded09e

Please sign in to comment.