Reflect protocol v6 changes in schema JSON output

[radeksimko]

Recent changes in #27699 and #27793 introduced some new fields for an attribute, which should probably be reflected somehow in the schema JSON output produced by terraform providers schema -json.

The SDK doesn't seem to support v6 yet, so this currently has no real impact on end-users AFAICT, but when I tried adding some of the new fields to the builtin provider I saw a crash.

Terraform Version

Modified version of e5538693ced0dc42bb90a3919875cb8bbd625736 (latest revision)

Modified schema

terraform/builtin/providers/terraform/data_source_state.go

Lines 59 to 76 in 59b71de

	"foo": {
	NestedType: &configschema.Object{
	Attributes: map[string]*configschema.Attribute{
	"one": {
	Type: cty.String,
	Optional: true,
	},
	"two": {
	Type: cty.Number,
	Optional: true,
	},
	},
	Nesting: configschema.NestingSingle,
	},
	Description: "Testing something",
	DescriptionKind: configschema.StringMarkdown,
	Optional: true,
	},

(feel free to checkout json-schema-export-crash branch)

Terraform Configuration Files

resource "terraform_remote_state" "test" {

}

Crash Output

2021-03-11T12:23:36.083Z [INFO]  Terraform version: 0.15.0 dev
2021-03-11T12:23:36.083Z [INFO]  Go runtime version: go1.16
2021-03-11T12:23:36.083Z [INFO]  CLI args: []string{"/Users/radeksimko/gopath/bin/terraform", "providers", "schema", "-json"}
2021-03-11T12:23:36.083Z [TRACE] Stdout is a terminal of width 182
2021-03-11T12:23:36.083Z [TRACE] Stderr is a terminal of width 182
2021-03-11T12:23:36.083Z [TRACE] Stdin is a terminal
2021-03-11T12:23:36.083Z [DEBUG] Attempting to open CLI config file: /Users/radeksimko/.terraformrc
2021-03-11T12:23:36.083Z [INFO]  Loading CLI configuration from /Users/radeksimko/.terraformrc
2021-03-11T12:23:36.084Z [DEBUG] checking for credentials in "/Users/radeksimko/.terraform.d/plugins"
2021-03-11T12:23:36.084Z [DEBUG] ignoring non-existing provider search directory terraform.d/plugins
2021-03-11T12:23:36.084Z [DEBUG] will search for provider plugins in /Users/radeksimko/.terraform.d/plugins
2021-03-11T12:23:36.084Z [DEBUG] ignoring non-existing provider search directory /Users/radeksimko/Library/Application Support/io.terraform/plugins
2021-03-11T12:23:36.084Z [DEBUG] ignoring non-existing provider search directory /Library/Application Support/io.terraform/plugins
2021-03-11T12:23:36.084Z [INFO]  CLI command args: []string{"providers", "schema", "-json"}
2021-03-11T12:23:36.085Z [TRACE] Meta.Backend: no config given or present on disk, so returning nil config
2021-03-11T12:23:36.085Z [TRACE] Meta.Backend: backend has not previously been initialized in this working directory
2021-03-11T12:23:36.085Z [DEBUG] New state was assigned lineage "c9080e6c-8149-656f-2396-72941fe0038b"
2021-03-11T12:23:36.085Z [TRACE] Meta.Backend: using default local state only (no backend configuration, and no existing initialized backend)
2021-03-11T12:23:36.085Z [TRACE] Meta.Backend: instantiated backend of type <nil>
2021-03-11T12:23:36.086Z [DEBUG] checking for provisioner in "."
2021-03-11T12:23:36.087Z [DEBUG] checking for provisioner in "/Users/radeksimko/gopath/bin"
2021-03-11T12:23:36.087Z [DEBUG] checking for provisioner in "/Users/radeksimko/.terraform.d/plugins"
2021-03-11T12:23:36.087Z [INFO]  Failed to read plugin lock file .terraform/plugins/darwin_amd64/lock.json: open .terraform/plugins/darwin_amd64/lock.json: no such file or directory
2021-03-11T12:23:36.087Z [TRACE] Meta.Backend: backend <nil> does not support operations, so wrapping it in a local backend
2021-03-11T12:23:36.087Z [TRACE] backend/local: requesting state manager for workspace "default"
2021-03-11T12:23:36.087Z [TRACE] backend/local: state manager for workspace "default" will:
 - read initial snapshot from terraform.tfstate
 - write new snapshots to terraform.tfstate
 - create any backup at terraform.tfstate.backup
2021-03-11T12:23:36.087Z [TRACE] backend/local: requesting state lock for workspace "default"
2021-03-11T12:23:36.087Z [TRACE] backend/local: reading remote state for workspace "default"
2021-03-11T12:23:36.087Z [TRACE] statemgr.Filesystem: reading initial snapshot from terraform.tfstate
2021-03-11T12:23:36.087Z [TRACE] statemgr.Filesystem: snapshot file has nil snapshot, but that's okay
2021-03-11T12:23:36.087Z [TRACE] statemgr.Filesystem: read nil snapshot
2021-03-11T12:23:36.087Z [DEBUG] backend/local: skipping refresh of managed resources
2021-03-11T12:23:36.087Z [TRACE] backend/local: retrieving local state snapshot for workspace "default"
2021-03-11T12:23:36.087Z [TRACE] backend/local: building context for current working directory
2021-03-11T12:23:36.087Z [TRACE] terraform.NewContext: starting
2021-03-11T12:23:36.087Z [TRACE] terraform.NewContext: loading provider schemas
2021-03-11T12:23:36.088Z [TRACE] LoadSchemas: retrieving schema for provider type "terraform.io/builtin/terraform"
2021-03-11T12:23:36.088Z [TRACE] terraform.NewContext: complete
2021-03-11T12:23:36.088Z [TRACE] backend/local: finished building terraform.Context

panic: unknown type implementation

goroutine 1 [running]:
github.com/zclconf/go-cty/cty.Type.MarshalJSON(0x0, 0x0, 0xd9c9d55d, 0x2fcb13150d2aa4a, 0xffffffffffffffff, 0x0, 0x0)
	/Users/radeksimko/gopath/pkg/mod/github.com/zclconf/go-cty@v1.8.0/cty/json.go:87 +0xd65
github.com/hashicorp/terraform/command/jsonprovider.marshalAttribute(0xc0005ab1c0, 0xc00036c330)
	/Users/radeksimko/gopath/src/github.com/hashicorp/terraform/command/jsonprovider/attribute.go:32 +0x3b
github.com/hashicorp/terraform/command/jsonprovider.marshalBlock(0xc00036c000, 0xc0004adc60)
	/Users/radeksimko/gopath/src/github.com/hashicorp/terraform/command/jsonprovider/block.go:63 +0x1fe
github.com/hashicorp/terraform/command/jsonprovider.marshalSchema(...)
	/Users/radeksimko/gopath/src/github.com/hashicorp/terraform/command/jsonprovider/schema.go:20
github.com/hashicorp/terraform/command/jsonprovider.marshalSchemas(0xc00036c060, 0xc00036c090, 0xc00036c2d0)
	/Users/radeksimko/gopath/src/github.com/hashicorp/terraform/command/jsonprovider/schema.go:31 +0x127
github.com/hashicorp/terraform/command/jsonprovider.marshalProvider(0xc00036c0c0, 0x339dc5b)
	/Users/radeksimko/gopath/src/github.com/hashicorp/terraform/command/jsonprovider/provider.go:62 +0xf0
github.com/hashicorp/terraform/command/jsonprovider.Marshal(0xc0004adc10, 0x0, 0x0, 0xc0005eb800, 0x1, 0x1)
	/Users/radeksimko/gopath/src/github.com/hashicorp/terraform/command/jsonprovider/provider.go:38 +0x305
github.com/hashicorp/terraform/command.(*ProvidersSchemaCommand).Run(0xc0001d2fc0, 0xc00004e0b0, 0x1, 0x1, 0xc0004ad170)
	/Users/radeksimko/gopath/src/github.com/hashicorp/terraform/command/providers_schema.go:100 +0x889
github.com/mitchellh/cli.(*CLI).Run(0xc000399180, 0xc0002fbb00, 0x33a6fb9, 0x10)
	/Users/radeksimko/gopath/pkg/mod/github.com/mitchellh/cli@v1.1.2/cli.go:262 +0x41a
main.wrappedMain(0x0)
	/Users/radeksimko/gopath/src/github.com/hashicorp/terraform/main.go:379 +0x10e8
main.realMain(0x0)
	/Users/radeksimko/gopath/src/github.com/hashicorp/terraform/main.go:115 +0x554
main.main()
	/Users/radeksimko/gopath/src/github.com/hashicorp/terraform/main.go:58 +0x25

Expected Behavior

Output JSON schema

Actual Behavior

Crash

Steps to Reproduce

1. terraform init
2. terraform providers schema -json

[mildwonkey]

ack thank you radek that got lost in the shuffle ❤️

[bflad]

Aside: Can/should we bump the version?

// FormatVersion represents the version of the json format and will be
// incremented for any change to this format that requires changes to a
// consuming parser.
const FormatVersion = "0.1"

I know these changes are not necessarily required but eventually downstream tooling (e.g. terraform-plugin-docs) will need to be updated with this support and it would be helpful to know compatibility. It might also be helpful to include a separate changelog for this particular versioning or some way for consumers to know what/how to update support over time.

[mildwonkey]

That is an excellent point @bflad , we've made a lot of additive changes and I have been neglecting that format version. I'll open a PR and make sure I ping our consumers. thanks for the nudge!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.