Add support for built-in functions #145

dbanck · 2022-09-23T14:55:17Z

This PR adds the function signatures for all previous Terraform versions, starting with 0.12. For versions >= 1.4, we use the new terraform metadata functions -json command to generate the signatures.

The changes are currently a no-op for users. However, we will follow up with modifications to the language server to obtain the functions for a specific Terraform version using FunctionsForVersion.

I've created a list of all Terraform functions with their introduction versions for a better overview and used that as a source for all the signatures.

TODO

Bump terraform-exec version after Add support for metadata functions -json command terraform-exec#358 is merged
- Pin to new terraform-exec release
Bump hcl-lang version after Add function signature types hcl-lang#207 is merged

Part of Support built-in functions terraform-ls#37

radeksimko

I'm assuming this is now expected, until 1.4.0 final actually lands, just surfacing it here for posterity:

2023/02/16 12:39:16 ensuring terraform is installed
2023/02/16 12:39:17 failed to obtain product version from "https://releases.hashicorp.com/terraform/1.4.0/index.json": 404 Not Found
exit status 1
internal/funcs/generated/doc.go:4: running "go": exit status 1

I left some comments in-line, mostly minor, but the escaping one should certainly be resolved before we merge it, especially if it affects upstream as well (i.e. the JSON TF Core produces is HTML escaped).

radeksimko · 2023-02-16T12:12:49Z

internal/funcs/0.12/base_functions.go

+				},
+			},
+			ReturnType:  cty.String,
+			Description: "`base64sha256` computes the SHA256 hash of a given string and encodes it with Base64. This is not equivalent to `base64encode(sha256(&amp;#34;test&amp;#34;))` since `sha256()` returns hexadecimal representation.",


I'm not sure where this came from and whether it affects other places in Core as well, but I see some HTML escaping in effect (&#...) which probably shouldn't be.

The context in which we need to escape the (Markdown) description text would be single-line Go string, i.e. " would need to be \" but that's about it. Single quotes and probably most other characters can stay as-is.

I don't know if there's any escaping function we could use in this context in the generator, here I suppose it's just a simple matter of manual Find+Replace.

Good catch! I fixed it in Terraform in hashicorp/terraform#32710 and replaced all occurrences here.

radeksimko · 2023-02-16T12:17:17Z

internal/funcs/generated/README.md

+
+This package can generate function signature files for Terraform >= 1.4 automatically.
+
+It is intended to run whenever HashiCorp releases a new Terraform. If the  Terraform version contains updated function signatures, it generates a new file for that version. When no changes are detected, one should commit the version bump in `gen/gen.go`.


Suggested change

It is intended to run whenever HashiCorp releases a new Terraform. If the Terraform version contains updated function signatures, it generates a new file for that version. When no changes are detected, one should commit the version bump in `gen/gen.go`.

It is intended to run whenever HashiCorp releases a new Terraform version. If the Terraform version contains updated function signatures, it generates a new file for that version. When no changes are detected, one should commit the version bump in `gen/gen.go`.

radeksimko · 2023-02-16T12:24:10Z

internal/funcs/generated/gen/gen.go

+		"escapeDescription": escapeDescription,
+	}).Parse(outputTpl)
+	if err != nil {
+		log.Fatal(err)


Nitpick: I know this is a script which will be executed by a maintainer, but it's still a good practice for any downstream functions to return error and only the top level to actually deal with surfacing it somehow.

This enables testability and reusability and helps readability (it's easier to understand the flow of the program as a whole), even though admittedly we probably won't be testing or reusing this code any time soon.

Oh, missed updating the error handling for the two templating functions. Thanks for pointing it out.

radeksimko · 2023-02-16T12:27:30Z

internal/funcs/generated/gen/gen.go

+}
+
+func escapeDescription(description string) string {
+	return strings.ReplaceAll(description, `\`, `\\`)


Again - not sure if the HTML escape characters come already in the JSON output - which would be something to address upstream, but either way we should make sure that double quotes are escaped correctly here too.

radeksimko · 2023-02-16T12:32:47Z

schema/functions.go

+		}
+	}
+
+	return nil, NoCompatibleFunctionsErr{Constraints: vc}


Given this should never happen, I'd consider not exporting this error - since we should never be catching it in LS.

We could go as far as panicing here, but it's probably still better not to, since it's a library and as such it should not decide how errors are surfaced. LS could however turn that error into panic.

Agreed — replaced it with a simple error string which we can turn into a panic upstream.

radeksimko · 2023-02-16T12:37:45Z

schema/functions.go

+)
+
+func FunctionsForVersion(v *version.Version) (map[string]schema.FunctionSignature, error) {
+	ver, err := semVer(v)


I think the semVer predates the Core() method upstream, which could now be used directly and we could get rid of semVer().

radeksimko

LGTM

dbanck added the enhancement New feature or request label Sep 23, 2022

dbanck self-assigned this Sep 23, 2022

dbanck mentioned this pull request Sep 23, 2022

Add support for function signature help hashicorp/terraform-ls#1077

Merged

dbanck force-pushed the f-function-support branch from 232bb2b to fb17b61 Compare October 7, 2022 16:02

dbanck force-pushed the f-function-support branch from ca85fec to 83f7af5 Compare December 6, 2022 13:13

dbanck force-pushed the f-function-support branch 2 times, most recently from 7dd453d to 41034ba Compare January 16, 2023 14:02

dbanck force-pushed the f-function-support branch from 41034ba to 3b18e7d Compare January 31, 2023 15:17

Add 1.3 as new schema version

19c0050

dbanck force-pushed the f-function-support branch 3 times, most recently from 3240078 to 2edfec1 Compare February 15, 2023 13:56

dbanck marked this pull request as ready for review February 15, 2023 14:02

dbanck requested a review from a team as a code owner February 15, 2023 14:02

dbanck force-pushed the f-function-support branch from 2edfec1 to 4a6f85e Compare February 15, 2023 14:15

radeksimko mentioned this pull request Feb 15, 2023

Add function signature types hashicorp/hcl-lang#207

Merged

Bump hcl-lang to 7a48063

817806e

dbanck force-pushed the f-function-support branch from 4a6f85e to ec8c2f8 Compare February 16, 2023 11:47

dbanck added 6 commits February 16, 2023 12:50

Add Terraform 0.12 functions

6a134a2

Add Terraform 0.13 functions

e490d15

Add Terraform 0.14 functions

71e635c

Add Terraform 0.15 functions

57f2df3

Add Terraform 1.3 functions

8361ecd

Generate function signatures for Terraform >= 1.4

aad1908

dbanck force-pushed the f-function-support branch from ec8c2f8 to aad1908 Compare February 16, 2023 11:52

radeksimko reviewed Feb 16, 2023

View reviewed changes

dbanck added 4 commits February 20, 2023 17:17

Install terraform-exec v0.18.0

5b2861c

Replace HTML entities

2f39e87

Escape quotes and generated with latest version

31c3642

Improve error handling

ac7cffe

dbanck added 4 commits February 20, 2023 17:17

Fix README typos

84c37bf

Replace ioutil usage

f7e6e8f

Replace semVer with version.Core

ddf148a

Replace NoCompatibleFunctionsErr with simple error

6e7a15d

radeksimko approved these changes Feb 20, 2023

View reviewed changes

dbanck merged commit d92602a into main Feb 20, 2023

dbanck deleted the f-function-support branch February 20, 2023 17:24

dbanck mentioned this pull request Mar 14, 2023

Add support for SignatureAtPos hashicorp/hcl-lang#135

Merged

This was referenced Apr 24, 2023

Support built-in functions hashicorp/terraform-ls#37

Closed

Support for built-in functions hashicorp/vscode-terraform#666

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for built-in functions #145

Add support for built-in functions #145

dbanck commented Sep 23, 2022 •

edited

Loading

radeksimko left a comment

radeksimko Feb 16, 2023

dbanck Feb 20, 2023

radeksimko Feb 16, 2023

radeksimko Feb 16, 2023

dbanck Feb 20, 2023

radeksimko Feb 16, 2023

radeksimko Feb 16, 2023

dbanck Feb 20, 2023

radeksimko Feb 16, 2023

radeksimko left a comment


		This package can generate function signature files for Terraform >= 1.4 automatically.

		It is intended to run whenever HashiCorp releases a new Terraform. If the Terraform version contains updated function signatures, it generates a new file for that version. When no changes are detected, one should commit the version bump in `gen/gen.go`.

Add support for built-in functions #145

Add support for built-in functions #145

Conversation

dbanck commented Sep 23, 2022 • edited Loading

TODO

radeksimko left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

radeksimko left a comment

Choose a reason for hiding this comment

dbanck commented Sep 23, 2022 •

edited

Loading