meta: Support Terraform Test Framework #1534
Comments
|
Related: #636 Maybe that issue can be broken down into the individual file formats but there may (or may not) be some higher-level work to do to support multiple formats easily/efficiently throughout all the libraries (hcl-lang + terraform-schema). Right now we support basically just |
|
note: as per https://developer.hashicorp.com/terraform/language/tests#syntax the extension is either |
|
Now that the test framework was released with terraform 1.6, I think this issue is more pressing as people want to use it effectively in vscode! |
|
Also don't forget the formatter. At the moment of writting this comment, the command |
|
Question: I'm a newbie on the internal of terraform in general but if the terraform-cli already parce the "test framework" file, this code cannot be re-user for the language server? or even the "tokenization" that need to be perform for the command |
|
@martinOrigin8Cares thanks for reminding me about formatting, I've added it to the list. Under the hood, both Terraform and terraform-ls, use https://github.com/hashicorp/hcl to parse/tokenize the Terraform HCL files. After that step both tools work a bit differently. terraform-ls supports any Terraform version >= 0.12 and we load the specific schema/features for the user's version. In terms of formatting: We currently use |
|
@dbanck Any progress on support for I'm keen to help out if required - have dug into the code a bit, but not quite sure where to start. |
|
From what I can tell the following is required: terraform-ls:internal/terraform/ast/tests.goAdd AST for test files// Copyright (c) HashiCorp, Inc.
// SPDX-License-Identifier: MPL-2.0
package ast
import (
"strings"
"github.com/hashicorp/hcl/v2"
)
type TestsFilename string
func NewTestsFilename(name string) (TestsFilename, bool) {
if IsTestsFilename(name) {
return TestsFilename(name), true
}
return "", false
}
func IsTestsFilename(name string) bool {
return strings.HasSuffix(name, ".tftest.hcl") ||
strings.HasSuffix(name, ".tftest.json")
}
func (tf TestsFilename) String() string {
return string(tf)
}
func (tf TestsFilename) IsJSON() bool {
return strings.HasSuffix(string(tf), ".json")
}
type TestsFiles map[TestsFilename]*hcl.File
func TestsFilesFromMap(m map[string]*hcl.File) TestsFiles {
mf := make(TestsFiles, len(m))
for name, file := range m {
mf[TestsFilename(name)] = file
}
return mf
}
func (tf TestsFiles) Copy() TestsFiles {
m := make(TestsFiles, len(tf))
for name, file := range tf {
m[name] = file
}
return m
}
type TestsDiags map[TestsFilename]hcl.Diagnostics
func TestsDiagsFromMap(m map[string]hcl.Diagnostics) TestsDiags {
mf := make(TestsDiags, len(m))
for name, file := range m {
mf[TestsFilename(name)] = file
}
return mf
}
func (td TestsDiags) Copy() TestsDiags {
m := make(TestsDiags, len(td))
for name, file := range td {
m[name] = file
}
return m
}
func (td TestsDiags) AsMap() map[string]hcl.Diagnostics {
m := make(map[string]hcl.Diagnostics, len(td))
for name, diags := range td {
m[string(name)] = diags
}
return m
}
func (td TestsDiags) Count() int {
count := 0
for _, diags := range td {
count += len(diags)
}
return count
}
type SourceTestsDiags map[DiagnosticSource]TestsDiags
func (std SourceTestsDiags) Count() int {
count := 0
for _, diags := range std {
count += diags.Count()
}
return count
}internal/langserver/handlers/did_change_watched_files.go// Line ~260
if !ast.IsModuleFilename(fi.Name()) && !ast.IsVarsFilename(fi.Name()) && !ast.IsTestsFilename(fi.Name()) {
...
}vscode-terraformsrc/extension.tsactivate > clientOptions > synchronize > fileEvents package.jsoncontributes > grammars syntaxes/tftest.tmGrammar.jsonKind of stuck at the
|
|
Hi @nagytech! Thanks for the offer to help! I'm actively researching and working on the Terraform test integration and we plan to ship it early next year. It will also require changes in terraform-schema and hcl-lang and some refactoring in terraform-ls. I will ticket out the work over the next month and we can see if anything comes up that you want to contribute. For the static grammar, we will probably reuse the existing Terraform grammar and extend the list of blocks. This way we avoid maintaining and shipping another grammar file. PS: If you want to keep digging, there are some poc branches in the repositories. |
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
|
@dbanck Hi Daniel, any updates on the status of this? Also happy to help if needed. Currently syntax validation is throwing errors for the new At the minimum, is there a way to just disable the errors temporarily on my end? I'm trying to create an AWS workshop on using this new feature of Terraform (as part of an advanced pt.2 follow up to the Intro to Terraform on AWS Workshop, and currently I have to quit and re-open vscode every time I make any modification to the files. This makes it quite difficult to take any screenshots or make video explanations. |
|
@novekm You shouldn't get validation errors for Terraform test files today, because the extension doesn't (yet) claim the new extension You can get syntax highlighting for test files by installing the HashiCorp HCL extension. Hope this helps! :) |
|
You can add to the checklist the support for
Also the content of those files is referenced in new |
|
An interesting catch I hit in a project using We have a CI step to run services/otel-graphite-collector/modules/otel_graphite_collector/tests/ecs_service.tftest.hcl
--- old/services/otel-graphite-collector/modules/otel_graphite_collector/tests/ecs_service.tftest.hcl
+++ new/services/otel-graphite-collector/modules/otel_graphite_collector/tests/ecs_service.tftest.hcl
@@ -6,7 +6,7 @@
command = plan
assert {
- condition = aws_ecs_service.this.desired_count > 0
+ condition = aws_ecs_service.this.desired_count > 0
error_message = "The desired_count must be greater than 0."
}
Exited with code exit status 3But unfortunately the Terraform extension isn't formatting these files yet so I'm manually having to run |
|
It has been almost a year and Terraform is currently at version 1.9. Any ETA on this issue? |
|
Still seems to be broken / missing as of 1.9.5 |
|
I am happy to say that this work has been prioritized and parts of this has already been released in https://github.com/hashicorp/vscode-terraform/releases/tag/v2.32.3. We now support syntax highlighting for Terraform Test and Terraform Mock files in v2.32.3. We will start to add new features in the coming weeks, and will update this ticket with links as they are delivered. |
Problem Statement
Terraform 1.6 will ship with an updated and finalised
terraform testcommand for running tests against your Terraform files. Tests will use a new file extension (.tftest.hcl&.tftest.json) and introduce newrun,assert,...blocks. Right now, a user will not get any syntax highlighting or advanced IDE features.User Impact
The
testcommand and test file syntax are aimed at module authors who wish to validate and test their shared modules. However, it is also possible to use thetestcommand to validate root modules. So any user writing tests for Terraform code will benefit from this.Expected User Experience
Syntax Highlighting
When you open a
.tftest.hclfile, it should be highlighted in a similar way to how Terraform is highlighted today. This should happen automatically, without the user having to set the file type manually. The vscode-hcl extension will provide some basic syntax highlighting for the files, but we can enhance this with support in vscode-terraform.Completion
A user can use auto-completion within a test file. Triggering completion at the top level will bring up all available blocks. Triggering completion within a block will bring up all available attributes and child blocks.
Hover
Hovering over any block, attribute or value will display additional information, similar to working within Terraform files.
References
Within a test file, a user can refer to resources and modules. The go-to definition should work for all resource and module sources and open the appropriate definition when triggered.
Completion should also bring up the existing resources which are defined in the Terraform files.
Semantic Tokens
We send accurate semantic tokens for all identifiers in a test file to further improve highlighting.
Formatting
Similar to how we format
.tffiles, we should implement formatting for.tftest.hclfiles.VS Code Integration
The Testing API allows VS Code extensions to discover tests in the workspace and publish results. Users can execute tests in the Test Explorer view, from decorations, and inside commands.
Proposal
.tftest.hclfiles.tftest.hclfiles.tftest.hclfiles.tftest.jsonfiles.tftest.hclfiles.tftest.hclfiles.tftest.hclfiles.tftest.hclfiles.tftest.hclfilesCommunity Note
The text was updated successfully, but these errors were encountered: