Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix crash when an unset variable is used #8837

Merged
merged 23 commits into from
Mar 10, 2020
Merged

Fix crash when an unset variable is used #8837

merged 23 commits into from
Mar 10, 2020

Conversation

azr
Copy link
Contributor

@azr azr commented Mar 4, 2020

This was due to a misunderstanding on my side. When a variable is not set one needs to set it to cty.NullVal(cty.DynamicPseudoType) , like so:

func (v *Variable) Value() (cty.Value, *hcl.Diagnostic) {
for _, value := range []cty.Value{
v.CmdValue,
v.VarfileValue,
v.EnvValue,
v.DefaultValue,
} {
if !value.IsNull() {
return value, nil
}
}
value := cty.NullVal(cty.DynamicPseudoType)
return value, &hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: fmt.Sprintf("Unset variable %q", v.Name),
Detail: "A used variable must be set or have a default value; see " +
"https://packer.io/docs/configuration/from-1.5/syntax.html for details.",
Context: v.block.DefRange.Ptr(),
}
}
type Variables map[string]*Variable
func (variables Variables) Values() map[string]cty.Value {
res := map[string]cty.Value{}
for k, v := range variables {
res[k], _ = v.Value()
// here the value might not be used and in that case we don't want to
// force users to set it. So this error is ignored. we still set it as
// it can be a `cty.NullVal(cty.DynamicPseudoType)`, which is the go
// cty value for 'unknown value' and should error when used.
}
return res
}

I choose not to error when a variable is not set and not used. But when someone does something like this:

variable "proxmox_username" {
  type = string
}

build {
  sources = [
    "source.null.null-builder${var.proxmox_username}",
  ]
}


source "null" "null-builder" {
}

The error is the following:


  on .vscode/hcl2/crash/var_with_no_default.pkr.hcl line 8, in build:
   8:     "source.null.null-builder${var.proxmox_username}",

Variables may not be used here.

Error: Unsuitable value type

  on .vscode/hcl2/crash/var_with_no_default.pkr.hcl line 7, in build:
   7:   sources = [
   8:     "source.null.null-builder${var.proxmox_username}",
   9:   ]

Unsuitable value: value must be known

fix #8730

@azr azr requested a review from a team as a code owner March 4, 2020 12:05
azr added 3 commits March 4, 2020 13:19
because a variable can be defined in the Locals block
Copy link
Contributor

@sylviamoss sylviamoss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 🎉

hcl2template/types.build.go Show resolved Hide resolved
hcl2template/types.packer_config.go Show resolved Hide resolved
@azr
Copy link
Contributor Author

azr commented Mar 4, 2020

Question for the team. With this PR a variable can be defined and have no value.But this would be different that for Terraform; so quoting this message:

By default, a Terraform variable is required and so Terraform would emit an error if it isn't set. An author can make a variable optional by providing a default value.

variable "example" {
  type    = string
  default = "Hello"
}

In some cases there isn't any reasonable default value, and the module simply wants to detect whether or not the value is set. In that case, the author can specify the default as null to say that the variable is optional but it has no default:

variable "example" {
  type    = string
  default = null
}

Should we do the same ?

@sylviamoss
Copy link
Contributor

Should we do the same ?

I think as a terraform user the one might expect to be the same. In terraform if the variable without a default is used, is it going to return an error as your implementation does?

I like the way you did and I think both solutions are good. If it's not too difficult, I would do it as terraform just for consistency. Maybe just add the option of setting default = null without removing what you did.

@@ -200,7 +214,7 @@ func (variables *Variables) decodeVariableBlock(block *hcl.Block, ectx *hcl.Eval
})
}

(*variables)[block.Labels[0]] = res
(*variables)[name] = res
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💯

// Description of the variable
Description string
// When Sensitive is set to true Packer will try it best to hide/obfuscate
// the variable from the output stream. By replacing the text.
Sensitive bool

block *hcl.Block
Range hcl.Range
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this so that we can keep track of the block when generating diagnostics?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes !

@@ -180,6 +179,4 @@ replace git.apache.org/thrift.git => github.com/apache/thrift v0.0.0-20180902110

replace github.com/gofrs/flock => github.com/azr/flock v0.0.0-20190823144736-958d66434653

replace github.com/zclconf/go-cty => github.com/azr/go-cty v1.1.1-0.20200203143058-28fcda2fe0cc
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My pr was merged; so this can be removed.

@azr azr changed the title Fix crash when an unset variable is used [WIP] Fix crash when an unset variable is used Mar 6, 2020
@azr azr changed the title [WIP] Fix crash when an unset variable is used Fix crash when an unset variable is used Mar 10, 2020
@azr
Copy link
Contributor Author

azr commented Mar 10, 2020

Hey folks ! I think this PR is good to review. I think the behaviour are super close to what we want in the RFC. But since this also fixes a crash I think we should merge it.

I'll gladly update the behaviours if/when the RFC changes later on 🙂.

@@ -29,6 +29,7 @@ variable "super_secret_password" {
description = <<IMSENSIBLE
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you mean "sensitive" here instead of "sensible"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes !

@SwampDragons SwampDragons merged commit 8a1caaa into master Mar 10, 2020
@@ -278,3 +278,48 @@ precedence over earlier ones:

~> **Important:** Variables with map and object values behave the same way as
other variables: the last value found overrides the previous values.

### A variable value must be known :
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💯 👍 🏆 Nicely done on adding the variables table.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks :D

@azr azr deleted the fix_8730 branch March 11, 2020 09:19
@SwampDragons SwampDragons added this to the 1.5.5 milestone Mar 12, 2020
dominikborkowski added a commit to dominikborkowski/packer-aws-runner that referenced this pull request Mar 18, 2020
Thanks to the following issue hashicorp/packer#8846, and the fact that they’ll be removing abiity to use a simple ‘go get’ (hashicorp/packer#8837 (comment)), it makes no sense to try to build it locally anymore.
nywilken added a commit that referenced this pull request Mar 27, 2020
…atching

Before change
```
413e19b Merge pull request #8942 from desolatorxxl/google-fix-ssh-keys-metadata
b81800d Merge pull request #8935 from zaventh/feature/start-on-boot
9486316 Merge pull request #8922 from hashicorp/f-vsphere_iso-export-ovf-options
56aebbe Merge pull request #8920 from rhencke/patch-1
d068430 make sure locals are evaluated only once variables are + test this (#8918)
3dae5df Merge pull request #8905 from hashicorp/fix_8493
811a730 Merge pull request #8907 from hashicorp/fix_8428
fa49d21 Merge pull request #8906 from hashicorp/fix_8904
23f5603 Merge pull request #8889 from hashicorp/hcl2_singular_blocks
dc9259f Merge pull request #8892 from zaventh/feature/vga-adapter
fc35f02 Merge pull request #8890 from hashicorp/fix_8880
7972ab7 Merge pull request #8735 from hashicorp/fix_plugin_loading
890d7b2 Merge pull request #8875 from hashicorp/fix_8812
e94ff70 Merge pull request #8883 from hashicorp/fix_8835
9075b80 Merge pull request #8891 from rhencke/patch-1
6477d8a Merge pull request #8882 from hashicorp/fix-var-file-hcl
6008f91 Merge pull request #8847 from takaishi/support-keyboard-interactive
5604561 Merge pull request #8877 from paulcichonski/remote-esxi-bastion
698f744 Merge pull request #8887 from hashicorp/untangle_ssh_docs_from_aws
aeedc9a Merge pull request #8879 from mbrancato/specify_keyvault_sku
5365fda Merge pull request #8884 from hashicorp/fix_codecov_config
4bd7b14 Merge pull request #8732 from jhawk28/reorder_cdrom_drive
072a71b Merge pull request #8863 from hashicorp/update_go-cty_regex
8a1caaa Merge pull request #8837 from hashicorp/fix_8730
7873cab Merge pull request #8858 from hashicorp/fix_8791
7e382d0 Merge pull request #8828 from mvitaly/fix_8816
8832b3e Merge pull request #8787 from jhawk28/vsphere_iso_multiple_disks
5281740 Merge pull request #8831 from rjhornsby/master
e35a872 Merge pull request #8830 from hashicorp/d-var-file-hcl2-not-yet
```

After change
```
⇶  git log v1.5.4...v1.5.5 --first-parent --oneline --grep="Merge pull request #[0-9]\+" --grep="(#[0-9]\+)$"
413e19b Merge pull request #8942 from desolatorxxl/google-fix-ssh-keys-metadata
c387dc2 builder/vsphere-clone: Find the vm within the folder (#8938)
b17b211 Add cleanup_remote_cache config option to vmware-iso (#8917)
e6368b9 Fix azure winrm_password attribution and allow to set winrm_username (#8928)
fcf10e9 Replace Amazon with Outscale for OSC BSU doc (#8944)
9240fb7 Fix typo in title (#8943)
2c6f096 Allow accepting image for the members in OpenStack builder (#8931)
b81800d Merge pull request #8935 from zaventh/feature/start-on-boot
daffd9c CONTRIBUTING: Update documentation for linting on Travis (#8933)
3a9d356 golangci-lint: Update --new-from-rev option to check only newly added commits (#8923)
97d797d Fix small typos in osc-bsuvolume.html.md (#8926)
9486316 Merge pull request #8922 from hashicorp/f-vsphere_iso-export-ovf-options
56aebbe Merge pull request #8920 from rhencke/patch-1
99b0b98 Add ovf export capability to vsphere builders (#8764)
d068430 make sure locals are evaluated only once variables are + test this (#8918)
ad8dafa HCL: add tests and fixes around var-file and var args  (#8914)
7979ab0 Add after_n_builds to codecov.yml (#8913)
3dae5df Merge pull request #8905 from hashicorp/fix_8493
811a730 Merge pull request #8907 from hashicorp/fix_8428
fa49d21 Merge pull request #8906 from hashicorp/fix_8904
b94937c Update provisioner_test.go (#8900)
2319521 Add iso config test for checksum from file specific case (#8897)
23f5603 Merge pull request #8889 from hashicorp/hcl2_singular_blocks
dc9259f Merge pull request #8892 from zaventh/feature/vga-adapter
690bf71 Add Codecov badge and remove report style (#8896)
fc35f02 Merge pull request #8890 from hashicorp/fix_8880
7972ab7 Merge pull request #8735 from hashicorp/fix_plugin_loading
890d7b2 Merge pull request #8875 from hashicorp/fix_8812
e94ff70 Merge pull request #8883 from hashicorp/fix_8835
```
@ghost
Copy link

ghost commented Apr 10, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@ghost ghost locked and limited conversation to collaborators Apr 10, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Packer with HCL crashes on missing variable default.
4 participants