cliconfig: Allow breaking the dependency lock file using the environment #32726
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
apparentlymart
added
the
1.4-backport
alisdair
approved these changes
Feb 22, 2023
Since it's already possible to activate the dependency lock file using an environment variable, we should allow opting in to it having broken behavior using the environment too. It's kinda odd in retrospect that TF_PLUGIN_CACHE_DIR is the only setting we allow to be configured both in the environment and the CLI configuration. That means that the infrastructure for dealing with that situation was relatively immature here and so I did some light refactoring to make it unit-testable without actually modifying the test program's environment.
apparentlymart
force-pushed
the
f-break-plugin-cache-with-env-var
branch
from
13a3f2f to
1016c09
Compare
|
This setting (in both of its forms) will be mentioned in the v1.4 Upgrade Guide once it's written, but we don't have that yet so I skipped doing that here and will instead contribute some text to the later PR for the upgrade guide. |
|
Reminder for the merging maintainer: if this is a user-visible change, please update the changelog on the appropriate release branch. |
|
Thanks @apparentlymart |
minamijoyo
added a commit
to minamijoyo/tfmigrate
that referenced
this pull request
Mar 9, 2023
Starting from Terraform v1.4, launching terraform providers in the
acceptance test has been failing more frequently with a text file busy
error.
```
--- FAIL: TestAccMultiStateMigratorApplySimple (1.07s)
multi_state_migrator_test.go:123: failed to run terraform init: failed to run command (exited 1): terraform init -input=false -no-color
stdout:
Initializing the backend...
Successfully configured the backend "s3"! Terraform will automatically
use this backend unless the backend configuration changes.
Initializing provider plugins...
- Finding latest version of hashicorp/null...
- Installing hashicorp/null v3.2.1...
stderr:
Error: Failed to install provider
Error while installing hashicorp/null v3.2.1: open
/tmp/plugin-cache/registry.terraform.io/hashicorp/null/3.2.1/linux_amd64/terraform-provider-null_v3.2.1_x5:
text file busy
```
After some investigation, I found Go's `os/exec.Cmd.Run()` does not wait
for the grandchild process to complete; from the point of view of
tfmigrate, the terraform command is the child process, and the provider
is the grandchild process.
golang/go#23019
If I understand correctly, this is not a Terraform issue and
theoretically should occur in versions older than v1.4; the changes in
v1.4 may have broken the balance of execution timing and made the test
very flaky. I experimented with inserting some sleep but could not get
the test to stabilize correctly. After trying various things, I found
that the test became stable by enabling the
`TF_PLUGIN_CACHE_MAY_BREAK_DEPENDENCY_LOCK_FILE` flag was introduced in
v1.4. This is an escape hatch to revert to the v1.3 equivalent of the
global cache behavior change in v1.4.
hashicorp/terraform#32726
This behavior change has already been addressed in the previous commit
using a local file system mirror, so activating this flag does not seem
to make any sense. Even though I have no other reasonable solutions now,
please let me know if anyone finds a better solution.
|
I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active contributions. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Since it's already possible to activate the dependency lock file using an environment variable, we should allow opting in to it having broken behavior using the environment too.
It's kinda odd in retrospect that
TF_PLUGIN_CACHE_DIRis the only setting we allow to be configured both in the environment and the CLI configuration. That means that the infrastructure for dealing with that situation was relatively immature here and so I did some light refactoring to make it unit-testable without actually modifying the test program's environment. To reduce the risk of that I wrote what might be considered an excessive amount of tests for such a small change.Closes #32656.