plugins/bundle: use unique temporary files #4786
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
In order to use the feature to persist activated bundles to disk in a cloud environment with shared storage, e.g. Kubernetes with the Amazon EFS storage driver, each instance of OPA needs to either synchronize their access to the temporary file using advisory file locks, or use unique temporary files. If not, then the following situation may occur: p1: open and trunc tmp file p1: write to tmp file p2: open and trunc tmp file p1: rename tmp file to dst p2: write to tmp file p2: rename tmp file to dst This may then lead to the persisted bundle being truncated or corrupted. Here the approach of using unique temporary files is chosen because it avoids the overhead of introducing file locks, and the additional dependency since Go lacks any such mechanisms in the standard library. This solution should avoid truncated or corrupt bundles as `rename()` is guaranteed to be atomic, even in file systems like NFS. Fixes: open-policy-agent#4782 Signed-off-by: Fredrik Appelros <fredrik.appelros@sinch.com>
JAORMX
approved these changes
Jun 22, 2022
srenatus
reviewed
Jul 12, 2022
| } | ||
|
|
||
| dest, err := os.OpenFile(filepath.Join(path, filename), os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0644) | ||
| dest, err := os.CreateTemp(path, ".bundle.tar.gz.*.tmp") |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In order to use the feature to persist activated bundles to disk in a
cloud environment with shared storage, e.g. Kubernetes with the Amazon
EFS storage driver, each instance of OPA needs to either synchronize
their access to the temporary file using advisory file locks, or use
unique temporary files. If not, then the following situation may occur:
p1: open and trunc tmp file
p1: write to tmp file
p2: open and trunc tmp file
p1: rename tmp file to dst
p2: write to tmp file
p2: rename tmp file to dst
This may then lead to the persisted bundle being truncated or corrupted.
Here the approach of using unique temporary files is chosen because it
avoids the overhead of introducing file locks, and the additional
dependency since Go lacks any such mechanisms in the standard library.
This solution should avoid truncated or corrupt bundles as
rename()isguaranteed to be atomic, even in file systems like NFS.
Fixes: #4782
Signed-off-by: Fredrik Appelros fredrik.appelros@sinch.com