Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI workflow ensures directories are writable before environment cleanup. #1273

Merged
merged 1 commit into from
Aug 23, 2024

Conversation

eap
Copy link
Collaborator

@eap eap commented Aug 23, 2024

A change in #1270 uncovered a behavior where go mod derived package install modules cannot be removed from our scratch directory due to undesirable folder permissions. This has been observed before and golang maintainers marked the behavior as WAI suggesting that people maintain these directory trees with go-language tools. In our case this is not highly feasible due to spack's intervention in the project structure meaning we must find a workaround.

The workaround here uses find to update directory permissions to ensure that write/execute is set on all directories in the tree. This has been tested locally but due to environment carryover we cannot ensure it has the desired effect for go packages in our CI system until it is submitted. (Ie; testing would require re-running an affected workflow and all other pull requests would still be subject to the issue and would break).

Testing

CI tests here will exercise the code and ensure it doesn't break our existing environment.

Issues addressed.

Bugfix for regression noted in #1270

@eap eap marked this pull request as ready for review August 23, 2024 19:00
@eap eap changed the title CI cleanup ensures directories are writable before environment cleanup. CI workflow ensures directories are writable before environment cleanup. Aug 23, 2024
@climbfuji
Copy link
Collaborator

Not just directories, but also files. Can you add

find ./* -type f -exec chmod u+w {} \;

please?

@eap
Copy link
Collaborator Author

eap commented Aug 23, 2024

@climbfuji - In testing, fixing the directory permission was adequate to allow the files to be deleted even with the read-only 444 permission on files. I also tested the full approach using a local go mod build tree and was able to clear the contents.

Works in osX and linux.

# Create directory and file both with `-r--r--r--` permission.
$ cd /tmp
$ mkdir not_deletable
$ echo "deletethis" > ./not_deletable/deleteme
$ chmod 444 not_deletable/deleteme
$ chmod 444 ./not_deletable

# Cannot delete this directory or contents.
$ rm -rf ./not_deletable
rm: cannot remove './not_deletable/deleteme': Permission denied

# Change directory only, it can be deleted.
$ chmod u+wx ./not_deletable
$ rm -rf ./not_deletable

@climbfuji
Copy link
Collaborator

Alright, if that's the case then let's go ahead and merge w/o further CI testing. We can always come back if we need to.

@climbfuji climbfuji merged commit a9c3d63 into JCSDA:develop Aug 23, 2024
8 checks passed
DavidHuber-NOAA added a commit to DavidHuber-NOAA/spack-stack that referenced this pull request Aug 26, 2024
* jcsda/develop:
  Bump Python to 3.11.7 (JCSDA#1217)
  Add a clause in the cleanup to fix directory permissions (JCSDA#1273)
  Bug fix: configure neptune-env variants in three templates: neptune-dev, skylab-dev, unified-dev (JCSDA#1268)
  Update site configs for Atlantis, Narwhal, Nautilus (JCSDA#1266)
  Configuration for bufr_query@0.0.2 and eckit@1.27.0 (JCSDA#1240)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants