Fix chdir and add tests for fsutils functions #45
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
aerorahul
reviewed
Nov 14, 2024
src/wxflow/fsutils.py
Outdated
| if missing_ok: | ||
| logger.warning(f"WARNING cannot remove the target path {dir_path} because it does not exist") | ||
| else: | ||
| raise exc |
There was a problem hiding this comment.
Suggested change
| raise exc | |
| raise FileNotFoundError() |
Should this raise this error specifically. Not sure how this works, if its trying to remove something that is not existant.
There was a problem hiding this comment.
Since exc is the original FileNotFoundError, raising exc is equivalent to raising FileNotFoundError. But, raising a custom FileNotFoundError with an more specific error message would be useful. I'll add that in.
So if the file doesn't exist, nothing will happen but rmtree will return a FileNotFoundError. If missing_ok=False, then this Exception will be passed on. Otherwise, you get a warning for trying.
aerorahul
reviewed
Nov 14, 2024
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #45 +/- ##
===========================================
+ Coverage 50.12% 52.41% +2.29%
===========================================
Files 18 18
Lines 1652 1679 +27
Branches 339 296 -43
===========================================
+ Hits 828 880 +52
+ Misses 765 749 -16
+ Partials 59 50 -9 ☔ View full report in Codecov by Sentry. |
aerorahul
approved these changes
Nov 14, 2024
This was referenced Nov 15, 2024
Merged
DavidHuber-NOAA
added a commit
to NOAA-EMC/global-workflow
that referenced
this pull request
Dec 9, 2024
…3105) # Description This adds the capability to archive the experiment directory. Additionally, this adds options to run `git status` and `git diff` on the `HOMEgfs` global workflow (but not the submodules) and store that information within the experiment directory's archive. These options are specified in `config.base` with the following defaults: ```bash export ARCH_EXPDIR='YES' # Archive the EXPDIR configs, XML, and database export ARCH_EXPDIR_FREQ=0 # How often to archive the EXPDIR in hours or 0 for first and last cycle only export ARCH_HASHES='YES' # Archive the hashes of the GW and submodules and 'git status' for each; requires ARCH_EXPDIR export ARCH_DIFFS='NO' # Archive the output of 'git diff' for the GW; requires ARCH_EXPDIR ``` Resolves #2994 # Type of change - [x] New feature (adds functionality) # Change characteristics <!-- Choose YES or NO from each of the following and delete the other --> - Is this a breaking change (a change in existing functionality)? NO - Does this change require a documentation update? YES - Does this change require an update to any of the following submodules? YES (If YES, please add a link to any PRs that are pending.) - [x] wxflow NOAA-EMC/wxflow#45 # How has this been tested? - [x] Local archiving on Hercules for a C48_ATM case - [x] Cycled testing on Hercules with `ARCH_DIFFS=YES` and `ARCH_EXPDIR_FREQ=6,12` - [x] Testing with `ARCH_EXPDIR=NO` or `ARCH_HASHES=NO` # Checklist - [x] Any dependent changes have been merged and published - [x] My code follows the style guidelines of this project - [x] I have performed a self-review of my own code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have documented my code, including function, input, and output descriptions - [x] My changes generate no new warnings - [x] New and existing tests pass with my changes - [x] This change is covered by an existing CI test or a new one has been added - [x] Any new scripts have been added to the .github/CODEOWNERS file with owners - [x] I have made corresponding changes to the system documentation if necessary --------- Co-authored-by: Walter Kolczynski - NOAA <Walter.Kolczynski@noaa.gov>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This modifies
chdirso that it produces an error if the target directory is not available and returns to the original working directory when thewithcompletes without producing a warning.Tests were added to verify that
chdir, as well as the otherfsutils, run as expected. While writing these tests, the following issues were found and fixed:mkdir_pwas simplified to use theos.mkdirexist_ok=Trueparameter instead of attempting to handle that in wxflow.missing_okparameter tormdirto allow missing directories to be skipped without error.rm_perror handling and added amissing_okparameter.Lastly, the only function that did not have a test added was
chgrp. I am not sure what groups are available on GitHub runners, so I am not sure how this could be tested reliably.Type of change
How Has This Been Tested?
Checklist