Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bugfix: Phase Diagram use case writes all output into OUTPUT_BASE instead of INPUT_BASE #1706

Closed
23 tasks
georgemccabe opened this issue Jul 25, 2022 · 0 comments · Fixed by #1708 or #1709
Closed
23 tasks
Assignees
Labels
component: use case wrapper METplus: Configuration priority: high High Priority reporting: DTC NOAA BASE NOAA Office of Atmospheric Research DTC Project requestor: NOAA/EMC NOAA Environmental Modeling Center type: bug Fix something that is not working

Comments

@georgemccabe
Copy link
Collaborator

georgemccabe commented Jul 25, 2022

This came up in discussion #1703

Describe the Problem

The model_applications/s2s_mjo/UserScript_obsERA_obsOnly_PhaseDiagram use case (previously in s2s) fails because the use case tries to write output data to a directory under {INPUT_BASE}. In most cases the input data is owned by another user so they cannot write output to that area. All use cases should always write any output and intermediate files relative to {OUTPUT_BASE} to prevent any issues like this.

Expected Behavior

The use case should run without error when the INPUT_BASE is owned by another user and the user running the use case does not have permission to write into INPUT_BASE. All files created by the use case should be written under OUTPUT_BASE.

Environment

Describe your runtime environment:
*1. Machine: seneca
*2. OS: Debian Buster
*3. METplus 4.1.0 and develop

To Reproduce

Describe the steps to reproduce the behavior:
1. Set INPUT_BASE to /d1/projects/METplus/METplus_Data
2. Run UserScript_obsERA_obsOnly_PhaseDiagram use case

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

2792541

Define the Metadata

Assignee

  • Select engineer(s) or no engineer required
  • Select scientist(s) or no scientist required

Labels

  • Select component(s)
  • Select priority
  • Select requestor(s)

Projects and Milestone

  • Select Organization level Project for support of the current coordinated release
  • Select Repository level Project for development toward the next official release or add alert: NEED PROJECT ASSIGNMENT label
  • Select Milestone as the next bugfix version

Define Related Issue(s)

Consider the impact to the other METplus components.

Bugfix Checklist

See the METplus Workflow for details.

  • Complete the issue definition above, including the Time Estimate and Funding Source.
  • Fork this repository or create a branch of main_<Version>.
    Branch name: bugfix_<Issue Number>_main_<Version>_<Description>
  • Fix the bug and test your changes.
  • Add/update log messages for easier debugging.
  • Add/update unit tests.
  • Add/update documentation.
  • Add any new Python packages to the METplus Components Python Requirements table.
  • Push local changes to GitHub.
  • Submit a pull request to merge into main_<Version>.
    Pull request: bugfix <Issue Number> main_<Version> <Description>
  • Define the pull request metadata, as permissions allow.
    Select: Reviewer(s) and Linked issues
    Select: Organization level software support Project for the current coordinated release
    Select: Milestone as the next bugfix version
  • Iterate until the reviewer(s) accept and merge your changes.
  • Delete your fork or branch.
  • Complete the steps above to fix the bug on the develop branch.
    Branch name: bugfix_<Issue Number>_develop_<Description>
    Pull request: bugfix <Issue Number> develop <Description>
    Select: Reviewer(s) and Linked issues
    Select: Repository level development cycle Project for the next official release
    Select: Milestone as the next official version
  • Close this issue.
@georgemccabe georgemccabe added type: bug Fix something that is not working alert: NEED MORE DEFINITION Not yet actionable, additional definition required alert: NEED ACCOUNT KEY Need to assign an account key to this issue alert: NEED CYCLE ASSIGNMENT Need to assign to a release development cycle labels Jul 25, 2022
@georgemccabe georgemccabe added this to the METplus-5.0.0 milestone Jul 25, 2022
georgemccabe added a commit that referenced this issue Jul 25, 2022
…se the file i/o logic will fail if the directory does not exist
georgemccabe added a commit that referenced this issue Jul 25, 2022
…ASE instead of INPUT_BASE and changed input directory path to find the correct file in the INPUT_BASE
@georgemccabe georgemccabe removed alert: NEED MORE DEFINITION Not yet actionable, additional definition required alert: NEED CYCLE ASSIGNMENT Need to assign to a release development cycle labels Jul 25, 2022
georgemccabe added a commit that referenced this issue Jul 25, 2022
…se the file i/o logic will fail if the directory does not exist
@TaraJensen TaraJensen added reporting: DTC NOAA BASE NOAA Office of Atmospheric Research DTC Project and removed alert: NEED ACCOUNT KEY Need to assign an account key to this issue labels Jul 25, 2022
@georgemccabe georgemccabe changed the title Phase Diagram use case fails because output is attempted to write into INPUT_BASE Bugfix: Phase Diagram use case writes all output into OUTPUT_BASE instead of INPUT_BASE Jul 27, 2022
georgemccabe pushed a commit that referenced this issue Apr 6, 2023
Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>
Co-authored-by: johnhg <johnhg@ucar.edu>
Co-authored-by: cristianastan2 <cstan@gmu.edu>
Co-authored-by: John Halley Gotway <johnhg@ucar.edu>
Co-authored-by: bikegeek <minnawin@ucar.edu>
Co-authored-by: Lisa Goodrich <lisag@ucar.edu>
Co-authored-by: Julie Prestopnik <jpresto@seneca.rap.ucar.edu>
Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com>
Co-authored-by: Hank Fisher <fisherh@ucar.edu>
Co-authored-by: Dan Adriaansen <dadriaan@ucar.edu>
Co-authored-by: jprestop <jpresto@ucar.edu>
Co-authored-by: Tracy Hertneky <hertneky@seneca.rap.ucar.edu>
Co-authored-by: j-opatz <59586397+j-opatz@users.noreply.github.com>
Co-authored-by: lisagoodrich <33230218+lisagoodrich@users.noreply.github.com>
Co-authored-by: bikegeek <3753118+bikegeek@users.noreply.github.com>
Co-authored-by: j-opatz <jopatz@ucar.edu>
Co-authored-by: Will Mayfield <59745143+willmayfield@users.noreply.github.com>
Co-authored-by: metplus-bot <97135045+metplus-bot@users.noreply.github.com>
Co-authored-by: Tracy Hertneky <39317287+hertneky@users.noreply.github.com>
fixing some spacing #770
Fixing formatting, typos etc.  #770
Fixing formatting  #770
fixing typos, spacing, etc. #770
fixing typos, grammar and punctuation. #770
Fixing TOC and coding block #770
fixing note #770
fix.  Updating bolding and italics for file names and directories #770
fixing space #770
fixing conflict #2 #770
fixing typos #770
fixing typos #2 #770
fixing formatting #770
fixing errors #650
fixing errors take 2 #650
fixing errors take 3 #650
fixing errors take 4 #650
fixing errors take 5 #650
fixing question order #650
fix #1706 fix PhaseDiagram use case to avoid writing into INPUT_BASE (#1708)
fix #1713 develop METPLOTPY_BASE (#1715)
fix #1691 remove whitespace from output file paths (#1721)
fix Contributor's Guide GitHub Workflow page (#1774)
fix release (#1790)
fix GitHub Actions warnings (#1864)
fix #1884 develop PCPCombine {custom} in subtract method (#1887)
fix #1939 develop - failure reading obs when zipped file also exists (#1941)
Closes #1986
fix develop Fix broken documentation links (#2004)
fix #2026 develop StatAnalysis looping (#2028)
fix priority of obs_window config variables so that wrapper-specific version is preferred over generic OBS_WINDOW_BEGIN/END (#2062)
fix #2070 var list numeric order (#2072)
fix #2087 develop docs_pdf (#2091)
fix #2096/#2098 develop - fix skip if output exists and do not error if no commands were run (#2099)
georgemccabe pushed a commit that referenced this issue Apr 28, 2023
Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>
Co-authored-by: johnhg <johnhg@ucar.edu>
Co-authored-by: cristianastan2 <cstan@gmu.edu>
Co-authored-by: John Halley Gotway <johnhg@ucar.edu>
Co-authored-by: bikegeek <minnawin@ucar.edu>
Co-authored-by: Lisa Goodrich <lisag@ucar.edu>
Co-authored-by: Julie Prestopnik <jpresto@seneca.rap.ucar.edu>
Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com>
Co-authored-by: Hank Fisher <fisherh@ucar.edu>
Co-authored-by: Dan Adriaansen <dadriaan@ucar.edu>
Co-authored-by: jprestop <jpresto@ucar.edu>
Co-authored-by: Tracy Hertneky <hertneky@seneca.rap.ucar.edu>
Co-authored-by: Giovanni Rosa <giovanni.rosa@unimol.it>
Co-authored-by: j-opatz <59586397+j-opatz@users.noreply.github.com>
Co-authored-by: lisagoodrich <33230218+lisagoodrich@users.noreply.github.com>
Co-authored-by: bikegeek <3753118+bikegeek@users.noreply.github.com>
Co-authored-by: j-opatz <jopatz@ucar.edu>
Co-authored-by: Will Mayfield <59745143+willmayfield@users.noreply.github.com>
Co-authored-by: metplus-bot <97135045+metplus-bot@users.noreply.github.com>
Co-authored-by: Tracy Hertneky <39317287+hertneky@users.noreply.github.com>
Co-authored-by: Giovanni Rosa <g.rosa1@studenti.unimol.it>
fixing errors #650
fixing errors take 2 #650
fixing errors take 3 #650
fixing errors take 4 #650
fixing errors take 5 #650
fixing question order #650
fix #1706 fix PhaseDiagram use case to avoid writing into INPUT_BASE (#1708)
fix #1713 develop METPLOTPY_BASE (#1715)
fix #1691 remove whitespace from output file paths (#1721)
fix Contributor's Guide GitHub Workflow page (#1774)
fix release (#1790)
fix GitHub Actions warnings (#1864)
fix #1884 develop PCPCombine {custom} in subtract method (#1887)
fix #1939 develop - failure reading obs when zipped file also exists (#1941)
Closes #1986
fix develop Fix broken documentation links (#2004)
fix #2026 develop StatAnalysis looping (#2028)
fix priority of obs_window config variables so that wrapper-specific version is preferred over generic OBS_WINDOW_BEGIN/END (#2062)
fix #2070 var list numeric order (#2072)
fix #2087 develop docs_pdf (#2091)
fix #2096/#2098 develop - fix skip if output exists and do not error if no commands were run (#2099)
Fix for Dockerfile smell DL4000 (#2112)
fix #2082 develop regrid.convert/censor_thresh/censor_val (#2140)
fix #2082 main_v5.0 regrid.convert/censor_thresh/censor_val (#2101)
fix #2137 develop PointStat -obs_valid_beg/end (#2141)
JohnHalleyGotway pushed a commit that referenced this issue May 11, 2023
Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>
Co-authored-by: johnhg <johnhg@ucar.edu>
Co-authored-by: cristianastan2 <cstan@gmu.edu>
Co-authored-by: John Halley Gotway <johnhg@ucar.edu>
Co-authored-by: bikegeek <minnawin@ucar.edu>
Co-authored-by: Lisa Goodrich <lisag@ucar.edu>
Co-authored-by: Julie Prestopnik <jpresto@seneca.rap.ucar.edu>
Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com>
Co-authored-by: Hank Fisher <fisherh@ucar.edu>
Co-authored-by: Dan Adriaansen <dadriaan@ucar.edu>
Co-authored-by: jprestop <jpresto@ucar.edu>
Co-authored-by: Tracy Hertneky <hertneky@seneca.rap.ucar.edu>
Co-authored-by: Giovanni Rosa <giovanni.rosa@unimol.it>
Co-authored-by: j-opatz <59586397+j-opatz@users.noreply.github.com>
Co-authored-by: lisagoodrich <33230218+lisagoodrich@users.noreply.github.com>
Co-authored-by: bikegeek <3753118+bikegeek@users.noreply.github.com>
Co-authored-by: j-opatz <jopatz@ucar.edu>
Co-authored-by: Will Mayfield <59745143+willmayfield@users.noreply.github.com>
Co-authored-by: metplus-bot <97135045+metplus-bot@users.noreply.github.com>
Co-authored-by: Tracy Hertneky <39317287+hertneky@users.noreply.github.com>
Co-authored-by: Giovanni Rosa <g.rosa1@studenti.unimol.it>
fixing errors #650
fixing errors take 2 #650
fixing errors take 3 #650
fixing errors take 4 #650
fixing errors take 5 #650
fixing question order #650
fix #1706 fix PhaseDiagram use case to avoid writing into INPUT_BASE (#1708)
fix #1713 develop METPLOTPY_BASE (#1715)
fix #1691 remove whitespace from output file paths (#1721)
fix Contributor's Guide GitHub Workflow page (#1774)
fix release (#1790)
fix GitHub Actions warnings (#1864)
fix #1884 develop PCPCombine {custom} in subtract method (#1887)
fix #1939 develop - failure reading obs when zipped file also exists (#1941)
Closes #1986
fix develop Fix broken documentation links (#2004)
fix #2026 develop StatAnalysis looping (#2028)
fix priority of obs_window config variables so that wrapper-specific version is preferred over generic OBS_WINDOW_BEGIN/END (#2062)
fix #2070 var list numeric order (#2072)
fix #2087 develop docs_pdf (#2091)
fix #2096/#2098 develop - fix skip if output exists and do not error if no commands were run (#2099)
Fix for Dockerfile smell DL4000 (#2112)
fix #2082 develop regrid.convert/censor_thresh/censor_val (#2140)
fix #2082 main_v5.0 regrid.convert/censor_thresh/censor_val (#2101)
fix #2137 develop PointStat -obs_valid_beg/end (#2141)
fix failured introduced by urllib3 (see urllib3/urllib3#2168)
fix #2161 develop PCPCombine additional field arguments in -subtract mode (#2162)
georgemccabe pushed a commit that referenced this issue Jun 5, 2023
Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>
Co-authored-by: johnhg <johnhg@ucar.edu>
Co-authored-by: cristianastan2 <cstan@gmu.edu>
Co-authored-by: John Halley Gotway <johnhg@ucar.edu>
Co-authored-by: bikegeek <minnawin@ucar.edu>
Co-authored-by: Lisa Goodrich <lisag@ucar.edu>
Co-authored-by: Julie Prestopnik <jpresto@seneca.rap.ucar.edu>
Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com>
Co-authored-by: Hank Fisher <fisherh@ucar.edu>
Co-authored-by: Dan Adriaansen <dadriaan@ucar.edu>
Co-authored-by: jprestop <jpresto@ucar.edu>
Co-authored-by: Tracy Hertneky <hertneky@seneca.rap.ucar.edu>
Co-authored-by: Giovanni Rosa <giovanni.rosa@unimol.it>
Co-authored-by: j-opatz <59586397+j-opatz@users.noreply.github.com>
Co-authored-by: Mrinal Biswas <biswas@seneca.rap.ucar.edu>
Co-authored-by: lisagoodrich <33230218+lisagoodrich@users.noreply.github.com>
Co-authored-by: bikegeek <3753118+bikegeek@users.noreply.github.com>
Co-authored-by: j-opatz <jopatz@ucar.edu>
Co-authored-by: Will Mayfield <59745143+willmayfield@users.noreply.github.com>
Co-authored-by: metplus-bot <97135045+metplus-bot@users.noreply.github.com>
Co-authored-by: Tracy Hertneky <39317287+hertneky@users.noreply.github.com>
Co-authored-by: Giovanni Rosa <g.rosa1@studenti.unimol.it>
Co-authored-by: mrinalbiswas <biswas@ucar.edu>
fixing errors #650
fixing errors take 2 #650
fixing errors take 3 #650
fixing errors take 4 #650
fixing errors take 5 #650
fixing question order #650
fix #1706 fix PhaseDiagram use case to avoid writing into INPUT_BASE (#1708)
fix #1713 develop METPLOTPY_BASE (#1715)
fix #1691 remove whitespace from output file paths (#1721)
fix Contributor's Guide GitHub Workflow page (#1774)
fix release (#1790)
fix GitHub Actions warnings (#1864)
fix #1884 develop PCPCombine {custom} in subtract method (#1887)
fix #1939 develop - failure reading obs when zipped file also exists (#1941)
Closes #1986
fix develop Fix broken documentation links (#2004)
fix #2026 develop StatAnalysis looping (#2028)
fix priority of obs_window config variables so that wrapper-specific version is preferred over generic OBS_WINDOW_BEGIN/END (#2062)
fix #2070 var list numeric order (#2072)
fix #2087 develop docs_pdf (#2091)
fix #2096/#2098 develop - fix skip if output exists and do not error if no commands were run (#2099)
Fix for Dockerfile smell DL4000 (#2112)
fix #2082 develop regrid.convert/censor_thresh/censor_val (#2140)
fix #2082 main_v5.0 regrid.convert/censor_thresh/censor_val (#2101)
fix #2137 develop PointStat -obs_valid_beg/end (#2141)
fix failured introduced by urllib3 (see urllib3/urllib3#2168)
fix #2161 develop PCPCombine additional field arguments in -subtract mode (#2162)
fix #2168 develop - StatAnalysis time shift (#2169)
fix releases. (#2183)
fix #2189 develop - spaces in complex thresholds (#2191)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component: use case wrapper METplus: Configuration priority: high High Priority reporting: DTC NOAA BASE NOAA Office of Atmospheric Research DTC Project requestor: NOAA/EMC NOAA Environmental Modeling Center type: bug Fix something that is not working
Projects
No open projects
3 participants