Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reduce R&D archive job memory requests #1144

Closed
KateFriedman-NOAA opened this issue Dec 5, 2022 · 2 comments · Fixed by #1183
Closed

Reduce R&D archive job memory requests #1144

KateFriedman-NOAA opened this issue Dec 5, 2022 · 2 comments · Fixed by #1183
Assignees
@KateFriedman-NOAA
Labels
bug Something isn't working

Comments

@KateFriedman-NOAA
Copy link
Member

Description

Raghu Reddy of Hera reported global-workflow user archive jobs backing up from over requesting memory on the service queue (login nodes). Need to reduce memory request for archive jobs on R&Ds. Value set from WCOSS2 commit (50GB) is too high for R&Ds without separate service nodes and likely too much for the archive jobs overall.

Requirements

Archive jobs on Hera and Orion request more appropriate memory amounts.
@KateFriedman-NOAA KateFriedman-NOAA added the bug Something isn't working label Dec 5, 2022
@KateFriedman-NOAA KateFriedman-NOAA self-assigned this Dec 5, 2022
@KateFriedman-NOAA KateFriedman-NOAA mentioned this issue Dec 5, 2022
Closed
KateFriedman-NOAA added a commit to KateFriedman-NOAA/global-workflow that referenced this issue Dec 5, 2022
@KateFriedman-NOAA
Update archive job memory request value
7ad555e 
- Revert memory request back to 2048M on R&Ds (may change after testing).
- Keep 50GB setting for WCOSS2.

Refs NOAA-EMC#1144
KateFriedman-NOAA added a commit to KateFriedman-NOAA/global-workflow that referenced this issue Dec 6, 2022
@KateFriedman-NOAA
Adjust archive job memory value based on machine
3dff45f 
- Revert default (R&D) archive memory value back to 2048M.
- Keep the 50GB value for WCOSS2 only.

Refs NOAA-EMC#1144
This was referenced Dec 6, 2022
Merged
Merged
@KateFriedman-NOAA
Copy link
Member Author

Testing the new POSTAMBLE_CMD on Hera. Will also test elsewhere and include reduced archive memory request. Once complete will submit PR into develop to complete this issue and issue #1145.

KateFriedman-NOAA added a commit to KateFriedman-NOAA/global-workflow that referenced this issue Dec 15, 2022
@KateFriedman-NOAA
Update archive job memory request value
5c6d266 
- Revert memory request back to 2048M on R&Ds (may change after testing).
- Keep 50GB setting for WCOSS2.

Refs NOAA-EMC#1144
@KateFriedman-NOAA KateFriedman-NOAA mentioned this issue Dec 15, 2022
Merged
9 tasks
@KateFriedman-NOAA
Copy link
Member Author

Separating this from the POSTAMBLE_CMD work in #1145. More work is needed on that still.

KateFriedman-NOAA added a commit that referenced this issue Dec 15, 2022
@KateFriedman-NOAA
Update archive job memory request value for R&Ds (#1183)
98545da 
- Revert memory request back to 2048M on R&Ds.
- Keep 50GB setting for WCOSS2.

Refs #1144
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@KateFriedman-NOAA KateFriedman-NOAA

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Update archive job memory request value for R&Ds KateFriedman-NOAA/global-workflow
1 participant
@KateFriedman-NOAA