Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Robustify GitHub Actions workflow #588

Closed
phil-blain opened this issue Mar 30, 2021 · 2 comments · Fixed by #586
Closed

Robustify GitHub Actions workflow #588

phil-blain opened this issue Mar 30, 2021 · 2 comments · Fixed by #586
Assignees
@apcraig
Labels
Scripts

Comments

@phil-blain
Copy link
Member

phil-blain commented Mar 30, 2021 •
edited
Loading

Sometimes the GitHub Actions CI runner encounters a broken connection to Zenodo:
https://github.com/CICE-Consortium/CICE/pull/587/checks?check_run_id=2230759756#step:11:58435

2021-03-30T19:22:27.0277120Z 2021-03-30 19:22:26 (4.69 MB/s) - Connection closed at byte 2537720689. Retrying.
2021-03-30T19:22:27.0277510Z 
2021-03-30T19:22:27.8913650Z --2021-03-30 19:22:27--  (try: 2)  https://zenodo.org/record/3728364/files/CICE_data_gx3_forcing_JRA55-20200320.tar.gz
2021-03-30T19:22:28.0625210Z Connecting to zenodo.org (zenodo.org)|137.138.76.77|:443... connected.
2021-03-30T19:22:28.5772140Z HTTP request sent, awaiting response... 429 Too Many Requests
2021-03-30T19:22:28.5773870Z 2021-03-30 19:22:28 ERROR 429: Too Many Requests.

The 'Download input data' step is successful in the workflow, despite wget encoutering an error. It's not the first time this happens, and I think I understand why. Each step in the workflow is ran by the shell specified by the "shell" keyword, or a platform default:
https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#using-a-specific-shell

We use the Cshell ("csh") (which in theory is not necessary, see #557) and thus use a custom shell:
https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#custom-shell

CICE/.github/workflows/test-cice.yml

Lines 18 to 20 in e204fb8

defaults:
run:
shell: /bin/csh {0}

but this invocation of csh does not ensure that any error encountered terminates the process, which is what is done by default for ex. Bash (see the Github documentation). So we should at least be using /bin/csh -e {0} to ensure errors are caught.
Also, we should invoke 'wget' with the --progress=dot:giga option to avoid countless number of lines in the CI log since the input data is really big.

Originally posted by @phil-blain in #587 (comment)
@phil-blain
Copy link
Member Author

[Aside: maybe a new label "CI" or "Continuous Integration" could be added for these types of issues.]

@apcraig apcraig self-assigned this Apr 1, 2021
apcraig added a commit to apcraig/CICE that referenced this issue Apr 1, 2021
@apcraig
Update gx3 and gx1 input filenames
4dc3bf1 
Update CI wget implementation CICE-Consortium#588
Create variable in ice_forcing.F90 called mixed_layer_depth_default and use it instead of c20
Fix bug in hourly output, created in time manager update CICE-Consortium#589
Set start year for all runs to 2005 and turn leap years on by default
Update documentation in history output section to add information about hist_avg namelist, noted in CICE-Consortium#566
@apcraig apcraig mentioned this issue Apr 1, 2021
Merged
16 tasks
apcraig added a commit that referenced this issue Apr 6, 2021
@apcraig
Update gx1 initial condition, Icepack, tests, and version. Fix hmix d…
afc3a58 
…efault value (#586)

* Update gx1 ic

- update set_nml.gx1prod to match current production system
- add apr 1 test case for gx1
- update landice tests, use gx1
- fix hmix default #585
- delete old code in ice_forcing.F90 (should have been done in earlier PR)

* update gx1coreii initial condition

* update icepack

* add gx1 debug test, expected to fail

* Update version number to 6.2.0

* Update gx3 and gx1 input filenames
Update CI wget implementation #588
Create variable in ice_forcing.F90 called mixed_layer_depth_default and use it instead of c20
Fix bug in hourly output, created in time manager update #589
Set start year for all runs to 2005 and turn leap years on by default
Update documentation in history output section to add information about hist_avg namelist, noted in #566

* update ic filenames, tests, and documentation

* update color links in test results wiki

* update hmix initialization
@phil-blain
Copy link
Member Author

Thanks @apcraig!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@apcraig apcraig

Labels
Scripts
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Update gx1 initial condition, Icepack, tests, and version. Fix hmix default value apcraig/CICE
2 participants
@apcraig @phil-blain