Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add R2D2 mysql dependencies and complete site config updates for unified environment #463

Merged
merged 81 commits into from
Mar 6, 2023

Conversation

climbfuji
Copy link
Collaborator

@climbfuji climbfuji commented Feb 1, 2023

Description

Changes in this PR:

  • Add dependencies for the upcoming R2D2 mysql backend implementation to various configs files,
  • Update the submodule pointer for spack for the corresponding changes there
  • Update CI scripts and documentation - avoid using the padded_length logic for now, too many issues. Instead, alwaus install in same location (env-name/env-dir)
  • Add Ubuntu CI on AWS EC2, currently only with Intel on the AWS Parallelcluster head node of JCSDA
  • Also update remaining site configs with changes required for the unified environment. The unified environment was introduced in Create template for unified environment, test install on orion and AWS parallelcluster #454, PR Update compiler/MPI entries in site configs for the UE #480 updated some of the site configs, this PR adds the remaining site configurations for fully supported sites

With this PR and previous PRs, the following sites have updated compiler/package files for (a) the unified environment and (b) mysql. For sites marked with an asterisk *, the updated unified-environment was built successfully (manually).

  • Orion (Intel, GNU) *
  • S4 (Intel, GNU) *
  • Cheyenne (Intel, GNU) *
  • Casper (Intel)
  • Discover (Intel, GNU) *
  • Hera (Intel, GNU)
  • Jet (Intel, GNU)
  • AWS Parallelcluster (Intel) *
  • Narwhal (Intel, GNU)
  • Gaea
  • noaa-aws
  • noaa-azure
  • noaa-gcloud

The following sites are still missing, but we can do those in a follow-up PR (as long as before we roll out the spack-stack 1.3.0 release) - but may want to consider moving this to a different "category" of support, as discussed in the last spack-stack meeting:

  • Acorn
  • Gaea c5
  • emc-rhel (if building the unified-dev env)
  • frontera
  • comet

Dependencies

Issues

Fixes #383

Testing

  • New Ubuntu and macOS CI tests passed for 45c5520
  • Manual tests on platforms listed above and on Dom's macOS

ulmononian and others added 24 commits September 27, 2022 16:41
…es/hpc-dev-v1 to configs/templates/unified-dev
Copy link
Collaborator

@srherbener srherbener left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the documentation updates I'm not seeing updated instructions for the Mac regarding the need to install mysql with homebrew and to mark mysql as an external package. Should brew install mysql be added to the list showing the other brew install commands? Also add the spack --not-buildable --scope system mysql to the list of spack find commands?

Also, should we put in the tip about shutting off the auto updates of dependencies in the Mac instructions. Ie, add export HOMEBREW_NO_AUTO_UPDATE=1 to your environment before running brew installs?

@climbfuji
Copy link
Collaborator Author

In the documentation updates I'm not seeing updated instructions for the Mac regarding the need to install mysql with homebrew and to mark mysql as an external package. Should brew install mysql be added to the list showing the other brew install commands? Also add the spack --not-buildable --scope system mysql to the list of spack find commands?

Also, should we put in the tip about shutting off the auto updates of dependencies in the Mac instructions. Ie, add export HOMEBREW_NO_AUTO_UPDATE=1 to your environment before running brew installs?

Indeed. Now the instructions are there. See 7952569.

Copy link
Collaborator

@srherbener srherbener left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great. Thanks!

@srherbener
Copy link
Collaborator

I'm testing on my Mac now with the proper handling of the mysql package. I should have a result later this afternoon.

Copy link
Collaborator

@ericlingerfelt ericlingerfelt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @climbfuji!

Copy link
Collaborator

@ulmononian ulmononian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

5stars. seems ready to roll!

@climbfuji climbfuji merged commit dc412b0 into JCSDA:develop Mar 6, 2023
@climbfuji climbfuji deleted the feature/r2d2_mysql branch March 6, 2023 22:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add new R2D2 thirdparty dependencies for mysql backend
4 participants