Code

Build aws updates prototype#305

Code

Closed

TerrenceMcGuinness-NOAA wants to merge 102 commits intodevelopTerrenceMcGuinness-NOAA/global-workflow:developfrom build_aws_updates_prototypeTerrenceMcGuinness-NOAA/global-workflow:build_aws_updates_prototypeCopy head branch name to clipboard

+73-7

Commits

Commits on Sep 25, 2024

• added GitHub pipeline for runner on pw

  

  tmcguinness

  committed
  0436e69

  Copy full SHA for 0436e69

• added repos name to chekcout

  

  tmcguinness

  committed
  7829f7d

  Copy full SHA for 7829f7d

• added ssh to hera from pw before actions/checkout@v3

  

  tmcguinness

  committed
  fc4bf1f

  Copy full SHA for fc4bf1f

• had to create dir on hera first

  

  tmcguinness

  committed
  a4126a3

  Copy full SHA for a4126a3

• ssh then two commands

  

  tmcguinness

  committed
  82f4b99

  Copy full SHA for 82f4b99

• cd to target dir firsts the mk

  

  tmcguinness

  committed
  581c31e

  Copy full SHA for 581c31e

• added -t and EOF for getting ssh to run in one shot

  

  tmcguinness

  committed
  9d79705

  Copy full SHA for 9d79705

• skipping checkout@v3 and pushing forwrad

  

  tmcguinness

  committed
  3d15521

  Copy full SHA for 3d15521

Commits on Sep 26, 2024

• ready to test CI pw pipeline using self hosted runner on pw

  

  tmcguinness

  committed
  25bc12b

  Copy full SHA for 25bc12b

• removed default branch in dispatch

  

  tmcguinness

  committed
  cb7fa7a

  Copy full SHA for cb7fa7a

• removed branch input as it not needed

  

  tmcguinness

  committed
  d56f313

  Copy full SHA for d56f313

• set BRANCH_NAME and PR_NUMBER in advance

  

  tmcguinness

  committed
  658e745

  Copy full SHA for 658e745

• set BRANCH_NAME and PR_NUMBER in advance

  

  tmcguinness

  committed
  08bc4b8

  Copy full SHA for 08bc4b8

• only add TEST_DIR in env and use githug when needed

  

  tmcguinness

  committed
  9f910f7

  Copy full SHA for 9f910f7

• had extra steps section now removed

  

  tmcguinness

  committed
  8d2fcfd

  Copy full SHA for 8d2fcfd

• forgot to put in expression for branch in checkout

  

  tmcguinness

  committed
  3c207b4

  Copy full SHA for 3c207b4

• needed env for TEST_DIR on subsquent sections

  

  tmcguinness

  committed
  0c8b9fa

  Copy full SHA for 0c8b9fa

• misspelled TEST_DIR in a few places

  

  tmcguinness

  committed
  00ff9f9

  Copy full SHA for 00ff9f9

• Had tags wrong for running on Runner in subsquent jobs

  

  tmcguinness

  committed
  57b8cd6

  Copy full SHA for 57b8cd6

• added shared env between jobs using ouputs for TEST_DIR

  

  Terrence McGuinness

  authored and

  Terrence McGuinness

  committed
  f78d573

  Copy full SHA for f78d573

• missing header and on: dispatch

  

  Terry McGuinness

  committed
  08ab7ec

  Copy full SHA for 08ab7ec

Commits on Sep 27, 2024

• fix new location of create_experment and added local env def into next job

  

  tmcguinness

  committed
  00f2d9b

  Copy full SHA for 00f2d9b

• Merge branch 'test_pw_ghrunner' of github.com:TerrenceMcGuinness-NOAA/global-workflow into test_pw_ghrunner

  

  tmcguinness

  committed
  09688e8

  Copy full SHA for 09688e8

• moved env section in create-experiments

  

  tmcguinness

  committed
  c0734a9

  Copy full SHA for c0734a9

• corrected path to getting TEST_DIR address in create-experiments

  

  tmcguinness

  committed
  146aa62

  Copy full SHA for 146aa62

• moved env for HOMEgfs into steps on create

  

  tmcguinness

  committed
  09f8075

  Copy full SHA for 09f8075

• updaed env for run experiments too

  

  tmcguinness

  committed
  daddf59

  Copy full SHA for daddf59

• update env section to define HOMEgfs in run

  

  tmcguinness

  committed
  d8b9b13

  Copy full SHA for d8b9b13

• need pr in path for cases

  

  tmcguinness

  committed
  6fa30fb

  Copy full SHA for 6fa30fb

• hard coded id so not to have to rebuild

  

  tmcguinness

  committed
  c4c99df

  Copy full SHA for c4c99df

• added overwrite to create experment

  

  tmcguinness

  committed
  71a8d87

  Copy full SHA for 71a8d87

• removed skips

  

  tmcguinness

  committed
  70e249a

  Copy full SHA for 70e249a

Commits on Sep 30, 2024

• removed direct git clone and used cross mounted path to RHHPCS directly using pw env inorder to use github action checkout@v3

  

  tmcguinness

  committed
  19c2c86

  Copy full SHA for 19c2c86

• made distinction between runners path and RDPHCS path with CHECKOUT_DIR_PW and TEST_DIR respectivily

  

  tmcguinness

  committed
  751054d

  Copy full SHA for 751054d

• fixed reference to CHECKOUT_DIR and updated submodule arguments to checkout@v3

  

  tmcguinness

  committed
  9b163e9

  Copy full SHA for 9b163e9

• updated GITHUB ENV and OUTPUT syntax

  

  tmcguinness

  committed
  bd8da4e

  Copy full SHA for bd8da4e

• added lfs and shallow depth for submodules to get checkout to work

  

  tmcguinness

  committed
  fd878f9

  Copy full SHA for fd878f9

• turn off git-lfs becasue it is not in the pw env

  

  tmcguinness

  committed
  15dbf30

  Copy full SHA for 15dbf30

• gdas checkout submodule failed again because it did not complete, removed lfs: false in case that was causing something to missed

  

  tmcguinness

  committed
  4a85a18

  Copy full SHA for 4a85a18

• added extra git configure to try to help with the checkout issues with the gdas.cd submodule clone issues

  

  tmcguinness

  committed
  ec4e850

  Copy full SHA for ec4e850

• upgrade checkout@v3 to checckout@v4.2.0

  

  tmcguinness

  committed
  7c11c96

  Copy full SHA for 7c11c96

• removed redundent specification of TEST_DIR when writing to GITHUB_OUTPUT

  

  tmcguinness

  committed
  bf42c6a

  Copy full SHA for bf42c6a

• also added verif-global.fd as a shallow clone

  

  tmcguinness

  committed
  0e91eba

  Copy full SHA for 0e91eba

Commits on Oct 1, 2024

• rsync check out dir after it is cloned on pw onto RDPHCS cross mounted link in HOME dir

  

  tmcguinness

  committed
  9c00b4e

  Copy full SHA for 9c00b4e

• worked on correct path for rsync to RDHPCS

  

  tmcguinness

  committed
  a5db8bd

  Copy full SHA for a5db8bd

• remore optons to chmod on dest in rysnc

  

  tmcguinness

  committed
  6454f41

  Copy full SHA for 6454f41

• Merge branch 'NOAA-EMC:develop' into test_pw_ghrunner

  TerrenceMcGuinness-NOAA

  authored
  e9f6e61

  Copy full SHA for e9f6e61

Commits on Oct 2, 2024

• update fms version to match spack stack on pw

  

  Parallel Works app-run user

  authored and

  Parallel Works app-run user

  committed
  9cfab8e

  Copy full SHA for 9cfab8e

• updated run.sh to use MACHINE_ID in load fv4gfs modules script

  

  Terry McGuinness

  committed
  249bd9f

  Copy full SHA for 249bd9f

Commits on Oct 3, 2024

• updated version of g2tmpl and change to module file noaacloud.intl for ufs_modle.fd

  

  Terry McGuinness

  committed
  6267f58

  Copy full SHA for 6267f58

• add patch for updating ufs_noaacloud.intel module file from global-workflow repo (temp)

  

  Terry McGuinness

  committed
  12fc5c0

  Copy full SHA for 12fc5c0

• add versioning for noaacloud and temp patch to ufs_module module file for noaacloud

  

  Terry McGuinness

  committed
  109b67c

  Copy full SHA for 109b67c

• removed githug action pipeline for pw to make this a build pr

  

  Terry McGuinness

  committed
  557a13f

  Copy full SHA for 557a13f

• moved source of ver for runtime in load fv3gfs modules back to before the module is loaded

  

  tmcguinness

  committed
  ca69f8a

  Copy full SHA for ca69f8a

• changed the runtime version of g2tmp down

  

  Terry McGuinness

  committed
  2366a9b

  Copy full SHA for 2366a9b

Commits on Oct 4, 2024

• update bucket to contrib in awspw hosts yaml file for ICs

  

  Parallel Works app-run user

  authored and

  Parallel Works app-run user

  committed
  068ada6

  Copy full SHA for 068ada6

• updated location of ICs (no bucker dir in /contrib)

  

  Terry McGuinness

  committed
  e6da9e8

  Copy full SHA for e6da9e8

• update start hour to 00 for C48_ATM

  

  Terry McGuinness

  committed
  5b07703

  Copy full SHA for 5b07703

Commits on Oct 7, 2024

• Merge branch 'NOAA-EMC:develop' into build_aws_updates

  TerrenceMcGuinness-NOAA

  authored
  ba76126

  Copy full SHA for ba76126

Commits on Oct 8, 2024

• added first pass at GitHub Action Pipelines for CI on pw

  

  Terry McGuinness

  committed
  40a8592

  Copy full SHA for 40a8592

• updated pw to paralleworks in lables to runners

  

  Terry McGuinness

  committed
  3616743

  Copy full SHA for 3616743

• trying checkout@v3 because Centos GLIB not supported

  

  Terry McGuinness

  committed
  267e39d

  Copy full SHA for 267e39d

Commits on Oct 9, 2024

• beefed up checkout action to v4 and added a bunch of helpers so run recurslivly on submodules as fast as possible

  

  tmcguinness

  committed
  1767803

  Copy full SHA for 1767803

• needed to indent steps on first stage

  

  tmcguinness

  committed
  2cd9d22

  Copy full SHA for 2cd9d22

• made sure suboudles where fully recursive

  

  tmcguinness

  committed
  a4de837

  Copy full SHA for a4de837

• updated submodule command for v4

  

  tmcguinness

  committed
  2c87707

  Copy full SHA for 2c87707

• add noaacloud config file for create experment bash script

  

  tmcguinness

  committed
  16cde87

  Copy full SHA for 16cde87

• added mkdir for RUNTESTS and added confg file under ci/platforms for noaacloud

  

  tmcguinness

  committed
  d6f383a

  Copy full SHA for d6f383a

• path to create experment was wrong

  

  tmcguinness

  committed
  1d5d291

  Copy full SHA for 1d5d291

• path to pr cases was also stale and wrong

  

  tmcguinness

  committed
  75514d2

  Copy full SHA for 75514d2

• removed bug feature HOMEgfs and updated runtime ver numbers for fv3gfs runtime module versions

  

  tmcguinness

  committed
  577e8df

  Copy full SHA for 577e8df

• needed third arg to run-check_ci.sh for getting the TESTDIR path with HOMEgfs overwriting global-workflow

  

  Terry McGuinness

  committed
  7a4dedf

  Copy full SHA for 7a4dedf

Commits on Oct 10, 2024

• removed cron bash ci values from platforms config for noaacload

  

  tmcguinness

  committed
  6c763cb

  Copy full SHA for 6c763cb

• Update config.noaacloud

  Show description for 508ffc6
  TerrenceMcGuinness-NOAA

  authored
  508ffc6

  Copy full SHA for 508ffc6

• Update globalworkflow-ci-pw-ssh.yaml

  TerrenceMcGuinness-NOAA

  authored
  e1448a4

  Copy full SHA for e1448a4

• added the switch overwirte to create experment in GitHub Pipeline CI scripts

  

  tmcguinness

  committed
  99e3cab

  Copy full SHA for 99e3cab

• Merge branch 'NOAA-EMC:develop' into build_aws_updates

  TerrenceMcGuinness-NOAA

  authored
  8c5369e

  Copy full SHA for 8c5369e

• using checkout v3 for centos 7

  

  tmcguinness

  committed
  cf284fc

  Copy full SHA for cf284fc

• revered to checkout v4 for centos 7 because found fix and reverted to /bucket as is now mounted

  

  tmcguinness

  committed
  cf41b80

  Copy full SHA for cf41b80

• putting g2tmpl_ver back to 1.13.0

  

  Terry McGuinness

  authored and

  Terry McGuinness

  committed
  3eea38d

  Copy full SHA for 3eea38d

• dropped a quote on the end of BACI_IC string

  

  tmcguinness

  committed
  25944e7

  Copy full SHA for 25944e7

• Merge branch 'build_aws_updates' of github.com:TerrenceMcGuinness-NOAA/global-workflow into build_aws_updates

  

  tmcguinness

  committed
  a059fd1

  Copy full SHA for a059fd1

• two runners, one for rocky the other for centos

  

  tmcguinness

  committed
  bfa65ce

  Copy full SHA for bfa65ce

• renamed actons for ssh, rocky, and centos

  

  tmcguinness

  committed
  7b20639

  Copy full SHA for 7b20639

Commits on Oct 11, 2024

• move source of run time module versions up a few lines to its original location in the script

  

  tmcguinness

  committed
  6a29eb6

  Copy full SHA for 6a29eb6

Commits on Oct 15, 2024

• remove runtime ver conflics in run noaacloud env settings

  

  tmcguinness

  committed
  17d061e

  Copy full SHA for 17d061e

• improved provision runner script from bash sheel check

  

  tmcguinness

  committed
  0f38f0d

  Copy full SHA for 0f38f0d

• couple of more shell check updates to provision runner

  

  tmcguinness

  committed
  1e297a3

  Copy full SHA for 1e297a3

• couple of more shell check updates to provision runner

  

  tmcguinness

  committed
  b01d0b0

  Copy full SHA for b01d0b0

• added last line to provision runner script

  

  tmcguinness

  committed
  bb5658d

  Copy full SHA for bb5658d

• Merge branch 'NOAA-EMC:develop' into build_aws_updates

  TerrenceMcGuinness-NOAA

  authored
  9af68fb

  Copy full SHA for 9af68fb

• chmod x on provision runner scripts and add UserBootstrap code for Centos

  

  tmcguinness

  committed
  1715e7d

  Copy full SHA for 1715e7d

Commits on Oct 16, 2024

• reduced this branch to only build run changes seperate to ci

  

  tmcguinness

  committed
  881f917

  Copy full SHA for 881f917

• added correct pipline for ci

  

  Terry McGuinness

  committed
  a5f4614

  Copy full SHA for a5f4614

Commits on Oct 18, 2024

• added rocky workflow for testing development

  

  tmcguinness

  committed
  ba8f0ff

  Copy full SHA for ba8f0ff

Commits on Oct 21, 2024

• added a kludge fix to mound epic contrib disk for access on the compute notes

  

  tmcguinness

  committed
  05ffb6d

  Copy full SHA for 05ffb6d

• added runtime ver for noaa cload rocky 8

  

  tmcguinness

  committed
  7926989

  Copy full SHA for 7926989

• added test if all ready mounted

  

  tmcguinness

  committed
  fd3ab64

  Copy full SHA for fd3ab64

• added to distinct runtime ver vor noaa cload for centos and rocky

  

  tmcguinness

  committed
  ca92d54

  Copy full SHA for ca92d54

• updated runtime versions for centos

  

  tmcguinness

  committed
  0fb55a2

  Copy full SHA for 0fb55a2

• remove github pipeline codes

  

  tmcguinness

  committed
  24d6388

  Copy full SHA for 24d6388

Commits on Oct 22, 2024

• Merge branch 'NOAA-EMC:develop' into build_aws_updates_prototype

  TerrenceMcGuinness-NOAA

  authored
  fe0e581

  Copy full SHA for fe0e581