Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add JEDI ATM equivalent to GSI eobs #2415

Closed
RussTreadon-NOAA opened this issue Mar 20, 2024 · 2 comments · Fixed by #2833
Closed

Add JEDI ATM equivalent to GSI eobs #2415

RussTreadon-NOAA opened this issue Mar 20, 2024 · 2 comments · Fixed by #2833
Assignees
@RussTreadon-NOAA
Labels
feature New feature or request

Comments

@RussTreadon-NOAA
Copy link
Contributor

RussTreadon-NOAA commented Mar 20, 2024 •
edited
Loading

What new functionality do you need?

Add a JEDI ATM equivalent of the GSI based eobs

What are the requirements for the new functionality?

Run JEDI local ensemble application in observer mode. This is done via a JEDI ATM eobs equivalent to run between the existing atmensanlinit and atmensanletkf.

Acceptance Criteria

Cycle JEDI ATM local ensemble DA more efficiently with the addition of an observer job

Suggest a solution (optional)

Add a new job, atmensanlobs, to scripts in workflow/ Anticipate making changes to

  • workflow/applications/applications.py
  • workflow/applications/gfs_cycled.py
  • workflow/rocoto/gfs_tasks.py
  • workflow/rocoto/tasks.py
@RussTreadon-NOAA RussTreadon-NOAA added feature New feature or request triage Issues that are triage labels Mar 20, 2024
@RussTreadon-NOAA RussTreadon-NOAA self-assigned this Mar 20, 2024
@RussTreadon-NOAA
Copy link
Contributor Author

Work for this issue will be done in RussTreadon-NOAA:feature/ensda_obs

@WalterKolczynski-NOAA WalterKolczynski-NOAA removed the triage Issues that are triage label Mar 20, 2024
RussTreadon-NOAA added a commit to RussTreadon-NOAA/global-workflow that referenced this issue Mar 20, 2024
@RussTreadon-NOAA
add plumbing for JEDI ATM local ensemble observer job (NOAA-EMC#2415)
80e21a9
RussTreadon-NOAA added a commit to RussTreadon-NOAA/global-workflow that referenced this issue Mar 20, 2024
@RussTreadon-NOAA
fix pycodestyle violation (NOAA-EMC#2415)
4ece0e2
@RussTreadon-NOAA RussTreadon-NOAA mentioned this issue Apr 24, 2024
Open
RussTreadon-NOAA added a commit to RussTreadon-NOAA/global-workflow that referenced this issue May 31, 2024
@RussTreadon-NOAA
update gdas and gsi_utils hashes (NOAA-EMC#2415)
df4456f
@RussTreadon-NOAA
Copy link
Contributor Author

Work with @TravisElless-NOAA to run GDASApp lgetkf in two steps: observer and solver. Doing so results in a significant reduction in both memory usage and run time over a single execution of lgtekf which includes both the observer and solver.

Given this result the effort to add a JEDI equivalent to eobs will be resumed.

@RussTreadon-NOAA RussTreadon-NOAA changed the title Add JEDI ATM equivalent to eomg & eobs Add JEDI ATM equivalent to GSI eobs Jul 31, 2024
This was referenced Aug 4, 2024
Merged
Merged
RussTreadon-NOAA added a commit to RussTreadon-NOAA/global-workflow that referenced this issue Aug 13, 2024
@RussTreadon-NOAA
initial commit of jedi local ensemble observer and solver jobs (NOAA-…
e90a403 
…EMC#2415)
This was referenced Aug 13, 2024
Closed
Closed
Closed
RussTreadon-NOAA added a commit to RussTreadon-NOAA/global-workflow that referenced this issue Aug 13, 2024
@RussTreadon-NOAA
update gdas.cd hash (NOAA-EMC#2415)
d659131
RussTreadon-NOAA added a commit to RussTreadon-NOAA/global-workflow that referenced this issue Aug 14, 2024
@RussTreadon-NOAA
update gdas.cd hash for aircraft (NOAA-EMC#2415)
083f1f5
RussTreadon-NOAA added a commit to RussTreadon-NOAA/global-workflow that referenced this issue Aug 14, 2024
@RussTreadon-NOAA
update gdas.cd hash to save lgetkf ensemble increments (NOAA-EMC#2415)
dddb9b3
RussTreadon-NOAA added a commit to RussTreadon-NOAA/global-workflow that referenced this issue Aug 14, 2024
@RussTreadon-NOAA
turn off fit2obs for jediatmvar (NOAA-EMC#2415)
52db22c
@RussTreadon-NOAA RussTreadon-NOAA mentioned this issue Aug 14, 2024
Merged
7 tasks
RussTreadon-NOAA added a commit to RussTreadon-NOAA/global-workflow that referenced this issue Aug 14, 2024
@RussTreadon-NOAA
update setup_expt.py to copy JEDI radiance bias correction files, upd…
57687c3 
…ate gdas.cd hash (NOAA-EMC#2415)
RussTreadon-NOAA added a commit to RussTreadon-NOAA/global-workflow that referenced this issue Aug 14, 2024
@RussTreadon-NOAA
correct pynorm errors in setup_expt.py (NOAA-EMC#2415)
1e40e2f
RussTreadon-NOAA added a commit to RussTreadon-NOAA/global-workflow that referenced this issue Aug 15, 2024
@RussTreadon-NOAA
add entries for lgetkf observer and solver jobs to machine specific e…
9abaf32 
…nv files, add marine entries to WCOSS2.env (NOAA-EMC#2415)
RussTreadon-NOAA added a commit to RussTreadon-NOAA/global-workflow that referenced this issue Aug 16, 2024
@RussTreadon-NOAA
activate C96C48_ufs_hybatmDA CI on hera, adjust hera resources, updat…
b8c0747 
…e gdas.cd hash (NOAA-EMC#2415)
RussTreadon-NOAA added a commit to RussTreadon-NOAA/global-workflow that referenced this issue Aug 16, 2024
@RussTreadon-NOAA
set WCOSS2 resources for atmanlvar and atmensanlobs (NOAA-EMC#2415)
30852ff
RussTreadon-NOAA added a commit to RussTreadon-NOAA/global-workflow that referenced this issue Aug 16, 2024
@RussTreadon-NOAA
set HERCULES resources for atmanlvar and atmensanlobs (NOAA-EMC#2415)
5ed24a4
RussTreadon-NOAA added a commit to RussTreadon-NOAA/global-workflow that referenced this issue Aug 17, 2024
@RussTreadon-NOAA
update gdas.cd hash to bring in updated jcb templates (NOAA-EMC#2415)
cf082b4
RussTreadon-NOAA added a commit to RussTreadon-NOAA/global-workflow that referenced this issue Aug 19, 2024
@RussTreadon-NOAA
add atmensanlobs and atmensanlsol to enkf.yaml.j2 (NOAA-EMC#2415)
252a9db
RussTreadon-NOAA added a commit to RussTreadon-NOAA/global-workflow that referenced this issue Aug 19, 2024
@RussTreadon-NOAA
generalize lgetkf observer and solver configuration, save observer an…
dd05d39 
…d solver yamls (NOAA-EMC#2415)
RussTreadon-NOAA added a commit to RussTreadon-NOAA/global-workflow that referenced this issue Aug 19, 2024
@RussTreadon-NOAA
correct pynorms violation (NOAA-EMC#2415)
b34013a
RussTreadon-NOAA added a commit to RussTreadon-NOAA/global-workflow that referenced this issue Aug 19, 2024
@RussTreadon-NOAA
update gdas.cd hash to include lgetkf observer and solver ctests (NOA…
218c851 
…A-EMC#2415)
RussTreadon-NOAA added a commit to RussTreadon-NOAA/global-workflow that referenced this issue Aug 20, 2024
@RussTreadon-NOAA
update sorc/gdas.cd hash to head of GDASApp develop (NOAA-EMC#2415)
b0e6fcf
RussTreadon-NOAA added a commit to RussTreadon-NOAA/global-workflow that referenced this issue Aug 21, 2024
@RussTreadon-NOAA
update stage_ic to copy JEDI radiance bias correction files (NOAA-EMC…
4211bf9 
…#2415)
RussTreadon-NOAA added a commit to RussTreadon-NOAA/global-workflow that referenced this issue Aug 21, 2024
@RussTreadon-NOAA
remove trailing whitespace to correct pynorm coding violation (NOAA-E…
3ef486c 
…MC#2415)
RussTreadon-NOAA added a commit to RussTreadon-NOAA/global-workflow that referenced this issue Aug 21, 2024
@RussTreadon-NOAA
add logic to toggle on/off generation of combined all-in-one lgetkf y…
bb98a18 
…aml (NOAA-EMC#2415)
RussTreadon-NOAA added a commit to RussTreadon-NOAA/global-workflow that referenced this issue Aug 21, 2024
@RussTreadon-NOAA
add logic to jinja template to archive appropriate atmens yaml file(s) (
74de6bc 
NOAA-EMC#2415)
RussTreadon-NOAA added a commit to RussTreadon-NOAA/global-workflow that referenced this issue Aug 22, 2024
@RussTreadon-NOAA
use update to populate jcb_config with appropriate yaml (NOAA-EMC#2415)
a8a528f
RussTreadon-NOAA added a commit to RussTreadon-NOAA/global-workflow that referenced this issue Aug 22, 2024
@RussTreadon-NOAA
rename observer and solver python methods as observe and solve (NOAA-…
28e6e5a 
…EMC#2415)
RussTreadon-NOAA added a commit to RussTreadon-NOAA/global-workflow that referenced this issue Aug 22, 2024
@RussTreadon-NOAA
rename init_observe and init_solve as init_observer and init_solver (N…
25ee2a9 
…OAA-EMC#2415)
RussTreadon-NOAA added a commit to RussTreadon-NOAA/global-workflow that referenced this issue Aug 23, 2024
@RussTreadon-NOAA
activate C96C48_ufs_hybatmDA CI on wcoss2 (NOAA-EMC#2415)
d9258f4
RussTreadon-NOAA added a commit to RussTreadon-NOAA/global-workflow that referenced this issue Aug 26, 2024
@RussTreadon-NOAA
update gdas.cd hash to enable GDASApp ctests to run lgetkf in multipl…
ee60eff 
…e modes (NOAA-EMC#2415)
WalterKolczynski-NOAA pushed a commit that referenced this issue Aug 28, 2024
@RussTreadon-NOAA
Add JEDI ATM lgetkf observer and solver jobs (#2833)
9ad7d3e 
This PR adds JEDI ATM lgetkf observer and solver jobs to
global-workflow. This is approach is akin GSI-based eobs and eupd.
Splitting the single JEDI ATM lgetkf job into separate observer and
solver jobs improves memory and computational efficiency.

Resolves #2415
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@RussTreadon-NOAA RussTreadon-NOAA

Labels
feature New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add JEDI ATM lgetkf observer and solver jobs RussTreadon-NOAA/global-workflow
2 participants
@RussTreadon-NOAA @WalterKolczynski-NOAA