Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Ctests for obsForge #4

Draft
wants to merge 6 commits into
base: main
Choose a base branch
from
Draft

Add Ctests for obsForge #4

wants to merge 6 commits into from

Conversation

emilyhcliu
Copy link

@emilyhcliu emilyhcliu commented Dec 8, 2024 •
edited
Loading

Add initial structure for Ctests
Tests added:

  • coding norms
  • satwnd_amv_goes
  • sfcsno

Tests data

  • Currently using full-resolution BUFR files (obsforge-0.0.0.tgz in EMCRZDM)
  • Working on reduced BUFR files (obsforge-0.0.1.tgz on EMCRZDM)
    This PR will be converted from draft to review when the reduced BUFR files are ready.

Issue #3

Here is the list of tests implemented:

orion-login-3[518] eliu$ ctest -N
Test project /work/noaa/da/eliu/ORION/EMC-obsForge/obsForge_ctest/build/obsForge
  Test  #1: obsForge_coding_norms
  Test  #2: test_obsforge_sfcsno_bufr2netcdf
  Test  #3: test_obsforge_sfcsno_bufr2netcdf_mpi2
  Test  #4: test_obsforge_sfcsno_script2netcdf
  Test  #5: test_obsforge_sfcsno_script2netcdf_mpi2
  Test  #6: test_obsforge_sfcsno_bufr_backend
  Test  #7: test_obsforge_sfcsno_bufr_backend_mpi2
  Test  #8: test_obsforge_sfcsno_script_backend
  Test  #9: test_obsforge_sfcsno_script_backend_mpi2
  Test #10: test_obsforge_satwnd_abi_goes-16_bufr2netcdf
  Test #11: test_obsforge_satwnd_abi_goes-17_bufr2netcdf
  Test #12: test_obsforge_satwnd_abi_goes-18_bufr2netcdf
  Test #13: test_obsforge_satwnd_abi_goes-16_bufr2netdf_mpi4
  Test #14: test_obsforge_satwnd_abi_goes-17_bufr2netcdf_mpi8
  Test #15: test_obsforge_satwnd_abi_goes-18_bufr2netcdf_mpi2
  Test #16: test_obsforge_satwnd_abi_goes-16_script2netcdf
  Test #17: test_obsforge_satwnd_abi_goes-17_script2netcdf
  Test #18: test_obsforge_satwnd_abi_goes-18_script2netcdf
  Test #19: test_obsforge_satwnd_abi_goes-16_script2netcdf_mpi4
  Test #20: test_obsforge_satwnd_abi_goes-17_script2netcdf_mpi8
  Test #21: test_obsforge_satwnd_abi_goes-18_script2netcdf_mpi2
  Test #22: test_obsforge_satwnd_abi_goes-16_bufr_backend
  Test #23: test_obsforge_satwnd_abi_goes-17_bufr_backend
  Test #24: test_obsforge_satwnd_abi_goes-18_bufr_backend
  Test #25: test_obsforge_satwnd_abi_goes-16_bufr_backend_mpi4
  Test #26: test_obsforge_satwnd_abi_goes-17_bufr_backend_mpi4
  Test #27: test_obsforge_satwnd_abi_goes-18_bufr_backend_mpi4
  Test #28: test_obsforge_satwnd_abi_goes-16_script_backend
  Test #29: test_obsforge_satwnd_abi_goes-17_script_backend
  Test #30: test_obsforge_satwnd_abi_goes-18_script_backend
  Test #31: test_obsforge_satwnd_abi_goes-16_script_backend_mpi4
  Test #32: test_obsforge_satwnd_abi_goes-17_script_backend_mpi4
  Test #33: test_obsforge_satwnd_abi_goes-18_script_backend_mpi4

Total Tests: 33

@emilyhcliu emilyhcliu marked this pull request as draft December 8, 2024 20:37
@emilyhcliu emilyhcliu mentioned this pull request Dec 8, 2024
Open
nicholasesposito
nicholasesposito reviewed Dec 9, 2024
View reviewed changes
test/testinput/bufr_bufr_backend_satwnd_amv_goes_mpi4.yaml Outdated
@@ -0,0 +1,58 @@
time window:
begin: "2018-04-14T21:00:00Z"
end: "2023-12-15T03:00:00Z"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For my own reference, why are the satellite yamls such large time windows?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is purposely done since we want to test several other cycles as well. So, we gave a big range.

nicholasesposito
nicholasesposito reviewed Dec 9, 2024
View reviewed changes
test/testinput/bufr_bufr_backend_satwnd_amv_goes_mpi4.yaml Outdated
time window:
begin: "2018-04-14T21:00:00Z"
end: "2023-12-15T03:00:00Z"

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additionally, I think you should add " bound to include: begin"

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

time window:
  begin:  # datetime in ISO format
  length: # duration in ISO format
  bound to include: end # which window bound is inclusive (options: begin, end (default)).

Since the end is inclusive already (default), we should add bound to include: begin.
I will add it. Thanks!

nicholasesposito
nicholasesposito reviewed Dec 9, 2024
View reviewed changes
test/testinput/bufr_bufr_backend_sfcsno.yaml Outdated
@@ -0,0 +1,20 @@
time window:
begin: "2018-04-14T21:00:00Z"
end: "2023-12-15T03:00:00Z"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll suggest adding:
bound to include: begin

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do

nicholasesposito
nicholasesposito reviewed Dec 9, 2024
View reviewed changes
test/testinput/bufr_bufr_backend_sfcsno_mpi2.yaml Outdated
@@ -0,0 +1,20 @@
time window:
begin: "2018-04-14T21:00:00Z"
end: "2023-12-15T03:00:00Z"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bound to include: begin

nicholasesposito
nicholasesposito reviewed Dec 9, 2024
View reviewed changes
test/testinput/bufr_script_backend_satwnd_amv_goes.yaml Outdated
@@ -0,0 +1,58 @@
time window:
begin: "2018-04-14T21:00:00Z"
end: "2023-12-15T03:00:00Z"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bound to include: begin

nicholasesposito
nicholasesposito reviewed Dec 9, 2024
View reviewed changes
test/testinput/bufr_script_backend_satwnd_amv_goes_mpi4.yaml Outdated
@@ -0,0 +1,58 @@
time window:
begin: "2018-04-14T21:00:00Z"
end: "2023-12-15T03:00:00Z"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bound to include: begin

nicholasesposito
nicholasesposito reviewed Dec 9, 2024
View reviewed changes
test/testinput/bufr_script_backend_sfcsno.yaml Outdated
@@ -0,0 +1,22 @@
time window:
begin: "2018-04-14T21:00:00Z"
end: "2023-12-15T03:00:00Z"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bound to include: begin

nicholasesposito
nicholasesposito reviewed Dec 9, 2024
View reviewed changes
test/testinput/bufr_script_backend_sfcsno_mpi2.yaml Outdated
@@ -0,0 +1,22 @@
time window:
begin: "2018-04-14T21:00:00Z"
end: "2023-12-15T03:00:00Z"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bound to include: begin

nicholasesposito
nicholasesposito reviewed Dec 9, 2024
View reviewed changes
test/testinput/bufr_bufr_backend_satwnd_amv_goes.yaml Outdated
@@ -0,0 +1,58 @@
time window:
begin: "2018-04-14T21:00:00Z"
end: "2023-12-15T03:00:00Z"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bound to include: begin

rmclaren
rmclaren reviewed Dec 9, 2024
View reviewed changes
test/testinput/bufr_bufr_backend_satwnd_amv_goes.yaml Outdated
engine:
type: bufr
obsfile: "./testdata/gdas.t00z.satwnd.tm00.bufr_d"
mapping file: "./testinput//bufr_satwnd_amv_goes_mapping.yaml"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

extra slash :)

test/CMakeLists.txt Outdated
Comment on lines 12 to 35
## Get the test data if we don't have it.
if(NOT EXISTS "${CMAKE_CURRENT_BINARY_DIR}/${FILE_COLLECTION}")
message(STATUS "Downloading: " ${CMAKE_CURRENT_BINARY_DIR}/${FILE_COLLECTION})
# file(DOWNLOAD
# ${DOWNLOAD_URL}/${FILE_COLLECTION}
# ${CMAKE_CURRENT_BINARY_DIR}/${FILE_COLLECTION}
# SHOW_PROGRESS
# STATUS download_status
# INACTIVITY_TIMEOUT 30
# )
#
# list(GET download_status 0 download_status_num)
#
# if(NOT download_status_num EQUAL 0 OR NOT EXISTS ${CMAKE_CURRENT_BINARY_DIR}/${FILE_COLLECTION})
# # Remove empty file if download doesn't complete
# file(REMOVE ${CMAKE_CURRENT_BINARY_DIR}/${BUFR_TAR})
# message(STATUS "Could not download test files, not building tests")
# return()
# endif()
#<<orig

#>>emily
file(COPY ${CMAKE_CURRENT_SOURCE_DIR}/${FILE_COLLECTION} DESTINATION ${CMAKE_CURRENT_BINARY_DIR})
#<<emily
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please delete all the commented out unused code.

test/CMakeLists.txt
bufr2netcdf/gdas.t00z.satwnd.abi_goes-18.tm00.nc)

# Perform script2netcdf tests
#if (${BUILD_PYTHON_BINDINGS})
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please delete this line if its not needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nicholasesposito nicholasesposito nicholasesposito left review comments

@rmclaren rmclaren rmclaren left review comments

@CoryMartin-NOAA CoryMartin-NOAA Awaiting requested review from CoryMartin-NOAA

@xincjin-NOAA xincjin-NOAA Awaiting requested review from xincjin-NOAA

@PraveenKumar-NOAA PraveenKumar-NOAA Awaiting requested review from PraveenKumar-NOAA

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@emilyhcliu @rmclaren @nicholasesposito