Skip to content

Config Files

Jump to bottom
Lori Garzio edited this page Jun 28, 2024 · 6 revisions

The QC configuration files are maintained and updated in example_configs, but must be added to the correct location in the fileserver. All of the configuration files can be modified by the user as necessary to work for different glider datasets and regions.

The deployment specific config files allow for adjusting the QC scripts for each deployment.

$HOME/deployments/YYYY/DEPLOYMENT_NAME/config/qc

  • qctests.yml: Change True to False for each test listed in the qctests.yml file to turn that test off for the deployment.

  • time_shift.yml: Lists the variable names for which the optimal time shift is calculated, and specifies the corresponding long_name for the time-adjusted variable as well as the default time shift. If the optimal shift calculated is zero or last value tested (the default is to test time shifts of 0 to 59 seconds), it means that the code couldn't find an optimal time shift with the values it tested. If this is the case, the code will use the closest non-nan shift from the previous segments. If there are no previous non-nan optimal shifts, the code will use the value from the configuration file (default is 30). To turn this test off for a deployment, delete or rename the time_shift.yml file in the ../DEPLOYMENT_NAME/config/qc directory, or alternatively set time_shift to False in the qctests.yml file.

All other QC configuration files that are not deployment specific are located here:

$HOME/qc/config

When these config files are updated in github the changes must be propagated to this directory.

  • global_regional: Configuration files that contain thresholds for QARTOD Climatology, Spike, and Rate of Change Tests. Different regional and seasonal thresholds can be specified using these files.

  • gross_flatline: Configuration files that contain thresholds for QARTOD Gross Range and Flat Line Tests, according to sensor specification sheets from the manufacturers.

  • ctd_hysteresis.yml: Thresholds used for the CTD hysteresis test to determine if the test should be run (comparing the data range to the test_threshold), and defining suspect and failed thresholds. These thresholds were determined by manually reviewing profiles from deployments with known CTD issues to flag bad profile pairs.

  • ctd_variables.yml: Mapping of CTD variable names from different instruments to a standard set of CTD variable names. Used in check_science_variables.py to check for values of 0.0000

  • oxygen_variables.yml: Mapping of all possible Dissolved Oxygen variable names from different instruments to a standard set of variables names. Used in check_science_variables.py to check for values of 0.000

  • science_variables.txt: List of possible science variable names. Used in check_science_variables.py to check for files that don't contain science data.

  • sciencevar_processing.yml: Specifications for advanced science variable processing, such as dissolved oxygen in mg/L and pH calculations from raw variables.

Clone this wiki locally