Home

the lterwg-som wiki

key file version 2

new features and checks wish list:

• project id detail in the hmgzd notes (see also issue #35) DONE
• units for variables such as clay content, which can be decimal fraction or percent fraction
• resolve soc vs. c_tot - issue #12
• date conversion - issue #40
• experiment and treatment levels to homog output - issues #13 DONE
• alignment needed to locations tab and homog output - issue #8 case-by-case
• elevate quality control:
  • non-decimal degrees in lat-lon (see also issue #34)
  • strings and weird characters (see also issue #34)
  • force numerical data types where expected (see also issue #34)
  • future has_experiments exist input in location tab cannot be null
  • if any exp == true then control ID in location tab must not be null
  • add min and max values to key file
• modified at timestamp to key file DONE (in key file update log)
• content of key files in a version controlled system DONE (captured in Aurora)

processing notes:

• add a sleep step before each iteration of touching a google file to help reduce errors
• take a snapshot of the Google Drive before each batch step (Julien will check with Nick)
• consider a unique naming system that would facilitate keeping the already homogenized files
• general workflow:
  1. back up Google Drive: snapshot 0
  2. get cell range stats from key files to identify cases where cell range references will break
  3. adjust key files that need cell range adjustment
  4. batch update key files to version 2 format
  5. edit each key file
  6. back up Google Drive: snapshot 1
  7. remove all homogenized products except the modified key file
  8. batch re-homogenize all data packages