Skip to content

Integration Test Procedure

Jim Hofman edited this page Feb 27, 2024 · 16 revisions

Running 'test_int_<pge>.sh' scripts on opera-pge-ci and opera-dev-pge.

Setup

  • Log on to opera-pge-ci or opera-dev-pge.
  • Use the 'docker images' command to see if there is an existing image you can use.
  • If no images exists follow the steps in the Building PGE Container Images to add an image.
  • Log on to aws with the 'aws-login' command and select 'pub' from the pulldown. This is required to enable the script to retrieve test data from the s3://operasds-dev-pge bucket.

Script setup of temporary directory content

  • When the test_int_<pge>.sh script calls the test_int_setup_test_data() function, asw s3 cp commands are used to setup the following test directories and files (this example happens to be for 'cslc_s1'):

    `drwxr-xr-x  3 <user> cloud-user    24 Sep 12 21:41 delivery_cslc_s1_final_0.5.1_expected_input_data`
    
    `drwxr-xr-x  4 <user> cloud-user    75 Sep 12 21:46 delivery_cslc_s1_final_0.5.1_expected_output`
    
    `drwxrwxr-x 12 <user> cloud-user  4096 Feb 27 04:06 output_cslc_s1`
    
    `drwxrwxr-x 12 <user> cloud-user  4096 Feb 27 04:59 output_cslc_s1_static`
    
    `drwxrwxr-x  2 <user> cloud-user   131 Feb 27 03:47 runconfig`
    
    `drwxrwxrwx  2 <user> cloud-user     6 Feb 27 03:47 scratch_cslc_s1`
    
    `-rw-rw-r--  1 <user> cloud-user 11814 Sep 12 21:59 validate_product.py`
    

Running an integration test shell script

  • If it does not already exist; use the mkdir command to create a 'temporary' directory within the directory containing the test_int<pge>.sh_ script
  • Below are the arguments to the script, controlled by the following switches:
    • -h or --help : (optional) for usage summary.
    • -t or --tag : (optional) user enters docker tag to select a docker image (default is '<user>-dev')
    • -i or --input-data : (optional) user enters path to custom input data
    • -e or --expected-data : (optional) user enters path to custom expected data
    • --runconfig : (optional) user enters path to custom runconfig file
    • --temp-root : (required) path to temporary directory created by user (directory name is up to the user (see above))
    • --no-cleanup : (optional) suspends the deletion of the temporary directory (Note: user is now responsible for cleanup)
    • --no-metrics : (optional) suspends the generation for runtime metrics
  • Sample commands
    • bash test_int_cslc_s1.sh --temp-root tmp/
    • bash test_int_disp_s1.sh --temp-root temp/ --tag jehofman-dev --no-metrics --no-cleanup
    • bash test_int_dswx_hls.sh --temp-root temp_dir/ -i <path to input data> -e <path expected data> --runconfig <path to runconfig file>
Clone this wiki locally