Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial support for PV and PVC creation #684

Merged
merged 56 commits into from
Mar 4, 2024
Merged

Initial support for PV and PVC creation #684

merged 56 commits into from
Mar 4, 2024

Commits on Feb 13, 2024

  1. Initial support for PV and PVC creation

    @tcassaert
    tcassaert committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    2145976 View commit details
    Browse the repository at this point in the history

Commits on Feb 15, 2024

  1. Merge branch 'master' into openeo_cdse_infra_46_fuse_mount_results_bu… 

    …cket
    @tcassaert
    tcassaert committed Feb 15, 2024
    Configuration menu
    Copy the full SHA
    7b75539 View commit details
    Browse the repository at this point in the history

  2. Allow different type of kube api clients

    @tcassaert
    tcassaert committed Feb 15, 2024
    Configuration menu
    Copy the full SHA
    b3c8811 View commit details
    Browse the repository at this point in the history

  3. Fix kube_client for jobtracker

    @tcassaert
    tcassaert committed Feb 15, 2024
    Configuration menu
    Copy the full SHA
    8787800 View commit details
    Browse the repository at this point in the history

Commits on Feb 22, 2024

  1. Initial support for PV and PVC creation

    @tcassaert
    tcassaert committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    c798667 View commit details
    Browse the repository at this point in the history

  2. Add PV and PVC deletion on job cancel

    @tcassaert
    tcassaert committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    182d040 View commit details
    Browse the repository at this point in the history

  3. Add volumeMount

    @tcassaert
    tcassaert committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    13f5adc View commit details
    Browse the repository at this point in the history

  4. Remove S3 batch results writing

    @tcassaert
    tcassaert committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    e465456 View commit details
    Browse the repository at this point in the history

  5. Increase default size PVC and adjust mount options for s3fs

    @tcassaert
    tcassaert committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    07f855e View commit details
    Browse the repository at this point in the history

  6. Re-apply correct api_instances

    @tcassaert
    tcassaert committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    32a247b View commit details
    Browse the repository at this point in the history

  7. Error out on non-matching api type

    @tcassaert
    tcassaert committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    9e4b8fa View commit details
    Browse the repository at this point in the history

  8. Fix the mountpath of the bucket

    @tcassaert
    tcassaert committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    df315c8 View commit details
    Browse the repository at this point in the history

  9. Remove unnecessary underscore

    @tcassaert
    tcassaert committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    1ce1d6f View commit details
    Browse the repository at this point in the history

  10. Remove print statement in favor of structured logging

    @tcassaert
    tcassaert committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    bac62ae View commit details
    Browse the repository at this point in the history

  11. Don't catch failed creation of PV and PVC. It should fail the deploym… 

    …ent of the batch job
    @tcassaert
    tcassaert committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    222b9ff View commit details
    Browse the repository at this point in the history

Commits on Feb 27, 2024

  1. orfeo upgrade: sar parameters are now doubles 

    #571
    @jdries @tcassaert
    jdries authored and tcassaert committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    37747b0 View commit details
    Browse the repository at this point in the history

  2. json output format: more robust with respect to unknown types 

    #643
    @jdries @tcassaert
    jdries authored and tcassaert committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    14936d2 View commit details
    Browse the repository at this point in the history

  3. changelog entry for gdal 3.8.4 

    #571
    @jdries @tcassaert
    jdries authored and tcassaert committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    44af25b View commit details
    Browse the repository at this point in the history

  4. add (skipped) test 

    Open-EO/openeo-python-driver#261
    @bossie @tcassaert
    bossie authored and tcassaert committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    c8990c5 View commit details
    Browse the repository at this point in the history

  5. simplify batch job results logic #676

    @bossie @tcassaert
    bossie authored and tcassaert committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    4aaf55d View commit details
    Browse the repository at this point in the history

  6. stac fix: processing:expression is not an array but an object

    @jdries @tcassaert
    jdries authored and tcassaert committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    e2e4933 View commit details
    Browse the repository at this point in the history

  7. stac: fix processing:expression 

    #588
    @jdries @tcassaert
    jdries authored and tcassaert committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    e1876d3 View commit details
    Browse the repository at this point in the history

  8. removal of file:nodata 

    #588
    @jdries @tcassaert
    jdries authored and tcassaert committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    a236d9f View commit details
    Browse the repository at this point in the history

  9. fix Log4j logging in local runs 

    Align Spark version with that of geotrellis-extensions.

java.lang.IllegalAccessError: class org.apache.logging.log4j.layout.template.json.JsonTemplateLayout$StringBuilderEncoder tried to access method 'void org.apache.logging.log4j.core.layout.TextEncoderHelper.encodeText(java.nio.charset.CharsetEncoder, java.nio.CharBuffer, java.nio.ByteBuffer, java.lang.StringBuilder, org.apache.logging.log4j.core.layout.ByteBufferDestination)' (org.apache.logging.log4j.layout.template.json.JsonTemplateLayout$StringBuilderEncoder and org.apache.logging.log4j.core.layout.TextEncoderHelper are in unnamed module of loader 'app')
    @bossie @tcassaert
    bossie authored and tcassaert committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    15d5b7e View commit details
    Browse the repository at this point in the history

  10. add partition check before resample_spatial 

    Open-EO/openeo-geotrellis-extensions#265
    @jdries @tcassaert
    jdries authored and tcassaert committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    0192add View commit details
    Browse the repository at this point in the history

  11. only repartition if cellsize is decreased 

    Open-EO/openeo-geotrellis-extensions#265
    @jdries @tcassaert
    jdries authored and tcassaert committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    b0da172 View commit details
    Browse the repository at this point in the history

  12. Add skipped test_layer_catalog_step_resolution. #616

    @EmileSonneveld @tcassaert
    EmileSonneveld authored and tcassaert committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    5969b5c View commit details
    Browse the repository at this point in the history

  13. Better logging. No is_layer_too_large when step is missing. #616

    @EmileSonneveld @tcassaert
    EmileSonneveld authored and tcassaert committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    cd2b80f View commit details
    Browse the repository at this point in the history

  14. Print CRS. Skip test.

    @EmileSonneveld @tcassaert
    EmileSonneveld authored and tcassaert committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    23817d6 View commit details
    Browse the repository at this point in the history

  15. WIP: Parsed a resolution in 92 out of 99 layers. #616

    @EmileSonneveld @tcassaert
    EmileSonneveld authored and tcassaert committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    f3426a7 View commit details
    Browse the repository at this point in the history

  16. Add get_GSD_in_meters method to GeopysparkCubeMetadata, need to use y… 

    …et. #616
    @EmileSonneveld @tcassaert
    EmileSonneveld authored and tcassaert committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    3ed0432 View commit details
    Browse the repository at this point in the history

  17. Convert GSD to resolution in layer CRS. Fix tests. #616

    @EmileSonneveld @tcassaert
    EmileSonneveld authored and tcassaert committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    b526e24 View commit details
    Browse the repository at this point in the history

  18. Use temporal step resolution.

    @EmileSonneveld @tcassaert
    EmileSonneveld authored and tcassaert committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    cb563a8 View commit details
    Browse the repository at this point in the history

  19. Code reviewing. Simplify test_layer_catalog_step_resolution. #616

    @EmileSonneveld @tcassaert
    EmileSonneveld authored and tcassaert committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    ff42df0 View commit details
    Browse the repository at this point in the history

  20. Better comment

    @EmileSonneveld @tcassaert
    EmileSonneveld authored and tcassaert committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    643c4f7 View commit details
    Browse the repository at this point in the history

  21. Better comment

    @EmileSonneveld @tcassaert
    EmileSonneveld authored and tcassaert committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    8cba809 View commit details
    Browse the repository at this point in the history

  22. Code reviewing. #616

    @EmileSonneveld @tcassaert
    EmileSonneveld authored and tcassaert committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    d27fe25 View commit details
    Browse the repository at this point in the history

  23. More reproject_cellsize. clean_number_pair.

    @EmileSonneveld @tcassaert
    EmileSonneveld authored and tcassaert committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    27cca2d View commit details
    Browse the repository at this point in the history

  24. Fix comment and add test with north pole.

    @EmileSonneveld @tcassaert
    EmileSonneveld authored and tcassaert committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    0af6e83 View commit details
    Browse the repository at this point in the history

  25. Fix for when band aliases are used. Found after running on many exist… 

    …ing process graphs.
    @EmileSonneveld @tcassaert
    EmileSonneveld authored and tcassaert committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    d72baca View commit details
    Browse the repository at this point in the history

  26. Fix for when no GSD found. Issue found by running validation on last … 

    …1000 process graphs from CDSE-prod.
    @EmileSonneveld @tcassaert
    EmileSonneveld authored and tcassaert committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    3a599d0 View commit details
    Browse the repository at this point in the history

  27. Move verify_for_synchronous_processing to GpsProcessing from Concrete… 

    …Processing. #616
    @EmileSonneveld @tcassaert
    EmileSonneveld authored and tcassaert committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    c1192e6 View commit details
    Browse the repository at this point in the history

  28. Issue #285 port s3_bucket_name config to GpsBackendConfig

    @soxofaan @tcassaert
    soxofaan authored and tcassaert committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    e7c4069 View commit details
    Browse the repository at this point in the history

  29. job tracker logging: put deploy env in rotating log file name

    @soxofaan @tcassaert
    soxofaan authored and tcassaert committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    28a0bbc View commit details
    Browse the repository at this point in the history

  30. unblock JobTracker on requests towards ETL API #690

    @bossie @tcassaert
    bossie authored and tcassaert committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    db4b40d View commit details
    Browse the repository at this point in the history

  31. remove unused get_etl_api import #690

    @bossie @tcassaert
    bossie authored and tcassaert committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    10b4d35 View commit details
    Browse the repository at this point in the history

  32. do not check on crs: if the vector cube already has a crs, the 'expec… 

    …ted' crs doesn't really matter and we return a correct object. The receiving end can then reproject as necessary.
    @jdries @tcassaert
    jdries authored and tcassaert committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    c24db71 View commit details
    Browse the repository at this point in the history

  33. Feature flag the batchjob result s3 bucket mount

    @tcassaert
    tcassaert committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    9e16390 View commit details
    Browse the repository at this point in the history

  34. Changelog note on MAX_CONTENT_LENGTH 

    Open-EO/openeo-python-driver#254
    @soxofaan @tcassaert
    soxofaan authored and tcassaert committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    e8420ec View commit details
    Browse the repository at this point in the history

  35. Initial support for PV and PVC creation

    @tcassaert
    tcassaert committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    d78bff0 View commit details
    Browse the repository at this point in the history

  36. Feature flag the batchjob result s3 bucket mount

    @tcassaert
    tcassaert committed Feb 27, 2024
    Configuration menu
    Copy the full SHA
    9a4af04 View commit details
    Browse the repository at this point in the history

  37. Merge branch 'master' into openeo_cdse_infra_46_fuse_mount_results_bu… 

    …cket
    @tcassaert
    tcassaert authored Feb 27, 2024
    Configuration menu
    Copy the full SHA
    338c94a View commit details
    Browse the repository at this point in the history

Commits on Feb 29, 2024

  1. Merge branch 'master' into openeo_cdse_infra_46_fuse_mount_results_bu… 

    …cket
    @tcassaert
    tcassaert committed Feb 29, 2024
    Configuration menu
    Copy the full SHA
    bf16fd2 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into openeo_cdse_infra_46_fuse_mount_results_bu… 

    …cket
    @soxofaan
    soxofaan committed Feb 29, 2024
    Configuration menu
    Copy the full SHA
    47f5f3d View commit details
    Browse the repository at this point in the history

  3. fixup! Feature flag the batchjob result s3 bucket mount

    @soxofaan
    soxofaan committed Feb 29, 2024
    Configuration menu
    Copy the full SHA
    023891a View commit details
    Browse the repository at this point in the history

Commits on Mar 4, 2024

  1. Also feature-flag the deletion of PV and PVC on job cancel

    @tcassaert
    tcassaert committed Mar 4, 2024
    Configuration menu
    Copy the full SHA
    b015a0b View commit details
    Browse the repository at this point in the history