feat(providers): add product types to dedl provider

[anesson-cs]

Add all missing providers on the dedl provider configuration from the DEDL portfolio.

Please note the addition of archive_depth parameter in this configuration to make direct access to product folder of all dedl product types. Furthermore, ISIMIP product types have been added while dedl still does not store them, but it should be the case very soon.

[github-actions]

Code Coverage (Ubuntu)

Filename                                     Stmts    Miss  Cover    Missing
-----------------------------------------  -------  ------  -------  ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__init__.py                                      8       0  100.00%
cli.py                                         323      64  80.19%   62-63, 87, 674-737, 839-890, 894
config.py                                      430      27  93.72%   80-82, 91, 99, 103-105, 176, 187, 692-694, 807-810, 853-854, 863-864, 969, 1032-1037, 1039
crunch.py                                        5       5  0.00%    20-24
api/__init__.py                                  0       0  100.00%
api/core.py                                    783      74  90.55%   317-318, 368, 658, 702-705, 743, 787, 821, 866-871, 897, 988, 1056, 1194, 1279-1291, 1327, 1329, 1357, 1361-1372, 1385-1391, 1474-1477, 1510-1530, 1582, 1599-1603, 1615-1618, 1954, 1978-1984, 2235, 2239-2243, 2252-2254, 2298-2299, 2328-2329
api/search_result.py                            58       4  93.10%   82, 91, 98, 112
api/product/__init__.py                          6       0  100.00%
api/product/_assets.py                          48       5  89.58%   90, 176, 184, 187-191
api/product/_product.py                        196      19  90.31%   70-72, 236-237, 312, 341, 398, 412-415, 428, 452-455, 515-518
api/product/metadata_mapping.py                695      64  90.79%   119-121, 218, 250-251, 297-298, 308-320, 322, 333, 398-399, 436, 457-460, 483, 491-492, 578-579, 603-604, 610-613, 628-629, 778, 824, 977, 986-990, 1007-1012, 1139, 1153-1173, 1193, 1198, 1327, 1341, 1366, 1412, 1464, 1487-1488, 1504-1508, 1524, 1532
api/product/drivers/__init__.py                 20       2  90.00%   33-36
api/product/drivers/base.py                     29       1  96.55%   98
api/product/drivers/generic.py                   7       0  100.00%
api/product/drivers/sentinel1.py                15       0  100.00%
api/product/drivers/sentinel2.py                15       0  100.00%
plugins/__init__.py                              0       0  100.00%
plugins/base.py                                 21       2  90.48%   48, 55
plugins/manager.py                             172      14  91.86%   106-111, 161, 199, 221, 225, 249, 389-392, 404-405
plugins/apis/__init__.py                         0       0  100.00%
plugins/apis/base.py                             4       0  100.00%
plugins/apis/ecmwf.py                           97      10  89.69%   171-173, 220-221, 247-249, 302-303
plugins/apis/usgs.py                           180      26  85.56%   156, 258, 292, 327-329, 334, 360-361, 366, 396-403, 414-419, 441-447, 478
plugins/authentication/__init__.py               6       1  83.33%   31
plugins/authentication/aws_auth.py              20       0  100.00%
plugins/authentication/base.py                  17       2  88.24%   45, 58
plugins/authentication/generic.py               14       2  85.71%   51, 61
plugins/authentication/header.py                19       0  100.00%
plugins/authentication/keycloak.py              46       7  84.78%   151-154, 175-180
plugins/authentication/oauth.py                 13       7  46.15%   43-45, 49-52
plugins/authentication/openid_connect.py       206      27  86.89%   80-81, 93-111, 149, 155-183, 191, 323-326, 352
plugins/authentication/qsauth.py                34       1  97.06%   91
plugins/authentication/sas_auth.py              47       1  97.87%   76
plugins/authentication/token.py                100      16  84.00%   141, 170, 172, 207-220, 276-280
plugins/authentication/token_exchange.py        36      14  61.11%   75, 92-120
plugins/crunch/__init__.py                       0       0  100.00%
plugins/crunch/base.py                          10       1  90.00%   43
plugins/crunch/filter_date.py                   59      14  76.27%   52-57, 69, 78, 87, 90, 102-104, 113-115, 122
plugins/crunch/filter_latest_intersect.py       47       8  82.98%   52-53, 69, 78-81, 83, 90-93
plugins/crunch/filter_latest_tpl_name.py        31       1  96.77%   83
plugins/crunch/filter_overlap.py                66      18  72.73%   28-30, 66-69, 76-79, 85, 93, 104-120
plugins/crunch/filter_property.py               30       5  83.33%   55-60, 63-64
plugins/download/__init__.py                     0       0  100.00%
plugins/download/aws.py                        491     152  69.04%   269, 282, 349-352, 366-370, 412-414, 418, 450-451, 457-461, 491, 523, 527, 534, 564-572, 576, 608-616, 627-629, 660-734, 752-810, 821-826, 879, 894-896, 899, 909-917, 925-938, 948-970, 977-989, 1027, 1053, 1098-1100, 1320
plugins/download/base.py                       253      42  83.40%   127, 155, 235-238, 291-292, 340-344, 350-351, 393, 396-410, 422, 426, 490-494, 524-525, 550-558, 560-564, 607, 629, 651, 659
plugins/download/creodias_s3.py                 25       9  64.00%   55-69
plugins/download/http.py                       529      95  82.04%   227, 323-326, 329, 336-341, 359-374, 391, 403, 451, 458-464, 482, 496, 510, 518-520, 536-541, 552, 570, 606-609, 638, 642, 662, 739, 758-769, 777-782, 792-809, 827, 857-866, 902, 927-928, 947-952, 958, 961, 978, 981, 996-997, 1032, 1094, 1109, 1168-1169, 1175, 1185, 1221, 1257, 1277, 1312-1314
plugins/download/s3rest.py                     116      24  79.31%   119, 153, 160, 195, 222-229, 232-234, 238, 249-255, 263-264, 267-271, 294, 315-318
plugins/search/__init__.py                      22       0  100.00%
plugins/search/base.py                         145      11  92.41%   101, 105, 129-135, 275, 295, 428
plugins/search/build_search_result.py          357      44  87.68%   239, 267-268, 304, 307, 378-381, 470-487, 515, 564, 566, 597, 631-633, 637, 661, 696, 747, 795-810, 853, 878, 881, 889, 1001-1002, 1008, 1019, 1077, 1128
plugins/search/cop_marine.py                   244      47  80.74%   56, 64-66, 76-77, 82, 87-88, 104, 106, 109, 175-176, 228, 234, 238, 242, 255, 266-267, 275, 303, 307, 322, 326, 330, 334, 338-342, 348-351, 354-368, 385, 434-438, 443, 455
plugins/search/creodias_s3.py                   25       1  96.00%   51
plugins/search/csw.py                          105      81  22.86%   98-99, 103-104, 112-160, 166-179, 187-219, 237-278
plugins/search/data_request_search.py          202      69  65.84%   189-192, 208, 219, 223-224, 235, 240, 245, 252, 265-268, 322-323, 327, 337-343, 348, 374-377, 385-396, 413, 415, 422-425, 427-428, 446-450, 483, 493, 504, 517, 523-538, 543
plugins/search/qssearch.py                     737      89  87.92%   450, 508, 522, 526-532, 560-564, 677-689, 734-737, 808-809, 857, 876, 883, 895, 952, 973, 976-977, 986-987, 996-997, 1006-1007, 1034, 1116-1121, 1125-1134, 1168, 1190, 1267, 1357, 1442-1443, 1453, 1530-1534, 1596, 1599, 1605-1606, 1627, 1655-1667, 1674, 1706-1708, 1718-1724, 1754, 1777, 1782-1783, 1798, 1814, 1899-1902, 1907-1910, 1919, 1949-1953, 1959
plugins/search/stac_list_assets.py              25      11  56.00%   44-51, 68, 75-85
plugins/search/static_stac_search.py            75      13  82.67%   98-124, 140, 153
rest/__init__.py                                 4       2  50.00%   21-22
rest/cache.py                                   33       7  78.79%   35-37, 53-55, 59, 68
rest/config.py                                  25       0  100.00%
rest/constants.py                                6       0  100.00%
rest/core.py                                   260      66  74.62%   258, 266, 284-301, 316-352, 446, 483-522, 705, 712-765
rest/errors.py                                  69       5  92.75%   106, 116, 127, 143-144
rest/server.py                                 192      24  87.50%   86, 109-111, 285-290, 318, 514-516, 533-538, 567, 569, 573-574, 578-579
rest/stac.py                                   319      63  80.25%   308, 330, 382-385, 412-439, 470-472, 495, 527-528, 610-650, 672-688, 780-784, 791, 845-846, 907, 997-999
rest/types/__init__.py                           0       0  100.00%
rest/types/collections_search.py                13      13  0.00%    18-44
rest/types/eodag_search.py                     180       6  96.67%   225-229, 282, 285, 353, 385
rest/types/queryables.py                        57       5  91.23%   93-98, 162
rest/types/stac_search.py                      125       7  94.40%   128, 174, 189-191, 199, 203
rest/utils/__init__.py                          94      12  87.23%   101-102, 121-123, 175, 185-199
rest/utils/cql_evaluate.py                      48       5  89.58%   69, 76, 90, 97, 105
rest/utils/rfc3339.py                           23       3  86.96%   48, 60, 62
types/__init__.py                              133      41  69.17%   63, 76-80, 91-103, 131-133, 140-145, 186, 227, 237-253, 258, 260, 282, 287, 295, 305
types/bbox.py                                   43      19  55.81%   46-61, 72-74, 85-87, 99-101, 113-115, 123
types/download_args.py                           9       0  100.00%
types/queryables.py                            108       1  99.07%   63
types/search_args.py                            70      18  74.29%   60-64, 71-88, 103
types/whoosh.py                                 81      16  80.25%   129-132, 136-143, 155-161, 174-176
utils/__init__.py                              538      40  92.57%   81, 198-199, 208-235, 238, 253, 333-337, 410-414, 433-435, 449, 541, 556, 596-597, 626, 1001-1004, 1012-1013, 1051-1052, 1099-1100, 1234
utils/exceptions.py                             46       0  100.00%
utils/import_system.py                          28      19  32.14%   64-78, 89-99
utils/logging.py                                28       1  96.43%   41
utils/notebook.py                               44      23  47.73%   25-29, 36-41, 58-62, 72-78, 83-87
utils/repr.py                                   38       8  78.95%   51, 53, 57, 98, 122-129
utils/requests.py                               55      29  47.27%   51-52, 64, 85-96, 107-124, 128
utils/rest.py                                   36       1  97.22%   55
utils/s3.py                                     65       3  95.38%   139-140, 203
utils/stac_reader.py                           111      45  59.46%   56-57, 63-85, 95-97, 101, 137, 153-156, 203-212, 222-252
TOTAL                                        10145    1612  84.11%

Diff against develop

Filename      Stmts    Miss  Cover
----------  -------  ------  --------
TOTAL             0       0  +100.00%

Results for commit: 8c28992

Minimum allowed coverage is 70%

♻️ This comment has been updated with latest results

[github-actions]

Code Coverage (Windows)

Filename                                     Stmts    Miss  Cover    Missing
-----------------------------------------  -------  ------  -------  --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__init__.py                                      8       0  100.00%
cli.py                                         323      64  80.19%   62-63, 87, 674-737, 839-890, 894
config.py                                      430      28  93.49%   80-82, 91, 99, 103-105, 176, 187, 692-694, 807-810, 853-854, 863-864, 969, 1000, 1032-1037, 1039
crunch.py                                        5       5  0.00%    20-24
api/__init__.py                                  0       0  100.00%
api/core.py                                    783      74  90.55%   317-318, 368, 658, 702-705, 743, 787, 821, 866-871, 897, 988, 1056, 1194, 1279-1291, 1327, 1329, 1357, 1361-1372, 1385-1391, 1474-1477, 1510-1530, 1582, 1599-1603, 1615-1618, 1954, 1978-1984, 2235, 2239-2243, 2252-2254, 2298-2299, 2328-2329
api/search_result.py                            58       4  93.10%   82, 91, 98, 112
api/product/__init__.py                          6       0  100.00%
api/product/_assets.py                          48       5  89.58%   90, 176, 184, 187-191
api/product/_product.py                        196      19  90.31%   70-72, 236-237, 312, 341, 398, 412-415, 428, 452-455, 515-518
api/product/metadata_mapping.py                695      65  90.65%   119-121, 218, 250-251, 297-298, 308-320, 322, 333, 398-399, 436, 457-460, 483, 491-492, 578-579, 603-604, 610-613, 628-629, 778, 824, 977, 986-990, 1007-1012, 1139, 1153-1173, 1193, 1198, 1327, 1341, 1366, 1412, 1464, 1487-1488, 1491, 1504-1508, 1524, 1532
api/product/drivers/__init__.py                 20       2  90.00%   33-36
api/product/drivers/base.py                     29       1  96.55%   98
api/product/drivers/generic.py                   7       0  100.00%
api/product/drivers/sentinel1.py                15       0  100.00%
api/product/drivers/sentinel2.py                15       0  100.00%
plugins/__init__.py                              0       0  100.00%
plugins/base.py                                 21       3  85.71%   48, 55, 68
plugins/manager.py                             172      14  91.86%   106-111, 161, 199, 221, 225, 249, 389-392, 404-405
plugins/apis/__init__.py                         0       0  100.00%
plugins/apis/base.py                             4       0  100.00%
plugins/apis/ecmwf.py                           97      10  89.69%   171-173, 220-221, 247-249, 302-303
plugins/apis/usgs.py                           180      26  85.56%   156, 258, 292, 327-329, 334, 360-361, 366, 396-403, 414-419, 441-447, 478
plugins/authentication/__init__.py               6       1  83.33%   31
plugins/authentication/aws_auth.py              20       0  100.00%
plugins/authentication/base.py                  17       2  88.24%   45, 58
plugins/authentication/generic.py               14       2  85.71%   51, 61
plugins/authentication/header.py                19       0  100.00%
plugins/authentication/keycloak.py              46       7  84.78%   151-154, 175-180
plugins/authentication/oauth.py                 13       7  46.15%   43-45, 49-52
plugins/authentication/openid_connect.py       206      27  86.89%   80-81, 93-111, 149, 155-183, 191, 323-326, 352
plugins/authentication/qsauth.py                34       1  97.06%   91
plugins/authentication/sas_auth.py              47       1  97.87%   76
plugins/authentication/token.py                100      16  84.00%   141, 170, 172, 207-220, 276-280
plugins/authentication/token_exchange.py        36      14  61.11%   75, 92-120
plugins/crunch/__init__.py                       0       0  100.00%
plugins/crunch/base.py                          10       1  90.00%   43
plugins/crunch/filter_date.py                   59      14  76.27%   52-57, 69, 78, 87, 90, 102-104, 113-115, 122
plugins/crunch/filter_latest_intersect.py       47      33  29.79%   49-54, 67-112
plugins/crunch/filter_latest_tpl_name.py        31       1  96.77%   83
plugins/crunch/filter_overlap.py                66      18  72.73%   28-30, 66-69, 76-79, 85, 93, 104-120
plugins/crunch/filter_property.py               30       5  83.33%   55-60, 63-64
plugins/download/__init__.py                     0       0  100.00%
plugins/download/aws.py                        491     152  69.04%   269, 282, 349-352, 366-370, 412-414, 418, 450-451, 457-461, 491, 523, 527, 534, 564-572, 576, 608-616, 627-629, 660-734, 752-810, 821-826, 879, 894-896, 899, 909-917, 925-938, 948-970, 977-989, 1027, 1053, 1098-1100, 1320
plugins/download/base.py                       253      44  82.61%   127, 155, 222-224, 235-238, 291-292, 340-344, 350-351, 393, 396-410, 422, 426, 490-494, 524-525, 550-558, 560-564, 607, 629, 651, 659
plugins/download/creodias_s3.py                 25       9  64.00%   55-69
plugins/download/http.py                       529      95  82.04%   227, 323-326, 329, 336-341, 359-374, 391, 403, 451, 458-464, 482, 496, 510, 518-520, 536-541, 552, 570, 606-609, 638, 642, 662, 739, 758-769, 777-782, 792-809, 827, 857-866, 902, 927-928, 947-952, 958, 961, 978, 981, 996-997, 1032, 1094, 1109, 1168-1169, 1175, 1185, 1221, 1257, 1277, 1312-1314
plugins/download/s3rest.py                     116      24  79.31%   119, 153, 160, 195, 222-229, 232-234, 238, 249-255, 263-264, 267-271, 294, 315-318
plugins/search/__init__.py                      22       0  100.00%
plugins/search/base.py                         145      11  92.41%   101, 105, 129-135, 275, 295, 428
plugins/search/build_search_result.py          357      45  87.39%   239, 267-268, 304, 307, 378-381, 470-487, 515, 564, 566, 597, 631-633, 637, 661, 696, 747, 761, 795-810, 853, 878, 881, 889, 1001-1002, 1008, 1019, 1077, 1128
plugins/search/cop_marine.py                   244      47  80.74%   56, 64-66, 76-77, 82, 87-88, 104, 106, 109, 175-176, 228, 234, 238, 242, 255, 266-267, 275, 303, 307, 322, 326, 330, 334, 338-342, 348-351, 354-368, 385, 434-438, 443, 455
plugins/search/creodias_s3.py                   25       1  96.00%   51
plugins/search/csw.py                          105      81  22.86%   98-99, 103-104, 112-160, 166-179, 187-219, 237-278
plugins/search/data_request_search.py          202      69  65.84%   189-192, 208, 219, 223-224, 235, 240, 245, 252, 265-268, 322-323, 327, 337-343, 348, 374-377, 385-396, 413, 415, 422-425, 427-428, 446-450, 483, 493, 504, 517, 523-538, 543
plugins/search/qssearch.py                     737     125  83.04%   450, 508, 522, 526-532, 560-564, 677-689, 734-737, 808-809, 857, 876, 883, 895, 952, 973, 976-977, 986-987, 996-997, 1006-1007, 1034, 1116-1121, 1125-1134, 1168, 1190, 1267, 1357, 1442-1443, 1453, 1530-1534, 1596, 1599, 1605-1606, 1627, 1655-1667, 1674, 1706-1708, 1718-1724, 1754, 1777, 1782-1783, 1798, 1814, 1882-1992
plugins/search/stac_list_assets.py              25      11  56.00%   44-51, 68, 75-85
plugins/search/static_stac_search.py            75      13  82.67%   98-124, 140, 153
rest/__init__.py                                 4       2  50.00%   21-22
rest/cache.py                                   33      22  33.33%   35-37, 44-70
rest/config.py                                  25       1  96.00%   35
rest/constants.py                                6       0  100.00%
rest/core.py                                   260     153  41.15%   157, 159, 161, 164-165, 179-189, 198-199, 205, 208, 249-303, 316-352, 383-420, 435-451, 467-476, 483-522, 539, 581-666, 705, 712-765
rest/errors.py                                  69      49  28.99%   60, 65-100, 105-108, 115-118, 126-147, 155-160, 175-181
rest/server.py                                 192     192  0.00%    18-592
rest/stac.py                                   319      68  78.68%   242, 308, 330, 382-385, 412-439, 470-472, 495, 527-528, 610-650, 672-688, 715, 780-784, 791, 845-846, 852, 907, 945, 978, 997-999
rest/types/__init__.py                           0       0  100.00%
rest/types/collections_search.py                13      13  0.00%    18-44
rest/types/eodag_search.py                     180      20  88.89%   225-229, 262-264, 282, 285, 291, 295, 353, 371-386
rest/types/queryables.py                        57      13  77.19%   50-51, 58-59, 66-67, 93-98, 107-108, 162
rest/types/stac_search.py                      125      11  91.20%   126-128, 174, 189-191, 199, 203, 251, 254
rest/utils/__init__.py                          94      30  68.09%   72-78, 98, 101-102, 121-123, 136, 143, 168-176, 183-204
rest/utils/cql_evaluate.py                      48       5  89.58%   69, 76, 90, 97, 105
rest/utils/rfc3339.py                           23       5  78.26%   43-44, 48, 60, 62
types/__init__.py                              133      43  67.67%   63, 67, 76-80, 91-103, 131-133, 140-145, 186, 200, 227, 237-253, 258, 260, 282, 287, 295, 305
types/bbox.py                                   43      19  55.81%   46-61, 72-74, 85-87, 99-101, 113-115, 123
types/download_args.py                           9       0  100.00%
types/queryables.py                            108       1  99.07%   63
types/search_args.py                            70      18  74.29%   60-64, 71-88, 103
types/whoosh.py                                 81      16  80.25%   129-132, 136-143, 155-161, 174-176
utils/__init__.py                              538      40  92.57%   81, 198-199, 208-235, 238, 253, 333-337, 410-414, 433-435, 449, 541, 556, 596-597, 626, 1001-1004, 1012-1013, 1051-1052, 1099-1100, 1234
utils/exceptions.py                             46       0  100.00%
utils/import_system.py                          28      19  32.14%   64-78, 89-99
utils/logging.py                                28       1  96.43%   41
utils/notebook.py                               44      23  47.73%   25-29, 36-41, 58-62, 72-78, 83-87
utils/repr.py                                   38       8  78.95%   51, 53, 57, 98, 122-129
utils/requests.py                               55      29  47.27%   51-52, 64, 85-96, 107-124, 128
utils/rest.py                                   36       1  97.22%   55
utils/s3.py                                     65       3  95.38%   139-140, 203
utils/stac_reader.py                           111      45  59.46%   56-57, 63-85, 95-97, 101, 137, 153-156, 203-212, 222-252
TOTAL                                        10145    2047  79.82%

Diff against develop

Filename      Stmts    Miss  Cover
----------  -------  ------  --------
TOTAL             0       0  +100.00%

Results for commit: 8c28992

Minimum allowed coverage is 70%

♻️ This comment has been updated with latest results

[github-actions]

Test Results

    4 files  ±0      4 suites  ±0   6m 36s ⏱️ +3s
  607 tests ±0    604 ✅ ±0   3 💤 ±0  0 ❌ ±0 
2 428 runs  ±0  2 330 ✅ ±0  98 💤 ±0  0 ❌ ±0 

Results for commit 8c28992. ± Comparison against base commit c5805cf.

♻️ This comment has been updated with latest results.

[jlahovnik]

I think the notebook docs/notebooks/api_user_guide/2_providers_products_available.ipynb has to be updated after adding all these product types

[anesson-cs]

I think the notebook docs/notebooks/api_user_guide/2_providers_products_available.ipynb has to be updated after adding all these product types

Done.