Landsat 8 and 5 download raises exception

[MWieland]

When downloading Landsat 8 data using the script below the following exception is raised:

['B1.TIF', 'B2.TIF', 'B3.TIF', 'B4.TIF', 'B5.TIF', 'B6.TIF', 'B7.TIF', 'B8.TIF', 'B9.TIF', 'B10.TIF', 'B11.TIF', 'BQA.TIF', 'MTL.txt', 'ANG.txt', 'README.GTF']
Traceback (most recent call last):
  File "test.py", line 11, in <module>
    product.download(out_dir=target_dir, progressbar=True)
  File "/users/wiel_mc/Source/miniconda3/envs/proc_s2_flood/lib/python3.6/site-packages/pylandsat/download.py", line 69, in download
    download_file(url, dst_dir, progressbar=progressbar, verify=True)
  File "/users/wiel_mc/Source/miniconda3/envs/proc_s2_flood/lib/python3.6/site-packages/pylandsat/utils.py", line 79, in download_file
    raise requests.exceptions.HTTPError(str(r.status_code))
requests.exceptions.HTTPError: 404

It seems that the file "README.GTF" is missing and therefore the exception is raised. All other files are downloaded correctly. This problem is present for all Landsat 8 images (at least the ones I tried) and can also be reported for Landsat 5. Landsat 7 seems to download without exceptions. In all cases it seems that there is a mismatch between what product.available returns and what actually is available for download.

from pylandsat import Product

product_srcid = "LC08_L1TP_193027_20200712_20200722_01_T1"
target_dir = "/download"

product = Product(product_srcid)
print(product.available)
product.download(out_dir=target_dir, progressbar=True)

pylandsat 0.4, python 3.6, ubuntu 18.4

[yannforget]

Is it still an issue? I could not reproduce this error.

[MWieland]

Thanks for looking into this. I just tried it again with a clean conda environment and pip install pylandsat on another machine. Running above example produces the same issue again. Anything I may miss regarding the setup?

pylandsat 0.5, python 3.7, ubuntu 20.4

[jha-vineet69]

I got the same error while downloading Landsat-8 data in Google Colab with pylandsat-0.5.
!pylandsat download 'LC08_L1TP_089078_20191106_20191115_01_T1'
Here's the output in case you couldn't reproduce the error.

All the other files were downloaded correctly though.

[jha-vineet69]

The error is that there is no README.GTF file for Landsat-8 data. This can be confirmed by browsing the directories in this AWS Bucket: Landsat-8 Data on AWS or GCP Bucket: Landsat-8 Data on GCP. Hope this helps in fixing the bug.

[yannforget]

Thanks! This helps

[yannforget]

Should be solved in v0.6.0. Thanks @jha-vineet69 👍

[MWieland]

Just tested it with v0.6.0 - works like a charm. Many thanks for the fix.