Sudden failure of retrival with H5Coro::Future read failure reporting

[trchudley]

Hi all, thanks for a great tool. I've been playing around with sliderule (version 4.9.1) for a couple of weeks now for downloading ATL06-SR data. This afternoon, my scripts started failing to download any data, even down to the most basic working examples. For instance, replicating the Boulder watershed demo:

from sliderule import icesat2

region = [ {"lon":-105.82971551223244, "lat": 39.81983728534918},
           {"lon":-105.30742121965137, "lat": 39.81983728534918},
           {"lon":-105.30742121965137, "lat": 40.164048017973755},
           {"lon":-105.82971551223244, "lat": 40.164048017973755},
           {"lon":-105.82971551223244, "lat": 39.81983728534918} ]

parms = {
    "poly": region,
    "srt": icesat2.SRT_LAND,
    "cnf": icesat2.CNF_SURFACE_HIGH,
    "ats": 10.0,
    "cnt": 10,
    "len": 40.0,
    "res": 20.0
}

# Request ATL06 Data
gdf = icesat2.atl06p(parms)

# Display Statistics
print("Received {} elevations".format(len(gdf)))

Returns a series of Exceptions, and no data:

Exception <-1>: Failure on resource ATL03_20191228133623_00280606_006_01.h5 track 2.1: H5Coro::Future read failure on [/gt2r/geolocation/reference_photon_lat](http://localhost:8888/gt2r/geolocation/reference_photon_lat)
Exception <-1>: Failure on resource ATL03_20191201024430_09960502_006_01.h5 track 1.0: H5Coro::Future read failure on [/gt1l/geolocation/reference_photon_lat](http://localhost:8888/gt1l/geolocation/reference_photon_lat)
Exception <-1>: Failure on resource ATL03_20191201024430_09960502_006_01.h5 track 2.0: H5Coro::Future read failure on [/gt2l/geolocation/reference_photon_lat](http://localhost:8888/gt2l/geolocation/reference_photon_lat)
Exception <-1>: Failure on resource ATL03_20191228133623_00280606_006_01.h5 track 1.0: H5Coro::Future read failure on [/gt1l/geolocation/reference_photon_lat](http://localhost:8888/gt1l/geolocation/reference_photon_lat)
Exception <-1>: Failure on resource ATL03_20200229222416_09960602_006_01.h5 track 2.0: H5Coro::Future read failure on [/gt2l/geolocation/reference_photon_lat](http://localhost:8888/gt2l/geolocation/reference_photon_lat)
Exception <-1>: Failure on resource ATL03_20200229222416_09960602_006_01.h5 track 1.0: H5Coro::Future read failure on [/gt1l/geolocation/reference_photon_lat](http://localhost:8888/gt1l/geolocation/reference_photon_lat)

[snip - 762 exception lines total]

Received 0 elevations

Is there anything obvious I'm missing?

[jpswinski]

@trchudley Thanks for reporting this issue. I am able to reproduce the problem and am currently investigating. I will post here as soon as it is resolved.

[jpswinski]

I've identified the immediate issue - the backend servers, starting this afternoon, are failing to authenticate to Earth Data Login and are therefore unable to obtain current credentials to access the h5 granules in S3. I don't know why yet... continuing to investigate.

[jpswinski]

The issue is either with our Earth Data Login (EDL) account, or with EDL itself. Even trying to log in manually met with a number of failures. I am reaching out to them now to see if we can get the matter resolved.

[trchudley]

Thanks @jpswinski, just glad to know I'm not going mad! Please do keep me in the loop, looking forward to jumping back in when it's sorted.

[jpswinski]

Someone at the NSIDC reported that EDL is in an outage right now. I've reached out directly to Earthdata Support since their status page shows normal operation.

[trchudley]

Hi @jpswinski, as of this morning everything seems to be working again:

Received 369932 elevations

Will leave you to close this in case there's anything that needs doing. Thanks for your prompt response and help!