-
Notifications
You must be signed in to change notification settings - Fork 12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ADD: GPM-API Tutorial #84
Conversation
👋 Thanks for opening this PR! The Cookbook will be automatically built with GitHub Actions. To see the status of your deployment, click below. |
Thanks @ghiggi, I'll merge tomorrow, if everything runs well. Agreed, to add changes in subsequent PR's. |
Absolutely I can @ghiggi 👍 I can take care of it tomorrow morning! |
@ghiggi There is an example how to grab the data from locally cached fsspec here (see also PyDDA Notebook): # Set the URL and path for the cloud
URL = "https://js2.jetstream-cloud.org:8001/"
path = f"pythia/radar/erad2024"
fs = fsspec.filesystem("s3", anon=True, client_kwargs=dict(endpoint_url=URL))
fs.glob(f"{path}/*")
files = fs.glob("pythia/radar/erad2024/gpm-api/*")
local_files = [
fsspec.open_local(
f"simplecache::{URL}{i}", s3={"anon": True}, filecache={"cache_storage": "."}
)
for i in files
]
# grab the file with eg. `local_files[0]` or directly down load the data like this: import os
import urllib.request
from pathlib import Path
# Set the URL for the cloud
URL = "https://js2.jetstream-cloud.org:8001/"
path = "pythia/radar/erad2024"
!mkdir -p data
files = [
"gpm-api/2A.GPM.DPR.V9-20211125.20230820-S213941-E231213.053847.V07B.zarr))",
"gpm-api/KNKX20230820_221341_V06.zarr",
]
for file in files:
file_remote = os.path.join(path, file)
file_local = os.path.join("data", Path(file).name)
if not os.path.exists(file_local):
print(f"downloading, {file_local}")
urllib.request.urlretrieve(f"{URL}{file_remote}", file_local)
# files available in local data-subfolder |
Added the docker tag and going to merge when ready. |
Thanks @ghiggi! |
This PR adds a draft of the tutorial introducing GPM-API that I think can already be merged.
I plan to fix a couple of stuffs and make a new package release in the coming days and then open another PR to update the dependencies if you agree.