Free data catalog and cloud processing system Google Earth Engine (GEE) provides a lot of of geospatial datasets. This project uses a chunked downloading to extract big rasters, even dozens of gigabyte each from Google Earth Engine. For an example, that is possible to download 500 GB raster in 12-24 hours.
For Google Earth Engine (GEE) access the service account key required (it's named /root/gee-export.json in the scripts), to create your own one follow the link Create and register a service account to use Earth Engine
Use provided script geotif-to-bqcsv.py to convert WGS 84 GeoTIFF files to BigQuery CSV data and table schema:
./geotif-to-bqcsv.py GeoTIFF_file [CSV_file]
or for batch conversion:
find . -name '*.tif' -print0 | parallel -0 geotif-to-bqcsv.py '{}' '{}'.csv
With just mandatory first argument the script returns corresponding BigQuery table schema only. With the optional second argument this script converts the entire GeoTIFF file to CSV output into the specified file and also prints the schema too.
See WorldPop.sh to extract data for 2020 year in WGS84 coordinates.
See AnnualNPP.sh to extract the entire dataset and convert it into WGS84 coordinates.
See GFS.sh to extract data for date "2021/04/13" and forecasting interval 384 hours in WGS84 coordinates.
