Upload disalexi data into Earth Engine / Compute Monthly summaries

[qjhart]

The supplied data from Martha contains daily data, that includes some caveats in terms of their use. These data need to be provided to EarthEngine in a manner consistant with the other monthly data. In order to do that, we need to :

• Either convert locally, and then upload the monthly results to Earth Engine. We could do that with some combination of Postgis, grass, ArcGIS, gdal.
• Convert the daily data to a single 365 band file ready for upload into EarthEngine. Do the math on EE.
• Convert the daily data to multiband tiffs (for each data type), and then use EE's new command line tools to upload the data into an ImageCollection.

This data is located at file://quinn.cws.ucdavis.edu/mnt/nas/ssj-disalexi

[qjhart]

In the end, I decided on the first method. This allowed me to calculate smaller geotiff files that could be more easily imported into EarthEngine. Then I could quickly convert these to the monthly values.

There were a couple of steps for that:
I needed to convert the data. You need to use Int16, because a daily ET can be negative.

cd /mnt/nas/quinn/ssj-disalexi/OUTPUT
for i in ETd_2015*.hdr; do 
 d=`basename $i .hdr | sed -e s'/ETd_//'`; 
 gdal_calc.py -A ETd_$d --outfile=$d.tif --calc="10*A" --type='Int16' --co 'COMPRESS=DEFLATE'; 
done

I need to authenticate myself, and set the proper project

gcloud auth login
gcloud config set project api-project-883511445304

Earthengine can only get data from Google cloud store, so I needed to get them into that, using gsutil, and a bucket that I already had.

gsutil cp 2015*.tif gs://qjhart-cats/

Finally, I used the earthengine command line tool to add these into earth engine.

for i in 2015*.tif; do 
 n=`echo $i | sed -e 's/^2015//' -e s/\.tif//`; 
 d=`date --date="2014-12-31 +$n days" --iso`; 
 earthengine upload image --asset_id=users/qjhart/ssj-delta-cu/ssj-disalexi/et_daily/$d \
 --time_start="$d" --pyramiding_policy=mean --nodata_value=-32767 gs://qjhart-cats/$i; 
done

[qjhart]

After uploading, I needed to convert the data into monthly sets. The earthengine script ssj-disalexi/daily_to_monthly.js performs that conversion.
NOTE --time_start is SUPER IMPORTANT. You will be sad and have to enter these data by hand if you forget