SATELLITE IMAGE ANALYSIS WITH MACHINE LEARNING

PRESENTATION SLIDES: project_report (pdf)

OUTPUT FILES FORMAT

Folders with results for CNN, SML algorithms are generated within each of the .SAFE output folder.

Format	Example	Use
{resolution}.tif	10m.tif	All ordered bands of S2/L8 stacked in a single file with 10m resolution (multispectral)
{resolution}-{bands}-{product}-{type}.tif	10m-9-built-prediction.tif	BUILT prediction(or confidence) file using 9 bands with 10m resolution
{resolution}-{bands}-{product}.tif	10m-9-pop.tif	SML based POP file using 9 bands
{resolution}-original-{product}.tif	10m-original-built.tif	Original GHSL products(BUILT,SPOP,SMOD) clipped as the same region and resolution as input file
metrics.json	metrics.json	Metrics calculated by comparing BUILT and Original GHSL-BUILT (meanIOU, F1 score etc.)

SAMPLE RESULTS

Input Data	Results	Algorithm
/netscratch/delhikar/test_data/	/netscratch/delhikar/results_all_new/	CNN/SML
/netscratch/delhikar/WSF/version2/test_data/	/netscratch/delhikar/WSF/version2/test_data_results_new/	WSF

SYMBOLIC MACHINE LEARNING (SML) BASED ALL PRODUCTS (Multiprocessing mode)

Generates all SML based products (BUILT,SPOP,SMOD) for a given directory containing a list of Sentinel2(.SAFE) or Landsat8 satellite folders.

Parameter	Use
-c	Configuration file path
-rf	Parent folder containing a list of s2(.SAFE) or l8 folders
-of	Output folder
-nb	No. of bands to be used for inference
-st	Satellite type - s2 or l8

Command:

$ python3 sml-all-products.py -c '<path to config.json>' -rf '<parent folder of bulk s2/l8 images>' -of '<output folder to store results>' -nb <no. of bands> -st '<satellite type- s2 or l9>'

Example:

$ python3 sml-all-products.py -c "/netscratch/delhikar/GHS-SML/version4/config.json" -rf  "/netscratch/delhikar/test_data/" -of  "/netscratch/delhikar/test_data_results" -st "s2"  -nb  9

CNN BASED PRODUCTS

Generates all CNN based products (BUILT,POP,SMOD) for a given root directory containing a list of processed Sentinel2(.SAFE) sub-folders.

Parameter	Use
-c	Configuration file path
-rf	Parent folder containing a list of .SAFE folders
-of	Output folder
-st	Satellite type - s2 or l8

Command:

$ python3 cnn-all-products.py -c '<path to config.json>' -rf '<parent folder with list of .SAFE folders>' -of '<output folder to store results>' -st '<satellite type- s2 or l9>'

Example:

$ python3 cnn-all-products.py -c "/netscratch/delhikar/GHS-SML/version4/config.json" -rf  "/netscratch/delhikar/test_data/" -of  "/netscratch/delhikar/test_data_results" -st "s2"  -st 's2'

SYMBOLIC MACHINE LEARNING (SML) TRAINING DATA DOWNLOADER

Execute in the current directory you want to download and organize training data for SML automatically. (Change username and password in the code).

Command:

$ python3 s2-data-downloader.py 

SYMBOLIC MACHINE LEARNING (SML) BASED BUILT TRAINING

Trains SML algorithm using above training data generated in previous step for a given number of bands.

Parameter	Use
-c	Configuration file path
-nb	No. of bands to be used for training

Command:

$ python3 sml-built-train.py -c '<path to config.json>' -nb <NO. OF BANDS> 

Example:

$ python3 sml-built-train.py -c "/netscratch/delhikar/GHS-SML/version4/config.json" -nb 9 

SYMBOLIC MACHINE LEARNING (SML) BASED BUILT

Generates SML-BUILT(prediction and confidence) products for a given multispectral image using specified number of bands

Parameter	Use
-c	Configuration file path
-nb	No. of bands to be used for inference
-i	A Multi Spectral (MS) TIF file, with 1-12 bands
-of	Output folder of results

Command:

$ python3 sml-built-generator.py -c '<path to config.json>' -nb <NO. OF BANDS> -i '<path to Multispectral (MS) image>' -of '<output folder path>'

Example:

$ python3 sml-built-generator.py -c "/netscratch/delhikar/GHS-SML/version4/config.json" -nb 9 -i "/netscratch/delhikar/results_all_new/S2A_MSIL1C_20190424T101031_N0207_R022_T33VXF_20190424T153347.SAFE/10m.tif" -of "/netscratch/delhikar/results_all_new/S2A_MSIL1C_20190424T101031_N0207_R022_T33VXF_20190424T153347.SAFE/SML/"

POPULATION GRID (POP)

Generates POP product for a given BUILT(prediction and confidence) products resulting from the above step

Parameter	Use
-c	Configuration file path
-cf	A GHS-BUILT (Confidence) TIF file, with 0-100%
-pr	A GHS-BUILT (Prediction) TIF file, with 0,1,2 (0=BUILT UP, 1=OTHERS , 2=WATER)
-of	Output folder to store results

Command:

$ python3 sml-spop-gpw-count-generator.py -c '<path to config.json>' -cf <'path to GHS-BUILT (confidence) file generated from previous step'> -pr '<path to GHS-BUILT (prediction) file generated from previous step>' -of '<Output folder>'  

Example:

$ python3 sml-spop-gpw-count-generator.py -c "/netscratch/delhikar/GHS-SML/version4/config.json" -cf "/netscratch/delhikar/results_all_new/S2A_MSIL1C_20190424T101031_N0207_R022_T33VXF_20190424T153347.SAFE/SML/10m-9-built-confidence.tif" -pr "/netscratch/delhikar/results_all_new/S2A_MSIL1C_20190424T101031_N0207_R022_T33VXF_20190424T153347.SAFE/SML/10m-9-built-prediction.tif" -of "/netscratch/delhikar/results_all_new/S2A_MSIL1C_20190424T101031_N0207_R022_T33VXF_20190424T153347.SAFE/SML/"

HUMAN SETTLEMENT MODEL GRID (SMOD)

Generates SMOD product from a given BUILT (prediction/confidence) and POP products resulted from the above steps.

Parameter	Use
-cf	A GHS-BUILT (confidence) TIF file, with 0-100%
-pr	A GHS-BUILT (prediction) TIF file, with 0,1,2 (0=BUILT UP, 1=UNKNOWN, 2=WATER)
-p	A GHS-POP TIF file
-of	Output folder

Command:

$ python3 sml-smod-generator.py -cf '<path to confidence file>' -pr 'path to prediction generated from previous step' -p '<path to GHS-POP generated from previous step>' -of '<Output folder>'

Example:

$ python3 sml-smod-generator.py -cf '/netscratch/delhikar/GHS-SML/version3/italy-confidence.tif' -pr '/netscratch/delhikar/GHS-SML/version3/italy-prediction.tif' -p '/netscratch/delhikar/GHS-SML/version3/test/italy-pop.tif' -of '/netscratch/delhikar/GHS-SML/version3/30m/results'

WORLD SETTLEMENT FOOTPRINT (WSF)-BUILT

Generates WSF based predictions for a given folder containing multitemporal scenes

Parameter	Use
-c	Configuration file path
-i	Multi Spectral (MS) TIF file, with 3 temporal scenes

Command:

$ python3 wsf.py -c '<path to config.json>' -if <INPUT FOLDER - MULTITEMPORAL PROCESSED SCENES> 

REFERENCES

Mentioned in project report: project_report (pdf)