High-throughput Onboard Hyperspectral Image Compression with Ground-based CNN Reconstruction

Code from paper High-throughput Onboard Hyperspectral Image Compression with Ground-based CNN Reconstruction, IEEE Transactions on Geoscience and Remote Sensing, 2019. The code implements image restoration for the bounded absolute error objective.

BibTex reference:

@article{valsesia2019high,
  title={High-throughput Onboard Hyperspectral Image Compression with Ground-based CNN Reconstruction},
  author={Valsesia, Diego and Magli, Enrico},
  journal={IEEE Transactions on Geoscience and Remote Sensing},
  year={2019},
  volume={},
  number={},
  pages={},
  doi={10.1109/TGRS.2019.2927434}, 
}

Requirements

• Python 3.6
• Pytorch 0.4.0

Python packages are listed in requirements.txt. Install them with:

pip install -r requirements.txt

Data format

Training data must be placed in the Data/prequantization/train/Q61/ directory. Substitute prequantization with ccsds when training the model to reconstruct from CCSDS 123.0-B-2, and Q61 with the desired quantization step size. The directory must contain two files:

• all_Q61.mat : the noisy data, e.g. the output of the prequantization method after dequantization, or the output of the CCSDS-123.0-B-2 lossy decompressor. A MAT v7.3 file generated by matlab (or a compatible h5 archive) containing a 4-D tensor named PallQ of type uint16 of size number of patches x 32 x 32 x 8;
• all_orig.mat : the clean data. A MAT v7.3 file generated by matlab (or a compatible h5 archive) containing a 4-D tensor named Pall of type uint16 of size number of patches x 32 x 32 x 8. Notice that they must be the clean versions of the same patches stored in all_Q61.mat, in the same order.

Test data must be placed in the Data/prequantization/test/Q61/ directory. Substitute prequantization with ccsds when using the model to reconstruct from CCSDS 123.0-B-2, and Q61 with the desired quantization step size. The file with the noisy data must be:

• scene_name_Q61.mat: scene_name is the same name specified in the launcher_test.sh script. The file is MAT v7.3 file generated by matlab (or a compatible h5 archive) containing an image to be restored as a 3-D tensor (lines x columns x bands) with type single and named img_q.

Training

Run the following script:

./launcher_train.sh

Customize the paramters in the script to train the desired model and quantization step size. A validation file must be provided for the test_quant_filename parameter and place in the directory specified by test_dir. This file has the same format as testing data. If a start_iter file with the iteration number if found in the log_dir training will load the last checkpoint from the Results/prequantization/Q61/saved_models directory and resume from there.

Testing

Run the following script:

./launcher_test.sh

Customize the paramters in the script to test the desired model and quantization step size.

Pretrained models

Pretrained networks are provided for the both the ccsds and prequantization algorithms for the folowing quantization step size: 3,7,11,15,21,31,41,61,101. They can be found in the Results directories. A decompressed test image is available under directory Data/prequantization/test/Q61/.