Projection-based-PC-Quality-Metric

Introduction

A novel joint geometry and color projection-based point cloud objective quality metric which solves the critical weakness of this type of quality metrics, i.e., the misalignment between the reference and degraded projected images. Moreover, this point cloud quality metric exploits the best performing 2D quality metrics in the literature to assess the quality of the projected images.

Installation Instructions

Image Quality Assessment (IQA) Models in PyTorch should be installed

pip install IQA_pytorch

PyTorch Image Quality (PIQ) is also required if you are intending to use HaarPSI metric

pip install piq

An Implementation of PSNR HVS is included in this repository. The corresponding papers are cited in our published article.

Open3d python packages are also necessary to read/write point clouds.

pip install open3d

Conda instalation

conda install -c open3d-admin -c conda-forge open3d

OpenCV is also necessary for image processing operations in the metric.

pip install opencv-python

You need to also install Numpy, Pandas, and PyTorch

NOTE: IQA and PIQ do not support newer versions of pytorch.

Usage

All metric parameters should be set inside the configuration file (config.ini) before running the metric. If it is the first time runing the metric and recolored point clouds are not available:

python3 compute_projqm.py -a reference PC -b degraded PC -c config.ini -o output.csv

Note that path to PCs is set inside the config file, they are brought to command line arguments to make it easier to run metric for a batch of PCs.

If you want to save point clouds, savePCs should be set to 1 (savePCs = 1)

If you want to save projected images, saveImages should be set to 1 (saveImages = 1). All images of the PC should be stored in a separate folder. The path to this folder is set inside the configuration file.

Precision of the input point clouds should also be set in the configuration file.

There is also a flag for each of the 2D quality metrics that you want to be included in the final results

You can see a sample configuration file below:

[Paths]
# Path to directory including reference point clouds
PCA = C:\AlirezaJav\Datasets\EPFL_MPEG_Codecs\stimuli\  
# Path to directory including degraded point clouds                           
PCB = C:\AlirezaJav\Datasets\EPFL_MPEG_Codecs\stimuli\	
# Path to directory including recolored reference point clouds                         
PCA_rec = C:\AlirezaJav\Projects\Projection-based Metric\Final Software\Recolored PCs\
# Path to directory including recolroed degraded point clouds 
PCB_rec = C:\AlirezaJav\Projects\Projection-based Metric\Final Software\Recolored PCs\
# Path to directory including six projected images of the reference point cloud (All six projected images of a point cloud should be in a folder)
RefImages = C:\AlirezaJav\Projects\Projection-based Metric\Final Software\RefImages\
# Path to directory including six projected images of the degraded point cloud (All six projected images of a point cloud should be in a folder)
DegImages = C:\AlirezaJav\Projects\Projection-based Metric\Final Software\DegImages\
# Path to directory including six projected images of the recolored reference point cloud (All six projected images of a point cloud should be in a folder)
RefImages_rec = C:\AlirezaJav\Projects\Projection-based Metric\Final Software\RecoloredRefImages\
# Path to directory including six projected images of the recolored degraded point cloud (All six projected images of a point cloud should be in a folder)
DegImages_rec = C:\AlirezaJav\Projects\Projection-based Metric\Final Software\RecoloredDegImages\
# Path to directory including six occupancy maps of the reference point cloud (All six occupancy maps of a point cloud should be in a folder)
RefOMs = C:\AlirezaJav\Projects\Projection-based Metric\Final Software\RefOMs\
# Path to directory including six occupancy maps of the degraded point cloud (All six occupancy maps of a point cloud should be in a folder)
DegOMs = C:\AlirezaJav\Projects\Projection-based Metric\Final Software\DegOms\

[Flags]
# Set to 1 if projected images are available and there is no need for projection, 0 otherwise
projected = 0
# Set to 1 if recolored point clouds are saved before and there is no need for recoloring, 0 otherwise
Recolored = 0
# Set to 1 to save recolored point clouds for further use
savePCs = 1
# Set to 1 to save projected images after pre-processing, otherwise 0 (they can be evaluate directly by 2D metric later)
saveImages = 1


[parameters]
# precision of the input PC
precision = 10
# window search size for filtering points after projection. Put zero if you don't want filtering (W = 2*window size + 1)
window size = 2

[2D Metrics]
# compute DISTS
dists = 1
# compute LPIPS
lpips = 1
# compute FSIM
fsim = 1
# compute VSI
vsi = 1
# compute HaarPSI
haarpsi = 1
# compute VIFp
vifp = 1
# compute SSIM
ssim = 1
# compute MS-SSIM
ms-ssim = 1
# compute HVS PSNR and HVS PSNR M
psnr-hvs = 1
# compute PSNR
psnr = 1

If Recolored PCs are already available, recolored flag should be set to 1 (Recolored = 1) and run the metric as follows:

python3 compute_projqm.py -a reference PC -b degraded PC -ar recolored reference PC -br recolored degraded PC -c config.ini -o output.csv

If projected images are saved and you need to only compute a new 2D metric, projected flag should be set to 1 (projected = 1) and run the metric as follows:

python3 compute_projqm.py -c config.ini -o output.csv

Performance

Below table shows the objective-Subjective correlation performance of this metric, compared with most famous state-of-the-art metrics using the MOS scores provied in M-PCCD dataset.

Type	Metric	SROCC	PLCC	RMSE
Point-to-Point	D1-PSNR	79.1	77.7	0.857
Point-to-Point	GH 98% PSNR	86.9	84.6	0.726
Point-to-Point	RA-PSNR (APD10)	90.2	88.8	0.626
Point-to-Point	Y-PSNR	66.2	67.1	1.009
Point-to-Plane	D2-PSNR	83.8	80.5	0.808
Point-to-Plane	GH 98% PSNR	87.9	84.3	0.731
Point-to-Plane	RA-PSNR (APD10)	89.9	88.9	0.622
Feature-Based	PointSSIM	91.8	92.6	0.514
Feature-Based	dgc	92.0	90.4	0.585
Feature-Based	HYL2	88.4	85.3	0.710
Feature-Based	PCMRR(MCCV)	90.7	90.2	0.573
Point-to_Distribution	P2D-JGY	93.8	92.9	0.503
Point-to_Distribution	LogP2D-JGY	93.8	92.9	0.502
Projection-based	JGC-ProjQM-FSIM	90.1	88.2	0.640
Projection-based	JGC-ProjQM-VSI	87.6	85.4	0.707
Projection-based	JGC-ProjQM-LPIPS	93.2	92.3	0.523
Projection-based	JGC-ProjQM-DISTS	95.6	94.7	0.439

Reference

If you are using this metric, please cite the following publications:

A. Javaheri, C. Brites, F. Pereira, J. Ascenso "Joint Geometry and Color Projection-based Point Cloud Quality Metric", arXiv preprint arXiv:2108.00054., July 2021.

This article is submitted to IEEE Access journal.