Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[mesh] Texturing: Multi-band blending #629

Merged
merged 54 commits into from
Jun 20, 2019
Merged

[mesh] Texturing: Multi-band blending #629

Show file tree
Hide file tree
Changes from 53 commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
9152f4b
[mvsData] Create Image class
cvere Mar 22, 2019
5dfaf62
[mesh] Texturing: add missing reference
cvere Mar 22, 2019
84ee7ba
[mesh] texturing: create laplacian pyramid and recompose the image
cvere Mar 22, 2019
cf02002
[mvsUtils] ImageCache : function to setCacheSize
cvere Mar 29, 2019
885d38d
[mesh] Texturing : create laplacian pyramid from downscaled images
cvere Mar 29, 2019
6ea3306
[mvsData] Image : Color interpolation valid on borders
cvere Mar 29, 2019
f4e9ebb
[mvsData] Image : operators
cvere Mar 29, 2019
b124297
[mvsData] Image : difference between different sizes images
cvere Mar 29, 2019
94ac053
[mesh] Texturing : new function to write a texture + fix padding
cvere Mar 29, 2019
d7b38d1
[mesh] Texturing : memory management
cvere Mar 29, 2019
9160841
[mesh] Texturing : multiBandBlending
cvere Mar 29, 2019
553e178
[mesh] Texturing : debug mode
cvere Mar 29, 2019
4e3383d
[mesh] Texturing : delete sizeKernel & maxNbImagesForFusion
cvere Mar 29, 2019
95f373d
[mesh] Texturing : optimize padding
cvere Mar 29, 2019
7dc8eb6
[mesh] Texturing: add downscale param for multiband blending
cvere Apr 5, 2019
f41101e
[mesh] Texturing: add nbBand param for multiband blending
cvere Apr 5, 2019
35d33a7
[mesh] Texturing: keep margin in memory use
cvere Apr 5, 2019
fa0f07c
[mesh] Texturing : optimize texture padding
cvere Apr 5, 2019
d2f29dd
[mvsData] Image: handle invalid pixels in color interpolation
cvere Apr 5, 2019
77b549d
Merge remote-tracking branch 'origin/develop' into dev_multiBandBlending
cvere Apr 9, 2019
8e6b830
[mesh] Texturing : error fix
cvere Apr 12, 2019
19dac6f
[mesh] Texturing : final memory optimization
cvere Apr 12, 2019
770499e
[mesh] Texturing: taking triangles scores into account
cvere Apr 15, 2019
000aba7
[utils] New main to test multi-band blending
cvere Apr 26, 2019
22096c7
[mvsUtils] load images from cache in a chosen colorspace
cvere Apr 26, 2019
9a85cd0
[mesh] texturing: write textures from sRGB colorspace
cvere Apr 26, 2019
c7929e8
[imageIO] image: fix error
cvere Apr 26, 2019
e841b64
[mvsData] Image: fix color interpolation and resize functions
cvere Apr 26, 2019
70f7a75
[mesh] texturing: taking triangles scores into account
cvere Apr 26, 2019
3202d80
[mesh] Texturing: fix number of contributions for multi-band blending
cvere Apr 26, 2019
788705a
[imageIO] add OutputFileColorSpace struct
cvere Apr 26, 2019
3b0681e
[mesh] texturing: debug mode
cvere Apr 26, 2019
d3c8ca7
[mesh] texturing: fix padding
cvere Apr 26, 2019
69998ac
[mesh] texturing: change memory estimation
cvere Apr 26, 2019
94546a2
[mesh] texturing: add comments
cvere Apr 26, 2019
4d126bf
[samples] Move main_multibandtest to samples
yann-lty May 13, 2019
29c9e1d
[mesh] Texturing: set parameters for multi-band blending
cvere May 7, 2019
da08a26
[mesh] Texturing: add multiBandNbContrib param
cvere May 13, 2019
e503f75
[mesh] Texturing: add log infos, comments and delete useless variables
cvere May 14, 2019
6249819
[mesh] Texturing: add debug mode
cvere May 14, 2019
b488f2c
[mesh] Texturing: fix number of contributions per band
cvere May 14, 2019
bd89d50
[mesh] Texturing: handle "0"or illogical contributions for MBB
cvere May 14, 2019
01d9862
Texturing: merge MultiBandBlending class in Image class + clean code
cvere Jun 4, 2019
c4d3678
Merge imageIO module into mvsData/imageIO
cvere Jun 5, 2019
973af01
Merge mvsUtils::ImagesCache::Img class into mvsData::Image
cvere Jun 5, 2019
17a5d51
[mesh][mvsUtils] check if mesh triangles reproject correctly
cvere Jun 12, 2019
1531052
[mesh] Texturing: fix nbContribMax calculus
cvere Jun 12, 2019
850ca7a
[mesh] Texturing: bug fix on a shared_ptr
cvere Jun 12, 2019
118ed54
[mesh] Texturing: code cleaning & setting param default value
cvere Jun 12, 2019
7fdc82b
[mesh] Texturing: set bestScoreThreshold default value to 0.1
cvere Jun 13, 2019
d794c42
code cleaning on struct OutputFileColorSpace
cvere Jun 14, 2019
8bc2e79
Merge remote-tracking branch 'origin/develop' into dev_multiBandBlending
cvere Jun 18, 2019
9f189bc
Explicit constructor for OutputFileColorSpace struct & missing reference
cvere Jun 20, 2019
956df2a
Merge branch 'develop' into dev_multiBandBlending
cvere Jun 20, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion src/aliceVision/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,6 @@ endif()
# MVS modules

if(ALICEVISION_BUILD_MVS)
add_subdirectory(imageIO)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We tried to keep a split between IO and data structures as almost all modules have dependencies to data structures.
So if we don't split it, all modules will have dependencies to IO.

But maybe this commit is a needed intermediate solution before a new change? If not, we do not want to merge mvsData and imageIO.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have a several files related to image IO:

  • image/io.hpp // SfM: functions to read/writes images
  • imageIO/image.hpp // MVS: functions to read/write images, relies on mvsData for data structure types (rgb, Color and now Image)
  • mvsData/imageIO.hpp (image.hpp on develop) // MVS: containing EImageFileType enum

The idea was to start simplifying things. We will probably want to merge image types and IO at some point, but it already seems weird to have, for the MVS part, this split between imageIO/image.hpp and mvsData/imageIO.hpp. And everything that uses mvsData also uses imageIO, so the merge does not seem that problematic IMHO. But if we want to keep this imageIO module as the one that will stay, maybe it could be the other way around: move what's inside mvsData/imageIO.hpp into the imageIO module.

add_subdirectory(lightingEstimation)
add_subdirectory(mesh)
add_subdirectory(mvsData)
Expand Down
1 change: 0 additions & 1 deletion src/aliceVision/depthMap/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,6 @@ alicevision_add_library(aliceVision_depthMap
${depthMap_cuda_files_sources}
PUBLIC_LINKS
aliceVision_mvsData
aliceVision_imageIO
aliceVision_mvsUtils
aliceVision_system
${Boost_FILESYSTEM_LIBRARY}
Expand Down
18 changes: 12 additions & 6 deletions src/aliceVision/depthMap/DepthSimMap.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
#include <aliceVision/mvsData/Color.hpp>
#include <aliceVision/mvsData/geometry.hpp>
#include <aliceVision/mvsData/jetColorMap.hpp>
#include <aliceVision/imageIO/image.hpp>
#include <aliceVision/mvsData/imageIO.hpp>

#include <iostream>

Expand Down Expand Up @@ -383,7 +383,9 @@ void DepthSimMap::saveToImage(std::string filename, float simThr)
}
}

imageIO::writeImage(filename, bufferWidth, h, colorBuffer, imageIO::EImageQuality::LOSSLESS, imageIO::EImageColorSpace::NO_CONVERSION);
using namespace imageIO;
OutputFileColorSpace colorspace(EImageColorSpace::NO_CONVERSION);
writeImage(filename, bufferWidth, h, colorBuffer, EImageQuality::LOSSLESS, colorspace);
}
catch(...)
{
Expand Down Expand Up @@ -415,8 +417,10 @@ void DepthSimMap::save(int rc, const StaticVector<int>& tcams)
metadata.push_back(oiio::ParamValue("AliceVision:P", oiio::TypeDesc(oiio::TypeDesc::DOUBLE, oiio::TypeDesc::MATRIX44), 1, matrixP.data()));
}

imageIO::writeImage(getFileNameFromIndex(mp, rc, mvsUtils::EFileType::depthMap, scale), width, height, depthMap->getDataWritable(), imageIO::EImageQuality::LOSSLESS, imageIO::EImageColorSpace::NO_CONVERSION, metadata);
imageIO::writeImage(getFileNameFromIndex(mp, rc, mvsUtils::EFileType::simMap, scale), width, height, simMap->getDataWritable(), imageIO::EImageQuality::OPTIMIZED, imageIO::EImageColorSpace::NO_CONVERSION, metadata);
using namespace imageIO;
OutputFileColorSpace colorspace(EImageColorSpace::NO_CONVERSION);
writeImage(getFileNameFromIndex(mp, rc, mvsUtils::EFileType::depthMap, scale), width, height, depthMap->getDataWritable(), EImageQuality::LOSSLESS, colorspace, metadata);
writeImage(getFileNameFromIndex(mp, rc, mvsUtils::EFileType::simMap, scale), width, height, simMap->getDataWritable(), EImageQuality::OPTIMIZED, colorspace, metadata);
}

void DepthSimMap::load(int rc, int fromScale)
Expand Down Expand Up @@ -466,8 +470,10 @@ void DepthSimMap::saveRefine(int rc, std::string depthMapFileName, std::string s
metadata.push_back(oiio::ParamValue("AliceVision:P", oiio::TypeDesc(oiio::TypeDesc::DOUBLE, oiio::TypeDesc::MATRIX44), 1, matrixP.data()));
}

imageIO::writeImage(depthMapFileName, width, height, depthMap, imageIO::EImageQuality::LOSSLESS, imageIO::EImageColorSpace::NO_CONVERSION, metadata);
imageIO::writeImage(simMapFileName, width, height, simMap, imageIO::EImageQuality::OPTIMIZED, imageIO::EImageColorSpace::NO_CONVERSION, metadata);
using namespace imageIO;
OutputFileColorSpace colorspace(EImageColorSpace::NO_CONVERSION);
writeImage(depthMapFileName, width, height, depthMap, EImageQuality::LOSSLESS, colorspace, metadata);
writeImage(simMapFileName, width, height, simMap, EImageQuality::OPTIMIZED, colorspace, metadata);
}

float DepthSimMap::getCellSmoothStep(int rc, const int cellId)
Expand Down
10 changes: 6 additions & 4 deletions src/aliceVision/depthMap/RefineRc.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
#include <aliceVision/mvsData/Point3d.hpp>
#include <aliceVision/mvsUtils/common.hpp>
#include <aliceVision/mvsUtils/fileIO.hpp>
#include <aliceVision/imageIO/image.hpp>
#include <aliceVision/mvsData/imageIO.hpp>
#include <aliceVision/alicevision_omp.hpp>

#include <boost/filesystem.hpp>
Expand Down Expand Up @@ -355,7 +355,7 @@ void estimateAndRefineDepthMaps(int cudaDeviceNo, mvsUtils::MultiViewParams* mp,
const int bandType = 0;

// load images from files into RAM
mvsUtils::ImagesCache ic(mp, bandType);
mvsUtils::ImagesCache ic(mp, bandType, imageIO::EImageColorSpace::LINEAR);
// load stuff on GPU memory and creates multi-level images and computes gradients
PlaneSweepingCuda cps(cudaDeviceNo, ic, mp, sgmScale);
// init plane sweeping parameters
Expand Down Expand Up @@ -388,7 +388,7 @@ void computeNormalMaps(int CUDADeviceNo, mvsUtils::MultiViewParams* mp, const St
const int bandType = 0;
const int wsh = 3;

mvsUtils::ImagesCache ic(mp, bandType);
mvsUtils::ImagesCache ic(mp, bandType, imageIO::EImageColorSpace::LINEAR);
PlaneSweepingCuda cps(CUDADeviceNo, ic, mp, 1);

for(const int rc : cams)
Expand All @@ -407,7 +407,9 @@ void computeNormalMaps(int CUDADeviceNo, mvsUtils::MultiViewParams* mp, const St

cps.computeNormalMap(&depthMap, &normalMap, rc, 1, igammaC, igammaP, wsh);

imageIO::writeImage(normalMapFilepath, mp->getWidth(rc), mp->getHeight(rc), normalMap.getDataWritable(), imageIO::EImageQuality::LOSSLESS, imageIO::EImageColorSpace::NO_CONVERSION);
using namespace imageIO;
OutputFileColorSpace colorspace(EImageColorSpace::NO_CONVERSION);
writeImage(normalMapFilepath, mp->getWidth(rc), mp->getHeight(rc), normalMap.getDataWritable(), EImageQuality::LOSSLESS, colorspace);
}
}
}
Expand Down
8 changes: 5 additions & 3 deletions src/aliceVision/depthMap/SemiGlobalMatchingRc.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,7 @@
#include <aliceVision/mvsData/Point3d.hpp>
#include <aliceVision/mvsUtils/common.hpp>
#include <aliceVision/mvsUtils/fileIO.hpp>
#include <aliceVision/imageIO/image.hpp>
#include <aliceVision/imageIO/imageScaledColors.hpp>
#include <aliceVision/mvsData/imageIO.hpp>
#include <aliceVision/alicevision_omp.hpp>

#include <boost/filesystem.hpp>
Expand Down Expand Up @@ -528,7 +527,10 @@ bool SemiGlobalMatchingRc::sgmrc(bool checkIfExists)
std::vector<unsigned short> volumeBestId(_volumeBestIdVal->size());
for(int i = 0; i < _volumeBestIdVal->size(); i++)
volumeBestId.at(i) = std::max(0, (*_volumeBestIdVal)[i].id);
imageIO::writeImage(_sp->getSGM_idDepthMapFileName(_sp->mp->getViewId(_rc), _scale, _step), _width, _height, volumeBestId, imageIO::EImageQuality::LOSSLESS, imageIO::EImageColorSpace::NO_CONVERSION);

using namespace imageIO;
OutputFileColorSpace colorspace(EImageColorSpace::NO_CONVERSION);
writeImage(_sp->getSGM_idDepthMapFileName(_sp->mp->getViewId(_rc), _scale, _step), _width, _height, volumeBestId, EImageQuality::LOSSLESS, colorspace);
}
return true;
}
Expand Down
4 changes: 2 additions & 2 deletions src/aliceVision/depthMap/cuda/PlaneSweepingCuda.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
namespace aliceVision {
namespace depthMap {

inline const uchar4 get( mvsUtils::ImagesCache::ImgPtr img, int x, int y )
inline const uchar4 get( mvsUtils::ImagesCache::ImgSharedPtr img, int x, int y )
{
const Color floatRGB = img->at(x,y) * 255.0f;

Expand Down Expand Up @@ -137,7 +137,7 @@ void cps_fillCameraData(mvsUtils::ImagesCache* ic, cameraStruct* cam, int c, mvs
// cam->tex_hmh_g->getBuffer(),
// cam->tex_hmh_b->getBuffer(), mp->indexes[c], mp, true, 1, 0);

mvsUtils::ImagesCache::ImgPtr img = ic->getImg_sync(c);
mvsUtils::ImagesCache::ImgSharedPtr img = ic->getImg_sync(c);

Pixel pix;
{
Expand Down
1 change: 0 additions & 1 deletion src/aliceVision/fuseCut/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@ alicevision_add_library(aliceVision_fuseCut
PUBLIC_LINKS
aliceVision_mvsData
aliceVision_mvsUtils
aliceVision_imageIO
aliceVision_mesh
aliceVision_system
Geogram::geogram
Expand Down
2 changes: 1 addition & 1 deletion src/aliceVision/fuseCut/DelaunayGraphCut.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
#include <aliceVision/mvsData/Point2d.hpp>
#include <aliceVision/mvsData/Universe.hpp>
#include <aliceVision/mvsUtils/fileIO.hpp>
#include <aliceVision/imageIO/image.hpp>
#include <aliceVision/mvsData/imageIO.hpp>
#include <aliceVision/alicevision_omp.hpp>

#include "nanoflann.hpp"
Expand Down
15 changes: 9 additions & 6 deletions src/aliceVision/fuseCut/Fuser.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,7 @@
#include <aliceVision/mvsData/Stat3d.hpp>
#include <aliceVision/mvsUtils/common.hpp>
#include <aliceVision/mvsUtils/fileIO.hpp>
#include <aliceVision/imageIO/image.hpp>
#include <aliceVision/imageIO/imageScaledColors.hpp>
#include <aliceVision/mvsData/imageIO.hpp>

#include <boost/filesystem.hpp>
#include <boost/accumulators/accumulators.hpp>
Expand Down Expand Up @@ -225,8 +224,10 @@ bool Fuser::filterGroupsRC(int rc, int pixSizeBall, int pixSizeBallWSP, int nNea
}

{
imageIO::transposeImage(h, w, numOfModalsMap);
imageIO::writeImage(getFileNameFromIndex(mp, rc, mvsUtils::EFileType::nmodMap), w, h, numOfModalsMap, imageIO::EImageQuality::LOSSLESS, imageIO::EImageColorSpace::NO_CONVERSION);
using namespace imageIO;
OutputFileColorSpace colorspace(EImageColorSpace::NO_CONVERSION);
transposeImage(h, w, numOfModalsMap);
writeImage(getFileNameFromIndex(mp, rc, mvsUtils::EFileType::nmodMap), w, h, numOfModalsMap, EImageQuality::LOSSLESS, colorspace);
}

delete numOfPtsMap;
Expand Down Expand Up @@ -322,8 +323,10 @@ bool Fuser::filterDepthMapsRC(int rc, int minNumOfModals, int minNumOfModalsWSP2
metadata.push_back(oiio::ParamValue("AliceVision:P", oiio::TypeDesc(oiio::TypeDesc::DOUBLE, oiio::TypeDesc::MATRIX44), 1, matrixP.data()));
}

imageIO::writeImage(getFileNameFromIndex(mp, rc, mvsUtils::EFileType::depthMap, 0), w, h, depthMap, imageIO::EImageQuality::LOSSLESS, imageIO::EImageColorSpace::NO_CONVERSION, metadata);
imageIO::writeImage(getFileNameFromIndex(mp, rc, mvsUtils::EFileType::simMap, 0), w, h, simMap, imageIO::EImageQuality::OPTIMIZED, imageIO::EImageColorSpace::NO_CONVERSION, metadata);
using namespace imageIO;
OutputFileColorSpace colorspace(EImageColorSpace::NO_CONVERSION);
writeImage(getFileNameFromIndex(mp, rc, mvsUtils::EFileType::depthMap, 0), w, h, depthMap, EImageQuality::LOSSLESS, colorspace, metadata);
writeImage(getFileNameFromIndex(mp, rc, mvsUtils::EFileType::simMap, 0), w, h, simMap, EImageQuality::OPTIMIZED, colorspace, metadata);

if(mp->verbose)
ALICEVISION_LOG_DEBUG(rc << " solved.");
Expand Down
23 changes: 0 additions & 23 deletions src/aliceVision/imageIO/CMakeLists.txt
View file
Open in desktop

This file was deleted.

78 changes: 0 additions & 78 deletions src/aliceVision/imageIO/imageScaledColors.cpp
View file
Open in desktop

This file was deleted.

28 changes: 0 additions & 28 deletions src/aliceVision/imageIO/imageScaledColors.hpp
View file
Open in desktop

This file was deleted.

1 change: 0 additions & 1 deletion src/aliceVision/mesh/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@ alicevision_add_library(aliceVision_mesh
PUBLIC_LINKS
aliceVision_mvsData
aliceVision_mvsUtils
aliceVision_imageIO
Geogram::geogram
${Boost_FILESYSTEM_LIBRARY}
PRIVATE_LINKS
Expand Down
Loading