Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add feature to choose voxel color mode when creating VoxelGrid from PointCloud #6937

Merged
merged 8 commits into from
Dec 19, 2024
Merged

Add feature to choose voxel color mode when creating VoxelGrid from PointCloud #6937

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
a5e657e
add feature to choose voxel color mode in VoxelGrid
rxba Aug 28, 2024
652c828
update CHANGELOG.md
rxba Aug 28, 2024
ae31d31
update tutorial for VoxelGrid color_mode change
rxba Aug 28, 2024
9f4b930
update CHANGELOG.md
rxba Aug 28, 2024
dd7e410
Merge branch 'main' into voxelgrid_color_mode
benjaminum Dec 19, 2024
41fe6b2
rename VoxelColorMode to VoxelPoolingMode
benjaminum Dec 19, 2024
786b11b
update CHANGELOG
benjaminum Dec 19, 2024
d3a4e39
apply style
benjaminum Dec 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@
- Split pybind declarations/definitions to avoid C++ types in Python docs (PR #6869)
- Fix minimal oriented bounding box of MeshBase derived classes and add new unit tests (PR #6898)
- Fix projection of point cloud to Depth/RGBD image if no position attribute is provided (PR #6880)
- Add choice of voxel color mode when creating VoxelGrid from PointCloud (PR #6937)

## 0.13

Expand Down
78 changes: 74 additions & 4 deletions cpp/open3d/geometry/VoxelGrid.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -189,30 +189,42 @@ class VoxelGrid : public Geometry3D {
double height,
double depth);

/// \enum VoxelColorMode
///
/// \brief Possible ways of determining voxel color from PointCloud.
enum class VoxelColorMode { AVG, MIN, MAX, SUM };
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Possible nitpick: can we call this "VoxelPoolingMode" or "VoxelPoolingMethod" instead of "VoxelColorMode"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the feedback! I'd tend to agree that "VoxelPoolingMode" is probably the better name, but I'll wait until a maintainer has had the chance to review the PR to get their feedback.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rxba Thanks for this feature contribution! It looks very useful.
@IDoCodingStuffs I agree, "VoxelPoolingMode" it is


/// Creates a VoxelGrid from a given PointCloud. The color value of a given
/// voxel is the average color value of the points that fall into it (if the
/// voxel is determined by the VoxelColorMode, e.g. by default the average
/// color value of the points that fall into it (if the
/// PointCloud has colors).
/// The bounds of the created VoxelGrid are computed from the PointCloud.
///
/// \param input The input PointCloud.
/// \param voxel_size Voxel size of of the VoxelGrid construction.
/// \param color_mode Mode of determining color for each voxel.
static std::shared_ptr<VoxelGrid> CreateFromPointCloud(
const PointCloud &input, double voxel_size);
const PointCloud &input,
double voxel_size,
VoxelColorMode color_mode = VoxelColorMode::AVG);

/// Creates a VoxelGrid from a given PointCloud. The color value of a given
/// voxel is the average color value of the points that fall into it (if the
/// voxel is determined by the VoxelColorMode, e.g. by default the average
/// color value of the points that fall into it (if the
/// PointCloud has colors).
/// The bounds of the created VoxelGrid are defined by the given parameters.
///
/// \param input The input PointCloud.
/// \param voxel_size Voxel size of of the VoxelGrid construction.
/// \param min_bound Minimum boundary point for the VoxelGrid to create.
/// \param max_bound Maximum boundary point for the VoxelGrid to create.
/// \param color_mode Mode of determining color for each voxel.
static std::shared_ptr<VoxelGrid> CreateFromPointCloudWithinBounds(
const PointCloud &input,
double voxel_size,
const Eigen::Vector3d &min_bound,
const Eigen::Vector3d &max_bound);
const Eigen::Vector3d &max_bound,
VoxelColorMode color_mode = VoxelColorMode::AVG);

/// Creates a VoxelGrid from a given TriangleMesh. No color information is
/// converted. The bounds of the created VoxelGrid are computed from the
Expand Down Expand Up @@ -294,5 +306,63 @@ class AvgColorVoxel {
Eigen::Vector3d color_;
};

/// \class AggColorVoxel
///
/// \brief Class to aggregate color values from different votes in one voxel
/// Can be used to compute min, max, average and sum voxel color.
class AggColorVoxel {
public:
AggColorVoxel()
: num_of_points_(0),
color_(0.0, 0.0, 0.0),
min_color_(Eigen::Vector3d::Constant(
std::numeric_limits<double>::max())),
max_color_(Eigen::Vector3d::Constant(
std::numeric_limits<double>::lowest())) {}

public:
void Add(const Eigen::Vector3i &voxel_index) {
if (num_of_points_ > 0 && voxel_index != voxel_index_) {
utility::LogWarning(
"Tried to aggregate ColorVoxel with different "
"voxel_index");
}
voxel_index_ = voxel_index;
}

void Add(const Eigen::Vector3i &voxel_index, const Eigen::Vector3d &color) {
Add(voxel_index);
color_ += color;
num_of_points_++;
min_color_ = min_color_.cwiseMin(color);
max_color_ = max_color_.cwiseMax(color);
}

Eigen::Vector3i GetVoxelIndex() const { return voxel_index_; }

Eigen::Vector3d GetAverageColor() const {
if (num_of_points_ > 0) {
return color_ / double(num_of_points_);
} else {
return color_;
}
}

Eigen::Vector3d GetMinColor() const { return min_color_; }

Eigen::Vector3d GetMaxColor() const { return max_color_; }

Eigen::Vector3d GetSumColor() const { return color_; }

public:
int num_of_points_;
Eigen::Vector3i voxel_index_;
Eigen::Vector3d color_;

private:
Eigen::Vector3d min_color_;
Eigen::Vector3d max_color_;
};

} // namespace geometry
} // namespace open3d
23 changes: 16 additions & 7 deletions cpp/open3d/geometry/VoxelGridFactory.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,8 @@ std::shared_ptr<VoxelGrid> VoxelGrid::CreateFromPointCloudWithinBounds(
const PointCloud &input,
double voxel_size,
const Eigen::Vector3d &min_bound,
const Eigen::Vector3d &max_bound) {
const Eigen::Vector3d &max_bound,
VoxelGrid::VoxelColorMode color_mode) {
auto output = std::make_shared<VoxelGrid>();
if (voxel_size <= 0.0) {
utility::LogError("voxel_size <= 0.");
Expand All @@ -57,7 +58,7 @@ std::shared_ptr<VoxelGrid> VoxelGrid::CreateFromPointCloudWithinBounds(
}
output->voxel_size_ = voxel_size;
output->origin_ = min_bound;
std::unordered_map<Eigen::Vector3i, AvgColorVoxel,
std::unordered_map<Eigen::Vector3i, AggColorVoxel,
utility::hash_eigen<Eigen::Vector3i>>
voxelindex_to_accpoint;
Eigen::Vector3d ref_coord;
Expand All @@ -76,9 +77,15 @@ std::shared_ptr<VoxelGrid> VoxelGrid::CreateFromPointCloudWithinBounds(
}
for (auto accpoint : voxelindex_to_accpoint) {
const Eigen::Vector3i &grid_index = accpoint.second.GetVoxelIndex();
const Eigen::Vector3d &color =
has_colors ? accpoint.second.GetAverageColor()
: Eigen::Vector3d(0, 0, 0);
// clang-format off
const Eigen::Vector3d &color = has_colors ?
(color_mode == VoxelColorMode::AVG ? accpoint.second.GetAverageColor()
: color_mode == VoxelColorMode::MIN ? accpoint.second.GetMinColor()
: color_mode == VoxelColorMode::MAX ? accpoint.second.GetMaxColor()
: color_mode == VoxelColorMode::SUM ? accpoint.second.GetSumColor()
: Eigen::Vector3d::Zero())
: Eigen::Vector3d::Zero();
// clang-format on
output->AddVoxel(geometry::Voxel(grid_index, color));
}
utility::LogDebug(
Expand All @@ -88,12 +95,14 @@ std::shared_ptr<VoxelGrid> VoxelGrid::CreateFromPointCloudWithinBounds(
}

std::shared_ptr<VoxelGrid> VoxelGrid::CreateFromPointCloud(
const PointCloud &input, double voxel_size) {
const PointCloud &input,
double voxel_size,
VoxelGrid::VoxelColorMode color_mode) {
Eigen::Vector3d voxel_size3(voxel_size, voxel_size, voxel_size);
Eigen::Vector3d min_bound = input.GetMinBound() - voxel_size3 * 0.5;
Eigen::Vector3d max_bound = input.GetMaxBound() + voxel_size3 * 0.5;
return CreateFromPointCloudWithinBounds(input, voxel_size, min_bound,
max_bound);
max_bound, color_mode);
}

std::shared_ptr<VoxelGrid> VoxelGrid::CreateFromTriangleMeshWithinBounds(
Expand Down
28 changes: 22 additions & 6 deletions cpp/pybind/geometry/voxelgrid.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,15 @@ void pybind_voxelgrid_definitions(py::module &m) {
static_cast<py::class_<VoxelGrid, PyGeometry3D<VoxelGrid>,
std::shared_ptr<VoxelGrid>, Geometry3D>>(
m.attr("VoxelGrid"));

py::enum_<VoxelGrid::VoxelColorMode> color_mode(
voxelgrid, "VoxelColorMode",
"Mode of determining color for each voxel.");
color_mode.value("AVG", VoxelGrid::VoxelColorMode::AVG)
.value("MIN", VoxelGrid::VoxelColorMode::MIN)
.value("MAX", VoxelGrid::VoxelColorMode::MAX)
.value("SUM", VoxelGrid::VoxelColorMode::SUM);

py::detail::bind_default_constructor<VoxelGrid>(voxelgrid);
py::detail::bind_copy_functions<VoxelGrid>(voxelgrid);
voxelgrid
Expand Down Expand Up @@ -129,19 +138,23 @@ void pybind_voxelgrid_definitions(py::module &m) {
.def_static("create_from_point_cloud",
&VoxelGrid::CreateFromPointCloud,
"Creates a VoxelGrid from a given PointCloud. The "
"color value of a given voxel is the average color "
"color value of a given voxel is determined by the "
"VoxelColorMode, e.g. by default the average color "
"value of the points that fall into it (if the "
"PointCloud has colors). The bounds of the created "
"VoxelGrid are computed from the PointCloud.",
"input"_a, "voxel_size"_a)
"input"_a, "voxel_size"_a,
"color_mode"_a = VoxelGrid::VoxelColorMode::AVG)
.def_static("create_from_point_cloud_within_bounds",
&VoxelGrid::CreateFromPointCloudWithinBounds,
"Creates a VoxelGrid from a given PointCloud. The "
"color value of a given voxel is the average color "
"color value of a given voxel is determined by the "
"VoxelColorMode, e.g. by default the average color "
"value of the points that fall into it (if the "
"PointCloud has colors). The bounds of the created "
"VoxelGrid are defined by the given parameters.",
"input"_a, "voxel_size"_a, "min_bound"_a, "max_bound"_a)
"input"_a, "voxel_size"_a, "min_bound"_a, "max_bound"_a,
"color_mode"_a = VoxelGrid::VoxelColorMode::AVG)
.def_static("create_from_triangle_mesh",
&VoxelGrid::CreateFromTriangleMesh,
"Creates a VoxelGrid from a given TriangleMesh. No "
Expand All @@ -162,6 +175,7 @@ void pybind_voxelgrid_definitions(py::module &m) {
"origin point.")
.def_readwrite("voxel_size", &VoxelGrid::voxel_size_,
"``float64`` Size of the voxel.");

docstring::ClassMethodDocInject(m, "VoxelGrid", "has_colors");
docstring::ClassMethodDocInject(m, "VoxelGrid", "has_voxels");
docstring::ClassMethodDocInject(m, "VoxelGrid", "get_voxel",
Expand Down Expand Up @@ -214,15 +228,17 @@ void pybind_voxelgrid_definitions(py::module &m) {
docstring::ClassMethodDocInject(
m, "VoxelGrid", "create_from_point_cloud",
{{"input", "The input PointCloud"},
{"voxel_size", "Voxel size of of the VoxelGrid construction."}});
{"voxel_size", "Voxel size of of the VoxelGrid construction."},
{"color_mode", "VoxelColorMode for determining voxel color."}});
docstring::ClassMethodDocInject(
m, "VoxelGrid", "create_from_point_cloud_within_bounds",
{{"input", "The input PointCloud"},
{"voxel_size", "Voxel size of of the VoxelGrid construction."},
{"min_bound",
"Minimum boundary point for the VoxelGrid to create."},
{"max_bound",
"Maximum boundary point for the VoxelGrid to create."}});
"Maximum boundary point for the VoxelGrid to create."},
{"color_mode", "VoxelColorMode for determining voxel color."}});
docstring::ClassMethodDocInject(
m, "VoxelGrid", "create_from_triangle_mesh",
{{"input", "The input TriangleMesh"},
Expand Down
2 changes: 1 addition & 1 deletion docs/jupyter/geometry/voxelization.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@
"metadata": {},
"source": [
"## From point cloud\n",
"The voxel grid can also be created from a point cloud using the method `create_from_point_cloud`. A voxel is occupied if at least one point of the point cloud is within the voxel. The color of the voxel is the average of all the points within the voxel. The argument `voxel_size` defines the resolution of the voxel grid."
"The voxel grid can also be created from a point cloud using the method `create_from_point_cloud`. A voxel is occupied if at least one point of the point cloud is within the voxel. The argument `voxel_size` defines the resolution of the voxel grid. By default, the color of the voxel is the average of all the points within the voxel. The argument `color_mode` can be changed to determine the color by average, min, max or sum value of the points, e.g. with `o3d.geometry.VoxelGrid.VoxelColorMode.MIN`."
]
},
{
Expand Down
Loading